System

Abstract

A system is provided.SOLUTION: A system comprising: means for selecting or generating a character preferred by a user; means for performing a qualification-related lesson using the selected or generated character; means for tracking and recording a learning progress situation in real time; means for generating an answer to a question of the user using the generated AI model and transmitting the answer through the character; means for automatically performing a learning level test after the learning and analyzing a result; and means for personalizing next learning content on the basis of the analyzed result. AI.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional learning systems, especially those aimed at users aiming to obtain qualifications, are heavily dependent on the quality of the instructor and class schedules, making it difficult to build an efficient and individualized learning system. Furthermore, they lack learning support that can flexibly adapt to the user's learning progress, and do not offer 24 / 7 learning support. To solve these problems, a system is needed that provides learning support through characters individually selected by the user and generates highly accurate answers and curricula using generative AI models. [Means for solving the problem]

[0005] The present invention provides a means for pre-learning learning material data using a generative AI model specialized for qualifications, a means for the user to select or generate a character of their choice, and a means for teaching qualification-related lessons using the selected or generated character. It also provides a means for tracking and recording learning progress in real time, and a means for generating answers to user questions using a generative AI model and communicating them through the character. Furthermore, it provides a means for automatically conducting a mastery test after the completion of learning and analyzing the results, and a means for personalizing the next learning content based on the analysis results, thereby constructing a system that comprehensively solves the problems faced by conventional learning systems.

[0006] A "generative AI model" is an artificial intelligence algorithm that specializes in learning data from a specific field or content, and generates appropriate answers or information in response to user input.

[0007] A "character" is a visual entity such as a virtual person, animal, or animation that can be selected or generated by the user and that explains or teaches educational materials.

[0008] "Study progress tracking" is the process of recording a user's progress as they study the learning material in real time, tracking that progress, and storing it in a database.

[0009] A "mastery test" is an automated test administered after a user completes a learning session, providing a means of assessing a user's understanding and learning outcomes.

[0010] "Personalized curriculum" refers to learning content and plans that are optimized based on each user's individual progress and learning outcomes.

[0011] "Real-time answer generation" is a process in which a generative AI model instantly creates answers to questions or doubts from users on the spot and communicates them to the user through a character. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0013] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0014] First, the terms used in the following description will be explained.

[0015] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0016] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0017] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0018] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0019] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0020] [First embodiment]

[0021] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0022] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0023] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0024] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0025] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0026] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0027] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0028] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0029] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0030] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0031] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0032] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0033] The present invention is a system that provides effective learning support for users aiming to obtain qualifications, and is characterized by learning support using generative AI models and characters. The following describes in detail the embodiments of the present invention.

[0034] 1. User registration and character creation

[0035] A user accesses the system, enters the required information such as name, email address, and password on the new registration page, and submits it. The server receives the entered information and stores it in a database. The server then sends a confirmation email to the user's email address. When the user clicks on the link in the confirmation email, the account is confirmed. Once confirmation is complete, the user proceeds to a page where they can select or create their favorite character. On this page, an interface is displayed to customize the character's appearance and behavior. Once the user has finished configuring their character, the information is sent to the server and stored.

[0036] 2. Select study materials and start studying

[0037] Users select the type of qualification within the system and choose the corresponding study materials. This selection is sent to the server, which loads the selected study material data into the generative AI model. Based on the loaded data, the server generates a curriculum specifically for the user, and the user is ready to study.

[0038] 3. Conducting a learning session

[0039] The user's device displays a screen for the learning session, simultaneously displaying the generated character and the learning content. The character explains the content of the learning material with voice and movement. The server monitors the user's progress in real time and stores this information in a database. If the user has any questions, they can enter them and the questions are sent to the server. The server uses a generative AI model to instantly generate answers and communicate them to the user through the character.

[0040] 4. Measurement and feedback

[0041] When a user finishes a learning session, they are automatically tested for their level of mastery. This test is administered by the server, and the test results are analyzed. Based on the analysis, the server identifies the user's weaknesses and areas for improvement, and personalizes the next learning session accordingly. This allows users to continue learning efficiently and in line with their individual needs.

[0042] Specific examples

[0043] For example, a user registers in the system aiming to obtain the "Bookkeeping Level 3" qualification. First, they access the system and create a new account. Next, they create a custom character based on Pikachu. After creating the character, they select the "Bookkeeping Level 3" study materials and begin a study session. Pikachu explains basic accounting concepts, and whenever the user asks a question, the generative AI model provides an instant answer. After the study session ends, a mastery test is automatically conducted, and the next study content is personalized based on the test results.

[0044] In this way, the system of the present invention can provide efficient and individually optimized learning support to users 24 hours a day, 365 days a year.

[0045] The processing flow will be explained below.

[0046] Step 1:

[0047] A user accesses the system and opens the new registration page. The terminal displays the registration form.

[0048] Step 2:

[0049] The user enters the required information (name, email address, password, etc.) and presses the send button. The device sends the input information to the server.

[0050] Step 3:

[0051] The server receives the input data and stores it in a database, then sends a confirmation email to the user's email address.

[0052] Step 4:

[0053] The user confirms the account by clicking the link in the confirmation email. The device clicks the link and communicates with the server.

[0054] Step 5:

[0055] Once the account is verified, the user is directed to a page where they can select or create their desired character, and the device will display a character generation tool.

[0056] Step 6:

[0057] The user customizes the character's appearance and movements, then presses a button to save the settings. The device then sends the settings information to the server.

[0058] Step 7:

[0059] The server receives the character setting information and stores it in a database.

[0060] Step 8:

[0061] The user selects the type of qualification and the corresponding learning material, and the terminal displays the qualification list.

[0062] Step 9:

[0063] The user selects the qualifications they wish to study and chooses the study materials, and the selections are sent to the server.

[0064] Step 10:

[0065] The server loads the necessary data into the generative AI model based on the selected teaching materials and generates a curriculum.

[0066] Step 11:

[0067] The learning session screen is displayed on the user's device. The device provides an interface that simultaneously displays the character and the learning material content.

[0068] Step 12:

[0069] The character explains the contents of the learning material using voice and actions, and the user can learn by watching this.

[0070] Step 13:

[0071] The server tracks the user's progress in real time and records it in a database.

[0072] Step 14:

[0073] The user enters a question or concern, and the device sends the question to the server.

[0074] Step 15:

[0075] The server uses a generative AI model to instantly generate an answer and send it to the device, where a character relays the answer to the user.

[0076] Step 16:

[0077] The user presses a button to end the learning session, and the device performs the termination process.

[0078] Step 17:

[0079] The server automatically conducts a learning level test, and the test screen is displayed on the user's device.

[0080] Step 18:

[0081] The user takes the test. The server analyzes the test results.

[0082] Step 19:

[0083] Based on the analysis results, the server extracts the user's weaknesses and areas for improvement and personalizes the next learning content.

[0084] Step 20:

[0085] The server sends personalized feedback and the next curriculum to the user's device, which displays it.

[0086] In this way, the system provides efficient and individually optimized learning support to users 24 hours a day, 365 days a year.

[0087] Example 1

[0088] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0089] In conventional qualification learning systems, it was difficult to monitor each user's learning progress individually and provide feedback in real time. Furthermore, there was a lack of a flexible system that could immediately respond to users' questions, which often led to a decline in learning efficiency. Furthermore, there were limited systems for personalizing users' learning content, making it difficult to provide learning curricula that met individual needs.

[0090] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0091] In this invention, the server includes means for generating an account based on information entered by the user and sending a confirmation email, means for selecting or generating a character preferred by the user, means for pre-learning learning material data using a generative AI model specialized for qualifications, means for conducting qualification-related lessons using the selected or generated character, means for generating a curriculum dedicated to the user, means for tracking and recording learning progress in real time, means for generating answers to user questions using a generative AI model and communicating them through the character, means for automatically conducting a mastery test after completion of learning and analyzing the results, and means for personalizing the next learning content based on the analysis results, thereby enabling efficient and individually optimized qualification learning.

[0092] "Means for creating an account and sending a confirmation email" refers to a method in which a user is prompted to enter the necessary information when registering with the system, a user account is created based on that information, and an email is automatically sent to confirm the registration.

[0093] The "means for selecting or generating a character" is a method for allowing a user to select or generate a virtual character to be used when studying, according to the user's preferences.

[0094] "Means of using a generative AI model to learn teaching material data in advance" refers to a method in which teaching material data related to a specific qualification is input into the AI ​​model in advance, and the AI ​​model learns based on that data.

[0095] The "means for conducting qualification-related lessons" is a method for using the generated character to conduct lessons to provide users with the knowledge necessary to obtain qualifications.

[0096] The "means for generating a curriculum specifically for a user" is a method for automatically creating an individually optimized learning plan according to the user's qualification acquisition goals and progress.

[0097] The "means for tracking and recording learning progress" is a method for monitoring a user's learning activities in real time and recording the progress in a database or the like.

[0098] "Means of generating answers using a generative AI model and communicating them through a character" refers to a method in which, when a user inputs a question, a generative AI model generates an answer to that question and communicates it to the user through a virtual character.

[0099] The "means for conducting a mastery test and analyzing the results" is a method for automatically conducting a test after the completion of a learning session and analyzing the results.

[0100] "Means for personalizing the next learning content" refers to a method for individually optimizing the next learning content based on the results of the mastery test and the user's progress.

[0101] The present invention is a system that provides effective learning support to users aiming to obtain qualifications, characterized by learning support using a generative AI model and a character. Specifically, the system includes means for generating an account based on information entered by the user and sending a confirmation email, means for selecting or generating a character preferred by the user, means for pre-learning learning material data using a generative AI model specialized for qualifications, means for conducting qualification-related lessons using the selected or generated character, means for generating a curriculum dedicated to the user, means for tracking and recording learning progress in real time, means for generating answers to user questions using a generative AI model and communicating them through a character, means for automatically conducting a mastery test after completion of learning and analyzing the results, and means for personalizing the next learning content based on the analysis results.

[0102] Users access this system using a web browser as an interface. When a user enters their name, email address, and password on the new registration page and submits it, the server receives the information and stores it in a MySQL database. The server then uses a mail server to send a confirmation email. This email contains a URL link for account confirmation, and the user completes account confirmation by clicking the link. Once confirmation is complete, the user can proceed to the character creation page, where an interface is provided for customizing the character's appearance and behavior. Once the user has finished configuring their character, the information is sent back to the server and stored in the database.

[0103] The user selects the type of qualification within the system and chooses the corresponding learning materials. The server loads the selected learning material data (PDF, video, audio files, etc.) into a generative AI model (e.g., OpenAI GPT-4) and generates a curriculum specifically for the user. The user is then ready to learn. The learning session screen is displayed on the user's device, displaying the generated character and the learning content simultaneously. The character explains the content of the learning materials with voice and movement. Voice synthesis software may also be used here.

[0104] The server monitors the user's progress in real time and stores that information in a database. When the user enters a question into a text box and submits it, the server inputs the question into a generative AI model to generate an answer. The answer is then communicated to the user via voice via a character.

[0105] Once a user has completed a learning session, they are automatically tested for their progress. The test is administered by the server and the results are analyzed in real time. Based on the test results, the user's weaknesses and areas for improvement are identified, and the next learning session is personalized.

[0106] As a concrete example, let's consider a case where a user is aiming for the "Bookkeeping Level 3" qualification. The user accesses the system and creates a new account. Next, they create a custom character based on Pikachu. After creating the character, they select the "Bookkeeping Level 3" study materials and begin a study session. Pikachu explains basic accounting concepts, and when the user enters a question about a specific journal entry, the server sends prompts to the generative AI model, providing an immediate answer. After the study session, a mastery test is automatically conducted, and the next study content is personalized based on the test results.

[0107] An example of a prompt sentence is, "Please explain basic journal entries as preparation for the Bookkeeping Level 3 exam." This enables the system to provide efficient, individually optimized learning support to users 24 hours a day, 365 days a year.

[0108] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0109] Step 1: The user accesses the system, enters their name, email address, and password on the new registration page, and submits the information. The entered information is sent to the server.

[0110] Input: User registration information (name, email address, password)

[0111] Output: Registration information sent to the server

[0112] What happens: The user fills in the form and clicks the submit button.

[0113] Step 2: The server receives the entered information and performs validation checks.

[0114] Input: User registration information

[0115] Output: Validation result (success or error)

[0116] What happens: The server validates the information entered to ensure it is in the correct format.

[0117] Step 3: If validation is successful, the server saves the registration information in a MySQL database.

[0118] Input: Validated registration information

[0119] Output: Save results to database

[0120] Specific operation: The server inserts the registration information into the user table in the database.

[0121] Step 4: The server sends a confirmation email to the user's email address, which contains a URL link to verify the account.

[0122] Input: User's email address and confirmation URL

[0123] Output: Confirmation email sent

[0124] Specific operation: The server sends a confirmation email via the mail server.

[0125] Step 5: The user opens the confirmation email and clicks on the confirmation link, which causes the server to activate the user's account.

[0126] Input: User confirmation link click

[0127] Output: Account enabled

[0128] Specific behavior: The server detects the click on the confirmation link and updates the account status accordingly.

[0129] Step 6: The user proceeds to the character generation page to customize the character's appearance and behavior.

[0130] Input: User's character setting information

[0131] Output: Customized character information

[0132] Specific actions: The user selects the character's appearance and actions through the interface and submits the configuration information.

[0133] Step 7: The server receives the character setting information and stores it in a database.

[0134] Input: User's character setting information

[0135] Output: Saved character information

[0136] Specific operation: The server inserts the character setting information into the character table of the database.

[0137] Step 8: The user selects the type of qualification and chooses the corresponding study material.

[0138] Input: User credential selection information

[0139] Output: Selected teaching material data

[0140] Specific Actions: The user selects a category of qualifications and then selects specific learning materials within that category.

[0141] Step 9: The server loads the selected teaching material data into the generative AI model.

[0142] Input: Selected teaching material data

[0143] Output: Teaching material data loaded into the generative AI model

[0144] Specific operation: The server inputs the teaching material data into the learning engine of the generative AI model.

[0145] Step 10: The server generates a curriculum specific to the user and stores it in the database.

[0146] Input: Teaching material data from a generative AI model

[0147] Output: User-specific curriculum

[0148] Specific operation: The server automatically generates a curriculum based on the user's progress and goals.

[0149] Step 11: A screen for the learning session is displayed on the user terminal.

[0150] Input: User-specific curriculum data

[0151] Output: Learning session screen

[0152] What happens: A user starts a learning session on a device and the interface is displayed.

[0153] Step 12: The character explains the content of the teaching material with voice and movement.

[0154] Input: Curriculum data and character setting information

[0155] Output: Audio explanation of teaching materials

[0156] Specific actions: The character explains to the user through movement and voice based on the content of the teaching material.

[0157] Step 13: The server monitors the user's progress in real time and stores the information in a database.

[0158] Input: User's learning progress data

[0159] Output: Saved learning progress data

[0160] Specific operation: The server records in real time how much progress the user has made in their studies.

[0161] Step 14: The user enters and submits a question during the study.

[0162] Input: User question text

[0163] Output: The question sent to the server

[0164] Specific actions: The user enters a question into the interface and clicks the submit button.

[0165] Step 15: The server inputs a question into the generative AI model, which generates an answer.

[0166] Input: User question text

[0167] Output: Answer from the generative AI model

[0168] Specific operation: The server inputs the question as a prompt into the AI ​​model, which generates an answer.

[0169] Step 16: The generated answer is spoken to the user through the character.

[0170] Input: Answer from a generative AI model

[0171] Output: Audio explanation of answers

[0172] Specific operation: The character vocalizes the answer from the generated AI model and conveys it to the user.

[0173] Step 17: When the user finishes the learning session, an automatic mastery test is administered.

[0174] Input: User's learning session end signal

[0175] Output: Start of mastery test

[0176] Specific operation: The server detects the end of the learning session and displays a test screen.

[0177] Step 18: The server analyzes the test results in real time and records the user's score and correct answer rate.

[0178] Input: User test answer data

[0179] Output: Parsed test results

[0180] Specific operation: The server analyzes the user's answers and calculates the score and percentage of correct answers.

[0181] Step 19: The server personalizes the next learning content based on the analysis results.

[0182] Input: Parsed test results

[0183] Output: Personalized upcoming curriculum

[0184] Specific operation: The server individually optimizes the next learning content based on the user's weaknesses and areas for improvement.

[0185] The above is the specific flow of the program processing of this system.

[0186] (Application example 1)

[0187] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0188] Traditional online learning systems offer limited interactive learning support and lack personalized feedback. They also lack real-time progress tracking and a virtual learning experience, which can reduce learning effectiveness. Furthermore, it is difficult for users to get immediate answers to their questions, which reduces learning efficiency.

[0189] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0190] In this invention, the server includes means for pre-learning learning material data using a generative AI model specialized for qualifications, means for selecting or generating a character preferred by the user, means for teaching qualification-related lessons using the selected or generated character, means for tracking and recording learning progress in real time, means for generating answers to the user's questions using the generative AI model and communicating them through the character, means for automatically conducting a mastery test after completion of learning and analyzing the results, means for personalizing the next learning content based on the analysis results, means for accessing as an avatar in a virtual space and providing interactive learning support, and means for the character to explain the learning material in a learning area in the virtual space. This allows users to receive personalized learning support, enabling effective and efficient learning.

[0191] A "qualification-specific generative AI model" is an artificial intelligence model designed to learn teaching material data corresponding to a specific qualification.

[0192] "Means for selecting or generating a character" refers to a mechanism such as an interface that allows a user to select a character of their choice or generate a new one.

[0193] "Means for teaching qualification-related lessons" is a function for teaching knowledge related to qualifications using selected or generated characters.

[0194] The "means for tracking and recording learning progress in real time" is a mechanism for monitoring the progress of a user during learning in real time and recording that data.

[0195] "A means of generating answers to questions using a generative AI model and communicating them through a character" is a function in which, when a user inputs a question, the generative AI model instantly generates an answer, and communicates that answer via voice or text through a character.

[0196] "Means for automatically conducting a mastery test after completion of learning and analyzing the results" is a function that automatically conducts a test after the learning session ends and analyzes the results.

[0197] "Means for personalizing the next learning content" refers to a system that optimizes and sets the next learning content for each individual user based on the test results.

[0198] "Means for providing interactive learning support through access as an avatar in a virtual space" is a system that allows users to log in to a virtual space, participate as an avatar, and receive interactive learning support.

[0199] "Means for a character to explain learning materials in a learning area within a virtual space" is a function that sets up a specific learning area within a virtual space and allows a character to explain learning materials there.

[0200] The present invention is a system that provides effective learning support for users aiming to obtain qualifications, and is characterized by learning support using generative AI models and characters. A specific method for implementing the present invention will be described in detail below.

[0201] First, when a user accesses the system and enters the required information such as name, email address, and password on the new registration page, the server receives the information and stores it in a database. Next, the server sends a confirmation email to the user's email address, and the user confirms the account by clicking the link in the confirmation email. Once confirmation is complete, the user proceeds to a page where they can select or create their favorite character. On this page, an interface is displayed where the user can customize the character's appearance and behavior.

[0202] Once the user has set up their character, the information is sent to the server and saved. Next, the user selects the type of qualification within the system and chooses the corresponding learning materials. This selection is sent to the server, which loads the selected learning materials data into a generative AI model and generates a curriculum specifically for the user.

[0203] The user's device displays a screen for the learning session, simultaneously displaying the generated character and the learning content. The character explains the content of the learning material with voice and movement. This allows the user to access the virtual space as an avatar and receive interactive learning support. If the user has any questions, they can enter them and the questions are sent to the server. The server uses a generative AI model to instantly generate an answer and communicates it to the user through the character.

[0204] After the learning session, a mastery test is automatically conducted. This test is administered by the server, and the test results are analyzed. Based on the analysis results, the server extracts the user's weaknesses and areas for improvement, and personalizes the next learning content based on these. This allows the user to receive efficient, individually optimized learning support.

[0205] Specific hardware used includes smartphones, smart glasses, or head-mounted displays (e.g., Oculus Rift). AWS and Google Cloud can be used as server platforms. Advanced artificial intelligence models such as GPT-4 are used as generative AI models. Character animation is performed using software such as Unity and Unreal Engine.

[0206] As a concrete example, consider the case where a user registers with the system aiming for the "Bookkeeping Level 3" qualification. First, the user accesses the system and creates a new account. Next, the user generates a character, selects the "Bookkeeping Level 3" study materials, and starts a study session. The character explains basic accounting concepts, and whenever the user asks a question, the generative AI model provides an immediate answer. For example, in response to a question such as "Please tell me the basics of journal entries," the character provides a detailed explanation. After the study session ends, a mastery test is conducted, and the next study content is personalized based on the test results.

[0207] An example of a prompt sentence for a generative AI model is, "The user is asking about the depreciation method for fixed assets. Please explain, including specific examples (for example, how to depreciate a machine worth 1 million yen over 10 years)."

[0208] In this way, the system of the present invention can provide efficient and individually optimized learning support to users 24 hours a day, 365 days a year.

[0209] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0210] Step 1:

[0211] User Registration

[0212] Input: The user enters their name, email address, and password.

[0213] Processing: The server receives this information and stores it in a database.

[0214] Output: A confirmation email is sent to the user.

[0215] How it works: The user clicks on the link in the confirmation email to confirm the account. The server activates the account.

[0216] Step 2:

[0217] Character Generation

[0218] Input: The user customizes the character's appearance and behavior.

[0219] Processing: The server receives the character setting information and stores it in the database.

[0220] Output: Customized character information.

[0221] Behavior: The user configures the character's appearance and behavior on the character customization screen and sends the settings to the server.

[0222] Step 3:

[0223] Selection of teaching materials

[0224] Input: User selects the qualification type and corresponding study material.

[0225] Processing: The server loads the selected learning materials into a generative AI model to generate a personalized curriculum.

[0226] Output: A personalized learning curriculum.

[0227] How it works: A user selects a certification type within the system and then selects study materials to begin learning.

[0228] Step 4:

[0229] Start a study session

[0230] Input: A request to start a learning session from the user's device.

[0231] Processing: The server sends the character and educational material data to the user's terminal.

[0232] Output: Screen display for the learning session.

[0233] How it works: A character and learning materials are displayed on the user's device, and the learning session begins. The character explains the content.

[0234] Step 5:

[0235] Question and Answering

[0236] Input: The question asked by the user.

[0237] Processing: The server sends the question to the generative AI model, which generates an answer.

[0238] Output: Display the answer on the user's terminal.

[0239] How it works: Users enter a question and the answer is instantly provided by a generative AI model. A character will then relay the answer.

[0240] Step 6:

[0241] Progress Tracking

[0242] Input: User's learning progress data.

[0243] Processing: The server tracks the progress in real time and stores it in a database.

[0244] Output: Updated progress data.

[0245] How it works: As you learn, your progress is recorded in real time.

[0246] Step 7:

[0247] Completion of study and mastery test

[0248] Input: Training session end signal.

[0249] Processing: The server automatically conducts the mastery test and analyzes the results.

[0250] Output: Test results and analysis report.

[0251] How it works: After the user has completed the learning, an automatic learning test is administered, and the results are sent to the server for analysis.

[0252] Step 8:

[0253] Personalize your next lesson

[0254] Input: Test results and analytical data.

[0255] Processing: The server customizes the next learning content based on the results.

[0256] Output: A personalized next-day learning curriculum.

[0257] How it works: Learning content is set based on the user's test results and is reflected in the next learning session.

[0258] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0259] The present invention is a system that provides effective learning support for users aiming to obtain qualifications, and is characterized by a generative AI model and character learning support, as well as an emotion engine that recognizes the user's emotions. The following describes in detail the embodiments of the present invention.

[0260] 1. User registration and character creation

[0261] A user accesses the system, enters the required information such as name, email address, and password on the new registration page, and submits it. The server receives the entered information and stores it in a database. The server then sends a confirmation email to the user's email address. When the user clicks on the link in the confirmation email, the account is confirmed. Once confirmation is complete, the user proceeds to a page where they can select or create their favorite character. On this page, an interface is displayed to customize the character's appearance and behavior. Once the user has finished configuring their character, the information is sent to the server and stored.

[0262] 2. Select study materials and start studying

[0263] Users select the type of qualification within the system and choose the corresponding study materials. This selection is sent to the server, which loads the selected study material data into the generative AI model. Based on the loaded data, the server generates a curriculum specifically for the user, and the user is ready to study.

[0264] 3. Conducting a learning session

[0265] The user's device displays a screen for the learning session, simultaneously displaying the generated character and the learning content. The character explains the content of the learning material with voice and actions. An emotion engine then recognizes the user's emotions from their facial expressions and vocalizations and provides real-time feedback. For example, if the emotion engine determines that the user is confused, the character will provide additional explanations and encouragement according to that emotion. The server monitors the user's progress and emotional changes in real time and stores this information in a database. If the user has any questions, they can enter them and the questions are sent to the server. The server then uses a generative AI model to instantly generate answers and communicate them to the user through the character.

[0266] 4. Measurement and feedback

[0267] When a user finishes a learning session, they are automatically tested for their level of mastery. This test is administered by the server, and the test results are analyzed. Based on the analysis results, the server identifies the user's weaknesses and areas for improvement, and personalizes the next learning content based on these. In addition, emotional data recognized by the emotion engine is also added to the analysis, enabling more accurate feedback and curriculum adjustments. This allows users to continue learning efficiently and in line with their individual needs and emotions.

[0268] Specific examples

[0269] For example, a user registers in the system aiming to obtain the "Bookkeeping Level 3" qualification. First, they access the system and create a new account. Next, they create a custom character based on Pikachu. After creating the character, they select the "Bookkeeping Level 3" study materials and begin a study session. Pikachu explains basic accounting concepts, and whenever the user asks a question, the generative AI model provides an instant answer. If the emotion engine detects a confused expression on the user's face during the study session, Pikachu will provide additional explanations or encouragement. After the study session ends, a mastery test is automatically conducted, and the next study content is personalized based on the test results.

[0270] In this way, by combining a generative AI model and an emotion engine, the system of the present invention is able to provide efficient and individually optimized learning support to users 24 hours a day, 365 days a year.

[0271] The processing flow will be explained below.

[0272] MODE FOR CARRYING OUT THE INVENTION (INCLUDING EMOTION ENGINE)

[0273] Step 1:

[0274] A user accesses the system and opens the new registration page. The terminal displays the registration form.

[0275] Step 2:

[0276] The user enters the required information (name, email address, password, etc.) and presses the send button. The device sends the input information to the server.

[0277] Step 3:

[0278] The server receives the input data and stores it in a database, then sends a confirmation email to the user's email address.

[0279] Step 4:

[0280] The user confirms their account by clicking the link in the confirmation email, and the device clicks the link to communicate with the server.

[0281] Step 5:

[0282] Once the account is verified, the user is directed to a page where they can select or create their desired character, and the device will display a character generation tool.

[0283] Step 6:

[0284] The user customizes the character's appearance and movements, then presses a button to save the settings. The device then sends the settings information to the server.

[0285] Step 7:

[0286] The server receives the character setting information and stores it in a database.

[0287] Step 8:

[0288] The user selects the type of qualification and the corresponding learning material, and the terminal displays the qualification list.

[0289] Step 9:

[0290] The user selects the qualifications they wish to study and chooses the study materials, and the selections are sent to the server.

[0291] Step 10:

[0292] The server loads the necessary data into the generative AI model based on the selected teaching materials and generates a curriculum.

[0293] Step 11:

[0294] The learning session screen is displayed on the user's device. The device provides an interface that simultaneously displays the character and the learning material content.

[0295] Step 12:

[0296] The character explains the contents of the learning material using voice and actions, and the emotion engine analyzes the user's facial expressions and vocalizations in real time to recognize the user's emotional state.

[0297] Step 13:

[0298] The emotion engine sends feedback to the server based on the user's emotional state, for example suggesting additional explanation or encouragement if the user is confused.

[0299] Step 14:

[0300] The server receives feedback from the emotion engine and instructs the character to respond appropriately. The character then conveys feedback to the user according to the emotion.

[0301] Step 15:

[0302] The server tracks the user's progress and emotional changes in real time and records them in a database.

[0303] Step 16:

[0304] The user enters a question or concern, and the device sends the question to the server.

[0305] Step 17:

[0306] The server uses a generative AI model to instantly generate answers and communicate them to the user through a character.

[0307] Step 18:

[0308] The user presses a button to end the learning session, and the device performs the termination process.

[0309] Step 19:

[0310] The server automatically conducts a learning level test, and the test screen is displayed on the user's device.

[0311] Step 20:

[0312] The user takes the test. The server analyzes the test results.

[0313] Step 21:

[0314] Based on the analysis results, the server identifies the user's weaknesses and areas for improvement, and personalizes the next lesson. The emotion engine also analyzes the emotional data it recognizes, improving the quality of feedback and the curriculum.

[0315] Step 22:

[0316] The server sends personalized feedback and the next curriculum to the user's device, which displays it.

[0317] In this way, by combining a generative AI model and an emotion engine, this system is able to provide efficient and individually optimized learning support to users 24 hours a day, 365 days a year.

[0318] Example 2

[0319] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0320] Conventional learning support systems have had difficulty providing personalized learning tailored to the individual needs and progress of users. Furthermore, they lacked the ability to recognize and respond to users' emotions and confusion in real time, making it difficult to maximize the effectiveness of learning. There was a need to provide a system that would solve these problems and enable users to study for qualifications efficiently and effectively.

[0321] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0322] In this invention, the server includes a means for pre-training learning material data using a generative AI model related to qualifications, a means for using an agent selected or generated by the user, and a means for tracking the user's learning progress in real time and recording it in a database. This enables personalized learning according to the user's individual needs and learning progress. The agent also deepens the user's understanding by conducting lessons involving simple movements. Furthermore, the emotion engine recognizes the user's facial expressions and vocalizations in real time and reflects that feedback to maximize learning effectiveness.

[0323] A "generative AI model for qualifications" is an artificial intelligence model that pre-learns educational material data related to specific qualifications and provides answers and educational materials in response to user questions and customizations.

[0324] An "agent" is a character or avatar that the user can select or create, and that character or avatar is responsible for explaining the learning content.

[0325] "Tracking" refers to the process of tracking a user's learning progress and behavior in real time and recording it in a database.

[0326] "Database" means a data storage system for managing user data, progress, learning outcomes, etc. collected within the system.

[0327] An "emotion engine" is an algorithm or software component that analyzes a user's facial expressions and tone of voice in real time and recognizes their emotions.

[0328] A "proficiency test" is a test that is automatically administered after a user completes a learning session to assess the user's learning progress.

[0329] "Personalization" refers to the process of individually optimizing the next learning content and curriculum based on each user's learning progress and weaknesses.

[0330] This invention is a system that provides effective learning support to users aiming to obtain qualifications, and in addition to learning support using generative AI models and agents, it features an emotion engine that recognizes the user's emotions.

[0331] User registration and agent creation

[0332] A user accesses the system, enters the required information such as name, email address, and password on the new registration page, and submits it. The server receives the entered information and stores it in a database. The server then sends a confirmation email to the user's email address. When the user clicks on the link in the confirmation email, the account is confirmed.

[0333] After verifying their account, users are taken to a page where they can select or create their preferred agent. This page provides an interface for customizing the agent's appearance and behavior. Once the user has configured their agent, the information is sent to the server and stored in a database.

[0334] Select study materials and start learning

[0335] Users select the type of qualification within the system and choose the corresponding study materials. The selection is sent to the server, which loads the selected study material data into a generative AI model. For example, GPT-3 or BERT can be used as the generative AI model. Based on the loaded data, the server generates a curriculum specifically for the user, and the user is ready to begin learning.

[0336] Running a learning session

[0337] The user's device displays a screen for the learning session, simultaneously displaying the generated agent and the learning content. The agent explains the content of the learning material with voice and action. The emotion engine recognizes the user's emotions from their facial expressions and vocalizations and provides feedback in real time. For example, if the emotion engine determines that the user is confused, the agent will provide additional explanations and encouragement according to that emotion. The server monitors the user's progress and emotional changes in real time and stores this information in a database. If the user has any questions, they can enter them and the questions are sent to the server. The server uses a generative AI model to instantly generate answers and communicates them to the user via the agent.

[0338] Measurement and feedback

[0339] When a user finishes a learning session, they are automatically tested for their level of mastery. This test is administered by the server, and the test results are analyzed. Based on the analysis results, the server identifies the user's weaknesses and areas for improvement, and personalizes the next learning content based on these. In addition, emotional data recognized by the emotion engine is also added to the analysis, enabling more accurate feedback and curriculum adjustments. This allows users to continue learning efficiently and in line with their individual needs and emotions.

[0340] Specific examples

[0341] For example, a user registers with the system aiming for the "Bookkeeping Level 3" certification. First, they access the system and create a new account, then generate a custom agent based on Pikachu. After generating the agent, they select the "Bookkeeping Level 3" study materials and begin a learning session. The agent explains basic accounting concepts, and whenever the user asks a question, the generated AI model provides an immediate answer. If the emotion engine detects a confused expression on the user's face during the learning session, the agent will provide additional explanations or encouragement. After the learning session ends, a mastery test is automatically conducted, and the next learning content is personalized based on the test results.

[0342] Prompt Sentence Examples

[0343] "Please explain a learning support system using custom agents with emotion recognition."

[0344] By combining a generative AI model and an emotion engine, the system of this invention can provide efficient and individually optimized learning support to users 24 hours a day, 365 days a year.

[0345] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0346] Step 1:

[0347] A user accesses the system, enters the required information such as name, email address, and password on the new registration page, and submits it. The input includes the user's personal information (name, email address, password). The output is the user information sent to the server.

[0348] Step 2:

[0349] The server receives the entered user information and stores it in a database.

[0350] Input: User's personal information (name, email address, password)

[0351] Data processing: Convert received information into the appropriate format and store it in a database

[0352] Output: Save confirmation message

[0353] Step 3:

[0354] The server sends a confirmation email to the user's email address, and when the user clicks on the link in the email, the account is authenticated.

[0355] Input: User's email address

[0356] Data processing: Generating and sending confirmation emails

[0357] Output: Account verification link

[0358] Step 4:

[0359] After completing the verification, the user proceeds to the agent creation page and uses the interface to customize the agent's appearance and behavior.

[0360] Input: User's agent selection and customization information

[0361] Data processing: Importing and verifying selected information

[0362] Output: Agent customization data

[0363] Step 5:

[0364] The server receives the customized agent information and stores it in a database.

[0365] Input: Agent customization information

[0366] Data processing: Format conversion and storage of received information

[0367] Output: Agent creation confirmation message

[0368] Step 6:

[0369] Users select the type of qualification within the system and choose the corresponding study materials.

[0370] Input:Select qualification type

[0371] Data processing: Search for teaching material data corresponding to the selected qualification

[0372] Output: List of teaching material data

[0373] Step 7:

[0374] The server loads the selected teaching material data into a generative AI model and generates a curriculum tailored to the user.

[0375] Input: Teaching material data

[0376] Data processing: Loading teaching material data and generating curriculum

[0377] Output: User-specific curriculum

[0378] Step 8:

[0379] The learning session screen is displayed on the user's device. The generated agent and the learning content are displayed simultaneously, and the agent explains the learning content with voice and actions.

[0380] Input: User-specific curriculum and agent data

[0381] Data processing: Synchronization of screen display with voice and action

[0382] Output: Learning screen

[0383] Step 9:

[0384] The emotion engine recognizes the user's facial expressions and vocalizations in real time and provides feedback. For example, if it detects a confused expression, the agent will provide additional explanation or encouragement.

[0385] Input: User's facial expression data and voice data

[0386] Data processing: Sentiment analysis and feedback content generation

[0387] Output: Feedback content

[0388] Step 10:

[0389] If a user has a question while studying, they can enter it and send it to the server.

[0390] Input: User question

[0391] Data processing: Question analysis and answer generation using an AI model

[0392] Output: Response data

[0393] Step 11:

[0394] The server uses a generative AI model to generate answers and communicates them to the user through an agent.

[0395] Input: Question data

[0396] Data processing: Answer generation using generative AI models

[0397] Output: Response via agent

[0398] Step 12:

[0399] Once the user has finished the learning session, the mastery test will begin automatically.

[0400] Input: End of study session trigger

[0401] Data processing: Implementing learning level tests and compiling the results

[0402] Output: Test results

[0403] Step 13:

[0404] The server analyzes the test results and extracts the user's strengths and weaknesses.

[0405] Input: Test result data

[0406] Data processing: Analyzing test results and identifying weaknesses

[0407] Output: Analysis results

[0408] Step 14:

[0409] The server personalizes the next learning content based on the analysis results and provides feedback taking into account the emotional data recognized by the emotion engine.

[0410] Input: Analysis results and sentiment data

[0411] Data processing: Customizing the next learning content

[0412] Output: Personalized learning and feedback

[0413] (Application example 2)

[0414] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0415] Conventional learning support systems have difficulty providing effective learning support based on the user's individual emotions and progress, and are insufficiently personalized to meet individual needs. Furthermore, when dealing with customers in brick-and-mortar stores, it is difficult to provide efficient explanations and customer service that reflect the customer's emotions and needs in real time. There is a need for a system that can solve these issues and provide support and services that are optimized for each user and customer.

[0416] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for pre-learning learning material data using a generative AI model specialized for qualifications; means for selecting or generating a character preferred by the user; means for conducting qualification-related lessons using the selected or generated character; means for tracking and recording learning progress in real time; means for generating answers to the user's questions using the generative AI model and communicating them through the character; means for automatically conducting a mastery test after completion of learning and analyzing the results; means for personalizing the next learning content based on the analysis results; means for analyzing the user's emotions using an emotion recognition engine and adjusting the character's movements and speech based on the data; and means for providing personalized product explanations and customer service to customers in stores as part of personalization. This enables effective learning support that reflects the user's emotions and progress in real time, making it possible to provide explanations and customer service optimized for each customer even in physical stores.

[0417] A "generative AI model specialized for qualifications" is an artificial intelligence model that pre-learns educational material data related to obtaining specific qualifications and generates appropriate answers and explanations.

[0418] An "emotion recognition engine" is a system that analyzes the emotions of a user, such as facial expressions and vocalizations, in real time, and extracts and utilizes that emotional data.

[0419] A "character" is a virtual avatar or character selected or created by the user that acts as a facilitator of learning or guidance.

[0420] "Means for tracking and recording learning progress in real time" refers to a mechanism for monitoring a user's learning activities and progress in real time and recording that information.

[0421] "Generating answers using a generative AI model" means that artificial intelligence generates appropriate answers to questions from users based on information it has learned in advance.

[0422] "Means of communicating through a character" refers to a method of communicating the generated answer or explanation to the user through the voice or actions of a character selected by the user.

[0423] The "means for automatically conducting a mastery test after completion of learning" is a mechanism for automatically conducting a test to measure the user's level of understanding after the learning session is completed.

[0424] The "means for analyzing the results" is a method for analyzing the results of the mastery test in detail and understanding the user's strengths and weaknesses.

[0425] "Means for personalizing the next learning content" is a mechanism that provides learning content optimized for the user's individual needs and progress based on the analysis results.

[0426] "Means for providing personalized product explanations and customer service" refers to a system for providing appropriate product explanations and customer service in physical stores based on the individual emotions and needs of customers.

[0427] The present invention relates to a system that provides learning support and customer service support in physical stores using a generative AI model and an emotion recognition engine. Specific examples for implementing the present invention are described in detail below.

[0428] 1. System Configuration

[0429] This system provides effective learning support for users who are aiming to serve customers in brick-and-mortar stores and obtain qualifications. The system is primarily composed of the following hardware and software:

[0430] Hardware: Smartphones, smart glasses

[0431] Software: Mobile app, server-side program (Python + Flask), database (MySQL), generative AI model (OpenAI's GPT-4), emotion recognition engine (Microsoft Azure Face API)

[0432] 2. Program Processing

[0433] Customer registration and character creation

[0434] The server creates an account based on the information the user entered and sends a confirmation email. When the user clicks the confirmation link, the account is activated and the user is then taken to the character creation page. The user customizes the character's appearance and behavior to their liking, and this information is sent to the server and stored in a database.

[0435] Product selection and introduction

[0436] Users select a product category through their smart device. The selection is sent to the server, which then loads the selected product data into the generative AI model, which then creates a personalized guidance curriculum for the customer based on the data.

[0437] Running a store navigation session

[0438] An information screen is displayed on the user's device, and a generated character explains the product. An emotion recognition engine analyzes the user's emotions from their facial expressions and vocalizations and provides feedback in real time. The server monitors the user's progress and emotions in real time and stores them in a database. When the user enters a question, the generative AI model instantly generates an answer and communicates it to the user through the character.

[0439] Measurement and feedback

[0440] After the guidance session ends, a satisfaction test is automatically conducted by the server. The test results are analyzed in detail by the server and feedback is provided to identify the user's strengths and weaknesses. Based on the analysis results, the next guidance content is personalized.

[0441] As a specific example, consider the case where a user visits a physical store and uses smart glasses to receive product information. At this time, a character explains the product based on product data previously learned by the generative AI model. If the emotion recognition engine detects a confused expression on the user's face, the character will provide additional explanation to help the user understand. In this way, effective customer service is possible, reflecting the user's emotions and progress in real time.

[0442] Example prompt sentence:

[0443] "Based on the product category selected by the customer, we use an emotion recognition engine to analyze the user's emotions and generate the most appropriate product description."

[0444] As described above, by combining a generative AI model and an emotion recognition engine, this system can provide individually optimized learning support and customer service to users and customers.

[0445] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0446] Step 1:

[0447] The user accesses the app from smart glasses or a smartphone, enters the required information such as name, email address, and password on the new registration page, and submits it. Input: Registration information entered by the user. Output: Registration information is sent to the server and saved in the database. Specifically, after the server receives the entered information, it saves it as a new record in the database.

[0448] Step 2:

[0449] The server sends a confirmation email to the user's email address. Input: The user's email address included in the registration information. Output: A confirmation email is sent to the user's email address. Specifically, the server generates and sends the confirmation email using the SMTP protocol.

[0450] Step 3:

[0451] When the user clicks the link in the confirmation email, the account is confirmed. Input: The confirmation link the user clicked. Output: The server activates the account. Specifically, the server updates the user's account record and sets the confirmation flag.

[0452] Step 4:

[0453] The user creates or selects a character of their choice. Input: Customization information for the appearance and behavior of the character selected by the user. Output: Character creation information is sent to the server and saved in a database. Specifically, the server receives the character information set by the user and saves it in a database.

[0454] Step 5:

[0455] The user selects a product category through the app. Input: The product category selected by the user. Output: The product category information is sent to the server and loaded into the generative AI model. Specifically, the server receives the product category information and passes it to the generative AI model.

[0456] Step 6:

[0457] The generative AI model generates a guidance curriculum based on product data. Input: Product category information. Output: Generated guidance curriculum. Specifically, the generative AI model analyzes the given product category information and generates an appropriate guidance curriculum.

[0458] Step 7:

[0459] An information screen is displayed on the user's device, and the generated character explains the product. Input: Generated information curriculum. Output: Product description displayed on the device. Specifically, the character explains the product using voice and actions based on the information curriculum obtained from the generation AI model.

[0460] Step 8:

[0461] The emotion recognition engine analyzes the user's facial expressions and vocalizations to recognize emotions. Input: User's facial expression and vocalization data. Output: Recognized emotion data. Specifically, the emotion recognition engine analyzes the user's facial images and voice data in real time to extract their emotional state.

[0462] Step 9:

[0463] The server monitors the user's progress and emotions in real time and records them in a database. Input: User's progress and emotion data. Output: Progress and emotion data stored in the database. Specifically, the server receives the user's learning progress and emotion data and stores it in the database.

[0464] Step 10:

[0465] The user inputs a question and sends it to the server. Input: User's question. Output: Answer generated by the generative AI model. Specifically, the server receives the question from the user and uses the generative AI model to generate an appropriate answer.

[0466] Step 11:

[0467] The generated answer is communicated to the user through the character. Input: The answer generated by the generative AI model. Output: The answer communicated to the user. The specific action is for the character to communicate the answer to the user through voice and action.

[0468] Step 12:

[0469] After the guidance session ends, a satisfaction test is automatically conducted. Input: Learning / guidance session end trigger. Output: Satisfaction test result. Specifically, the server detects the end of the session and automatically conducts a satisfaction test.

[0470] Step 13:

[0471] The server analyzes the test results and identifies the user's strengths and weaknesses. Input: Satisfaction test results. Output: Analysis results. Specifically, the server analyzes the test results in detail to extract the user's level of understanding and areas for improvement.

[0472] Step 14:

[0473] Based on the analysis results, the next announcement content is personalized. Input: Analysis results. Output: Personalized next announcement content. Specifically, the server generates announcement content optimized for the user's individual needs based on the analysis results.

[0474] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0475] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0476] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0477] [Second embodiment]

[0478] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0479] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0480] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0481] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0482] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0483] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0484] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0485] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0486] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0487] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0488] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0489] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0490] The present invention is a system that provides effective learning support for users aiming to obtain qualifications, and is characterized by learning support using generative AI models and characters. The following describes in detail the embodiments of the present invention.

[0491] 1. User registration and character creation

[0492] A user accesses the system, enters the required information such as name, email address, and password on the new registration page, and submits it. The server receives the entered information and stores it in a database. The server then sends a confirmation email to the user's email address. When the user clicks on the link in the confirmation email, the account is confirmed. Once confirmation is complete, the user proceeds to a page where they can select or create their favorite character. On this page, an interface is displayed to customize the character's appearance and behavior. Once the user has finished configuring their character, the information is sent to the server and stored.

[0493] 2. Select study materials and start studying

[0494] Users select the type of qualification within the system and choose the corresponding study materials. This selection is sent to the server, which loads the selected study material data into the generative AI model. Based on the loaded data, the server generates a curriculum specifically for the user, and the user is ready to study.

[0495] 3. Conducting a learning session

[0496] The user's device displays a screen for the learning session, simultaneously displaying the generated character and the learning content. The character explains the content of the learning material with voice and movement. The server monitors the user's progress in real time and stores this information in a database. If the user has any questions, they can enter them and the questions are sent to the server. The server uses a generative AI model to instantly generate answers and communicate them to the user through the character.

[0497] 4. Measurement and feedback

[0498] When a user finishes a learning session, they are automatically tested for their level of mastery. This test is administered by the server, and the test results are analyzed. Based on the analysis, the server identifies the user's weaknesses and areas for improvement, and personalizes the next learning session accordingly. This allows users to continue learning efficiently and in line with their individual needs.

[0499] Specific examples

[0500] For example, a user registers in the system aiming to obtain the "Bookkeeping Level 3" qualification. First, they access the system and create a new account. Next, they create a custom character based on Pikachu. After creating the character, they select the "Bookkeeping Level 3" study materials and begin a study session. Pikachu explains basic accounting concepts, and whenever the user asks a question, the generative AI model provides an instant answer. After the study session ends, a mastery test is automatically conducted, and the next study content is personalized based on the test results.

[0501] In this way, the system of the present invention can provide efficient and individually optimized learning support to users 24 hours a day, 365 days a year.

[0502] The processing flow will be explained below.

[0503] Step 1:

[0504] A user accesses the system and opens the new registration page. The terminal displays the registration form.

[0505] Step 2:

[0506] The user enters the required information (name, email address, password, etc.) and presses the send button. The device sends the input information to the server.

[0507] Step 3:

[0508] The server receives the input data and stores it in a database, then sends a confirmation email to the user's email address.

[0509] Step 4:

[0510] The user confirms the account by clicking the link in the confirmation email. The device clicks the link and communicates with the server.

[0511] Step 5:

[0512] Once the account is verified, the user is directed to a page where they can select or create their desired character, and the device will display a character generation tool.

[0513] Step 6:

[0514] The user customizes the character's appearance and movements, then presses a button to save the settings. The device then sends the settings information to the server.

[0515] Step 7:

[0516] The server receives the character setting information and stores it in a database.

[0517] Step 8:

[0518] The user selects the type of qualification and the corresponding learning material, and the terminal displays the qualification list.

[0519] Step 9:

[0520] The user selects the qualifications they wish to study and chooses the study materials, and the selections are sent to the server.

[0521] Step 10:

[0522] The server loads the necessary data into the generative AI model based on the selected teaching materials and generates a curriculum.

[0523] Step 11:

[0524] The learning session screen is displayed on the user's device. The device provides an interface that simultaneously displays the character and the learning material content.

[0525] Step 12:

[0526] The character explains the contents of the learning material using voice and actions, and the user can learn by watching this.

[0527] Step 13:

[0528] The server tracks the user's progress in real time and records it in a database.

[0529] Step 14:

[0530] The user enters a question or concern, and the device sends the question to the server.

[0531] Step 15:

[0532] The server uses a generative AI model to instantly generate an answer and send it to the device, where a character relays the answer to the user.

[0533] Step 16:

[0534] The user presses a button to end the learning session, and the device performs the termination process.

[0535] Step 17:

[0536] The server automatically conducts a learning level test, and the test screen is displayed on the user's device.

[0537] Step 18:

[0538] The user takes the test. The server analyzes the test results.

[0539] Step 19:

[0540] Based on the analysis results, the server extracts the user's weaknesses and areas for improvement and personalizes the next learning content.

[0541] Step 20:

[0542] The server sends personalized feedback and the next curriculum to the user's device, which displays it.

[0543] In this way, the system provides efficient and individually optimized learning support to users 24 hours a day, 365 days a year.

[0544] Example 1

[0545] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0546] In conventional qualification learning systems, it was difficult to monitor each user's learning progress individually and provide feedback in real time. Furthermore, there was a lack of a flexible system that could immediately respond to users' questions, which often led to a decline in learning efficiency. Furthermore, there were limited systems for personalizing users' learning content, making it difficult to provide learning curricula that met individual needs.

[0547] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0548] In this invention, the server includes means for generating an account based on information entered by the user and sending a confirmation email, means for selecting or generating a character preferred by the user, means for pre-learning learning material data using a generative AI model specialized for qualifications, means for conducting qualification-related lessons using the selected or generated character, means for generating a curriculum dedicated to the user, means for tracking and recording learning progress in real time, means for generating answers to user questions using a generative AI model and communicating them through the character, means for automatically conducting a mastery test after completion of learning and analyzing the results, and means for personalizing the next learning content based on the analysis results, thereby enabling efficient and individually optimized qualification learning.

[0549] "Means for creating an account and sending a confirmation email" refers to a method in which a user is prompted to enter the necessary information when registering with the system, a user account is created based on that information, and an email is automatically sent to confirm the registration.

[0550] The "means for selecting or generating a character" is a method for allowing a user to select or generate a virtual character to be used when studying, according to the user's preferences.

[0551] "Means of using a generative AI model to learn teaching material data in advance" refers to a method in which teaching material data related to a specific qualification is input into the AI ​​model in advance, and the AI ​​model learns based on that data.

[0552] The "means for conducting qualification-related lessons" is a method for using the generated character to conduct lessons to provide users with the knowledge necessary to obtain qualifications.

[0553] The "means for generating a curriculum specifically for a user" is a method for automatically creating an individually optimized learning plan according to the user's qualification acquisition goals and progress.

[0554] The "means for tracking and recording learning progress" is a method for monitoring a user's learning activities in real time and recording the progress in a database or the like.

[0555] "Means of generating answers using a generative AI model and communicating them through a character" refers to a method in which, when a user inputs a question, a generative AI model generates an answer to that question and communicates it to the user through a virtual character.

[0556] The "means for conducting a mastery test and analyzing the results" is a method for automatically conducting a test after the completion of a learning session and analyzing the results.

[0557] "Means for personalizing the next learning content" refers to a method for individually optimizing the next learning content based on the results of the mastery test and the user's progress.

[0558] The present invention is a system that provides effective learning support to users aiming to obtain qualifications, characterized by learning support using a generative AI model and a character. Specifically, the system includes means for generating an account based on information entered by the user and sending a confirmation email, means for selecting or generating a character preferred by the user, means for pre-learning learning material data using a generative AI model specialized for qualifications, means for conducting qualification-related lessons using the selected or generated character, means for generating a curriculum dedicated to the user, means for tracking and recording learning progress in real time, means for generating answers to user questions using a generative AI model and communicating them through a character, means for automatically conducting a mastery test after completion of learning and analyzing the results, and means for personalizing the next learning content based on the analysis results.

[0559] Users access this system using a web browser as an interface. When a user enters their name, email address, and password on the new registration page and submits it, the server receives the information and stores it in a MySQL database. The server then uses a mail server to send a confirmation email. This email contains a URL link for account confirmation, and the user completes account confirmation by clicking the link. Once confirmation is complete, the user can proceed to the character creation page, where an interface is provided for customizing the character's appearance and behavior. Once the user has finished configuring their character, the information is sent back to the server and stored in the database.

[0560] The user selects the type of qualification within the system and chooses the corresponding learning materials. The server loads the selected learning material data (PDF, video, audio files, etc.) into a generative AI model (e.g., OpenAI GPT-4) and generates a curriculum specifically for the user. The user is then ready to learn. The learning session screen is displayed on the user's device, displaying the generated character and the learning content simultaneously. The character explains the content of the learning materials with voice and movement. Voice synthesis software may also be used here.

[0561] The server monitors the user's progress in real time and stores that information in a database. When the user enters a question into a text box and submits it, the server inputs the question into a generative AI model to generate an answer. The answer is then communicated to the user via voice via a character.

[0562] Once a user has completed a learning session, they are automatically tested for their progress. The test is administered by the server and the results are analyzed in real time. Based on the test results, the user's weaknesses and areas for improvement are identified, and the next learning session is personalized.

[0563] As a concrete example, let's consider a case where a user is aiming for the "Bookkeeping Level 3" qualification. The user accesses the system and creates a new account. Next, they create a custom character based on Pikachu. After creating the character, they select the "Bookkeeping Level 3" study materials and begin a study session. Pikachu explains basic accounting concepts, and when the user enters a question about a specific journal entry, the server sends prompts to the generative AI model, providing an immediate answer. After the study session, a mastery test is automatically conducted, and the next study content is personalized based on the test results.

[0564] An example of a prompt sentence is, "Please explain basic journal entries as preparation for the Bookkeeping Level 3 exam." This enables the system to provide efficient, individually optimized learning support to users 24 hours a day, 365 days a year.

[0565] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0566] Step 1: The user accesses the system, enters their name, email address, and password on the new registration page, and submits the information. The entered information is sent to the server.

[0567] Input: User registration information (name, email address, password)

[0568] Output: Registration information sent to the server

[0569] What happens: The user fills in the form and clicks the submit button.

[0570] Step 2: The server receives the entered information and performs validation checks.

[0571] Input: User registration information

[0572] Output: Validation result (success or error)

[0573] What happens: The server validates the information entered to ensure it is in the correct format.

[0574] Step 3: If validation is successful, the server saves the registration information in a MySQL database.

[0575] Input: Validated registration information

[0576] Output: Save results to database

[0577] Specific operation: The server inserts the registration information into the user table in the database.

[0578] Step 4: The server sends a confirmation email to the user's email address, which contains a URL link to verify the account.

[0579] Input: User's email address and confirmation URL

[0580] Output: Confirmation email sent

[0581] Specific operation: The server sends a confirmation email via the mail server.

[0582] Step 5: The user opens the confirmation email and clicks on the confirmation link, which causes the server to activate the user's account.

[0583] Input: User confirmation link click

[0584] Output: Account enabled

[0585] Specific behavior: The server detects the click on the confirmation link and updates the account status accordingly.

[0586] Step 6: The user proceeds to the character generation page to customize the character's appearance and behavior.

[0587] Input: User's character setting information

[0588] Output: Customized character information

[0589] Specific actions: The user selects the character's appearance and actions through the interface and submits the configuration information.

[0590] Step 7: The server receives the character setting information and stores it in a database.

[0591] Input: User's character setting information

[0592] Output: Saved character information

[0593] Specific operation: The server inserts the character setting information into the character table of the database.

[0594] Step 8: The user selects the type of qualification and chooses the corresponding study material.

[0595] Input: User credential selection information

[0596] Output: Selected teaching material data

[0597] Specific Actions: The user selects a category of qualifications and then selects specific learning materials within that category.

[0598] Step 9: The server loads the selected teaching material data into the generative AI model.

[0599] Input: Selected teaching material data

[0600] Output: Teaching material data loaded into the generative AI model

[0601] Specific operation: The server inputs the teaching material data into the learning engine of the generative AI model.

[0602] Step 10: The server generates a curriculum specific to the user and stores it in the database.

[0603] Input: Teaching material data from a generative AI model

[0604] Output: User-specific curriculum

[0605] Specific operation: The server automatically generates a curriculum based on the user's progress and goals.

[0606] Step 11: A screen for the learning session is displayed on the user terminal.

[0607] Input: User-specific curriculum data

[0608] Output: Learning session screen

[0609] What happens: A user starts a learning session on a device and the interface is displayed.

[0610] Step 12: The character explains the content of the teaching material with voice and movement.

[0611] Input: Curriculum data and character setting information

[0612] Output: Audio explanation of teaching materials

[0613] Specific actions: The character explains to the user through movement and voice based on the content of the teaching material.

[0614] Step 13: The server monitors the user's progress in real time and stores the information in a database.

[0615] Input: User's learning progress data

[0616] Output: Saved learning progress data

[0617] Specific operation: The server records in real time how much progress the user has made in their studies.

[0618] Step 14: The user enters and submits a question during the study.

[0619] Input: User question text

[0620] Output: The question sent to the server

[0621] Specific actions: The user enters a question into the interface and clicks the submit button.

[0622] Step 15: The server inputs a question into the generative AI model, which generates an answer.

[0623] Input: User question text

[0624] Output: Answer from the generative AI model

[0625] Specific operation: The server inputs the question as a prompt into the AI ​​model, which generates an answer.

[0626] Step 16: The generated answer is spoken to the user through the character.

[0627] Input: Answer from a generative AI model

[0628] Output: Audio explanation of answers

[0629] Specific operation: The character vocalizes the answer from the generated AI model and conveys it to the user.

[0630] Step 17: When the user finishes the learning session, an automatic mastery test is administered.

[0631] Input: User's learning session end signal

[0632] Output: Start of mastery test

[0633] Specific operation: The server detects the end of the learning session and displays a test screen.

[0634] Step 18: The server analyzes the test results in real time and records the user's score and correct answer rate.

[0635] Input: User test answer data

[0636] Output: Parsed test results

[0637] Specific operation: The server analyzes the user's answers and calculates the score and percentage of correct answers.

[0638] Step 19: The server personalizes the next learning content based on the analysis results.

[0639] Input: Parsed test results

[0640] Output: Personalized upcoming curriculum

[0641] Specific operation: The server individually optimizes the next learning content based on the user's weaknesses and areas for improvement.

[0642] The above is the specific flow of the program processing of this system.

[0643] (Application example 1)

[0644] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0645] Traditional online learning systems offer limited interactive learning support and lack personalized feedback. They also lack real-time progress tracking and a virtual learning experience, which can reduce learning effectiveness. Furthermore, it is difficult for users to get immediate answers to their questions, which reduces learning efficiency.

[0646] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0647] In this invention, the server includes means for pre-learning learning material data using a generative AI model specialized for qualifications, means for selecting or generating a character preferred by the user, means for teaching qualification-related lessons using the selected or generated character, means for tracking and recording learning progress in real time, means for generating answers to the user's questions using the generative AI model and communicating them through the character, means for automatically conducting a mastery test after completion of learning and analyzing the results, means for personalizing the next learning content based on the analysis results, means for accessing as an avatar in a virtual space and providing interactive learning support, and means for the character to explain the learning material in a learning area in the virtual space. This allows users to receive personalized learning support, enabling effective and efficient learning.

[0648] A "qualification-specific generative AI model" is an artificial intelligence model designed to learn teaching material data corresponding to a specific qualification.

[0649] "Means for selecting or generating a character" refers to a mechanism such as an interface that allows a user to select a character of their choice or generate a new one.

[0650] "Means for teaching qualification-related lessons" is a function for teaching knowledge related to qualifications using selected or generated characters.

[0651] The "means for tracking and recording learning progress in real time" is a mechanism for monitoring the progress of a user during learning in real time and recording that data.

[0652] "A means of generating answers to questions using a generative AI model and communicating them through a character" is a function in which, when a user inputs a question, the generative AI model instantly generates an answer, and communicates that answer via voice or text through a character.

[0653] "Means for automatically conducting a mastery test after completion of learning and analyzing the results" is a function that automatically conducts a test after the learning session ends and analyzes the results.

[0654] "Means for personalizing the next learning content" refers to a system that optimizes and sets the next learning content for each individual user based on the test results.

[0655] "Means for providing interactive learning support through access as an avatar in a virtual space" is a system that allows users to log in to a virtual space, participate as an avatar, and receive interactive learning support.

[0656] "Means for a character to explain learning materials in a learning area within a virtual space" is a function that sets up a specific learning area within a virtual space and allows a character to explain learning materials there.

[0657] The present invention is a system that provides effective learning support for users aiming to obtain qualifications, and is characterized by learning support using generative AI models and characters. A specific method for implementing the present invention will be described in detail below.

[0658] First, when a user accesses the system and enters the required information such as name, email address, and password on the new registration page, the server receives the information and stores it in a database. Next, the server sends a confirmation email to the user's email address, and the user confirms the account by clicking the link in the confirmation email. Once confirmation is complete, the user proceeds to a page where they can select or create their favorite character. On this page, an interface is displayed where the user can customize the character's appearance and behavior.

[0659] Once the user has set up their character, the information is sent to the server and saved. Next, the user selects the type of qualification within the system and chooses the corresponding learning materials. This selection is sent to the server, which loads the selected learning materials data into a generative AI model and generates a curriculum specifically for the user.

[0660] The user's device displays a screen for the learning session, simultaneously displaying the generated character and the learning content. The character explains the content of the learning material with voice and movement. This allows the user to access the virtual space as an avatar and receive interactive learning support. If the user has any questions, they can enter them and the questions are sent to the server. The server uses a generative AI model to instantly generate an answer and communicates it to the user through the character.

[0661] After the learning session, a mastery test is automatically conducted. This test is administered by the server, and the test results are analyzed. Based on the analysis results, the server extracts the user's weaknesses and areas for improvement, and personalizes the next learning content based on these. This allows the user to receive efficient, individually optimized learning support.

[0662] Specific hardware used includes smartphones, smart glasses, or head-mounted displays (e.g., Oculus Rift). AWS and Google Cloud can be used as server platforms. Advanced artificial intelligence models such as GPT-4 are used as generative AI models. Character animation is performed using software such as Unity and Unreal Engine.

[0663] As a concrete example, consider the case where a user registers with the system aiming for the "Bookkeeping Level 3" qualification. First, the user accesses the system and creates a new account. Next, the user generates a character, selects the "Bookkeeping Level 3" study materials, and starts a study session. The character explains basic accounting concepts, and whenever the user asks a question, the generative AI model provides an immediate answer. For example, in response to a question such as "Please tell me the basics of journal entries," the character provides a detailed explanation. After the study session ends, a mastery test is conducted, and the next study content is personalized based on the test results.

[0664] An example of a prompt sentence for a generative AI model is, "The user is asking about the depreciation method for fixed assets. Please explain, including specific examples (for example, how to depreciate a machine worth 1 million yen over 10 years)."

[0665] In this way, the system of the present invention can provide efficient and individually optimized learning support to users 24 hours a day, 365 days a year.

[0666] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0667] Step 1:

[0668] User Registration

[0669] Input: The user enters their name, email address, and password.

[0670] Processing: The server receives this information and stores it in a database.

[0671] Output: A confirmation email is sent to the user.

[0672] How it works: The user clicks on the link in the confirmation email to confirm the account. The server activates the account.

[0673] Step 2:

[0674] Character Generation

[0675] Input: The user customizes the character's appearance and behavior.

[0676] Processing: The server receives the character setting information and stores it in the database.

[0677] Output: Customized character information.

[0678] Behavior: The user configures the character's appearance and behavior on the character customization screen and sends the settings to the server.

[0679] Step 3:

[0680] Selection of teaching materials

[0681] Input: User selects the qualification type and corresponding study material.

[0682] Processing: The server loads the selected learning materials into a generative AI model to generate a personalized curriculum.

[0683] Output: A personalized learning curriculum.

[0684] How it works: A user selects a certification type within the system and then selects study materials to begin learning.

[0685] Step 4:

[0686] Start a study session

[0687] Input: A request to start a learning session from the user's device.

[0688] Processing: The server sends the character and educational material data to the user's terminal.

[0689] Output: Screen display for the learning session.

[0690] How it works: A character and learning materials are displayed on the user's device, and the learning session begins. The character explains the content.

[0691] Step 5:

[0692] Question and Answering

[0693] Input: The question asked by the user.

[0694] Processing: The server sends the question to the generative AI model, which generates an answer.

[0695] Output: Display the answer on the user's terminal.

[0696] How it works: Users enter a question and the answer is instantly provided by a generative AI model. A character will then relay the answer.

[0697] Step 6:

[0698] Progress Tracking

[0699] Input: User's learning progress data.

[0700] Processing: The server tracks the progress in real time and stores it in a database.

[0701] Output: Updated progress data.

[0702] How it works: As you learn, your progress is recorded in real time.

[0703] Step 7:

[0704] Completion of study and mastery test

[0705] Input: Training session end signal.

[0706] Processing: The server automatically conducts the mastery test and analyzes the results.

[0707] Output: Test results and analysis report.

[0708] How it works: After the user has completed the learning, an automatic learning test is administered, and the results are sent to the server for analysis.

[0709] Step 8:

[0710] Personalize your next lesson

[0711] Input: Test results and analytical data.

[0712] Processing: The server customizes the next learning content based on the results.

[0713] Output: A personalized next-day learning curriculum.

[0714] How it works: Learning content is set based on the user's test results and is reflected in the next learning session.

[0715] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0716] The present invention is a system that provides effective learning support for users aiming to obtain qualifications, and is characterized by a generative AI model and character learning support, as well as an emotion engine that recognizes the user's emotions. The following describes in detail the embodiments of the present invention.

[0717] 1. User registration and character creation

[0718] A user accesses the system, enters the required information such as name, email address, and password on the new registration page, and submits it. The server receives the entered information and stores it in a database. The server then sends a confirmation email to the user's email address. When the user clicks on the link in the confirmation email, the account is confirmed. Once confirmation is complete, the user proceeds to a page where they can select or create their favorite character. On this page, an interface is displayed to customize the character's appearance and behavior. Once the user has finished configuring their character, the information is sent to the server and stored.

[0719] 2. Select study materials and start studying

[0720] Users select the type of qualification within the system and choose the corresponding study materials. This selection is sent to the server, which loads the selected study material data into the generative AI model. Based on the loaded data, the server generates a curriculum specifically for the user, and the user is ready to study.

[0721] 3. Conducting a learning session

[0722] The user's device displays a screen for the learning session, simultaneously displaying the generated character and the learning content. The character explains the content of the learning material with voice and actions. An emotion engine then recognizes the user's emotions from their facial expressions and vocalizations and provides real-time feedback. For example, if the emotion engine determines that the user is confused, the character will provide additional explanations and encouragement according to that emotion. The server monitors the user's progress and emotional changes in real time and stores this information in a database. If the user has any questions, they can enter them and the questions are sent to the server. The server then uses a generative AI model to instantly generate answers and communicate them to the user through the character.

[0723] 4. Measurement and feedback

[0724] When a user finishes a learning session, they are automatically tested for their level of mastery. This test is administered by the server, and the test results are analyzed. Based on the analysis results, the server identifies the user's weaknesses and areas for improvement, and personalizes the next learning content based on these. In addition, emotional data recognized by the emotion engine is also added to the analysis, enabling more accurate feedback and curriculum adjustments. This allows users to continue learning efficiently and in line with their individual needs and emotions.

[0725] Specific examples

[0726] For example, a user registers in the system aiming to obtain the "Bookkeeping Level 3" qualification. First, they access the system and create a new account. Next, they create a custom character based on Pikachu. After creating the character, they select the "Bookkeeping Level 3" study materials and begin a study session. Pikachu explains basic accounting concepts, and whenever the user asks a question, the generative AI model provides an instant answer. If the emotion engine detects a confused expression on the user's face during the study session, Pikachu will provide additional explanations or encouragement. After the study session ends, a mastery test is automatically conducted, and the next study content is personalized based on the test results.

[0727] In this way, by combining a generative AI model and an emotion engine, the system of the present invention is able to provide efficient and individually optimized learning support to users 24 hours a day, 365 days a year.

[0728] The processing flow will be explained below.

[0729] MODE FOR CARRYING OUT THE INVENTION (INCLUDING EMOTION ENGINE)

[0730] Step 1:

[0731] A user accesses the system and opens the new registration page. The terminal displays the registration form.

[0732] Step 2:

[0733] The user enters the required information (name, email address, password, etc.) and presses the send button. The device sends the input information to the server.

[0734] Step 3:

[0735] The server receives the input data and stores it in a database, then sends a confirmation email to the user's email address.

[0736] Step 4:

[0737] The user confirms their account by clicking the link in the confirmation email, and the device clicks the link to communicate with the server.

[0738] Step 5:

[0739] Once the account is verified, the user is directed to a page where they can select or create their desired character, and the device will display a character generation tool.

[0740] Step 6:

[0741] The user customizes the character's appearance and movements, then presses a button to save the settings. The device then sends the settings information to the server.

[0742] Step 7:

[0743] The server receives the character setting information and stores it in a database.

[0744] Step 8:

[0745] The user selects the type of qualification and the corresponding learning material, and the terminal displays the qualification list.

[0746] Step 9:

[0747] The user selects the qualifications they wish to study and chooses the study materials, and the selections are sent to the server.

[0748] Step 10:

[0749] The server loads the necessary data into the generative AI model based on the selected teaching materials and generates a curriculum.

[0750] Step 11:

[0751] The learning session screen is displayed on the user's device. The device provides an interface that simultaneously displays the character and the learning material content.

[0752] Step 12:

[0753] The character explains the contents of the learning material using voice and actions, and the emotion engine analyzes the user's facial expressions and vocalizations in real time to recognize the user's emotional state.

[0754] Step 13:

[0755] The emotion engine sends feedback to the server based on the user's emotional state, for example suggesting additional explanation or encouragement if the user is confused.

[0756] Step 14:

[0757] The server receives feedback from the emotion engine and instructs the character to respond appropriately. The character then conveys feedback to the user according to the emotion.

[0758] Step 15:

[0759] The server tracks the user's progress and emotional changes in real time and records them in a database.

[0760] Step 16:

[0761] The user enters a question or concern, and the device sends the question to the server.

[0762] Step 17:

[0763] The server uses a generative AI model to instantly generate answers and communicate them to the user through a character.

[0764] Step 18:

[0765] The user presses a button to end the learning session, and the device performs the termination process.

[0766] Step 19:

[0767] The server automatically conducts a learning level test, and the test screen is displayed on the user's device.

[0768] Step 20:

[0769] The user takes the test. The server analyzes the test results.

[0770] Step 21:

[0771] Based on the analysis results, the server identifies the user's weaknesses and areas for improvement, and personalizes the next lesson. The emotion engine also analyzes the emotional data it recognizes, improving the quality of feedback and the curriculum.

[0772] Step 22:

[0773] The server sends personalized feedback and the next curriculum to the user's device, which displays it.

[0774] In this way, by combining a generative AI model and an emotion engine, this system is able to provide efficient and individually optimized learning support to users 24 hours a day, 365 days a year.

[0775] Example 2

[0776] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0777] Conventional learning support systems have had difficulty providing personalized learning tailored to the individual needs and progress of users. Furthermore, they lacked the ability to recognize and respond to users' emotions and confusion in real time, making it difficult to maximize the effectiveness of learning. There was a need to provide a system that would solve these problems and enable users to study for qualifications efficiently and effectively.

[0778] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0779] In this invention, the server includes a means for pre-training learning material data using a generative AI model related to qualifications, a means for using an agent selected or generated by the user, and a means for tracking the user's learning progress in real time and recording it in a database. This enables personalized learning according to the user's individual needs and learning progress. The agent also deepens the user's understanding by conducting lessons involving simple movements. Furthermore, the emotion engine recognizes the user's facial expressions and vocalizations in real time and reflects that feedback to maximize learning effectiveness.

[0780] A "generative AI model for qualifications" is an artificial intelligence model that pre-learns educational material data related to specific qualifications and provides answers and educational materials in response to user questions and customizations.

[0781] An "agent" is a character or avatar that the user can select or create, and that character or avatar is responsible for explaining the learning content.

[0782] "Tracking" refers to the process of tracking a user's learning progress and behavior in real time and recording it in a database.

[0783] "Database" means a data storage system for managing user data, progress, learning outcomes, etc. collected within the system.

[0784] An "emotion engine" is an algorithm or software component that analyzes a user's facial expressions and tone of voice in real time and recognizes their emotions.

[0785] A "proficiency test" is a test that is automatically administered after a user completes a learning session to assess the user's learning progress.

[0786] "Personalization" refers to the process of individually optimizing the next learning content and curriculum based on each user's learning progress and weaknesses.

[0787] This invention is a system that provides effective learning support to users aiming to obtain qualifications, and in addition to learning support using generative AI models and agents, it features an emotion engine that recognizes the user's emotions.

[0788] User registration and agent creation

[0789] A user accesses the system, enters the required information such as name, email address, and password on the new registration page, and submits it. The server receives the entered information and stores it in a database. The server then sends a confirmation email to the user's email address. When the user clicks on the link in the confirmation email, the account is confirmed.

[0790] After verifying their account, users are taken to a page where they can select or create their preferred agent. This page provides an interface for customizing the agent's appearance and behavior. Once the user has configured their agent, the information is sent to the server and stored in a database.

[0791] Select study materials and start learning

[0792] Users select the type of qualification within the system and choose the corresponding study materials. The selection is sent to the server, which loads the selected study material data into a generative AI model. For example, GPT-3 or BERT can be used as the generative AI model. Based on the loaded data, the server generates a curriculum specifically for the user, and the user is ready to begin learning.

[0793] Running a learning session

[0794] The user's device displays a screen for the learning session, simultaneously displaying the generated agent and the learning content. The agent explains the content of the learning material with voice and action. The emotion engine recognizes the user's emotions from their facial expressions and vocalizations and provides feedback in real time. For example, if the emotion engine determines that the user is confused, the agent will provide additional explanations and encouragement according to that emotion. The server monitors the user's progress and emotional changes in real time and stores this information in a database. If the user has any questions, they can enter them and the questions are sent to the server. The server uses a generative AI model to instantly generate answers and communicates them to the user via the agent.

[0795] Measurement and feedback

[0796] When a user finishes a learning session, they are automatically tested for their level of mastery. This test is administered by the server, and the test results are analyzed. Based on the analysis results, the server identifies the user's weaknesses and areas for improvement, and personalizes the next learning content based on these. In addition, emotional data recognized by the emotion engine is also added to the analysis, enabling more accurate feedback and curriculum adjustments. This allows users to continue learning efficiently and in line with their individual needs and emotions.

[0797] Specific examples

[0798] For example, a user registers with the system aiming for the "Bookkeeping Level 3" certification. First, they access the system and create a new account, then generate a custom agent based on Pikachu. After generating the agent, they select the "Bookkeeping Level 3" study materials and begin a learning session. The agent explains basic accounting concepts, and whenever the user asks a question, the generated AI model provides an immediate answer. If the emotion engine detects a confused expression on the user's face during the learning session, the agent will provide additional explanations or encouragement. After the learning session ends, a mastery test is automatically conducted, and the next learning content is personalized based on the test results.

[0799] Prompt Sentence Examples

[0800] "Please explain a learning support system using custom agents with emotion recognition."

[0801] By combining a generative AI model and an emotion engine, the system of this invention can provide efficient and individually optimized learning support to users 24 hours a day, 365 days a year.

[0802] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0803] Step 1:

[0804] A user accesses the system, enters the required information such as name, email address, and password on the new registration page, and submits it. The input includes the user's personal information (name, email address, password). The output is the user information sent to the server.

[0805] Step 2:

[0806] The server receives the entered user information and stores it in a database.

[0807] Input: User's personal information (name, email address, password)

[0808] Data processing: Convert received information into the appropriate format and store it in a database

[0809] Output: Save confirmation message

[0810] Step 3:

[0811] The server sends a confirmation email to the user's email address, and when the user clicks on the link in the email, the account is authenticated.

[0812] Input: User's email address

[0813] Data processing: Generating and sending confirmation emails

[0814] Output: Account verification link

[0815] Step 4:

[0816] After completing the verification, the user proceeds to the agent creation page and uses the interface to customize the agent's appearance and behavior.

[0817] Input: User's agent selection and customization information

[0818] Data processing: Importing and verifying selected information

[0819] Output: Agent customization data

[0820] Step 5:

[0821] The server receives the customized agent information and stores it in a database.

[0822] Input: Agent customization information

[0823] Data processing: Format conversion and storage of received information

[0824] Output: Agent creation confirmation message

[0825] Step 6:

[0826] Users select the type of qualification within the system and choose the corresponding study materials.

[0827] Input:Select qualification type

[0828] Data processing: Search for teaching material data corresponding to the selected qualification

[0829] Output: List of teaching material data

[0830] Step 7:

[0831] The server loads the selected teaching material data into a generative AI model and generates a curriculum tailored to the user.

[0832] Input: Teaching material data

[0833] Data processing: Loading teaching material data and generating curriculum

[0834] Output: User-specific curriculum

[0835] Step 8:

[0836] The learning session screen is displayed on the user's device. The generated agent and the learning content are displayed simultaneously, and the agent explains the learning content with voice and actions.

[0837] Input: User-specific curriculum and agent data

[0838] Data processing: Synchronization of screen display with voice and action

[0839] Output: Learning screen

[0840] Step 9:

[0841] The emotion engine recognizes the user's facial expressions and vocalizations in real time and provides feedback. For example, if it detects a confused expression, the agent will provide additional explanation or encouragement.

[0842] Input: User's facial expression data and voice data

[0843] Data processing: Sentiment analysis and feedback content generation

[0844] Output: Feedback content

[0845] Step 10:

[0846] If a user has a question while studying, they can enter it and send it to the server.

[0847] Input: User question

[0848] Data processing: Question analysis and answer generation using an AI model

[0849] Output: Response data

[0850] Step 11:

[0851] The server uses a generative AI model to generate answers and communicates them to the user through an agent.

[0852] Input: Question data

[0853] Data processing: Answer generation using generative AI models

[0854] Output: Response via agent

[0855] Step 12:

[0856] Once the user has finished the learning session, the mastery test will begin automatically.

[0857] Input: End of study session trigger

[0858] Data processing: Implementing learning level tests and compiling the results

[0859] Output: Test results

[0860] Step 13:

[0861] The server analyzes the test results and extracts the user's strengths and weaknesses.

[0862] Input: Test result data

[0863] Data processing: Analyzing test results and identifying weaknesses

[0864] Output: Analysis results

[0865] Step 14:

[0866] The server personalizes the next learning content based on the analysis results and provides feedback taking into account the emotional data recognized by the emotion engine.

[0867] Input: Analysis results and sentiment data

[0868] Data processing: Customizing the next learning content

[0869] Output: Personalized learning and feedback

[0870] (Application example 2)

[0871] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0872] Conventional learning support systems have difficulty providing effective learning support based on the user's individual emotions and progress, and are insufficiently personalized to meet individual needs. Furthermore, when dealing with customers in brick-and-mortar stores, it is difficult to provide efficient explanations and customer service that reflect the customer's emotions and needs in real time. There is a need for a system that can solve these issues and provide support and services that are optimized for each user and customer.

[0873] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for pre-learning learning material data using a generative AI model specialized for qualifications; means for selecting or generating a character preferred by the user; means for conducting qualification-related lessons using the selected or generated character; means for tracking and recording learning progress in real time; means for generating answers to the user's questions using the generative AI model and communicating them through the character; means for automatically conducting a mastery test after completion of learning and analyzing the results; means for personalizing the next learning content based on the analysis results; means for analyzing the user's emotions using an emotion recognition engine and adjusting the character's movements and speech based on the data; and means for providing personalized product explanations and customer service to customers in stores as part of personalization. This enables effective learning support that reflects the user's emotions and progress in real time, making it possible to provide explanations and customer service optimized for each customer even in physical stores.

[0874] A "generative AI model specialized for qualifications" is an artificial intelligence model that pre-learns educational material data related to obtaining specific qualifications and generates appropriate answers and explanations.

[0875] An "emotion recognition engine" is a system that analyzes the emotions of a user, such as facial expressions and vocalizations, in real time, and extracts and utilizes that emotional data.

[0876] A "character" is a virtual avatar or character selected or created by the user that acts as a facilitator of learning or guidance.

[0877] "Means for tracking and recording learning progress in real time" refers to a mechanism for monitoring a user's learning activities and progress in real time and recording that information.

[0878] "Generating answers using a generative AI model" means that artificial intelligence generates appropriate answers to questions from users based on information it has learned in advance.

[0879] "Means of communicating through a character" refers to a method of communicating the generated answer or explanation to the user through the voice or actions of a character selected by the user.

[0880] The "means for automatically conducting a mastery test after completion of learning" is a mechanism for automatically conducting a test to measure the user's level of understanding after the learning session is completed.

[0881] The "means for analyzing the results" is a method for analyzing the results of the mastery test in detail and understanding the user's strengths and weaknesses.

[0882] "Means for personalizing the next learning content" is a mechanism that provides learning content optimized for the user's individual needs and progress based on the analysis results.

[0883] "Means for providing personalized product explanations and customer service" refers to a system for providing appropriate product explanations and customer service in physical stores based on the individual emotions and needs of customers.

[0884] The present invention relates to a system that provides learning support and customer service support in physical stores using a generative AI model and an emotion recognition engine. Specific examples for implementing the present invention are described in detail below.

[0885] 1. System Configuration

[0886] This system provides effective learning support for users who are aiming to serve customers in brick-and-mortar stores and obtain qualifications. The system is primarily composed of the following hardware and software:

[0887] Hardware: Smartphones, smart glasses

[0888] Software: Mobile app, server-side program (Python + Flask), database (MySQL), generative AI model (OpenAI's GPT-4), emotion recognition engine (Microsoft Azure Face API)

[0889] 2. Program Processing

[0890] Customer registration and character creation

[0891] The server creates an account based on the information the user entered and sends a confirmation email. When the user clicks the confirmation link, the account is activated and the user is then taken to the character creation page. The user customizes the character's appearance and behavior to their liking, and this information is sent to the server and stored in a database.

[0892] Product selection and introduction

[0893] Users select a product category through their smart device. The selection is sent to the server, which then loads the selected product data into the generative AI model, which then creates a personalized guidance curriculum for the customer based on the data.

[0894] Running a store navigation session

[0895] An information screen is displayed on the user's device, and a generated character explains the product. An emotion recognition engine analyzes the user's emotions from their facial expressions and vocalizations and provides feedback in real time. The server monitors the user's progress and emotions in real time and stores them in a database. When the user enters a question, the generative AI model instantly generates an answer and communicates it to the user through the character.

[0896] Measurement and feedback

[0897] After the guidance session ends, a satisfaction test is automatically conducted by the server. The test results are analyzed in detail by the server and feedback is provided to identify the user's strengths and weaknesses. Based on the analysis results, the next guidance content is personalized.

[0898] As a specific example, consider the case where a user visits a physical store and uses smart glasses to receive product information. At this time, a character explains the product based on product data previously learned by the generative AI model. If the emotion recognition engine detects a confused expression on the user's face, the character will provide additional explanation to help the user understand. In this way, effective customer service is possible, reflecting the user's emotions and progress in real time.

[0899] Example prompt sentence:

[0900] "Based on the product category selected by the customer, we use an emotion recognition engine to analyze the user's emotions and generate the most appropriate product description."

[0901] As described above, by combining a generative AI model and an emotion recognition engine, this system can provide individually optimized learning support and customer service to users and customers.

[0902] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0903] Step 1:

[0904] The user accesses the app from smart glasses or a smartphone, enters the required information such as name, email address, and password on the new registration page, and submits it. Input: Registration information entered by the user. Output: Registration information is sent to the server and saved in the database. Specifically, after the server receives the entered information, it saves it as a new record in the database.

[0905] Step 2:

[0906] The server sends a confirmation email to the user's email address. Input: The user's email address included in the registration information. Output: A confirmation email is sent to the user's email address. Specifically, the server generates and sends the confirmation email using the SMTP protocol.

[0907] Step 3:

[0908] When the user clicks the link in the confirmation email, the account is confirmed. Input: The confirmation link the user clicked. Output: The server activates the account. Specifically, the server updates the user's account record and sets the confirmation flag.

[0909] Step 4:

[0910] The user creates or selects a character of their choice. Input: Customization information for the appearance and behavior of the character selected by the user. Output: Character creation information is sent to the server and saved in a database. Specifically, the server receives the character information set by the user and saves it in a database.

[0911] Step 5:

[0912] The user selects a product category through the app. Input: The product category selected by the user. Output: The product category information is sent to the server and loaded into the generative AI model. Specifically, the server receives the product category information and passes it to the generative AI model.

[0913] Step 6:

[0914] The generative AI model generates a guidance curriculum based on product data. Input: Product category information. Output: Generated guidance curriculum. Specifically, the generative AI model analyzes the given product category information and generates an appropriate guidance curriculum.

[0915] Step 7:

[0916] An information screen is displayed on the user's device, and the generated character explains the product. Input: Generated information curriculum. Output: Product description displayed on the device. Specifically, the character explains the product using voice and actions based on the information curriculum obtained from the generation AI model.

[0917] Step 8:

[0918] The emotion recognition engine analyzes the user's facial expressions and vocalizations to recognize emotions. Input: User's facial expression and vocalization data. Output: Recognized emotion data. Specifically, the emotion recognition engine analyzes the user's facial images and voice data in real time to extract their emotional state.

[0919] Step 9:

[0920] The server monitors the user's progress and emotions in real time and records them in a database. Input: User's progress and emotion data. Output: Progress and emotion data stored in the database. Specifically, the server receives the user's learning progress and emotion data and stores it in the database.

[0921] Step 10:

[0922] The user inputs a question and sends it to the server. Input: User's question. Output: Answer generated by the generative AI model. Specifically, the server receives the question from the user and uses the generative AI model to generate an appropriate answer.

[0923] Step 11:

[0924] The generated answer is communicated to the user through the character. Input: The answer generated by the generative AI model. Output: The answer communicated to the user. The specific action is for the character to communicate the answer to the user through voice and action.

[0925] Step 12:

[0926] After the guidance session ends, a satisfaction test is automatically conducted. Input: Learning / guidance session end trigger. Output: Satisfaction test result. Specifically, the server detects the end of the session and automatically conducts a satisfaction test.

[0927] Step 13:

[0928] The server analyzes the test results and identifies the user's strengths and weaknesses. Input: Satisfaction test results. Output: Analysis results. Specifically, the server analyzes the test results in detail to extract the user's level of understanding and areas for improvement.

[0929] Step 14:

[0930] Based on the analysis results, the next announcement content is personalized. Input: Analysis results. Output: Personalized next announcement content. Specifically, the server generates announcement content optimized for the user's individual needs based on the analysis results.

[0931] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0932] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0933] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0934] [Third embodiment]

[0935] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0936] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0937] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0938] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0939] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0940] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0941] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0942] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0943] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0944] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0945] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0946] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0947] The present invention is a system that provides effective learning support for users aiming to obtain qualifications, and is characterized by learning support using generative AI models and characters. The following describes in detail the embodiments of the present invention.

[0948] 1. User registration and character creation

[0949] A user accesses the system, enters the required information such as name, email address, and password on the new registration page, and submits it. The server receives the entered information and stores it in a database. The server then sends a confirmation email to the user's email address. When the user clicks on the link in the confirmation email, the account is confirmed. Once confirmation is complete, the user proceeds to a page where they can select or create their favorite character. On this page, an interface is displayed to customize the character's appearance and behavior. Once the user has finished configuring their character, the information is sent to the server and stored.

[0950] 2. Select study materials and start studying

[0951] Users select the type of qualification within the system and choose the corresponding study materials. This selection is sent to the server, which loads the selected study material data into the generative AI model. Based on the loaded data, the server generates a curriculum specifically for the user, and the user is ready to study.

[0952] 3. Conducting a learning session

[0953] The user's device displays a screen for the learning session, simultaneously displaying the generated character and the learning content. The character explains the content of the learning material with voice and movement. The server monitors the user's progress in real time and stores this information in a database. If the user has any questions, they can enter them and the questions are sent to the server. The server uses a generative AI model to instantly generate answers and communicate them to the user through the character.

[0954] 4. Measurement and feedback

[0955] When a user finishes a learning session, they are automatically tested for their level of mastery. This test is administered by the server, and the test results are analyzed. Based on the analysis, the server identifies the user's weaknesses and areas for improvement, and personalizes the next learning session accordingly. This allows users to continue learning efficiently and in line with their individual needs.

[0956] Specific examples

[0957] For example, a user registers in the system aiming to obtain the "Bookkeeping Level 3" qualification. First, they access the system and create a new account. Next, they create a custom character based on Pikachu. After creating the character, they select the "Bookkeeping Level 3" study materials and begin a study session. Pikachu explains basic accounting concepts, and whenever the user asks a question, the generative AI model provides an instant answer. After the study session ends, a mastery test is automatically conducted, and the next study content is personalized based on the test results.

[0958] In this way, the system of the present invention can provide efficient and individually optimized learning support to users 24 hours a day, 365 days a year.

[0959] The processing flow will be explained below.

[0960] Step 1:

[0961] A user accesses the system and opens the new registration page. The terminal displays the registration form.

[0962] Step 2:

[0963] The user enters the required information (name, email address, password, etc.) and presses the send button. The device sends the input information to the server.

[0964] Step 3:

[0965] The server receives the input data and stores it in a database, then sends a confirmation email to the user's email address.

[0966] Step 4:

[0967] The user confirms the account by clicking the link in the confirmation email. The device clicks the link and communicates with the server.

[0968] Step 5:

[0969] Once the account is verified, the user is directed to a page where they can select or create their desired character, and the device will display a character generation tool.

[0970] Step 6:

[0971] The user customizes the character's appearance and movements, then presses a button to save the settings. The device then sends the settings information to the server.

[0972] Step 7:

[0973] The server receives the character setting information and stores it in a database.

[0974] Step 8:

[0975] The user selects the type of qualification and the corresponding learning material, and the terminal displays the qualification list.

[0976] Step 9:

[0977] The user selects the qualifications they wish to study and chooses the study materials, and the selections are sent to the server.

[0978] Step 10:

[0979] The server loads the necessary data into the generative AI model based on the selected teaching materials and generates a curriculum.

[0980] Step 11:

[0981] The learning session screen is displayed on the user's device. The device provides an interface that simultaneously displays the character and the learning material content.

[0982] Step 12:

[0983] The character explains the contents of the learning material using voice and actions, and the user can learn by watching this.

[0984] Step 13:

[0985] The server tracks the user's progress in real time and records it in a database.

[0986] Step 14:

[0987] The user enters a question or concern, and the device sends the question to the server.

[0988] Step 15:

[0989] The server uses a generative AI model to instantly generate an answer and send it to the device, where a character relays the answer to the user.

[0990] Step 16:

[0991] The user presses a button to end the learning session, and the device performs the termination process.

[0992] Step 17:

[0993] The server automatically conducts a learning level test, and the test screen is displayed on the user's device.

[0994] Step 18:

[0995] The user takes the test. The server analyzes the test results.

[0996] Step 19:

[0997] Based on the analysis results, the server extracts the user's weaknesses and areas for improvement and personalizes the next learning content.

[0998] Step 20:

[0999] The server sends personalized feedback and the next curriculum to the user's device, which displays it.

[1000] In this way, the system provides efficient and individually optimized learning support to users 24 hours a day, 365 days a year.

[1001] Example 1

[1002] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1003] In conventional qualification learning systems, it was difficult to monitor each user's learning progress individually and provide feedback in real time. Furthermore, there was a lack of a flexible system that could immediately respond to users' questions, which often led to a decline in learning efficiency. Furthermore, there were limited systems for personalizing users' learning content, making it difficult to provide learning curricula that met individual needs.

[1004] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1005] In this invention, the server includes means for generating an account based on information entered by the user and sending a confirmation email, means for selecting or generating a character preferred by the user, means for pre-learning learning material data using a generative AI model specialized for qualifications, means for conducting qualification-related lessons using the selected or generated character, means for generating a curriculum dedicated to the user, means for tracking and recording learning progress in real time, means for generating answers to user questions using a generative AI model and communicating them through the character, means for automatically conducting a mastery test after completion of learning and analyzing the results, and means for personalizing the next learning content based on the analysis results, thereby enabling efficient and individually optimized qualification learning.

[1006] "Means for creating an account and sending a confirmation email" refers to a method in which a user is prompted to enter the necessary information when registering with the system, a user account is created based on that information, and an email is automatically sent to confirm the registration.

[1007] The "means for selecting or generating a character" is a method for allowing a user to select or generate a virtual character to be used when studying, according to the user's preferences.

[1008] "Means of using a generative AI model to learn teaching material data in advance" refers to a method in which teaching material data related to a specific qualification is input into the AI ​​model in advance, and the AI ​​model learns based on that data.

[1009] The "means for conducting qualification-related lessons" is a method for using the generated character to conduct lessons to provide users with the knowledge necessary to obtain qualifications.

[1010] The "means for generating a curriculum specifically for a user" is a method for automatically creating an individually optimized learning plan according to the user's qualification acquisition goals and progress.

[1011] The "means for tracking and recording learning progress" is a method for monitoring a user's learning activities in real time and recording the progress in a database or the like.

[1012] "Means of generating answers using a generative AI model and communicating them through a character" refers to a method in which, when a user inputs a question, a generative AI model generates an answer to that question and communicates it to the user through a virtual character.

[1013] The "means for conducting a mastery test and analyzing the results" is a method for automatically conducting a test after the completion of a learning session and analyzing the results.

[1014] "Means for personalizing the next learning content" refers to a method for individually optimizing the next learning content based on the results of the mastery test and the user's progress.

[1015] The present invention is a system that provides effective learning support to users aiming to obtain qualifications, characterized by learning support using a generative AI model and a character. Specifically, the system includes means for generating an account based on information entered by the user and sending a confirmation email, means for selecting or generating a character preferred by the user, means for pre-learning learning material data using a generative AI model specialized for qualifications, means for conducting qualification-related lessons using the selected or generated character, means for generating a curriculum dedicated to the user, means for tracking and recording learning progress in real time, means for generating answers to user questions using a generative AI model and communicating them through a character, means for automatically conducting a mastery test after completion of learning and analyzing the results, and means for personalizing the next learning content based on the analysis results.

[1016] Users access this system using a web browser as an interface. When a user enters their name, email address, and password on the new registration page and submits it, the server receives the information and stores it in a MySQL database. The server then uses a mail server to send a confirmation email. This email contains a URL link for account confirmation, and the user completes account confirmation by clicking the link. Once confirmation is complete, the user can proceed to the character creation page, where an interface is provided for customizing the character's appearance and behavior. Once the user has finished configuring their character, the information is sent back to the server and stored in the database.

[1017] The user selects the type of qualification within the system and chooses the corresponding learning materials. The server loads the selected learning material data (PDF, video, audio files, etc.) into a generative AI model (e.g., OpenAI GPT-4) and generates a curriculum specifically for the user. The user is then ready to learn. The learning session screen is displayed on the user's device, displaying the generated character and the learning content simultaneously. The character explains the content of the learning materials with voice and movement. Voice synthesis software may also be used here.

[1018] The server monitors the user's progress in real time and stores that information in a database. When the user enters a question into a text box and submits it, the server inputs the question into a generative AI model to generate an answer. The answer is then communicated to the user via voice via a character.

[1019] Once a user has completed a learning session, they are automatically tested for their progress. The test is administered by the server and the results are analyzed in real time. Based on the test results, the user's weaknesses and areas for improvement are identified, and the next learning session is personalized.

[1020] As a concrete example, let's consider a case where a user is aiming for the "Bookkeeping Level 3" qualification. The user accesses the system and creates a new account. Next, they create a custom character based on Pikachu. After creating the character, they select the "Bookkeeping Level 3" study materials and begin a study session. Pikachu explains basic accounting concepts, and when the user enters a question about a specific journal entry, the server sends prompts to the generative AI model, providing an immediate answer. After the study session, a mastery test is automatically conducted, and the next study content is personalized based on the test results.

[1021] An example of a prompt sentence is, "Please explain basic journal entries as preparation for the Bookkeeping Level 3 exam." This enables the system to provide efficient, individually optimized learning support to users 24 hours a day, 365 days a year.

[1022] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1023] Step 1: The user accesses the system, enters their name, email address, and password on the new registration page, and submits the information. The entered information is sent to the server.

[1024] Input: User registration information (name, email address, password)

[1025] Output: Registration information sent to the server

[1026] What happens: The user fills in the form and clicks the submit button.

[1027] Step 2: The server receives the entered information and performs validation checks.

[1028] Input: User registration information

[1029] Output: Validation result (success or error)

[1030] What happens: The server validates the information entered to ensure it is in the correct format.

[1031] Step 3: If validation is successful, the server saves the registration information in a MySQL database.

[1032] Input: Validated registration information

[1033] Output: Save results to database

[1034] Specific operation: The server inserts the registration information into the user table in the database.

[1035] Step 4: The server sends a confirmation email to the user's email address, which contains a URL link to verify the account.

[1036] Input: User's email address and confirmation URL

[1037] Output: Confirmation email sent

[1038] Specific operation: The server sends a confirmation email via the mail server.

[1039] Step 5: The user opens the confirmation email and clicks on the confirmation link, which causes the server to activate the user's account.

[1040] Input: User confirmation link click

[1041] Output: Account enabled

[1042] Specific behavior: The server detects the click on the confirmation link and updates the account status accordingly.

[1043] Step 6: The user proceeds to the character generation page to customize the character's appearance and behavior.

[1044] Input: User's character setting information

[1045] Output: Customized character information

[1046] Specific actions: The user selects the character's appearance and actions through the interface and submits the configuration information.

[1047] Step 7: The server receives the character setting information and stores it in a database.

[1048] Input: User's character setting information

[1049] Output: Saved character information

[1050] Specific operation: The server inserts the character setting information into the character table of the database.

[1051] Step 8: The user selects the type of qualification and chooses the corresponding study material.

[1052] Input: User credential selection information

[1053] Output: Selected teaching material data

[1054] Specific Actions: The user selects a category of qualifications and then selects specific learning materials within that category.

[1055] Step 9: The server loads the selected teaching material data into the generative AI model.

[1056] Input: Selected teaching material data

[1057] Output: Teaching material data loaded into the generative AI model

[1058] Specific operation: The server inputs the teaching material data into the learning engine of the generative AI model.

[1059] Step 10: The server generates a curriculum specific to the user and stores it in the database.

[1060] Input: Teaching material data from a generative AI model

[1061] Output: User-specific curriculum

[1062] Specific operation: The server automatically generates a curriculum based on the user's progress and goals.

[1063] Step 11: A screen for the learning session is displayed on the user terminal.

[1064] Input: User-specific curriculum data

[1065] Output: Learning session screen

[1066] What happens: A user starts a learning session on a device and the interface is displayed.

[1067] Step 12: The character explains the content of the teaching material with voice and movement.

[1068] Input: Curriculum data and character setting information

[1069] Output: Audio explanation of teaching materials

[1070] Specific actions: The character explains to the user through movement and voice based on the content of the teaching material.

[1071] Step 13: The server monitors the user's progress in real time and stores the information in a database.

[1072] Input: User's learning progress data

[1073] Output: Saved learning progress data

[1074] Specific operation: The server records in real time how much progress the user has made in their studies.

[1075] Step 14: The user enters and submits a question during the study.

[1076] Input: User question text

[1077] Output: The question sent to the server

[1078] Specific actions: The user enters a question into the interface and clicks the submit button.

[1079] Step 15: The server inputs a question into the generative AI model, which generates an answer.

[1080] Input: User question text

[1081] Output: Answer from the generative AI model

[1082] Specific operation: The server inputs the question as a prompt into the AI ​​model, which generates an answer.

[1083] Step 16: The generated answer is spoken to the user through the character.

[1084] Input: Answer from a generative AI model

[1085] Output: Audio explanation of answers

[1086] Specific operation: The character vocalizes the answer from the generated AI model and conveys it to the user.

[1087] Step 17: When the user finishes the learning session, an automatic mastery test is administered.

[1088] Input: User's learning session end signal

[1089] Output: Start of mastery test

[1090] Specific operation: The server detects the end of the learning session and displays a test screen.

[1091] Step 18: The server analyzes the test results in real time and records the user's score and correct answer rate.

[1092] Input: User test answer data

[1093] Output: Parsed test results

[1094] Specific operation: The server analyzes the user's answers and calculates the score and percentage of correct answers.

[1095] Step 19: The server personalizes the next learning content based on the analysis results.

[1096] Input: Parsed test results

[1097] Output: Personalized upcoming curriculum

[1098] Specific operation: The server individually optimizes the next learning content based on the user's weaknesses and areas for improvement.

[1099] The above is the specific flow of the program processing of this system.

[1100] (Application example 1)

[1101] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1102] Traditional online learning systems offer limited interactive learning support and lack personalized feedback. They also lack real-time progress tracking and a virtual learning experience, which can reduce learning effectiveness. Furthermore, it is difficult for users to get immediate answers to their questions, which reduces learning efficiency.

[1103] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1104] In this invention, the server includes means for pre-learning learning material data using a generative AI model specialized for qualifications, means for selecting or generating a character preferred by the user, means for teaching qualification-related lessons using the selected or generated character, means for tracking and recording learning progress in real time, means for generating answers to the user's questions using the generative AI model and communicating them through the character, means for automatically conducting a mastery test after completion of learning and analyzing the results, means for personalizing the next learning content based on the analysis results, means for accessing as an avatar in a virtual space and providing interactive learning support, and means for the character to explain the learning material in a learning area in the virtual space. This allows users to receive personalized learning support, enabling effective and efficient learning.

[1105] A "qualification-specific generative AI model" is an artificial intelligence model designed to learn teaching material data corresponding to a specific qualification.

[1106] "Means for selecting or generating a character" refers to a mechanism such as an interface that allows a user to select a character of their choice or generate a new one.

[1107] "Means for teaching qualification-related lessons" is a function for teaching knowledge related to qualifications using selected or generated characters.

[1108] The "means for tracking and recording learning progress in real time" is a mechanism for monitoring the progress of a user during learning in real time and recording that data.

[1109] "A means of generating answers to questions using a generative AI model and communicating them through a character" is a function in which, when a user inputs a question, the generative AI model instantly generates an answer, and communicates that answer via voice or text through a character.

[1110] "Means for automatically conducting a mastery test after completion of learning and analyzing the results" is a function that automatically conducts a test after the learning session ends and analyzes the results.

[1111] "Means for personalizing the next learning content" refers to a system that optimizes and sets the next learning content for each individual user based on the test results.

[1112] "Means for providing interactive learning support through access as an avatar in a virtual space" is a system that allows users to log in to a virtual space, participate as an avatar, and receive interactive learning support.

[1113] "Means for a character to explain learning materials in a learning area within a virtual space" is a function that sets up a specific learning area within a virtual space and allows a character to explain learning materials there.

[1114] The present invention is a system that provides effective learning support for users aiming to obtain qualifications, and is characterized by learning support using generative AI models and characters. A specific method for implementing the present invention will be described in detail below.

[1115] First, when a user accesses the system and enters the required information such as name, email address, and password on the new registration page, the server receives the information and stores it in a database. Next, the server sends a confirmation email to the user's email address, and the user confirms the account by clicking the link in the confirmation email. Once confirmation is complete, the user proceeds to a page where they can select or create their favorite character. On this page, an interface is displayed where the user can customize the character's appearance and behavior.

[1116] Once the user has set up their character, the information is sent to the server and saved. Next, the user selects the type of qualification within the system and chooses the corresponding learning materials. This selection is sent to the server, which loads the selected learning materials data into a generative AI model and generates a curriculum specifically for the user.

[1117] The user's device displays a screen for the learning session, simultaneously displaying the generated character and the learning content. The character explains the content of the learning material with voice and movement. This allows the user to access the virtual space as an avatar and receive interactive learning support. If the user has any questions, they can enter them and the questions are sent to the server. The server uses a generative AI model to instantly generate an answer and communicates it to the user through the character.

[1118] After the learning session, a mastery test is automatically conducted. This test is administered by the server, and the test results are analyzed. Based on the analysis results, the server extracts the user's weaknesses and areas for improvement, and personalizes the next learning content based on these. This allows the user to receive efficient, individually optimized learning support.

[1119] Specific hardware used includes smartphones, smart glasses, or head-mounted displays (e.g., Oculus Rift). AWS and Google Cloud can be used as server platforms. Advanced artificial intelligence models such as GPT-4 are used as generative AI models. Character animation is performed using software such as Unity and Unreal Engine.

[1120] As a concrete example, consider the case where a user registers with the system aiming for the "Bookkeeping Level 3" qualification. First, the user accesses the system and creates a new account. Next, the user generates a character, selects the "Bookkeeping Level 3" study materials, and starts a study session. The character explains basic accounting concepts, and whenever the user asks a question, the generative AI model provides an immediate answer. For example, in response to a question such as "Please tell me the basics of journal entries," the character provides a detailed explanation. After the study session ends, a mastery test is conducted, and the next study content is personalized based on the test results.

[1121] An example of a prompt sentence for a generative AI model is, "The user is asking about the depreciation method for fixed assets. Please explain, including specific examples (for example, how to depreciate a machine worth 1 million yen over 10 years)."

[1122] In this way, the system of the present invention can provide efficient and individually optimized learning support to users 24 hours a day, 365 days a year.

[1123] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1124] Step 1:

[1125] User Registration

[1126] Input: The user enters their name, email address, and password.

[1127] Processing: The server receives this information and stores it in a database.

[1128] Output: A confirmation email is sent to the user.

[1129] How it works: The user clicks on the link in the confirmation email to confirm the account. The server activates the account.

[1130] Step 2:

[1131] Character Generation

[1132] Input: The user customizes the character's appearance and behavior.

[1133] Processing: The server receives the character setting information and stores it in the database.

[1134] Output: Customized character information.

[1135] Behavior: The user configures the character's appearance and behavior on the character customization screen and sends the settings to the server.

[1136] Step 3:

[1137] Selection of teaching materials

[1138] Input: User selects the qualification type and corresponding study material.

[1139] Processing: The server loads the selected learning materials into a generative AI model to generate a personalized curriculum.

[1140] Output: A personalized learning curriculum.

[1141] How it works: A user selects a certification type within the system and then selects study materials to begin learning.

[1142] Step 4:

[1143] Start a study session

[1144] Input: A request to start a learning session from the user's device.

[1145] Processing: The server sends the character and educational material data to the user's terminal.

[1146] Output: Screen display for the learning session.

[1147] How it works: A character and learning materials are displayed on the user's device, and the learning session begins. The character explains the content.

[1148] Step 5:

[1149] Question and Answering

[1150] Input: The question asked by the user.

[1151] Processing: The server sends the question to the generative AI model, which generates an answer.

[1152] Output: Display the answer on the user's terminal.

[1153] How it works: Users enter a question and the answer is instantly provided by a generative AI model. A character will then relay the answer.

[1154] Step 6:

[1155] Progress Tracking

[1156] Input: User's learning progress data.

[1157] Processing: The server tracks the progress in real time and stores it in a database.

[1158] Output: Updated progress data.

[1159] How it works: As you learn, your progress is recorded in real time.

[1160] Step 7:

[1161] Completion of study and mastery test

[1162] Input: Training session end signal.

[1163] Processing: The server automatically conducts the mastery test and analyzes the results.

[1164] Output: Test results and analysis report.

[1165] How it works: After the user has completed the learning, an automatic learning test is administered, and the results are sent to the server for analysis.

[1166] Step 8:

[1167] Personalize your next lesson

[1168] Input: Test results and analytical data.

[1169] Processing: The server customizes the next learning content based on the results.

[1170] Output: A personalized next-day learning curriculum.

[1171] How it works: Learning content is set based on the user's test results and is reflected in the next learning session.

[1172] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1173] The present invention is a system that provides effective learning support for users aiming to obtain qualifications, and is characterized by a generative AI model and character learning support, as well as an emotion engine that recognizes the user's emotions. The following describes in detail the embodiments of the present invention.

[1174] 1. User registration and character creation

[1175] A user accesses the system, enters the required information such as name, email address, and password on the new registration page, and submits it. The server receives the entered information and stores it in a database. The server then sends a confirmation email to the user's email address. When the user clicks on the link in the confirmation email, the account is confirmed. Once confirmation is complete, the user proceeds to a page where they can select or create their favorite character. On this page, an interface is displayed to customize the character's appearance and behavior. Once the user has finished configuring their character, the information is sent to the server and stored.

[1176] 2. Select study materials and start studying

[1177] Users select the type of qualification within the system and choose the corresponding study materials. This selection is sent to the server, which loads the selected study material data into the generative AI model. Based on the loaded data, the server generates a curriculum specifically for the user, and the user is ready to study.

[1178] 3. Conducting a learning session

[1179] The user's device displays a screen for the learning session, simultaneously displaying the generated character and the learning content. The character explains the content of the learning material with voice and actions. An emotion engine then recognizes the user's emotions from their facial expressions and vocalizations and provides real-time feedback. For example, if the emotion engine determines that the user is confused, the character will provide additional explanations and encouragement according to that emotion. The server monitors the user's progress and emotional changes in real time and stores this information in a database. If the user has any questions, they can enter them and the questions are sent to the server. The server then uses a generative AI model to instantly generate answers and communicate them to the user through the character.

[1180] 4. Measurement and feedback

[1181] When a user finishes a learning session, they are automatically tested for their level of mastery. This test is administered by the server, and the test results are analyzed. Based on the analysis results, the server identifies the user's weaknesses and areas for improvement, and personalizes the next learning content based on these. In addition, emotional data recognized by the emotion engine is also added to the analysis, enabling more accurate feedback and curriculum adjustments. This allows users to continue learning efficiently and in line with their individual needs and emotions.

[1182] Specific examples

[1183] For example, a user registers in the system aiming to obtain the "Bookkeeping Level 3" qualification. First, they access the system and create a new account. Next, they create a custom character based on Pikachu. After creating the character, they select the "Bookkeeping Level 3" study materials and begin a study session. Pikachu explains basic accounting concepts, and whenever the user asks a question, the generative AI model provides an instant answer. If the emotion engine detects a confused expression on the user's face during the study session, Pikachu will provide additional explanations or encouragement. After the study session ends, a mastery test is automatically conducted, and the next study content is personalized based on the test results.

[1184] In this way, by combining a generative AI model and an emotion engine, the system of the present invention is able to provide efficient and individually optimized learning support to users 24 hours a day, 365 days a year.

[1185] The processing flow will be explained below.

[1186] MODE FOR CARRYING OUT THE INVENTION (INCLUDING EMOTION ENGINE)

[1187] Step 1:

[1188] A user accesses the system and opens the new registration page. The terminal displays the registration form.

[1189] Step 2:

[1190] The user enters the required information (name, email address, password, etc.) and presses the send button. The device sends the input information to the server.

[1191] Step 3:

[1192] The server receives the input data and stores it in a database, then sends a confirmation email to the user's email address.

[1193] Step 4:

[1194] The user confirms their account by clicking the link in the confirmation email, and the device clicks the link to communicate with the server.

[1195] Step 5:

[1196] Once the account is verified, the user is directed to a page where they can select or create their desired character, and the device will display a character generation tool.

[1197] Step 6:

[1198] The user customizes the character's appearance and movements, then presses a button to save the settings. The device then sends the settings information to the server.

[1199] Step 7:

[1200] The server receives the character setting information and stores it in a database.

[1201] Step 8:

[1202] The user selects the type of qualification and the corresponding learning material, and the terminal displays the qualification list.

[1203] Step 9:

[1204] The user selects the qualifications they wish to study and chooses the study materials, and the selections are sent to the server.

[1205] Step 10:

[1206] The server loads the necessary data into the generative AI model based on the selected teaching materials and generates a curriculum.

[1207] Step 11:

[1208] The learning session screen is displayed on the user's device. The device provides an interface that simultaneously displays the character and the learning material content.

[1209] Step 12:

[1210] The character explains the contents of the learning material using voice and actions, and the emotion engine analyzes the user's facial expressions and vocalizations in real time to recognize the user's emotional state.

[1211] Step 13:

[1212] The emotion engine sends feedback to the server based on the user's emotional state, for example suggesting additional explanation or encouragement if the user is confused.

[1213] Step 14:

[1214] The server receives feedback from the emotion engine and instructs the character to respond appropriately. The character then conveys feedback to the user according to the emotion.

[1215] Step 15:

[1216] The server tracks the user's progress and emotional changes in real time and records them in a database.

[1217] Step 16:

[1218] The user enters a question or concern, and the device sends the question to the server.

[1219] Step 17:

[1220] The server uses a generative AI model to instantly generate answers and communicate them to the user through a character.

[1221] Step 18:

[1222] The user presses a button to end the learning session, and the device performs the termination process.

[1223] Step 19:

[1224] The server automatically conducts a learning level test, and the test screen is displayed on the user's device.

[1225] Step 20:

[1226] The user takes the test. The server analyzes the test results.

[1227] Step 21:

[1228] Based on the analysis results, the server identifies the user's weaknesses and areas for improvement, and personalizes the next lesson. The emotion engine also analyzes the emotional data it recognizes, improving the quality of feedback and the curriculum.

[1229] Step 22:

[1230] The server sends personalized feedback and the next curriculum to the user's device, which displays it.

[1231] In this way, by combining a generative AI model and an emotion engine, this system is able to provide efficient and individually optimized learning support to users 24 hours a day, 365 days a year.

[1232] Example 2

[1233] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1234] Conventional learning support systems have had difficulty providing personalized learning tailored to the individual needs and progress of users. Furthermore, they lacked the ability to recognize and respond to users' emotions and confusion in real time, making it difficult to maximize the effectiveness of learning. There was a need to provide a system that would solve these problems and enable users to study for qualifications efficiently and effectively.

[1235] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1236] In this invention, the server includes a means for pre-training learning material data using a generative AI model related to qualifications, a means for using an agent selected or generated by the user, and a means for tracking the user's learning progress in real time and recording it in a database. This enables personalized learning according to the user's individual needs and learning progress. The agent also deepens the user's understanding by conducting lessons involving simple movements. Furthermore, the emotion engine recognizes the user's facial expressions and vocalizations in real time and reflects that feedback to maximize learning effectiveness.

[1237] A "generative AI model for qualifications" is an artificial intelligence model that pre-learns educational material data related to specific qualifications and provides answers and educational materials in response to user questions and customizations.

[1238] An "agent" is a character or avatar that the user can select or create, and that character or avatar is responsible for explaining the learning content.

[1239] "Tracking" refers to the process of tracking a user's learning progress and behavior in real time and recording it in a database.

[1240] "Database" means a data storage system for managing user data, progress, learning outcomes, etc. collected within the system.

[1241] An "emotion engine" is an algorithm or software component that analyzes a user's facial expressions and tone of voice in real time and recognizes their emotions.

[1242] A "proficiency test" is a test that is automatically administered after a user completes a learning session to assess the user's learning progress.

[1243] "Personalization" refers to the process of individually optimizing the next learning content and curriculum based on each user's learning progress and weaknesses.

[1244] This invention is a system that provides effective learning support to users aiming to obtain qualifications, and in addition to learning support using generative AI models and agents, it features an emotion engine that recognizes the user's emotions.

[1245] User registration and agent creation

[1246] A user accesses the system, enters the required information such as name, email address, and password on the new registration page, and submits it. The server receives the entered information and stores it in a database. The server then sends a confirmation email to the user's email address. When the user clicks on the link in the confirmation email, the account is confirmed.

[1247] After verifying their account, users are taken to a page where they can select or create their preferred agent. This page provides an interface for customizing the agent's appearance and behavior. Once the user has configured their agent, the information is sent to the server and stored in a database.

[1248] Select study materials and start learning

[1249] Users select the type of qualification within the system and choose the corresponding study materials. The selection is sent to the server, which loads the selected study material data into a generative AI model. For example, GPT-3 or BERT can be used as the generative AI model. Based on the loaded data, the server generates a curriculum specifically for the user, and the user is ready to begin learning.

[1250] Running a learning session

[1251] The user's device displays a screen for the learning session, simultaneously displaying the generated agent and the learning content. The agent explains the content of the learning material with voice and action. The emotion engine recognizes the user's emotions from their facial expressions and vocalizations and provides feedback in real time. For example, if the emotion engine determines that the user is confused, the agent will provide additional explanations and encouragement according to that emotion. The server monitors the user's progress and emotional changes in real time and stores this information in a database. If the user has any questions, they can enter them and the questions are sent to the server. The server uses a generative AI model to instantly generate answers and communicates them to the user via the agent.

[1252] Measurement and feedback

[1253] When a user finishes a learning session, they are automatically tested for their level of mastery. This test is administered by the server, and the test results are analyzed. Based on the analysis results, the server identifies the user's weaknesses and areas for improvement, and personalizes the next learning content based on these. In addition, emotional data recognized by the emotion engine is also added to the analysis, enabling more accurate feedback and curriculum adjustments. This allows users to continue learning efficiently and in line with their individual needs and emotions.

[1254] Specific examples

[1255] For example, a user registers with the system aiming for the "Bookkeeping Level 3" certification. First, they access the system and create a new account, then generate a custom agent based on Pikachu. After generating the agent, they select the "Bookkeeping Level 3" study materials and begin a learning session. The agent explains basic accounting concepts, and whenever the user asks a question, the generated AI model provides an immediate answer. If the emotion engine detects a confused expression on the user's face during the learning session, the agent will provide additional explanations or encouragement. After the learning session ends, a mastery test is automatically conducted, and the next learning content is personalized based on the test results.

[1256] Prompt Sentence Examples

[1257] "Please explain a learning support system using custom agents with emotion recognition."

[1258] By combining a generative AI model and an emotion engine, the system of this invention can provide efficient and individually optimized learning support to users 24 hours a day, 365 days a year.

[1259] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1260] Step 1:

[1261] A user accesses the system, enters the required information such as name, email address, and password on the new registration page, and submits it. The input includes the user's personal information (name, email address, password). The output is the user information sent to the server.

[1262] Step 2:

[1263] The server receives the entered user information and stores it in a database.

[1264] Input: User's personal information (name, email address, password)

[1265] Data processing: Convert received information into the appropriate format and store it in a database

[1266] Output: Save confirmation message

[1267] Step 3:

[1268] The server sends a confirmation email to the user's email address, and when the user clicks on the link in the email, the account is authenticated.

[1269] Input: User's email address

[1270] Data processing: Generating and sending confirmation emails

[1271] Output: Account verification link

[1272] Step 4:

[1273] After completing the verification, the user proceeds to the agent creation page and uses the interface to customize the agent's appearance and behavior.

[1274] Input: User's agent selection and customization information

[1275] Data processing: Importing and verifying selected information

[1276] Output: Agent customization data

[1277] Step 5:

[1278] The server receives the customized agent information and stores it in a database.

[1279] Input: Agent customization information

[1280] Data processing: Format conversion and storage of received information

[1281] Output: Agent creation confirmation message

[1282] Step 6:

[1283] Users select the type of qualification within the system and choose the corresponding study materials.

[1284] Input:Select qualification type

[1285] Data processing: Search for teaching material data corresponding to the selected qualification

[1286] Output: List of teaching material data

[1287] Step 7:

[1288] The server loads the selected teaching material data into a generative AI model and generates a curriculum tailored to the user.

[1289] Input: Teaching material data

[1290] Data processing: Loading teaching material data and generating curriculum

[1291] Output: User-specific curriculum

[1292] Step 8:

[1293] The learning session screen is displayed on the user's device. The generated agent and the learning content are displayed simultaneously, and the agent explains the learning content with voice and actions.

[1294] Input: User-specific curriculum and agent data

[1295] Data processing: Synchronization of screen display with voice and action

[1296] Output: Learning screen

[1297] Step 9:

[1298] The emotion engine recognizes the user's facial expressions and vocalizations in real time and provides feedback. For example, if it detects a confused expression, the agent will provide additional explanation or encouragement.

[1299] Input: User's facial expression data and voice data

[1300] Data processing: Sentiment analysis and feedback content generation

[1301] Output: Feedback content

[1302] Step 10:

[1303] If a user has a question while studying, they can enter it and send it to the server.

[1304] Input: User question

[1305] Data processing: Question analysis and answer generation using an AI model

[1306] Output: Response data

[1307] Step 11:

[1308] The server uses a generative AI model to generate answers and communicates them to the user through an agent.

[1309] Input: Question data

[1310] Data processing: Answer generation using generative AI models

[1311] Output: Response via agent

[1312] Step 12:

[1313] Once the user has finished the learning session, the mastery test will begin automatically.

[1314] Input: End of study session trigger

[1315] Data processing: Implementing learning level tests and compiling the results

[1316] Output: Test results

[1317] Step 13:

[1318] The server analyzes the test results and extracts the user's strengths and weaknesses.

[1319] Input: Test result data

[1320] Data processing: Analyzing test results and identifying weaknesses

[1321] Output: Analysis results

[1322] Step 14:

[1323] The server personalizes the next learning content based on the analysis results and provides feedback taking into account the emotional data recognized by the emotion engine.

[1324] Input: Analysis results and sentiment data

[1325] Data processing: Customizing the next learning content

[1326] Output: Personalized learning and feedback

[1327] (Application example 2)

[1328] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1329] Conventional learning support systems have difficulty providing effective learning support based on the user's individual emotions and progress, and are insufficiently personalized to meet individual needs. Furthermore, when dealing with customers in brick-and-mortar stores, it is difficult to provide efficient explanations and customer service that reflect the customer's emotions and needs in real time. There is a need for a system that can solve these issues and provide support and services that are optimized for each user and customer.

[1330] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for pre-learning learning material data using a generative AI model specialized for qualifications; means for selecting or generating a character preferred by the user; means for conducting qualification-related lessons using the selected or generated character; means for tracking and recording learning progress in real time; means for generating answers to the user's questions using the generative AI model and communicating them through the character; means for automatically conducting a mastery test after completion of learning and analyzing the results; means for personalizing the next learning content based on the analysis results; means for analyzing the user's emotions using an emotion recognition engine and adjusting the character's movements and speech based on the data; and means for providing personalized product explanations and customer service to customers in stores as part of personalization. This enables effective learning support that reflects the user's emotions and progress in real time, making it possible to provide explanations and customer service optimized for each customer even in physical stores.

[1331] A "generative AI model specialized for qualifications" is an artificial intelligence model that pre-learns educational material data related to obtaining specific qualifications and generates appropriate answers and explanations.

[1332] An "emotion recognition engine" is a system that analyzes the emotions of a user, such as facial expressions and vocalizations, in real time, and extracts and utilizes that emotional data.

[1333] A "character" is a virtual avatar or character selected or created by the user that acts as a facilitator of learning or guidance.

[1334] "Means for tracking and recording learning progress in real time" refers to a mechanism for monitoring a user's learning activities and progress in real time and recording that information.

[1335] "Generating answers using a generative AI model" means that artificial intelligence generates appropriate answers to questions from users based on information it has learned in advance.

[1336] "Means of communicating through a character" refers to a method of communicating the generated answer or explanation to the user through the voice or actions of a character selected by the user.

[1337] The "means for automatically conducting a mastery test after completion of learning" is a mechanism for automatically conducting a test to measure the user's level of understanding after the learning session is completed.

[1338] The "means for analyzing the results" is a method for analyzing the results of the mastery test in detail and understanding the user's strengths and weaknesses.

[1339] "Means for personalizing the next learning content" is a mechanism that provides learning content optimized for the user's individual needs and progress based on the analysis results.

[1340] "Means for providing personalized product explanations and customer service" refers to a system for providing appropriate product explanations and customer service in physical stores based on the individual emotions and needs of customers.

[1341] The present invention relates to a system that provides learning support and customer service support in physical stores using a generative AI model and an emotion recognition engine. Specific examples for implementing the present invention are described in detail below.

[1342] 1. System Configuration

[1343] This system provides effective learning support for users who are aiming to serve customers in brick-and-mortar stores and obtain qualifications. The system is primarily composed of the following hardware and software:

[1344] Hardware: Smartphones, smart glasses

[1345] Software: Mobile app, server-side program (Python + Flask), database (MySQL), generative AI model (OpenAI's GPT-4), emotion recognition engine (Microsoft Azure Face API)

[1346] 2. Program Processing

[1347] Customer registration and character creation

[1348] The server creates an account based on the information the user entered and sends a confirmation email. When the user clicks the confirmation link, the account is activated and the user is then taken to the character creation page. The user customizes the character's appearance and behavior to their liking, and this information is sent to the server and stored in a database.

[1349] Product selection and introduction

[1350] Users select a product category through their smart device. The selection is sent to the server, which then loads the selected product data into the generative AI model, which then creates a personalized guidance curriculum for the customer based on the data.

[1351] Running a store navigation session

[1352] An information screen is displayed on the user's device, and a generated character explains the product. An emotion recognition engine analyzes the user's emotions from their facial expressions and vocalizations and provides feedback in real time. The server monitors the user's progress and emotions in real time and stores them in a database. When the user enters a question, the generative AI model instantly generates an answer and communicates it to the user through the character.

[1353] Measurement and feedback

[1354] After the guidance session ends, a satisfaction test is automatically conducted by the server. The test results are analyzed in detail by the server and feedback is provided to identify the user's strengths and weaknesses. Based on the analysis results, the next guidance content is personalized.

[1355] As a specific example, consider the case where a user visits a physical store and uses smart glasses to receive product information. At this time, a character explains the product based on product data previously learned by the generative AI model. If the emotion recognition engine detects a confused expression on the user's face, the character will provide additional explanation to help the user understand. In this way, effective customer service is possible, reflecting the user's emotions and progress in real time.

[1356] Example prompt sentence:

[1357] "Based on the product category selected by the customer, we use an emotion recognition engine to analyze the user's emotions and generate the most appropriate product description."

[1358] As described above, by combining a generative AI model and an emotion recognition engine, this system can provide individually optimized learning support and customer service to users and customers.

[1359] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1360] Step 1:

[1361] The user accesses the app from smart glasses or a smartphone, enters the required information such as name, email address, and password on the new registration page, and submits it. Input: Registration information entered by the user. Output: Registration information is sent to the server and saved in the database. Specifically, after the server receives the entered information, it saves it as a new record in the database.

[1362] Step 2:

[1363] The server sends a confirmation email to the user's email address. Input: The user's email address included in the registration information. Output: A confirmation email is sent to the user's email address. Specifically, the server generates and sends the confirmation email using the SMTP protocol.

[1364] Step 3:

[1365] When the user clicks the link in the confirmation email, the account is confirmed. Input: The confirmation link the user clicked. Output: The server activates the account. Specifically, the server updates the user's account record and sets the confirmation flag.

[1366] Step 4:

[1367] The user creates or selects a character of their choice. Input: Customization information for the appearance and behavior of the character selected by the user. Output: Character creation information is sent to the server and saved in a database. Specifically, the server receives the character information set by the user and saves it in a database.

[1368] Step 5:

[1369] The user selects a product category through the app. Input: The product category selected by the user. Output: The product category information is sent to the server and loaded into the generative AI model. Specifically, the server receives the product category information and passes it to the generative AI model.

[1370] Step 6:

[1371] The generative AI model generates a guidance curriculum based on product data. Input: Product category information. Output: Generated guidance curriculum. Specifically, the generative AI model analyzes the given product category information and generates an appropriate guidance curriculum.

[1372] Step 7:

[1373] An information screen is displayed on the user's device, and the generated character explains the product. Input: Generated information curriculum. Output: Product description displayed on the device. Specifically, the character explains the product using voice and actions based on the information curriculum obtained from the generation AI model.

[1374] Step 8:

[1375] The emotion recognition engine analyzes the user's facial expressions and vocalizations to recognize emotions. Input: User's facial expression and vocalization data. Output: Recognized emotion data. Specifically, the emotion recognition engine analyzes the user's facial images and voice data in real time to extract their emotional state.

[1376] Step 9:

[1377] The server monitors the user's progress and emotions in real time and records them in a database. Input: User's progress and emotion data. Output: Progress and emotion data stored in the database. Specifically, the server receives the user's learning progress and emotion data and stores it in the database.

[1378] Step 10:

[1379] The user inputs a question and sends it to the server. Input: User's question. Output: Answer generated by the generative AI model. Specifically, the server receives the question from the user and uses the generative AI model to generate an appropriate answer.

[1380] Step 11:

[1381] The generated answer is communicated to the user through the character. Input: The answer generated by the generative AI model. Output: The answer communicated to the user. The specific action is for the character to communicate the answer to the user through voice and action.

[1382] Step 12:

[1383] After the guidance session ends, a satisfaction test is automatically conducted. Input: Learning / guidance session end trigger. Output: Satisfaction test result. Specifically, the server detects the end of the session and automatically conducts a satisfaction test.

[1384] Step 13:

[1385] The server analyzes the test results and identifies the user's strengths and weaknesses. Input: Satisfaction test results. Output: Analysis results. Specifically, the server analyzes the test results in detail to extract the user's level of understanding and areas for improvement.

[1386] Step 14:

[1387] Based on the analysis results, the next announcement content is personalized. Input: Analysis results. Output: Personalized next announcement content. Specifically, the server generates announcement content optimized for the user's individual needs based on the analysis results.

[1388] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1389] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1390] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1391] [Fourth embodiment]

[1392] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1393] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1394] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1395] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1396] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1397] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1398] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1399] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1400] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1401] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1402] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1403] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1404] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1405] The present invention is a system that provides effective learning support for users aiming to obtain qualifications, and is characterized by learning support using generative AI models and characters. The following describes in detail the embodiments of the present invention.

[1406] 1. User registration and character creation

[1407] A user accesses the system, enters the required information such as name, email address, and password on the new registration page, and submits it. The server receives the entered information and stores it in a database. The server then sends a confirmation email to the user's email address. When the user clicks on the link in the confirmation email, the account is confirmed. Once confirmation is complete, the user proceeds to a page where they can select or create their favorite character. On this page, an interface is displayed to customize the character's appearance and behavior. Once the user has finished configuring their character, the information is sent to the server and stored.

[1408] 2. Select study materials and start studying

[1409] Users select the type of qualification within the system and choose the corresponding study materials. This selection is sent to the server, which loads the selected study material data into the generative AI model. Based on the loaded data, the server generates a curriculum specifically for the user, and the user is ready to study.

[1410] 3. Conducting a learning session

[1411] The user's device displays a screen for the learning session, simultaneously displaying the generated character and the learning content. The character explains the content of the learning material with voice and movement. The server monitors the user's progress in real time and stores this information in a database. If the user has any questions, they can enter them and the questions are sent to the server. The server uses a generative AI model to instantly generate answers and communicate them to the user through the character.

[1412] 4. Measurement and feedback

[1413] When a user finishes a learning session, they are automatically tested for their level of mastery. This test is administered by the server, and the test results are analyzed. Based on the analysis, the server identifies the user's weaknesses and areas for improvement, and personalizes the next learning session accordingly. This allows users to continue learning efficiently and in line with their individual needs.

[1414] Specific examples

[1415] For example, a user registers in the system aiming to obtain the "Bookkeeping Level 3" qualification. First, they access the system and create a new account. Next, they create a custom character based on Pikachu. After creating the character, they select the "Bookkeeping Level 3" study materials and begin a study session. Pikachu explains basic accounting concepts, and whenever the user asks a question, the generative AI model provides an instant answer. After the study session ends, a mastery test is automatically conducted, and the next study content is personalized based on the test results.

[1416] In this way, the system of the present invention can provide efficient and individually optimized learning support to users 24 hours a day, 365 days a year.

[1417] The processing flow will be explained below.

[1418] Step 1:

[1419] A user accesses the system and opens the new registration page. The terminal displays the registration form.

[1420] Step 2:

[1421] The user enters the required information (name, email address, password, etc.) and presses the send button. The device sends the input information to the server.

[1422] Step 3:

[1423] The server receives the input data and stores it in a database, then sends a confirmation email to the user's email address.

[1424] Step 4:

[1425] The user confirms the account by clicking the link in the confirmation email. The device clicks the link and communicates with the server.

[1426] Step 5:

[1427] Once the account is verified, the user is directed to a page where they can select or create their desired character, and the device will display a character generation tool.

[1428] Step 6:

[1429] The user customizes the character's appearance and movements, then presses a button to save the settings. The device then sends the settings information to the server.

[1430] Step 7:

[1431] The server receives the character setting information and stores it in a database.

[1432] Step 8:

[1433] The user selects the type of qualification and the corresponding learning material, and the terminal displays the qualification list.

[1434] Step 9:

[1435] The user selects the qualifications they wish to study and chooses the study materials, and the selections are sent to the server.

[1436] Step 10:

[1437] The server loads the necessary data into the generative AI model based on the selected teaching materials and generates a curriculum.

[1438] Step 11:

[1439] The learning session screen is displayed on the user's device. The device provides an interface that simultaneously displays the character and the learning material content.

[1440] Step 12:

[1441] The character explains the contents of the learning material using voice and actions, and the user can learn by watching this.

[1442] Step 13:

[1443] The server tracks the user's progress in real time and records it in a database.

[1444] Step 14:

[1445] The user enters a question or concern, and the device sends the question to the server.

[1446] Step 15:

[1447] The server uses a generative AI model to instantly generate an answer and send it to the device, where a character relays the answer to the user.

[1448] Step 16:

[1449] The user presses a button to end the learning session, and the device performs the termination process.

[1450] Step 17:

[1451] The server automatically conducts a learning level test, and the test screen is displayed on the user's device.

[1452] Step 18:

[1453] The user takes the test. The server analyzes the test results.

[1454] Step 19:

[1455] Based on the analysis results, the server extracts the user's weaknesses and areas for improvement and personalizes the next learning content.

[1456] Step 20:

[1457] The server sends personalized feedback and the next curriculum to the user's device, which displays it.

[1458] In this way, the system provides efficient and individually optimized learning support to users 24 hours a day, 365 days a year.

[1459] Example 1

[1460] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1461] In conventional qualification learning systems, it was difficult to monitor each user's learning progress individually and provide feedback in real time. Furthermore, there was a lack of a flexible system that could immediately respond to users' questions, which often led to a decline in learning efficiency. Furthermore, there were limited systems for personalizing users' learning content, making it difficult to provide learning curricula that met individual needs.

[1462] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1463] In this invention, the server includes means for generating an account based on information entered by the user and sending a confirmation email, means for selecting or generating a character preferred by the user, means for pre-learning learning material data using a generative AI model specialized for qualifications, means for conducting qualification-related lessons using the selected or generated character, means for generating a curriculum dedicated to the user, means for tracking and recording learning progress in real time, means for generating answers to user questions using a generative AI model and communicating them through the character, means for automatically conducting a mastery test after completion of learning and analyzing the results, and means for personalizing the next learning content based on the analysis results, thereby enabling efficient and individually optimized qualification learning.

[1464] "Means for creating an account and sending a confirmation email" refers to a method in which a user is prompted to enter the necessary information when registering with the system, a user account is created based on that information, and an email is automatically sent to confirm the registration.

[1465] The "means for selecting or generating a character" is a method for allowing a user to select or generate a virtual character to be used when studying, according to the user's preferences.

[1466] "Means of using a generative AI model to learn teaching material data in advance" refers to a method in which teaching material data related to a specific qualification is input into the AI ​​model in advance, and the AI ​​model learns based on that data.

[1467] The "means for conducting qualification-related lessons" is a method for using the generated character to conduct lessons to provide users with the knowledge necessary to obtain qualifications.

[1468] The "means for generating a curriculum specifically for a user" is a method for automatically creating an individually optimized learning plan according to the user's qualification acquisition goals and progress.

[1469] The "means for tracking and recording learning progress" is a method for monitoring a user's learning activities in real time and recording the progress in a database or the like.

[1470] "Means of generating answers using a generative AI model and communicating them through a character" refers to a method in which, when a user inputs a question, a generative AI model generates an answer to that question and communicates it to the user through a virtual character.

[1471] The "means for conducting a mastery test and analyzing the results" is a method for automatically conducting a test after the completion of a learning session and analyzing the results.

[1472] "Means for personalizing the next learning content" refers to a method for individually optimizing the next learning content based on the results of the mastery test and the user's progress.

[1473] The present invention is a system that provides effective learning support to users aiming to obtain qualifications, characterized by learning support using a generative AI model and a character. Specifically, the system includes means for generating an account based on information entered by the user and sending a confirmation email, means for selecting or generating a character preferred by the user, means for pre-learning learning material data using a generative AI model specialized for qualifications, means for conducting qualification-related lessons using the selected or generated character, means for generating a curriculum dedicated to the user, means for tracking and recording learning progress in real time, means for generating answers to user questions using a generative AI model and communicating them through a character, means for automatically conducting a mastery test after completion of learning and analyzing the results, and means for personalizing the next learning content based on the analysis results.

[1474] Users access this system using a web browser as an interface. When a user enters their name, email address, and password on the new registration page and submits it, the server receives the information and stores it in a MySQL database. The server then uses a mail server to send a confirmation email. This email contains a URL link for account confirmation, and the user completes account confirmation by clicking the link. Once confirmation is complete, the user can proceed to the character creation page, where an interface is provided for customizing the character's appearance and behavior. Once the user has finished configuring their character, the information is sent back to the server and stored in the database.

[1475] The user selects the type of qualification within the system and chooses the corresponding learning materials. The server loads the selected learning material data (PDF, video, audio files, etc.) into a generative AI model (e.g., OpenAI GPT-4) and generates a curriculum specifically for the user. The user is then ready to learn. The learning session screen is displayed on the user's device, displaying the generated character and the learning content simultaneously. The character explains the content of the learning materials with voice and movement. Voice synthesis software may also be used here.

[1476] The server monitors the user's progress in real time and stores that information in a database. When the user enters a question into a text box and submits it, the server inputs the question into a generative AI model to generate an answer. The answer is then communicated to the user via voice via a character.

[1477] Once a user has completed a learning session, they are automatically tested for their progress. The test is administered by the server and the results are analyzed in real time. Based on the test results, the user's weaknesses and areas for improvement are identified, and the next learning session is personalized.

[1478] As a concrete example, let's consider a case where a user is aiming for the "Bookkeeping Level 3" qualification. The user accesses the system and creates a new account. Next, they create a custom character based on Pikachu. After creating the character, they select the "Bookkeeping Level 3" study materials and begin a study session. Pikachu explains basic accounting concepts, and when the user enters a question about a specific journal entry, the server sends prompts to the generative AI model, providing an immediate answer. After the study session, a mastery test is automatically conducted, and the next study content is personalized based on the test results.

[1479] An example of a prompt sentence is, "Please explain basic journal entries as preparation for the Bookkeeping Level 3 exam." This enables the system to provide efficient, individually optimized learning support to users 24 hours a day, 365 days a year.

[1480] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1481] Step 1: The user accesses the system, enters their name, email address, and password on the new registration page, and submits the information. The entered information is sent to the server.

[1482] Input: User registration information (name, email address, password)

[1483] Output: Registration information sent to the server

[1484] What happens: The user fills in the form and clicks the submit button.

[1485] Step 2: The server receives the entered information and performs validation checks.

[1486] Input: User registration information

[1487] Output: Validation result (success or error)

[1488] What happens: The server validates the information entered to ensure it is in the correct format.

[1489] Step 3: If validation is successful, the server saves the registration information in a MySQL database.

[1490] Input: Validated registration information

[1491] Output: Save results to database

[1492] Specific operation: The server inserts the registration information into the user table in the database.

[1493] Step 4: The server sends a confirmation email to the user's email address, which contains a URL link to verify the account.

[1494] Input: User's email address and confirmation URL

[1495] Output: Confirmation email sent

[1496] Specific operation: The server sends a confirmation email via the mail server.

[1497] Step 5: The user opens the confirmation email and clicks on the confirmation link, which causes the server to activate the user's account.

[1498] Input: User confirmation link click

[1499] Output: Account enabled

[1500] Specific behavior: The server detects the click on the confirmation link and updates the account status accordingly.

[1501] Step 6: The user proceeds to the character generation page to customize the character's appearance and behavior.

[1502] Input: User's character setting information

[1503] Output: Customized character information

[1504] Specific actions: The user selects the character's appearance and actions through the interface and submits the configuration information.

[1505] Step 7: The server receives the character setting information and stores it in a database.

[1506] Input: User's character setting information

[1507] Output: Saved character information

[1508] Specific operation: The server inserts the character setting information into the character table of the database.

[1509] Step 8: The user selects the type of qualification and chooses the corresponding study material.

[1510] Input: User credential selection information

[1511] Output: Selected teaching material data

[1512] Specific Actions: The user selects a category of qualifications and then selects specific learning materials within that category.

[1513] Step 9: The server loads the selected teaching material data into the generative AI model.

[1514] Input: Selected teaching material data

[1515] Output: Teaching material data loaded into the generative AI model

[1516] Specific operation: The server inputs the teaching material data into the learning engine of the generative AI model.

[1517] Step 10: The server generates a curriculum specific to the user and stores it in the database.

[1518] Input: Teaching material data from a generative AI model

[1519] Output: User-specific curriculum

[1520] Specific operation: The server automatically generates a curriculum based on the user's progress and goals.

[1521] Step 11: A screen for the learning session is displayed on the user terminal.

[1522] Input: User-specific curriculum data

[1523] Output: Learning session screen

[1524] What happens: A user starts a learning session on a device and the interface is displayed.

[1525] Step 12: The character explains the content of the teaching material with voice and movement.

[1526] Input: Curriculum data and character setting information

[1527] Output: Audio explanation of teaching materials

[1528] Specific actions: The character explains to the user through movement and voice based on the content of the teaching material.

[1529] Step 13: The server monitors the user's progress in real time and stores the information in a database.

[1530] Input: User's learning progress data

[1531] Output: Saved learning progress data

[1532] Specific operation: The server records in real time how much progress the user has made in their studies.

[1533] Step 14: The user enters and submits a question during the study.

[1534] Input: User question text

[1535] Output: The question sent to the server

[1536] Specific actions: The user enters a question into the interface and clicks the submit button.

[1537] Step 15: The server inputs a question into the generative AI model, which generates an answer.

[1538] Input: User question text

[1539] Output: Answer from the generative AI model

[1540] Specific operation: The server inputs the question as a prompt into the AI ​​model, which generates an answer.

[1541] Step 16: The generated answer is spoken to the user through the character.

[1542] Input: Answer from a generative AI model

[1543] Output: Audio explanation of answers

[1544] Specific operation: The character vocalizes the answer from the generated AI model and conveys it to the user.

[1545] Step 17: When the user finishes the learning session, an automatic mastery test is administered.

[1546] Input: User's learning session end signal

[1547] Output: Start of mastery test

[1548] Specific operation: The server detects the end of the learning session and displays a test screen.

[1549] Step 18: The server analyzes the test results in real time and records the user's score and correct answer rate.

[1550] Input: User test answer data

[1551] Output: Parsed test results

[1552] Specific operation: The server analyzes the user's answers and calculates the score and percentage of correct answers.

[1553] Step 19: The server personalizes the next learning content based on the analysis results.

[1554] Input: Parsed test results

[1555] Output: Personalized upcoming curriculum

[1556] Specific operation: The server individually optimizes the next learning content based on the user's weaknesses and areas for improvement.

[1557] The above is the specific flow of the program processing of this system.

[1558] (Application example 1)

[1559] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1560] Traditional online learning systems offer limited interactive learning support and lack personalized feedback. They also lack real-time progress tracking and a virtual learning experience, which can reduce learning effectiveness. Furthermore, it is difficult for users to get immediate answers to their questions, which reduces learning efficiency.

[1561] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1562] In this invention, the server includes means for pre-learning learning material data using a generative AI model specialized for qualifications, means for selecting or generating a character preferred by the user, means for teaching qualification-related lessons using the selected or generated character, means for tracking and recording learning progress in real time, means for generating answers to the user's questions using the generative AI model and communicating them through the character, means for automatically conducting a mastery test after completion of learning and analyzing the results, means for personalizing the next learning content based on the analysis results, means for accessing as an avatar in a virtual space and providing interactive learning support, and means for the character to explain the learning material in a learning area in the virtual space. This allows users to receive personalized learning support, enabling effective and efficient learning.

[1563] A "qualification-specific generative AI model" is an artificial intelligence model designed to learn teaching material data corresponding to a specific qualification.

[1564] "Means for selecting or generating a character" refers to a mechanism such as an interface that allows a user to select a character of their choice or generate a new one.

[1565] "Means for teaching qualification-related lessons" is a function for teaching knowledge related to qualifications using selected or generated characters.

[1566] The "means for tracking and recording learning progress in real time" is a mechanism for monitoring the progress of a user during learning in real time and recording that data.

[1567] "A means of generating answers to questions using a generative AI model and communicating them through a character" is a function in which, when a user inputs a question, the generative AI model instantly generates an answer, and communicates that answer via voice or text through a character.

[1568] "Means for automatically conducting a mastery test after completion of learning and analyzing the results" is a function that automatically conducts a test after the learning session ends and analyzes the results.

[1569] "Means for personalizing the next learning content" refers to a system that optimizes and sets the next learning content for each individual user based on the test results.

[1570] "Means for providing interactive learning support through access as an avatar in a virtual space" is a system that allows users to log in to a virtual space, participate as an avatar, and receive interactive learning support.

[1571] "Means for a character to explain learning materials in a learning area within a virtual space" is a function that sets up a specific learning area within a virtual space and allows a character to explain learning materials there.

[1572] The present invention is a system that provides effective learning support for users aiming to obtain qualifications, and is characterized by learning support using generative AI models and characters. A specific method for implementing the present invention will be described in detail below.

[1573] First, when a user accesses the system and enters the required information such as name, email address, and password on the new registration page, the server receives the information and stores it in a database. Next, the server sends a confirmation email to the user's email address, and the user confirms the account by clicking the link in the confirmation email. Once confirmation is complete, the user proceeds to a page where they can select or create their favorite character. On this page, an interface is displayed where the user can customize the character's appearance and behavior.

[1574] Once the user has set up their character, the information is sent to the server and saved. Next, the user selects the type of qualification within the system and chooses the corresponding learning materials. This selection is sent to the server, which loads the selected learning materials data into a generative AI model and generates a curriculum specifically for the user.

[1575] The user's device displays a screen for the learning session, simultaneously displaying the generated character and the learning content. The character explains the content of the learning material with voice and movement. This allows the user to access the virtual space as an avatar and receive interactive learning support. If the user has any questions, they can enter them and the questions are sent to the server. The server uses a generative AI model to instantly generate an answer and communicates it to the user through the character.

[1576] After the learning session, a mastery test is automatically conducted. This test is administered by the server, and the test results are analyzed. Based on the analysis results, the server extracts the user's weaknesses and areas for improvement, and personalizes the next learning content based on these. This allows the user to receive efficient, individually optimized learning support.

[1577] Specific hardware used includes smartphones, smart glasses, or head-mounted displays (e.g., Oculus Rift). AWS and Google Cloud can be used as server platforms. Advanced artificial intelligence models such as GPT-4 are used as generative AI models. Character animation is performed using software such as Unity and Unreal Engine.

[1578] As a concrete example, consider the case where a user registers with the system aiming for the "Bookkeeping Level 3" qualification. First, the user accesses the system and creates a new account. Next, the user generates a character, selects the "Bookkeeping Level 3" study materials, and starts a study session. The character explains basic accounting concepts, and whenever the user asks a question, the generative AI model provides an immediate answer. For example, in response to a question such as "Please tell me the basics of journal entries," the character provides a detailed explanation. After the study session ends, a mastery test is conducted, and the next study content is personalized based on the test results.

[1579] An example of a prompt sentence for a generative AI model is, "The user is asking about the depreciation method for fixed assets. Please explain, including specific examples (for example, how to depreciate a machine worth 1 million yen over 10 years)."

[1580] In this way, the system of the present invention can provide efficient and individually optimized learning support to users 24 hours a day, 365 days a year.

[1581] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1582] Step 1:

[1583] User Registration

[1584] Input: The user enters their name, email address, and password.

[1585] Processing: The server receives this information and stores it in a database.

[1586] Output: A confirmation email is sent to the user.

[1587] How it works: The user clicks on the link in the confirmation email to confirm the account. The server activates the account.

[1588] Step 2:

[1589] Character Generation

[1590] Input: The user customizes the character's appearance and behavior.

[1591] Processing: The server receives the character setting information and stores it in the database.

[1592] Output: Customized character information.

[1593] Behavior: The user configures the character's appearance and behavior on the character customization screen and sends the settings to the server.

[1594] Step 3:

[1595] Selection of teaching materials

[1596] Input: User selects the qualification type and corresponding study material.

[1597] Processing: The server loads the selected learning materials into a generative AI model to generate a personalized curriculum.

[1598] Output: A personalized learning curriculum.

[1599] How it works: A user selects a certification type within the system and then selects study materials to begin learning.

[1600] Step 4:

[1601] Start a study session

[1602] Input: A request to start a learning session from the user's device.

[1603] Processing: The server sends the character and educational material data to the user's terminal.

[1604] Output: Screen display for the learning session.

[1605] How it works: A character and learning materials are displayed on the user's device, and the learning session begins. The character explains the content.

[1606] Step 5:

[1607] Question and Answering

[1608] Input: The question asked by the user.

[1609] Processing: The server sends the question to the generative AI model, which generates an answer.

[1610] Output: Display the answer on the user's terminal.

[1611] How it works: Users enter a question and the answer is instantly provided by a generative AI model. A character will then relay the answer.

[1612] Step 6:

[1613] Progress Tracking

[1614] Input: User's learning progress data.

[1615] Processing: The server tracks the progress in real time and stores it in a database.

[1616] Output: Updated progress data.

[1617] How it works: As you learn, your progress is recorded in real time.

[1618] Step 7:

[1619] Completion of study and mastery test

[1620] Input: Training session end signal.

[1621] Processing: The server automatically conducts the mastery test and analyzes the results.

[1622] Output: Test results and analysis report.

[1623] How it works: After the user has completed the learning, an automatic learning test is administered, and the results are sent to the server for analysis.

[1624] Step 8:

[1625] Personalize your next lesson

[1626] Input: Test results and analytical data.

[1627] Processing: The server customizes the next learning content based on the results.

[1628] Output: A personalized next-day learning curriculum.

[1629] How it works: Learning content is set based on the user's test results and is reflected in the next learning session.

[1630] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1631] The present invention is a system that provides effective learning support for users aiming to obtain qualifications, and is characterized by a generative AI model and character learning support, as well as an emotion engine that recognizes the user's emotions. The following describes in detail the embodiments of the present invention.

[1632] 1. User registration and character creation

[1633] A user accesses the system, enters the required information such as name, email address, and password on the new registration page, and submits it. The server receives the entered information and stores it in a database. The server then sends a confirmation email to the user's email address. When the user clicks on the link in the confirmation email, the account is confirmed. Once confirmation is complete, the user proceeds to a page where they can select or create their favorite character. On this page, an interface is displayed to customize the character's appearance and behavior. Once the user has finished configuring their character, the information is sent to the server and stored.

[1634] 2. Select study materials and start studying

[1635] Users select the type of qualification within the system and choose the corresponding study materials. This selection is sent to the server, which loads the selected study material data into the generative AI model. Based on the loaded data, the server generates a curriculum specifically for the user, and the user is ready to study.

[1636] 3. Conducting a learning session

[1637] The user's device displays a screen for the learning session, simultaneously displaying the generated character and the learning content. The character explains the content of the learning material with voice and actions. An emotion engine then recognizes the user's emotions from their facial expressions and vocalizations and provides real-time feedback. For example, if the emotion engine determines that the user is confused, the character will provide additional explanations and encouragement according to that emotion. The server monitors the user's progress and emotional changes in real time and stores this information in a database. If the user has any questions, they can enter them and the questions are sent to the server. The server then uses a generative AI model to instantly generate answers and communicate them to the user through the character.

[1638] 4. Measurement and feedback

[1639] When a user finishes a learning session, they are automatically tested for their level of mastery. This test is administered by the server, and the test results are analyzed. Based on the analysis results, the server identifies the user's weaknesses and areas for improvement, and personalizes the next learning content based on these. In addition, emotional data recognized by the emotion engine is also added to the analysis, enabling more accurate feedback and curriculum adjustments. This allows users to continue learning efficiently and in line with their individual needs and emotions.

[1640] Specific examples

[1641] For example, a user registers in the system aiming to obtain the "Bookkeeping Level 3" qualification. First, they access the system and create a new account. Next, they create a custom character based on Pikachu. After creating the character, they select the "Bookkeeping Level 3" study materials and begin a study session. Pikachu explains basic accounting concepts, and whenever the user asks a question, the generative AI model provides an instant answer. If the emotion engine detects a confused expression on the user's face during the study session, Pikachu will provide additional explanations or encouragement. After the study session ends, a mastery test is automatically conducted, and the next study content is personalized based on the test results.

[1642] In this way, by combining a generative AI model and an emotion engine, the system of the present invention is able to provide efficient and individually optimized learning support to users 24 hours a day, 365 days a year.

[1643] The processing flow will be explained below.

[1644] MODE FOR CARRYING OUT THE INVENTION (INCLUDING EMOTION ENGINE)

[1645] Step 1:

[1646] A user accesses the system and opens the new registration page. The terminal displays the registration form.

[1647] Step 2:

[1648] The user enters the required information (name, email address, password, etc.) and presses the send button. The device sends the input information to the server.

[1649] Step 3:

[1650] The server receives the input data and stores it in a database, then sends a confirmation email to the user's email address.

[1651] Step 4:

[1652] The user confirms their account by clicking the link in the confirmation email, and the device clicks the link to communicate with the server.

[1653] Step 5:

[1654] Once the account is verified, the user is directed to a page where they can select or create their desired character, and the device will display a character generation tool.

[1655] Step 6:

[1656] The user customizes the character's appearance and movements, then presses a button to save the settings. The device then sends the settings information to the server.

[1657] Step 7:

[1658] The server receives the character setting information and stores it in a database.

[1659] Step 8:

[1660] The user selects the type of qualification and the corresponding learning material, and the terminal displays the qualification list.

[1661] Step 9:

[1662] The user selects the qualifications they wish to study and chooses the study materials, and the selections are sent to the server.

[1663] Step 10:

[1664] The server loads the necessary data into the generative AI model based on the selected teaching materials and generates a curriculum.

[1665] Step 11:

[1666] The learning session screen is displayed on the user's device. The device provides an interface that simultaneously displays the character and the learning material content.

[1667] Step 12:

[1668] The character explains the contents of the learning material using voice and actions, and the emotion engine analyzes the user's facial expressions and vocalizations in real time to recognize the user's emotional state.

[1669] Step 13:

[1670] The emotion engine sends feedback to the server based on the user's emotional state, for example suggesting additional explanation or encouragement if the user is confused.

[1671] Step 14:

[1672] The server receives feedback from the emotion engine and instructs the character to respond appropriately. The character then conveys feedback to the user according to the emotion.

[1673] Step 15:

[1674] The server tracks the user's progress and emotional changes in real time and records them in a database.

[1675] Step 16:

[1676] The user enters a question or concern, and the device sends the question to the server.

[1677] Step 17:

[1678] The server uses a generative AI model to instantly generate answers and communicate them to the user through a character.

[1679] Step 18:

[1680] The user presses a button to end the learning session, and the device performs the termination process.

[1681] Step 19:

[1682] The server automatically conducts a learning level test, and the test screen is displayed on the user's device.

[1683] Step 20:

[1684] The user takes the test. The server analyzes the test results.

[1685] Step 21:

[1686] Based on the analysis results, the server identifies the user's weaknesses and areas for improvement, and personalizes the next lesson. The emotion engine also analyzes the emotional data it recognizes, improving the quality of feedback and the curriculum.

[1687] Step 22:

[1688] The server sends personalized feedback and the next curriculum to the user's device, which displays it.

[1689] In this way, by combining a generative AI model and an emotion engine, this system is able to provide efficient and individually optimized learning support to users 24 hours a day, 365 days a year.

[1690] Example 2

[1691] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1692] Conventional learning support systems have had difficulty providing personalized learning tailored to the individual needs and progress of users. Furthermore, they lacked the ability to recognize and respond to users' emotions and confusion in real time, making it difficult to maximize the effectiveness of learning. There was a need to provide a system that would solve these problems and enable users to study for qualifications efficiently and effectively.

[1693] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1694] In this invention, the server includes a means for pre-training learning material data using a generative AI model related to qualifications, a means for using an agent selected or generated by the user, and a means for tracking the user's learning progress in real time and recording it in a database. This enables personalized learning according to the user's individual needs and learning progress. The agent also deepens the user's understanding by conducting lessons involving simple movements. Furthermore, the emotion engine recognizes the user's facial expressions and vocalizations in real time and reflects that feedback to maximize learning effectiveness.

[1695] A "generative AI model for qualifications" is an artificial intelligence model that pre-learns educational material data related to specific qualifications and provides answers and educational materials in response to user questions and customizations.

[1696] An "agent" is a character or avatar that the user can select or create, and that character or avatar is responsible for explaining the learning content.

[1697] "Tracking" refers to the process of tracking a user's learning progress and behavior in real time and recording it in a database.

[1698] "Database" means a data storage system for managing user data, progress, learning outcomes, etc. collected within the system.

[1699] An "emotion engine" is an algorithm or software component that analyzes a user's facial expressions and tone of voice in real time and recognizes their emotions.

[1700] A "proficiency test" is a test that is automatically administered after a user completes a learning session to assess the user's learning progress.

[1701] "Personalization" refers to the process of individually optimizing the next learning content and curriculum based on each user's learning progress and weaknesses.

[1702] This invention is a system that provides effective learning support to users aiming to obtain qualifications, and in addition to learning support using generative AI models and agents, it features an emotion engine that recognizes the user's emotions.

[1703] User registration and agent creation

[1704] A user accesses the system, enters the required information such as name, email address, and password on the new registration page, and submits it. The server receives the entered information and stores it in a database. The server then sends a confirmation email to the user's email address. When the user clicks on the link in the confirmation email, the account is confirmed.

[1705] After verifying their account, users are taken to a page where they can select or create their preferred agent. This page provides an interface for customizing the agent's appearance and behavior. Once the user has configured their agent, the information is sent to the server and stored in a database.

[1706] Select study materials and start learning

[1707] Users select the type of qualification within the system and choose the corresponding study materials. The selection is sent to the server, which loads the selected study material data into a generative AI model. For example, GPT-3 or BERT can be used as the generative AI model. Based on the loaded data, the server generates a curriculum specifically for the user, and the user is ready to begin learning.

[1708] Running a learning session

[1709] The user's device displays a screen for the learning session, simultaneously displaying the generated agent and the learning content. The agent explains the content of the learning material with voice and action. The emotion engine recognizes the user's emotions from their facial expressions and vocalizations and provides feedback in real time. For example, if the emotion engine determines that the user is confused, the agent will provide additional explanations and encouragement according to that emotion. The server monitors the user's progress and emotional changes in real time and stores this information in a database. If the user has any questions, they can enter them and the questions are sent to the server. The server uses a generative AI model to instantly generate answers and communicates them to the user via the agent.

[1710] Measurement and feedback

[1711] When a user finishes a learning session, they are automatically tested for their level of mastery. This test is administered by the server, and the test results are analyzed. Based on the analysis results, the server identifies the user's weaknesses and areas for improvement, and personalizes the next learning content based on these. In addition, emotional data recognized by the emotion engine is also added to the analysis, enabling more accurate feedback and curriculum adjustments. This allows users to continue learning efficiently and in line with their individual needs and emotions.

[1712] Specific examples

[1713] For example, a user registers with the system aiming for the "Bookkeeping Level 3" certification. First, they access the system and create a new account, then generate a custom agent based on Pikachu. After generating the agent, they select the "Bookkeeping Level 3" study materials and begin a learning session. The agent explains basic accounting concepts, and whenever the user asks a question, the generated AI model provides an immediate answer. If the emotion engine detects a confused expression on the user's face during the learning session, the agent will provide additional explanations or encouragement. After the learning session ends, a mastery test is automatically conducted, and the next learning content is personalized based on the test results.

[1714] Prompt Sentence Examples

[1715] "Please explain a learning support system using custom agents with emotion recognition."

[1716] By combining a generative AI model and an emotion engine, the system of this invention can provide efficient and individually optimized learning support to users 24 hours a day, 365 days a year.

[1717] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1718] Step 1:

[1719] A user accesses the system, enters the required information such as name, email address, and password on the new registration page, and submits it. The input includes the user's personal information (name, email address, password). The output is the user information sent to the server.

[1720] Step 2:

[1721] The server receives the entered user information and stores it in a database.

[1722] Input: User's personal information (name, email address, password)

[1723] Data processing: Convert received information into the appropriate format and store it in a database

[1724] Output: Save confirmation message

[1725] Step 3:

[1726] The server sends a confirmation email to the user's email address, and when the user clicks on the link in the email, the account is authenticated.

[1727] Input: User's email address

[1728] Data processing: Generating and sending confirmation emails

[1729] Output: Account verification link

[1730] Step 4:

[1731] After completing the verification, the user proceeds to the agent creation page and uses the interface to customize the agent's appearance and behavior.

[1732] Input: User's agent selection and customization information

[1733] Data processing: Importing and verifying selected information

[1734] Output: Agent customization data

[1735] Step 5:

[1736] The server receives the customized agent information and stores it in a database.

[1737] Input: Agent customization information

[1738] Data processing: Format conversion and storage of received information

[1739] Output: Agent creation confirmation message

[1740] Step 6:

[1741] Users select the type of qualification within the system and choose the corresponding study materials.

[1742] Input:Select qualification type

[1743] Data processing: Search for teaching material data corresponding to the selected qualification

[1744] Output: List of teaching material data

[1745] Step 7:

[1746] The server loads the selected teaching material data into a generative AI model and generates a curriculum tailored to the user.

[1747] Input: Teaching material data

[1748] Data processing: Loading teaching material data and generating curriculum

[1749] Output: User-specific curriculum

[1750] Step 8:

[1751] The learning session screen is displayed on the user's device. The generated agent and the learning content are displayed simultaneously, and the agent explains the learning content with voice and actions.

[1752] Input: User-specific curriculum and agent data

[1753] Data processing: Synchronization of screen display with voice and action

[1754] Output: Learning screen

[1755] Step 9:

[1756] The emotion engine recognizes the user's facial expressions and vocalizations in real time and provides feedback. For example, if it detects a confused expression, the agent will provide additional explanation or encouragement.

[1757] Input: User's facial expression data and voice data

[1758] Data processing: Sentiment analysis and feedback content generation

[1759] Output: Feedback content

[1760] Step 10:

[1761] If a user has a question while studying, they can enter it and send it to the server.

[1762] Input: User question

[1763] Data processing: Question analysis and answer generation using an AI model

[1764] Output: Response data

[1765] Step 11:

[1766] The server uses a generative AI model to generate answers and communicates them to the user through an agent.

[1767] Input: Question data

[1768] Data processing: Answer generation using generative AI models

[1769] Output: Response via agent

[1770] Step 12:

[1771] Once the user has finished the learning session, the mastery test will begin automatically.

[1772] Input: End of study session trigger

[1773] Data processing: Implementing learning level tests and compiling the results

[1774] Output: Test results

[1775] Step 13:

[1776] The server analyzes the test results and extracts the user's strengths and weaknesses.

[1777] Input: Test result data

[1778] Data processing: Analyzing test results and identifying weaknesses

[1779] Output: Analysis results

[1780] Step 14:

[1781] The server personalizes the next learning content based on the analysis results and provides feedback taking into account the emotional data recognized by the emotion engine.

[1782] Input: Analysis results and sentiment data

[1783] Data processing: Customizing the next learning content

[1784] Output: Personalized learning and feedback

[1785] (Application example 2)

[1786] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1787] Conventional learning support systems have difficulty providing effective learning support based on the user's individual emotions and progress, and are insufficiently personalized to meet individual needs. Furthermore, when dealing with customers in brick-and-mortar stores, it is difficult to provide efficient explanations and customer service that reflect the customer's emotions and needs in real time. There is a need for a system that can solve these issues and provide support and services that are optimized for each user and customer.

[1788] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for pre-learning learning material data using a generative AI model specialized for qualifications; means for selecting or generating a character preferred by the user; means for conducting qualification-related lessons using the selected or generated character; means for tracking and recording learning progress in real time; means for generating answers to the user's questions using the generative AI model and communicating them through the character; means for automatically conducting a mastery test after completion of learning and analyzing the results; means for personalizing the next learning content based on the analysis results; means for analyzing the user's emotions using an emotion recognition engine and adjusting the character's movements and speech based on the data; and means for providing personalized product explanations and customer service to customers in stores as part of personalization. This enables effective learning support that reflects the user's emotions and progress in real time, making it possible to provide explanations and customer service optimized for each customer even in physical stores.

[1789] A "generative AI model specialized for qualifications" is an artificial intelligence model that pre-learns educational material data related to obtaining specific qualifications and generates appropriate answers and explanations.

[1790] An "emotion recognition engine" is a system that analyzes the emotions of a user, such as facial expressions and vocalizations, in real time, and extracts and utilizes that emotional data.

[1791] A "character" is a virtual avatar or character selected or created by the user that acts as a facilitator of learning or guidance.

[1792] "Means for tracking and recording learning progress in real time" refers to a mechanism for monitoring a user's learning activities and progress in real time and recording that information.

[1793] "Generating answers using a generative AI model" means that artificial intelligence generates appropriate answers to questions from users based on information it has learned in advance.

[1794] "Means of communicating through a character" refers to a method of communicating the generated answer or explanation to the user through the voice or actions of a character selected by the user.

[1795] The "means for automatically conducting a mastery test after completion of learning" is a mechanism for automatically conducting a test to measure the user's level of understanding after the learning session is completed.

[1796] The "means for analyzing the results" is a method for analyzing the results of the mastery test in detail and understanding the user's strengths and weaknesses.

[1797] "Means for personalizing the next learning content" is a mechanism that provides learning content optimized for the user's individual needs and progress based on the analysis results.

[1798] "Means for providing personalized product explanations and customer service" refers to a system for providing appropriate product explanations and customer service in physical stores based on the individual emotions and needs of customers.

[1799] The present invention relates to a system that provides learning support and customer service support in physical stores using a generative AI model and an emotion recognition engine. Specific examples for implementing the present invention are described in detail below.

[1800] 1. System Configuration

[1801] This system provides effective learning support for users who are aiming to serve customers in brick-and-mortar stores and obtain qualifications. The system is primarily composed of the following hardware and software:

[1802] Hardware: Smartphones, smart glasses

[1803] Software: Mobile app, server-side program (Python + Flask), database (MySQL), generative AI model (OpenAI's GPT-4), emotion recognition engine (Microsoft Azure Face API)

[1804] 2. Program Processing

[1805] Customer registration and character creation

[1806] The server creates an account based on the information the user entered and sends a confirmation email. When the user clicks the confirmation link, the account is activated and the user is then taken to the character creation page. The user customizes the character's appearance and behavior to their liking, and this information is sent to the server and stored in a database.

[1807] Product selection and introduction

[1808] Users select a product category through their smart device. The selection is sent to the server, which then loads the selected product data into the generative AI model, which then creates a personalized guidance curriculum for the customer based on the data.

[1809] Running a store navigation session

[1810] An information screen is displayed on the user's device, and a generated character explains the product. An emotion recognition engine analyzes the user's emotions from their facial expressions and vocalizations and provides feedback in real time. The server monitors the user's progress and emotions in real time and stores them in a database. When the user enters a question, the generative AI model instantly generates an answer and communicates it to the user through the character.

[1811] Measurement and feedback

[1812] After the guidance session ends, a satisfaction test is automatically conducted by the server. The test results are analyzed in detail by the server and feedback is provided to identify the user's strengths and weaknesses. Based on the analysis results, the next guidance content is personalized.

[1813] As a specific example, consider the case where a user visits a physical store and uses smart glasses to receive product information. At this time, a character explains the product based on product data previously learned by the generative AI model. If the emotion recognition engine detects a confused expression on the user's face, the character will provide additional explanation to help the user understand. In this way, effective customer service is possible, reflecting the user's emotions and progress in real time.

[1814] Example prompt sentence:

[1815] "Based on the product category selected by the customer, we use an emotion recognition engine to analyze the user's emotions and generate the most appropriate product description."

[1816] As described above, by combining a generative AI model and an emotion recognition engine, this system can provide individually optimized learning support and customer service to users and customers.

[1817] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1818] Step 1:

[1819] The user accesses the app from smart glasses or a smartphone, enters the required information such as name, email address, and password on the new registration page, and submits it. Input: Registration information entered by the user. Output: Registration information is sent to the server and saved in the database. Specifically, after the server receives the entered information, it saves it as a new record in the database.

[1820] Step 2:

[1821] The server sends a confirmation email to the user's email address. Input: The user's email address included in the registration information. Output: A confirmation email is sent to the user's email address. Specifically, the server generates and sends the confirmation email using the SMTP protocol.

[1822] Step 3:

[1823] When the user clicks the link in the confirmation email, the account is confirmed. Input: The confirmation link the user clicked. Output: The server activates the account. Specifically, the server updates the user's account record and sets the confirmation flag.

[1824] Step 4:

[1825] The user creates or selects a character of their choice. Input: Customization information for the appearance and behavior of the character selected by the user. Output: Character creation information is sent to the server and saved in a database. Specifically, the server receives the character information set by the user and saves it in a database.

[1826] Step 5:

[1827] The user selects a product category through the app. Input: The product category selected by the user. Output: The product category information is sent to the server and loaded into the generative AI model. Specifically, the server receives the product category information and passes it to the generative AI model.

[1828] Step 6:

[1829] The generative AI model generates a guidance curriculum based on product data. Input: Product category information. Output: Generated guidance curriculum. Specifically, the generative AI model analyzes the given product category information and generates an appropriate guidance curriculum.

[1830] Step 7:

[1831] An information screen is displayed on the user's device, and the generated character explains the product. Input: Generated information curriculum. Output: Product description displayed on the device. Specifically, the character explains the product using voice and actions based on the information curriculum obtained from the generation AI model.

[1832] Step 8:

[1833] The emotion recognition engine analyzes the user's facial expressions and vocalizations to recognize emotions. Input: User's facial expression and vocalization data. Output: Recognized emotion data. Specifically, the emotion recognition engine analyzes the user's facial images and voice data in real time to extract their emotional state.

[1834] Step 9:

[1835] The server monitors the user's progress and emotions in real time and records them in a database. Input: User's progress and emotion data. Output: Progress and emotion data stored in the database. Specifically, the server receives the user's learning progress and emotion data and stores it in the database.

[1836] Step 10:

[1837] The user inputs a question and sends it to the server. Input: User's question. Output: Answer generated by the generative AI model. Specifically, the server receives the question from the user and uses the generative AI model to generate an appropriate answer.

[1838] Step 11:

[1839] The generated answer is communicated to the user through the character. Input: The answer generated by the generative AI model. Output: The answer communicated to the user. The specific action is for the character to communicate the answer to the user through voice and action.

[1840] Step 12:

[1841] After the guidance session ends, a satisfaction test is automatically conducted. Input: Learning / guidance session end trigger. Output: Satisfaction test result. Specifically, the server detects the end of the session and automatically conducts a satisfaction test.

[1842] Step 13:

[1843] The server analyzes the test results and identifies the user's strengths and weaknesses. Input: Satisfaction test results. Output: Analysis results. Specifically, the server analyzes the test results in detail to extract the user's level of understanding and areas for improvement.

[1844] Step 14:

[1845] Based on the analysis results, the next announcement content is personalized. Input: Analysis results. Output: Personalized next announcement content. Specifically, the server generates announcement content optimized for the user's individual needs based on the analysis results.

[1846] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1847] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1848] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1849] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1850] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1851] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1852] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1853] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Bra...

Claims

1. A means of pre-training teaching material data using a generative AI model specialized for qualifications, and A means for a user to select or generate a character of their choice; a means for conducting qualification-related lessons using the selected or generated character; A means to track and record learning progress in real time; A method to generate answers to user questions using a generative AI model and communicate them through characters, A means to automatically conduct a mastery test after the learning is completed and analyze the results, A system that includes a means to personalize the next learning content based on the analysis results.

2. 2. The system according to claim 1, further comprising means for creating an account based on information entered by the user and sending a confirmation email.

3. 2. The system according to claim 1, further comprising means for the character to give a lesson while making simple movements.

4. 2. The system according to claim 1, further comprising means for extracting weak points of the user based on the results of the mastery test and suggesting points to be improved.

5. 2. The system according to claim 1, further comprising means for generating and providing an individual curriculum according to the progress of the user.

Images