System

Abstract

A system is provided.SOLUTION: A system comprising: means for collecting and storing user profile information; means for generating an appropriate virtual adventure plan based on the profile information; means for providing language learning and motor tasks based on the generated adventure plan; means for monitoring the progress of the language learning and motor tasks and providing real-time feedback; and means for generating an overall assessment and suggesting a next adventure plan when the adventure is completed.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In today's society, where elderly people face challenges such as feelings of isolation, declining cognitive function, and maintaining their health, traditional learning and exercise methods are difficult to provide sustainable and enjoyable experiences for them. Furthermore, for the new generation of elderly people who are familiar with the digital world, traditional methods may not be enough to satisfy them. To solve this problem, new forms of entertainment that allow people to enjoy language learning and exercise at the same time are needed. [Means for solving the problem]

[0005] The present invention provides a means for collecting and storing user profile information. It also includes a means for generating an appropriate virtual adventure plan based on that information. It also includes a means for providing language learning tasks and exercise tasks based on the generated adventure plan, allowing the user to learn and exercise while having fun. It also has a function for monitoring the progress of the language learning tasks and exercise tasks and providing feedback in real time. It also includes a means for generating a comprehensive evaluation and proposing the next adventure plan upon completion of the adventure. This provides a system that helps seniors maintain their cognitive function and promote their health without feeling isolated.

[0006] "Profile information" refers to information about a user's personal characteristics and preferences, such as their language learning level, cultural interests, and exercise preferences.

[0007] A "virtual adventure plan" is a set of adventure scenarios and tasks that are generated based on the user's profile information and that the user experiences in a virtual space.

[0008] A "language learning task" is a learning activity that includes quiz-style questions and other activities aimed at improving language skills.

[0009] An "exercise task" is a physical activity that a user performs based on exercise instructions displayed on the screen.

[0010] "Real-time feedback" refers to feedback that monitors the user's progress in language learning or exercise and provides appropriate advice and evaluation in real time.

[0011] The "overall evaluation" is the result of an overall evaluation of the user's performance, generated after the virtual adventure plan is completed.

[0012] A "next adventure plan" is a set of new virtual adventure scenarios or tasks that are suggested to the user after the adventure they have completed. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0014] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0017] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0018] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0019] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0021] [First embodiment]

[0022] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0023] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0026] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0029] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0033] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0034] This invention is a system for helping seniors learn languages ​​and improve their health while having fun. The system collects user profile information and provides an optimal virtual adventure plan based on that information. It combines language learning tasks with physical activities to provide users with a fun and fulfilling experience.

[0035] Explaining program processing in natural language

[0036] 1. User Registration and Login

[0037] User: Opens the app and enters their name, age, email address, and password on the new registration screen.

[0038] Terminal: Send this information to the server and display a message indicating registration is complete.

[0039] Server: Validates the input information and stores it in a database. Once stored, it provides login authentication to the device.

[0040] 2. Profile Settings

[0041] User: After logging in, enter your language learning level, cultural interests, exercise preferences, etc. in the profile settings screen.

[0042] Terminal: Sends the entered information to the server and prepares for the next step.

[0043] Server: Stores profile information and prepares the next appropriate virtual adventure plan.

[0044] 3. Start your virtual adventure

[0045] Server: Runs an algorithm to generate an adventure plan based on the user's profile and sends the plan to the device.

[0046] Terminal: Displays the adventure start screen and provides a dialogue interface with the robot avatar.

[0047] User: Press the "Start Adventure" button to begin the virtual adventure.

[0048] 4. Combining language learning and exercise

[0049] Server: Generates language learning tasks (e.g., Japanese quizzes) and exercise tasks (e.g., stretching).

[0050] Terminal: displays language learning tasks and provides an interface for users to enter answers.

[0051] User: Completes a quiz and then performs a motor task.

[0052] Terminal: Sends quiz answers to the server and displays instructions for the exercise task.

[0053] Server: Determines the answers to the quiz and generates the next task based on the progress.

[0054] 5. Real-time feedback

[0055] Server: Monitors language learning and motor progress and provides real-time feedback.

[0056] Terminal: Shows feedback messages and notifies the user of the next task.

[0057] User: Receive feedback and move on to the next task.

[0058] 6. Adventure Completion and Evaluation

[0059] Server: Once the adventure is completed, generate an overall evaluation and propose a plan for the next adventure.

[0060] Device: Displays the overall rating screen and presents the user with options for their next adventure.

[0061] User: Checks the rating and decides whether to select the next adventure or quit.

[0062] Specific examples

[0063] Example: A 60-year-old user is interested in learning Japanese and Spanish culture.

[0064] User: Launch the app, register, and enter the required information. Then, on the profile settings screen, enter that you have "Intermediate Japanese" and are "Interested in Spanish culture."

[0065] Terminal: Sends this information to the server and receives matching adventure plans from the server.

[0066] Server: Generates and sends a virtual adventure plan for a trip to Spain based on the user's information.

[0067] User: The adventure begins and the user answers a Japanese quiz (e.g., "Please translate this sentence: Hola! ¿Cómo estás?").

[0068] Terminal: Sends the quiz results to the server and then displays the stretching exercise task.

[0069] Server: Evaluates the user's answers, monitors the progress of the motor task, and generates feedback.

[0070] On the device: Provides real-time feedback and suggests next steps to the user.

[0071] User: Completes all tasks, receives final evaluation, and decides whether to select the next adventure plan.

[0072] In this way, users can learn languages ​​and promote health through exercise while enjoying different cultures in a virtual environment. This system helps seniors improve their cognitive function and maintain their health without feeling isolated.

[0073] The processing flow will be explained below.

[0074] Step 1:

[0075] User: Opens the app and enters their name, age, email address, and password on the new registration screen.

[0076] Terminal: Collects the entered information and sends it to the server.

[0077] Server: Validates the received user information and stores it in the database if there are no problems. Returns a registration completion message to the terminal.

[0078] Device: Receives the registration completion message and notifies the user. The device transitions to the login screen.

[0079] Step 2:

[0080] User: After logging in, enter your language level, cultural interests, and exercise preferences in the profile settings screen.

[0081] Device: Collects profile information entered and sends it to the server.

[0082] Server: Receives the profile information, stores it in a database, and associates the profile information with the user account.

[0083] Step 3:

[0084] Server: Generates an appropriate virtual adventure plan based on the user's profile information. Sends the generated adventure plan to the device.

[0085] Terminal: Displays the adventure start screen and provides a dialogue interface with the robot avatar.

[0086] User: Press the "Start Adventure" button to begin the virtual adventure.

[0087] Step 4:

[0088] Server: Creates a scenario for generating language learning tasks and exercise tasks. For the language learning tasks, it presents quiz-style questions to the user.

[0089] Terminal: Provides an interface that displays language learning tasks and allows users to enter answers.

[0090] User: Answers the quiz and presses the submit button.

[0091] Terminal: Sends the quiz answers to the server.

[0092] Server: Determines whether the quiz is correct or incorrect and returns the result to the device.

[0093] Terminal: Displays the judgment result and then the motor task screen.

[0094] User: Follow the exercise instructions displayed on the screen and perform physical exercise.

[0095] Step 5:

[0096] Server: Monitors the user's progress in language learning and motor tasks and generates feedback in real time.

[0097] Terminal: Display a feedback message to the user, asking if the user is ready to proceed to the next task.

[0098] User: Review the feedback and move on to the next task.

[0099] Step 6:

[0100] Server: Checks that all tasks are completed, evaluates the user's performance, and generates the next adventure plan and sends it to the device.

[0101] Device: Displays an overall rating screen and offers the user options for their next adventure.

[0102] User: Check the overall rating and decide whether to select the next adventure plan or exit the system.

[0103] This series of steps provides a system that allows elderly people to have fun, learn about different cultures and languages, and get appropriate exercise at the same time.

[0104] Example 1

[0105] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0106] There is a need for a system that allows elderly people to learn languages ​​and promote their health while having fun. However, many elderly people find that current learning methods lack enjoyment and continuity. Therefore, there is an urgent need to develop a system that allows elderly people to continue enjoying activities without straining themselves. There is also a need for a system that can efficiently combine language learning and exercise. Furthermore, there is a need for a system that can provide an optimized virtual adventure plan for each individual user and provide feedback in real time.

[0107] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0108] In this invention, the server includes means for collecting and storing user attribute information, means for generating an appropriate virtual adventure plan based on the attribute information, and means for providing language learning tasks and exercise tasks based on the generated adventure plan. This allows the user to enjoy learning a language and exercising through an individually optimized adventure plan. The server also includes means for monitoring the progress of the language learning tasks and exercise tasks and providing immediate feedback, and means for generating a comprehensive evaluation and proposing the next adventure plan upon completion of the adventure. This allows the user to check their progress in real time and proceed to the next step. This also allows elderly people to continue enjoying learning and exercise without straining themselves.

[0109] "Attribute information" refers to personal data about the user, such as age, interests, language learning level, and type of exercise.

[0110] A "virtual adventure plan" refers to a series of adventures or tasks to be performed in a virtual environment that are generated based on the user's attribute information.

[0111] A "language learning task" refers to an activity in which a user solves a specified language problem or challenge.

[0112] "Exercise task" refers to an activity that includes instructions for a movement or exercise that a user should perform.

[0113] "Immediate feedback" refers to providing real-time evaluation and advice on the results of a user's language learning or motor task.

[0114] "Overall evaluation" refers to a comprehensive evaluation of the results of all language learning tasks and motor tasks performed by the user.

[0115] "When the adventure is complete" refers to the point at which the user has completed all assigned language learning and physical tasks.

[0116] A "virtual adventure plan" refers to an adventurous activity or experience in a virtual environment that is individually generated based on the user's attribute information.

[0117] This invention is a system for elderly people to learn languages ​​and improve their health while having fun. The system collects user attribute information and provides an optimal virtual adventure plan based on that information. It combines language learning tasks with physical activities to provide users with a fun and fulfilling experience.

[0118] A specific embodiment of this system is as follows.

[0119] 1. Hardware and Software

[0120] server:

[0121] Hardware: A high-performance server (e.g., Dell PowerEdge series).

[0122] Software: database management system (e.g., MySQL®), programming language (e.g., Python) for executing algorithms.

[0123] Device:

[0124] Hardware: Tablet, smartphone, or PC.

[0125] Software: Dedicated application (e.g., iOS or ANDROID app), browser-based interface.

[0126] User:

[0127] Hardware: Devices operated by users, such as tablets and smartphones.

[0128] Software: The dedicated application or browser-based interface.

[0129] 2. Data processing and calculation

[0130] User Registration and Login:

[0131] The user opens the app and enters their name, age, email address, and password on the new registration screen. The device sends this information to the server, which verifies the data and stores it in a database. When logging in, the server verifies the email address and password entered by the user, generates an authentication token, and sends it to the device.

[0132] Profile Settings:

[0133] After logging in, users enter their language learning level, cultural interests, exercise preferences, etc. on the profile setting screen. The device sends the entered information to the server, which then stores it in a database.

[0134] Create and launch a virtual adventure:

[0135] The server uses an algorithm to generate a virtual adventure plan based on the stored user attribute information. The generated plan is sent to the terminal, and the user presses the "Start Adventure" button to begin the virtual adventure.

[0136] Verbal learning and motor task performance:

[0137] The server generates language learning tasks (e.g., "Please translate this phrase") and exercise tasks (e.g., "Stretch for 5 minutes") based on the user's progress. The device presents these to the user and sends the user's answers and exercise progress to the server. The server evaluates the answers and generates the next task.

[0138] Real-time feedback:

[0139] The server monitors the user's progress in real time and generates and sends feedback to the device, allowing the user to move on to the next task.

[0140] Adventure completion and rating:

[0141] When the adventure is completed, the server generates a comprehensive evaluation and proposes the next adventure plan. The terminal presents the evaluation to the user, who then decides whether to select the next adventure.

[0142] 3. Examples and prompts

[0143] Examples:

[0144] If a 60-year-old user is interested in learning Japanese and French culture, the user launches the app, registers, and enters the necessary information. Then, on the profile settings screen, the user enters "Intermediate Japanese" and "Interested in French culture." Based on this information, the server generates a virtual adventure plan for a "trip to France" and sends it to the device. The user begins the adventure, answers a Japanese quiz (e.g., "Translate this sentence: Bonjour!"), and then performs an exercise task (e.g., "Stretch for 5 minutes").

[0145] Example prompt sentence:

[0146] Here is an example of a prompt to provide more details to a generative AI model:

[0147] Describe the specific steps of a system that helps elderly people improve their health while learning a language. The system's process consists of the following steps: user registration and login, profile setup, adventure plan generation and initiation, language learning and exercise task execution, real-time feedback, and adventure completion and evaluation. For each step, provide a detailed explanation of how the server, device, and user operate.

[0148] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0149] Step 1:

[0150] User Registration and Login

[0151] input:

[0152] The user opens the app and enters their name, age, email address, and password on the new registration screen.

[0153] Data processing and calculation:

[0154] The device sends the entered information to the server, which checks the integrity of the data and stores it in a database if there are no problems. When logging in, the entered email address and password are compared with the database, and if they match, an authentication token is generated.

[0155] output:

[0156] A registration completion message is displayed and an authentication token is returned.

[0157] Specific behavior:

[0158] The user enters the name "tanaka60", age "60", email address "tanaka60@example.com", and password "password123", and the device sends this to the server. The server stores it in a database, generates an authentication token, and sends it to the device. The device displays "Registration completed" to the user.

[0159] Step 2:

[0160] Profile Settings

[0161] input:

[0162] After logging in, users enter their language learning level, cultural interests, and exercise preferences on the profile settings screen.

[0163] Data processing and calculation:

[0164] The terminal transmits the input information to a server, which then stores the attribute information in a database.

[0165] output:

[0166] A save completion message will be displayed and preparations for the next step will be made.

[0167] Specific behavior:

[0168] The user selects "Japanese Beginner's," "French Culture," and "Yoga," and the device sends this to the server. The server saves this and displays the message "Profile saved."

[0169] Step 3:

[0170] Creating and launching a virtual adventure

[0171] input:

[0172] The server generates a virtual adventure plan based on the stored profile information.

[0173] Data processing and calculation:

[0174] The server uses an algorithm to generate a virtual adventure plan and transmits it to the terminal.

[0175] output:

[0176] The generated adventure plan is displayed and a "Start Adventure" button is presented to the user.

[0177] Specific behavior:

[0178] The server generates an adventure plan based on the theme of "Travel to France," combining Japanese conversation quizzes and yoga exercise tasks, and sends it to the terminal. The terminal then displays "Do you want to start your adventure traveling to France?" and presents the user with a "Start Adventure" button.

[0179] Step 4:

[0180] Verbal learning and motor task performance

[0181] input:

[0182] The server generates language learning tasks and motor tasks and sends them to the terminal.

[0183] Data processing and calculation:

[0184] The server generates questions, receives answers, and evaluates the user's progress, then generates instructions for the motor task.

[0185] output:

[0186] The language learning task and the motor task are displayed on the screen, and the user's answers and motor execution data are transmitted.

[0187] Specific behavior:

[0188] The server generates a quiz question, "Bonjour! Please translate this phrase into Japanese," and sends it to the device. The user answers, "Hello!", and the device sends this to the server. The server evaluates the answer, then generates an exercise task, "Please stretch for 5 minutes," and sends it to the device. The user performs the exercise and presses the "Task Complete" button.

[0189] Step 5:

[0190] Real-time feedback and next steps

[0191] input:

[0192] Receive the user's answers and exercise progress data.

[0193] Data processing and calculation:

[0194] The server evaluates the answers and generates feedback to send to the user.

[0195] output:

[0196] Get real-time feedback messages and next task notifications.

[0197] Specific behavior:

[0198] The server generates a message saying "Great, you got it right!" and prepares to present the next quiz or exercise task. The device displays a feedback message and offers the user a "Proceed to next task" button.

[0199] Step 6:

[0200] Adventure completion and evaluation

[0201] input:

[0202] Collect data when all tasks are completed.

[0203] Data processing and calculation:

[0204] The server generates an overall rating and suggests the next adventure plan.

[0205] output:

[0206] An overall evaluation screen will be displayed, and options for the next adventure will be presented.

[0207] Specific behavior:

[0208] The server evaluates the user's overall performance and generates a message saying, "Adventure successful! We recommend a trip to Italy as your next adventure." The device displays the evaluation results along with the next options, providing the user with an interface to select "Start the next adventure" or "End."

[0209] (Application example 1)

[0210] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0211] This invention relates to a system for helping seniors learn languages ​​and promote their health while having fun. Specifically, the system aims to reduce loneliness and improve cognitive function and health by providing a virtual adventure that combines language learning and physical exercise through a realistic experience within a brick-and-mortar store. It is also required to provide real-time feedback and closely monitor the user's progress.

[0212] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0213] In this invention, the server includes means for collecting and storing personal information of the user, means for generating an appropriate virtual adventure plan based on the personal information, means for providing language learning tasks and physical exercise tasks based on the generated adventure plan, means for monitoring the progress of the language learning tasks and physical exercise tasks and providing feedback in real time, means for generating an overall evaluation and proposing a next adventure plan when the adventure is completed, means for providing an adventure scenario within the physical store, means for performing the language learning tasks and physical exercise tasks at specified locations while walking within the physical store, and means for providing real-time evaluation and feedback. This enables the user to improve their health while having fun learning a language through a realistic experience within the physical store.

[0214] "User personal information" refers to information such as the user's name, age, language learning level, cultural interests, and physical activity preferences.

[0215] "Virtual adventure plan" refers to a plan that includes language learning tasks and physical exercise tasks, generated based on the user's personal information.

[0216] A "language learning task" refers to the task of quiz-style questions posed to the user and judging their answers.

[0217] "Physical exercise task" refers to an exercise performed by a user based on exercise instructions displayed on a screen.

[0218] "Brick and mortar store" refers to a retail store or facility that a user can actually visit.

[0219] A "virtual adventure scenario" is a scenario that a user experiences in a physical store, and refers to content that is constructed based on a virtual adventure plan.

[0220] "Real-time feedback" refers to assessment and instruction provided in real time on a user's progress as they perform a language learning task or physical movement task.

[0221] "Overall Grade" refers to the overall grade or rating provided upon completing a series of tasks in a virtual adventure.

[0222] The system embodying this invention collects personal information from users, creates a virtual adventure plan, and provides language learning tasks and physical exercise tasks to promote language learning and health for the elderly. Specific embodiments are described below.

[0223] 1. User Registration and Profile Settings

[0224] The server collects personal information entered by the user (such as name, age, language learning level, cultural interests, physical activity preferences, etc.) and stores it in a database. Django is a suitable server-side framework for this.

[0225] The device sends the information entered by the user to the server via the interface and prepares to proceed to the next step. The device can be a smartphone or tablet, and the application is implemented using React Native.

[0226] 2. Creating and launching a virtual adventure plan

[0227] The server generates a virtual adventure plan based on the collected personal information using Python scripts and serverless architectures such as AWS Lambda. Specifically, the plan is automatically created to include language learning tasks and physical exercise tasks that match the user's interests and level.

[0228] The device displays the generated adventure plan to the user and prompts them to start the adventure. An adventure scenario in a physical store is displayed, and when the user presses the "Start Adventure" button, the actual task begins.

[0229] 3. Providing language learning and physical motor tasks

[0230] The server creates language learning tasks (e.g., quiz-style questions) and physical exercise tasks (e.g., stretching and walking) and sends them to the terminal. PostgreSQL is often used as the database.

[0231] The terminal displays the language learning tasks and provides an interface where the user can enter answers, and displays instructions for the physical motor tasks so that the user can perform them.

[0232] 4. Real-time feedback

[0233] The server monitors the progress of language learning tasks and physical motor tasks and generates real-time feedback using real-time communication over WebSockets.

[0234] The terminal displays the feedback messages received from the server in real time and notifies the user of the next task.

[0235] 5. Adventure Completion and Evaluation

[0236] Once the adventure is completed, the server generates an overall evaluation and proposes a plan for the next virtual adventure. Based on the evaluation algorithm, the evaluation is calculated based on the user's performance and progress.

[0237] The terminal displays the overall rating and presents the user with options for the next adventure.

[0238] Specific examples

[0239] Example: If a 60-year-old user is interested in learning Japanese and Spanish culture, the user opens the app, enters their personal information, and then enters "Intermediate Japanese" and "Interested in Spanish culture" in the profile settings screen. The user is then presented with a virtual adventure scenario set in a physical store, where they answer Japanese language quizzes and perform stretching exercises in designated areas. The device displays real-time feedback, allowing the user to track their progress as they progress through the adventure.

[0240] Prompt Sentence Examples

[0241] "I want to design a system that helps seniors learn languages ​​and promote their health while having fun. This system collects users' personal information and then provides them with an optimal virtual adventure plan based on that information. The application combines language learning tasks with physical exercise tasks to provide a fun and fulfilling experience. For example, if a 65-year-old user is interested in French culture, the app can have them complete quizzes about French tourist attractions and related stretching exercises, and provide real-time feedback. Please tell me more about how you will implement this system and the technologies you will use."

[0242] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0243] Step 1:

[0244] User registration and profile settings

[0245] Users open a smartphone app and enter personal information such as their name, age, language learning level, cultural interests, and physical activity preferences.

[0246] Input: Name, age, language level, cultural interests, physical activity preferences

[0247] The terminal transmits the input information to the server.

[0248] Input: User personal information

[0249] The server stores the received information in a database. The information is managed using the Django framework.

[0250] Output: User's personal information stored in a database

[0251] The server sends a message to the terminal confirming that the save has been completed.

[0252] Output: Save complete confirmation message

[0253] Step 2:

[0254] Virtual adventure planning

[0255] The server generates a virtual adventure plan based on the user's personal information. The algorithm used is implemented in Python and processed in a scalable manner using AWS Lambda.

[0256] Input: User personal information

[0257] Data processing: Data calculations to generate optimal adventure plans based on the user's interests and learning level

[0258] Output: Virtual adventure plan

[0259] The server transmits the generated adventure plan to the terminal.

[0260] Output: Adventure plan sent to terminal

[0261] Step 3:

[0262] The start of a virtual adventure

[0263] The terminal displays the generated adventure plan to the user and provides the user with a "Start Adventure" button.

[0264] Input: Virtual adventure plan

[0265] The user presses the "Start Adventure" button.

[0266] Output: Instructions to start the adventure

[0267] The device displays the adventure's progress screen and provides instructions within the physical store.

[0268] Step 4:

[0269] Providing language learning and physical motor tasks

[0270] The server generates language learning tasks (e.g., quiz-style questions) and physical exercise tasks (e.g., stretching and walking) based on the virtual adventure plan and sends them to the terminal.

[0271] Input: Virtual adventure plan

[0272] Data processing: generating language learning tasks and physical movement tasks

[0273] Output: The generated tasks

[0274] The terminal displays the language learning task and provides an interface where the user can enter answers, as well as displaying instructions for the physical movement task.

[0275] Input: The generated task

[0276] Output: Task display screen

[0277] The user answers the tasks and performs the exercises.

[0278] Step 5:

[0279] Real-time feedback

[0280] The server monitors the user's progress and generates real-time feedback, using WebSockets for real-time communication.

[0281] Input: User's answer results and exercise progress

[0282] Data processing: generating feedback

[0283] Output: Real-time feedback

[0284] The terminal displays the received feedback to the user in real time.

[0285] Output: Display feedback

[0286] Step 6:

[0287] Adventure completion and evaluation

[0288] After the adventure is completed, the server generates an overall rating and proposes the next adventure plan. The rating is calculated based on the rating algorithm.

[0289] Input: All task progress data for the user

[0290] Data processing: Generating an overall rating

[0291] Output: Overall evaluation, new adventure plan

[0292] The terminal displays the overall rating to the user and offers options for the next adventure.

[0293] Output: Overall rating and next adventure plan

[0294] The user selects the next adventure or ends it.

[0295] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0296] This invention is a system for helping seniors learn languages ​​and improve their health while having fun, and it can improve the quality of the user experience by incorporating an emotion engine. The system collects user profile information and provides a virtual adventure plan based on that information. It combines language learning tasks with physical tasks to provide a fun and fulfilling experience for the user, and includes real-time feedback and emotion recognition adaptation.

[0297] Explaining program processing in natural language

[0298] 1. User Registration and Login

[0299] User: Opens the app and enters their name, age, email address, and password on the new registration screen.

[0300] Terminal: Collects the entered information and sends it to the server.

[0301] Server: Validates the received user information and stores it in the database if there are no problems. Returns a registration completion message to the terminal.

[0302] Device: Receives the registration completion message and notifies the user. The device transitions to the login screen.

[0303] 2. Profile Settings

[0304] User: After logging in, enter your language level, cultural interests, and exercise preferences in the profile settings screen.

[0305] Device: Collects profile information entered and sends it to the server.

[0306] Server: Receives the profile information, stores it in a database, and associates the profile information with the user account.

[0307] 3. Start your virtual adventure

[0308] Server: Runs an algorithm to generate an appropriate virtual adventure plan based on the user's profile information and sends the plan to the device.

[0309] Terminal: Displays the adventure start screen and provides a dialogue interface with the robot avatar.

[0310] User: Press the "Start Adventure" button to begin the virtual adventure.

[0311] 4. Combining language learning and exercise

[0312] Server: Creates scenarios for language learning tasks and motor tasks. Provides quiz-style questions for the language learning tasks.

[0313] Terminal: Provides an interface that displays language learning tasks and allows users to enter answers.

[0314] User: Answers the quiz and presses the submit button.

[0315] Terminal: Sends the quiz answers to the server.

[0316] Server: Determines whether the quiz is correct or incorrect and returns the result to the device.

[0317] Terminal: Displays the judgment results and instructions for the motor task.

[0318] User: Follow the displayed exercise instructions and perform physical exercise.

[0319] 5. Real-time feedback and emotion recognition

[0320] Server: Monitors the progress of language learning and motor tasks, generates real-time feedback, and uses an emotion engine to analyze the user's voice and facial expressions to obtain emotional data.

[0321] Terminal: Display additional emotion-based feedback to the user along with the feedback message.

[0322] User: Review the feedback and move on to the next task.

[0323] 6. Completing and evaluating the adventure, and proposing the next adventure plan

[0324] Server: Checks that all tasks are completed, evaluates the user's overall performance, and generates the next adventure plan taking into account the emotional data and sends it to the device.

[0325] Device: Displays an overall rating screen and offers the user options for their next adventure.

[0326] User: Check the overall rating, choose your next adventure plan, or exit the system.

[0327] Specific examples

[0328] Example: A 60-year-old user is interested in learning Japanese and Spanish culture.

[0329] User: Launch the app, register, and enter the required information. Then, on the profile settings screen, enter that you have "Intermediate Japanese" and are "Interested in Spanish culture."

[0330] Terminal: Sends this information to the server and receives matching adventure plans from the server.

[0331] Server: Generates and sends a virtual adventure plan for a trip to Spain based on the user's information.

[0332] User: The adventure begins and the user answers a Japanese quiz (e.g., "Please translate this sentence: Hola! ¿Cómo estás?").

[0333] Terminal: Sends the quiz results to the server and then displays the stretching exercise task.

[0334] Server: Evaluates the user's answers, monitors the progress of the motor task, and generates feedback. It also uses an emotion engine to analyze the user's facial expressions and voice to obtain emotional data.

[0335] On-device: Display feedback messages in real time and provide additional feedback based on sentiment.

[0336] User: Completes all tasks, receives final evaluation, and decides whether to select the next adventure plan.

[0337] In this way, users can learn languages ​​and improve their health through exercise while enjoying different cultures in a virtual environment. Furthermore, the emotional engine optimizes the user experience to match each individual's emotional state, helping seniors improve their cognitive function and maintain their health without feeling isolated.

[0338] The processing flow will be explained below.

[0339] Step 1:

[0340] User: Opens the app and enters their name, age, email address, and password on the new registration screen.

[0341] Step 2:

[0342] Terminal: Collects the entered information and sends it to the server.

[0343] Step 3:

[0344] Server: Validates the received user information and stores it in the database if there are no problems. Returns a registration completion message to the terminal.

[0345] Step 4:

[0346] Device: Receives the registration completion message and notifies the user. The device transitions to the login screen.

[0347] Step 5:

[0348] User: After logging in, enter your language level, cultural interests, and exercise preferences in the profile settings screen.

[0349] Step 6:

[0350] Device: Collects profile information entered and sends it to the server.

[0351] Step 7:

[0352] Server: Receives the profile information, stores it in a database, and associates the profile information with the user account.

[0353] Step 8:

[0354] Server: Runs an algorithm to generate an appropriate virtual adventure plan based on the user's profile information and sends the plan to the device.

[0355] Step 9:

[0356] Terminal: Displays the adventure start screen and provides a dialogue interface with the robot avatar.

[0357] Step 10:

[0358] User: Press the "Start Adventure" button to begin the virtual adventure.

[0359] Step 11:

[0360] Server: Creates scenarios for language learning tasks and motor tasks. Provides quiz-style questions for the language learning tasks.

[0361] Step 12:

[0362] Terminal: Provides an interface that displays language learning tasks and allows users to enter answers.

[0363] Step 13:

[0364] User: Answers the quiz and presses the submit button.

[0365] Step 14:

[0366] Terminal: Sends the quiz answers to the server.

[0367] Step 15:

[0368] Server: Determines whether the quiz is correct or incorrect and returns the result to the device.

[0369] Step 16:

[0370] Terminal: Displays the judgment result and then the motor task screen.

[0371] Step 17:

[0372] User: Follow the displayed exercise instructions and perform physical exercise.

[0373] Step 18:

[0374] Server: Monitors the progress of language learning and motor tasks, generates real-time feedback, and uses an emotion engine to analyze the user's voice and facial expressions to obtain emotional data.

[0375] Step 19:

[0376] Terminal: Display additional emotion-based feedback to the user along with the feedback message.

[0377] Step 20:

[0378] User: Review the feedback and move on to the next task.

[0379] Step 21:

[0380] Server: Checks that all tasks are completed, evaluates the user's overall performance, and generates the next adventure plan taking into account the emotional data and sends it to the device.

[0381] Step 22:

[0382] Device: Displays an overall rating screen and offers the user options for their next adventure.

[0383] Step 23:

[0384] User: Check the overall rating, choose your next adventure plan, or exit the system.

[0385] Example 2

[0386] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0387] In modern society, it is becoming increasingly important for older adults to maintain cognitive function through language learning and promote health through physical activity. However, traditional learning and exercise programs often lack enjoyment and make continued participation difficult. Furthermore, they lack approaches that take into account users' emotions and individual interests, resulting in a one-sided user experience. This increases the likelihood that older adults will feel isolated, preventing them from fully realizing the benefits of learning and exercise. Therefore, there is a need for a system that allows older adults to learn languages ​​and promote their health while having fun.

[0388] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting and storing user profile information, means for generating an appropriate virtual adventure plan based on the profile information, means for providing language learning tasks and exercise tasks based on the generated adventure plan, means for monitoring the progress of the language learning tasks and exercise tasks and providing feedback in real time, means for analyzing the user's voice and facial expressions using an emotion engine to obtain emotion data, and means for generating an overall evaluation and suggesting the next adventure plan when the adventure is completed. This enables elderly people to learn a language while having fun, promote their health, and provide an optimal user experience tailored to their individual emotions and interests.

[0389] "User" refers to the person who uses the system, and is often the elderly in particular.

[0390] "Profile information" is information that indicates a user's individual characteristics and preferences, such as their language learning level, cultural interests, and exercise preferences.

[0391] A "virtual adventure plan" is a plan containing learning and exercise scenarios in a virtual environment that is generated based on a user's profile information.

[0392] A "language learning task" refers to a learning activity that a user performs with the goal of acquiring a language, and includes quiz-style questions.

[0393] "Exercise task" refers to an exercise activity that a user performs to promote physical health, and is performed according to exercise instructions displayed on the screen.

[0394] "Progress monitoring" is the process by which the system monitors the user's progress in language learning and motor tasks in real time.

[0395] "Real-time feedback" refers to the immediate response and evaluation of a user's activities, helping them improve their learning and exercise.

[0396] "Emotion engine" refers to the technology and algorithms used to analyze a user's voice and facial expressions and obtain emotional data.

[0397] "Emotion data" is information about the user's emotional state obtained by the emotion engine.

[0398] "Overall Rating" refers to the overall rating of a user's performance generated by the system when the user completes an adventure task.

[0399] "Next adventure plan" refers to a new virtual adventure plan that the user should undertake next, which is generated based on the overall evaluation and emotional data.

[0400] This invention is a system for helping seniors learn languages ​​and improve their health while having fun, and it can improve the quality of the user experience by incorporating an emotion engine. The system collects user profile information and provides a virtual adventure plan based on that information. It combines language learning tasks with physical tasks to provide a fun and fulfilling experience for the user, and includes real-time feedback and emotion recognition adaptation.

[0401] User Registration and Login

[0402] A user launches the application using a device and enters their name, age, email address, and password on the new registration screen. The device collects this information and sends it to the server. The server verifies the received information and stores it in a database. Once registration is complete, the device displays a registration completion message and the user is taken to the login screen.

[0403] Profile Settings

[0404] After logging in, users enter their language level, cultural interests, and exercise preferences on the profile settings screen. The device collects this profile information and sends it to the server. The server verifies the information and stores it in a database. This associates the profile information with the user's account.

[0405] Start your virtual adventure

[0406] The server generates an appropriate virtual adventure plan using a generative AI model based on the user's profile information. This plan is sent to the device, which displays a start-of-adventure screen. The user presses the "Start Adventure" button to begin the virtual adventure. The adventure plan includes language learning tasks and physical tasks.

[0407] Combining language learning and exercise

[0408] The server generates language learning quizzes based on the user's progress and sends them to the device. The device displays the quizzes, and the user answers them. The answers are sent to the server, which judges whether they are correct or incorrect and returns the results to the device. The device displays the judgment results and then displays instructions for the exercise task. The user follows the exercise instructions and performs physical exercise.

[0409] Real-time feedback and emotion recognition

[0410] The server monitors the progress of language learning and motor tasks and generates real-time feedback. It also uses an emotion engine to analyze the user's voice and facial expressions to obtain emotional data. The device displays additional feedback based on the user's emotions along with the feedback message.

[0411] Completing and evaluating adventures, and proposing plans for the next adventure

[0412] The server confirms that all tasks have been completed and comprehensively evaluates the user's performance. It then generates the next adventure plan taking into account the emotional data and sends it to the device. The device then displays the comprehensive evaluation screen, allowing the user to select the next adventure plan or exit the system.

[0413] Specific examples

[0414] Example: A 60-year-old user is interested in learning Japanese and Spanish culture.

[0415] The user starts the app, registers, and enters the required information. Then, on the profile settings screen, the user enters that they have an intermediate level of Japanese and are interested in Spanish culture.

[0416] The terminal transmits this information to the server and receives matching adventure plans from the server.

[0417] The server generates and transmits a virtual adventure plan for a "trip to Spain" based on the user's information.

[0418] The user begins the adventure and answers Japanese quizzes (e.g., "Please translate this sentence: Hola! ¿Cómo estás?").

[0419] The terminal transmits the quiz results to the server, and then displays the stretching exercise task.

[0420] The server evaluates the user's answers, monitors the progress of the motor task, and generates feedback. It also uses an emotion engine to analyze the user's facial expressions and voice to obtain emotional data.

[0421] The device displays feedback messages in real time and provides additional feedback based on emotions.

[0422] The user completes all tasks, receives a final evaluation, and decides whether to select the next adventure plan.

[0423] The system allows users to learn languages ​​while enjoying different cultures and promote health through exercise. It also uses an emotion engine to optimize the user experience to match each individual's emotional state, helping seniors improve their cognitive function and maintain their health without feeling isolated.

[0424] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0425] Step 1: User enters registration information

[0426] How it works: A user enters information such as their name, age, email address, and password on the app's new registration screen.

[0427] Input: User-entered data (name, age, email address, password)

[0428] Output: User information entered into the terminal

[0429] Step 2: The device sends the user information to the server

[0430] How it works: The device collects the information entered by the user, converts it to JSON format, and sends it to the server via an HTTP POST request.

[0431] Input: User information entered into the terminal

[0432] Output: User information sent to the server

[0433] Step 3: The server validates the user information and stores it in the database

[0434] Operation: The server validates the data received based on the validation rules, and if there are no problems, saves it in the database. It also generates a registration completion message and returns it to the terminal.

[0435] Input: Submitted user information

[0436] Output: User information saved in the database, registration completion message

[0437] Step 4: Your device will display a registration complete message

[0438] Operation: The terminal receives a registration completion message from the server and displays a dialog box informing the user that registration is complete.

[0439] Input: Registration completion message

[0440] Output: Notification to the user that registration is complete

[0441] Step 5: User Enters Profile Information

[0442] How it works: After logging in, users enter their language level, cultural interests, and exercise preferences in the profile settings screen.

[0443] Input: User-entered data (language level, cultural interests, exercise preferences)

[0444] Output: Profile information entered on the device

[0445] Step 6: The device sends the profile information to the server

[0446] How it works: The device converts the entered profile information into JSON format and sends it to the server via an HTTP POST request.

[0447] Input: Profile information entered on the device

[0448] Output: Profile information sent to the server

[0449] Step 7: The server saves the profile information to a database

[0450] Operation: The server validates the received profile information, stores it in a database, and associates the profile information with the user account.

[0451] Input: Submitted profile information

[0452] Output: Profile information stored in a database

[0453] Step 8: The server generates a virtual adventure plan

[0454] How it works: The server uses a generative AI model to generate an appropriate virtual adventure plan based on the user's profile information. The plan is then sent to the device.

[0455] Input: User profile information

[0456] Output: Generated virtual adventure plan, submitted adventure plan

[0457] Step 9: The device displays the adventure plan, and the user begins the adventure.

[0458] Operation: The terminal displays the virtual adventure plan received from the server and provides an interface for the user to click a "Start Adventure" button.

[0459] Input: Received adventure plan

[0460] Output: Adventure start screen, user starts adventure

[0461] Step 10: Server generates language learning quiz

[0462] How it works: The server uses a generative AI model to create language learning quizzes based on the user's progress and sends them to the device.

[0463] Input: User progress information

[0464] Output: Generated language learning quiz, submitted quiz

[0465] Step 11: The device displays the quiz and the user answers it.

[0466] How it works: A device displays language learning quizzes and provides an interface for users to enter answers.

[0467] Input: Received language learning quiz

[0468] Output: Quiz displayed, answers given by the user

[0469] Step 12: The device sends the answer to the server

[0470] How it works: The device converts the user's answers into JSON format and sends them to the server via an HTTP POST request.

[0471] Input: User's answer

[0472] Output: The answer sent to the server

[0473] Step 13: The server determines whether the quiz is correct or not and returns the result to the device.

[0474] Operation: The server judges the correctness of the received answer and returns the result to the terminal.

[0475] Input: Submitted answer

[0476] Output: Quiz results, data returned to the device

[0477] Step 14: The device displays the judgment result and the exercise task.

[0478] Operation: The device receives the judgment result from the server and displays it to the user, and then displays instructions for the next exercise task.

[0479] Input: Judgment result

[0480] Output: Display of judgment results, display of motor task instructions

[0481] Step 15: The user performs the motor task

[0482] Action: The user follows the displayed exercise instructions and performs the physical exercise.

[0483] Input: Motor task instructions

[0484] Output: The motor task performed

[0485] Step 16: Server generates feedback and captures emotion data

[0486] Operation: The server monitors the progress of language learning and motor tasks, generates real-time feedback, and utilizes an emotion engine to analyze the user's voice and facial expressions to obtain emotional data.

[0487] Input: Progress data, voice data, facial expression data

[0488] Output: Generated feedback, captured emotion data

[0489] Step 17: The device displays a feedback message and additional emotional feedback

[0490] Behavior: The device displays a feedback message from the server and additional feedback based on the emotion.

[0491] Input: Feedback message, additional feedback based on sentiment data

[0492] Output: The feedback message that is displayed to the user

[0493] Step 18: The server generates an overall rating and proposes the next adventure plan.

[0494] How it works: The server verifies that all tasks have been completed, evaluates the user's overall performance, and generates a next adventure plan taking into account the emotional data and sends it to the device.

[0495] Input: Task progress data, emotion data

[0496] Output: Generated overall evaluation, proposal for next adventure plan

[0497] Step 19: Your device will display your overall rating and options for your next adventure.

[0498] Operation: The device displays the overall rating from the server and presents the user with options for their next adventure plan.

[0499] Input: Overall rating, next adventure plan

[0500] Output: Overall rating and next adventure options displayed to the user

[0501] Step 20: User selects next adventure plan or exits the system

[0502] Action: The user selects the next adventure plan or exits the system.

[0503] Input: Next adventure plan option

[0504] Output: Next adventure plan selected or system termination

[0505] (Application example 2)

[0506] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0507] Systems that enable elderly people to learn languages ​​and promote their health while having fun are required to enhance the user experience and provide optimal feedback and content according to each individual's emotional state. However, existing systems lack the ability to appropriately recognize the user's emotional state and adapt feedback and content accordingly, which limits the user experience.

[0508] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0509] In this invention, the server includes means for collecting and storing user profile information, means for generating an appropriate virtual adventure plan based on the profile information, means for providing language learning tasks and exercise tasks based on the generated adventure plan, means for analyzing the user's emotional state using an emotion engine and adapting feedback and content, means for monitoring the progress of the language learning tasks and exercise tasks and providing feedback in real time, and means for generating an overall evaluation and suggesting the next adventure plan when the adventure is completed. This enables optimal feedback and content to be provided according to the user's emotional state, allowing elderly people to learn languages ​​more enjoyably and improve their health.

[0510] "User profile information" refers to information about a user's individual characteristics, interests, and abilities, and specifically includes language learning level, cultural interests, exercise preferences, and emotional data.

[0511] A "virtual adventure plan" is an activity plan for a virtual space that is generated based on the user's profile information, and includes language learning tasks and physical activities.

[0512] A "language learning task" is a quiz-style question given to a user, and is a challenge for the user to improve their language ability through their answers.

[0513] An "exercise task" is a physical exercise performed by a user according to exercise instructions displayed on the screen.

[0514] An "emotion engine" is a technology that analyzes a user's voice and facial expressions to recognize their emotional state.

[0515] "Feedback" refers to providing real-time evaluations and reactions to the user's actions and answers, and assisting the user in their learning or athletic progress.

[0516] "Emotional state" refers to the emotion the user is currently feeling, and is analyzed from voice, facial expressions, etc.

[0517] The "overall evaluation" is an index that evaluates the overall results of the language learning task and the motor task and indicates the user's performance.

[0518] The "next adventure plan" is a new virtual adventure plan that is proposed after the current adventure plan is completed.

[0519] The present invention is an interactive system that supports users in language learning and health promotion, and can provide optimal feedback and content according to the user's emotional state. The system implements various functions, including collecting and storing user profile information, generating virtual adventure plans, providing language learning tasks and exercise tasks, monitoring progress in real time, analyzing emotions using an emotion engine, generating overall ratings, and suggesting the next adventure plan.

[0520] Explaining program processing in natural language

[0521] The server performs the following process: Collects and stores the user's profile information (language learning level, cultural interests, exercise preferences, and emotional data). Generates an appropriate virtual adventure plan based on the user's profile information. This virtual adventure plan includes a language learning task and an exercise task. The language learning task involves presenting quiz-style questions, and the user improves their language ability through their answers. The exercise task involves performing exercises based on the exercise instructions displayed on the screen.

[0522] The server monitors the progress of language learning and motor tasks and provides real-time feedback. It also uses an emotion engine to analyze the user's voice and facial expressions to recognize their emotional state and adapt the feedback and content accordingly. By providing optimal feedback based on their emotional state, users can learn languages ​​and improve their health while having more fun. When the adventure is complete, the server evaluates the user's overall performance and suggests a plan for the next adventure.

[0523] The system runs on devices such as smartphones and tablets, and the server manages user data using cloud services such as AWS, Google® Cloud, Firebase, and MongoDB. The emotion engine uses Microsoft® Azure® Emotion API and Google Cloud Vision API.

[0524] Adding specific examples

[0525] As a concrete example, the following system usage scenario can be considered: If a 60-year-old user is interested in learning Japanese and Spanish culture, he or she will use the system in the following steps.

[0526] 1. User: Launch the app, register, and enter the required information. Then, on the profile settings screen, enter "Intermediate Japanese" and "Interested in Spanish culture."

[0527] 2. Terminal: Sends this information to the server and receives matching adventure plans from the server.

[0528] 3. Server: Based on the user's information, it generates and sends a virtual adventure plan for a trip to Spain.

[0529] 4. User: The adventure begins and the user answers a Japanese quiz (e.g., "Please translate this sentence: Hola! ¿Cómo estás?").

[0530] 5. Terminal: Sends the quiz results to the server and then displays the stretching exercise task.

[0531] 6. Server: Evaluates the user's answers, monitors the progress of the motor task, generates feedback, and uses an emotion engine to analyze the user's facial expressions and voice to obtain emotional data.

[0532] 7. Terminal: Display feedback messages in real time and provide additional feedback based on emotions.

[0533] 8. User: Completes all tasks, receives final evaluation, and decides whether to select the next adventure plan.

[0534] Prompt Sentence Examples

[0535] Next, a specific example of a prompt sentence to be input to the generative AI model is shown below.

[0536] Invention: A system for elderly people to learn languages ​​and improve their health while having fun

[0537] Application example: Interactive health language learning app

[0538] User Story: Seniors use their smartphones to learn a new language and get some exercise, and emotional feedback helps them enjoy more content.

[0539] How the program works: A user registers and sets up a profile. They then embark on a virtual adventure tailored to their profile, completing language quizzes and physical tasks and receiving real-time feedback. The program also uses an emotion engine to collect emotional data, which is then reflected in the next adventure plan.

[0540] In this way, the present invention provides an optimal interactive system for users, even elderly people, to learn languages ​​and improve their health while having fun.

[0541] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0542] Step 1:

[0543] User Registration and Login

[0544] The user launches the app and enters their name, age, email address, and password on the new registration screen.

[0545] The terminal transmits the input information to the server.

[0546] The server verifies the received user information, and if there are no problems, stores it in a database and returns a registration completion message to the terminal.

[0547] Input: User information (name, age, email address, password)

[0548] Data processing / calculation: Checking the validity of information, saving to database

[0549] Output: Registration complete message

[0550] The terminal receives the registration completion message and notifies the user, and transitions to the login screen.

[0551] Step 2:

[0552] Profile Settings

[0553] After logging in, users enter their language level, cultural interests, exercise preferences, and emotional data on the profile setting screen.

[0554] The terminal transmits the entered profile information to the server.

[0555] The server receives the profile information, stores it in a database, and associates the profile information with a user account.

[0556] Input: Profile information (language level, cultural interests, exercise preferences, emotional data)

[0557] Data processing / calculation: Formatting profile data and saving to database

[0558] Output: Save confirmation message

[0559] Step 3:

[0560] Start your virtual adventure

[0561] The server runs an algorithm to generate an appropriate virtual adventure plan based on the user's profile information and transmits the plan to the terminal.

[0562] Input: User profile information

[0563] Data processing / calculation: Execution of adventure plan generation algorithm

[0564] Output: Virtual adventure plan

[0565] The terminal displays a screen for starting the adventure and provides an interface for interacting with the robot avatar.

[0566] The user presses the "Start Adventure" button to begin the virtual adventure.

[0567] Step 4:

[0568] Combining language learning and exercise

[0569] The server creates scenarios for language learning tasks and physical tasks, and presents quiz-style questions as language learning tasks.

[0570] Enter: Virtual Adventure Plan

[0571] Data processing / calculation: Creating scenarios for language learning tasks and motor tasks

[0572] Output: Verbal learning task and motor task

[0573] The terminal provides an interface that displays the language learning task and allows the user to enter answers.

[0574] The user answers the quiz and presses the send button.

[0575] The terminal transmits the answers to the quiz to the server.

[0576] The server judges whether the quiz is correct or incorrect and returns the result of the judgment to the terminal.

[0577] Input: User's quiz answer

[0578] Data processing / calculation: Answer determination

[0579] Output: Judgment result

[0580] The terminal displays the judgment result and displays instructions for the exercise task.

[0581] The user follows the displayed exercise instructions and performs physical exercise.

[0582] Step 5:

[0583] Real-time feedback and emotion recognition

[0584] The server monitors the progress of language learning and motor tasks, generates real-time feedback, and uses an emotion engine to analyze the user's voice and facial expressions to obtain emotional data.

[0585] Input: Progress data of language learning tasks and motor tasks, user voice and facial expression data

[0586] Data processing / calculation: Feedback generation, emotional state analysis

[0587] Output: Real-time feedback, emotional data

[0588] The terminal displays additional emotion-based feedback to the user along with the feedback message.

[0589] The user reviews the feedback and proceeds to the next task.

[0590] Step 6:

[0591] Completing and evaluating adventures, and proposing plans for the next adventure

[0592] The server confirms that all tasks have been completed, evaluates the user's overall performance, and generates a next adventure plan taking into account the emotional data and sends it to the device.

[0593] Input: Language learning task and motor task completion data, emotion data

[0594] Data processing / calculation: Comprehensive evaluation generation, next adventure plan generation

[0595] Output: Overall rating, next adventure plan

[0596] The terminal displays a comprehensive evaluation screen and provides the user with options for the next adventure.

[0597] The user checks the overall rating and selects the next adventure plan or exits the system.

[0598] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0599] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0600] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0601] [Second embodiment]

[0602] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0603] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0604] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0605] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0606] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0607] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0608] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0609] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0610] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0611] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0612] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0613] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0614] This invention is a system for helping seniors learn languages ​​and improve their health while having fun. The system collects user profile information and provides an optimal virtual adventure plan based on that information. It combines language learning tasks with physical activities to provide users with a fun and fulfilling experience.

[0615] Explaining program processing in natural language

[0616] 1. User Registration and Login

[0617] User: Opens the app and enters their name, age, email address, and password on the new registration screen.

[0618] Terminal: Send this information to the server and display a message indicating registration is complete.

[0619] Server: Validates the input information and stores it in a database. Once stored, it provides login authentication to the device.

[0620] 2. Profile Settings

[0621] User: After logging in, enter your language learning level, cultural interests, exercise preferences, etc. in the profile settings screen.

[0622] Terminal: Sends the entered information to the server and prepares for the next step.

[0623] Server: Stores profile information and prepares the next appropriate virtual adventure plan.

[0624] 3. Start your virtual adventure

[0625] Server: Runs an algorithm to generate an adventure plan based on the user's profile and sends the plan to the device.

[0626] Terminal: Displays the adventure start screen and provides a dialogue interface with the robot avatar.

[0627] User: Press the "Start Adventure" button to begin the virtual adventure.

[0628] 4. Combining language learning and exercise

[0629] Server: Generates language learning tasks (e.g., Japanese quizzes) and exercise tasks (e.g., stretching).

[0630] Terminal: displays language learning tasks and provides an interface for users to enter answers.

[0631] User: Completes a quiz and then performs a motor task.

[0632] Terminal: Sends quiz answers to the server and displays instructions for the exercise task.

[0633] Server: Determines the answers to the quiz and generates the next task based on the progress.

[0634] 5. Real-time feedback

[0635] Server: Monitors language learning and motor progress and provides real-time feedback.

[0636] Terminal: Shows feedback messages and notifies the user of the next task.

[0637] User: Receive feedback and move on to the next task.

[0638] 6. Adventure Completion and Evaluation

[0639] Server: Once the adventure is completed, generate an overall evaluation and propose a plan for the next adventure.

[0640] Device: Displays the overall rating screen and presents the user with options for their next adventure.

[0641] User: Checks the rating and decides whether to select the next adventure or quit.

[0642] Specific examples

[0643] Example: A 60-year-old user is interested in learning Japanese and Spanish culture.

[0644] User: Launch the app, register, and enter the required information. Then, on the profile settings screen, enter that you have "Intermediate Japanese" and are "Interested in Spanish culture."

[0645] Terminal: Sends this information to the server and receives matching adventure plans from the server.

[0646] Server: Generates and sends a virtual adventure plan for a trip to Spain based on the user's information.

[0647] User: The adventure begins and the user answers a Japanese quiz (e.g., "Please translate this sentence: Hola! ¿Cómo estás?").

[0648] Terminal: Sends the quiz results to the server and then displays the stretching exercise task.

[0649] Server: Evaluates the user's answers, monitors the progress of the motor task, and generates feedback.

[0650] On the device: Provides real-time feedback and suggests next steps to the user.

[0651] User: Completes all tasks, receives final evaluation, and decides whether to select the next adventure plan.

[0652] In this way, users can learn languages ​​and promote health through exercise while enjoying different cultures in a virtual environment. This system helps seniors improve their cognitive function and maintain their health without feeling isolated.

[0653] The processing flow will be explained below.

[0654] Step 1:

[0655] User: Opens the app and enters their name, age, email address, and password on the new registration screen.

[0656] Terminal: Collects the entered information and sends it to the server.

[0657] Server: Validates the received user information and stores it in the database if there are no problems. Returns a registration completion message to the terminal.

[0658] Device: Receives the registration completion message and notifies the user. The device transitions to the login screen.

[0659] Step 2:

[0660] User: After logging in, enter your language level, cultural interests, and exercise preferences in the profile settings screen.

[0661] Device: Collects profile information entered and sends it to the server.

[0662] Server: Receives the profile information, stores it in a database, and associates the profile information with the user account.

[0663] Step 3:

[0664] Server: Generates an appropriate virtual adventure plan based on the user's profile information. Sends the generated adventure plan to the device.

[0665] Terminal: Displays the adventure start screen and provides a dialogue interface with the robot avatar.

[0666] User: Press the "Start Adventure" button to begin the virtual adventure.

[0667] Step 4:

[0668] Server: Creates a scenario for generating language learning tasks and exercise tasks. For the language learning tasks, it presents quiz-style questions to the user.

[0669] Terminal: Provides an interface that displays language learning tasks and allows users to enter answers.

[0670] User: Answers the quiz and presses the submit button.

[0671] Terminal: Sends the quiz answers to the server.

[0672] Server: Determines whether the quiz is correct or incorrect and returns the result to the device.

[0673] Terminal: Displays the judgment result and then the motor task screen.

[0674] User: Follow the exercise instructions displayed on the screen and perform physical exercise.

[0675] Step 5:

[0676] Server: Monitors the user's progress in language learning and motor tasks and generates feedback in real time.

[0677] Terminal: Display a feedback message to the user, asking if the user is ready to proceed to the next task.

[0678] User: Review the feedback and move on to the next task.

[0679] Step 6:

[0680] Server: Checks that all tasks are completed, evaluates the user's performance, and generates the next adventure plan and sends it to the device.

[0681] Device: Displays an overall rating screen and offers the user options for their next adventure.

[0682] User: Check the overall rating and decide whether to select the next adventure plan or exit the system.

[0683] This series of steps provides a system that allows elderly people to have fun, learn about different cultures and languages, and get appropriate exercise at the same time.

[0684] Example 1

[0685] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0686] There is a need for a system that allows elderly people to learn languages ​​and promote their health while having fun. However, many elderly people find that current learning methods lack enjoyment and continuity. Therefore, there is an urgent need to develop a system that allows elderly people to continue enjoying activities without straining themselves. There is also a need for a system that can efficiently combine language learning and exercise. Furthermore, there is a need for a system that can provide an optimized virtual adventure plan for each individual user and provide feedback in real time.

[0687] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0688] In this invention, the server includes means for collecting and storing user attribute information, means for generating an appropriate virtual adventure plan based on the attribute information, and means for providing language learning tasks and exercise tasks based on the generated adventure plan. This allows the user to enjoy learning a language and exercising through an individually optimized adventure plan. The server also includes means for monitoring the progress of the language learning tasks and exercise tasks and providing immediate feedback, and means for generating a comprehensive evaluation and proposing the next adventure plan upon completion of the adventure. This allows the user to check their progress in real time and proceed to the next step. This also allows elderly people to continue enjoying learning and exercise without straining themselves.

[0689] "Attribute information" refers to personal data about the user, such as age, interests, language learning level, and type of exercise.

[0690] A "virtual adventure plan" refers to a series of adventures or tasks to be performed in a virtual environment that are generated based on the user's attribute information.

[0691] A "language learning task" refers to an activity in which a user solves a specified language problem or challenge.

[0692] "Exercise task" refers to an activity that includes instructions for a movement or exercise that a user should perform.

[0693] "Immediate feedback" refers to providing real-time evaluation and advice on the results of a user's language learning or motor task.

[0694] "Overall evaluation" refers to a comprehensive evaluation of the results of all language learning tasks and motor tasks performed by the user.

[0695] "When the adventure is complete" refers to the point at which the user has completed all assigned language learning and physical tasks.

[0696] A "virtual adventure plan" refers to an adventurous activity or experience in a virtual environment that is individually generated based on the user's attribute information.

[0697] This invention is a system for elderly people to learn languages ​​and improve their health while having fun. The system collects user attribute information and provides an optimal virtual adventure plan based on that information. It combines language learning tasks with physical activities to provide users with a fun and fulfilling experience.

[0698] A specific embodiment of this system is as follows.

[0699] 1. Hardware and Software

[0700] server:

[0701] Hardware: A high-performance server (e.g., Dell PowerEdge series).

[0702] Software: Database management system (e.g., MySQL), programming language to execute the algorithms (e.g., Python).

[0703] Device:

[0704] Hardware: Tablet, smartphone, or PC.

[0705] Software: Dedicated application (e.g., iOS or Android app), browser-based interface.

[0706] User:

[0707] Hardware: Devices operated by users, such as tablets and smartphones.

[0708] Software: The dedicated application or browser-based interface.

[0709] 2. Data processing and calculation

[0710] User Registration and Login:

[0711] The user opens the app and enters their name, age, email address, and password on the new registration screen. The device sends this information to the server, which verifies the data and stores it in a database. When logging in, the server verifies the email address and password entered by the user, generates an authentication token, and sends it to the device.

[0712] Profile Settings:

[0713] After logging in, users enter their language learning level, cultural interests, exercise preferences, etc. on the profile setting screen. The device sends the entered information to the server, which then stores it in a database.

[0714] Create and launch a virtual adventure:

[0715] The server uses an algorithm to generate a virtual adventure plan based on the stored user attribute information. The generated plan is sent to the terminal, and the user presses the "Start Adventure" button to begin the virtual adventure.

[0716] Verbal learning and motor task performance:

[0717] The server generates language learning tasks (e.g., "Please translate this phrase") and exercise tasks (e.g., "Stretch for 5 minutes") based on the user's progress. The device presents these to the user and sends the user's answers and exercise progress to the server. The server evaluates the answers and generates the next task.

[0718] Real-time feedback:

[0719] The server monitors the user's progress in real time and generates and sends feedback to the device, allowing the user to move on to the next task.

[0720] Adventure completion and rating:

[0721] When the adventure is completed, the server generates a comprehensive evaluation and proposes the next adventure plan. The terminal presents the evaluation to the user, who then decides whether to select the next adventure.

[0722] 3. Examples and prompts

[0723] Examples:

[0724] If a 60-year-old user is interested in learning Japanese and French culture, the user launches the app, registers, and enters the necessary information. Then, on the profile settings screen, the user enters "Intermediate Japanese" and "Interested in French culture." Based on this information, the server generates a virtual adventure plan for a "trip to France" and sends it to the device. The user begins the adventure, answers a Japanese quiz (e.g., "Translate this sentence: Bonjour!"), and then performs an exercise task (e.g., "Stretch for 5 minutes").

[0725] Example prompt sentence:

[0726] Here is an example of a prompt to provide more details to a generative AI model:

[0727] Describe the specific steps of a system that helps elderly people improve their health while learning a language. The system's process consists of the following steps: user registration and login, profile setup, adventure plan generation and initiation, language learning and exercise task execution, real-time feedback, and adventure completion and evaluation. For each step, provide a detailed explanation of how the server, device, and user operate.

[0728] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0729] Step 1:

[0730] User Registration and Login

[0731] input:

[0732] The user opens the app and enters their name, age, email address, and password on the new registration screen.

[0733] Data processing and calculation:

[0734] The device sends the entered information to the server, which checks the integrity of the data and stores it in a database if there are no problems. When logging in, the entered email address and password are compared with the database, and if they match, an authentication token is generated.

[0735] output:

[0736] A registration completion message is displayed and an authentication token is returned.

[0737] Specific behavior:

[0738] The user enters the name "tanaka60", age "60", email address "tanaka60@example.com", and password "password123", and the device sends this to the server. The server stores it in a database, generates an authentication token, and sends it to the device. The device displays "Registration completed" to the user.

[0739] Step 2:

[0740] Profile Settings

[0741] input:

[0742] After logging in, users enter their language learning level, cultural interests, and exercise preferences on the profile settings screen.

[0743] Data processing and calculation:

[0744] The terminal transmits the input information to a server, which then stores the attribute information in a database.

[0745] output:

[0746] A save completion message will be displayed and preparations for the next step will be made.

[0747] Specific behavior:

[0748] The user selects "Japanese Beginner's," "French Culture," and "Yoga," and the device sends this to the server. The server saves this and displays the message "Profile saved."

[0749] Step 3:

[0750] Creating and launching a virtual adventure

[0751] input:

[0752] The server generates a virtual adventure plan based on the stored profile information.

[0753] Data processing and calculation:

[0754] The server uses an algorithm to generate a virtual adventure plan and transmits it to the terminal.

[0755] output:

[0756] The generated adventure plan is displayed and a "Start Adventure" button is presented to the user.

[0757] Specific behavior:

[0758] The server generates an adventure plan based on the theme of "Travel to France," combining Japanese conversation quizzes and yoga exercise tasks, and sends it to the terminal. The terminal then displays "Do you want to start your adventure traveling to France?" and presents the user with a "Start Adventure" button.

[0759] Step 4:

[0760] Verbal learning and motor task performance

[0761] input:

[0762] The server generates language learning tasks and motor tasks and sends them to the terminal.

[0763] Data processing and calculation:

[0764] The server generates questions, receives answers, and evaluates the user's progress, then generates instructions for the motor task.

[0765] output:

[0766] The language learning task and the motor task are displayed on the screen, and the user's answers and motor execution data are transmitted.

[0767] Specific behavior:

[0768] The server generates a quiz question, "Bonjour! Please translate this phrase into Japanese," and sends it to the device. The user answers, "Hello!", and the device sends this to the server. The server evaluates the answer, then generates an exercise task, "Please stretch for 5 minutes," and sends it to the device. The user performs the exercise and presses the "Task Complete" button.

[0769] Step 5:

[0770] Real-time feedback and next steps

[0771] input:

[0772] Receive the user's answers and exercise progress data.

[0773] Data processing and calculation:

[0774] The server evaluates the answers and generates feedback to send to the user.

[0775] output:

[0776] Get real-time feedback messages and next task notifications.

[0777] Specific behavior:

[0778] The server generates a message saying "Great, you got it right!" and prepares to present the next quiz or exercise task. The device displays a feedback message and offers the user a "Proceed to next task" button.

[0779] Step 6:

[0780] Adventure completion and evaluation

[0781] input:

[0782] Collect data when all tasks are completed.

[0783] Data processing and calculation:

[0784] The server generates an overall rating and suggests the next adventure plan.

[0785] output:

[0786] An overall evaluation screen will be displayed, and options for the next adventure will be presented.

[0787] Specific behavior:

[0788] The server evaluates the user's overall performance and generates a message saying, "Adventure successful! We recommend a trip to Italy as your next adventure." The device displays the evaluation results along with the next options, providing the user with an interface to select "Start the next adventure" or "End."

[0789] (Application example 1)

[0790] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0791] This invention relates to a system for helping seniors learn languages ​​and promote their health while having fun. Specifically, the system aims to reduce loneliness and improve cognitive function and health by providing a virtual adventure that combines language learning and physical exercise through a realistic experience within a brick-and-mortar store. It is also required to provide real-time feedback and closely monitor the user's progress.

[0792] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0793] In this invention, the server includes means for collecting and storing personal information of the user, means for generating an appropriate virtual adventure plan based on the personal information, means for providing language learning tasks and physical exercise tasks based on the generated adventure plan, means for monitoring the progress of the language learning tasks and physical exercise tasks and providing feedback in real time, means for generating an overall evaluation and proposing a next adventure plan when the adventure is completed, means for providing an adventure scenario within the physical store, means for performing the language learning tasks and physical exercise tasks at specified locations while walking within the physical store, and means for providing real-time evaluation and feedback. This enables the user to improve their health while having fun learning a language through a realistic experience within the physical store.

[0794] "User personal information" refers to information such as the user's name, age, language learning level, cultural interests, and physical activity preferences.

[0795] "Virtual adventure plan" refers to a plan that includes language learning tasks and physical exercise tasks, generated based on the user's personal information.

[0796] A "language learning task" refers to the task of quiz-style questions posed to the user and judging their answers.

[0797] "Physical exercise task" refers to an exercise performed by a user based on exercise instructions displayed on a screen.

[0798] "Brick and mortar store" refers to a retail store or facility that a user can actually visit.

[0799] A "virtual adventure scenario" is a scenario that a user experiences in a physical store, and refers to content that is constructed based on a virtual adventure plan.

[0800] "Real-time feedback" refers to assessment and instruction provided in real time on a user's progress as they perform a language learning task or physical movement task.

[0801] "Overall Grade" refers to the overall grade or rating provided upon completing a series of tasks in a virtual adventure.

[0802] The system embodying this invention collects personal information from users, creates a virtual adventure plan, and provides language learning tasks and physical exercise tasks to promote language learning and health for the elderly. Specific embodiments are described below.

[0803] 1. User Registration and Profile Settings

[0804] The server collects personal information entered by the user (such as name, age, language learning level, cultural interests, physical activity preferences, etc.) and stores it in a database. Django is a suitable server-side framework for this.

[0805] The device sends the information entered by the user to the server via the interface and prepares to proceed to the next step. The device can be a smartphone or tablet, and the application is implemented using React Native.

[0806] 2. Creating and launching a virtual adventure plan

[0807] The server generates a virtual adventure plan based on the collected personal information using Python scripts and serverless architectures such as AWS Lambda, and automatically creates a plan that includes language learning tasks and physical exercise tasks tailored to the user's interests and level.

[0808] The device displays the generated adventure plan to the user and prompts them to start the adventure. An adventure scenario in a physical store is displayed, and when the user presses the "Start Adventure" button, the actual task begins.

[0809] 3. Providing language learning and physical motor tasks

[0810] The server creates language learning tasks (e.g., quiz-style questions) and physical exercise tasks (e.g., stretching and walking) and sends them to the terminal. PostgreSQL is often used as the database.

[0811] The terminal displays the language learning tasks and provides an interface where the user can enter answers, and displays instructions for the physical motor tasks so that the user can perform them.

[0812] 4. Real-time feedback

[0813] The server monitors the progress of language learning tasks and physical motor tasks and generates real-time feedback using real-time communication over WebSockets.

[0814] The terminal displays the feedback messages received from the server in real time and notifies the user of the next task.

[0815] 5. Adventure Completion and Evaluation

[0816] Once the adventure is completed, the server generates an overall evaluation and proposes a plan for the next virtual adventure. Based on the evaluation algorithm, the evaluation is calculated based on the user's performance and progress.

[0817] The terminal displays the overall rating and presents the user with options for the next adventure.

[0818] Specific examples

[0819] Example: If a 60-year-old user is interested in learning Japanese and Spanish culture, the user opens the app, enters their personal information, and then enters "Intermediate Japanese" and "Interested in Spanish culture" in the profile settings screen. The user is then presented with a virtual adventure scenario set in a physical store, where they answer Japanese language quizzes and perform stretching exercises in designated areas. The device displays real-time feedback, allowing the user to track their progress as they progress through the adventure.

[0820] Prompt Sentence Examples

[0821] "I want to design a system that helps seniors learn languages ​​and promote their health while having fun. This system collects users' personal information and then provides them with an optimal virtual adventure plan based on that information. The application combines language learning tasks with physical exercise tasks to provide a fun and fulfilling experience. For example, if a 65-year-old user is interested in French culture, the app can have them complete quizzes about French tourist attractions and related stretching exercises, and provide real-time feedback. Please tell me more about how you will implement this system and the technologies you will use."

[0822] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0823] Step 1:

[0824] User registration and profile settings

[0825] Users open a smartphone app and enter personal information such as their name, age, language learning level, cultural interests, and physical activity preferences.

[0826] Input: Name, age, language level, cultural interests, physical activity preferences

[0827] The terminal transmits the input information to the server.

[0828] Input: User personal information

[0829] The server stores the received information in a database. The information is managed using the Django framework.

[0830] Output: User's personal information stored in a database

[0831] The server sends a message to the terminal confirming that the save has been completed.

[0832] Output: Save complete confirmation message

[0833] Step 2:

[0834] Virtual adventure planning

[0835] The server generates a virtual adventure plan based on the user's personal information. The algorithm used is implemented in Python and processed in a scalable manner using AWS Lambda.

[0836] Input: User personal information

[0837] Data processing: Data calculations to generate optimal adventure plans based on the user's interests and learning level

[0838] Output: Virtual adventure plan

[0839] The server transmits the generated adventure plan to the terminal.

[0840] Output: Adventure plan sent to terminal

[0841] Step 3:

[0842] The start of a virtual adventure

[0843] The terminal displays the generated adventure plan to the user and provides the user with a "Start Adventure" button.

[0844] Input: Virtual adventure plan

[0845] The user presses the "Start Adventure" button.

[0846] Output: Instructions to start the adventure

[0847] The device displays the adventure's progress screen and provides instructions within the physical store.

[0848] Step 4:

[0849] Providing language learning and physical motor tasks

[0850] The server generates language learning tasks (e.g., quiz-style questions) and physical exercise tasks (e.g., stretching and walking) based on the virtual adventure plan and sends them to the terminal.

[0851] Input: Virtual adventure plan

[0852] Data processing: generating language learning tasks and physical movement tasks

[0853] Output: The generated tasks

[0854] The terminal displays the language learning task and provides an interface where the user can enter answers, as well as displaying instructions for the physical movement task.

[0855] Input: The generated task

[0856] Output: Task display screen

[0857] The user answers the tasks and performs the exercises.

[0858] Step 5:

[0859] Real-time feedback

[0860] The server monitors the user's progress and generates real-time feedback, using WebSockets for real-time communication.

[0861] Input: User's answer results and exercise progress

[0862] Data processing: generating feedback

[0863] Output: Real-time feedback

[0864] The terminal displays the received feedback to the user in real time.

[0865] Output: Display feedback

[0866] Step 6:

[0867] Adventure completion and evaluation

[0868] After the adventure is completed, the server generates an overall rating and proposes the next adventure plan. The rating is calculated based on the rating algorithm.

[0869] Input: All task progress data for the user

[0870] Data processing: Generating an overall rating

[0871] Output: Overall evaluation, new adventure plan

[0872] The terminal displays the overall rating to the user and offers options for the next adventure.

[0873] Output: Overall rating and next adventure plan

[0874] The user selects the next adventure or ends it.

[0875] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0876] This invention is a system for helping seniors learn languages ​​and improve their health while having fun, and it can improve the quality of the user experience by incorporating an emotion engine. The system collects user profile information and provides a virtual adventure plan based on that information. It combines language learning tasks with physical tasks to provide a fun and fulfilling experience for the user, and includes real-time feedback and emotion recognition adaptation.

[0877] Explaining program processing in natural language

[0878] 1. User Registration and Login

[0879] User: Opens the app and enters their name, age, email address, and password on the new registration screen.

[0880] Terminal: Collects the entered information and sends it to the server.

[0881] Server: Validates the received user information and stores it in the database if there are no problems. Returns a registration completion message to the terminal.

[0882] Device: Receives the registration completion message and notifies the user. The device transitions to the login screen.

[0883] 2. Profile Settings

[0884] User: After logging in, enter your language level, cultural interests, and exercise preferences in the profile settings screen.

[0885] Device: Collects profile information entered and sends it to the server.

[0886] Server: Receives the profile information, stores it in a database, and associates the profile information with the user account.

[0887] 3. Start your virtual adventure

[0888] Server: Runs an algorithm to generate an appropriate virtual adventure plan based on the user's profile information and sends the plan to the device.

[0889] Terminal: Displays the adventure start screen and provides a dialogue interface with the robot avatar.

[0890] User: Press the "Start Adventure" button to begin the virtual adventure.

[0891] 4. Combining language learning and exercise

[0892] Server: Creates scenarios for language learning tasks and motor tasks. Provides quiz-style questions for the language learning tasks.

[0893] Terminal: Provides an interface that displays language learning tasks and allows users to enter answers.

[0894] User: Answers the quiz and presses the submit button.

[0895] Terminal: Sends the quiz answers to the server.

[0896] Server: Determines whether the quiz is correct or incorrect and returns the result to the device.

[0897] Terminal: Displays the judgment results and instructions for the motor task.

[0898] User: Follow the displayed exercise instructions and perform physical exercise.

[0899] 5. Real-time feedback and emotion recognition

[0900] Server: Monitors the progress of language learning and motor tasks, generates real-time feedback, and uses an emotion engine to analyze the user's voice and facial expressions to obtain emotional data.

[0901] Terminal: Display additional emotion-based feedback to the user along with the feedback message.

[0902] User: Review the feedback and move on to the next task.

[0903] 6. Completing and evaluating the adventure, and proposing the next adventure plan

[0904] Server: Checks that all tasks are completed, evaluates the user's overall performance, and generates the next adventure plan taking into account the emotional data and sends it to the device.

[0905] Device: Displays an overall rating screen and offers the user options for their next adventure.

[0906] User: Check the overall rating, choose your next adventure plan, or exit the system.

[0907] Specific examples

[0908] Example: A 60-year-old user is interested in learning Japanese and Spanish culture.

[0909] User: Launch the app, register, and enter the required information. Then, on the profile settings screen, enter that you have "Intermediate Japanese" and are "Interested in Spanish culture."

[0910] Terminal: Sends this information to the server and receives matching adventure plans from the server.

[0911] Server: Generates and sends a virtual adventure plan for a trip to Spain based on the user's information.

[0912] User: The adventure begins and the user answers a Japanese quiz (e.g., "Please translate this sentence: Hola! ¿Cómo estás?").

[0913] Terminal: Sends the quiz results to the server and then displays the stretching exercise task.

[0914] Server: Evaluates the user's answers, monitors the progress of the motor task, and generates feedback. It also uses an emotion engine to analyze the user's facial expressions and voice to obtain emotional data.

[0915] On-device: Display feedback messages in real time and provide additional feedback based on sentiment.

[0916] User: Completes all tasks, receives final evaluation, and decides whether to select the next adventure plan.

[0917] In this way, users can learn languages ​​and improve their health through exercise while enjoying different cultures in a virtual environment. Furthermore, the emotional engine optimizes the user experience to match each individual's emotional state, helping seniors improve their cognitive function and maintain their health without feeling isolated.

[0918] The processing flow will be explained below.

[0919] Step 1:

[0920] User: Opens the app and enters their name, age, email address, and password on the new registration screen.

[0921] Step 2:

[0922] Terminal: Collects the entered information and sends it to the server.

[0923] Step 3:

[0924] Server: Validates the received user information and stores it in the database if there are no problems. Returns a registration completion message to the terminal.

[0925] Step 4:

[0926] Device: Receives the registration completion message and notifies the user. The device transitions to the login screen.

[0927] Step 5:

[0928] User: After logging in, enter your language level, cultural interests, and exercise preferences in the profile settings screen.

[0929] Step 6:

[0930] Device: Collects profile information entered and sends it to the server.

[0931] Step 7:

[0932] Server: Receives the profile information, stores it in a database, and associates the profile information with the user account.

[0933] Step 8:

[0934] Server: Runs an algorithm to generate an appropriate virtual adventure plan based on the user's profile information and sends the plan to the device.

[0935] Step 9:

[0936] Terminal: Displays the adventure start screen and provides a dialogue interface with the robot avatar.

[0937] Step 10:

[0938] User: Press the "Start Adventure" button to begin the virtual adventure.

[0939] Step 11:

[0940] Server: Creates scenarios for language learning tasks and motor tasks. Provides quiz-style questions for the language learning tasks.

[0941] Step 12:

[0942] Terminal: Provides an interface that displays language learning tasks and allows users to enter answers.

[0943] Step 13:

[0944] User: Answers the quiz and presses the submit button.

[0945] Step 14:

[0946] Terminal: Sends the quiz answers to the server.

[0947] Step 15:

[0948] Server: Determines whether the quiz is correct or incorrect and returns the result to the device.

[0949] Step 16:

[0950] Terminal: Displays the judgment result and then the motor task screen.

[0951] Step 17:

[0952] User: Follow the displayed exercise instructions and perform physical exercise.

[0953] Step 18:

[0954] Server: Monitors the progress of language learning and motor tasks, generates real-time feedback, and uses an emotion engine to analyze the user's voice and facial expressions to obtain emotional data.

[0955] Step 19:

[0956] Terminal: Display additional emotion-based feedback to the user along with the feedback message.

[0957] Step 20:

[0958] User: Review the feedback and move on to the next task.

[0959] Step 21:

[0960] Server: Checks that all tasks are completed, evaluates the user's overall performance, and generates the next adventure plan taking into account the emotional data and sends it to the device.

[0961] Step 22:

[0962] Device: Displays an overall rating screen and offers the user options for their next adventure.

[0963] Step 23:

[0964] User: Check the overall rating, choose your next adventure plan, or exit the system.

[0965] Example 2

[0966] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0967] In modern society, it is becoming increasingly important for older adults to maintain cognitive function through language learning and promote health through physical activity. However, traditional learning and exercise programs often lack enjoyment and make continued participation difficult. Furthermore, they lack approaches that take into account users' emotions and individual interests, resulting in a one-sided user experience. This increases the likelihood that older adults will feel isolated, preventing them from fully realizing the benefits of learning and exercise. Therefore, there is a need for a system that allows older adults to learn languages ​​and promote their health while having fun.

[0968] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting and storing user profile information, means for generating an appropriate virtual adventure plan based on the profile information, means for providing language learning tasks and exercise tasks based on the generated adventure plan, means for monitoring the progress of the language learning tasks and exercise tasks and providing feedback in real time, means for analyzing the user's voice and facial expressions using an emotion engine to obtain emotion data, and means for generating an overall evaluation and suggesting the next adventure plan when the adventure is completed. This enables elderly people to learn a language while having fun, promote their health, and provide an optimal user experience tailored to their individual emotions and interests.

[0969] "User" refers to the person who uses the system, and is often the elderly in particular.

[0970] "Profile information" is information that indicates a user's individual characteristics and preferences, such as their language learning level, cultural interests, and exercise preferences.

[0971] A "virtual adventure plan" is a plan containing learning and exercise scenarios in a virtual environment that is generated based on a user's profile information.

[0972] A "language learning task" refers to a learning activity that a user performs with the goal of acquiring a language, and includes quiz-style questions.

[0973] "Exercise task" refers to an exercise activity that a user performs to promote physical health, and is performed according to exercise instructions displayed on the screen.

[0974] "Progress monitoring" is the process by which the system monitors the user's progress in language learning and motor tasks in real time.

[0975] "Real-time feedback" refers to the immediate response and evaluation of a user's activities, helping them improve their learning and exercise.

[0976] "Emotion engine" refers to the technology and algorithms used to analyze a user's voice and facial expressions and obtain emotional data.

[0977] "Emotion data" is information about the user's emotional state obtained by the emotion engine.

[0978] "Overall Rating" refers to the overall rating of a user's performance generated by the system when the user completes an adventure task.

[0979] "Next adventure plan" refers to a new virtual adventure plan that the user should undertake next, which is generated based on the overall evaluation and emotional data.

[0980] This invention is a system for helping seniors learn languages ​​and improve their health while having fun, and it can improve the quality of the user experience by incorporating an emotion engine. The system collects user profile information and provides a virtual adventure plan based on that information. It combines language learning tasks with physical tasks to provide a fun and fulfilling experience for the user, and includes real-time feedback and emotion recognition adaptation.

[0981] User Registration and Login

[0982] A user launches the application using a device and enters their name, age, email address, and password on the new registration screen. The device collects this information and sends it to the server. The server verifies the received information and stores it in a database. Once registration is complete, the device displays a registration completion message and the user is taken to the login screen.

[0983] Profile Settings

[0984] After logging in, users enter their language level, cultural interests, and exercise preferences on the profile settings screen. The device collects this profile information and sends it to the server. The server verifies the information and stores it in a database. This associates the profile information with the user's account.

[0985] Start your virtual adventure

[0986] The server generates an appropriate virtual adventure plan using a generative AI model based on the user's profile information. This plan is sent to the device, which displays a start-of-adventure screen. The user presses the "Start Adventure" button to begin the virtual adventure. The adventure plan includes language learning tasks and physical tasks.

[0987] Combining language learning and exercise

[0988] The server generates language learning quizzes based on the user's progress and sends them to the device. The device displays the quizzes, and the user answers them. The answers are sent to the server, which judges whether they are correct or incorrect and returns the results to the device. The device displays the judgment results and then displays instructions for the exercise task. The user follows the exercise instructions and performs physical exercise.

[0989] Real-time feedback and emotion recognition

[0990] The server monitors the progress of language learning and motor tasks and generates real-time feedback. It also uses an emotion engine to analyze the user's voice and facial expressions to obtain emotional data. The device displays additional feedback based on the user's emotions along with the feedback message.

[0991] Completing and evaluating adventures, and proposing plans for the next adventure

[0992] The server confirms that all tasks have been completed and comprehensively evaluates the user's performance. It then generates the next adventure plan taking into account the emotional data and sends it to the device. The device then displays the comprehensive evaluation screen, allowing the user to select the next adventure plan or exit the system.

[0993] Specific examples

[0994] Example: A 60-year-old user is interested in learning Japanese and Spanish culture.

[0995] The user starts the app, registers, and enters the required information. Then, on the profile settings screen, the user enters that they have an intermediate level of Japanese and are interested in Spanish culture.

[0996] The terminal transmits this information to the server and receives matching adventure plans from the server.

[0997] The server generates and transmits a virtual adventure plan for a "trip to Spain" based on the user's information.

[0998] The user begins the adventure and answers Japanese quizzes (e.g., "Please translate this sentence: Hola! ¿Cómo estás?").

[0999] The terminal transmits the quiz results to the server, and then displays the stretching exercise task.

[1000] The server evaluates the user's answers, monitors the progress of the motor task, and generates feedback. It also uses an emotion engine to analyze the user's facial expressions and voice to obtain emotional data.

[1001] The device displays feedback messages in real time and provides additional feedback based on emotions.

[1002] The user completes all tasks, receives a final evaluation, and decides whether to select the next adventure plan.

[1003] The system allows users to learn languages ​​while enjoying different cultures and promote health through exercise. It also uses an emotion engine to optimize the user experience to match each individual's emotional state, helping seniors improve their cognitive function and maintain their health without feeling isolated.

[1004] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1005] Step 1: User enters registration information

[1006] How it works: A user enters information such as their name, age, email address, and password on the app's new registration screen.

[1007] Input: User-entered data (name, age, email address, password)

[1008] Output: User information entered into the terminal

[1009] Step 2: The device sends the user information to the server

[1010] How it works: The device collects the information entered by the user, converts it to JSON format, and sends it to the server via an HTTP POST request.

[1011] Input: User information entered into the terminal

[1012] Output: User information sent to the server

[1013] Step 3: The server validates the user information and stores it in the database

[1014] Operation: The server validates the data received based on the validation rules, and if there are no problems, saves it in the database. It also generates a registration completion message and returns it to the terminal.

[1015] Input: Submitted user information

[1016] Output: User information saved in the database, registration completion message

[1017] Step 4: Your device will display a registration complete message

[1018] Operation: The terminal receives a registration completion message from the server and displays a dialog box informing the user that registration is complete.

[1019] Input: Registration completion message

[1020] Output: Notification to the user that registration is complete

[1021] Step 5: User Enters Profile Information

[1022] How it works: After logging in, users enter their language level, cultural interests, and exercise preferences in the profile settings screen.

[1023] Input: User-entered data (language level, cultural interests, exercise preferences)

[1024] Output: Profile information entered on the device

[1025] Step 6: The device sends the profile information to the server

[1026] How it works: The device converts the entered profile information into JSON format and sends it to the server via an HTTP POST request.

[1027] Input: Profile information entered on the device

[1028] Output: Profile information sent to the server

[1029] Step 7: The server saves the profile information to a database

[1030] Operation: The server validates the received profile information, stores it in a database, and associates the profile information with the user account.

[1031] Input: Submitted profile information

[1032] Output: Profile information stored in a database

[1033] Step 8: The server generates a virtual adventure plan

[1034] How it works: The server uses a generative AI model to generate an appropriate virtual adventure plan based on the user's profile information. The plan is then sent to the device.

[1035] Input: User profile information

[1036] Output: Generated virtual adventure plan, submitted adventure plan

[1037] Step 9: The device displays the adventure plan, and the user begins the adventure.

[1038] Operation: The terminal displays the virtual adventure plan received from the server and provides an interface for the user to click a "Start Adventure" button.

[1039] Input: Received adventure plan

[1040] Output: Adventure start screen, user starts adventure

[1041] Step 10: Server generates language learning quiz

[1042] How it works: The server uses a generative AI model to create language learning quizzes based on the user's progress and sends them to the device.

[1043] Input: User progress information

[1044] Output: Generated language learning quiz, submitted quiz

[1045] Step 11: The device displays the quiz and the user answers it.

[1046] How it works: A device displays language learning quizzes and provides an interface for users to enter answers.

[1047] Input: Received language learning quiz

[1048] Output: Quiz displayed, answers given by the user

[1049] Step 12: The device sends the answer to the server

[1050] How it works: The device converts the user's answers into JSON format and sends them to the server via an HTTP POST request.

[1051] Input: User's answer

[1052] Output: The answer sent to the server

[1053] Step 13: The server determines whether the quiz is correct or not and returns the result to the device.

[1054] Operation: The server judges the correctness of the received answer and returns the result to the terminal.

[1055] Input: Submitted answer

[1056] Output: Quiz results, data returned to the device

[1057] Step 14: The device displays the judgment result and the exercise task.

[1058] Operation: The device receives the judgment result from the server and displays it to the user, and then displays instructions for the next exercise task.

[1059] Input: Judgment result

[1060] Output: Display of judgment results, display of motor task instructions

[1061] Step 15: The user performs the motor task

[1062] Action: The user follows the displayed exercise instructions and performs the physical exercise.

[1063] Input: Motor task instructions

[1064] Output: The motor task performed

[1065] Step 16: Server generates feedback and captures emotion data

[1066] Operation: The server monitors the progress of language learning and motor tasks, generates real-time feedback, and utilizes an emotion engine to analyze the user's voice and facial expressions to obtain emotional data.

[1067] Input: Progress data, voice data, facial expression data

[1068] Output: Generated feedback, captured emotion data

[1069] Step 17: The device displays a feedback message and additional emotional feedback

[1070] Behavior: The device displays a feedback message from the server and additional feedback based on the emotion.

[1071] Input: Feedback message, additional feedback based on sentiment data

[1072] Output: The feedback message that is displayed to the user

[1073] Step 18: The server generates an overall rating and proposes the next adventure plan.

[1074] How it works: The server verifies that all tasks have been completed, evaluates the user's overall performance, and generates a next adventure plan taking into account the emotional data and sends it to the device.

[1075] Input: Task progress data, emotion data

[1076] Output: Generated overall evaluation, proposal for next adventure plan

[1077] Step 19: Your device will display your overall rating and options for your next adventure.

[1078] Operation: The device displays the overall rating from the server and presents the user with options for their next adventure plan.

[1079] Input: Overall rating, next adventure plan

[1080] Output: Overall rating and next adventure options displayed to the user

[1081] Step 20: User selects next adventure plan or exits the system

[1082] Action: The user selects the next adventure plan or exits the system.

[1083] Input: Next adventure plan option

[1084] Output: Next adventure plan selected or system termination

[1085] (Application example 2)

[1086] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1087] Systems that enable elderly people to learn languages ​​and promote their health while having fun are required to enhance the user experience and provide optimal feedback and content according to each individual's emotional state. However, existing systems lack the ability to appropriately recognize the user's emotional state and adapt feedback and content accordingly, which limits the user experience.

[1088] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1089] In this invention, the server includes means for collecting and storing user profile information, means for generating an appropriate virtual adventure plan based on the profile information, means for providing language learning tasks and exercise tasks based on the generated adventure plan, means for analyzing the user's emotional state using an emotion engine and adapting feedback and content, means for monitoring the progress of the language learning tasks and exercise tasks and providing feedback in real time, and means for generating an overall evaluation and suggesting the next adventure plan when the adventure is completed. This enables optimal feedback and content to be provided according to the user's emotional state, allowing elderly people to learn languages ​​more enjoyably and improve their health.

[1090] "User profile information" refers to information about a user's individual characteristics, interests, and abilities, and specifically includes language learning level, cultural interests, exercise preferences, and emotional data.

[1091] A "virtual adventure plan" is an activity plan for a virtual space that is generated based on the user's profile information, and includes language learning tasks and physical activities.

[1092] A "language learning task" is a quiz-style question given to a user, and is a challenge for the user to improve their language ability through their answers.

[1093] An "exercise task" is a physical exercise performed by a user according to exercise instructions displayed on the screen.

[1094] An "emotion engine" is a technology that analyzes a user's voice and facial expressions to recognize their emotional state.

[1095] "Feedback" refers to providing real-time evaluations and reactions to the user's actions and answers, and assisting the user in their learning or athletic progress.

[1096] "Emotional state" refers to the emotion the user is currently feeling, and is analyzed from voice, facial expressions, etc.

[1097] The "overall evaluation" is an index that evaluates the overall results of the language learning task and the motor task and indicates the user's performance.

[1098] The "next adventure plan" is a new virtual adventure plan that is proposed after the current adventure plan is completed.

[1099] The present invention is an interactive system that supports users in language learning and health promotion, and can provide optimal feedback and content according to the user's emotional state. The system implements various functions, including collecting and storing user profile information, generating virtual adventure plans, providing language learning tasks and exercise tasks, monitoring progress in real time, analyzing emotions using an emotion engine, generating overall ratings, and suggesting the next adventure plan.

[1100] Explaining program processing in natural language

[1101] The server performs the following process: Collects and stores the user's profile information (language learning level, cultural interests, exercise preferences, and emotional data). Generates an appropriate virtual adventure plan based on the user's profile information. This virtual adventure plan includes a language learning task and an exercise task. The language learning task involves presenting quiz-style questions, and the user improves their language ability through their answers. The exercise task involves performing exercises based on the exercise instructions displayed on the screen.

[1102] The server monitors the progress of language learning and motor tasks and provides real-time feedback. It also uses an emotion engine to analyze the user's voice and facial expressions to recognize their emotional state and adapt the feedback and content accordingly. By providing optimal feedback based on their emotional state, users can learn languages ​​and improve their health while having more fun. When the adventure is complete, the server evaluates the user's overall performance and suggests a plan for the next adventure.

[1103] The system runs on devices such as smartphones and tablets, and the server manages user data using cloud services such as AWS, Google Cloud, Firebase, and MongoDB. The emotion engine uses the Microsoft Azure Emotion API and Google Cloud Vision API.

[1104] Adding specific examples

[1105] As a concrete example, the following system usage scenario can be considered: If a 60-year-old user is interested in learning Japanese and Spanish culture, he or she will use the system in the following steps.

[1106] 1. User: Launch the app, register, and enter the required information. Then, on the profile settings screen, enter "Intermediate Japanese" and "Interested in Spanish culture."

[1107] 2. Terminal: Sends this information to the server and receives matching adventure plans from the server.

[1108] 3. Server: Based on the user's information, it generates and sends a virtual adventure plan for a trip to Spain.

[1109] 4. User: The adventure begins and the user answers a Japanese quiz (e.g., "Please translate this sentence: Hola! ¿Cómo estás?").

[1110] 5. Terminal: Sends the quiz results to the server and then displays the stretching exercise task.

[1111] 6. Server: Evaluates the user's answers, monitors the progress of the motor task, generates feedback, and uses an emotion engine to analyze the user's facial expressions and voice to obtain emotional data.

[1112] 7. Terminal: Display feedback messages in real time and provide additional feedback based on emotions.

[1113] 8. User: Completes all tasks, receives final evaluation, and decides whether to select the next adventure plan.

[1114] Prompt Sentence Examples

[1115] Next, a specific example of a prompt sentence to be input to the generative AI model is shown below.

[1116] Invention: A system for elderly people to learn languages ​​and improve their health while having fun

[1117] Application example: Interactive health language learning app

[1118] User Story: Seniors use their smartphones to learn a new language and get some exercise, and emotional feedback helps them enjoy more content.

[1119] How the program works: A user registers and sets up a profile. They then embark on a virtual adventure tailored to their profile, completing language quizzes and physical tasks and receiving real-time feedback. The program also uses an emotion engine to collect emotional data, which is then reflected in the next adventure plan.

[1120] In this way, the present invention provides an optimal interactive system for users, even elderly people, to learn languages ​​and improve their health while having fun.

[1121] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1122] Step 1:

[1123] User Registration and Login

[1124] The user launches the app and enters their name, age, email address, and password on the new registration screen.

[1125] The terminal transmits the input information to the server.

[1126] The server verifies the received user information, and if there are no problems, stores it in a database and returns a registration completion message to the terminal.

[1127] Input: User information (name, age, email address, password)

[1128] Data processing / calculation: Checking the validity of information, saving to database

[1129] Output: Registration complete message

[1130] The terminal receives the registration completion message and notifies the user, and transitions to the login screen.

[1131] Step 2:

[1132] Profile Settings

[1133] After logging in, users enter their language level, cultural interests, exercise preferences, and emotional data on the profile setting screen.

[1134] The terminal transmits the entered profile information to the server.

[1135] The server receives the profile information, stores it in a database, and associates the profile information with a user account.

[1136] Input: Profile information (language level, cultural interests, exercise preferences, emotional data)

[1137] Data processing / calculation: Formatting profile data and saving to database

[1138] Output: Save confirmation message

[1139] Step 3:

[1140] Start your virtual adventure

[1141] The server runs an algorithm to generate an appropriate virtual adventure plan based on the user's profile information and transmits the plan to the terminal.

[1142] Input: User profile information

[1143] Data processing / calculation: Execution of adventure plan generation algorithm

[1144] Output: Virtual adventure plan

[1145] The terminal displays a screen for starting the adventure and provides an interface for interacting with the robot avatar.

[1146] The user presses the "Start Adventure" button to begin the virtual adventure.

[1147] Step 4:

[1148] Combining language learning and exercise

[1149] The server creates scenarios for language learning tasks and physical tasks, and presents quiz-style questions as language learning tasks.

[1150] Enter: Virtual Adventure Plan

[1151] Data processing / calculation: Creating scenarios for language learning tasks and motor tasks

[1152] Output: Verbal learning task and motor task

[1153] The terminal provides an interface that displays the language learning task and allows the user to enter answers.

[1154] The user answers the quiz and presses the send button.

[1155] The terminal transmits the answers to the quiz to the server.

[1156] The server judges whether the quiz is correct or incorrect and returns the result of the judgment to the terminal.

[1157] Input: User's quiz answer

[1158] Data processing / calculation: Answer determination

[1159] Output: Judgment result

[1160] The terminal displays the judgment result and displays instructions for the exercise task.

[1161] The user follows the displayed exercise instructions and performs physical exercise.

[1162] Step 5:

[1163] Real-time feedback and emotion recognition

[1164] The server monitors the progress of language learning and motor tasks, generates real-time feedback, and uses an emotion engine to analyze the user's voice and facial expressions to obtain emotional data.

[1165] Input: Progress data of language learning tasks and motor tasks, user voice and facial expression data

[1166] Data processing / calculation: Feedback generation, emotional state analysis

[1167] Output: Real-time feedback, emotional data

[1168] The terminal displays additional emotion-based feedback to the user along with the feedback message.

[1169] The user reviews the feedback and proceeds to the next task.

[1170] Step 6:

[1171] Completing and evaluating adventures, and proposing plans for the next adventure

[1172] The server confirms that all tasks have been completed, evaluates the user's overall performance, and generates a next adventure plan taking into account the emotional data and sends it to the device.

[1173] Input: Language learning task and motor task completion data, emotion data

[1174] Data processing / calculation: Comprehensive evaluation generation, next adventure plan generation

[1175] Output: Overall rating, next adventure plan

[1176] The terminal displays a comprehensive evaluation screen and provides the user with options for the next adventure.

[1177] The user checks the overall rating and selects the next adventure plan or exits the system.

[1178] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1179] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1180] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1181] [Third embodiment]

[1182] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1183] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1184] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1185] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1186] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1187] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1188] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1189] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1190] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1191] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1192] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1193] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1194] This invention is a system for helping seniors learn languages ​​and improve their health while having fun. The system collects user profile information and provides an optimal virtual adventure plan based on that information. It combines language learning tasks with physical activities to provide users with a fun and fulfilling experience.

[1195] Explaining program processing in natural language

[1196] 1. User Registration and Login

[1197] User: Opens the app and enters their name, age, email address, and password on the new registration screen.

[1198] Terminal: Send this information to the server and display a message indicating registration is complete.

[1199] Server: Validates the input information and stores it in a database. Once stored, it provides login authentication to the device.

[1200] 2. Profile Settings

[1201] User: After logging in, enter your language learning level, cultural interests, exercise preferences, etc. in the profile settings screen.

[1202] Terminal: Sends the entered information to the server and prepares for the next step.

[1203] Server: Stores profile information and prepares the next appropriate virtual adventure plan.

[1204] 3. Start your virtual adventure

[1205] Server: Runs an algorithm to generate an adventure plan based on the user's profile and sends the plan to the device.

[1206] Terminal: Displays the adventure start screen and provides a dialogue interface with the robot avatar.

[1207] User: Press the "Start Adventure" button to begin the virtual adventure.

[1208] 4. Combining language learning and exercise

[1209] Server: Generates language learning tasks (e.g., Japanese quizzes) and exercise tasks (e.g., stretching).

[1210] Terminal: displays language learning tasks and provides an interface for users to enter answers.

[1211] User: Completes a quiz and then performs a motor task.

[1212] Terminal: Sends quiz answers to the server and displays instructions for the exercise task.

[1213] Server: Determines the answers to the quiz and generates the next task based on the progress.

[1214] 5. Real-time feedback

[1215] Server: Monitors language learning and motor progress and provides real-time feedback.

[1216] Terminal: Shows feedback messages and notifies the user of the next task.

[1217] User: Receive feedback and move on to the next task.

[1218] 6. Adventure Completion and Evaluation

[1219] Server: Once the adventure is completed, generate an overall evaluation and propose a plan for the next adventure.

[1220] Device: Displays the overall rating screen and presents the user with options for their next adventure.

[1221] User: Checks the rating and decides whether to select the next adventure or quit.

[1222] Specific examples

[1223] Example: A 60-year-old user is interested in learning Japanese and Spanish culture.

[1224] User: Launch the app, register, and enter the required information. Then, on the profile settings screen, enter that you have "Intermediate Japanese" and are "Interested in Spanish culture."

[1225] Terminal: Sends this information to the server and receives matching adventure plans from the server.

[1226] Server: Generates and sends a virtual adventure plan for a trip to Spain based on the user's information.

[1227] User: The adventure begins and the user answers a Japanese quiz (e.g., "Please translate this sentence: Hola! ¿Cómo estás?").

[1228] Terminal: Sends the quiz results to the server and then displays the stretching exercise task.

[1229] Server: Evaluates the user's answers, monitors the progress of the motor task, and generates feedback.

[1230] On the device: Provides real-time feedback and suggests next steps to the user.

[1231] User: Completes all tasks, receives final evaluation, and decides whether to select the next adventure plan.

[1232] In this way, users can learn languages ​​and promote health through exercise while enjoying different cultures in a virtual environment. This system helps seniors improve their cognitive function and maintain their health without feeling isolated.

[1233] The processing flow will be explained below.

[1234] Step 1:

[1235] User: Opens the app and enters their name, age, email address, and password on the new registration screen.

[1236] Terminal: Collects the entered information and sends it to the server.

[1237] Server: Validates the received user information and stores it in the database if there are no problems. Returns a registration completion message to the terminal.

[1238] Device: Receives the registration completion message and notifies the user. The device transitions to the login screen.

[1239] Step 2:

[1240] User: After logging in, enter your language level, cultural interests, and exercise preferences in the profile settings screen.

[1241] Device: Collects profile information entered and sends it to the server.

[1242] Server: Receives the profile information, stores it in a database, and associates the profile information with the user account.

[1243] Step 3:

[1244] Server: Generates an appropriate virtual adventure plan based on the user's profile information. Sends the generated adventure plan to the device.

[1245] Terminal: Displays the adventure start screen and provides a dialogue interface with the robot avatar.

[1246] User: Press the "Start Adventure" button to begin the virtual adventure.

[1247] Step 4:

[1248] Server: Creates a scenario for generating language learning tasks and exercise tasks. For the language learning tasks, it presents quiz-style questions to the user.

[1249] Terminal: Provides an interface that displays language learning tasks and allows users to enter answers.

[1250] User: Answers the quiz and presses the submit button.

[1251] Terminal: Sends the quiz answers to the server.

[1252] Server: Determines whether the quiz is correct or incorrect and returns the result to the device.

[1253] Terminal: Displays the judgment result and then the motor task screen.

[1254] User: Follow the exercise instructions displayed on the screen and perform physical exercise.

[1255] Step 5:

[1256] Server: Monitors the user's progress in language learning and motor tasks and generates feedback in real time.

[1257] Terminal: Display a feedback message to the user, asking if the user is ready to proceed to the next task.

[1258] User: Review the feedback and move on to the next task.

[1259] Step 6:

[1260] Server: Checks that all tasks are completed, evaluates the user's performance, and generates the next adventure plan and sends it to the device.

[1261] Device: Displays an overall rating screen and offers the user options for their next adventure.

[1262] User: Check the overall rating and decide whether to select the next adventure plan or exit the system.

[1263] This series of steps provides a system that allows elderly people to have fun, learn about different cultures and languages, and get appropriate exercise at the same time.

[1264] Example 1

[1265] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1266] There is a need for a system that allows elderly people to learn languages ​​and promote their health while having fun. However, many elderly people find that current learning methods lack enjoyment and continuity. Therefore, there is an urgent need to develop a system that allows elderly people to continue enjoying activities without straining themselves. There is also a need for a system that can efficiently combine language learning and exercise. Furthermore, there is a need for a system that can provide an optimized virtual adventure plan for each individual user and provide feedback in real time.

[1267] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1268] In this invention, the server includes means for collecting and storing user attribute information, means for generating an appropriate virtual adventure plan based on the attribute information, and means for providing language learning tasks and exercise tasks based on the generated adventure plan. This allows the user to enjoy learning a language and exercising through an individually optimized adventure plan. The server also includes means for monitoring the progress of the language learning tasks and exercise tasks and providing immediate feedback, and means for generating a comprehensive evaluation and proposing the next adventure plan upon completion of the adventure. This allows the user to check their progress in real time and proceed to the next step. This also allows elderly people to continue enjoying learning and exercise without straining themselves.

[1269] "Attribute information" refers to personal data about the user, such as age, interests, language learning level, and type of exercise.

[1270] A "virtual adventure plan" refers to a series of adventures or tasks to be performed in a virtual environment that are generated based on the user's attribute information.

[1271] A "language learning task" refers to an activity in which a user solves a specified language problem or challenge.

[1272] "Exercise task" refers to an activity that includes instructions for a movement or exercise that a user should perform.

[1273] "Immediate feedback" refers to providing real-time evaluation and advice on the results of a user's language learning or motor task.

[1274] "Overall evaluation" refers to a comprehensive evaluation of the results of all language learning tasks and motor tasks performed by the user.

[1275] "When the adventure is complete" refers to the point at which the user has completed all assigned language learning and physical tasks.

[1276] A "virtual adventure plan" refers to an adventurous activity or experience in a virtual environment that is individually generated based on the user's attribute information.

[1277] This invention is a system for elderly people to learn languages ​​and improve their health while having fun. The system collects user attribute information and provides an optimal virtual adventure plan based on that information. It combines language learning tasks with physical activities to provide users with a fun and fulfilling experience.

[1278] A specific embodiment of this system is as follows.

[1279] 1. Hardware and Software

[1280] server:

[1281] Hardware: A high-performance server (e.g., Dell PowerEdge series).

[1282] Software: Database management system (e.g., MySQL), programming language to execute the algorithms (e.g., Python).

[1283] Device:

[1284] Hardware: Tablet, smartphone, or PC.

[1285] Software: Dedicated application (e.g., iOS or Android app), browser-based interface.

[1286] User:

[1287] Hardware: Devices operated by users, such as tablets and smartphones.

[1288] Software: The dedicated application or browser-based interface.

[1289] 2. Data processing and calculation

[1290] User Registration and Login:

[1291] The user opens the app and enters their name, age, email address, and password on the new registration screen. The device sends this information to the server, which verifies the data and stores it in a database. When logging in, the server verifies the email address and password entered by the user, generates an authentication token, and sends it to the device.

[1292] Profile Settings:

[1293] After logging in, users enter their language learning level, cultural interests, exercise preferences, etc. on the profile setting screen. The device sends the entered information to the server, which then stores it in a database.

[1294] Create and launch a virtual adventure:

[1295] The server uses an algorithm to generate a virtual adventure plan based on the stored user attribute information. The generated plan is sent to the terminal, and the user presses the "Start Adventure" button to begin the virtual adventure.

[1296] Verbal learning and motor task performance:

[1297] The server generates language learning tasks (e.g., "Please translate this phrase") and exercise tasks (e.g., "Stretch for 5 minutes") based on the user's progress. The device presents these to the user and sends the user's answers and exercise progress to the server. The server evaluates the answers and generates the next task.

[1298] Real-time feedback:

[1299] The server monitors the user's progress in real time and generates and sends feedback to the device, allowing the user to move on to the next task.

[1300] Adventure completion and rating:

[1301] When the adventure is completed, the server generates a comprehensive evaluation and proposes the next adventure plan. The terminal presents the evaluation to the user, who then decides whether to select the next adventure.

[1302] 3. Examples and prompts

[1303] Examples:

[1304] If a 60-year-old user is interested in learning Japanese and French culture, the user launches the app, registers, and enters the necessary information. Then, on the profile settings screen, the user enters "Intermediate Japanese" and "Interested in French culture." Based on this information, the server generates a virtual adventure plan for a "trip to France" and sends it to the device. The user begins the adventure, answers a Japanese quiz (e.g., "Translate this sentence: Bonjour!"), and then performs an exercise task (e.g., "Stretch for 5 minutes").

[1305] Example prompt sentence:

[1306] Here is an example of a prompt to provide more details to a generative AI model:

[1307] Describe the specific steps of a system that helps elderly people improve their health while learning a language. The system's process consists of the following steps: user registration and login, profile setup, adventure plan generation and initiation, language learning and exercise task execution, real-time feedback, and adventure completion and evaluation. For each step, provide a detailed explanation of how the server, device, and user operate.

[1308] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1309] Step 1:

[1310] User Registration and Login

[1311] input:

[1312] The user opens the app and enters their name, age, email address, and password on the new registration screen.

[1313] Data processing and calculation:

[1314] The device sends the entered information to the server, which checks the integrity of the data and stores it in a database if there are no problems. When logging in, the entered email address and password are compared with the database, and if they match, an authentication token is generated.

[1315] output:

[1316] A registration completion message is displayed and an authentication token is returned.

[1317] Specific behavior:

[1318] The user enters the name "tanaka60", age "60", email address "tanaka60@example.com", and password "password123", and the device sends this to the server. The server stores it in a database, generates an authentication token, and sends it to the device. The device displays "Registration completed" to the user.

[1319] Step 2:

[1320] Profile Settings

[1321] input:

[1322] After logging in, users enter their language learning level, cultural interests, and exercise preferences on the profile settings screen.

[1323] Data processing and calculation:

[1324] The terminal transmits the input information to a server, which then stores the attribute information in a database.

[1325] output:

[1326] A save completion message will be displayed and preparations for the next step will be made.

[1327] Specific behavior:

[1328] The user selects "Japanese Beginner's," "French Culture," and "Yoga," and the device sends this to the server. The server saves this and displays the message "Profile saved."

[1329] Step 3:

[1330] Creating and launching a virtual adventure

[1331] input:

[1332] The server generates a virtual adventure plan based on the stored profile information.

[1333] Data processing and calculation:

[1334] The server uses an algorithm to generate a virtual adventure plan and transmits it to the terminal.

[1335] output:

[1336] The generated adventure plan is displayed and a "Start Adventure" button is presented to the user.

[1337] Specific behavior:

[1338] The server generates an adventure plan based on the theme of "Travel to France," combining Japanese conversation quizzes and yoga exercise tasks, and sends it to the terminal. The terminal then displays "Do you want to start your adventure traveling to France?" and presents the user with a "Start Adventure" button.

[1339] Step 4:

[1340] Verbal learning and motor task performance

[1341] input:

[1342] The server generates language learning tasks and motor tasks and sends them to the terminal.

[1343] Data processing and calculation:

[1344] The server generates questions, receives answers, and evaluates the user's progress, then generates instructions for the motor task.

[1345] output:

[1346] The language learning task and the motor task are displayed on the screen, and the user's answers and motor execution data are transmitted.

[1347] Specific behavior:

[1348] The server generates a quiz question, "Bonjour! Please translate this phrase into Japanese," and sends it to the device. The user answers, "Hello!", and the device sends this to the server. The server evaluates the answer, then generates an exercise task, "Please stretch for 5 minutes," and sends it to the device. The user performs the exercise and presses the "Task Complete" button.

[1349] Step 5:

[1350] Real-time feedback and next steps

[1351] input:

[1352] Receive the user's answers and exercise progress data.

[1353] Data processing and calculation:

[1354] The server evaluates the answers and generates feedback to send to the user.

[1355] output:

[1356] Get real-time feedback messages and next task notifications.

[1357] Specific behavior:

[1358] The server generates a message saying "Great, you got it right!" and prepares to present the next quiz or exercise task. The device displays a feedback message and offers the user a "Proceed to next task" button.

[1359] Step 6:

[1360] Adventure completion and evaluation

[1361] input:

[1362] Collect data when all tasks are completed.

[1363] Data processing and calculation:

[1364] The server generates an overall rating and suggests the next adventure plan.

[1365] output:

[1366] An overall evaluation screen will be displayed, and options for the next adventure will be presented.

[1367] Specific behavior:

[1368] The server evaluates the user's overall performance and generates a message saying, "Adventure successful! We recommend a trip to Italy as your next adventure." The device displays the evaluation results along with the next options, providing the user with an interface to select "Start the next adventure" or "End."

[1369] (Application example 1)

[1370] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1371] This invention relates to a system for helping seniors learn languages ​​and promote their health while having fun. Specifically, the system aims to reduce loneliness and improve cognitive function and health by providing a virtual adventure that combines language learning and physical exercise through a realistic experience within a brick-and-mortar store. It is also required to provide real-time feedback and closely monitor the user's progress.

[1372] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1373] In this invention, the server includes means for collecting and storing personal information of the user, means for generating an appropriate virtual adventure plan based on the personal information, means for providing language learning tasks and physical exercise tasks based on the generated adventure plan, means for monitoring the progress of the language learning tasks and physical exercise tasks and providing feedback in real time, means for generating an overall evaluation and proposing a next adventure plan when the adventure is completed, means for providing an adventure scenario within the physical store, means for performing the language learning tasks and physical exercise tasks at specified locations while walking within the physical store, and means for providing real-time evaluation and feedback. This enables the user to improve their health while having fun learning a language through a realistic experience within the physical store.

[1374] "User personal information" refers to information such as the user's name, age, language learning level, cultural interests, and physical activity preferences.

[1375] "Virtual adventure plan" refers to a plan that includes language learning tasks and physical exercise tasks, generated based on the user's personal information.

[1376] A "language learning task" refers to the task of quiz-style questions posed to the user and judging their answers.

[1377] "Physical exercise task" refers to an exercise performed by a user based on exercise instructions displayed on a screen.

[1378] "Brick and mortar store" refers to a retail store or facility that a user can actually visit.

[1379] A "virtual adventure scenario" is a scenario that a user experiences in a physical store, and refers to content that is constructed based on a virtual adventure plan.

[1380] "Real-time feedback" refers to assessment and instruction provided in real time on a user's progress as they perform a language learning task or physical movement task.

[1381] "Overall Grade" refers to the overall grade or rating provided upon completing a series of tasks in a virtual adventure.

[1382] The system embodying this invention collects personal information from users, creates a virtual adventure plan, and provides language learning tasks and physical exercise tasks to promote language learning and health for the elderly. Specific embodiments are described below.

[1383] 1. User Registration and Profile Settings

[1384] The server collects personal information entered by the user (such as name, age, language learning level, cultural interests, physical activity preferences, etc.) and stores it in a database. Django is a suitable server-side framework for this.

[1385] The device sends the information entered by the user to the server via the interface and prepares to proceed to the next step. The device can be a smartphone or tablet, and the application is implemented using React Native.

[1386] 2. Creating and launching a virtual adventure plan

[1387] The server generates a virtual adventure plan based on the collected personal information using Python scripts and serverless architectures such as AWS Lambda, and automatically creates a plan that includes language learning tasks and physical exercise tasks tailored to the user's interests and level.

[1388] The device displays the generated adventure plan to the user and prompts them to start the adventure. An adventure scenario in a physical store is displayed, and when the user presses the "Start Adventure" button, the actual task begins.

[1389] 3. Providing language learning and physical motor tasks

[1390] The server creates language learning tasks (e.g., quiz-style questions) and physical exercise tasks (e.g., stretching and walking) and sends them to the terminal. PostgreSQL is often used as the database.

[1391] The terminal displays the language learning tasks and provides an interface where the user can enter answers, and displays instructions for the physical motor tasks so that the user can perform them.

[1392] 4. Real-time feedback

[1393] The server monitors the progress of language learning tasks and physical motor tasks and generates real-time feedback using real-time communication over WebSockets.

[1394] The terminal displays the feedback messages received from the server in real time and notifies the user of the next task.

[1395] 5. Adventure Completion and Evaluation

[1396] Once the adventure is completed, the server generates an overall evaluation and proposes a plan for the next virtual adventure. Based on the evaluation algorithm, the evaluation is calculated based on the user's performance and progress.

[1397] The terminal displays the overall rating and presents the user with options for the next adventure.

[1398] Specific examples

[1399] Example: If a 60-year-old user is interested in learning Japanese and Spanish culture, the user opens the app, enters their personal information, and then enters "Intermediate Japanese" and "Interested in Spanish culture" in the profile settings screen. The user is then presented with a virtual adventure scenario set in a physical store, where they answer Japanese language quizzes and perform stretching exercises in designated areas. The device displays real-time feedback, allowing the user to track their progress as they progress through the adventure.

[1400] Prompt Sentence Examples

[1401] "I want to design a system that helps seniors learn languages ​​and promote their health while having fun. This system collects users' personal information and then provides them with an optimal virtual adventure plan based on that information. The application combines language learning tasks with physical exercise tasks to provide a fun and fulfilling experience. For example, if a 65-year-old user is interested in French culture, the app can have them complete quizzes about French tourist attractions and related stretching exercises, and provide real-time feedback. Please tell me more about how you will implement this system and the technologies you will use."

[1402] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1403] Step 1:

[1404] User registration and profile settings

[1405] Users open a smartphone app and enter personal information such as their name, age, language learning level, cultural interests, and physical activity preferences.

[1406] Input: Name, age, language level, cultural interests, physical activity preferences

[1407] The terminal transmits the input information to the server.

[1408] Input: User personal information

[1409] The server stores the received information in a database. The information is managed using the Django framework.

[1410] Output: User's personal information stored in a database

[1411] The server sends a message to the terminal confirming that the save has been completed.

[1412] Output: Save complete confirmation message

[1413] Step 2:

[1414] Virtual adventure planning

[1415] The server generates a virtual adventure plan based on the user's personal information. The algorithm used is implemented in Python and processed in a scalable manner using AWS Lambda.

[1416] Input: User personal information

[1417] Data processing: Data calculations to generate optimal adventure plans based on the user's interests and learning level

[1418] Output: Virtual adventure plan

[1419] The server transmits the generated adventure plan to the terminal.

[1420] Output: Adventure plan sent to terminal

[1421] Step 3:

[1422] The start of a virtual adventure

[1423] The terminal displays the generated adventure plan to the user and provides the user with a "Start Adventure" button.

[1424] Input: Virtual adventure plan

[1425] The user presses the "Start Adventure" button.

[1426] Output: Instructions to start the adventure

[1427] The device displays the adventure's progress screen and provides instructions within the physical store.

[1428] Step 4:

[1429] Providing language learning and physical motor tasks

[1430] The server generates language learning tasks (e.g., quiz-style questions) and physical exercise tasks (e.g., stretching and walking) based on the virtual adventure plan and sends them to the terminal.

[1431] Input: Virtual adventure plan

[1432] Data processing: generating language learning tasks and physical movement tasks

[1433] Output: The generated tasks

[1434] The terminal displays the language learning task and provides an interface where the user can enter answers, as well as displaying instructions for the physical movement task.

[1435] Input: The generated task

[1436] Output: Task display screen

[1437] The user answers the tasks and performs the exercises.

[1438] Step 5:

[1439] Real-time feedback

[1440] The server monitors the user's progress and generates real-time feedback, using WebSockets for real-time communication.

[1441] Input: User's answer results and exercise progress

[1442] Data processing: generating feedback

[1443] Output: Real-time feedback

[1444] The terminal displays the received feedback to the user in real time.

[1445] Output: Display feedback

[1446] Step 6:

[1447] Adventure completion and evaluation

[1448] After the adventure is completed, the server generates an overall rating and proposes the next adventure plan. The rating is calculated based on the rating algorithm.

[1449] Input: All task progress data for the user

[1450] Data processing: Generating an overall rating

[1451] Output: Overall evaluation, new adventure plan

[1452] The terminal displays the overall rating to the user and offers options for the next adventure.

[1453] Output: Overall rating and next adventure plan

[1454] The user selects the next adventure or ends it.

[1455] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1456] This invention is a system for helping seniors learn languages ​​and improve their health while having fun, and it can improve the quality of the user experience by incorporating an emotion engine. The system collects user profile information and provides a virtual adventure plan based on that information. It combines language learning tasks with physical tasks to provide a fun and fulfilling experience for the user, and includes real-time feedback and emotion recognition adaptation.

[1457] Explaining program processing in natural language

[1458] 1. User Registration and Login

[1459] User: Opens the app and enters their name, age, email address, and password on the new registration screen.

[1460] Terminal: Collects the entered information and sends it to the server.

[1461] Server: Validates the received user information and stores it in the database if there are no problems. Returns a registration completion message to the terminal.

[1462] Device: Receives the registration completion message and notifies the user. The device transitions to the login screen.

[1463] 2. Profile Settings

[1464] User: After logging in, enter your language level, cultural interests, and exercise preferences in the profile settings screen.

[1465] Device: Collects profile information entered and sends it to the server.

[1466] Server: Receives the profile information, stores it in a database, and associates the profile information with the user account.

[1467] 3. Start your virtual adventure

[1468] Server: Runs an algorithm to generate an appropriate virtual adventure plan based on the user's profile information and sends the plan to the device.

[1469] Terminal: Displays the adventure start screen and provides a dialogue interface with the robot avatar.

[1470] User: Press the "Start Adventure" button to begin the virtual adventure.

[1471] 4. Combining language learning and exercise

[1472] Server: Creates scenarios for language learning tasks and motor tasks. Provides quiz-style questions for the language learning tasks.

[1473] Terminal: Provides an interface that displays language learning tasks and allows users to enter answers.

[1474] User: Answers the quiz and presses the submit button.

[1475] Terminal: Sends the quiz answers to the server.

[1476] Server: Determines whether the quiz is correct or incorrect and returns the result to the device.

[1477] Terminal: Displays the judgment results and instructions for the motor task.

[1478] User: Follow the displayed exercise instructions and perform physical exercise.

[1479] 5. Real-time feedback and emotion recognition

[1480] Server: Monitors the progress of language learning and motor tasks, generates real-time feedback, and uses an emotion engine to analyze the user's voice and facial expressions to obtain emotional data.

[1481] Terminal: Display additional emotion-based feedback to the user along with the feedback message.

[1482] User: Review the feedback and move on to the next task.

[1483] 6. Completing and evaluating the adventure, and proposing the next adventure plan

[1484] Server: Checks that all tasks are completed, evaluates the user's overall performance, and generates the next adventure plan taking into account the emotional data and sends it to the device.

[1485] Device: Displays an overall rating screen and offers the user options for their next adventure.

[1486] User: Check the overall rating, choose your next adventure plan, or exit the system.

[1487] Specific examples

[1488] Example: A 60-year-old user is interested in learning Japanese and Spanish culture.

[1489] User: Launch the app, register, and enter the required information. Then, on the profile settings screen, enter that you have "Intermediate Japanese" and are "Interested in Spanish culture."

[1490] Terminal: Sends this information to the server and receives matching adventure plans from the server.

[1491] Server: Generates and sends a virtual adventure plan for a trip to Spain based on the user's information.

[1492] User: The adventure begins and the user answers a Japanese quiz (e.g., "Please translate this sentence: Hola! ¿Cómo estás?").

[1493] Terminal: Sends the quiz results to the server and then displays the stretching exercise task.

[1494] Server: Evaluates the user's answers, monitors the progress of the motor task, and generates feedback. It also uses an emotion engine to analyze the user's facial expressions and voice to obtain emotional data.

[1495] On-device: Display feedback messages in real time and provide additional feedback based on sentiment.

[1496] User: Completes all tasks, receives final evaluation, and decides whether to select the next adventure plan.

[1497] In this way, users can learn languages ​​and improve their health through exercise while enjoying different cultures in a virtual environment. Furthermore, the emotional engine optimizes the user experience to match each individual's emotional state, helping seniors improve their cognitive function and maintain their health without feeling isolated.

[1498] The processing flow will be explained below.

[1499] Step 1:

[1500] User: Opens the app and enters their name, age, email address, and password on the new registration screen.

[1501] Step 2:

[1502] Terminal: Collects the entered information and sends it to the server.

[1503] Step 3:

[1504] Server: Validates the received user information and stores it in the database if there are no problems. Returns a registration completion message to the terminal.

[1505] Step 4:

[1506] Device: Receives the registration completion message and notifies the user. The device transitions to the login screen.

[1507] Step 5:

[1508] User: After logging in, enter your language level, cultural interests, and exercise preferences in the profile settings screen.

[1509] Step 6:

[1510] Device: Collects profile information entered and sends it to the server.

[1511] Step 7:

[1512] Server: Receives the profile information, stores it in a database, and associates the profile information with the user account.

[1513] Step 8:

[1514] Server: Runs an algorithm to generate an appropriate virtual adventure plan based on the user's profile information and sends the plan to the device.

[1515] Step 9:

[1516] Terminal: Displays the adventure start screen and provides a dialogue interface with the robot avatar.

[1517] Step 10:

[1518] User: Press the "Start Adventure" button to begin the virtual adventure.

[1519] Step 11:

[1520] Server: Creates scenarios for language learning tasks and motor tasks. Provides quiz-style questions for the language learning tasks.

[1521] Step 12:

[1522] Terminal: Provides an interface that displays language learning tasks and allows users to enter answers.

[1523] Step 13:

[1524] User: Answers the quiz and presses the submit button.

[1525] Step 14:

[1526] Terminal: Sends the quiz answers to the server.

[1527] Step 15:

[1528] Server: Determines whether the quiz is correct or incorrect and returns the result to the device.

[1529] Step 16:

[1530] Terminal: Displays the judgment result and then the motor task screen.

[1531] Step 17:

[1532] User: Follow the displayed exercise instructions and perform physical exercise.

[1533] Step 18:

[1534] Server: Monitors the progress of language learning and motor tasks, generates real-time feedback, and uses an emotion engine to analyze the user's voice and facial expressions to obtain emotional data.

[1535] Step 19:

[1536] Terminal: Display additional emotion-based feedback to the user along with the feedback message.

[1537] Step 20:

[1538] User: Review the feedback and move on to the next task.

[1539] Step 21:

[1540] Server: Checks that all tasks are completed, evaluates the user's overall performance, and generates the next adventure plan taking into account the emotional data and sends it to the device.

[1541] Step 22:

[1542] Device: Displays an overall rating screen and offers the user options for their next adventure.

[1543] Step 23:

[1544] User: Check the overall rating, choose your next adventure plan, or exit the system.

[1545] Example 2

[1546] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1547] In modern society, it is becoming increasingly important for older adults to maintain cognitive function through language learning and promote health through physical activity. However, traditional learning and exercise programs often lack enjoyment and make continued participation difficult. Furthermore, they lack approaches that take into account users' emotions and individual interests, resulting in a one-sided user experience. This increases the likelihood that older adults will feel isolated, preventing them from fully realizing the benefits of learning and exercise. Therefore, there is a need for a system that allows older adults to learn languages ​​and promote their health while having fun.

[1548] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting and storing user profile information, means for generating an appropriate virtual adventure plan based on the profile information, means for providing language learning tasks and exercise tasks based on the generated adventure plan, means for monitoring the progress of the language learning tasks and exercise tasks and providing feedback in real time, means for analyzing the user's voice and facial expressions using an emotion engine to obtain emotion data, and means for generating an overall evaluation and suggesting the next adventure plan when the adventure is completed. This enables elderly people to learn a language while having fun, promote their health, and provide an optimal user experience tailored to their individual emotions and interests.

[1549] "User" refers to the person who uses the system, and is often the elderly in particular.

[1550] "Profile information" is information that indicates a user's individual characteristics and preferences, such as their language learning level, cultural interests, and exercise preferences.

[1551] A "virtual adventure plan" is a plan containing learning and exercise scenarios in a virtual environment that is generated based on a user's profile information.

[1552] A "language learning task" refers to a learning activity that a user performs with the goal of acquiring a language, and includes quiz-style questions.

[1553] "Exercise task" refers to an exercise activity that a user performs to promote physical health, and is performed according to exercise instructions displayed on the screen.

[1554] "Progress monitoring" is the process by which the system monitors the user's progress in language learning and motor tasks in real time.

[1555] "Real-time feedback" refers to the immediate response and evaluation of a user's activities, helping them improve their learning and exercise.

[1556] "Emotion engine" refers to the technology and algorithms used to analyze a user's voice and facial expressions and obtain emotional data.

[1557] "Emotion data" is information about the user's emotional state obtained by the emotion engine.

[1558] "Overall Rating" refers to the overall rating of a user's performance generated by the system when the user completes an adventure task.

[1559] "Next adventure plan" refers to a new virtual adventure plan that the user should undertake next, which is generated based on the overall evaluation and emotional data.

[1560] This invention is a system for helping seniors learn languages ​​and improve their health while having fun, and it can improve the quality of the user experience by incorporating an emotion engine. The system collects user profile information and provides a virtual adventure plan based on that information. It combines language learning tasks with physical tasks to provide a fun and fulfilling experience for the user, and includes real-time feedback and emotion recognition adaptation.

[1561] User Registration and Login

[1562] A user launches the application using a device and enters their name, age, email address, and password on the new registration screen. The device collects this information and sends it to the server. The server verifies the received information and stores it in a database. Once registration is complete, the device displays a registration completion message and the user is taken to the login screen.

[1563] Profile Settings

[1564] After logging in, users enter their language level, cultural interests, and exercise preferences on the profile settings screen. The device collects this profile information and sends it to the server. The server verifies the information and stores it in a database. This associates the profile information with the user's account.

[1565] Start your virtual adventure

[1566] The server generates an appropriate virtual adventure plan using a generative AI model based on the user's profile information. This plan is sent to the device, which displays a start-of-adventure screen. The user presses the "Start Adventure" button to begin the virtual adventure. The adventure plan includes language learning tasks and physical tasks.

[1567] Combining language learning and exercise

[1568] The server generates language learning quizzes based on the user's progress and sends them to the device. The device displays the quizzes, and the user answers them. The answers are sent to the server, which judges whether they are correct or incorrect and returns the results to the device. The device displays the judgment results and then displays instructions for the exercise task. The user follows the exercise instructions and performs physical exercise.

[1569] Real-time feedback and emotion recognition

[1570] The server monitors the progress of language learning and motor tasks and generates real-time feedback. It also uses an emotion engine to analyze the user's voice and facial expressions to obtain emotional data. The device displays additional feedback based on the user's emotions along with the feedback message.

[1571] Completing and evaluating adventures, and proposing plans for the next adventure

[1572] The server confirms that all tasks have been completed and comprehensively evaluates the user's performance. It then generates the next adventure plan taking into account the emotional data and sends it to the device. The device then displays the comprehensive evaluation screen, allowing the user to select the next adventure plan or exit the system.

[1573] Specific examples

[1574] Example: A 60-year-old user is interested in learning Japanese and Spanish culture.

[1575] The user starts the app, registers, and enters the required information. Then, on the profile settings screen, the user enters that they have an intermediate level of Japanese and are interested in Spanish culture.

[1576] The terminal transmits this information to the server and receives matching adventure plans from the server.

[1577] The server generates and transmits a virtual adventure plan for a "trip to Spain" based on the user's information.

[1578] The user begins the adventure and answers Japanese quizzes (e.g., "Please translate this sentence: Hola! ¿Cómo estás?").

[1579] The terminal transmits the quiz results to the server, and then displays the stretching exercise task.

[1580] The server evaluates the user's answers, monitors the progress of the motor task, and generates feedback. It also uses an emotion engine to analyze the user's facial expressions and voice to obtain emotional data.

[1581] The device displays feedback messages in real time and provides additional feedback based on emotions.

[1582] The user completes all tasks, receives a final evaluation, and decides whether to select the next adventure plan.

[1583] The system allows users to learn languages ​​while enjoying different cultures and promote health through exercise. It also uses an emotion engine to optimize the user experience to match each individual's emotional state, helping seniors improve their cognitive function and maintain their health without feeling isolated.

[1584] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1585] Step 1: User enters registration information

[1586] How it works: A user enters information such as their name, age, email address, and password on the app's new registration screen.

[1587] Input: User-entered data (name, age, email address, password)

[1588] Output: User information entered into the terminal

[1589] Step 2: The device sends the user information to the server

[1590] How it works: The device collects the information entered by the user, converts it to JSON format, and sends it to the server via an HTTP POST request.

[1591] Input: User information entered into the terminal

[1592] Output: User information sent to the server

[1593] Step 3: The server validates the user information and stores it in the database

[1594] Operation: The server validates the data received based on the validation rules, and if there are no problems, saves it in the database. It also generates a registration completion message and returns it to the terminal.

[1595] Input: Submitted user information

[1596] Output: User information saved in the database, registration completion message

[1597] Step 4: Your device will display a registration complete message

[1598] Operation: The terminal receives a registration completion message from the server and displays a dialog box informing the user that registration is complete.

[1599] Input: Registration completion message

[1600] Output: Notification to the user that registration is complete

[1601] Step 5: User Enters Profile Information

[1602] How it works: After logging in, users enter their language level, cultural interests, and exercise preferences in the profile settings screen.

[1603] Input: User-entered data (language level, cultural interests, exercise preferences)

[1604] Output: Profile information entered on the device

[1605] Step 6: The device sends the profile information to the server

[1606] How it works: The device converts the entered profile information into JSON format and sends it to the server via an HTTP POST request.

[1607] Input: Profile information entered on the device

[1608] Output: Profile information sent to the server

[1609] Step 7: The server saves the profile information to a database

[1610] Operation: The server validates the received profile information, stores it in a database, and associates the profile information with the user account.

[1611] Input: Submitted profile information

[1612] Output: Profile information stored in a database

[1613] Step 8: The server generates a virtual adventure plan

[1614] How it works: The server uses a generative AI model to generate an appropriate virtual adventure plan based on the user's profile information. The plan is then sent to the device.

[1615] Input: User profile information

[1616] Output: Generated virtual adventure plan, submitted adventure plan

[1617] Step 9: The device displays the adventure plan, and the user begins the adventure.

[1618] Operation: The terminal displays the virtual adventure plan received from the server and provides an interface for the user to click a "Start Adventure" button.

[1619] Input: Received adventure plan

[1620] Output: Adventure start screen, user starts adventure

[1621] Step 10: Server generates language learning quiz

[1622] How it works: The server uses a generative AI model to create language learning quizzes based on the user's progress and sends them to the device.

[1623] Input: User progress information

[1624] Output: Generated language learning quiz, submitted quiz

[1625] Step 11: The device displays the quiz and the user answers it.

[1626] How it works: A device displays language learning quizzes and provides an interface for users to enter answers.

[1627] Input: Received language learning quiz

[1628] Output: Quiz displayed, answers given by the user

[1629] Step 12: The device sends the answer to the server

[1630] How it works: The device converts the user's answers into JSON format and sends them to the server via an HTTP POST request.

[1631] Input: User's answer

[1632] Output: The answer sent to the server

[1633] Step 13: The server determines whether the quiz is correct or not and returns the result to the device.

[1634] Operation: The server judges the correctness of the received answer and returns the result to the terminal.

[1635] Input: Submitted answer

[1636] Output: Quiz results, data returned to the device

[1637] Step 14: The device displays the judgment result and the exercise task.

[1638] Operation: The device receives the judgment result from the server and displays it to the user, and then displays instructions for the next exercise task.

[1639] Input: Judgment result

[1640] Output: Display of judgment results, display of motor task instructions

[1641] Step 15: The user performs the motor task

[1642] Action: The user follows the displayed exercise instructions and performs the physical exercise.

[1643] Input: Motor task instructions

[1644] Output: The motor task performed

[1645] Step 16: Server generates feedback and captures emotion data

[1646] Operation: The server monitors the progress of language learning and motor tasks, generates real-time feedback, and utilizes an emotion engine to analyze the user's voice and facial expressions to obtain emotional data.

[1647] Input: Progress data, voice data, facial expression data

[1648] Output: Generated feedback, captured emotion data

[1649] Step 17: The device displays a feedback message and additional emotional feedback

[1650] Behavior: The device displays a feedback message from the server and additional feedback based on the emotion.

[1651] Input: Feedback message, additional feedback based on sentiment data

[1652] Output: The feedback message that is displayed to the user

[1653] Step 18: The server generates an overall rating and proposes the next adventure plan.

[1654] How it works: The server verifies that all tasks have been completed, evaluates the user's overall performance, and generates a next adventure plan taking into account the emotional data and sends it to the device.

[1655] Input: Task progress data, emotion data

[1656] Output: Generated overall evaluation, proposal for next adventure plan

[1657] Step 19: Your device will display your overall rating and options for your next adventure.

[1658] Operation: The device displays the overall rating from the server and presents the user with options for their next adventure plan.

[1659] Input: Overall rating, next adventure plan

[1660] Output: Overall rating and next adventure options displayed to the user

[1661] Step 20: User selects next adventure plan or exits the system

[1662] Action: The user selects the next adventure plan or exits the system.

[1663] Input: Next adventure plan option

[1664] Output: Next adventure plan selected or system termination

[1665] (Application example 2)

[1666] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1667] Systems that enable elderly people to learn languages ​​and promote their health while having fun are required to enhance the user experience and provide optimal feedback and content according to each individual's emotional state. However, existing systems lack the ability to appropriately recognize the user's emotional state and adapt feedback and content accordingly, which limits the user experience.

[1668] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1669] In this invention, the server includes means for collecting and storing user profile information, means for generating an appropriate virtual adventure plan based on the profile information, means for providing language learning tasks and exercise tasks based on the generated adventure plan, means for analyzing the user's emotional state using an emotion engine and adapting feedback and content, means for monitoring the progress of the language learning tasks and exercise tasks and providing feedback in real time, and means for generating an overall evaluation and suggesting the next adventure plan when the adventure is completed. This enables optimal feedback and content to be provided according to the user's emotional state, allowing elderly people to learn languages ​​more enjoyably and improve their health.

[1670] "User profile information" refers to information about a user's individual characteristics, interests, and abilities, and specifically includes language learning level, cultural interests, exercise preferences, and emotional data.

[1671] A "virtual adventure plan" is an activity plan for a virtual space that is generated based on the user's profile information, and includes language learning tasks and physical activities.

[1672] A "language learning task" is a quiz-style question given to a user, and is a challenge for the user to improve their language ability through their answers.

[1673] An "exercise task" is a physical exercise performed by a user according to exercise instructions displayed on the screen.

[1674] An "emotion engine" is a technology that analyzes a user's voice and facial expressions to recognize their emotional state.

[1675] "Feedback" refers to providing real-time evaluations and reactions to the user's actions and answers, and assisting the user in their learning or athletic progress.

[1676] "Emotional state" refers to the emotion the user is currently feeling, and is analyzed from voice, facial expressions, etc.

[1677] The "overall evaluation" is an index that evaluates the overall results of the language learning task and the motor task and indicates the user's performance.

[1678] The "next adventure plan" is a new virtual adventure plan that is proposed after the current adventure plan is completed.

[1679] The present invention is an interactive system that supports users in language learning and health promotion, and can provide optimal feedback and content according to the user's emotional state. The system implements various functions, including collecting and storing user profile information, generating virtual adventure plans, providing language learning tasks and exercise tasks, monitoring progress in real time, analyzing emotions using an emotion engine, generating overall ratings, and suggesting the next adventure plan.

[1680] Explaining program processing in natural language

[1681] The server performs the following process: Collects and stores the user's profile information (language learning level, cultural interests, exercise preferences, and emotional data). Generates an appropriate virtual adventure plan based on the user's profile information. This virtual adventure plan includes a language learning task and an exercise task. The language learning task involves presenting quiz-style questions, and the user improves their language ability through their answers. The exercise task involves performing exercises based on the exercise instructions displayed on the screen.

[1682] The server monitors the progress of language learning and motor tasks and provides real-time feedback. It also uses an emotion engine to analyze the user's voice and facial expressions to recognize their emotional state and adapt the feedback and content accordingly. By providing optimal feedback based on their emotional state, users can learn languages ​​and improve their health while having more fun. When the adventure is complete, the server evaluates the user's overall performance and suggests a plan for the next adventure.

[1683] The system runs on devices such as smartphones and tablets, and the server manages user data using cloud services such as AWS, Google Cloud, Firebase, and MongoDB. The emotion engine uses the Microsoft Azure Emotion API and Google Cloud Vision API.

[1684] Adding specific examples

[1685] As a concrete example, the following system usage scenario can be considered: If a 60-year-old user is interested in learning Japanese and Spanish culture, he or she will use the system in the following steps.

[1686] 1. User: Launch the app, register, and enter the required information. Then, on the profile settings screen, enter "Intermediate Japanese" and "Interested in Spanish culture."

[1687] 2. Terminal: Sends this information to the server and receives matching adventure plans from the server.

[1688] 3. Server: Based on the user's information, it generates and sends a virtual adventure plan for a trip to Spain.

[1689] 4. User: The adventure begins and the user answers a Japanese quiz (e.g., "Please translate this sentence: Hola! ¿Cómo estás?").

[1690] 5. Terminal: Sends the quiz results to the server and then displays the stretching exercise task.

[1691] 6. Server: Evaluates the user's answers, monitors the progress of the motor task, generates feedback, and uses an emotion engine to analyze the user's facial expressions and voice to obtain emotional data.

[1692] 7. Terminal: Display feedback messages in real time and provide additional feedback based on emotions.

[1693] 8. User: Completes all tasks, receives final evaluation, and decides whether to select the next adventure plan.

[1694] Prompt Sentence Examples

[1695] Next, a specific example of a prompt sentence to be input to the generative AI model is shown below.

[1696] Invention: A system for elderly people to learn languages ​​and improve their health while having fun

[1697] Application example: Interactive health language learning app

[1698] User Story: Seniors use their smartphones to learn a new language and get some exercise, and emotional feedback helps them enjoy more content.

[1699] How the program works: A user registers and sets up a profile. They then embark on a virtual adventure tailored to their profile, completing language quizzes and physical tasks and receiving real-time feedback. The program also uses an emotion engine to collect emotional data, which is then reflected in the next adventure plan.

[1700] In this way, the present invention provides an optimal interactive system for users, even elderly people, to learn languages ​​and improve their health while having fun.

[1701] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1702] Step 1:

[1703] User Registration and Login

[1704] The user launches the app and enters their name, age, email address, and password on the new registration screen.

[1705] The terminal transmits the input information to the server.

[1706] The server verifies the received user information, and if there are no problems, stores it in a database and returns a registration completion message to the terminal.

[1707] Input: User information (name, age, email address, password)

[1708] Data processing / calculation: Checking the validity of information, saving to database

[1709] Output: Registration complete message

[1710] The terminal receives the registration completion message and notifies the user, and transitions to the login screen.

[1711] Step 2:

[1712] Profile Settings

[1713] After logging in, users enter their language level, cultural interests, exercise preferences, and emotional data on the profile setting screen.

[1714] The terminal transmits the entered profile information to the server.

[1715] The server receives the profile information, stores it in a database, and associates the profile information with a user account.

[1716] Input: Profile information (language level, cultural interests, exercise preferences, emotional data)

[1717] Data processing / calculation: Formatting profile data and saving to database

[1718] Output: Save confirmation message

[1719] Step 3:

[1720] Start your virtual adventure

[1721] The server runs an algorithm to generate an appropriate virtual adventure plan based on the user's profile information and transmits the plan to the terminal.

[1722] Input: User profile information

[1723] Data processing / calculation: Execution of adventure plan generation algorithm

[1724] Output: Virtual adventure plan

[1725] The terminal displays a screen for starting the adventure and provides an interface for interacting with the robot avatar.

[1726] The user presses the "Start Adventure" button to begin the virtual adventure.

[1727] Step 4:

[1728] Combining language learning and exercise

[1729] The server creates scenarios for language learning tasks and physical tasks, and presents quiz-style questions as language learning tasks.

[1730] Enter: Virtual Adventure Plan

[1731] Data processing / calculation: Creating scenarios for language learning tasks and motor tasks

[1732] Output: Verbal learning task and motor task

[1733] The terminal provides an interface that displays the language learning task and allows the user to enter answers.

[1734] The user answers the quiz and presses the send button.

[1735] The terminal transmits the answers to the quiz to the server.

[1736] The server judges whether the quiz is correct or incorrect and returns the result of the judgment to the terminal.

[1737] Input: User's quiz answer

[1738] Data processing / calculation: Answer determination

[1739] Output: Judgment result

[1740] The terminal displays the judgment result and displays instructions for the exercise task.

[1741] The user follows the displayed exercise instructions and performs physical exercise.

[1742] Step 5:

[1743] Real-time feedback and emotion recognition

[1744] The server monitors the progress of language learning and motor tasks, generates real-time feedback, and uses an emotion engine to analyze the user's voice and facial expressions to obtain emotional data.

[1745] Input: Progress data of language learning tasks and motor tasks, user voice and facial expression data

[1746] Data processing / calculation: Feedback generation, emotional state analysis

[1747] Output: Real-time feedback, emotional data

[1748] The terminal displays additional emotion-based feedback to the user along with the feedback message.

[1749] The user reviews the feedback and proceeds to the next task.

[1750] Step 6:

[1751] Completing and evaluating adventures, and proposing plans for the next adventure

[1752] The server confirms that all tasks have been completed, evaluates the user's overall performance, and generates a next adventure plan taking into account the emotional data and sends it to the device.

[1753] Input: Language learning task and motor task completion data, emotion data

[1754] Data processing / calculation: Comprehensive evaluation generation, next adventure plan generation

[1755] Output: Overall rating, next adventure plan

[1756] The terminal displays a comprehensive evaluation screen and provides the user with options for the next adventure.

[1757] The user checks the overall rating and selects the next adventure plan or exits the system.

[1758] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1759] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1760] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1761] [Fourth embodiment]

[1762] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1763] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1764] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1765] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1766] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1767] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1768] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1769] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1770] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1771] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1772] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1773] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1774] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1775] This invention is a system for helping seniors learn languages ​​and improve their health while having fun. The system collects user profile information and provides an optimal virtual adventure plan based on that information. It combines language learning tasks with physical activities to provide users with a fun and fulfilling experience.

[1776] Explaining program processing in natural language

[1777] 1. User Registration and Login

[1778] User: Opens the app and enters their name, age, email address, and password on the new registration screen.

[1779] Terminal: Send this information to the server and display a message indicating registration is complete.

[1780] Server: Validates the input information and stores it in a database. Once stored, it provides login authentication to the device.

[1781] 2. Profile Settings

[1782] User: After logging in, enter your language learning level, cultural interests, exercise preferences, etc. in the profile settings screen.

[1783] Terminal: Sends the entered information to the server and prepares for the next step.

[1784] Server: Stores profile information and prepares the next appropriate virtual adventure plan.

[1785] 3. Start your virtual adventure

[1786] Server: Runs an algorithm to generate an adventure plan based on the user's profile and sends the plan to the device.

[1787] Terminal: Displays the adventure start screen and provides a dialogue interface with the robot avatar.

[1788] User: Press the "Start Adventure" button to begin the virtual adventure.

[1789] 4. Combining language learning and exercise

[1790] Server: Generates language learning tasks (e.g., Japanese quizzes) and exercise tasks (e.g., stretching).

[1791] Terminal: displays language learning tasks and provides an interface for users to enter answers.

[1792] User: Completes a quiz and then performs a motor task.

[1793] Terminal: Sends quiz answers to the server and displays instructions for the exercise task.

[1794] Server: Determines the answers to the quiz and generates the next task based on the progress.

[1795] 5. Real-time feedback

[1796] Server: Monitors language learning and motor progress and provides real-time feedback.

[1797] Terminal: Shows feedback messages and notifies the user of the next task.

[1798] User: Receive feedback and move on to the next task.

[1799] 6. Adventure Completion and Evaluation

[1800] Server: Once the adventure is completed, generate an overall evaluation and propose a plan for the next adventure.

[1801] Device: Displays the overall rating screen and presents the user with options for their next adventure.

[1802] User: Checks the rating and decides whether to select the next adventure or quit.

[1803] Specific examples

[1804] Example: A 60-year-old user is interested in learning Japanese and Spanish culture.

[1805] User: Launch the app, register, and enter the required information. Then, on the profile settings screen, enter that you have "Intermediate Japanese" and are "Interested in Spanish culture."

[1806] Terminal: Sends this information to the server and receives matching adventure plans from the server.

[1807] Server: Generates and sends a virtual adventure plan for a trip to Spain based on the user's information.

[1808] User: The adventure begins and the user answers a Japanese quiz (e.g., "Please translate this sentence: Hola! ¿Cómo estás?").

[1809] Terminal: Sends the quiz results to the server and then displays the stretching exercise task.

[1810] Server: Evaluates the user's answers, monitors the progress of the motor task, and generates feedback.

[1811] On the device: Provides real-time feedback and suggests next steps to the user.

[1812] User: Completes all tasks, receives final evaluation, and decides whether to select the next adventure plan.

[1813] In this way, users can learn languages ​​and promote health through exercise while enjoying different cultures in a virtual environment. This system helps seniors improve their cognitive function and maintain their health without feeling isolated.

[1814] The processing flow will be explained below.

[1815] Step 1:

[1816] User: Opens the app and enters their name, age, email address, and password on the new registration screen.

[1817] Terminal: Collects the entered information and sends it to the server.

[1818] Server: Validates the received user information and stores it in the database if there are no problems. Returns a registration completion message to the terminal.

[1819] Device: Receives the registration completion message and notifies the user. The device transitions to the login screen.

[1820] Step 2:

[1821] User: After logging in, enter your language level, cultural interests, and exercise preferences in the profile settings screen.

[1822] Device: Collects profile information entered and sends it to the server.

[1823] Server: Receives the profile information, stores it in a database, and associates the profile information with the user account.

[1824] Step 3:

[1825] Server: Generates an appropriate virtual adventure plan based on the user's profile information. Sends the generated adventure plan to the device.

[1826] Terminal: Displays the adventure start screen and provides a dialogue interface with the robot avatar.

[1827] User: Press the "Start Adventure" button to begin the virtual adventure.

[1828] Step 4:

[1829] Server: Creates a scenario for generating language learning tasks and exercise tasks. For the language learning tasks, it presents quiz-style questions to the user.

[1830] Terminal: Provides an interface that displays language learning tasks and allows users to enter answers.

[1831] User: Answers the quiz and presses the submit button.

[1832] Terminal: Sends the quiz answers to the server.

[1833] Server: Determines whether the quiz is correct or incorrect and returns the result to the device.

[1834] Terminal: Displays the judgment result and then the motor task screen.

[1835] User: Follow the exercise instructions displayed on the screen and perform physical exercise.

[1836] Step 5:

[1837] Server: Monitors the user's progress in language learning and motor tasks and generates feedback in real time.

[1838] Terminal: Display a feedback message to the user, asking if the user is ready to proceed to the next task.

[1839] User: Review the feedback and move on to the next task.

[1840] Step 6:

[1841] Server: Checks that all tasks are completed, evaluates the user's performance, and generates the next adventure plan and sends it to the device.

[1842] Device: Displays an overall rating screen and offers the user options for their next adventure.

[1843] User: Check the overall rating and decide whether to select the next adventure plan or exit the system.

[1844] This series of steps provides a system that allows elderly people to have fun, learn about different cultures and languages, and get appropriate exercise at the same time.

[1845] Example 1

[1846] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1847] There is a need for a system that allows elderly people to learn languages ​​and promote their health while having fun. However, many elderly people find that current learning methods lack enjoyment and continuity. Therefore, there is an urgent need to develop a system that allows elderly people to continue enjoying activities without straining themselves. There is also a need for a system that can efficiently combine language learning and exercise. Furthermore, there is a need for a system that can provide an optimized virtual adventure plan for each individual user and provide feedback in real time.

[1848] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1849] In this invention, the server includes means for collecting and storing user attribute information, means for generating an appropriate virtual adventure plan based on the attribute information, and means for providing language learning tasks and exercise tasks based on the generated adventure plan. This allows the user to enjoy learning a language and exercising through an individually optimized adventure plan. The server also includes means for monitoring the progress of the language learning tasks and exercise tasks and providing immediate feedback, and means for generating a comprehensive evaluation and proposing the next adventure plan upon completion of the adventure. This allows the user to check their progress in real time and proceed to the next step. This also allows elderly people to continue enjoying learning and exercise without straining themselves.

[1850] "Attribute information" refers to personal data about the user, such as age, interests, language learning level, and type of exercise.

[1851] A "virtual adventure plan" refers to a series of adventures or tasks to be performed in a virtual environment that are generated based on the user's attribute information.

[1852] A "language learning task" refers to an activity in which a user solves a specified language problem or challenge.

[1853] "Exercise task" refers to an activity that includes instructions for a movement or exercise that a user should perform.

[1854] "Immediate feedback" refers to providing real-time evaluation and advice on the results of a user's language learning or motor task.

[1855] "Overall evaluation" refers to a comprehensive evaluation of the results of all language learning tasks and motor tasks performed by the user.

[1856] "When the adventure is complete" refers to the point at which the user has completed all assigned language learning and physical tasks.

[1857] A "virtual adventure plan" refers to an adventurous activity or experience in a virtual environment that is individually generated based on the user's attribute information.

[1858] This invention is a system for elderly people to learn languages ​​and improve their health while having fun. The system collects user attribute information and provides an optimal virtual adventure plan based on that information. It combines language learning tasks with physical activities to provide users with a fun and fulfilling experience.

[1859] A specific embodiment of this system is as follows.

[1860] 1. Hardware and Software

[1861] server:

[1862] Hardware: A high-performance server (e.g., Dell PowerEdge series).

[1863] Software: Database management system (e.g., MySQL), programming language to execute the algorithms (e.g., Python).

[1864] Device:

[1865] Hardware: Tablet, smartphone, or PC.

[1866] Software: Dedicated application (e.g., iOS or Android app), browser-based interface.

[1867] User:

[1868] Hardware: Devices operated by users, such as tablets and smartphones.

[1869] Software: The dedicated application or browser-based interface.

[1870] 2. Data processing and calculation

[1871] User Registration and Login:

[1872] The user opens the app and enters their name, age, email address, and password on the new registration screen. The device sends this information to the server, which verifies the data and stores it in a database. When logging in, the server verifies the email address and password entered by the user, generates an authentication token, and sends it to the device.

[1873] Profile Settings:

[1874] After logging in, users enter their language learning level, cultural interests, exercise preferences, etc. on the profile setting screen. The device sends the entered information to the server, which then stores it in a database.

[1875] Create and launch a virtual adventure:

[1876] The server uses an algorithm to generate a virtual adventure plan based on the stored user attribute information. The generated plan is sent to the terminal, and the user presses the "Start Adventure" button to begin the virtual adventure.

[1877] Verbal learning and motor task performance:

[1878] The server generates language learning tasks (e.g., "Please translate this phrase") and exercise tasks (e.g., "Stretch for 5 minutes") based on the user's progress. The device presents these to the user and sends the user's answers and exercise progress to the server. The server evaluates the answers and generates the next task.

[1879] Real-time feedback:

[1880] The server monitors the user's progress in real time and generates and sends feedback to the device, allowing the user to move on to the next task.

[1881] Adventure completion and rating:

[1882] When the adventure is completed, the server generates a comprehensive evaluation and proposes the next adventure plan. The terminal presents the evaluation to the user, who then decides whether to select the next adventure.

[1883] 3. Examples and prompts

[1884] Examples:

[1885] If a 60-year-old user is interested in learning Japanese and French culture, the user launches the app, registers, and enters the necessary information. Then, on the profile settings screen, the user enters "Intermediate Japanese" and "Interested in French culture." Based on this information, the server generates a virtual adventure plan for a "trip to France" and sends it to the device. The user begins the adventure, answers a Japanese quiz (e.g., "Translate this sentence: Bonjour!"), and then performs an exercise task (e.g., "Stretch for 5 minutes").

[1886] Example prompt sentence:

[1887] Here is an example of a prompt to provide more details to a generative AI model:

[1888] Describe the specific steps of a system that helps elderly people improve their health while learning a language. The system's process consists of the following steps: user registration and login, profile setup, adventure plan generation and initiation, language learning and exercise task execution, real-time feedback, and adventure completion and evaluation. For each step, provide a detailed explanation of how the server, device, and user operate.

[1889] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1890] Step 1:

[1891] User Registration and Login

[1892] input:

[1893] The user opens the app and enters their name, age, email address, and password on the new registration screen.

[1894] Data processing and calculation:

[1895] The device sends the entered information to the server, which checks the integrity of the data and stores it in a database if there are no problems. When logging in, the entered email address and password are compared with the database, and if they match, an authentication token is generated.

[1896] output:

[1897] A registration completion message is displayed and an authentication token is returned.

[1898] Specific behavior:

[1899] The user enters the name "tanaka60", age "60", email address "tanaka60@example.com", and password "password123", and the device sends this to the server. The server stores it in a database, generates an authentication token, and sends it to the device. The device displays "Registration completed" to the user.

[1900] Step 2:

[1901] Profile Settings

[1902] input:

[1903] After logging in, users enter their language learning level, cultural interests, and exercise preferences on the profile settings screen.

[1904] Data processing and calculation:

[1905] The terminal transmits the input information to a server, which then stores the attribute information in a database.

[1906] output:

[1907] A save completion message will be displayed and preparations for the next step will be made.

[1908] Specific behavior:

[1909] The user selects "Japanese Beginner's," "French Culture," and "Yoga," and the device sends this to the server. The server saves this and displays the message "Profile saved."

[1910] Step 3:

[1911] Creating and launching a virtual adventure

[1912] input:

[1913] The server generates a virtual adventure plan based on the stored profile information.

[1914] Data processing and calculation:

[1915] The server uses an algorithm to generate a virtual adventure plan and transmits it to the terminal.

[1916] output:

[1917] The generated adventure plan is displayed and a "Start Adventure" button is presented to the user.

[1918] Specific behavior:

[1919] The server generates an adventure plan based on the theme of "Travel to France," combining Japanese conversation quizzes and yoga exercise tasks, and sends it to the terminal. The terminal then displays "Do you want to start your adventure traveling to France?" and presents the user with a "Start Adventure" button.

[1920] Step 4:

[1921] Verbal learning and motor task performance

[1922] input:

[1923] The server generates language learning tasks and motor tasks and sends them to the terminal.

[1924] Data processing and calculation:

[1925] The server generates questions, receives answers, and evaluates the user's progress, then generates instructions for the motor task.

[1926] output:

[1927] The language learning task and the motor task are displayed on the screen, and the user's answers and motor execution data are transmitted.

[1928] Specific behavior:

[1929] The server generates a quiz question, "Bonjour! Please translate this phrase into Japanese," and sends it to the device. The user answers, "Hello!", and the device sends this to the server. The server evaluates the answer, then generates an exercise task, "Please stretch for 5 minutes," and sends it to the device. The user performs the exercise and presses the "Task Complete" button.

[1930] Step 5:

[1931] Real-time feedback and next steps

[1932] input:

[1933] Receive the user's answers and exercise progress data.

[1934] Data processing and calculation:

[1935] The server evaluates the answers and generates feedback to send to the user.

[1936] output:

[1937] Get real-time feedback messages and next task notifications.

[1938] Specific behavior:

[1939] The server generates a message saying "Great, you got it right!" and prepares to present the next quiz or exercise task. The device displays a feedback message and offers the user a "Proceed to next task" button.

[1940] Step 6:

[1941] Adventure completion and evaluation

[1942] input:

[1943] Collect data when all tasks are completed.

[1944] Data processing and calculation:

[1945] The server generates an overall rating and suggests the next adventure plan.

[1946] output:

[1947] An overall evaluation screen will be displayed, and options for the next adventure will be presented.

[1948] Specific behavior:

[1949] The server evaluates the user's overall performance and generates a message saying, "Adventure successful! We recommend a trip to Italy as your next adventure." The device displays the evaluation results along with the next options, providing the user with an interface to select "Start the next adventure" or "End."

[1950] (Application example 1)

[1951] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1952] This invention relates to a system for helping seniors learn languages ​​and promote their health while having fun. Specifically, the system aims to reduce loneliness and improve cognitive function and health by providing a virtual adventure that combines language learning and physical exercise through a realistic experience within a brick-and-mortar store. It is also required to provide real-time feedback and closely monitor the user's progress.

[1953] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1954] In this invention, the server includes means for collecting and storing personal information of the user, means for generating an appropriate virtual adventure plan based on the personal information, means for providing language learning tasks and physical exercise tasks based on the generated adventure plan, means for monitoring the progress of the language learning tasks and physical exercise tasks and providing feedback in real time, means for generating an overall evaluation and proposing a next adventure plan when the adventure is completed, means for providing an adventure scenario within the physical store, means for performing the language learning tasks and physical exercise tasks at specified locations while walking within the physical store, and means for providing real-time evaluation and feedback. This enables the user to improve their health while having fun learning a language through a realistic experience within the physical store.

[1955] "User personal information" refers to information such as the user's name, age, language learning level, cultural interests, and physical activity preferences.

[1956] "Virtual adventure plan" refers to a plan that includes language learning tasks and physical exercise tasks, generated based on the user's personal information.

[1957] A "language learning task" refers to the task of quiz-style questions posed to the user and judging their answers.

[1958] "Physical exercise task" refers to an exercise performed by a user based on exercise instructions displayed on a screen.

[1959] "Brick and mortar store" refers to a retail store or facility that a user can actually visit.

[1960] A "virtual adventure scenario" is a scenario that a user experiences in a physical store, and refers to content that is constructed based on a virtual adventure plan.

[1961] "Real-time feedback" refers to assessment and instruction provided in real time on a user's progress as they perform a language learning task or physical movement task.

[1962] "Overall Grade" refers to the overall grade or rating provided upon completing a series of tasks in a virtual adventure.

[1963] The system embodying this invention collects personal information from users, creates a virtual adventure plan, and provides language learning tasks and physical exercise tasks to promote language learning and health for the elderly. Specific embodiments are described below.

[1964] 1. User Registration and Profile Settings

[1965] The server collects personal information entered by the user (such as name, age, language learning level, cultural interests, physical activity preferences, etc.) and stores it in a database. Django is a suitable server-side framework for this.

[1966] The device sends the information entered by the user to the server via the interface and prepares to proceed to the next step. The device can be a smartphone or tablet, and the application is implemented using React Native.

[1967] 2. Creating and launching a virtual adventure plan

[1968] The server generates a virtual adventure plan based on the collected personal information using Python scripts and serverless architectures such as AWS Lambda, and automatically creates a plan that includes language learning tasks and physical exercise tasks tailored to the user's interests and level.

[1969] The device displays the generated adventure plan to the user and prompts them to start the adventure. An adventure scenario in a physical store is displayed, and when the user presses the "Start Adventure" button, the actual task begins.

[1970] 3. Providing language learning and physical motor tasks

[1971] The server creates language learning tasks (e.g., quiz-style questions) and physical exercise tasks (e.g., stretching and walking) and sends them to the terminal. PostgreSQL is often used as the database.

[1972] The terminal displays the language learning tasks and provides an interface where the user can enter answers, and displays instructions for the physical motor tasks so that the user can perform them.

[1973] 4. Real-time feedback

[1974] The server monitors the progress of language learning tasks and physical motor tasks and generates real-time feedback using real-time communication over WebSockets.

[1975] The terminal displays the feedback messages received from the server in real time and notifies the user of the next task.

[1976] 5. Adventure Completion and Evaluation

[1977] Once the adventure is completed, the server generates an overall evaluation and proposes a plan for the next virtual adventure. Based on the evaluation algorithm, the evaluation is calculated based on the user's performance and progress.

[1978] The terminal displays the overall rating and presents the user with options for the next adventure.

[1979] Specific examples

[1980] Example: If a 60-year-old user is interested in learning Japanese and Spanish culture, the user opens the app, enters their personal information, and then enters "Intermediate Japanese" and "Interested in Spanish culture" in the profile settings screen. The user is then presented with a virtual adventure scenario set in a physical store, where they answer Japanese language quizzes and perform stretching exercises in designated areas. The device displays real-time feedback, allowing the user to track their progress as they progress through the adventure.

[1981] Prompt Sentence Examples

[1982] "I want to design a system that helps seniors learn languages ​​and promote their health while having fun. This system collects users' personal information and then provides them with an optimal virtual adventure plan based on that information. The application combines language learning tasks with physical exercise tasks to provide a fun and fulfilling experience. For example, if a 65-year-old user is interested in French culture, the app can have them complete quizzes about French tourist attractions and related stretching exercises, and provide real-time feedback. Please tell me more about how you will implement this system and the technologies you will use."

[1983] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1984] Step 1:

[1985] User registration and profile settings

[1986] Users open a smartphone app and enter personal information such as their name, age, language learning level, cultural interests, and physical activity preferences.

[1987] Input: Name, age, language level, cultural interests, physical activity preferences

[1988] The terminal transmits the input information to the server.

[1989] Input: User personal information

[1990] The server stores the received information in a database. The information is managed using the Django framework.

[1991] Output: User's personal information stored in a database

[1992] The server sends a message to the terminal confirming that the save has been completed.

[1993] Output: Save complete confirmation message

[1994] Step 2:

[1995] Virtual adventure planning

[1996] The server generates a virtual adventure plan based on the user's personal information. The algorithm used is implemented in Python and processed in a scalable manner using AWS Lambda.

[1997] Input: User personal information

[1998] Data processing: Data calculations to generate optimal adventure plans based on the user's interests and learning level

[1999] Output: Virtual adventure plan

[2000] The server transmits the generated adventure plan to the terminal.

[2001] Output: Adventure plan sent to terminal

[2002] Step 3:

[2003] The start of a virtual adventure

[2004] The terminal displays the generated adventure plan to the user and provides the user with a "Start Adventure" button.

[2005] Input: Virtual adventure plan

[2006] The user presses the "Start Adventure" button.

[2007] Output: Instructions to start the adventure

[2008] The device displays the adventure's progress screen and provides instructions within the physical store.

[2009] Step 4:

[2010] Providing language learning and physical motor tasks

[2011] The server generates language learning tasks (e.g., quiz-style questions) and physical exercise tasks (e.g., stretching and walking) based on the virtual adventure plan and sends them to the terminal.

[2012] Input: Virtual adventure plan

[2013] Data processing: generating language learning tasks and physical movement tasks

[2014] Output: The generated tasks

[2015] The terminal displays the language learning task and provides an interface where the user can enter answers, as well as displaying instructions for the physical movement task.

[2016] Input: The generated task

[2017] Output: Task display screen

[2018] The user answers the tasks and performs the exercises.

[2019] Step 5:

[2020] Real-time feedback

[2021] The server monitors the user's progress and generates real-time feedback, using WebSockets for real-time communication.

[2022] Input: User's answer results and exercise progress

[2023] Data processing: generating feedback

[2024] Output: Real-time feedback

[2025] The terminal displays the received feedback to the user in real time.

[2026] Output: Display feedback

[2027] Step 6:

[2028] Adventure completion and evaluation

[2029] After the adventure is completed, the server generates an overall rating and proposes the next adventure plan. The rating is calculated based on the rating algorithm.

[2030] Input: All task progress data for the user

[2031] Data processing: Generating an overall rating

[2032] Output: Overall evaluation, new adventure plan

[2033] The terminal displays the overall rating to the user and offers options for the next adventure.

[2034] Output: Overall rating and next adventure plan

[2035] The user selects the next adventure or ends it.

[2036] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2037] This invention is a system for helping seniors learn languages ​​and improve their health while having fun, and it can improve the quality of the user experience by incorporating an emotion engine. The system collects user profile information and provides a virtual adventure plan based on that information. It combines language learning tasks with physical tasks to provide a fun and fulfilling experience for the user, and includes real-time feedback and emotion recognition adaptation.

[2038] Explaining program processing in natural language

[2039] 1. User Registration and Login

[2040] User: Opens the app and enters their name, age, email address, and password on the new registration screen.

[2041] Terminal: Collects the entered information and sends it to the server.

[2042] Server: Validates the received user information and stores it in the database if there are no problems. Returns a registration completion message to the terminal.

[2043] Device: Receives the registration completion message and notifies the user. The device transitions to the login screen.

[2044] 2. Profile Settings

[2045] User: After logging in, enter your language level, cultural interests, and exercise preferences in the profile settings screen.

[2046] Device: Collects profile information entered and sends it to the server.

[2047] Server: Receives the profile information, stores it in a database, and associates the profile information with the user account.

[2048] 3. Start your virtual adventure

[2049] Server: Runs an algorithm to generate an appropriate virtual adventure plan based on the user's profile information and sends the plan to the device.

[2050] Terminal: Displays the adventure start screen and provides a dialogue interface with the robot avatar.

[2051] User: Press the "Start Adventure" button to begin the virtual adventure.

[2052] 4. Combining language learning and exercise

[2053] Server: Creates scenarios for language learning tasks and motor tasks. Provides quiz-style questions for the language learning tasks.

[2054] Terminal: Provides an interface that displays language learning tasks and allows users to enter answers.

[2055] User: Answers the quiz and presses the submit button.

[2056] Terminal: Sends the quiz answers to the server.

[2057] Server: Determines whether the quiz is correct or incorrect and returns the result to the device.

[2058] Terminal: Displays the judgment results and instructions for the motor task.

[2059] User: Follow the displayed exercise instructions and perform physical exercise.

[2060] 5. Real-time feedback and emotion recognition

[2061] Server: Monitors the progress of language learning and motor tasks, generates real-time feedback, and uses an emotion engine to analyze the user's voice and facial expressions to obtain emotional data.

[2062] Terminal: Display additional emotion-based feedback to the user along with the feedback message.

[2063] User: Review the feedback and move on to the next task.

[2064] 6. Completing and evaluating the adventure, and proposing the next adventure plan

[2065] Server: Checks that all tasks are completed, evaluates the user's overall performance, and generates the next adventure plan taking into account the emotional data and sends it to the device.

[2066] Device: Displays an overall rating screen and offers the user options for their next adventure.

[2067] User: Check the overall rating, choose your next adventure plan, or exit the system.

[2068] Specific examples

[2069] Example: A 60-year-old user is interested in learning Japanese and Spanish culture.

[2070] User: Launch the app, register, and enter the required information. Then, on the profile settings screen, enter that you have "Intermediate Japanese" and are "Interested in Spanish culture."

[2071] Terminal: Sends this information to the server and receives matching adventure plans from the server.

[2072] Server: Generates and sends a virtual adventure plan for a trip to Spain based on the user's information.

[2073] User: The adventure begins and the user answers a Japanese quiz (e.g., "Please translate this sentence: Hola! ¿Cómo estás?").

[2074] Terminal: Sends the quiz results to the server and then displays the stretching exercise task.

[2075] Server: Evaluates the user's answers, monitors the progress of the motor task, and generates feedback. It also uses an emotion engine to analyze the user's facial expressions and voice to obtain emotional data.

[2076] On-device: Display feedback messages in real time and provide additional feedback based on sentiment.

[2077] User: Completes all tasks, receives final evaluation, and decides whether to select the next adventure plan.

[2078] In this way, users can learn languages ​​and improve their health through exercise while enjoying different cultures in a virtual environment. Furthermore, the emotional engine optimizes the user experience to match each individual's emotional state, helping seniors improve their cognitive function and maintain their health without feeling isolated.

[2079] The processing flow will be explained below.

[2080] Step 1:

[2081] User: Opens the app and enters their name, age, email address, and password on the new registration screen.

[2082] Step 2:

[2083] Terminal: Collects the entered information and sends it to the server.

[2084] Step 3:

[2085] Server: Validates the received user information and stores it in the database if there are no problems. Returns a registration completion message to the terminal.

[2086] Step 4:

[2087] Device: Receives the registration completion message and notifies the user. The device transitions to the login screen.

[2088] Step 5:

[2089] User: After logging in, enter your language level, cultural interests, and exercise preferences in the profile settings screen.

[2090] Step 6:

[2091] Device: Collects profile information entered and sends it to the server.

[2092] Step 7:

[2093] Server: Receives the profile information, stores it in a database, and associates the profile information with the user account.

[2094] Step 8:

[2095] Server: Runs an algorithm to generate an appropriate virtual adventure plan based on the user's profile information and sends the plan to the device.

[2096] Step 9:

[2097] Terminal: Displays the adventure start screen and provides a dialogue interface with the robot avatar.

[2098] Step 10:

[2099] User: Press the "Start Adventure" button to begin the virtual adventure.

[2100] Step 11:

[2101] Server: Creates scenarios for language learning tasks and motor tasks. Provides quiz-style questions for the language learning tasks.

[2102] Step 12:

[2103] Terminal: Provides an interface that displays language learning tasks and allows users to enter answers.

[2104] Step 13:

[2105] User: Answers the quiz and presses the submit button.

[2106] Step 14:

[2107] Terminal: Sends the quiz answers to the server.

[2108] Step 15:

[2109] Server: Determines whether the quiz is correct or incorrect and returns the result to the device.

[2110] Step 16:

[2111] Terminal: Displays the judgment result and then the motor task screen.

[2112] Step 17:

[2113] User: Follow the displayed exercise instructions and perform physical exercise.

[2114] Step 18:

[2115] Server: Monitors the progress of language learning and motor tasks, generates real-time feedback, and uses an emotion engine to analyze the user's voice and facial expressions to obtain emotional data.

[2116] Step 19:

[2117] Terminal: Display additional emotion-based feedback to the user along with the feedback message.

[2118] Step 20:

[2119] User: Review the feedback and move on to the next task.

[2120] Step 21:

[2121] Server: Checks that all tasks are completed, evaluates the user's overall performance, and generates the next adventure plan taking into account the emotional data and sends it to the device.

[2122] Step 22:

[2123] Device: Displays an overall rating screen and offers the user options for their next adventure.

[2124] Step 23:

[2125] User: Check the overall rating, choose your next adventure plan, or exit the system.

[2126] Example 2

[2127] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2128] In modern society, it is becoming increasingly important for older adults to maintain cognitive function through language learning and promote health through physical activity. However, traditional learning and exercise programs often lack enjoyment and make continued participation difficult. Furthermore, they lack approaches that take into account users' emotions and individual interests, resulting in a one-sided user experience. This increases the likelihood that older adults will feel isolated, preventing them from fully realizing the benefits of learning and exercise. Therefore, there is a need for a system that allows older adults to learn languages ​​and promote their health while having fun.

[2129] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting and storing user profile information, means for generating an appropriate virtual adventure plan based on the profile information, means for providing language learning tasks and exercise tasks based on the generated adventure plan, means for monitoring the progress of the language learning tasks and exercise tasks and providing feedback in real time, means for analyzing the user's voice and facial expressions using an emotion engine to obtain emotion data, and means for generating an overall evaluation and suggesting the next adventure plan when the adventure is completed. This enables elderly people to learn a language while having fun, promote their health, and provide an optimal user experience tailored to their individual emotions and interests.

[2130] "User" refers to the person who uses the system, and is often the elderly in particular.

[2131] "Profile information" is information that indicates a user's individual characteristics and preferences, such as their language learning level, cultural interests, and exercise preferences.

[2132] A "virtual adventure plan" is a plan containing learning and exercise scenarios in a virtual environment that is generated based on a user's profile information.

[2133] A "language learning task" refers to a learning activity that a user performs with the goal of acquiring a language, and includes quiz-style questions.

[2134] "Exercise task" refers to an exercise activity that a user performs to promote physical health, and is performed according to exercise instructions displayed on the screen.

[2135] "Progress monitoring" is the process by which the system monitors the user's progress in language learning and motor tasks in real time.

[2136] "Real-time feedback" refers to the immediate response and evaluation of a user's activities, helping them improve their learning and exercise.

[2137] "Emotion engine" refers to the technology and algorithms used to analyze a user's voice and facial expressions and obtain emotional data.

[2138] "Emotion data" is information about the user's emotional state obtained by the emotion engine.

[2139] "Overall Rating" refers to the overall rating of a user's performance generated by the system when the user completes an adventure task.

[2140] "Next adventure plan" refers to a new virtual adventure plan that the user should undertake next, which is generated based on the overall evaluation and emotional data.

[2141] This invention is a system for helping seniors learn languages ​​and improve their health while having fun, and it can improve the quality of the user experience by incorporating an emotion engine. The system collects user profile information and provides a virtual adventure plan based on that information. It combines language learning tasks with physical tasks to provide a fun and fulfilling experience for the user, and includes real-time feedback and emotion recognition adaptation.

[2142] User Registration and Login

[2143] A user launches the application using a device and enters their name, age, email address, and password on the new registration screen. The device collects this information and sends it to the server. The server verifies the received information and stores it in a database. Once registration is complete, the device displays a registration completion message and the user is taken to the login screen.

[2144] Profile Settings

[2145] After logging in, users enter their language level, cultural interests, and exercise preferences on the profile settings screen. The device collects this profile information and sends it to the server. The server verifies the information and stores it in a database. This associates the profile information with the user's account.

[2146] Start your virtual adventure

[2147] The server generates an appropriate virtual adventure plan using a generative AI model based on the user's profile information. This plan is sent to the device, which displays a start-of-adventure screen. The user presses the "Start Adventure" button to begin the virtual adventure. The adventure plan includes language learning tasks and physical tasks.

[2148] Combining language learning and exercise

[2149] The server generates language learning quizzes based on the user's progress and sends them to the device. The device displays the quizzes, and the user answers them. The answers are sent to the server, which judges whether they are correct or incorrect and returns the results to the device. The device displays the judgment results and then displays instructions for the exercise task. The user follows the exercise instructions and performs physical exercise.

[2150] Real-time feedback and emotion recognition

[2151] The server monitors the progress of language learning and motor tasks and generates real-time feedback. It also uses an emotion engine to analyze the user's voice and facial expressions to obtain emotional data. The device displays additional feedback based on the user's emotions along with the feedback message.

[2152] Completing and evaluating adventures, and proposing plans for the next adventure

[2153] The server confirms that all tasks have been completed and comprehensively evaluates the user's performance. It then generates the next adventure plan taking into account the emotional data and sends it to the device. The device then displays the comprehensive evaluation screen, allowing the user to select the next adventure plan or exit the system.

[2154] Specific examples

[2155] Example: A 60-year-old user is interested in learning Japanese and Spanish culture.

[2156] The user starts the app, registers, and enters the required information. Then, on the profile settings screen, the user enters that they have an intermediate level of Japanese and are interested in Spanish culture.

[2157] The terminal transmits this information to the server and receives matching adventure plans from the server.

[2158] The server generates and transmits a virtual adventure plan for a "trip to Spain" based on the user's information.

[2159] The user begins the adventure and answers Japanese quizzes (e.g., "Please translate this sentence: Hola! ¿Cómo estás?").

[2160] The terminal transmits the quiz results to the server, and then displays the stretching exercise task.

[2161] The server evaluates the user's answers, monitors the progress of the motor task, and generates feedback. It also uses an emotion engine to analyze the user's facial expressions and voice to obtain emotional data.

[2162] The device displays feedback messages in real time and provides additional feedback based on emotions.

[2163] The user completes all tasks, receives a final evaluation, and decides whether to select the next adventure plan.

[2164] The system allows users to learn languages ​​while enjoying different cultures and promote health through exercise. It also uses an emotion engine to optimize the user experience to match each individual's emotional state, helping seniors improve their cognitive function and maintain their health without feeling isolated.

[2165] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2166] Step 1: User enters registration information

[2167] How it works: A user enters information such as their name, age, email address, and password on the app's new registration screen.

[2168] Input: User-entered data (name, age, email address, password)

[2169] Output: User information entered into the terminal

[2170] Step 2: The device sends the user information to the server

[2171] How it works: The device collects the information entered by the user, converts it to JSON format, and sends it to the server via an HTTP POST request.

[2172] Input: User information entered into the terminal

[2173] Output: User information sent to the server

[2174] Step 3: The server validates the user information and stores it in the database

[2175] Operation: The server validates the data received based on the validation rules, and if there are no problems, saves it in the database. It also generates a registration completion message and returns it to the terminal.

[2176] Input: Submitted user information

[2177] Output: User information saved in the database, registration completion message

[2178] Step 4: Your device will display a registration complete message

[2179] Operation: The terminal receives a registration completion message from the server and displays a dialog box informing the user that registration is complete.

[2180] Input: Registration completion message

[2181] Output: Notification to the user that registration is complete

[2182] Step 5: User Enters Profile Information

[2183] How it works: After logging in, users enter their language level, cultural interests, and exercise preferences in the profile settings screen.

[2184] Input: User-entered data (language level, cultural interests, exercise preferences)

[2185] Output: Profile information entered on the device

[2186] Step 6: The device sends the profile information to the server

[2187] How it works: The device converts the entered profile information into JSON format and sends it to the server via an HTTP POST request.

[2188] Input: Profile information entered on the device

[2189] Output: Profile information sent to the server

[2190] Step 7: The server saves the profile information to a database

[2191] Operation: The server validates the received profile information, stores it in a database, and associates the profile information with the user account.

[2192] Input: Submitted profile information

[2193] Output: Profile information stored in a database

[2194] Step 8: The server generates a virtual adventure plan

[2195] How it works: The server uses a generative AI model to generate an appropriate virtual adventure plan based on the user's profile information. The plan is then sent to the device.

[2196] Input: User profile information

[2197] Output: Generated virtual adventure plan, submitted adventure plan

[2198] Step 9: The device displays the adventure plan, and the user begins the adventure.

[2199] Operation: The terminal displays the virtual adventure plan received from the server and provides an interface for the user to click a "Start Adventure" button.

[2200] Input: Received adventure plan

[2201] Output: Adventure start screen, user starts adventure

[2202] Step 10: Server generates language learning quiz

[2203] How it works: The server uses a generative AI model to create language learning quizzes based on the user's progress and sends them to the device.

[2204] Input: User progress information

[2205] Output: Generated language learning quiz, submitted quiz

[2206] Step 11: The device displays the quiz and the user answers it.

[2207] How it works: A device displays language learning quizzes and provides an interface for users to enter answers.

[2208] Input: Received language learning quiz

[2209] Output: Quiz displayed, answers given by the user

[2210] Step 12: The device sends the answer to the server

[2211] How it works: The device converts the user's answers into JSON format and sends them to the server via an HTTP POST request.

[2212] Input: User's answer

[2213] Output: The answer sent to the server

[2214] Step 13: The server determines whether the quiz is correct or not and returns the result to the device.

[2215] Operation: The server judges the correctness of the received answer and returns the result to the terminal.

[2216] Input: Submitted answer

[2217] Output: Quiz results, data returned to the device

[2218] Step 14: The device displays the judgment result and the exercise task.

[2219] Operation: The device receives the judgment result from the server and displays it to the user, and then displays instructions for the next exercise task.

[2220] Input: Judgment result

[2221] Output: Display of judgment results, display of motor task instructions

[2222] Step 15: The user performs the motor task

[2223] Action: The user follows the displayed exercise instructions and performs the physical exercise.

[2224] Input: Motor task instructions

[2225] Output: The motor task performed

[2226] Step 16: Server generates feedback and captures emotion data

[2227] Operation: The server monitors the progress of language learning and motor tasks, generates real-time feedback, and utilizes an emotion engine to analyze the user's voice and facial expressions to obtain emotional data.

[2228] Input: Progress data, voice data, facial expression data

[2229] Output: Generated feedback, captured emotion data

[2230] Step 17: The device displays a feedback message and additional emotional feedback

[2231] Behavior: The device displays a feedback message from the server and additional feedback based on the emotion.

[2232] Input: Feedback message, additional feedback based on sentiment data

[2233] Output: The feedback message that is displayed to the user

[2234] Step 18: The server generates an overall rating and proposes the next adventure plan.

[2235] How it works: The server verifies that all tasks have been completed, evaluates the user's overall performance, and generates a next adventure plan taking into account the emotional data and sends it to the device.

[2236] Input: Task progress data, emotion data

[2237] Output: Generated overall evaluation, proposal for next adventure plan

[2238] Step 19: Your device will display your overall rating and options for your next adventure.

[2239] Operation: The device displays the overall rating from the server and presents the user with options for their next adventure plan.

[2240] Input: Overall rating, next adventure plan

[2241] Output: Overall rating and next adventure options displayed to the user

[2242] Step 20: User selects next adventure plan or exits the system

[2243] Action: The user selects the next adventure plan or exits the system.

[2244] Input: Next adventure plan option

[2245] Output: Next adventure plan selected or system termination

[2246] (Application example 2)

[2247] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2248] Systems that enable elderly people to learn languages ​​and promote their health while having fun are required to enhance the user experience and provide optimal feedback and content according to each individual's emotional state. However, existing systems lack the ability to appropriately recognize the user's emotional state and adapt feedback and content accordingly, which limits the user experience.

[2249] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2250] In this invention, the server includes means for collecting and storing user profile information, means for generating an appropriate virtual adventure plan based on the profile information, means for providing language learning tasks and exercise tasks based on the generated adventure plan, means for analyzing the user's emotional state using an emotion engine and adapting feedback and content, means for monitoring the progress of the language learning tasks and exercise tasks and providing feedback in real time, and means for generating an overall evaluation and suggesting the next adventure plan when the adventure is completed. This enables optimal feedback and content to be provided according to the user's emotional state, allowing elderly people to learn languages ​​more enjoyably and improve their health.

[2251] "User profile information" refers to information about a user's individual characteristics, interests, and abilities, and specifically includes language learning level, cultural interests, exercise preferences, and emotional data.

[2252] A "virtual adventure plan" is an activity plan for a virtual space that is generated based on the user's profile information, and includes language learning tasks and physical activities.

[2253] A "language learning task" is a quiz-style question given to a user, and is a challenge for the user to improve their language ability through their answers.

[2254] An "exercise task" is a physical exercise performed by a user according to exercise instructions displayed on the screen.

[2255] An "emotion engine" is a technology that analyzes a user's voice and facial expressions to recognize their emotional state.

[2256] "Feedback" refers to providing real-time evaluations and reactions to the user's actions and answers, and assisting the user in their learning or athletic progress.

[2257] "Emotional state" refers to the emotion the user is currently feeling, and is analyzed from voice, facial expressions, etc.

[2258] The "overall evaluation" is an index that evaluates the overall results of the language learning task and the motor task and indicates the user's performance.

[2259] The "next adventure plan" is a new virtual adventure plan that is proposed after the current adventure plan is completed.

[2260] The present invention is an interactive system that supports users in language learning and health promotion, and can provide optimal feedback and content according to the user's emotional state. The system implements various functions, including collecting and storing user profile information, generating virtual adventure plans, providing language learning tasks and exercise tasks, monitoring progress in real time, analyzing emotions using an emotion engine, generating overall ratings, and suggesting the next adventure plan.

[2261] Explaining program processing in natural language

[2262] The server performs the following process: Collects and stores the user's profile information (language learning level, cultural interests, exercise preferences, and emotional data). Generates an appropriate virtual adventure plan based on the user's profile information. This virtual adventure plan includes a language learning task and an exercise task. The language learning task involves presenting quiz-style questions, and the user improves their language ability through their answers. The exercise task involves performing exercises based on the exercise instructions displayed on the screen.

[2263] The server monitors the progress of language learning and motor tasks and provides real-time feedback. It also uses an emotion engine to analyze the user's voice and facial expressions to recognize their emotional state and adapt the feedback and content accordingly. By providing optimal feedback based on their emotional state, users can learn languages ​​and improve their health while having more fun. When the adventure is complete, the server evaluates the user's overall performance and suggests a plan for the next adventure.

[2264] The system runs on devices such as smartphones and tablets, and the server manages user data using cloud services such as AWS, Google Cloud, Firebase, and MongoDB. The emotion engine uses the Microsoft Azure Emotion API and Google Cloud Vision API.

[2265] Adding specific examples

[2266] As a concrete example, the following system usage scenario can be considered: If a 60-year-old user is interested in learning Japanese and Spanish culture, he or she will use the system in the following steps.

[2267] 1. User: Launch the app, register, and enter the required information. Then, on the profile settings screen, enter "Intermediate Japanese" and "Interested in Spanish culture."

[2268] 2. Terminal: Sends this information to the server and receives matching adventure plans from the server.

[2269] 3. Server: Based on the user's information, it generates and sends a virtual adventure plan for a trip to Spain.

[2270] 4. User: The adventure begins and the user answers a Japanese quiz (e.g., "Please translate this sentence: Hola! ¿Cómo estás?").

[2271] 5. Terminal: Sends the quiz results to the server and then displays the stretching exercise task.

[2272] 6. Server: Evaluates the user's answers, monitors the progress of the motor task, generates feedback, and uses an emotion engine to analyze the user's facial expressions and voice to obtain emotional data.

[2273] 7. Terminal: Display feedback messag...

Claims

1. means for collecting and storing user profile information; means for generating an appropriate virtual adventure plan based on the profile information; means for providing language learning tasks and motor tasks based on the generated adventure plan; means for monitoring progress of said language learning task and motor task and providing real-time feedback; When the adventure is completed, a means for generating a comprehensive evaluation and proposing a next adventure plan is provided. A system including:

2. 2. The system of claim 1, wherein the user profile information includes a level of language learning, cultural interests, and exercise preferences.

3. 2. The system of claim 1, wherein the language learning task includes quiz-style questions for the user and determining the answers to those questions, and the exercise task includes exercise based on exercise instructions displayed on a screen.

Images