System

Abstract

A system is provided.SOLUTION: A system comprising: means for collecting photo, calendar, and diary data from a user's terminal; artificial intelligence module means for analyzing the collected data and generating a quiz; means for delivering the generated quiz to the user's terminal; means for receiving the user's answer and determining whether the answer is correct or incorrect; means for evaluating the user's memory and cognitive risk based on the determination; and means for providing feedback to the user recommending an evaluation result and an appropriate action.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Dementia is a serious health problem common to both young and elderly people, and early detection and prevention are important. However, early symptoms of dementia are difficult to recognize, and people are reluctant to be told by others, so detection is often delayed. Furthermore, existing methods make it difficult for individual users to maintain enjoyment and motivation. Given this background, there is a need for a system that can monitor one's memory status in an enjoyable way on a daily basis and support the early detection and prevention of dementia. [Means for solving the problem]

[0005] The present invention provides a system including: means for collecting photo, calendar, and diary data from a user's device; an artificial intelligence module for analyzing the collected data and generating quizzes; means for delivering the generated quizzes to the user's device; means for receiving the user's answers and determining whether the answers are correct; means for assessing the user's memory and cognitive risk based on the assessment results; and means for providing the user with feedback recommending the assessment results and appropriate actions. This system allows users to check their own memory status while having fun on a daily basis and recognize cognitive risks early. Furthermore, the system allows users to take appropriate action by recommending a visit to a medical institution based on the assessment results.

[0006] "Device" means a computing device that a user uses on a daily basis, such as a smartphone or tablet.

[0007] "Data" refers to the information the system collects to track your memories and behavior, including photos, calendar events, diary entries, etc.

[0008] "Analysis" refers to the process of extracting meaningful information from collected data and generating a quiz.

[0009] "Artificial Intelligence Module" refers to the software component used to analyze data and automatically generate appropriate quizzes for users.

[0010] "Quiz" refers to content in the form of questions generated to stimulate the user's memory.

[0011] "Delivery" refers to the process of sending the generated quiz to the user's device and prompting them to answer.

[0012] "Answer" means a response provided by a User to a Quiz.

[0013] "Verification" refers to the process of evaluating whether a user's answer is correct.

[0014] "Memory and cognitive risk assessment" refers to the process of quantitatively or qualitatively assessing a user's memory ability and / or risk of developing dementia.

[0015] "Feedback" refers to the information provided to a User regarding the evaluation results and recommended actions.

[0016] "Recommendation to visit a medical institution" refers to the process of recommending that the user receive appropriate medical services based on the user's evaluation results.

[0017] "System" refers to a comprehensive component that includes multiple elements such as the above-mentioned terminals, data, analysis, artificial intelligence modules, quizzes, and distribution. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0019] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0022] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0023] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0024] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0026] [First embodiment]

[0027] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0028] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0031] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0034] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0038] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0039] Introduction

[0040] The present invention relates to a system that aims to activate a user's memory in a fun way and to detect and prevent dementia early. Specifically, the system generates quizzes based on data collected from the user's device and periodically evaluates the user's memory and cognitive status. The following describes in detail the embodiments of the present invention.

[0041] System configuration

[0042] This system mainly consists of the following components:

[0043] 1. User's Device

[0044] Devices that users use on a daily basis, such as smartphones and tablets.

[0045] The terminal performs the functions of collecting data, displaying quizzes, accepting answers, and providing feedback.

[0046] 2. Server

[0047] Analyzes data, generates quizzes, judges answers, and assesses memory and cognitive risk.

[0048] It is equipped with an artificial intelligence module that automates data analysis and quiz generation.

[0049] 3. Network

[0050] An internet connection that allows communication between your device and our server.

[0051] Program processing

[0052] Data collection

[0053] Server: Sends data collection requests to the user's device.

[0054] On your device: With your permission, we collect photos, calendar events, and diary data.

[0055] Terminal: Sends collected data to the server.

[0056] Examples:

[0057] Photos taken over the weekend, appointments on your calendar, and events recorded in your autobiography app.

[0058] Generate a quiz

[0059] Server: Analyzes the received data using an artificial intelligence module and generates a quiz to stimulate the user's memory.

[0060] Server: Sends the generated quiz to the user's device.

[0061] Examples:

[0062] Photo analysis results: "Where was this photo taken?"

[0063] Calendar event analysis results: "What events happened on this day?"

[0064] Quiz distribution and answers

[0065] On the device: Show the quiz notification to the user and open the quiz screen.

[0066] User: Answers the quiz by selecting options or entering text in response to the questions presented.

[0067] Terminal: Sends the user's answer to the server.

[0068] Examples:

[0069] Choose a location from the options for the photo question.

[0070] Enter detailed text for calendar events.

[0071] Judging and evaluating answers

[0072] Server: Compares the received answer with the correct answer database and determines whether it is correct or not.

[0073] Server: Records the results of the judgment and calculates values ​​to assess the user's memory and cognitive risk.

[0074] Examples:

[0075] If the user answers "park" and the correct answer is "park," it is determined to be the correct answer.

[0076] Based on the memory risk assessment, it is compared with the user's past answer history.

[0077] Providing Feedback

[0078] Server: Generates feedback to the user based on the judgment results.

[0079] Server: Sends feedback to the user's device.

[0080] Terminal: Show a feedback screen to the user.

[0081] Examples:

[0082] If the correct answer rate is high, the feedback is "You have maintained a good memory."

[0083] If the perceived risk is high, you will be notified that "we recommend that you visit a medical institution."

[0084] Example

[0085] User A launches Recoco Memory for the first time and allows the app to access photos, calendar, and diary information. The server receives the initial data and begins analysis. Based on the analysis results, multiple quizzes are generated from User A's photos and calendar events from the past week. For example, the server generates a quiz such as "Where did you visit last weekend?" and sends it to the user's device.

[0086] When User A answers the quiz, the answer data is sent to the server for analysis and evaluation. The server determines whether User A's answer is correct and evaluates the memory status based on the result. The evaluation result is sent as feedback to User A's device, and User A receives a notification that their memory status is "good."

[0087] In this way, by using the system of the present invention, users can enjoy checking their memories and assessing cognitive risk in their daily lives.

[0088] The processing flow will be explained below.

[0089] Step 1:

[0090] Users: Download and install the app on their devices such as smartphones and tablets.

[0091] Step 2:

[0092] Device: When the app is launched for the first time, a user registration screen is displayed and the user is asked to enter personal information (such as name, email address, and password).

[0093] Step 3:

[0094] Server: Receives user registration information, creates a new user account in the database, and sends a success message to the terminal.

[0095] Step 4:

[0096] On device: Prompt the user for permission to access their photos, calendar, and diary data.

[0097] Step 5:

[0098] User: Select "Allow" in the permission dialog to grant the app data access.

[0099] Step 6:

[0100] Device: Collects data for which permission has been granted (photos, calendar, diary information) and sends it to the server.

[0101] Step 7:

[0102] Server: The AI ​​module analyzes the received data and generates quizzes based on the photos and calendar information.

[0103] For example, generate questions based on photo locations or calendar events.

[0104] Step 8:

[0105] Server: Sends the generated quiz to the user's device.

[0106] Step 9:

[0107] On the device: A quiz notification is displayed to the user. When the user taps the notification, the quiz screen is displayed.

[0108] Step 10:

[0109] User: Answers the quiz by selecting options or entering text in response to the questions presented.

[0110] Step 11:

[0111] Terminal: Sends the user's answer to the server.

[0112] Step 12:

[0113] Server: Compares the received answer with the correct answer database and determines whether it is correct or not.

[0114] Step 13:

[0115] Server: Records the results of the judgment and calculates values ​​to assess the user's memory and cognitive risk.

[0116] Example: Calculating accuracy rate and memory risk score.

[0117] Step 14:

[0118] Server: Generates feedback to provide to the user based on the judgment results.

[0119] Step 15:

[0120] Server: Sends feedback to the user's device.

[0121] Step 16:

[0122] Device: A feedback screen is displayed to the user, presenting the evaluation results and recommended actions (e.g., visiting a medical institution).

[0123] Step 17:

[0124] User: Review the feedback and take any recommended action (e.g., seek medical attention) if necessary.

[0125] This series of steps allows users to have fun while checking their memories and assessing cognitive risk.

[0126] Example 1

[0127] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0128] There is a lack of efficient systems to stimulate users' memories in an enjoyable way and to detect and prevent dementia early. In addition, there are insufficient means to accurately assess users' memory status and cognitive risk and provide feedback.

[0129] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0130] In this invention, the server includes means for collecting image data, schedule management data, and record data from the user's computing device, means for analyzing the collected data and using a machine learning algorithm to generate questions, and means for delivering the generated questions to the user's computing device, thereby making it possible to automatically generate questions to stimulate the user's memory, evaluate memory and cognitive risks based on the answers, and provide appropriate feedback.

[0131] "Computing device" refers to an information processing device that users use on a daily basis, and specifically includes smartphones, tablets, personal computers, etc.

[0132] "Image data" refers to photographs taken by users and image files acquired by users.

[0133] "Schedule management data" refers to calendar application data that records a user's schedule and appointments.

[0134] "Recorded data" refers to text data recorded by the user as a diary or memo.

[0135] "Machine learning algorithm" refers to an algorithm used to analyze collected data and generate questions to stimulate a user's memory, and may include an artificial intelligence module.

[0136] "Feedback" refers to information and advice provided to users based on the results of an assessment of their memory status and cognitive risk.

[0137] "Medical institution" refers to a medical facility such as a hospital or clinic that provides medical services.

[0138] MODE FOR CARRYING OUT THE INVENTION

[0139] Introduction

[0140] The present invention relates to a system for activating a user's memory in a fun way, with the aim of early detection and prevention of dementia. The system generates quizzes based on data collected from the user's computing device and periodically assesses the user's memory and cognitive status. Specific embodiments for implementing the invention are described in detail below.

[0141] System configuration

[0142] This system mainly consists of the following components:

[0143] 1. Your Computing Device

[0144] Information processing devices such as smartphones, tablets, and personal computers.

[0145] The computing device has the functions of collecting data, displaying quizzes, accepting answers, and providing feedback.

[0146] 2. Server

[0147] Analyzes data, generates quizzes, judges answers, and assesses memory and cognitive risk.

[0148] It is equipped with an artificial intelligence module that automates data analysis and quiz generation (e.g., GPT-3).

[0149] 3. Network

[0150] An internet connection that allows communication between the computing device and the server.

[0151] Data collection

[0152] The server sends a data collection request to the user's computing device. Based on the user's permission, the computing device collects image data, schedule management data, and record data. The computing device then sends the collected data to the server. Specific examples include photos taken by the user over the weekend, appointments listed on a calendar, and events recorded in a personal history app.

[0153] Generate a quiz

[0154] The server analyzes the received data using an artificial intelligence module and generates a quiz to stimulate the user's memory. The server then sends the generated quiz to the user's computing device. Specific examples include quizzes such as "Where was this photo taken?" based on the analysis of a photo, or "What events happened on this day?" based on the analysis of calendar events.

[0155] Quiz distribution and answers

[0156] The computing device displays a quiz notification to the user and opens the quiz screen. The user answers the quiz by selecting options or entering text in response to the presented questions. The computing device then sends the user's answers to the server. Specific examples of answers include selecting a location from options in response to a photo question or entering detailed actions in response to a calendar event.

[0157] Judging and evaluating answers

[0158] The server compares the received answer with a correct answer database to determine whether it is correct or incorrect. The server then records the result of the determination and calculates a value to evaluate the user's memory and cognitive risk. For example, if the user answers "park" and the correct answer is "park," the answer is determined to be correct. Furthermore, based on the memory risk assessment, the result is compared with the user's past answer history.

[0159] Providing Feedback

[0160] The server generates feedback for the user based on the assessment results. The server then sends the generated feedback to the user's computing device. The computing device displays a feedback screen to the user. For example, if the accuracy rate is high, feedback such as "Your memory is in good condition" is displayed, and if the cognitive risk is high, a notification such as "We recommend that you visit a medical institution" is displayed.

[0161] Example

[0162] User A launches the system for the first time and allows the app to access image data, schedule management data, and record data. The server receives the initial data and begins analysis. Based on the analysis results, multiple quizzes are generated from User A's photos and calendar events from the past week. For example, the server generates a quiz question such as, "Where did you visit last weekend?" and sends it to the user's computing device. When User A answers the quiz, the answer data is sent to the server for analysis and evaluation. The server determines whether User A's answer is correct and evaluates his or her memory status based on the result. The evaluation result is sent as feedback to User A's computing device, and User A is notified that his or her memory status is "good." In this way, using the system of the present invention, users can enjoy checking their memory and assessing cognitive risk in their daily lives.

[0163] Examples of prompt statements

[0164] "Generate quizzes from the user's photo data collected by the system, deliver the generated quizzes to the user's computing device, and provide feedback based on the results."

[0165] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0166] Step 1: Submit a data collection request

[0167] The server sends a data collection request to the user's computing device, the request including information directing the collection of image data, schedule management data, and record data.

[0168] Input: None

[0169] Output: Data collection request

[0170] Step 2: Data collection

[0171] The device collects image data, schedule management data, and record data from the user based on the data collection request received from the server. Specifically, it obtains photos from the camera roll of the smartphone or tablet, schedules from the calendar app, and text information from the diary app.

[0172] Input: Data Collection Request

[0173] Output: Collected data (photos, schedules, diaries)

[0174] Step 3: Sending data

[0175] The terminal transmits collected image data, schedule management data, and record data to the server. This data is transmitted based on the user's permission.

[0176] Input: Collected data

[0177] Output: Send data to the server

[0178] Step 4: Data analysis

[0179] The server analyzes the received data using a generative AI model (e.g., GPT-3). This analysis includes photo recognition, text analysis, and calendar event identification. Specifically, image recognition algorithms are used to analyze the content of photos, and natural language processing algorithms are used to analyze text data.

[0180] Input: Collected data

[0181] Output: Analysis results

[0182] Step 5: Quiz Generation

[0183] The server generates quizzes to stimulate the user's memory based on the analysis results. For example, it generates questions such as "Where was this photo taken?" based on the location information of the photo. It also generates questions such as "What events happened on this day?" based on calendar events.

[0184] Input: Analysis results, generated AI model

[0185] Output: The generated quiz

[0186] Step 6: Stream the quiz

[0187] The server then sends the generated quiz to the user's computing device, formatted in a way that is easy for the user to answer.

[0188] Input: Generated quiz

[0189] Output:Quiz delivery

[0190] Step 7: Quiz Notification and Display

[0191] The device notifies the user of the quiz received from the server and displays the quiz screen. Specifically, the device notifies the user of the existence of a new quiz using the notification center or in-app notifications, and displays the quiz screen.

[0192] Input:Quiz broadcast

[0193] Output: Quiz notification to user, quiz screen display

[0194] Step 8: Take the quiz

[0195] The user can select an answer from the options or enter a text answer for the quiz. Specifically, the user selects the correct answer from the options displayed or enters the answer using the keyboard.

[0196] Input: Quiz screen

[0197] Output: User's answer

[0198] Step 9: Submit your answers

[0199] The device sends the user's answers to the server, where they are accurately recorded and securely transmitted to the server.

[0200] Input: User's answer

[0201] Output: Send the answer to the server

[0202] Step 10: Determine the answer

[0203] The server compares the received answer with a database of correct answers to determine whether the answer is correct or not. Specifically, it matches the user's answer with the correct answers stored in the database.

[0204] Input: User's answer

[0205] Output: Judgment result

[0206] Step 11: Valuation Calculation

[0207] The server calculates a numerical value to evaluate the user's memory and cognitive risk based on the judgment result. For example, the server evaluates the user's memory ability and cognitive risk based on the accuracy rate and reaction time, and generates a score.

[0208] Input: Judgment result

[0209] Output: Evaluation score

[0210] Step 12: Generate feedback

[0211] The server generates feedback for the user based on the evaluation score, such as a message indicating good memory status or a message recommending that the user seek medical attention.

[0212] Input: Rating score

[0213] Output: Feedback message

[0214] Step 13: Submit your feedback

[0215] The server then transmits the generated feedback to the user's computing device, where the feedback is user-friendly and encourages action.

[0216] Input: Feedback message

[0217] Output: Send feedback to the user

[0218] Step 14: Viewing feedback

[0219] The device displays the feedback message received from the server to the user, specifically, by providing the feedback to the user using an in-app notification or a pop-up window.

[0220] Input: Send Feedback

[0221] Output: Show feedback

[0222] (Application example 1)

[0223] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0224] In an aging society, the prevention and early detection of dementia are important issues. However, existing methods lack systems that allow users to enjoyably assess memory in their daily lives and determine the risk of cognitive function without visiting a medical institution. Furthermore, even in physical stores, there are no systems that promote memory activation through the shopping experience, so there is a need to integrate dementia prevention and shopping.

[0225] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0226] In this invention, the server includes means for collecting photo, calendar, and diary data, as well as purchase and check-in histories from the user's device, an artificial intelligence module for analyzing the collected data and generating quizzes, and means for delivering and notifying the generated quizzes to the user's device. This enables the user's memory to be stimulated in an enjoyable way through everyday shopping experiences, thereby enabling the prevention and early detection of dementia.

[0227] "User device" refers to a device such as a smartphone, tablet, or smart glasses that a user uses on a daily basis to collect data, display quizzes, and accept answers.

[0228] "Data collection" is the process of collecting purchase history, check-in history, photos, calendar, and diary data from users' devices.

[0229] "Purchase history" refers to a record and detailed information about products purchased by a user in a physical store.

[0230] "Check-in history" is a record of a user's visits to specific locations or stores, and is data that includes location information.

[0231] The "artificial intelligence module" is a system consisting of programs and algorithms for analyzing collected data and generating quizzes to stimulate the user's memory.

[0232] "Quiz generation" is the process of automatically creating questions to assess a user's memory based on collected and analyzed data.

[0233] "Quiz delivery" is the process of sending the generated quiz to the user's device and notifying them.

[0234] "Answer determination" is the process of analyzing the answers given by the user to the quiz and comparing them with the correct answers to determine whether they are correct or incorrect.

[0235] "Memory status assessment" is the process of assessing a user's memory and cognitive function by comparing it with the user's past answer history.

[0236] "Feedback" is the process of informing the user of information about their memory and cognitive status and appropriate actions based on their answers and the assessment results.

[0237] "Recommendation to visit a medical institution" is the process of notifying the user to visit a medical institution based on the evaluation results.

[0238] "Quantification" is the process of converting evaluation results and answer history into quantitative data to make it easier to evaluate the user's memory state.

[0239] Configuring the various components

[0240] The present invention relates to a system for activating a user's memory and assessing their cognitive status. The system comprises the following components:

[0241] 1. User's Device

[0242] These are devices that users use on a daily basis (e.g., smartphones, tablets, smart glasses, etc.). These devices collect data such as users' purchase history, check-in history, photos, calendars, and diaries. They also perform functions such as displaying quizzes, accepting answers, and providing feedback.

[0243] 2. Server

[0244] The server analyzes data, generates quizzes, judges answers, and assesses memory and cognitive risk. It is equipped with an artificial intelligence module to automate these processes, and often uses cloud services such as Google Cloud and Amazon Web Services (AWS).

[0245] 3. Network

[0246] Refers to an internet connection that enables communication between a device and a server, and can use Wi-Fi, 4G, 5G, or other mobile networks.

[0247] Data collection

[0248] 1. Terminal

[0249] With user permission, data such as purchase history, check-in history, photos, calendar events, and diary entries are collected and transmitted securely to the server.

[0250] Generate a quiz

[0251] 2. Server

[0252] The received data is analyzed using an artificial intelligence module (e.g., TensorFlow, PyTorch) to generate a quiz to stimulate the user's memory. The quiz is automatically created and sent to the user's device.

[0253] Quiz distribution and answers

[0254] 3. Terminal

[0255] A quiz notification is displayed to the user, and the quiz screen opens. The user answers the quiz by selecting options or entering text in response to the questions presented.

[0256] 4. Users

[0257] The user answers the quiz appropriately and transmits the answers to the server via the terminal.

[0258] Judging and evaluating answers

[0259] 5. Server

[0260] The received answer is compared with a correct answer database to determine whether it is correct or incorrect. The result of the determination is then recorded and compared with past answer history to evaluate the memory status.

[0261] Providing Feedback

[0262] 6. Server

[0263] Based on the results of the assessment, feedback is generated for the user, including an assessment of memory status and recommended actions (e.g., a recommendation to see a doctor). The generated feedback is sent to the user's device.

[0264] 7. Terminal

[0265] A feedback screen is displayed to the user, allowing them to check their memory status and recommended actions.

[0266] Specific examples

[0267] When a user is shopping in a physical store, they may purchase a specific item at the register (a retro-designed drink that they have purchased in the past). The system collects this data and generates a quiz based on their purchase history, asking, "What flavor was the retro cola you purchased two weeks ago?" If the user answers the quiz correctly, the system provides feedback such as, "Your memory is good!"

[0268] Prompt Sentence Examples

[0269] "What flavor of retro cola did you buy two weeks ago?"

[0270] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0271] Step 1:

[0272] Data collection

[0273] Device: First, with the user's permission, data such as purchase history, check-in history, photos, calendar events, and diary entries are collected. This data is either collected automatically by the application or entered manually by the user. The collected data is converted into an appropriate data format, such as JSON, and sent to the server while protecting privacy.

[0274] Input: User purchase history, check-in history, photos, calendar events, diary data

[0275] Output: JSON data formatted to send to the server

[0276] Step 2:

[0277] Generate a quiz

[0278] Server: Analyzes the received data. Specifically, it extracts information for generating quizzes from the collected data through text analysis (natural language processing) and image analysis. It uses artificial intelligence modules (e.g., TensorFlow, PyTorch) to automatically generate quizzes that stimulate the user's memory.

[0279] Input: JSON data sent from the terminal

[0280] Output: Generated quiz questions and choices

[0281] Step 3:

[0282] Quiz broadcast

[0283] Server: The server delivers the generated quiz to the user's device and sends a notification. The quiz content is pushed to the device using the REST API.

[0284] On the device: The received notification is displayed to the user and the quiz screen is opened. When the user taps the notification, the specific quiz content is displayed.

[0285] Input: Generated quiz

[0286] Output: Quiz notification and quiz screen

[0287] Step 4:

[0288] Quiz Answers

[0289] User: Answers the quiz by selecting an answer from the options provided or by entering text.

[0290] Terminal: Receives the user's answer and sends it to the server. The data sent is in JSON format, including the answer.

[0291] Input: User's quiz answer

[0292] Output: Answer data to send to the server

[0293] Step 5:

[0294] Judging and evaluating answers

[0295] Server: Compares the received user answers with a database of correct answers to determine whether they are correct or not. It also compares the answers with the user's past answer history to assess memory status and cognitive risk. Statistical analysis and machine learning algorithms are used for this.

[0296] Input: User's answer data

[0297] Output: Correct / incorrect judgment and memory status evaluation results

[0298] Step 6:

[0299] Providing Feedback

[0300] Server: Generates feedback to the user based on the correctness of the answers and the evaluation results. The feedback includes an evaluation of memory status and recommended actions (e.g., recommendation to see a doctor).

[0301] Device: Receives the generated feedback and displays it to the user, allowing the user to check their own memory status and course of action.

[0302] Input: Correct / incorrect judgment and memory status evaluation results

[0303] Output: Feedback that is displayed on the user's device

[0304] Examples of prompt statements

[0305] "What flavor of retro cola did you buy two weeks ago?"

[0306] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0307] Introduction

[0308] The present invention relates to a system that aims to activate a user's memory in a fun way and to detect and prevent dementia early. In particular, the present invention provides a more personalized service by recognizing the user's emotions and adjusting the quiz generation and feedback content.

[0309] System configuration

[0310] The system consists of the following main components:

[0311] 1. User's Device

[0312] These are devices that users use on a daily basis, such as smartphones and tablets.

[0313] It performs functions such as data collection, emotion recognition, quiz display, answer acceptance, and feedback provision.

[0314] 2. Server

[0315] Analyzes data, generates quizzes, judges answers, and assesses memory and cognitive risk.

[0316] It is equipped with an artificial intelligence module and an emotion engine to automate data analysis and quiz generation.

[0317] 3. Network

[0318] It is an internet connection that allows communication between the device and the server.

[0319] Program processing

[0320] Data collection and emotion recognition

[0321] Server: Sends data collection requests to the user's device.

[0322] On the device: With the user's permission, it collects photos, calendar events, and diary data, and also uses the device's camera and microphone to capture the user's facial expressions and voice, which are then sent to the emotion engine.

[0323] Emotion engine: Analyzes collected facial and voice data to recognize the user's emotions.

[0324] Terminal: Sends collected data and sentiment analysis results to the server.

[0325] Examples:

[0326] When a user takes a photo or records an event in the calendar, the device sends this data to the server, and at the same time collects the user's facial expressions and voice and sends them to the emotion engine.

[0327] Quiz Generation

[0328] Server: Analyzes the received data using an artificial intelligence module and generates a quiz based on the user's memory and emotional state.

[0329] Server: Adjusts the content and difficulty of the quiz based on the emotions recognized by the emotion engine.

[0330] For example: if the user is having fun, generate a challenging quiz, if the user is stressed, generate an easy quiz.

[0331] Examples:

[0332] Photo analysis results: "Where was this taken?"

[0333] Calendar analysis results: "What events were there on this day?"

[0334] Quiz distribution and answers

[0335] Server: Sends the generated quiz to the user's device.

[0336] Device: Show the quiz notification to the user and transition to the quiz screen.

[0337] User: Answers the quiz by selecting options or entering text in response to the questions presented.

[0338] Terminal: Sends the user's answer to the server.

[0339] Examples:

[0340] A user receives a quiz notification and selects the correct answer to the question, "Where was this photo taken?"

[0341] Judging and evaluating answers

[0342] Server: Compares the received answer with the correct answer database and determines whether it is correct or not.

[0343] Server: Combines the judgment results with data from the emotion engine to calculate a numerical value for assessing the user's memory and cognitive risk.

[0344] Example: Calculating the accuracy rate and memory risk score.

[0345] Examples:

[0346] If the user answers "park" and it is the correct answer, it is determined to be the correct answer and compared with the past answer history to evaluate the memory risk.

[0347] Providing Feedback

[0348] Server: Generates personalized feedback based on the judgment results and emotion data.

[0349] Server: Sends feedback to the user's device.

[0350] Device: A feedback screen is displayed to the user, presenting the evaluation results and recommended actions (e.g., visiting a medical institution).

[0351] Examples:

[0352] If the correct answer rate is high, the system will provide feedback such as "Your memory is in good condition," and if the emotional data indicates that the user is under stress, it will also display advice such as "Relax and try the next quiz."

[0353] Example

[0354] User B launched Recoco Memory for the first time and allowed the app to access photos, calendar, and diary information. He also allowed the app to access the camera and microphone for emotion recognition. The server received the initial data and emotion data and began analysis. Based on the analysis results, multiple quizzes were generated from User B's photos and calendar events from the past week. For example, the server generated a quiz asking, "Where did you visit last weekend?" and selected a slightly more challenging quiz based on the user's emotional state (enjoyment).

[0355] When User B answers the quiz, the answer data and the latest emotional data are sent to the server for analysis and evaluation. The server determines whether User B's answer is correct and evaluates his / her memory and emotional state based on the result and the emotional data. As feedback, the device displays a notification saying "Good memory state" and "Enjoy the next quiz."

[0356] In this way, by using the system of the present invention, users can not only have fun while checking their memory and assessing cognitive risk, but also receive personalized services that take into account their emotional state.

[0357] The processing flow will be explained below.

[0358] Step 1:

[0359] User: Download and install Ricoco Memory on a device such as a smartphone or tablet.

[0360] Step 2:

[0361] Device: When the app is launched for the first time, a user registration screen is displayed and the user is asked to enter personal information (such as name, email address, and password).

[0362] Step 3:

[0363] Server: Receives user registration information, creates a new user account in the database, and sends a success message to the terminal.

[0364] Step 4:

[0365] Device: Prompts the user for permission to access photos, calendar, diary data, and the camera and microphone.

[0366] Step 5:

[0367] User: Select "Allow" in the permission dialog to grant the app data access.

[0368] Step 6:

[0369] Device: Collects data with permission (photos, calendar events, diary information) and sends it to the server. It also uses the camera and microphone to transmit the user's facial expressions and voice to the emotion engine.

[0370] Step 7:

[0371] Emotion engine: Analyzes collected facial and voice data to recognize the user's emotions.

[0372] Step 8:

[0373] Terminal: Sends collected data and sentiment analysis results to the server.

[0374] Step 9:

[0375] Server: Analyzes the received data using an AI module and generates a quiz based on the user's memory and emotional state. The emotion engine adjusts the content and difficulty of the quiz based on the emotions recognized by the server.

[0376] For example: if the user is having fun, generate a challenging quiz, if the user is stressed, generate an easy quiz.

[0377] Step 10:

[0378] Server: Sends the generated quiz to the user's device.

[0379] Step 11:

[0380] Device: Show the quiz notification to the user and transition to the quiz screen.

[0381] Step 12:

[0382] User: Answers the quiz by selecting options or entering text in response to the questions presented.

[0383] Step 13:

[0384] Terminal: Sends the user's answer to the server.

[0385] Step 14:

[0386] Server: Compares the received answer with the correct answer database and determines whether it is correct or not.

[0387] Step 15:

[0388] Server: Combines the judgment results with data from the emotion engine to calculate a numerical value for assessing the user's memory and cognitive risk.

[0389] Example: Calculating the accuracy rate and memory risk score.

[0390] Step 16:

[0391] Server: Generates personalized feedback based on the judgment results and emotion data.

[0392] Step 17:

[0393] Server: Sends feedback to the user's device.

[0394] Step 18:

[0395] Device: A feedback screen is displayed to the user, presenting the evaluation results and recommended actions (e.g., visiting a medical institution).

[0396] Step 19:

[0397] User: Review the feedback and take any recommended action (e.g., seek medical attention) if necessary.

[0398] Through this series of steps, users can not only enjoyably check their memories and assess their cognitive risks, but also receive personalized services that take into account their emotional state.

[0399] Example 2

[0400] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0401] In modern society, early detection and prevention of dementia are important issues, especially in the health management of the elderly. However, conventional memory assessment tools are monotonous, and users lack motivation to continue using them. Furthermore, there is a need for systems that do not take into account the user's emotional state and can be used in a fun and stress-free manner. Furthermore, there is a lack of personalized feedback and recommendations for appropriate actions tailored to individual needs. In response to these challenges, the present invention provides a system that allows users to activate their memory in an enjoyable way and provides personalized services based on their emotional state.

[0402] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0403] In this invention, the server includes means for collecting photo, schedule, and document data from the user's terminal, means for analyzing the collected data and the user's facial expressions and voice to recognize the emotional state, an artificial intelligence module for generating quizzes based on the analyzed data and the emotional state, means for adjusting the content and difficulty of the generated quizzes based on the emotional state, means for delivering the generated quizzes to the user's terminal, means for receiving the user's answers and determining whether the answers are correct, means for assessing the user's memory and cognitive risk based on the assessment results and the emotional state, and means for providing the user with feedback recommending the assessment results and appropriate actions. This allows the user to have fun while activating their memory and receiving personalized feedback and recommended actions.

[0404] "User's device" refers to a device that a user uses on a daily basis, such as a smartphone or tablet.

[0405] "Photo" refers to a still image taken by a User using a Device.

[0406] "Calendar" refers to the schedule information that a user records on a device.

[0407] "Document" refers to text data that a user creates or saves using a device.

[0408] "Means of collecting data" refers to the functionality for obtaining data such as photos, schedules, and documents from the user's device.

[0409] "Facial expressions and voice" refers to the user's facial expressions and spoken voice data.

[0410] "Means for recognizing emotional states" refers to a function that analyzes collected facial and voice data to identify the user's emotions (happiness, sadness, surprise, etc.).

[0411] "Artificial intelligence module means" refers to a processing system that uses artificial intelligence to analyze data and generate quizzes.

[0412] "Quiz" refers to questions or problems designed to assess a user's memory or cognitive abilities.

[0413] "Means for adjusting the content and difficulty of the quiz" refers to a function for appropriately adjusting the content and difficulty of the quiz questions based on the user's emotional state.

[0414] "Means for delivering quizzes" refers to the function for sending the generated quiz to the user's terminal.

[0415] "Means for receiving answers" refers to a function for transmitting the answers given by the user to the quiz from the user's terminal to the server and receiving them.

[0416] "Means for determining whether the answer is correct" refers to a function for comparing the received answer with pre-set correct answer data and determining whether the answer is correct or incorrect.

[0417] "Means for assessing memory and cognitive risk" refers to a function for assessing the risk of a user's memory and cognitive state based on the correctness or incorrectness of answers and emotional state.

[0418] "Means for providing feedback" refers to a function for generating feedback to recommend appropriate actions to the user based on the evaluation results and sending it to the user's device.

[0419] This invention relates to a system that aims to stimulate a user's memory in a fun way and to detect and prevent dementia early. In particular, it provides a more personalized service by recognizing the user's emotions and adjusting the quiz generation and feedback content.

[0420] System configuration

[0421] The system consists of the following main components:

[0422] 1. User's Device

[0423] These are devices that users use on a daily basis, such as smartphones and tablets. Specific hardware examples include Android and iOS-based devices.

[0424] Software: A dedicated application that collects data, recognizes emotions, displays quizzes, accepts answers, and provides feedback.

[0425] 2. Server

[0426] The server has the functionality to analyze data, generate quizzes, judge answers, and assess memory and cognitive risk.

[0427] Software: Artificial intelligence module and emotion engine, which automates data analysis and quiz generation.

[0428] 3. Network

[0429] An internet connection that allows communication between a device and a server, allowing data to be exchanged.

[0430] Data collection and emotion recognition

[0431] Server: Sends a data collection request to the user's device. The user's device receives this request and, after obtaining the user's permission, collects photo, calendar, and document data. It also uses the device's camera and microphone to collect the user's facial and voice data and sends it to the emotion engine. The emotion engine analyzes this data and recognizes the user's emotions.

[0432] Examples:

[0433] Users take everyday photos with their smartphone camera and register travel events in their calendar. The device sends this data to the server and simultaneously collects the user's facial expressions and voice. The emotion engine detects "enjoyment" from the user's smile, tone of voice, etc.

[0434] Generate a quiz

[0435] Server: Receives the results from the emotion engine and generates a quiz based on the user's memory and emotional state.

[0436] Artificial Intelligence Module: This determines the content and difficulty of the questions. Quizzes are created using a generative AI model, with example prompts such as:

[0437] Example prompt sentence:

[0438] "Generate the following quiz based on user A's photo data. User A's emotional state seems to be fun. The content of the quiz should be something that user A would enjoy. Specific example questions include, 'Where was this photo taken?' and 'What was this event?'"

[0439] The generated quizzes are adjusted in content and difficulty based on the emotional state.

[0440] Quiz distribution and answers

[0441] Server: Sends the generated quiz to the user's device.

[0442] Device: Show the quiz notification to the user and transition to the quiz screen.

[0443] Examples:

[0444] Users are given a quiz asking, "Where was this photo taken?" and are asked to select the appropriate location.

[0445] Judging and evaluating answers

[0446] Server: Receives the user's answer data, compares it with the correct answer database, and determines whether it is correct or incorrect. The result of the evaluation is combined with the emotion engine data to calculate a value that evaluates the user's memory and cognitive risk.

[0447] Examples:

[0448] If the user answers "park," if that is the correct answer, it is determined to be the correct answer and compared with past answer history to assess memory risk.

[0449] Providing Feedback

[0450] Server: Generates personalized feedback based on the judgment results and emotion data. The feedback is sent to the device and displayed on the feedback screen.

[0451] Examples:

[0452] If the correct answer rate is high, the system will provide feedback such as "Your memory is in good condition," and if it determines that the user is feeling stressed, it will advise them to "relax and try the next quiz."

[0453] This allows users to have fun while activating their memories and receiving personalized feedback.

[0454] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0455] Step 1:

[0456] Server: Sends data collection requests to the device.

[0457] Input: None

[0458] Output: Sending a data collection request

[0459] Specific operation: The server sends data collection instructions to the user's device via an HTTP request, which includes the specific data to collect (photos, schedules, documents) and how to collect it.

[0460] Step 2:

[0461] Device: Obtains user permission to collect data. If the user gives permission, the device will collect photos, calendar, and document data. It will also use the device's camera and microphone to collect facial and voice data.

[0462] Input: Data collection request from the server

[0463] Output: Collected data (photos, schedules, documents, facial expressions, voice)

[0464] How it works: The device running the app collects photos taken by the user, event information registered in the calendar, and document data, while also using the camera and microphone to collect real-time facial expressions and voice recordings.

[0465] Step 3:

[0466] Emotion engine: Analyzes collected facial and voice data to recognize the user's emotional state.

[0467] Input: Collected facial expression and voice data

[0468] Output: Emotion recognition result

[0469] Specific operation: The device sends the facial expression data captured by the device to an image recognition algorithm, and the recorded voice data to a voice analysis engine, which then recognizes the user's emotions (happiness, sadness, surprise, etc.).

[0470] Step 4:

[0471] Terminal: Sends collected data and sentiment analysis results to the server.

[0472] Input: Collected data and emotion recognition results

[0473] Output: Send data to the server

[0474] What happens: The device converts the data into the appropriate format and sends it to the server. The data is sent via an HTTP POST request, subject to user permission.

[0475] Step 5:

[0476] Server: Inputs the received data into an artificial intelligence module and generates a quiz based on the user's memory and emotional state.

[0477] Input: Collected data and emotion recognition results

[0478] Output: The generated quiz

[0479] How it works: The server inputs data into the AI ​​module and generates a quiz, such as "Based on your calendar events, what were the events on this day?"

[0480] Step 6:

[0481] Server: Adjusts the content and difficulty of the quiz based on the emotions recognized by the emotion engine.

[0482] Input: Generated quiz and emotion recognition results

[0483] Output: Adjusted quiz

[0484] What it does: If the user is having fun, choose a challenging quiz; if the user is stressed, choose an easy quiz.

[0485] Step 7:

[0486] Server: Sends the generated quiz to the user's device.

[0487] Input: Adjusted Quiz

[0488] Output: Quiz stream

[0489] Specific operation: The server sends the quiz in JSON format to the device, and the device receives it and displays a quiz notification.

[0490] Step 8:

[0491] Device: Show the quiz notification to the user and transition to the quiz screen.

[0492] Input: Quiz Stream

[0493] Output: Quiz screen display

[0494] What happens: When the user clicks on the notification, the application transitions to a quiz screen where the question is displayed.

[0495] Step 9:

[0496] User: Answers the quiz by selecting options or entering text in response to the questions presented.

[0497] Input: Question on the quiz screen

[0498] Output: User's answer

[0499] Specific behavior: The user selects an answer from the options and clicks the "Submit" button.

[0500] Step 10:

[0501] Terminal: Sends the user's answer to the server.

[0502] Input: User's answer

[0503] Output: Send the answer to the server

[0504] Specific operation: The device converts the answer data into an appropriate format and sends it to the server via an HTTP POST request.

[0505] Step 11:

[0506] Server: Compares the received answer data with the correct answer database to determine whether it is correct or not.

[0507] Input: User's answer

[0508] Output: Judgment result

[0509] Specific operation: The server compares the received answer with the correct answer data and determines whether it is correct or incorrect.

[0510] Step 12:

[0511] Server: Combines the judgment results and data from the emotion engine to assess the user's memory and cognitive risk.

[0512] Input: Judgment results and emotion recognition results

[0513] Output: Memory and cognitive risk assessment

[0514] Specific operation: The server calculates an evaluation score based on the correct answer rate, past answer history, and emotional data.

[0515] Step 13:

[0516] Server: Generates personalized feedback based on the judgment results and emotion data.

[0517] Input: Memory and Cognitive Risk Assessment

[0518] Output: Feedback message

[0519] Specific behavior: Generate a specific feedback message according to the user's state (e.g., "Your memory is in good condition"). Examples of prompt sentences include:

[0520] Example prompt sentence:

[0521] "Generate the following feedback based on User B's quiz answer data and emotional data. User B's accuracy rate is high and their emotional state is positive. The feedback should include praise for User B's memory and positive advice for the next quiz."

[0522] Step 14:

[0523] Server: Sends feedback to the user's device.

[0524] Input: Feedback message

[0525] Output: Feedback distribution

[0526] Specific operation: The server sends the generated feedback message to the terminal, and the terminal receives it and notifies the user.

[0527] Step 15:

[0528] On the device: A feedback screen is displayed to the user, showing the evaluation results and recommended actions.

[0529] Input: Feedback message

[0530] Output: Display feedback

[0531] Specific actions: The device displays a feedback message and provides the user with specific advice or recommended actions.

[0532] Through these steps, the system allows users to have fun while activating their memories and receive personalized feedback and recommendations for action.

[0533] (Application example 2)

[0534] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0535] To activate users' memories in a fun way and to detect and prevent dementia early, personalized services are required. However, existing technologies only provide uniform services without considering the user's emotions, making effective memory activation difficult. Furthermore, there are also few ways for users to check their knowledge and memories while having fun. Therefore, a system is needed that provides personalized quizzes and feedback based on the user's emotions, and that can activate memory and assess cognitive risk.

[0536] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0537] In this invention, the server includes: means for collecting photo, calendar, and diary data from the user's terminal; emotion engine means for analyzing the collected data and recognizing the user's emotions; artificial intelligence module means for generating quizzes based on the recognized emotions; means for delivering the generated quizzes to the user's terminal; means for receiving the user's answers and determining whether the answers are correct; means for assessing the user's memory and cognitive risk based on the determination results; means for providing the user with feedback on the evaluation results and recommending appropriate actions; and means for adjusting the difficulty of the quiz in accordance with the user's emotions based on the presentation of the quiz. This allows for the provision of personalized quizzes and feedback in accordance with the user's emotions, enabling memory activation and cognitive risk assessment in a fun environment.

[0538] "User's device" refers to a device that a user uses on a daily basis, including a smartphone or tablet.

[0539] "Photo" refers to image data taken by a user using a camera.

[0540] "Calendar" refers to a scheduling feature or application that allows users to manage their schedules and events.

[0541] A "diary" refers to text data that users use to record their daily events and thoughts.

[0542] "Data collection means" refers to a method or device that provides functionality for acquiring photo, calendar, and diary data from a user's device.

[0543] The "emotion engine" is a software module that analyzes collected data and recognizes the user's emotions.

[0544] The "quiz generation means" is a system for generating quizzes based on recognized emotions using artificial intelligence.

[0545] The "quiz distribution means" refers to a method or device for transmitting the generated quiz to the user's terminal and displaying the quiz.

[0546] The "answer determination means" is a system that receives the user's answer and determines whether it is correct or incorrect.

[0547] The "memory and cognitive risk assessment means" refers to a function or device for assessing the memory state and cognitive risk of a user based on the judgment results.

[0548] The "feedback providing means" is a function that provides the user with the evaluation results and feedback to recommend appropriate actions.

[0549] The "difficulty level adjustment means" is a system for adjusting the difficulty level of a quiz in accordance with the user's emotions based on the presentation of the quiz.

[0550] System Overview

[0551] The system of this invention aims to stimulate the user's memory in a fun way and to detect and prevent dementia early. The system also recognizes the user's emotions and adjusts the quiz generation and feedback content to provide personalized services.

[0552] Hardware and Software Configuration

[0553] Hardware

[0554] User devices: everyday devices such as smartphones and tablets.

[0555] Server: A server for analyzing data, generating quizzes, determining answers, and assessing memory and cognitive risk.

[0556] software

[0557] Data collection software: Collects photos, calendar, and diary data from users' devices.

[0558] Emotion engine: Software that analyzes collected data and recognizes the user's emotions.

[0559] Artificial Intelligence Module: Software for generating quizzes.

[0560] Feedback generation module: Software for generating feedback based on the answer results.

[0561] System Operation

[0562] 1. Data Collection

[0563] Your device will collect photos, calendar and diary data, with your permission.

[0564] 2. Emotion recognition

[0565] The collected data is sent to a server and analyzed by an emotion engine, which, for example, analyzes photos and audio data to recognize emotions from the user's facial expressions and voice.

[0566] 3. Quiz Generation

[0567] The AI ​​module generates quizzes based on the collected data and emotion recognition results, adjusting the difficulty of the quiz depending on the user's emotional state.

[0568] 4. Quiz distribution and answer processing

[0569] The generated quiz is delivered to the user's device, where the user answers the quiz and the device sends the answers to the server.

[0570] 5. Assessment and evaluation of answers

[0571] The server judges the answers and assesses the user's memory and cognitive risk based on the results and emotional data.

[0572] 6. Providing Feedback

[0573] The feedback generation module generates feedback based on the evaluation results and provides it to the user's terminal.

[0574] Specific examples

[0575] For example, here's a scenario where a user answers a quiz about recent events:

[0576] 1. Use photos taken by users at places they visited last weekend.

[0577] 2. The emotion engine analyzes the user's happy facial expression from the photo.

[0578] 3. The artificial intelligence module generates a quiz question: "Where did you visit last weekend?"

[0579] 4. The user answers the quiz and the answers are sent to the server.

[0580] 5. The server determines whether the answer is correct or incorrect and assesses the user's memory status and cognitive risk.

[0581] 6. The feedback generation module generates feedback such as "Good memory" and "Have fun trying the next quiz," and displays it on the user's device.

[0582] In this way, the system of the present invention provides personalized quizzes and feedback according to the user's emotions, enabling memory activation and cognitive risk assessment in a fun way.

[0583] Prompt Sentence Examples

[0584] "Analyze the photos taken by the user and their emotions at the time, and generate a related quiz. Provide appropriate feedback based on the user's answers to the quiz and their emotional data."

[0585] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0586] Step 1:

[0587] Data collection

[0588] The device collects the user's photos, calendar, and diary data with the user's permission.

[0589] Input: The user's device receives collected photos, calendar events, and diary entries.

[0590] Output: Collected data is sent to the server.

[0591] Step 2:

[0592] emotion recognition

[0593] The server receives the collected data and analyzes it using an emotion engine, analyzing the user's facial expressions from photos and the tone of voice from audio data to recognize emotions.

[0594] Input: Collected photos and audio data.

[0595] Data processing / calculation: The emotion engine uses image recognition and voice analysis algorithms to identify emotions.

[0596] Output: Recognized user emotion data.

[0597] Step 3:

[0598] Quiz Generation

[0599] The server uses an artificial intelligence module to generate quizzes based on the collected data and the user's perceived emotions, adjusting the difficulty of the quiz depending on the user's emotional state: increasing the difficulty if the user is having fun, and decreasing the difficulty if the user is feeling stressed.

[0600] Input: Collected data, recognized emotion data.

[0601] Data processing / calculation: An artificial intelligence module analyzes the data and determines the content and difficulty of the quiz.

[0602] Output: The generated quiz.

[0603] Step 4:

[0604] Quiz broadcast

[0605] The server sends the generated quiz to the user's device, which displays a quiz notification and transitions to the quiz screen.

[0606] Input: The generated quiz.

[0607] Output: The quiz is displayed on the user's device.

[0608] Step 5:

[0609] Answer acceptance

[0610] The user answers the presented quiz by selecting an option or entering text.

[0611] Input: The user's answer to the quiz.

[0612] Output: The device sends the user's answer to the server.

[0613] Step 6:

[0614] Answer assessment and evaluation

[0615] The server compares the received answer with a database of correct answers to determine whether it is correct or not, and simultaneously evaluates the user's memory risk in combination with the recognized emotion data.

[0616] Input: User's answer, correct answer database, recognized emotion data.

[0617] Data processing / calculation: A correct / incorrect judgment algorithm is used to calculate the correct answer rate and assess memory risk.

[0618] Output: Judgment result and memory risk score.

[0619] Step 7:

[0620] Providing Feedback

[0621] The server generates feedback based on the assessment results and emotional data, converts the assessment results into numerical values, and compares them with the user's past answer history. It also provides feedback recommending appropriate actions.

[0622] Input: Judgment results, emotion data, past answer history.

[0623] Data processing / computation: Feedback generation algorithms are used to quantify assessment results and recommend actions.

[0624] Output: Feedback is displayed on the user's device.

[0625] Through these steps, users can have fun while activating their memory and undergoing a cognitive risk assessment.

[0626] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0627] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0628] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0629] [Second embodiment]

[0630] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0631] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0632] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0633] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0634] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0635] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0636] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0637] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0638] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0639] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0640] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0641] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0642] Introduction

[0643] The present invention relates to a system that aims to activate a user's memory in a fun way and to detect and prevent dementia early. Specifically, the system generates quizzes based on data collected from the user's device and periodically evaluates the user's memory and cognitive status. The following describes in detail the embodiments of the present invention.

[0644] System configuration

[0645] This system mainly consists of the following components:

[0646] 1. User's Device

[0647] Devices that users use on a daily basis, such as smartphones and tablets.

[0648] The terminal performs the functions of collecting data, displaying quizzes, accepting answers, and providing feedback.

[0649] 2. Server

[0650] Analyzes data, generates quizzes, judges answers, and assesses memory and cognitive risk.

[0651] It is equipped with an artificial intelligence module that automates data analysis and quiz generation.

[0652] 3. Network

[0653] An internet connection that allows communication between your device and our server.

[0654] Program processing

[0655] Data collection

[0656] Server: Sends data collection requests to the user's device.

[0657] On your device: With your permission, we collect photos, calendar events, and diary data.

[0658] Terminal: Sends collected data to the server.

[0659] Examples:

[0660] Photos taken over the weekend, appointments on your calendar, and events recorded in your autobiography app.

[0661] Generate a quiz

[0662] Server: Analyzes the received data using an artificial intelligence module and generates a quiz to stimulate the user's memory.

[0663] Server: Sends the generated quiz to the user's device.

[0664] Examples:

[0665] Photo analysis results: "Where was this photo taken?"

[0666] Calendar event analysis results: "What events happened on this day?"

[0667] Quiz distribution and answers

[0668] On the device: Show the quiz notification to the user and open the quiz screen.

[0669] User: Answers the quiz by selecting options or entering text in response to the questions presented.

[0670] Terminal: Sends the user's answer to the server.

[0671] Examples:

[0672] Choose a location from the options for the photo question.

[0673] Enter detailed text for calendar events.

[0674] Judging and evaluating answers

[0675] Server: Compares the received answer with the correct answer database and determines whether it is correct or not.

[0676] Server: Records the results of the judgment and calculates values ​​to assess the user's memory and cognitive risk.

[0677] Examples:

[0678] If the user answers "park" and the correct answer is "park," it is determined to be the correct answer.

[0679] Based on the memory risk assessment, it is compared with the user's past answer history.

[0680] Providing Feedback

[0681] Server: Generates feedback to the user based on the judgment results.

[0682] Server: Sends feedback to the user's device.

[0683] Terminal: Show a feedback screen to the user.

[0684] Examples:

[0685] If the correct answer rate is high, the feedback is "You have maintained a good memory."

[0686] If the perceived risk is high, you will be notified that "we recommend that you visit a medical institution."

[0687] Example

[0688] User A launches Recoco Memory for the first time and allows the app to access photos, calendar, and diary information. The server receives the initial data and begins analysis. Based on the analysis results, multiple quizzes are generated from User A's photos and calendar events from the past week. For example, the server generates a quiz such as "Where did you visit last weekend?" and sends it to the user's device.

[0689] When User A answers the quiz, the answer data is sent to the server for analysis and evaluation. The server determines whether User A's answer is correct and evaluates the memory status based on the result. The evaluation result is sent as feedback to User A's device, and User A receives a notification that their memory status is "good."

[0690] In this way, by using the system of the present invention, users can enjoy checking their memories and assessing cognitive risk in their daily lives.

[0691] The processing flow will be explained below.

[0692] Step 1:

[0693] Users: Download and install the app on their devices such as smartphones and tablets.

[0694] Step 2:

[0695] Device: When the app is launched for the first time, a user registration screen is displayed and the user is asked to enter personal information (such as name, email address, and password).

[0696] Step 3:

[0697] Server: Receives user registration information, creates a new user account in the database, and sends a success message to the terminal.

[0698] Step 4:

[0699] On device: Prompt the user for permission to access their photos, calendar, and diary data.

[0700] Step 5:

[0701] User: Select "Allow" in the permission dialog to grant the app data access.

[0702] Step 6:

[0703] Device: Collects data for which permission has been granted (photos, calendar, diary information) and sends it to the server.

[0704] Step 7:

[0705] Server: The AI ​​module analyzes the received data and generates quizzes based on the photos and calendar information.

[0706] For example, generate questions based on photo locations or calendar events.

[0707] Step 8:

[0708] Server: Sends the generated quiz to the user's device.

[0709] Step 9:

[0710] On the device: A quiz notification is displayed to the user. When the user taps the notification, the quiz screen is displayed.

[0711] Step 10:

[0712] User: Answers the quiz by selecting options or entering text in response to the questions presented.

[0713] Step 11:

[0714] Terminal: Sends the user's answer to the server.

[0715] Step 12:

[0716] Server: Compares the received answer with the correct answer database and determines whether it is correct or not.

[0717] Step 13:

[0718] Server: Records the results of the judgment and calculates values ​​to assess the user's memory and cognitive risk.

[0719] Example: Calculating accuracy rate and memory risk score.

[0720] Step 14:

[0721] Server: Generates feedback to provide to the user based on the judgment results.

[0722] Step 15:

[0723] Server: Sends feedback to the user's device.

[0724] Step 16:

[0725] Device: A feedback screen is displayed to the user, presenting the evaluation results and recommended actions (e.g., visiting a medical institution).

[0726] Step 17:

[0727] User: Review the feedback and take any recommended action (e.g., seek medical attention) if necessary.

[0728] This series of steps allows users to have fun while checking their memories and assessing cognitive risk.

[0729] Example 1

[0730] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0731] There is a lack of efficient systems to stimulate users' memories in an enjoyable way and to detect and prevent dementia early. In addition, there are insufficient means to accurately assess users' memory status and cognitive risk and provide feedback.

[0732] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0733] In this invention, the server includes means for collecting image data, schedule management data, and record data from the user's computing device, means for analyzing the collected data and using a machine learning algorithm to generate questions, and means for delivering the generated questions to the user's computing device, thereby making it possible to automatically generate questions to stimulate the user's memory, evaluate memory and cognitive risks based on the answers, and provide appropriate feedback.

[0734] "Computing device" refers to an information processing device that users use on a daily basis, and specifically includes smartphones, tablets, personal computers, etc.

[0735] "Image data" refers to photographs taken by users and image files acquired by users.

[0736] "Schedule management data" refers to calendar application data that records a user's schedule and appointments.

[0737] "Recorded data" refers to text data recorded by the user as a diary or memo.

[0738] "Machine learning algorithm" refers to an algorithm used to analyze collected data and generate questions to stimulate a user's memory, and may include an artificial intelligence module.

[0739] "Feedback" refers to information and advice provided to users based on the results of an assessment of their memory status and cognitive risk.

[0740] "Medical institution" refers to a medical facility such as a hospital or clinic that provides medical services.

[0741] MODE FOR CARRYING OUT THE INVENTION

[0742] Introduction

[0743] The present invention relates to a system for activating a user's memory in a fun way, with the aim of early detection and prevention of dementia. The system generates quizzes based on data collected from the user's computing device and periodically assesses the user's memory and cognitive status. Specific embodiments for implementing the invention are described in detail below.

[0744] System configuration

[0745] This system mainly consists of the following components:

[0746] 1. Your Computing Device

[0747] Information processing devices such as smartphones, tablets, and personal computers.

[0748] The computing device has the functions of collecting data, displaying quizzes, accepting answers, and providing feedback.

[0749] 2. Server

[0750] Analyzes data, generates quizzes, judges answers, and assesses memory and cognitive risk.

[0751] It is equipped with an artificial intelligence module that automates data analysis and quiz generation (e.g., GPT-3).

[0752] 3. Network

[0753] An internet connection that allows communication between the computing device and the server.

[0754] Data collection

[0755] The server sends a data collection request to the user's computing device. Based on the user's permission, the computing device collects image data, schedule management data, and record data. The computing device then sends the collected data to the server. Specific examples include photos taken by the user over the weekend, appointments listed on a calendar, and events recorded in a personal history app.

[0756] Generate a quiz

[0757] The server analyzes the received data using an artificial intelligence module and generates a quiz to stimulate the user's memory. The server then sends the generated quiz to the user's computing device. Specific examples include quizzes such as "Where was this photo taken?" based on the analysis of a photo, or "What events happened on this day?" based on the analysis of calendar events.

[0758] Quiz distribution and answers

[0759] The computing device displays a quiz notification to the user and opens the quiz screen. The user answers the quiz by selecting options or entering text in response to the presented questions. The computing device then sends the user's answers to the server. Specific examples of answers include selecting a location from options in response to a photo question or entering detailed actions in response to a calendar event.

[0760] Judging and evaluating answers

[0761] The server compares the received answer with a correct answer database to determine whether it is correct or incorrect. The server then records the result of the determination and calculates a value to evaluate the user's memory and cognitive risk. For example, if the user answers "park" and the correct answer is "park," the answer is determined to be correct. Furthermore, based on the memory risk assessment, the result is compared with the user's past answer history.

[0762] Providing Feedback

[0763] The server generates feedback for the user based on the assessment results. The server then sends the generated feedback to the user's computing device. The computing device displays a feedback screen to the user. For example, if the accuracy rate is high, feedback such as "Your memory is in good condition" is displayed, and if the cognitive risk is high, a notification such as "We recommend that you visit a medical institution" is displayed.

[0764] Example

[0765] User A launches the system for the first time and allows the app to access image data, schedule management data, and record data. The server receives the initial data and begins analysis. Based on the analysis results, multiple quizzes are generated from User A's photos and calendar events from the past week. For example, the server generates a quiz question such as, "Where did you visit last weekend?" and sends it to the user's computing device. When User A answers the quiz, the answer data is sent to the server for analysis and evaluation. The server determines whether User A's answer is correct and evaluates his or her memory status based on the result. The evaluation result is sent as feedback to User A's computing device, and User A is notified that his or her memory status is "good." In this way, using the system of the present invention, users can enjoy checking their memory and assessing cognitive risk in their daily lives.

[0766] Examples of prompt statements

[0767] "Generate quizzes from the user's photo data collected by the system, deliver the generated quizzes to the user's computing device, and provide feedback based on the results."

[0768] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0769] Step 1: Submit a data collection request

[0770] The server sends a data collection request to the user's computing device, the request including information directing the collection of image data, schedule management data, and record data.

[0771] Input: None

[0772] Output: Data collection request

[0773] Step 2: Data collection

[0774] The device collects image data, schedule management data, and record data from the user based on the data collection request received from the server. Specifically, it obtains photos from the camera roll of the smartphone or tablet, schedules from the calendar app, and text information from the diary app.

[0775] Input: Data Collection Request

[0776] Output: Collected data (photos, schedules, diaries)

[0777] Step 3: Sending data

[0778] The terminal transmits collected image data, schedule management data, and record data to the server. This data is transmitted based on the user's permission.

[0779] Input: Collected data

[0780] Output: Send data to the server

[0781] Step 4: Data analysis

[0782] The server analyzes the received data using a generative AI model (e.g., GPT-3). This analysis includes photo recognition, text analysis, and calendar event identification. Specifically, image recognition algorithms are used to analyze the content of photos, and natural language processing algorithms are used to analyze text data.

[0783] Input: Collected data

[0784] Output: Analysis results

[0785] Step 5: Quiz Generation

[0786] The server generates quizzes to stimulate the user's memory based on the analysis results. For example, it generates questions such as "Where was this photo taken?" based on the location information of the photo. It also generates questions such as "What events happened on this day?" based on calendar events.

[0787] Input: Analysis results, generated AI model

[0788] Output: The generated quiz

[0789] Step 6: Stream the quiz

[0790] The server then sends the generated quiz to the user's computing device, formatted in a way that is easy for the user to answer.

[0791] Input: Generated quiz

[0792] Output:Quiz delivery

[0793] Step 7: Quiz Notification and Display

[0794] The device notifies the user of the quiz received from the server and displays the quiz screen. Specifically, the device notifies the user of the existence of a new quiz using the notification center or in-app notifications, and displays the quiz screen.

[0795] Input:Quiz broadcast

[0796] Output: Quiz notification to user, quiz screen display

[0797] Step 8: Take the quiz

[0798] The user can select an answer from the options or enter a text answer for the quiz. Specifically, the user selects the correct answer from the options displayed or enters the answer using the keyboard.

[0799] Input: Quiz screen

[0800] Output: User's answer

[0801] Step 9: Submit your answers

[0802] The device sends the user's answers to the server, where they are accurately recorded and securely transmitted to the server.

[0803] Input: User's answer

[0804] Output: Send the answer to the server

[0805] Step 10: Determine the answer

[0806] The server compares the received answer with a database of correct answers to determine whether the answer is correct or not. Specifically, it matches the user's answer with the correct answers stored in the database.

[0807] Input: User's answer

[0808] Output: Judgment result

[0809] Step 11: Valuation Calculation

[0810] The server calculates a numerical value to evaluate the user's memory and cognitive risk based on the judgment result. For example, the server evaluates the user's memory ability and cognitive risk based on the accuracy rate and reaction time, and generates a score.

[0811] Input: Judgment result

[0812] Output: Evaluation score

[0813] Step 12: Generate feedback

[0814] The server generates feedback for the user based on the evaluation score, such as a message indicating good memory status or a message recommending that the user seek medical attention.

[0815] Input: Rating score

[0816] Output: Feedback message

[0817] Step 13: Submit your feedback

[0818] The server then transmits the generated feedback to the user's computing device, where the feedback is user-friendly and encourages action.

[0819] Input: Feedback message

[0820] Output: Send feedback to the user

[0821] Step 14: Viewing feedback

[0822] The device displays the feedback message received from the server to the user, specifically, by providing the feedback to the user using an in-app notification or a pop-up window.

[0823] Input: Send Feedback

[0824] Output: Show feedback

[0825] (Application example 1)

[0826] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0827] In an aging society, the prevention and early detection of dementia are important issues. However, existing methods lack systems that allow users to enjoyably assess memory in their daily lives and determine the risk of cognitive function without visiting a medical institution. Furthermore, even in physical stores, there are no systems that promote memory activation through the shopping experience, so there is a need to integrate dementia prevention and shopping.

[0828] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0829] In this invention, the server includes means for collecting photo, calendar, and diary data, as well as purchase and check-in histories from the user's device, an artificial intelligence module for analyzing the collected data and generating quizzes, and means for delivering and notifying the generated quizzes to the user's device. This enables the user's memory to be stimulated in an enjoyable way through everyday shopping experiences, thereby enabling the prevention and early detection of dementia.

[0830] "User device" refers to a device such as a smartphone, tablet, or smart glasses that a user uses on a daily basis to collect data, display quizzes, and accept answers.

[0831] "Data collection" is the process of collecting purchase history, check-in history, photos, calendar, and diary data from users' devices.

[0832] "Purchase history" refers to a record and detailed information about products purchased by a user in a physical store.

[0833] "Check-in history" is a record of a user's visits to specific locations or stores, and is data that includes location information.

[0834] The "artificial intelligence module" is a system consisting of programs and algorithms for analyzing collected data and generating quizzes to stimulate the user's memory.

[0835] "Quiz generation" is the process of automatically creating questions to assess a user's memory based on collected and analyzed data.

[0836] "Quiz delivery" is the process of sending the generated quiz to the user's device and notifying them.

[0837] "Answer determination" is the process of analyzing the answers given by the user to the quiz and comparing them with the correct answers to determine whether they are correct or incorrect.

[0838] "Memory status assessment" is the process of assessing a user's memory and cognitive function by comparing it with the user's past answer history.

[0839] "Feedback" is the process of informing the user of information about their memory and cognitive status and appropriate actions based on their answers and the assessment results.

[0840] "Recommendation to visit a medical institution" is the process of notifying the user to visit a medical institution based on the evaluation results.

[0841] "Quantification" is the process of converting evaluation results and answer history into quantitative data to make it easier to evaluate the user's memory state.

[0842] Configuring the various components

[0843] The present invention relates to a system for activating a user's memory and assessing their cognitive status. The system comprises the following components:

[0844] 1. User's Device

[0845] These are devices that users use on a daily basis (e.g., smartphones, tablets, smart glasses, etc.). These devices collect data such as users' purchase history, check-in history, photos, calendars, and diaries. They also perform functions such as displaying quizzes, accepting answers, and providing feedback.

[0846] 2. Server

[0847] The server analyzes data, generates quizzes, judges answers, and assesses memory and cognitive risk. It is equipped with an artificial intelligence module to automate these processes, and often uses cloud services such as Google Cloud and Amazon Web Services (AWS).

[0848] 3. Network

[0849] Refers to an internet connection that enables communication between a device and a server, and can use Wi-Fi, 4G, 5G, or other mobile networks.

[0850] Data collection

[0851] 1. Terminal

[0852] With user permission, data such as purchase history, check-in history, photos, calendar events, and diary entries are collected and transmitted securely to the server.

[0853] Generate a quiz

[0854] 2. Server

[0855] The received data is analyzed using an artificial intelligence module (e.g., TensorFlow, PyTorch) to generate a quiz to stimulate the user's memory. The quiz is automatically created and sent to the user's device.

[0856] Quiz distribution and answers

[0857] 3. Terminal

[0858] A quiz notification is displayed to the user, and the quiz screen opens. The user answers the quiz by selecting options or entering text in response to the questions presented.

[0859] 4. Users

[0860] The user answers the quiz appropriately and transmits the answers to the server via the terminal.

[0861] Judging and evaluating answers

[0862] 5. Server

[0863] The received answer is compared with a correct answer database to determine whether it is correct or incorrect. The result of the determination is then recorded and compared with past answer history to evaluate the memory status.

[0864] Providing Feedback

[0865] 6. Server

[0866] Based on the results of the assessment, feedback is generated for the user, including an assessment of memory status and recommended actions (e.g., a recommendation to see a doctor). The generated feedback is sent to the user's device.

[0867] 7. Terminal

[0868] A feedback screen is displayed to the user, allowing them to check their memory status and recommended actions.

[0869] Specific examples

[0870] When a user is shopping in a physical store, they may purchase a specific item at the register (a retro-designed drink that they have purchased in the past). The system collects this data and generates a quiz based on their purchase history, asking, "What flavor was the retro cola you purchased two weeks ago?" If the user answers the quiz correctly, the system provides feedback such as, "Your memory is good!"

[0871] Prompt Sentence Examples

[0872] "What flavor of retro cola did you buy two weeks ago?"

[0873] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0874] Step 1:

[0875] Data collection

[0876] Device: First, with the user's permission, data such as purchase history, check-in history, photos, calendar events, and diary entries are collected. This data is either collected automatically by the application or entered manually by the user. The collected data is converted into an appropriate data format, such as JSON, and sent to the server while protecting privacy.

[0877] Input: User purchase history, check-in history, photos, calendar events, diary data

[0878] Output: JSON data formatted to send to the server

[0879] Step 2:

[0880] Generate a quiz

[0881] Server: Analyzes the received data. Specifically, it extracts information for generating quizzes from the collected data through text analysis (natural language processing) and image analysis. It uses artificial intelligence modules (e.g., TensorFlow, PyTorch) to automatically generate quizzes that stimulate the user's memory.

[0882] Input: JSON data sent from the terminal

[0883] Output: Generated quiz questions and choices

[0884] Step 3:

[0885] Quiz broadcast

[0886] Server: The server delivers the generated quiz to the user's device and sends a notification. The quiz content is pushed to the device using the REST API.

[0887] On the device: The received notification is displayed to the user and the quiz screen is opened. When the user taps the notification, the specific quiz content is displayed.

[0888] Input: Generated quiz

[0889] Output: Quiz notification and quiz screen

[0890] Step 4:

[0891] Quiz Answers

[0892] User: Answers the quiz by selecting an answer from the options provided or by entering text.

[0893] Terminal: Receives the user's answer and sends it to the server. The data sent is in JSON format, including the answer.

[0894] Input: User's quiz answer

[0895] Output: Answer data to send to the server

[0896] Step 5:

[0897] Judging and evaluating answers

[0898] Server: Compares the received user answers with a database of correct answers to determine whether they are correct or not. It also compares the answers with the user's past answer history to assess memory status and cognitive risk. Statistical analysis and machine learning algorithms are used for this.

[0899] Input: User's answer data

[0900] Output: Correct / incorrect judgment and memory status evaluation results

[0901] Step 6:

[0902] Providing Feedback

[0903] Server: Generates feedback to the user based on the correctness of the answers and the evaluation results. The feedback includes an evaluation of memory status and recommended actions (e.g., recommendation to see a doctor).

[0904] Device: Receives the generated feedback and displays it to the user, allowing the user to check their own memory status and course of action.

[0905] Input: Correct / incorrect judgment and memory status evaluation results

[0906] Output: Feedback that is displayed on the user's device

[0907] Examples of prompt statements

[0908] "What flavor of retro cola did you buy two weeks ago?"

[0909] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0910] Introduction

[0911] The present invention relates to a system that aims to activate a user's memory in a fun way and to detect and prevent dementia early. In particular, the present invention provides a more personalized service by recognizing the user's emotions and adjusting the quiz generation and feedback content.

[0912] System configuration

[0913] The system consists of the following main components:

[0914] 1. User's Device

[0915] These are devices that users use on a daily basis, such as smartphones and tablets.

[0916] It performs functions such as data collection, emotion recognition, quiz display, answer acceptance, and feedback provision.

[0917] 2. Server

[0918] Analyzes data, generates quizzes, judges answers, and assesses memory and cognitive risk.

[0919] It is equipped with an artificial intelligence module and an emotion engine to automate data analysis and quiz generation.

[0920] 3. Network

[0921] It is an internet connection that allows communication between the device and the server.

[0922] Program processing

[0923] Data collection and emotion recognition

[0924] Server: Sends data collection requests to the user's device.

[0925] On the device: With the user's permission, it collects photos, calendar events, and diary data, and also uses the device's camera and microphone to capture the user's facial expressions and voice, which are then sent to the emotion engine.

[0926] Emotion engine: Analyzes collected facial and voice data to recognize the user's emotions.

[0927] Terminal: Sends collected data and sentiment analysis results to the server.

[0928] Examples:

[0929] When a user takes a photo or records an event in the calendar, the device sends this data to the server, and at the same time collects the user's facial expressions and voice and sends them to the emotion engine.

[0930] Quiz Generation

[0931] Server: Analyzes the received data using an artificial intelligence module and generates a quiz based on the user's memory and emotional state.

[0932] Server: Adjusts the content and difficulty of the quiz based on the emotions recognized by the emotion engine.

[0933] For example: if the user is having fun, generate a challenging quiz, if the user is stressed, generate an easy quiz.

[0934] Examples:

[0935] Photo analysis results: "Where was this taken?"

[0936] Calendar analysis results: "What events were there on this day?"

[0937] Quiz distribution and answers

[0938] Server: Sends the generated quiz to the user's device.

[0939] Device: Show the quiz notification to the user and transition to the quiz screen.

[0940] User: Answers the quiz by selecting options or entering text in response to the questions presented.

[0941] Terminal: Sends the user's answer to the server.

[0942] Examples:

[0943] A user receives a quiz notification and selects the correct answer to the question, "Where was this photo taken?"

[0944] Judging and evaluating answers

[0945] Server: Compares the received answer with the correct answer database and determines whether it is correct or not.

[0946] Server: Combines the judgment results with data from the emotion engine to calculate a numerical value for assessing the user's memory and cognitive risk.

[0947] Example: Calculating the accuracy rate and memory risk score.

[0948] Examples:

[0949] If the user answers "park" and it is the correct answer, it is determined to be the correct answer and compared with the past answer history to evaluate the memory risk.

[0950] Providing Feedback

[0951] Server: Generates personalized feedback based on the judgment results and emotion data.

[0952] Server: Sends feedback to the user's device.

[0953] Device: A feedback screen is displayed to the user, presenting the evaluation results and recommended actions (e.g., visiting a medical institution).

[0954] Examples:

[0955] If the correct answer rate is high, the system will provide feedback such as "Your memory is in good condition," and if the emotional data indicates that the user is under stress, it will also display advice such as "Relax and try the next quiz."

[0956] Example

[0957] User B launched Recoco Memory for the first time and allowed the app to access photos, calendar, and diary information. He also allowed the app to access the camera and microphone for emotion recognition. The server received the initial data and emotion data and began analysis. Based on the analysis results, multiple quizzes were generated from User B's photos and calendar events from the past week. For example, the server generated a quiz asking, "Where did you visit last weekend?" and selected a slightly more challenging quiz based on the user's emotional state (enjoyment).

[0958] When User B answers the quiz, the answer data and the latest emotional data are sent to the server for analysis and evaluation. The server determines whether User B's answer is correct and evaluates his / her memory and emotional state based on the result and the emotional data. As feedback, the device displays a notification saying "Good memory state" and "Enjoy the next quiz."

[0959] In this way, by using the system of the present invention, users can not only have fun while checking their memory and assessing cognitive risk, but also receive personalized services that take into account their emotional state.

[0960] The processing flow will be explained below.

[0961] Step 1:

[0962] User: Download and install Ricoco Memory on a device such as a smartphone or tablet.

[0963] Step 2:

[0964] Device: When the app is launched for the first time, a user registration screen is displayed and the user is asked to enter personal information (such as name, email address, and password).

[0965] Step 3:

[0966] Server: Receives user registration information, creates a new user account in the database, and sends a success message to the terminal.

[0967] Step 4:

[0968] Device: Prompts the user for permission to access photos, calendar, diary data, and the camera and microphone.

[0969] Step 5:

[0970] User: Select "Allow" in the permission dialog to grant the app data access.

[0971] Step 6:

[0972] Device: Collects data with permission (photos, calendar events, diary information) and sends it to the server. It also uses the camera and microphone to transmit the user's facial expressions and voice to the emotion engine.

[0973] Step 7:

[0974] Emotion engine: Analyzes collected facial and voice data to recognize the user's emotions.

[0975] Step 8:

[0976] Terminal: Sends collected data and sentiment analysis results to the server.

[0977] Step 9:

[0978] Server: Analyzes the received data using an AI module and generates a quiz based on the user's memory and emotional state. The emotion engine adjusts the content and difficulty of the quiz based on the emotions recognized by the server.

[0979] For example: if the user is having fun, generate a challenging quiz, if the user is stressed, generate an easy quiz.

[0980] Step 10:

[0981] Server: Sends the generated quiz to the user's device.

[0982] Step 11:

[0983] Device: Show the quiz notification to the user and transition to the quiz screen.

[0984] Step 12:

[0985] User: Answers the quiz by selecting options or entering text in response to the questions presented.

[0986] Step 13:

[0987] Terminal: Sends the user's answer to the server.

[0988] Step 14:

[0989] Server: Compares the received answer with the correct answer database and determines whether it is correct or not.

[0990] Step 15:

[0991] Server: Combines the judgment results with data from the emotion engine to calculate a numerical value for assessing the user's memory and cognitive risk.

[0992] Example: Calculating the accuracy rate and memory risk score.

[0993] Step 16:

[0994] Server: Generates personalized feedback based on the judgment results and emotion data.

[0995] Step 17:

[0996] Server: Sends feedback to the user's device.

[0997] Step 18:

[0998] Device: A feedback screen is displayed to the user, presenting the evaluation results and recommended actions (e.g., visiting a medical institution).

[0999] Step 19:

[1000] User: Review the feedback and take any recommended action (e.g., seek medical attention) if necessary.

[1001] Through this series of steps, users can not only enjoyably check their memories and assess their cognitive risks, but also receive personalized services that take into account their emotional state.

[1002] Example 2

[1003] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1004] In modern society, early detection and prevention of dementia are important issues, especially in the health management of the elderly. However, conventional memory assessment tools are monotonous, and users lack motivation to continue using them. Furthermore, there is a need for systems that do not take into account the user's emotional state and can be used in a fun and stress-free manner. Furthermore, there is a lack of personalized feedback and recommendations for appropriate actions tailored to individual needs. In response to these challenges, the present invention provides a system that allows users to activate their memory in an enjoyable way and provides personalized services based on their emotional state.

[1005] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1006] In this invention, the server includes means for collecting photo, schedule, and document data from the user's terminal, means for analyzing the collected data and the user's facial expressions and voice to recognize the emotional state, an artificial intelligence module for generating quizzes based on the analyzed data and the emotional state, means for adjusting the content and difficulty of the generated quizzes based on the emotional state, means for delivering the generated quizzes to the user's terminal, means for receiving the user's answers and determining whether the answers are correct, means for assessing the user's memory and cognitive risk based on the assessment results and the emotional state, and means for providing the user with feedback recommending the assessment results and appropriate actions. This allows the user to have fun while activating their memory and receiving personalized feedback and recommended actions.

[1007] "User's device" refers to a device that a user uses on a daily basis, such as a smartphone or tablet.

[1008] "Photo" refers to a still image taken by a User using a Device.

[1009] "Calendar" refers to the schedule information that a user records on a device.

[1010] "Document" refers to text data that a user creates or saves using a device.

[1011] "Means of collecting data" refers to the functionality for obtaining data such as photos, schedules, and documents from the user's device.

[1012] "Facial expressions and voice" refers to the user's facial expressions and spoken voice data.

[1013] "Means for recognizing emotional states" refers to a function that analyzes collected facial and voice data to identify the user's emotions (happiness, sadness, surprise, etc.).

[1014] "Artificial intelligence module means" refers to a processing system that uses artificial intelligence to analyze data and generate quizzes.

[1015] "Quiz" refers to questions or problems designed to assess a user's memory or cognitive abilities.

[1016] "Means for adjusting the content and difficulty of the quiz" refers to a function for appropriately adjusting the content and difficulty of the quiz questions based on the user's emotional state.

[1017] "Means for delivering quizzes" refers to the function for sending the generated quiz to the user's terminal.

[1018] "Means for receiving answers" refers to a function for transmitting the answers given by the user to the quiz from the user's terminal to the server and receiving them.

[1019] "Means for determining whether the answer is correct" refers to a function for comparing the received answer with pre-set correct answer data and determining whether the answer is correct or incorrect.

[1020] "Means for assessing memory and cognitive risk" refers to a function for assessing the risk of a user's memory and cognitive state based on the correctness or incorrectness of answers and emotional state.

[1021] "Means for providing feedback" refers to a function for generating feedback to recommend appropriate actions to the user based on the evaluation results and sending it to the user's device.

[1022] This invention relates to a system that aims to stimulate a user's memory in a fun way and to detect and prevent dementia early. In particular, it provides a more personalized service by recognizing the user's emotions and adjusting the quiz generation and feedback content.

[1023] System configuration

[1024] The system consists of the following main components:

[1025] 1. User's Device

[1026] These are devices that users use on a daily basis, such as smartphones and tablets. Specific hardware examples include Android and iOS-based devices.

[1027] Software: A dedicated application that collects data, recognizes emotions, displays quizzes, accepts answers, and provides feedback.

[1028] 2. Server

[1029] The server has the functionality to analyze data, generate quizzes, judge answers, and assess memory and cognitive risk.

[1030] Software: Artificial intelligence module and emotion engine, which automates data analysis and quiz generation.

[1031] 3. Network

[1032] An internet connection that allows communication between a device and a server, allowing data to be exchanged.

[1033] Data collection and emotion recognition

[1034] Server: Sends a data collection request to the user's device. The user's device receives this request and, after obtaining the user's permission, collects photo, calendar, and document data. It also uses the device's camera and microphone to collect the user's facial and voice data and sends it to the emotion engine. The emotion engine analyzes this data and recognizes the user's emotions.

[1035] Examples:

[1036] Users take everyday photos with their smartphone camera and register travel events in their calendar. The device sends this data to the server and simultaneously collects the user's facial expressions and voice. The emotion engine detects "enjoyment" from the user's smile, tone of voice, etc.

[1037] Generate a quiz

[1038] Server: Receives the results from the emotion engine and generates a quiz based on the user's memory and emotional state.

[1039] Artificial Intelligence Module: This determines the content and difficulty of the questions. Quizzes are created using a generative AI model, with example prompts such as:

[1040] Example prompt sentence:

[1041] "Generate the following quiz based on user A's photo data. User A's emotional state seems to be fun. The content of the quiz should be something that user A would enjoy. Specific example questions include, 'Where was this photo taken?' and 'What was this event?'"

[1042] The generated quizzes are adjusted in content and difficulty based on the emotional state.

[1043] Quiz distribution and answers

[1044] Server: Sends the generated quiz to the user's device.

[1045] Device: Show the quiz notification to the user and transition to the quiz screen.

[1046] Examples:

[1047] Users are given a quiz asking, "Where was this photo taken?" and are asked to select the appropriate location.

[1048] Judging and evaluating answers

[1049] Server: Receives the user's answer data, compares it with the correct answer database, and determines whether it is correct or incorrect. The result of the evaluation is combined with the emotion engine data to calculate a value that evaluates the user's memory and cognitive risk.

[1050] Examples:

[1051] If the user answers "park," if that is the correct answer, it is determined to be the correct answer and compared with past answer history to assess memory risk.

[1052] Providing Feedback

[1053] Server: Generates personalized feedback based on the judgment results and emotion data. The feedback is sent to the device and displayed on the feedback screen.

[1054] Examples:

[1055] If the correct answer rate is high, the system will provide feedback such as "Your memory is in good condition," and if it determines that the user is feeling stressed, it will advise them to "relax and try the next quiz."

[1056] This allows users to have fun while activating their memories and receiving personalized feedback.

[1057] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1058] Step 1:

[1059] Server: Sends data collection requests to the device.

[1060] Input: None

[1061] Output: Sending a data collection request

[1062] Specific operation: The server sends data collection instructions to the user's device via an HTTP request, which includes the specific data to collect (photos, schedules, documents) and how to collect it.

[1063] Step 2:

[1064] Device: Obtains user permission to collect data. If the user gives permission, the device will collect photos, calendar, and document data. It will also use the device's camera and microphone to collect facial and voice data.

[1065] Input: Data collection request from the server

[1066] Output: Collected data (photos, schedules, documents, facial expressions, voice)

[1067] How it works: The device running the app collects photos taken by the user, event information registered in the calendar, and document data, while also using the camera and microphone to collect real-time facial expressions and voice recordings.

[1068] Step 3:

[1069] Emotion engine: Analyzes collected facial and voice data to recognize the user's emotional state.

[1070] Input: Collected facial expression and voice data

[1071] Output: Emotion recognition result

[1072] Specific operation: The device sends the facial expression data captured by the device to an image recognition algorithm, and the recorded voice data to a voice analysis engine, which then recognizes the user's emotions (happiness, sadness, surprise, etc.).

[1073] Step 4:

[1074] Terminal: Sends collected data and sentiment analysis results to the server.

[1075] Input: Collected data and emotion recognition results

[1076] Output: Send data to the server

[1077] What happens: The device converts the data into the appropriate format and sends it to the server. The data is sent via an HTTP POST request, subject to user permission.

[1078] Step 5:

[1079] Server: Inputs the received data into an artificial intelligence module and generates a quiz based on the user's memory and emotional state.

[1080] Input: Collected data and emotion recognition results

[1081] Output: The generated quiz

[1082] How it works: The server inputs data into the AI ​​module and generates a quiz, such as "Based on your calendar events, what were the events on this day?"

[1083] Step 6:

[1084] Server: Adjusts the content and difficulty of the quiz based on the emotions recognized by the emotion engine.

[1085] Input: Generated quiz and emotion recognition results

[1086] Output: Adjusted quiz

[1087] What it does: If the user is having fun, choose a challenging quiz; if the user is stressed, choose an easy quiz.

[1088] Step 7:

[1089] Server: Sends the generated quiz to the user's device.

[1090] Input: Adjusted Quiz

[1091] Output: Quiz stream

[1092] Specific operation: The server sends the quiz in JSON format to the device, and the device receives it and displays a quiz notification.

[1093] Step 8:

[1094] Device: Show the quiz notification to the user and transition to the quiz screen.

[1095] Input: Quiz Stream

[1096] Output: Quiz screen display

[1097] What happens: When the user clicks on the notification, the application transitions to a quiz screen where the question is displayed.

[1098] Step 9:

[1099] User: Answers the quiz by selecting options or entering text in response to the questions presented.

[1100] Input: Question on the quiz screen

[1101] Output: User's answer

[1102] Specific behavior: The user selects an answer from the options and clicks the "Submit" button.

[1103] Step 10:

[1104] Terminal: Sends the user's answer to the server.

[1105] Input: User's answer

[1106] Output: Send the answer to the server

[1107] Specific operation: The device converts the answer data into an appropriate format and sends it to the server via an HTTP POST request.

[1108] Step 11:

[1109] Server: Compares the received answer data with the correct answer database to determine whether it is correct or not.

[1110] Input: User's answer

[1111] Output: Judgment result

[1112] Specific operation: The server compares the received answer with the correct answer data and determines whether it is correct or incorrect.

[1113] Step 12:

[1114] Server: Combines the judgment results and data from the emotion engine to assess the user's memory and cognitive risk.

[1115] Input: Judgment results and emotion recognition results

[1116] Output: Memory and cognitive risk assessment

[1117] Specific operation: The server calculates an evaluation score based on the correct answer rate, past answer history, and emotional data.

[1118] Step 13:

[1119] Server: Generates personalized feedback based on the judgment results and emotion data.

[1120] Input: Memory and Cognitive Risk Assessment

[1121] Output: Feedback message

[1122] Specific behavior: Generate a specific feedback message according to the user's state (e.g., "Your memory is in good condition"). Examples of prompt sentences include:

[1123] Example prompt sentence:

[1124] "Generate the following feedback based on User B's quiz answer data and emotional data. User B's accuracy rate is high and their emotional state is positive. The feedback should include praise for User B's memory and positive advice for the next quiz."

[1125] Step 14:

[1126] Server: Sends feedback to the user's device.

[1127] Input: Feedback message

[1128] Output: Feedback distribution

[1129] Specific operation: The server sends the generated feedback message to the terminal, and the terminal receives it and notifies the user.

[1130] Step 15:

[1131] On the device: A feedback screen is displayed to the user, showing the evaluation results and recommended actions.

[1132] Input: Feedback message

[1133] Output: Display feedback

[1134] Specific actions: The device displays a feedback message and provides the user with specific advice or recommended actions.

[1135] Through these steps, the system allows users to have fun while activating their memories and receive personalized feedback and recommendations for action.

[1136] (Application example 2)

[1137] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1138] To activate users' memories in a fun way and to detect and prevent dementia early, personalized services are required. However, existing technologies only provide uniform services without considering the user's emotions, making effective memory activation difficult. Furthermore, there are also few ways for users to check their knowledge and memories while having fun. Therefore, a system is needed that provides personalized quizzes and feedback based on the user's emotions, and that can activate memory and assess cognitive risk.

[1139] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1140] In this invention, the server includes: means for collecting photo, calendar, and diary data from the user's terminal; emotion engine means for analyzing the collected data and recognizing the user's emotions; artificial intelligence module means for generating quizzes based on the recognized emotions; means for delivering the generated quizzes to the user's terminal; means for receiving the user's answers and determining whether the answers are correct; means for assessing the user's memory and cognitive risk based on the determination results; means for providing the user with feedback on the evaluation results and recommending appropriate actions; and means for adjusting the difficulty of the quiz in accordance with the user's emotions based on the presentation of the quiz. This allows for the provision of personalized quizzes and feedback in accordance with the user's emotions, enabling memory activation and cognitive risk assessment in a fun environment.

[1141] "User's device" refers to a device that a user uses on a daily basis, including a smartphone or tablet.

[1142] "Photo" refers to image data taken by a user using a camera.

[1143] "Calendar" refers to a scheduling feature or application that allows users to manage their schedules and events.

[1144] A "diary" refers to text data that users use to record their daily events and thoughts.

[1145] "Data collection means" refers to a method or device that provides functionality for acquiring photo, calendar, and diary data from a user's device.

[1146] The "emotion engine" is a software module that analyzes collected data and recognizes the user's emotions.

[1147] The "quiz generation means" is a system for generating quizzes based on recognized emotions using artificial intelligence.

[1148] The "quiz distribution means" refers to a method or device for transmitting the generated quiz to the user's terminal and displaying the quiz.

[1149] The "answer determination means" is a system that receives the user's answer and determines whether it is correct or incorrect.

[1150] The "memory and cognitive risk assessment means" refers to a function or device for assessing the memory state and cognitive risk of a user based on the judgment results.

[1151] The "feedback providing means" is a function that provides the user with the evaluation results and feedback to recommend appropriate actions.

[1152] The "difficulty level adjustment means" is a system for adjusting the difficulty level of a quiz in accordance with the user's emotions based on the presentation of the quiz.

[1153] System Overview

[1154] The system of this invention aims to stimulate the user's memory in a fun way and to detect and prevent dementia early. The system also recognizes the user's emotions and adjusts the quiz generation and feedback content to provide personalized services.

[1155] Hardware and Software Configuration

[1156] Hardware

[1157] User devices: everyday devices such as smartphones and tablets.

[1158] Server: A server for analyzing data, generating quizzes, determining answers, and assessing memory and cognitive risk.

[1159] software

[1160] Data collection software: Collects photos, calendar, and diary data from users' devices.

[1161] Emotion engine: Software that analyzes collected data and recognizes the user's emotions.

[1162] Artificial Intelligence Module: Software for generating quizzes.

[1163] Feedback generation module: Software for generating feedback based on the answer results.

[1164] System Operation

[1165] 1. Data Collection

[1166] Your device will collect photos, calendar and diary data, with your permission.

[1167] 2. Emotion recognition

[1168] The collected data is sent to a server and analyzed by an emotion engine, which, for example, analyzes photos and audio data to recognize emotions from the user's facial expressions and voice.

[1169] 3. Quiz Generation

[1170] The AI ​​module generates quizzes based on the collected data and emotion recognition results, adjusting the difficulty of the quiz depending on the user's emotional state.

[1171] 4. Quiz distribution and answer processing

[1172] The generated quiz is delivered to the user's device, where the user answers the quiz and the device sends the answers to the server.

[1173] 5. Assessment and evaluation of answers

[1174] The server judges the answers and assesses the user's memory and cognitive risk based on the results and emotional data.

[1175] 6. Providing Feedback

[1176] The feedback generation module generates feedback based on the evaluation results and provides it to the user's terminal.

[1177] Specific examples

[1178] For example, here's a scenario where a user answers a quiz about recent events:

[1179] 1. Use photos taken by users at places they visited last weekend.

[1180] 2. The emotion engine analyzes the user's happy facial expression from the photo.

[1181] 3. The artificial intelligence module generates a quiz question: "Where did you visit last weekend?"

[1182] 4. The user answers the quiz and the answers are sent to the server.

[1183] 5. The server determines whether the answer is correct or incorrect and assesses the user's memory status and cognitive risk.

[1184] 6. The feedback generation module generates feedback such as "Good memory" and "Have fun trying the next quiz," and displays it on the user's device.

[1185] In this way, the system of the present invention provides personalized quizzes and feedback according to the user's emotions, enabling memory activation and cognitive risk assessment in a fun way.

[1186] Prompt Sentence Examples

[1187] "Analyze the photos taken by the user and their emotions at the time, and generate a related quiz. Provide appropriate feedback based on the user's answers to the quiz and their emotional data."

[1188] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1189] Step 1:

[1190] Data collection

[1191] The device collects the user's photos, calendar, and diary data with the user's permission.

[1192] Input: The user's device receives collected photos, calendar events, and diary entries.

[1193] Output: Collected data is sent to the server.

[1194] Step 2:

[1195] emotion recognition

[1196] The server receives the collected data and analyzes it using an emotion engine, analyzing the user's facial expressions from photos and the tone of voice from audio data to recognize emotions.

[1197] Input: Collected photos and audio data.

[1198] Data processing / calculation: The emotion engine uses image recognition and voice analysis algorithms to identify emotions.

[1199] Output: Recognized user emotion data.

[1200] Step 3:

[1201] Quiz Generation

[1202] The server uses an artificial intelligence module to generate quizzes based on the collected data and the user's perceived emotions, adjusting the difficulty of the quiz depending on the user's emotional state: increasing the difficulty if the user is having fun, and decreasing the difficulty if the user is feeling stressed.

[1203] Input: Collected data, recognized emotion data.

[1204] Data processing / calculation: An artificial intelligence module analyzes the data and determines the content and difficulty of the quiz.

[1205] Output: The generated quiz.

[1206] Step 4:

[1207] Quiz broadcast

[1208] The server sends the generated quiz to the user's device, which displays a quiz notification and transitions to the quiz screen.

[1209] Input: The generated quiz.

[1210] Output: The quiz is displayed on the user's device.

[1211] Step 5:

[1212] Answer acceptance

[1213] The user answers the presented quiz by selecting an option or entering text.

[1214] Input: The user's answer to the quiz.

[1215] Output: The device sends the user's answer to the server.

[1216] Step 6:

[1217] Answer assessment and evaluation

[1218] The server compares the received answer with a database of correct answers to determine whether it is correct or not, and simultaneously evaluates the user's memory risk in combination with the recognized emotion data.

[1219] Input: User's answer, correct answer database, recognized emotion data.

[1220] Data processing / calculation: A correct / incorrect judgment algorithm is used to calculate the correct answer rate and assess memory risk.

[1221] Output: Judgment result and memory risk score.

[1222] Step 7:

[1223] Providing Feedback

[1224] The server generates feedback based on the assessment results and emotional data, converts the assessment results into numerical values, and compares them with the user's past answer history. It also provides feedback recommending appropriate actions.

[1225] Input: Judgment results, emotion data, past answer history.

[1226] Data processing / computation: Feedback generation algorithms are used to quantify assessment results and recommend actions.

[1227] Output: Feedback is displayed on the user's device.

[1228] Through these steps, users can have fun while activating their memory and undergoing a cognitive risk assessment.

[1229] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1230] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1231] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1232] [Third embodiment]

[1233] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1234] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1235] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1236] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1237] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1238] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1239] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1240] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1241] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1242] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1243] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1244] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1245] Introduction

[1246] The present invention relates to a system that aims to activate a user's memory in a fun way and to detect and prevent dementia early. Specifically, the system generates quizzes based on data collected from the user's device and periodically evaluates the user's memory and cognitive status. The following describes in detail the embodiments of the present invention.

[1247] System configuration

[1248] This system mainly consists of the following components:

[1249] 1. User's Device

[1250] Devices that users use on a daily basis, such as smartphones and tablets.

[1251] The terminal performs the functions of collecting data, displaying quizzes, accepting answers, and providing feedback.

[1252] 2. Server

[1253] Analyzes data, generates quizzes, judges answers, and assesses memory and cognitive risk.

[1254] It is equipped with an artificial intelligence module that automates data analysis and quiz generation.

[1255] 3. Network

[1256] An internet connection that allows communication between your device and our server.

[1257] Program processing

[1258] Data collection

[1259] Server: Sends data collection requests to the user's device.

[1260] On your device: With your permission, we collect photos, calendar events, and diary data.

[1261] Terminal: Sends collected data to the server.

[1262] Examples:

[1263] Photos taken over the weekend, appointments on your calendar, and events recorded in your autobiography app.

[1264] Generate a quiz

[1265] Server: Analyzes the received data using an artificial intelligence module and generates a quiz to stimulate the user's memory.

[1266] Server: Sends the generated quiz to the user's device.

[1267] Examples:

[1268] Photo analysis results: "Where was this photo taken?"

[1269] Calendar event analysis results: "What events happened on this day?"

[1270] Quiz distribution and answers

[1271] On the device: Show the quiz notification to the user and open the quiz screen.

[1272] User: Answers the quiz by selecting options or entering text in response to the questions presented.

[1273] Terminal: Sends the user's answer to the server.

[1274] Examples:

[1275] Choose a location from the options for the photo question.

[1276] Enter detailed text for calendar events.

[1277] Judging and evaluating answers

[1278] Server: Compares the received answer with the correct answer database and determines whether it is correct or not.

[1279] Server: Records the results of the judgment and calculates values ​​to assess the user's memory and cognitive risk.

[1280] Examples:

[1281] If the user answers "park" and the correct answer is "park," it is determined to be the correct answer.

[1282] Based on the memory risk assessment, it is compared with the user's past answer history.

[1283] Providing Feedback

[1284] Server: Generates feedback to the user based on the judgment results.

[1285] Server: Sends feedback to the user's device.

[1286] Terminal: Show a feedback screen to the user.

[1287] Examples:

[1288] If the correct answer rate is high, the feedback is "You have maintained a good memory."

[1289] If the perceived risk is high, you will be notified that "we recommend that you visit a medical institution."

[1290] Example

[1291] User A launches Recoco Memory for the first time and allows the app to access photos, calendar, and diary information. The server receives the initial data and begins analysis. Based on the analysis results, multiple quizzes are generated from User A's photos and calendar events from the past week. For example, the server generates a quiz such as "Where did you visit last weekend?" and sends it to the user's device.

[1292] When User A answers the quiz, the answer data is sent to the server for analysis and evaluation. The server determines whether User A's answer is correct and evaluates the memory status based on the result. The evaluation result is sent as feedback to User A's device, and User A receives a notification that their memory status is "good."

[1293] In this way, by using the system of the present invention, users can enjoy checking their memories and assessing cognitive risk in their daily lives.

[1294] The processing flow will be explained below.

[1295] Step 1:

[1296] Users: Download and install the app on their devices such as smartphones and tablets.

[1297] Step 2:

[1298] Device: When the app is launched for the first time, a user registration screen is displayed and the user is asked to enter personal information (such as name, email address, and password).

[1299] Step 3:

[1300] Server: Receives user registration information, creates a new user account in the database, and sends a success message to the terminal.

[1301] Step 4:

[1302] On device: Prompt the user for permission to access their photos, calendar, and diary data.

[1303] Step 5:

[1304] User: Select "Allow" in the permission dialog to grant the app data access.

[1305] Step 6:

[1306] Device: Collects data for which permission has been granted (photos, calendar, diary information) and sends it to the server.

[1307] Step 7:

[1308] Server: The AI ​​module analyzes the received data and generates quizzes based on the photos and calendar information.

[1309] For example, generate questions based on photo locations or calendar events.

[1310] Step 8:

[1311] Server: Sends the generated quiz to the user's device.

[1312] Step 9:

[1313] On the device: A quiz notification is displayed to the user. When the user taps the notification, the quiz screen is displayed.

[1314] Step 10:

[1315] User: Answers the quiz by selecting options or entering text in response to the questions presented.

[1316] Step 11:

[1317] Terminal: Sends the user's answer to the server.

[1318] Step 12:

[1319] Server: Compares the received answer with the correct answer database and determines whether it is correct or not.

[1320] Step 13:

[1321] Server: Records the results of the judgment and calculates values ​​to assess the user's memory and cognitive risk.

[1322] Example: Calculating accuracy rate and memory risk score.

[1323] Step 14:

[1324] Server: Generates feedback to provide to the user based on the judgment results.

[1325] Step 15:

[1326] Server: Sends feedback to the user's device.

[1327] Step 16:

[1328] Device: A feedback screen is displayed to the user, presenting the evaluation results and recommended actions (e.g., visiting a medical institution).

[1329] Step 17:

[1330] User: Review the feedback and take any recommended action (e.g., seek medical attention) if necessary.

[1331] This series of steps allows users to have fun while checking their memories and assessing cognitive risk.

[1332] Example 1

[1333] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1334] There is a lack of efficient systems to stimulate users' memories in an enjoyable way and to detect and prevent dementia early. In addition, there are insufficient means to accurately assess users' memory status and cognitive risk and provide feedback.

[1335] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1336] In this invention, the server includes means for collecting image data, schedule management data, and record data from the user's computing device, means for analyzing the collected data and using a machine learning algorithm to generate questions, and means for delivering the generated questions to the user's computing device, thereby making it possible to automatically generate questions to stimulate the user's memory, evaluate memory and cognitive risks based on the answers, and provide appropriate feedback.

[1337] "Computing device" refers to an information processing device that users use on a daily basis, and specifically includes smartphones, tablets, personal computers, etc.

[1338] "Image data" refers to photographs taken by users and image files acquired by users.

[1339] "Schedule management data" refers to calendar application data that records a user's schedule and appointments.

[1340] "Recorded data" refers to text data recorded by the user as a diary or memo.

[1341] "Machine learning algorithm" refers to an algorithm used to analyze collected data and generate questions to stimulate a user's memory, and may include an artificial intelligence module.

[1342] "Feedback" refers to information and advice provided to users based on the results of an assessment of their memory status and cognitive risk.

[1343] "Medical institution" refers to a medical facility such as a hospital or clinic that provides medical services.

[1344] MODE FOR CARRYING OUT THE INVENTION

[1345] Introduction

[1346] The present invention relates to a system for activating a user's memory in a fun way, with the aim of early detection and prevention of dementia. The system generates quizzes based on data collected from the user's computing device and periodically assesses the user's memory and cognitive status. Specific embodiments for implementing the invention are described in detail below.

[1347] System configuration

[1348] This system mainly consists of the following components:

[1349] 1. Your Computing Device

[1350] Information processing devices such as smartphones, tablets, and personal computers.

[1351] The computing device has the functions of collecting data, displaying quizzes, accepting answers, and providing feedback.

[1352] 2. Server

[1353] Analyzes data, generates quizzes, judges answers, and assesses memory and cognitive risk.

[1354] It is equipped with an artificial intelligence module that automates data analysis and quiz generation (e.g., GPT-3).

[1355] 3. Network

[1356] An internet connection that allows communication between the computing device and the server.

[1357] Data collection

[1358] The server sends a data collection request to the user's computing device. Based on the user's permission, the computing device collects image data, schedule management data, and record data. The computing device then sends the collected data to the server. Specific examples include photos taken by the user over the weekend, appointments listed on a calendar, and events recorded in a personal history app.

[1359] Generate a quiz

[1360] The server analyzes the received data using an artificial intelligence module and generates a quiz to stimulate the user's memory. The server then sends the generated quiz to the user's computing device. Specific examples include quizzes such as "Where was this photo taken?" based on the analysis of a photo, or "What events happened on this day?" based on the analysis of calendar events.

[1361] Quiz distribution and answers

[1362] The computing device displays a quiz notification to the user and opens the quiz screen. The user answers the quiz by selecting options or entering text in response to the presented questions. The computing device then sends the user's answers to the server. Specific examples of answers include selecting a location from options in response to a photo question or entering detailed actions in response to a calendar event.

[1363] Judging and evaluating answers

[1364] The server compares the received answer with a correct answer database to determine whether it is correct or incorrect. The server then records the result of the determination and calculates a value to evaluate the user's memory and cognitive risk. For example, if the user answers "park" and the correct answer is "park," the answer is determined to be correct. Furthermore, based on the memory risk assessment, the result is compared with the user's past answer history.

[1365] Providing Feedback

[1366] The server generates feedback for the user based on the assessment results. The server then sends the generated feedback to the user's computing device. The computing device displays a feedback screen to the user. For example, if the accuracy rate is high, feedback such as "Your memory is in good condition" is displayed, and if the cognitive risk is high, a notification such as "We recommend that you visit a medical institution" is displayed.

[1367] Example

[1368] User A launches the system for the first time and allows the app to access image data, schedule management data, and record data. The server receives the initial data and begins analysis. Based on the analysis results, multiple quizzes are generated from User A's photos and calendar events from the past week. For example, the server generates a quiz question such as, "Where did you visit last weekend?" and sends it to the user's computing device. When User A answers the quiz, the answer data is sent to the server for analysis and evaluation. The server determines whether User A's answer is correct and evaluates his or her memory status based on the result. The evaluation result is sent as feedback to User A's computing device, and User A is notified that his or her memory status is "good." In this way, using the system of the present invention, users can enjoy checking their memory and assessing cognitive risk in their daily lives.

[1369] Examples of prompt statements

[1370] "Generate quizzes from the user's photo data collected by the system, deliver the generated quizzes to the user's computing device, and provide feedback based on the results."

[1371] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1372] Step 1: Submit a data collection request

[1373] The server sends a data collection request to the user's computing device, the request including information directing the collection of image data, schedule management data, and record data.

[1374] Input: None

[1375] Output: Data collection request

[1376] Step 2: Data collection

[1377] The device collects image data, schedule management data, and record data from the user based on the data collection request received from the server. Specifically, it obtains photos from the camera roll of the smartphone or tablet, schedules from the calendar app, and text information from the diary app.

[1378] Input: Data Collection Request

[1379] Output: Collected data (photos, schedules, diaries)

[1380] Step 3: Sending data

[1381] The terminal transmits collected image data, schedule management data, and record data to the server. This data is transmitted based on the user's permission.

[1382] Input: Collected data

[1383] Output: Send data to the server

[1384] Step 4: Data analysis

[1385] The server analyzes the received data using a generative AI model (e.g., GPT-3). This analysis includes photo recognition, text analysis, and calendar event identification. Specifically, image recognition algorithms are used to analyze the content of photos, and natural language processing algorithms are used to analyze text data.

[1386] Input: Collected data

[1387] Output: Analysis results

[1388] Step 5: Quiz Generation

[1389] The server generates quizzes to stimulate the user's memory based on the analysis results. For example, it generates questions such as "Where was this photo taken?" based on the location information of the photo. It also generates questions such as "What events happened on this day?" based on calendar events.

[1390] Input: Analysis results, generated AI model

[1391] Output: The generated quiz

[1392] Step 6: Stream the quiz

[1393] The server then sends the generated quiz to the user's computing device, formatted in a way that is easy for the user to answer.

[1394] Input: Generated quiz

[1395] Output:Quiz delivery

[1396] Step 7: Quiz Notification and Display

[1397] The device notifies the user of the quiz received from the server and displays the quiz screen. Specifically, the device notifies the user of the existence of a new quiz using the notification center or in-app notifications, and displays the quiz screen.

[1398] Input:Quiz broadcast

[1399] Output: Quiz notification to user, quiz screen display

[1400] Step 8: Take the quiz

[1401] The user can select an answer from the options or enter a text answer for the quiz. Specifically, the user selects the correct answer from the options displayed or enters the answer using the keyboard.

[1402] Input: Quiz screen

[1403] Output: User's answer

[1404] Step 9: Submit your answers

[1405] The device sends the user's answers to the server, where they are accurately recorded and securely transmitted to the server.

[1406] Input: User's answer

[1407] Output: Send the answer to the server

[1408] Step 10: Determine the answer

[1409] The server compares the received answer with a database of correct answers to determine whether the answer is correct or not. Specifically, it matches the user's answer with the correct answers stored in the database.

[1410] Input: User's answer

[1411] Output: Judgment result

[1412] Step 11: Valuation Calculation

[1413] The server calculates a numerical value to evaluate the user's memory and cognitive risk based on the judgment result. For example, the server evaluates the user's memory ability and cognitive risk based on the accuracy rate and reaction time, and generates a score.

[1414] Input: Judgment result

[1415] Output: Evaluation score

[1416] Step 12: Generate feedback

[1417] The server generates feedback for the user based on the evaluation score, such as a message indicating good memory status or a message recommending that the user seek medical attention.

[1418] Input: Rating score

[1419] Output: Feedback message

[1420] Step 13: Submit your feedback

[1421] The server then transmits the generated feedback to the user's computing device, where the feedback is user-friendly and encourages action.

[1422] Input: Feedback message

[1423] Output: Send feedback to the user

[1424] Step 14: Viewing feedback

[1425] The device displays the feedback message received from the server to the user, specifically, by providing the feedback to the user using an in-app notification or a pop-up window.

[1426] Input: Send Feedback

[1427] Output: Show feedback

[1428] (Application example 1)

[1429] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1430] In an aging society, the prevention and early detection of dementia are important issues. However, existing methods lack systems that allow users to enjoyably assess memory in their daily lives and determine the risk of cognitive function without visiting a medical institution. Furthermore, even in physical stores, there are no systems that promote memory activation through the shopping experience, so there is a need to integrate dementia prevention and shopping.

[1431] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1432] In this invention, the server includes means for collecting photo, calendar, and diary data, as well as purchase and check-in histories from the user's device, an artificial intelligence module for analyzing the collected data and generating quizzes, and means for delivering and notifying the generated quizzes to the user's device. This enables the user's memory to be stimulated in an enjoyable way through everyday shopping experiences, thereby enabling the prevention and early detection of dementia.

[1433] "User device" refers to a device such as a smartphone, tablet, or smart glasses that a user uses on a daily basis to collect data, display quizzes, and accept answers.

[1434] "Data collection" is the process of collecting purchase history, check-in history, photos, calendar, and diary data from users' devices.

[1435] "Purchase history" refers to a record and detailed information about products purchased by a user in a physical store.

[1436] "Check-in history" is a record of a user's visits to specific locations or stores, and is data that includes location information.

[1437] The "artificial intelligence module" is a system consisting of programs and algorithms for analyzing collected data and generating quizzes to stimulate the user's memory.

[1438] "Quiz generation" is the process of automatically creating questions to assess a user's memory based on collected and analyzed data.

[1439] "Quiz delivery" is the process of sending the generated quiz to the user's device and notifying them.

[1440] "Answer determination" is the process of analyzing the answers given by the user to the quiz and comparing them with the correct answers to determine whether they are correct or incorrect.

[1441] "Memory status assessment" is the process of assessing a user's memory and cognitive function by comparing it with the user's past answer history.

[1442] "Feedback" is the process of informing the user of information about their memory and cognitive status and appropriate actions based on their answers and the assessment results.

[1443] "Recommendation to visit a medical institution" is the process of notifying the user to visit a medical institution based on the evaluation results.

[1444] "Quantification" is the process of converting evaluation results and answer history into quantitative data to make it easier to evaluate the user's memory state.

[1445] Configuring the various components

[1446] The present invention relates to a system for activating a user's memory and assessing their cognitive status. The system comprises the following components:

[1447] 1. User's Device

[1448] These are devices that users use on a daily basis (e.g., smartphones, tablets, smart glasses, etc.). These devices collect data such as users' purchase history, check-in history, photos, calendars, and diaries. They also perform functions such as displaying quizzes, accepting answers, and providing feedback.

[1449] 2. Server

[1450] The server analyzes data, generates quizzes, judges answers, and assesses memory and cognitive risk. It is equipped with an artificial intelligence module to automate these processes, and often uses cloud services such as Google Cloud and Amazon Web Services (AWS).

[1451] 3. Network

[1452] Refers to an internet connection that enables communication between a device and a server, and can use Wi-Fi, 4G, 5G, or other mobile networks.

[1453] Data collection

[1454] 1. Terminal

[1455] With user permission, data such as purchase history, check-in history, photos, calendar events, and diary entries are collected and transmitted securely to the server.

[1456] Generate a quiz

[1457] 2. Server

[1458] The received data is analyzed using an artificial intelligence module (e.g., TensorFlow, PyTorch) to generate a quiz to stimulate the user's memory. The quiz is automatically created and sent to the user's device.

[1459] Quiz distribution and answers

[1460] 3. Terminal

[1461] A quiz notification is displayed to the user, and the quiz screen opens. The user answers the quiz by selecting options or entering text in response to the questions presented.

[1462] 4. Users

[1463] The user answers the quiz appropriately and transmits the answers to the server via the terminal.

[1464] Judging and evaluating answers

[1465] 5. Server

[1466] The received answer is compared with a correct answer database to determine whether it is correct or incorrect. The result of the determination is then recorded and compared with past answer history to evaluate the memory status.

[1467] Providing Feedback

[1468] 6. Server

[1469] Based on the results of the assessment, feedback is generated for the user, including an assessment of memory status and recommended actions (e.g., a recommendation to see a doctor). The generated feedback is sent to the user's device.

[1470] 7. Terminal

[1471] A feedback screen is displayed to the user, allowing them to check their memory status and recommended actions.

[1472] Specific examples

[1473] When a user is shopping in a physical store, they may purchase a specific item at the register (a retro-designed drink that they have purchased in the past). The system collects this data and generates a quiz based on their purchase history, asking, "What flavor was the retro cola you purchased two weeks ago?" If the user answers the quiz correctly, the system provides feedback such as, "Your memory is good!"

[1474] Prompt Sentence Examples

[1475] "What flavor of retro cola did you buy two weeks ago?"

[1476] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1477] Step 1:

[1478] Data collection

[1479] Device: First, with the user's permission, data such as purchase history, check-in history, photos, calendar events, and diary entries are collected. This data is either collected automatically by the application or entered manually by the user. The collected data is converted into an appropriate data format, such as JSON, and sent to the server while protecting privacy.

[1480] Input: User purchase history, check-in history, photos, calendar events, diary data

[1481] Output: JSON data formatted to send to the server

[1482] Step 2:

[1483] Generate a quiz

[1484] Server: Analyzes the received data. Specifically, it extracts information for generating quizzes from the collected data through text analysis (natural language processing) and image analysis. It uses artificial intelligence modules (e.g., TensorFlow, PyTorch) to automatically generate quizzes that stimulate the user's memory.

[1485] Input: JSON data sent from the terminal

[1486] Output: Generated quiz questions and choices

[1487] Step 3:

[1488] Quiz broadcast

[1489] Server: The server delivers the generated quiz to the user's device and sends a notification. The quiz content is pushed to the device using the REST API.

[1490] On the device: The received notification is displayed to the user and the quiz screen is opened. When the user taps the notification, the specific quiz content is displayed.

[1491] Input: Generated quiz

[1492] Output: Quiz notification and quiz screen

[1493] Step 4:

[1494] Quiz Answers

[1495] User: Answers the quiz by selecting an answer from the options provided or by entering text.

[1496] Terminal: Receives the user's answer and sends it to the server. The data sent is in JSON format, including the answer.

[1497] Input: User's quiz answer

[1498] Output: Answer data to send to the server

[1499] Step 5:

[1500] Judging and evaluating answers

[1501] Server: Compares the received user answers with a database of correct answers to determine whether they are correct or not. It also compares the answers with the user's past answer history to assess memory status and cognitive risk. Statistical analysis and machine learning algorithms are used for this.

[1502] Input: User's answer data

[1503] Output: Correct / incorrect judgment and memory status evaluation results

[1504] Step 6:

[1505] Providing Feedback

[1506] Server: Generates feedback to the user based on the correctness of the answers and the evaluation results. The feedback includes an evaluation of memory status and recommended actions (e.g., recommendation to see a doctor).

[1507] Device: Receives the generated feedback and displays it to the user, allowing the user to check their own memory status and course of action.

[1508] Input: Correct / incorrect judgment and memory status evaluation results

[1509] Output: Feedback that is displayed on the user's device

[1510] Examples of prompt statements

[1511] "What flavor of retro cola did you buy two weeks ago?"

[1512] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1513] Introduction

[1514] The present invention relates to a system that aims to activate a user's memory in a fun way and to detect and prevent dementia early. In particular, the present invention provides a more personalized service by recognizing the user's emotions and adjusting the quiz generation and feedback content.

[1515] System configuration

[1516] The system consists of the following main components:

[1517] 1. User's Device

[1518] These are devices that users use on a daily basis, such as smartphones and tablets.

[1519] It performs functions such as data collection, emotion recognition, quiz display, answer acceptance, and feedback provision.

[1520] 2. Server

[1521] Analyzes data, generates quizzes, judges answers, and assesses memory and cognitive risk.

[1522] It is equipped with an artificial intelligence module and an emotion engine to automate data analysis and quiz generation.

[1523] 3. Network

[1524] It is an internet connection that allows communication between the device and the server.

[1525] Program processing

[1526] Data collection and emotion recognition

[1527] Server: Sends data collection requests to the user's device.

[1528] On the device: With the user's permission, it collects photos, calendar events, and diary data, and also uses the device's camera and microphone to capture the user's facial expressions and voice, which are then sent to the emotion engine.

[1529] Emotion engine: Analyzes collected facial and voice data to recognize the user's emotions.

[1530] Terminal: Sends collected data and sentiment analysis results to the server.

[1531] Examples:

[1532] When a user takes a photo or records an event in the calendar, the device sends this data to the server, and at the same time collects the user's facial expressions and voice and sends them to the emotion engine.

[1533] Quiz Generation

[1534] Server: Analyzes the received data using an artificial intelligence module and generates a quiz based on the user's memory and emotional state.

[1535] Server: Adjusts the content and difficulty of the quiz based on the emotions recognized by the emotion engine.

[1536] For example: if the user is having fun, generate a challenging quiz, if the user is stressed, generate an easy quiz.

[1537] Examples:

[1538] Photo analysis results: "Where was this taken?"

[1539] Calendar analysis results: "What events were there on this day?"

[1540] Quiz distribution and answers

[1541] Server: Sends the generated quiz to the user's device.

[1542] Device: Show the quiz notification to the user and transition to the quiz screen.

[1543] User: Answers the quiz by selecting options or entering text in response to the questions presented.

[1544] Terminal: Sends the user's answer to the server.

[1545] Examples:

[1546] A user receives a quiz notification and selects the correct answer to the question, "Where was this photo taken?"

[1547] Judging and evaluating answers

[1548] Server: Compares the received answer with the correct answer database and determines whether it is correct or not.

[1549] Server: Combines the judgment results with data from the emotion engine to calculate a numerical value for assessing the user's memory and cognitive risk.

[1550] Example: Calculating the accuracy rate and memory risk score.

[1551] Examples:

[1552] If the user answers "park" and it is the correct answer, it is determined to be the correct answer and compared with the past answer history to evaluate the memory risk.

[1553] Providing Feedback

[1554] Server: Generates personalized feedback based on the judgment results and emotion data.

[1555] Server: Sends feedback to the user's device.

[1556] Device: A feedback screen is displayed to the user, presenting the evaluation results and recommended actions (e.g., visiting a medical institution).

[1557] Examples:

[1558] If the correct answer rate is high, the system will provide feedback such as "Your memory is in good condition," and if the emotional data indicates that the user is under stress, it will also display advice such as "Relax and try the next quiz."

[1559] Example

[1560] User B launched Recoco Memory for the first time and allowed the app to access photos, calendar, and diary information. He also allowed the app to access the camera and microphone for emotion recognition. The server received the initial data and emotion data and began analysis. Based on the analysis results, multiple quizzes were generated from User B's photos and calendar events from the past week. For example, the server generated a quiz asking, "Where did you visit last weekend?" and selected a slightly more challenging quiz based on the user's emotional state (enjoyment).

[1561] When User B answers the quiz, the answer data and the latest emotional data are sent to the server for analysis and evaluation. The server determines whether User B's answer is correct and evaluates his / her memory and emotional state based on the result and the emotional data. As feedback, the device displays a notification saying "Good memory state" and "Enjoy the next quiz."

[1562] In this way, by using the system of the present invention, users can not only have fun while checking their memory and assessing cognitive risk, but also receive personalized services that take into account their emotional state.

[1563] The processing flow will be explained below.

[1564] Step 1:

[1565] User: Download and install Ricoco Memory on a device such as a smartphone or tablet.

[1566] Step 2:

[1567] Device: When the app is launched for the first time, a user registration screen is displayed and the user is asked to enter personal information (such as name, email address, and password).

[1568] Step 3:

[1569] Server: Receives user registration information, creates a new user account in the database, and sends a success message to the terminal.

[1570] Step 4:

[1571] Device: Prompts the user for permission to access photos, calendar, diary data, and the camera and microphone.

[1572] Step 5:

[1573] User: Select "Allow" in the permission dialog to grant the app data access.

[1574] Step 6:

[1575] Device: Collects data with permission (photos, calendar events, diary information) and sends it to the server. It also uses the camera and microphone to transmit the user's facial expressions and voice to the emotion engine.

[1576] Step 7:

[1577] Emotion engine: Analyzes collected facial and voice data to recognize the user's emotions.

[1578] Step 8:

[1579] Terminal: Sends collected data and sentiment analysis results to the server.

[1580] Step 9:

[1581] Server: Analyzes the received data using an AI module and generates a quiz based on the user's memory and emotional state. The emotion engine adjusts the content and difficulty of the quiz based on the emotions recognized by the server.

[1582] For example: if the user is having fun, generate a challenging quiz, if the user is stressed, generate an easy quiz.

[1583] Step 10:

[1584] Server: Sends the generated quiz to the user's device.

[1585] Step 11:

[1586] Device: Show the quiz notification to the user and transition to the quiz screen.

[1587] Step 12:

[1588] User: Answers the quiz by selecting options or entering text in response to the questions presented.

[1589] Step 13:

[1590] Terminal: Sends the user's answer to the server.

[1591] Step 14:

[1592] Server: Compares the received answer with the correct answer database and determines whether it is correct or not.

[1593] Step 15:

[1594] Server: Combines the judgment results with data from the emotion engine to calculate a numerical value for assessing the user's memory and cognitive risk.

[1595] Example: Calculating the accuracy rate and memory risk score.

[1596] Step 16:

[1597] Server: Generates personalized feedback based on the judgment results and emotion data.

[1598] Step 17:

[1599] Server: Sends feedback to the user's device.

[1600] Step 18:

[1601] Device: A feedback screen is displayed to the user, presenting the evaluation results and recommended actions (e.g., visiting a medical institution).

[1602] Step 19:

[1603] User: Review the feedback and take any recommended action (e.g., seek medical attention) if necessary.

[1604] Through this series of steps, users can not only enjoyably check their memories and assess their cognitive risks, but also receive personalized services that take into account their emotional state.

[1605] Example 2

[1606] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1607] In modern society, early detection and prevention of dementia are important issues, especially in the health management of the elderly. However, conventional memory assessment tools are monotonous, and users lack motivation to continue using them. Furthermore, there is a need for systems that do not take into account the user's emotional state and can be used in a fun and stress-free manner. Furthermore, there is a lack of personalized feedback and recommendations for appropriate actions tailored to individual needs. In response to these challenges, the present invention provides a system that allows users to activate their memory in an enjoyable way and provides personalized services based on their emotional state.

[1608] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1609] In this invention, the server includes means for collecting photo, schedule, and document data from the user's terminal, means for analyzing the collected data and the user's facial expressions and voice to recognize the emotional state, an artificial intelligence module for generating quizzes based on the analyzed data and the emotional state, means for adjusting the content and difficulty of the generated quizzes based on the emotional state, means for delivering the generated quizzes to the user's terminal, means for receiving the user's answers and determining whether the answers are correct, means for assessing the user's memory and cognitive risk based on the assessment results and the emotional state, and means for providing the user with feedback recommending the assessment results and appropriate actions. This allows the user to have fun while activating their memory and receiving personalized feedback and recommended actions.

[1610] "User's device" refers to a device that a user uses on a daily basis, such as a smartphone or tablet.

[1611] "Photo" refers to a still image taken by a User using a Device.

[1612] "Calendar" refers to the schedule information that a user records on a device.

[1613] "Document" refers to text data that a user creates or saves using a device.

[1614] "Means of collecting data" refers to the functionality for obtaining data such as photos, schedules, and documents from the user's device.

[1615] "Facial expressions and voice" refers to the user's facial expressions and spoken voice data.

[1616] "Means for recognizing emotional states" refers to a function that analyzes collected facial and voice data to identify the user's emotions (happiness, sadness, surprise, etc.).

[1617] "Artificial intelligence module means" refers to a processing system that uses artificial intelligence to analyze data and generate quizzes.

[1618] "Quiz" refers to questions or problems designed to assess a user's memory or cognitive abilities.

[1619] "Means for adjusting the content and difficulty of the quiz" refers to a function for appropriately adjusting the content and difficulty of the quiz questions based on the user's emotional state.

[1620] "Means for delivering quizzes" refers to the function for sending the generated quiz to the user's terminal.

[1621] "Means for receiving answers" refers to a function for transmitting the answers given by the user to the quiz from the user's terminal to the server and receiving them.

[1622] "Means for determining whether the answer is correct" refers to a function for comparing the received answer with pre-set correct answer data and determining whether the answer is correct or incorrect.

[1623] "Means for assessing memory and cognitive risk" refers to a function for assessing the risk of a user's memory and cognitive state based on the correctness or incorrectness of answers and emotional state.

[1624] "Means for providing feedback" refers to a function for generating feedback to recommend appropriate actions to the user based on the evaluation results and sending it to the user's device.

[1625] This invention relates to a system that aims to stimulate a user's memory in a fun way and to detect and prevent dementia early. In particular, it provides a more personalized service by recognizing the user's emotions and adjusting the quiz generation and feedback content.

[1626] System configuration

[1627] The system consists of the following main components:

[1628] 1. User's Device

[1629] These are devices that users use on a daily basis, such as smartphones and tablets. Specific hardware examples include Android and iOS-based devices.

[1630] Software: A dedicated application that collects data, recognizes emotions, displays quizzes, accepts answers, and provides feedback.

[1631] 2. Server

[1632] The server has the functionality to analyze data, generate quizzes, judge answers, and assess memory and cognitive risk.

[1633] Software: Artificial intelligence module and emotion engine, which automates data analysis and quiz generation.

[1634] 3. Network

[1635] An internet connection that allows communication between a device and a server, allowing data to be exchanged.

[1636] Data collection and emotion recognition

[1637] Server: Sends a data collection request to the user's device. The user's device receives this request and, after obtaining the user's permission, collects photo, calendar, and document data. It also uses the device's camera and microphone to collect the user's facial and voice data and sends it to the emotion engine. The emotion engine analyzes this data and recognizes the user's emotions.

[1638] Examples:

[1639] Users take everyday photos with their smartphone camera and register travel events in their calendar. The device sends this data to the server and simultaneously collects the user's facial expressions and voice. The emotion engine detects "enjoyment" from the user's smile, tone of voice, etc.

[1640] Generate a quiz

[1641] Server: Receives the results from the emotion engine and generates a quiz based on the user's memory and emotional state.

[1642] Artificial Intelligence Module: This determines the content and difficulty of the questions. Quizzes are created using a generative AI model, with example prompts such as:

[1643] Example prompt sentence:

[1644] "Generate the following quiz based on user A's photo data. User A's emotional state seems to be fun. The content of the quiz should be something that user A would enjoy. Specific example questions include, 'Where was this photo taken?' and 'What was this event?'"

[1645] The generated quizzes are adjusted in content and difficulty based on the emotional state.

[1646] Quiz distribution and answers

[1647] Server: Sends the generated quiz to the user's device.

[1648] Device: Show the quiz notification to the user and transition to the quiz screen.

[1649] Examples:

[1650] Users are given a quiz asking, "Where was this photo taken?" and are asked to select the appropriate location.

[1651] Judging and evaluating answers

[1652] Server: Receives the user's answer data, compares it with the correct answer database, and determines whether it is correct or incorrect. The result of the evaluation is combined with the emotion engine data to calculate a value that evaluates the user's memory and cognitive risk.

[1653] Examples:

[1654] If the user answers "park," if that is the correct answer, it is determined to be the correct answer and compared with past answer history to assess memory risk.

[1655] Providing Feedback

[1656] Server: Generates personalized feedback based on the judgment results and emotion data. The feedback is sent to the device and displayed on the feedback screen.

[1657] Examples:

[1658] If the correct answer rate is high, the system will provide feedback such as "Your memory is in good condition," and if it determines that the user is feeling stressed, it will advise them to "relax and try the next quiz."

[1659] This allows users to have fun while activating their memories and receiving personalized feedback.

[1660] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1661] Step 1:

[1662] Server: Sends data collection requests to the device.

[1663] Input: None

[1664] Output: Sending a data collection request

[1665] Specific operation: The server sends data collection instructions to the user's device via an HTTP request, which includes the specific data to collect (photos, schedules, documents) and how to collect it.

[1666] Step 2:

[1667] Device: Obtains user permission to collect data. If the user gives permission, the device will collect photos, calendar, and document data. It will also use the device's camera and microphone to collect facial and voice data.

[1668] Input: Data collection request from the server

[1669] Output: Collected data (photos, schedules, documents, facial expressions, voice)

[1670] How it works: The device running the app collects photos taken by the user, event information registered in the calendar, and document data, while also using the camera and microphone to collect real-time facial expressions and voice recordings.

[1671] Step 3:

[1672] Emotion engine: Analyzes collected facial and voice data to recognize the user's emotional state.

[1673] Input: Collected facial expression and voice data

[1674] Output: Emotion recognition result

[1675] Specific operation: The device sends the facial expression data captured by the device to an image recognition algorithm, and the recorded voice data to a voice analysis engine, which then recognizes the user's emotions (happiness, sadness, surprise, etc.).

[1676] Step 4:

[1677] Terminal: Sends collected data and sentiment analysis results to the server.

[1678] Input: Collected data and emotion recognition results

[1679] Output: Send data to the server

[1680] What happens: The device converts the data into the appropriate format and sends it to the server. The data is sent via an HTTP POST request, subject to user permission.

[1681] Step 5:

[1682] Server: Inputs the received data into an artificial intelligence module and generates a quiz based on the user's memory and emotional state.

[1683] Input: Collected data and emotion recognition results

[1684] Output: The generated quiz

[1685] How it works: The server inputs data into the AI ​​module and generates a quiz, such as "Based on your calendar events, what were the events on this day?"

[1686] Step 6:

[1687] Server: Adjusts the content and difficulty of the quiz based on the emotions recognized by the emotion engine.

[1688] Input: Generated quiz and emotion recognition results

[1689] Output: Adjusted quiz

[1690] What it does: If the user is having fun, choose a challenging quiz; if the user is stressed, choose an easy quiz.

[1691] Step 7:

[1692] Server: Sends the generated quiz to the user's device.

[1693] Input: Adjusted Quiz

[1694] Output: Quiz stream

[1695] Specific operation: The server sends the quiz in JSON format to the device, and the device receives it and displays a quiz notification.

[1696] Step 8:

[1697] Device: Show the quiz notification to the user and transition to the quiz screen.

[1698] Input: Quiz Stream

[1699] Output: Quiz screen display

[1700] What happens: When the user clicks on the notification, the application transitions to a quiz screen where the question is displayed.

[1701] Step 9:

[1702] User: Answers the quiz by selecting options or entering text in response to the questions presented.

[1703] Input: Question on the quiz screen

[1704] Output: User's answer

[1705] Specific behavior: The user selects an answer from the options and clicks the "Submit" button.

[1706] Step 10:

[1707] Terminal: Sends the user's answer to the server.

[1708] Input: User's answer

[1709] Output: Send the answer to the server

[1710] Specific operation: The device converts the answer data into an appropriate format and sends it to the server via an HTTP POST request.

[1711] Step 11:

[1712] Server: Compares the received answer data with the correct answer database to determine whether it is correct or not.

[1713] Input: User's answer

[1714] Output: Judgment result

[1715] Specific operation: The server compares the received answer with the correct answer data and determines whether it is correct or incorrect.

[1716] Step 12:

[1717] Server: Combines the judgment results and data from the emotion engine to assess the user's memory and cognitive risk.

[1718] Input: Judgment results and emotion recognition results

[1719] Output: Memory and cognitive risk assessment

[1720] Specific operation: The server calculates an evaluation score based on the correct answer rate, past answer history, and emotional data.

[1721] Step 13:

[1722] Server: Generates personalized feedback based on the judgment results and emotion data.

[1723] Input: Memory and Cognitive Risk Assessment

[1724] Output: Feedback message

[1725] Specific behavior: Generate a specific feedback message according to the user's state (e.g., "Your memory is in good condition"). Examples of prompt sentences include:

[1726] Example prompt sentence:

[1727] "Generate the following feedback based on User B's quiz answer data and emotional data. User B's accuracy rate is high and their emotional state is positive. The feedback should include praise for User B's memory and positive advice for the next quiz."

[1728] Step 14:

[1729] Server: Sends feedback to the user's device.

[1730] Input: Feedback message

[1731] Output: Feedback distribution

[1732] Specific operation: The server sends the generated feedback message to the terminal, and the terminal receives it and notifies the user.

[1733] Step 15:

[1734] On the device: A feedback screen is displayed to the user, showing the evaluation results and recommended actions.

[1735] Input: Feedback message

[1736] Output: Display feedback

[1737] Specific actions: The device displays a feedback message and provides the user with specific advice or recommended actions.

[1738] Through these steps, the system allows users to have fun while activating their memories and receive personalized feedback and recommendations for action.

[1739] (Application example 2)

[1740] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1741] To activate users' memories in a fun way and to detect and prevent dementia early, personalized services are required. However, existing technologies only provide uniform services without considering the user's emotions, making effective memory activation difficult. Furthermore, there are also few ways for users to check their knowledge and memories while having fun. Therefore, a system is needed that provides personalized quizzes and feedback based on the user's emotions, and that can activate memory and assess cognitive risk.

[1742] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1743] In this invention, the server includes: means for collecting photo, calendar, and diary data from the user's terminal; emotion engine means for analyzing the collected data and recognizing the user's emotions; artificial intelligence module means for generating quizzes based on the recognized emotions; means for delivering the generated quizzes to the user's terminal; means for receiving the user's answers and determining whether the answers are correct; means for assessing the user's memory and cognitive risk based on the determination results; means for providing the user with feedback on the evaluation results and recommending appropriate actions; and means for adjusting the difficulty of the quiz in accordance with the user's emotions based on the presentation of the quiz. This allows for the provision of personalized quizzes and feedback in accordance with the user's emotions, enabling memory activation and cognitive risk assessment in a fun environment.

[1744] "User's device" refers to a device that a user uses on a daily basis, including a smartphone or tablet.

[1745] "Photo" refers to image data taken by a user using a camera.

[1746] "Calendar" refers to a scheduling feature or application that allows users to manage their schedules and events.

[1747] A "diary" refers to text data that users use to record their daily events and thoughts.

[1748] "Data collection means" refers to a method or device that provides functionality for acquiring photo, calendar, and diary data from a user's device.

[1749] The "emotion engine" is a software module that analyzes collected data and recognizes the user's emotions.

[1750] The "quiz generation means" is a system for generating quizzes based on recognized emotions using artificial intelligence.

[1751] The "quiz distribution means" refers to a method or device for transmitting the generated quiz to the user's terminal and displaying the quiz.

[1752] The "answer determination means" is a system that receives the user's answer and determines whether it is correct or incorrect.

[1753] The "memory and cognitive risk assessment means" refers to a function or device for assessing the memory state and cognitive risk of a user based on the judgment results.

[1754] The "feedback providing means" is a function that provides the user with the evaluation results and feedback to recommend appropriate actions.

[1755] The "difficulty level adjustment means" is a system for adjusting the difficulty level of a quiz in accordance with the user's emotions based on the presentation of the quiz.

[1756] System Overview

[1757] The system of this invention aims to stimulate the user's memory in a fun way and to detect and prevent dementia early. The system also recognizes the user's emotions and adjusts the quiz generation and feedback content to provide personalized services.

[1758] Hardware and Software Configuration

[1759] Hardware

[1760] User devices: everyday devices such as smartphones and tablets.

[1761] Server: A server for analyzing data, generating quizzes, determining answers, and assessing memory and cognitive risk.

[1762] software

[1763] Data collection software: Collects photos, calendar, and diary data from users' devices.

[1764] Emotion engine: Software that analyzes collected data and recognizes the user's emotions.

[1765] Artificial Intelligence Module: Software for generating quizzes.

[1766] Feedback generation module: Software for generating feedback based on the answer results.

[1767] System Operation

[1768] 1. Data Collection

[1769] Your device will collect photos, calendar and diary data, with your permission.

[1770] 2. Emotion recognition

[1771] The collected data is sent to a server and analyzed by an emotion engine, which, for example, analyzes photos and audio data to recognize emotions from the user's facial expressions and voice.

[1772] 3. Quiz Generation

[1773] The AI ​​module generates quizzes based on the collected data and emotion recognition results, adjusting the difficulty of the quiz depending on the user's emotional state.

[1774] 4. Quiz distribution and answer processing

[1775] The generated quiz is delivered to the user's device, where the user answers the quiz and the device sends the answers to the server.

[1776] 5. Assessment and evaluation of answers

[1777] The server judges the answers and assesses the user's memory and cognitive risk based on the results and emotional data.

[1778] 6. Providing Feedback

[1779] The feedback generation module generates feedback based on the evaluation results and provides it to the user's terminal.

[1780] Specific examples

[1781] For example, here's a scenario where a user answers a quiz about recent events:

[1782] 1. Use photos taken by users at places they visited last weekend.

[1783] 2. The emotion engine analyzes the user's happy facial expression from the photo.

[1784] 3. The artificial intelligence module generates a quiz question: "Where did you visit last weekend?"

[1785] 4. The user answers the quiz and the answers are sent to the server.

[1786] 5. The server determines whether the answer is correct or incorrect and assesses the user's memory status and cognitive risk.

[1787] 6. The feedback generation module generates feedback such as "Good memory" and "Have fun trying the next quiz," and displays it on the user's device.

[1788] In this way, the system of the present invention provides personalized quizzes and feedback according to the user's emotions, enabling memory activation and cognitive risk assessment in a fun way.

[1789] Prompt Sentence Examples

[1790] "Analyze the photos taken by the user and their emotions at the time, and generate a related quiz. Provide appropriate feedback based on the user's answers to the quiz and their emotional data."

[1791] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1792] Step 1:

[1793] Data collection

[1794] The device collects the user's photos, calendar, and diary data with the user's permission.

[1795] Input: The user's device receives collected photos, calendar events, and diary entries.

[1796] Output: Collected data is sent to the server.

[1797] Step 2:

[1798] emotion recognition

[1799] The server receives the collected data and analyzes it using an emotion engine, analyzing the user's facial expressions from photos and the tone of voice from audio data to recognize emotions.

[1800] Input: Collected photos and audio data.

[1801] Data processing / calculation: The emotion engine uses image recognition and voice analysis algorithms to identify emotions.

[1802] Output: Recognized user emotion data.

[1803] Step 3:

[1804] Quiz Generation

[1805] The server uses an artificial intelligence module to generate quizzes based on the collected data and the user's perceived emotions, adjusting the difficulty of the quiz depending on the user's emotional state: increasing the difficulty if the user is having fun, and decreasing the difficulty if the user is feeling stressed.

[1806] Input: Collected data, recognized emotion data.

[1807] Data processing / calculation: An artificial intelligence module analyzes the data and determines the content and difficulty of the quiz.

[1808] Output: The generated quiz.

[1809] Step 4:

[1810] Quiz broadcast

[1811] The server sends the generated quiz to the user's device, which displays a quiz notification and transitions to the quiz screen.

[1812] Input: The generated quiz.

[1813] Output: The quiz is displayed on the user's device.

[1814] Step 5:

[1815] Answer acceptance

[1816] The user answers the presented quiz by selecting an option or entering text.

[1817] Input: The user's answer to the quiz.

[1818] Output: The device sends the user's answer to the server.

[1819] Step 6:

[1820] Answer assessment and evaluation

[1821] The server compares the received answer with a database of correct answers to determine whether it is correct or not, and simultaneously evaluates the user's memory risk in combination with the recognized emotion data.

[1822] Input: User's answer, correct answer database, recognized emotion data.

[1823] Data processing / calculation: A correct / incorrect judgment algorithm is used to calculate the correct answer rate and assess memory risk.

[1824] Output: Judgment result and memory risk score.

[1825] Step 7:

[1826] Providing Feedback

[1827] The server generates feedback based on the assessment results and emotional data, converts the assessment results into numerical values, and compares them with the user's past answer history. It also provides feedback recommending appropriate actions.

[1828] Input: Judgment results, emotion data, past answer history.

[1829] Data processing / computation: Feedback generation algorithms are used to quantify assessment results and recommend actions.

[1830] Output: Feedback is displayed on the user's device.

[1831] Through these steps, users can have fun while activating their memory and undergoing a cognitive risk assessment.

[1832] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1833] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1834] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1835] [Fourth embodiment]

[1836] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1837] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1838] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1839] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1840] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1841] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1842] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1843] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1844] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1845] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1846] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1847] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1848] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1849] Introduction

[1850] The present invention relates to a system that aims to activate a user's memory in a fun way and to detect and prevent dementia early. Specifically, the system generates quizzes based on data collected from the user's device and periodically evaluates the user's memory and cognitive status. The following describes in detail the embodiments of the present invention.

[1851] System configuration

[1852] This system mainly consists of the following components:

[1853] 1. User's Device

[1854] Devices that users use on a daily basis, such as smartphones and tablets.

[1855] The terminal performs the functions of collecting data, displaying quizzes, accepting answers, and providing feedback.

[1856] 2. Server

[1857] Analyzes data, generates quizzes, judges answers, and assesses memory and cognitive risk.

[1858] It is equipped with an artificial intelligence module that automates data analysis and quiz generation.

[1859] 3. Network

[1860] An internet connection that allows communication between your device and our server.

[1861] Program processing

[1862] Data collection

[1863] Server: Sends data collection requests to the user's device.

[1864] On your device: With your permission, we collect photos, calendar events, and diary data.

[1865] Terminal: Sends collected data to the server.

[1866] Examples:

[1867] Photos taken over the weekend, appointments on your calendar, and events recorded in your autobiography app.

[1868] Generate a quiz

[1869] Server: Analyzes the received data using an artificial intelligence module and generates a quiz to stimulate the user's memory.

[1870] Server: Sends the generated quiz to the user's device.

[1871] Examples:

[1872] Photo analysis results: "Where was this photo taken?"

[1873] Calendar event analysis results: "What events happened on this day?"

[1874] Quiz distribution and answers

[1875] Device: Shows the quiz notification to the user and opens the quiz screen.

[1876] User: Answers the quiz by selecting options or entering text in response to the questions presented.

[1877] Terminal: Sends the user's answer to the server.

[1878] Examples:

[1879] Choose a location from the options for the photo question.

[1880] Enter detailed text for calendar events.

[1881] Judging and evaluating answers

[1882] Server: Compares the received answer with the correct answer database and determines whether it is correct or not.

[1883] Server: Records the results of the judgment and calculates values ​​to assess the user's memory and cognitive risk.

[1884] Examples:

[1885] If the user answers "park" and the correct answer is "park," it is determined to be the correct answer.

[1886] Based on the memory risk assessment, it is compared with the user's past answer history.

[1887] Providing Feedback

[1888] Server: Generates feedback to the user based on the judgment results.

[1889] Server: Sends feedback to the user's device.

[1890] Terminal: Show a feedback screen to the user.

[1891] Examples:

[1892] If the correct answer rate is high, the feedback is "You have maintained a good memory."

[1893] If the perceived risk is high, you will be notified that "we recommend that you visit a medical institution."

[1894] Example

[1895] User A launches Recoco Memory for the first time and allows the app to access photos, calendar, and diary information. The server receives the initial data and begins analysis. Based on the analysis results, multiple quizzes are generated from User A's photos and calendar events from the past week. For example, the server generates a quiz such as "Where did you visit last weekend?" and sends it to the user's device.

[1896] When User A answers the quiz, the answer data is sent to the server for analysis and evaluation. The server determines whether User A's answer is correct and evaluates the memory status based on the result. The evaluation result is sent as feedback to User A's device, and User A receives a notification that their memory status is "good."

[1897] In this way, by using the system of the present invention, users can enjoy checking their memories and assessing cognitive risk in their daily lives.

[1898] The processing flow will be explained below.

[1899] Step 1:

[1900] Users: Download and install the app on their devices such as smartphones and tablets.

[1901] Step 2:

[1902] Device: When the app is launched for the first time, a user registration screen is displayed and the user is asked to enter personal information (such as name, email address, and password).

[1903] Step 3:

[1904] Server: Receives user registration information, creates a new user account in the database, and sends a success message to the terminal.

[1905] Step 4:

[1906] On device: Prompt the user for permission to access their photos, calendar, and diary data.

[1907] Step 5:

[1908] User: Select "Allow" in the permission dialog to grant the app data access.

[1909] Step 6:

[1910] Device: Collects data for which permission has been granted (photos, calendar, diary information) and sends it to the server.

[1911] Step 7:

[1912] Server: The AI ​​module analyzes the received data and generates quizzes based on the photos and calendar information.

[1913] For example, generate questions based on photo locations or calendar events.

[1914] Step 8:

[1915] Server: Sends the generated quiz to the user's device.

[1916] Step 9:

[1917] On the device: A quiz notification is displayed to the user. When the user taps the notification, the quiz screen is displayed.

[1918] Step 10:

[1919] User: Answers the quiz by selecting options or entering text in response to the questions presented.

[1920] Step 11:

[1921] Terminal: Sends the user's answer to the server.

[1922] Step 12:

[1923] Server: Compares the received answer with the correct answer database and determines whether it is correct or not.

[1924] Step 13:

[1925] Server: Records the results of the judgment and calculates values ​​to assess the user's memory and cognitive risk.

[1926] Example: Calculating accuracy rate and memory risk score.

[1927] Step 14:

[1928] Server: Generates feedback to provide to the user based on the judgment results.

[1929] Step 15:

[1930] Server: Sends feedback to the user's device.

[1931] Step 16:

[1932] Device: A feedback screen is displayed to the user, presenting the evaluation results and recommended actions (e.g., visiting a medical institution).

[1933] Step 17:

[1934] User: Review the feedback and take any recommended action (e.g., seek medical attention) if necessary.

[1935] This series of steps allows users to have fun while checking their memories and assessing cognitive risk.

[1936] Example 1

[1937] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1938] There is a lack of efficient systems to stimulate users' memories in an enjoyable way and to detect and prevent dementia early. In addition, there are insufficient means to accurately assess users' memory status and cognitive risk and provide feedback.

[1939] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1940] In this invention, the server includes means for collecting image data, schedule management data, and record data from the user's computing device, means for analyzing the collected data and using a machine learning algorithm to generate questions, and means for delivering the generated questions to the user's computing device, thereby making it possible to automatically generate questions to stimulate the user's memory, evaluate memory and cognitive risks based on the answers, and provide appropriate feedback.

[1941] "Computing device" refers to an information processing device that users use on a daily basis, and specifically includes smartphones, tablets, personal computers, etc.

[1942] "Image data" refers to photographs taken by users and image files acquired by users.

[1943] "Schedule management data" refers to calendar application data that records a user's schedule and appointments.

[1944] "Recorded data" refers to text data recorded by the user as a diary or memo.

[1945] "Machine learning algorithm" refers to an algorithm used to analyze collected data and generate questions to stimulate a user's memory, and may include an artificial intelligence module.

[1946] "Feedback" refers to information and advice provided to users based on the results of an assessment of their memory status and cognitive risk.

[1947] "Medical institution" refers to a medical facility such as a hospital or clinic that provides medical services.

[1948] MODE FOR CARRYING OUT THE INVENTION

[1949] Introduction

[1950] The present invention relates to a system for activating a user's memory in a fun way, with the aim of early detection and prevention of dementia. The system generates quizzes based on data collected from the user's computing device and periodically assesses the user's memory and cognitive status. Specific embodiments for implementing the invention are described in detail below.

[1951] System configuration

[1952] This system mainly consists of the following components:

[1953] 1. Your Computing Device

[1954] Information processing devices such as smartphones, tablets, and personal computers.

[1955] The computing device has the functions of collecting data, displaying quizzes, accepting answers, and providing feedback.

[1956] 2. Server

[1957] Analyzes data, generates quizzes, judges answers, and assesses memory and cognitive risk.

[1958] It is equipped with an artificial intelligence module that automates data analysis and quiz generation (e.g., GPT-3).

[1959] 3. Network

[1960] An internet connection that allows communication between the computing device and the server.

[1961] Data collection

[1962] The server sends a data collection request to the user's computing device. Based on the user's permission, the computing device collects image data, schedule management data, and record data. The computing device then sends the collected data to the server. Specific examples include photos taken by the user over the weekend, appointments listed on a calendar, and events recorded in a personal history app.

[1963] Generate a quiz

[1964] The server analyzes the received data using an artificial intelligence module and generates a quiz to stimulate the user's memory. The server then sends the generated quiz to the user's computing device. Specific examples include quizzes such as "Where was this photo taken?" based on the analysis of a photo, or "What events happened on this day?" based on the analysis of calendar events.

[1965] Quiz distribution and answers

[1966] The computing device displays a quiz notification to the user and opens the quiz screen. The user answers the quiz by selecting options or entering text in response to the presented questions. The computing device then sends the user's answers to the server. Specific examples of answers include selecting a location from options in response to a photo question or entering detailed actions in response to a calendar event.

[1967] Judging and evaluating answers

[1968] The server compares the received answer with a correct answer database to determine whether it is correct or incorrect. The server then records the result of the determination and calculates a value to evaluate the user's memory and cognitive risk. For example, if the user answers "park" and the correct answer is "park," the answer is determined to be correct. Furthermore, based on the memory risk assessment, the result is compared with the user's past answer history.

[1969] Providing Feedback

[1970] The server generates feedback for the user based on the assessment results. The server then sends the generated feedback to the user's computing device. The computing device displays a feedback screen to the user. For example, if the accuracy rate is high, feedback such as "Your memory is in good condition" is displayed, and if the cognitive risk is high, a notification such as "We recommend that you visit a medical institution" is displayed.

[1971] Example

[1972] User A launches the system for the first time and allows the app to access image data, schedule management data, and record data. The server receives the initial data and begins analysis. Based on the analysis results, multiple quizzes are generated from User A's photos and calendar events from the past week. For example, the server generates a quiz question such as, "Where did you visit last weekend?" and sends it to the user's computing device. When User A answers the quiz, the answer data is sent to the server for analysis and evaluation. The server determines whether User A's answer is correct and evaluates his or her memory status based on the result. The evaluation result is sent as feedback to User A's computing device, and User A is notified that his or her memory status is "good." In this way, using the system of the present invention, users can enjoy checking their memory and assessing cognitive risk in their daily lives.

[1973] Examples of prompt statements

[1974] "Generate quizzes from the user's photo data collected by the system, deliver the generated quizzes to the user's computing device, and provide feedback based on the results."

[1975] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1976] Step 1: Submit a data collection request

[1977] The server sends a data collection request to the user's computing device, the request including information directing the collection of image data, schedule management data, and record data.

[1978] Input: None

[1979] Output: Data collection request

[1980] Step 2: Data collection

[1981] The device collects image data, schedule management data, and record data from the user based on the data collection request received from the server. Specifically, it obtains photos from the camera roll of the smartphone or tablet, schedules from the calendar app, and text information from the diary app.

[1982] Input: Data Collection Request

[1983] Output: Collected data (photos, schedules, diaries)

[1984] Step 3: Sending data

[1985] The terminal transmits collected image data, schedule management data, and record data to the server. This data is transmitted based on the user's permission.

[1986] Input: Collected data

[1987] Output: Send data to the server

[1988] Step 4: Data analysis

[1989] The server analyzes the received data using a generative AI model (e.g., GPT-3). This analysis includes photo recognition, text analysis, and calendar event identification. Specifically, image recognition algorithms are used to analyze the content of photos, and natural language processing algorithms are used to analyze text data.

[1990] Input: Collected data

[1991] Output: Analysis results

[1992] Step 5: Quiz Generation

[1993] The server generates quizzes to stimulate the user's memory based on the analysis results. For example, it generates questions such as "Where was this photo taken?" based on the location information of the photo. It also generates questions such as "What events happened on this day?" based on calendar events.

[1994] Input: Analysis results, generated AI model

[1995] Output: The generated quiz

[1996] Step 6: Stream the quiz

[1997] The server then sends the generated quiz to the user's computing device, formatted in a way that is easy for the user to answer.

[1998] Input: Generated quiz

[1999] Output:Quiz delivery

[2000] Step 7: Quiz Notification and Display

[2001] The device notifies the user of the quiz received from the server and displays the quiz screen. Specifically, the device notifies the user of the existence of a new quiz using the notification center or in-app notifications, and displays the quiz screen.

[2002] Input:Quiz broadcast

[2003] Output: Quiz notification to user, quiz screen display

[2004] Step 8: Take the quiz

[2005] The user can select an answer from the options or enter a text answer for the quiz. Specifically, the user selects the correct answer from the options displayed or enters the answer using the keyboard.

[2006] Input: Quiz screen

[2007] Output: User's answer

[2008] Step 9: Submit your answers

[2009] The device sends the user's answers to the server, where they are accurately recorded and securely transmitted to the server.

[2010] Input: User's answer

[2011] Output: Send the answer to the server

[2012] Step 10: Determine the answer

[2013] The server compares the received answer with a database of correct answers to determine whether the answer is correct or not. Specifically, it matches the user's answer with the correct answers stored in the database.

[2014] Input: User's answer

[2015] Output: Judgment result

[2016] Step 11: Valuation Calculation

[2017] The server calculates a numerical value to evaluate the user's memory and cognitive risk based on the judgment result. For example, the server evaluates the user's memory ability and cognitive risk based on the accuracy rate and reaction time, and generates a score.

[2018] Input: Judgment result

[2019] Output: Evaluation score

[2020] Step 12: Generate feedback

[2021] The server generates feedback for the user based on the evaluation score, such as a message indicating good memory status or a message recommending that the user seek medical attention.

[2022] Input: Rating score

[2023] Output: Feedback message

[2024] Step 13: Submit your feedback

[2025] The server then transmits the generated feedback to the user's computing device, where the feedback is user-friendly and encourages action.

[2026] Input: Feedback message

[2027] Output: Send feedback to the user

[2028] Step 14: Viewing feedback

[2029] The device displays the feedback message received from the server to the user, specifically, by providing the feedback to the user using an in-app notification or a pop-up window.

[2030] Input: Send Feedback

[2031] Output: Show feedback

[2032] (Application example 1)

[2033] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2034] In an aging society, the prevention and early detection of dementia are important issues. However, existing methods lack systems that allow users to enjoyably assess memory in their daily lives and determine the risk of cognitive function without visiting a medical institution. Furthermore, even in physical stores, there are no systems that promote memory activation through the shopping experience, so there is a need to integrate dementia prevention and shopping.

[2035] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2036] In this invention, the server includes means for collecting photo, calendar, and diary data, as well as purchase and check-in histories from the user's device, an artificial intelligence module for analyzing the collected data and generating quizzes, and means for delivering and notifying the generated quizzes to the user's device. This enables the user's memory to be stimulated in an enjoyable way through everyday shopping experiences, thereby enabling the prevention and early detection of dementia.

[2037] "User device" refers to a device such as a smartphone, tablet, or smart glasses that a user uses on a daily basis to collect data, display quizzes, and accept answers.

[2038] "Data collection" is the process of collecting purchase history, check-in history, photos, calendar, and diary data from users' devices.

[2039] "Purchase history" refers to a record and detailed information about products purchased by a user in a physical store.

[2040] "Check-in history" is a record of a user's visits to specific locations or stores, and is data that includes location information.

[2041] The "artificial intelligence module" is a system consisting of programs and algorithms for analyzing collected data and generating quizzes to stimulate the user's memory.

[2042] "Quiz generation" is the process of automatically creating questions to assess a user's memory based on collected and analyzed data.

[2043] "Quiz delivery" is the process of sending the generated quiz to the user's device and notifying them.

[2044] "Answer determination" is the process of analyzing the answers given by the user to the quiz and comparing them with the correct answers to determine whether they are correct or incorrect.

[2045] "Memory status assessment" is the process of assessing a user's memory and cognitive function by comparing it with the user's past answer history.

[2046] "Feedback" is the process of informing the user of information about their memory and cognitive status and appropriate actions based on their answers and the assessment results.

[2047] "Recommendation to visit a medical institution" is the process of notifying the user to visit a medical institution based on the evaluation results.

[2048] "Quantification" is the process of converting evaluation results and answer history into quantitative data to make it easier to evaluate the user's memory state.

[2049] Configuring the various components

[2050] The present invention relates to a system for activating a user's memory and assessing their cognitive status. The system comprises the following components:

[2051] 1. User's Device

[2052] These are devices that users use on a daily basis (e.g., smartphones, tablets, smart glasses, etc.). These devices collect data such as users' purchase history, check-in history, photos, calendars, and diaries. They also perform functions such as displaying quizzes, accepting answers, and providing feedback.

[2053] 2. Server

[2054] The server analyzes data, generates quizzes, judges answers, and assesses memory and cognitive risk. It is equipped with an artificial intelligence module to automate these processes, and often uses cloud services such as Google Cloud and Amazon Web Services (AWS).

[2055] 3. Network

[2056] Refers to an internet connection that enables communication between a device and a server, and can use Wi-Fi, 4G, 5G, or other mobile networks.

[2057] Data collection

[2058] 1. Terminal

[2059] With user permission, data such as purchase history, check-in history, photos, calendar events, and diary entries are collected and transmitted securely to the server.

[2060] Generate a quiz

[2061] 2. Server

[2062] The received data is analyzed using an artificial intelligence module (e.g., TensorFlow, PyTorch) to generate a quiz to stimulate the user's memory. The quiz is automatically created and sent to the user's device.

[2063] Quiz distribution and answers

[2064] 3. Terminal

[2065] A quiz notification is displayed to the user, and the quiz screen opens. The user answers the quiz by selecting options or entering text in response to the questions presented.

[2066] 4. Users

[2067] The user answers the quiz appropriately and transmits the answers to the server via the terminal.

[2068] Judging and evaluating answers

[2069] 5. Server

[2070] The received answer is compared with a correct answer database to determine whether it is correct or incorrect. The result of the determination is then recorded and compared with past answer history to evaluate the memory status.

[2071] Providing Feedback

[2072] 6. Server

[2073] Based on the results of the assessment, feedback is generated for the user, including an assessment of memory status and recommended actions (e.g., a recommendation to see a doctor). The generated feedback is sent to the user's device.

[2074] 7. Terminal

[2075] A feedback screen is displayed to the user, allowing them to check their memory status and recommended actions.

[2076] Specific examples

[2077] When a user is shopping in a physical store, they may purchase a specific item at the register (a retro-designed drink that they have purchased in the past). The system collects this data and generates a quiz based on their purchase history, asking, "What flavor was the retro cola you purchased two weeks ago?" If the user answers the quiz correctly, the system provides feedback such as, "Your memory is good!"

[2078] Prompt Sentence Examples

[2079] "What flavor of retro cola did you buy two weeks ago?"

[2080] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2081] Step 1:

[2082] Data collection

[2083] Device: First, with the user's permission, data such as purchase history, check-in history, photos, calendar events, and diary entries are collected. This data is either collected automatically by the application or entered manually by the user. The collected data is converted into an appropriate data format, such as JSON, and sent to the server while protecting privacy.

[2084] Input: User purchase history, check-in history, photos, calendar events, diary data

[2085] Output: JSON data formatted to send to the server

[2086] Step 2:

[2087] Generate a quiz

[2088] Server: Analyzes the received data. Specifically, it extracts information for generating quizzes from the collected data through text analysis (natural language processing) and image analysis. It uses artificial intelligence modules (e.g., TensorFlow, PyTorch) to automatically generate quizzes that stimulate the user's memory.

[2089] Input: JSON data sent from the terminal

[2090] Output: Generated quiz questions and choices

[2091] Step 3:

[2092] Quiz broadcast

[2093] Server: The server delivers the generated quiz to the user's device and sends a notification. The quiz content is pushed to the device using the REST API.

[2094] On the device: The received notification is displayed to the user and the quiz screen is opened. When the user taps the notification, the specific quiz content is displayed.

[2095] Input: Generated quiz

[2096] Output: Quiz notification and quiz screen

[2097] Step 4:

[2098] Quiz Answers

[2099] User: Answers the quiz by selecting an answer from the options provided or by entering text.

[2100] Terminal: Receives the user's answer and sends it to the server. The data sent is in JSON format, including the answer.

[2101] Input: User's quiz answer

[2102] Output: Answer data to send to the server

[2103] Step 5:

[2104] Judging and evaluating answers

[2105] Server: Compares the received user answers with a database of correct answers to determine whether they are correct or not. It also compares the answers with the user's past answer history to assess memory status and cognitive risk. Statistical analysis and machine learning algorithms are used for this.

[2106] Input: User's answer data

[2107] Output: Correct / incorrect judgment and memory status evaluation results

[2108] Step 6:

[2109] Providing Feedback

[2110] Server: Generates feedback to the user based on the correctness of the answers and the evaluation results. The feedback includes an evaluation of memory status and recommended actions (e.g., recommendation to see a doctor).

[2111] Device: Receives the generated feedback and displays it to the user, allowing the user to check their own memory status and course of action.

[2112] Input: Correct / incorrect judgment and memory status evaluation results

[2113] Output: Feedback that is displayed on the user's device

[2114] Examples of prompt statements

[2115] "What flavor of retro cola did you buy two weeks ago?"

[2116] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2117] Introduction

[2118] The present invention relates to a system that aims to activate a user's memory in a fun way and to detect and prevent dementia early. In particular, the present invention provides a more personalized service by recognizing the user's emotions and adjusting the quiz generation and feedback content.

[2119] System configuration

[2120] The system consists of the following main components:

[2121] 1. User's Device

[2122] These are devices that users use on a daily basis, such as smartphones and tablets.

[2123] It performs functions such as data collection, emotion recognition, quiz display, answer acceptance, and feedback provision.

[2124] 2. Server

[2125] Analyzes data, generates quizzes, judges answers, and assesses memory and cognitive risk.

[2126] It is equipped with an artificial intelligence module and an emotion engine to automate data analysis and quiz generation.

[2127] 3. Network

[2128] It is an internet connection that allows communication between the device and the server.

[2129] Program processing

[2130] Data collection and emotion recognition

[2131] Server: Sends data collection requests to the user's device.

[2132] On the device: With the user's permission, it collects photos, calendar events, and diary data, and also uses the device's camera and microphone to capture the user's facial expressions and voice, which are then sent to the emotion engine.

[2133] Emotion engine: Analyzes collected facial and voice data to recognize the user's emotions.

[2134] Terminal: Sends collected data and sentiment analysis results to the server.

[2135] Examples:

[2136] When a user takes a photo or records an event in the calendar, the device sends this data to the server, and at the same time collects the user's facial expressions and voice and sends them to the emotion engine.

[2137] Quiz Generation

[2138] Server: Analyzes the received data using an artificial intelligence module and generates a quiz based on the user's memory and emotional state.

[2139] Server: Adjusts the content and difficulty of the quiz based on the emotions recognized by the emotion engine.

[2140] For example: if the user is having fun, generate a challenging quiz, if the user is stressed, generate an easy quiz.

[2141] Examples:

[2142] Photo analysis results: "Where was this taken?"

[2143] Calendar analysis results: "What events were there on this day?"

[2144] Quiz distribution and answers

[2145] Server: Sends the generated quiz to the user's device.

[2146] Device: Show the quiz notification to the user and transition to the quiz screen.

[2147] User: Answers the quiz by selecting options or entering text in response to the questions presented.

[2148] Terminal: Sends the user's answer to the server.

[2149] Examples:

[2150] A user receives a quiz notification and selects the correct answer to the question, "Where was this photo taken?"

[2151] Judging and evaluating answers

[2152] Server: Compares the received answer with the correct answer database and determines whether it is correct or not.

[2153] Server: Combines the judgment results with data from the emotion engine to calculate a numerical value for assessing the user's memory and cognitive risk.

[2154] Example: Calculating the accuracy rate and memory risk score.

[2155] Examples:

[2156] If the user answers "park" and it is the correct answer, it is determined to be the correct answer and compared with the past answer history to evaluate the memory risk.

[2157] Providing Feedback

[2158] Server: Generates personalized feedback based on the judgment results and emotion data.

[2159] Server: Sends feedback to the user's device.

[2160] Device: A feedback screen is displayed to the user, presenting the evaluation results and recommended actions (e.g., visiting a medical institution).

[2161] Examples:

[2162] If the correct answer rate is high, the system will provide feedback such as "Your memory is in good condition," and if the emotional data indicates that the user is under stress, it will also display advice such as "Relax and try the next quiz."

[2163] Example

[2164] User B launched Recoco Memory for the first time and allowed the app to access photos, calendar, and diary information. He also allowed the app to access the camera and microphone for emotion recognition. The server received the initial data and emotion data and began analysis. Based on the analysis results, multiple quizzes were generated from User B's photos and calendar events from the past week. For example, the server generated a quiz asking, "Where did you visit last weekend?" and selected a slightly more challenging quiz based on the user's emotional state (enjoyment).

[2165] When User B answers the quiz, the answer data and the latest emotional data are sent to the server for analysis and evaluation. The server determines whether User B's answer is correct and evaluates his / her memory and emotional state based on the result and the emotional data. As feedback, the device displays a notification saying "Good memory state" and "Enjoy the next quiz."

[2166] In this way, by using the system of the present invention, users can not only have fun while checking their memory and assessing cognitive risk, but also receive personalized services that take into account their emotional state.

[2167] The processing flow will be explained below.

[2168] Step 1:

[2169] User: Download and install Ricoco Memory on a device such as a smartphone or tablet.

[2170] Step 2:

[2171] Device: When the app is launched for the first time, a user registration screen is displayed and the user is asked to enter personal information (such as name, email address, and password).

[2172] Step 3:

[2173] Server: Receives user registration information, creates a new user account in the database, and sends a success message to the terminal.

[2174] Step 4:

[2175] Device: Prompts the user for permission to access photos, calendar, diary data, and the camera and microphone.

[2176] Step 5:

[2177] User: Select "Allow" in the permission dialog to grant the app data access.

[2178] Step 6:

[2179] Device: Collects data with permission (photos, calendar events, diary information) and sends it to the server. It also uses the camera and microphone to transmit the user's facial expressions and voice to the emotion engine.

[2180] Step 7:

[2181] Emotion engine: Analyzes collected facial and voice data to recognize the user's emotions.

[2182] Step 8:

[2183] Terminal: Sends collected data and sentiment analysis results to the server.

[2184] Step 9:

[2185] Server: Analyzes the received data using an AI module and generates a quiz based on the user's memory and emotional state. The emotion engine adjusts the content and difficulty of the quiz based on the emotions recognized by the server.

[2186] For example: if the user is having fun, generate a challenging quiz, if the user is stressed, generate an easy quiz.

[2187] Step 10:

[2188] Server: Sends the generated quiz to the user's device.

[2189] Step 11:

[2190] Device: Show the quiz notification to the user and transition to the quiz screen.

[2191] Step 12:

[2192] User: Answers the quiz by selecting options or entering text in response to the questions presented.

[2193] Step 13:

[2194] Terminal: Sends the user's answer to the server.

[2195] Step 14:

[2196] Server: Compares the received answer with the correct answer database and determines whether it is correct or not.

[2197] Step 15:

[2198] Server: Combines the judgment results with data from the emotion engine to calculate a numerical value for assessing the user's memory and cognitive risk.

[2199] Example: Calculating the accuracy rate and memory risk score.

[2200] Step 16:

[2201] Server: Generates personalized feedback based on the judgment results and emotion data.

[2202] Step 17:

[2203] Server: Sends feedback to the user's device.

[2204] Step 18:

[2205] Device: A feedback screen is displayed to the user, presenting the evaluation results and recommended actions (e.g., visiting a medical institution).

[2206] Step 19:

[2207] User: Review the feedback and take any recommended action (e.g., seek medical attention) if necessary.

[2208] Through this series of steps, users can not only enjoyably check their memories and assess their cognitive risks, but also receive personalized services that take into account their emotional state.

[2209] Example 2

[2210] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2211] In modern society, early detection and prevention of dementia are important issues, especially in the health management of the elderly. However, conventional memory assessment tools are monotonous, and users lack motivation to continue using them. Furthermore, there is a need for systems that do not take into account the user's emotional state and can be used in a fun and stress-free manner. Furthermore, there is a lack of personalized feedback and recommendations for appropriate actions tailored to individual needs. In response to these challenges, the present invention provides a system that allows users to activate their memory in an enjoyable way and provides personalized services based on their emotional state.

[2212] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2213] In this invention, the server includes means for collecting photo, schedule, and document data from the user's terminal, means for analyzing the collected data and the user's facial expressions and voice to recognize the emotional state, an artificial intelligence module for generating quizzes based on the analyzed data and the emotional state, means for adjusting the content and difficulty of the generated quizzes based on the emotional state, means for delivering the generated quizzes to the user's terminal, means for receiving the user's answers and determining whether the answers are correct, means for assessing the user's memory and cognitive risk based on the assessment results and the emotional state, and means for providing the user with feedback recommending the assessment results and appropriate actions. This allows the user to have fun while activating their memory and receiving personalized feedback and recommended actions.

[2214] "User's device" refers to a device that a user uses on a daily basis, such as a smartphone or tablet.

[2215] "Photo" refers to a still image taken by a User using a Device.

[2216] "Calendar" refers to the schedule information that a user records on a device.

[2217] "Document" refers to text data that a user creates or saves using a device.

[2218] "Means of collecting data" refers to the functionality for obtaining data such as photos, schedules, and documents from the user's device.

[2219] "Facial expressions and voice" refers to the user's facial expressions and spoken voice data.

[2220] "Means for recognizing emotional states" refers to a function that analyzes collected facial and voice data to identify the user's emotions (happiness, sadness, surprise, etc.).

[2221] "Artificial intelligence module means" refers to a processing system that uses artificial intelligence to analyze data and generate quizzes.

[2222] "Quiz" refers to questions or problems designed to assess a user's memory or cognitive abilities.

[2223] "Means for adjusting the content and difficulty of the quiz" refers to a function for appropriately adjusting the content and difficulty of the quiz questions based on the user's emotional state.

[2224] "Means for delivering quizzes" refers to the function for sending the generated quiz to the user's terminal.

[2225] "Means for receiving answers" refers to a function for transmitting the answers given by the user to the quiz from the user's terminal to the server and receiving them.

[2226] "Means for determining whether the answer is correct" refers to a function for comparing the received answer with pre-set correct answer data and determining whether the answer is correct or incorrect.

[2227] "Means for assessing memory and cognitive risk" refers to a function for assessing the risk of a user's memory and cognitive state based on the correctness or incorrectness of answers and emotional state.

[2228] "Means for providing feedback" refers to a function for generating feedback to recommend appropriate actions to the user based on the evaluation results and sending it to the user's device.

[2229] This invention relates to a system that aims to stimulate a user's memory in a fun way and to detect and prevent dementia early. In particular, it provides a more personalized service by recognizing the user's emotions and adjusting the quiz generation and feedback content.

[2230] System configuration

[2231] The system consists of the following main components:

[2232] 1. User's Device

[2233] These are devices that users use on a daily basis, such as smartphones and tablets. Specific hardware examples include Android and iOS-based devices.

[2234] Software: A dedicated application that collects data, recognizes emotions, displays quizzes, accepts answers, and provides feedback.

[2235] 2. Server

[2236] The server has the functionality to analyze data, generate quizzes, judge answers, and assess memory and cognitive risk.

[2237] Software: Artificial intelligence module and emotion engine, which automates data analysis and quiz generation.

[2238] 3. Network

[2239] An internet connection that allows communication between a device and a server, allowing data to be exchanged.

[2240] Data collection and emotion recognition

[2241] Server: Sends a data collection request to the user's device. The user's device receives this request and, after obtaining the user's permission, collects photo, calendar, and document data. It also uses the device's camera and microphone to collect the user's facial and voice data and sends it to the emotion engine. The emotion engine analyzes this data and recognizes the user's emotions.

[2242] Examples:

[2243] Users take everyday photos with their smartphone camera and register travel events in their calendar. The device sends this data to the server and simultaneously collects the user's facial expressions and voice. The emotion engine detects "enjoyment" from the user's smile, tone of voice, etc.

[2244] Generate a quiz

[2245] Server: Receives the results from the emotion engine and generates a quiz based on the user's memory and emotional state.

[2246] Artificial Intelligence Module: This determines the content and difficulty of the questions. Quizzes are created using a generative AI model, with example prompts such as:

[2247] Example prompt sentence:

[2248] "Generate the following quiz based on user A's photo data. User A's emotional state seems to be fun. The content of the quiz should be something that user A would enjoy. Specific example questions include, 'Where was this photo taken?' and 'What was this event?'"

[2249] The generated quizzes are adjusted in content and difficulty based on the emotional state.

[2250] Quiz distribution and answers

[2251] Server: Sends the generated quiz to the user's device.

[2252] Device: Show the quiz notification to the user and transition to the quiz screen.

[2253] Examples:

[2254] Users are given a quiz asking, "Where was this photo taken?" and are asked to select the appropriate location.

[2255] Judging and evaluating answers

[2256] Server: Receives the user's answer data, compares it with the correct answer database, and determines whether it is correct or incorrect. The result of the evaluation is combined with the emotion engine data to calculate a value that evaluates the user's memory and cognitive risk.

[2257] Examples:

[2258] If the user answers "park," if that is the correct answer, it is determined to be the correct answer and compared with past answer history to assess memory risk.

[2259] Providing Feedback

[2260] Server: Generates personalized feedback based on the judgment results and emotion data. The feedback is sent to the device and displayed on the feedback screen.

[2261] Examples:

[2262] If the correct answer rate is high, the system will provide feedback such as "Your memory is in good condition," and if it determines that the user is feeling stressed, it will advise them to "relax and try the next quiz."

[2263] This allows users to have fun while activating their memories and receiving personalized feedback.

[2264] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2265] Step 1:

[2266] Server: Sends data collection requests to the device.

[2267] Input: None

[2268] Output: Sending a data collection request

[2269] Specific operation: The server sends data collection instructions to the user's device via an HTTP request, which includes the specific data to collect (photos, schedules, documents) and how to collect it.

[2270] Step 2:

[2271] Device: Obtains user permission to collect data. If the user gives permission, the device will collect photos, calendar, and document data. It will also use the device's camera and microphone to collect facial and voice data.

[2272] Input: Data collection request from the server

[2273] Output: Collected data (photos, schedules, documents, facial expressions, voice)

[2274] How it works: The device running the app collects photos taken by the user, event information registered in the calendar, and document data, while also using the camera and microphone to collect real-time facial expressions and voice recordings.

[2275] Step 3:

[2276] Emotion engine: Analyzes collected facial and voice data to recognize the user's emotional state.

[2277] Input: Collected facial expression and voice data

[2278] Output: Emotion recognition result

[2279] Specific operation: The device sends the facial expression data captured by the device to an image recognition algorithm, and the recorded voice data to a voice analysis engine, which then recognizes the user's emotions (happiness, sadness, surprise, etc.).

[2280] Step 4:

[2281] Terminal: Sends collected data and sentiment analysis results to the server.

[2282] Input: Collected data and emotion recognition results

[2283] Output: Send data to the server

[2284] What happens: The device converts the data into the appropriate format and sends it to the server. The data is sent via an HTTP POST request, subject to user permission.

[2285] Step 5:

[2286] Server: Inputs the received data into an artificial intelligence module and generates a quiz based on the user's memory and emotional state.

[2287] Input: Collected data and emotion recognition results

[2288] Output: The generated quiz

[2289] How it works: The server inputs data into the AI ​​module and generates a quiz, such as "Based on your calendar events, what were the events on this day?"

[2290] Step 6:

[2291] Server: Adjusts the content and difficulty of the quiz based on the emotions recognized by the emotion engine.

[2292] Input: Generated quiz and emotion recognition results

[2293] Output: Adjusted quiz

[2294] What it does: If the user is having fun, choose a challenging quiz; if the user is stressed, choose an easy quiz.

[2295] Step 7:

[2296] Server: Sends the generated quiz to the user's device.

[2297] Input: Adjusted Quiz

[2298] Output: Quiz stream

[2299] Specific operation: The server sends the quiz in JSON format to the device, and the device receives it and displays a quiz notification.

[2300] Step 8:

[2301] Device: Show the quiz notification to the user and transition to the quiz screen.

[2302] Input: Quiz Stream

[2303] Output: Quiz screen display

[2304] What happens: When the user clicks on the notification, the application transitions to a quiz screen where the question is displayed.

[2305] Step 9:

[2306] User: Answers the quiz by selecting options or entering text in response to the questions presented.

[2307] Input: Question on the quiz screen

[2308] Output: User's answer

[2309] Specific behavior: The user selects an answer from the options and clicks the "Submit" button.

[2310] Step 10:

[2311] Terminal: Sends the user's answer to the server.

[2312] Input: User's answer

[2313] Output: Send the answer to the server

[2314] Specific operation: The device converts the answer data into an appropriate format and sends it to the server via an HTTP POST request.

[2315] Step 11:

[2316] Server: Compares the received answer data with the correct answer database to determine whether it is correct or not.

[2317] Input: User's answer

[2318] Output: Judgment result

[2319] Specific operation: The server compares the received answer with the correct answer data and determines whether it is correct or incorrect.

[2320] Step 12:

[2321] Server: Combines the judgment results and data from the emotion engine to assess the user's memory and cognitive risk.

[2322] Input: Judgment results and emotion recognition results

[2323] Output: Memory and cognitive risk assessment

[2324] Specific operation: The server calculates an evaluation score based on the correct answer rate, past answer history, and emotional data.

[2325] Step 13:

[2326] Server: Generates personalized feedback based on the judgment results and emotion data.

[2327] Input: Memory and Cognitive Risk Assessment

[2328] Output: Feedback message

[2329] Specific behavior: Generate a specific feedback message according to the user's state (e.g., "Your memory is in good condition"). Examples of prompt sentences include:

[2330] Example prompt sentence:

[2331] "Generate the following feedback based on User B's quiz answer data and emotional data. User B's accuracy rate is high and their emotional state is positive. The feedback should include praise for User B's memory and positive advice for the next quiz."

[2332] Step 14:

[2333] Server: Sends feedback to the user's device.

[2334] Input: Feedback message

[2335] Output: Feedback distribution

[2336] Specific operation: The server sends the generated feedback message to the terminal, and the terminal receives it and notifies the user.

[2337] Step 15:

[2338] On the device: A feedback screen is displayed to the user, showing the evaluation results and recommended actions.

[2339] Input: Feedback message

[2340] Output: Display feedback

[2341] Specific actions: The device displays a feedback message and provides the user with specific advice or recommended actions.

[2342] Through these steps, the system allows users to have fun while activating their memories and receive personalized feedback and recommendations for action.

[2343] (Application example 2)

[2344] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2345] To activate users' memories in a fun way and to detect and prevent dementia early, personalized services are required. However, existing technologies only provide uniform services without considering the user's emotions, making effective memory activation difficult. Furthermore, there are also few ways for users to check their knowledge and memories while having fun. Therefore, a system is needed that provides personalized quizzes and feedback based on the user's emotions, and that can activate memory and assess cognitive risk.

[2346] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2347] In this invention, the server includes: means for collecting photo, calendar, and diary data from the user's terminal; emotion engine means for analyzing the collected data and recognizing the user's emotions; artificial intelligence module means for generating quizzes based on the recognized emotions; means for delivering the generated quizzes to the user's terminal; means for receiving the user's answers and determining whether the answers are correct; means for assessing the user's memory and cognitive risk based on the determination results; means for providing the user with feedback on the evaluation results and recommending appropriate actions; and means for adjusting the difficulty of the quiz in accordance with the user's emotions based on the presentation of the quiz. This allows for the provision of personalized quizzes and feedback in accordance with the user's emotions, enabling memory activation and cognitive risk assessment in a fun environment.

[2348] "User's device" refers to a device that a user uses on a daily basis, including a smartphone or tablet.

[2349] "Photo" refers to image data taken by a user using a camera.

[2350] "Calendar" refers to a scheduling feature or application that allows users to manage their schedules and events.

[2351] A "diary" refers to text data that users use to record their daily events and thoughts.

[2352] "Data collection means" refers to a method or device that provides functionality for acquiring photo, calendar, and diary data from a user's device.

[2353] The "emotion engine" is a software module that analyzes collected data and recognizes the user's emotions.

[2354] The "quiz generation means" is a system for generating quizzes based on recognized emotions using artificial intelligence.

[2355] The "quiz distribution means" refers to a method or device for transmitting the generated quiz to the user's terminal and displaying the quiz.

[2356] The "answer determination means" is a system that receives the user's answer and determines whether it is correct or incorrect.

[2357] The "memory and cognitive risk assessment means" refers to a function or device for assessing the memory state and cognitive risk of a user based on the judgment results.

[2358] The "feedback providing means" is a function that provides the user with the evaluation results and feedback to recommend appropriate actions.

[2359] The "difficulty level adjustment means" is a system for adjusting the difficulty level of a quiz in accordance with the user's emotions based on the presentation of the quiz.

[2360] System Overview

[2361] The system of this invention aims to stimulate the user's memory in a fun way and to detect and prevent dementia early. The system also recognizes the user's emotions and adjusts the quiz generation and feedback content to provide personalized services.

[2362] Hardware and Software Configuration

[2363] Hardware

[2364] User devices: everyday devices such as smartphones and tablets.

[2365] Server: A server for analyzing data, generating quizzes, determining answers, and assessing memory and cognitive risk.

[2366] software

[2367] Data collection software: Collects photos, calendar, and diary data from users' devices.

[2368] Emotion engine: Software that analyzes collected data and recognizes the user's emotions.

[2369] Artificial Intelligence Module: Software for generating quizzes.

[2370] Feedback generation module: Software for generating feedback based on the answer results.

[2371] System Operation

[2372] 1. Data Collection

[2373] Your device will collect photos, calendar and diary data, with your permission.

[2374] 2. Emotion recognition

[2375] The collected data is sent to a server and analyzed by an emotion engine, which, for example, analyzes photos and audio data to recognize emotions from the user's facial expressions and voice.

[2376] 3. Quiz Generation

[2377] The AI ​​module generates quizzes based on the collected data and emotion recognition results, adjusting the difficulty of the quiz depending on the user's emotional state.

[2378] 4. Quiz distribution and answer processing

[2379] The generated quiz is delivered to the user's device, where the user answers the quiz and the device sends the answers to the server.

[2380] 5. Assessment and evaluation of answers

[2381] The server judges the answers and assesses the user's memory and cognitive risk based on the results and emotional data.

[2382] 6. Providing Feedback

[2383] The feedback generation module generates feedback based on the evaluation results and provides it to the user's terminal.

[2384] Specific examples

[2385] For example, here's a scenario where a user answers a quiz about recent events:

[2386] 1. Use photos taken by users at places they visited last weekend.

[2387] 2. The emotion engine analyzes the user's happy facial expression from the photo.

[2388] 3. The artificial intelligence module generates a quiz question: "Where did you visit last weekend?"

[2389] 4. The user answers the quiz and the answers are sent to the server.

[2390] 5. The server determines whether the answer is correct or incorrect and assesses the user's memory status and cognitive risk.

[2391] 6. The feedback generation module generates feedback such as "Good memory" and "Have fun trying the next quiz," and displays it on the user's device.

[2392] In this way, the system of the present invention provides personalized quizzes and feedback according to the user's emotions, enabling memory activation and cognitive risk assessment in a fun way.

[2393] Prompt Sentence Examples

[2394] "Analyze the photos taken by the user and their emotions at the time, and generate a related quiz. Provide appropriate feedback based on the user's answers to the quiz and their emotional data."

[2395] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2396] Step 1:

[2397] Data collection

[2398] The device collects the user's photos, calendar, and diary data with the user's permission.

[2399] Input: The user's device receives collected photos, calendar events, and diary entries.

[2400] Output: Collected data is sent to the server.

[2401] Step 2:

[2402] emotion recognition

[2403] The server receives the collected data and analyzes it using an emotion engine, analyzing the user's facial expressions from photos and the tone of voice from audio data to recognize emotions.

[2404] Input: Collected photos and audio data.

[2405] Data processing / calculation: The emotion engine uses image recognition and voice analysis algorithms to identify emotions.

[2406] Output: Recognized user emotion data.

[2407] Step 3:

[2408] Quiz Generation

[2409] The server uses an artificial intelligence module to generate quizzes based on the collected data and the user's perceived emotions, adjusting the difficulty of the quiz depending on the user's emotional state: increasing the difficulty if the user is having fun, and decreasing the difficulty if the user is feeling stressed.

[2410] Input: Collected data, recognized emotion data.

[2411] Data processing / calculation: An artificial intelligence module analyzes the data and determines the content and difficulty of the quiz.

[2412] Output: The generated quiz.

[2413] Step 4:

[2414] Quiz broadcast

[2415] The server sends the generated quiz to the user's device, which displays a quiz notification and transitions to the quiz screen.

[2416] Input: The generated quiz.

[2417] Output: The quiz is displayed on the user's device.

[2418] Step 5:

[2419] Answer acceptance

[2420] The user answers the presented quiz by selecting an option or entering text.

[2421] Input: The user's answer to the quiz.

[2422] Output: The device sends the user's answer to the server.

[2423] Step 6:

[2424] Answer assessment and evaluation

[2425] The server compares the received answer with a database of correct answers to determine whether it is correct or not, and simultaneously evaluates the user's memory risk in combination with the recognized emotion data.

[2426] Input: User's answer, correct answer database, recognized emotion data.

[2427] Data processing / calculation: A correct / incorrect judgment algorithm is used to calculate the correct answer rate and assess memory risk.

[2428] Output: Judgment result and memory risk score.

[2429] Step 7:

[2430] Providing Feedback

[2431] The server generates feedback based on the assessment results and emotional data, converts the assessment results into numerical values, and compares them with the user's past answer history. It also provides feedback recommending appropriate actions.

[2432] Input: Judgment results, emotion data, past answer history.

[2433] Data processing / computation: Feedback generation algorithms are used to quantify assessment results and recommend actions.

[2434] Output: Feedback is displayed on the user's device.

[2435] Through these steps, users can have fun while activating their memory and undergoing a cognitive risk assessment.

[2436] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2437] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2438] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2439] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2440] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2441] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2442] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2443] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2444] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2445] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2446] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2447] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2448] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2449] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2450] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2451] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2452] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2453] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2454] Furthermore, the hardware struc...

Claims

1. A means of collecting photo, calendar, and diary data from the user's device; an artificial intelligence module means for analyzing the collected data and generating quizzes; A means for delivering the generated quiz to the user's device; means for receiving the user's answer and determining whether the answer is correct; A means for assessing the memory and cognitive risk of the user based on the determination result; a means of providing users with feedback on the results of the assessment and recommendations for appropriate action; A system including:

2. The system according to claim 1 , further comprising a means for recommending to the user that he or she visit a medical institution based on the evaluation result.

3. 2. The system of claim 1, further comprising means for quantifying the evaluation result and comparing it with the user's past answer history.

Images