System

Abstract

A system is provided.SOLUTION: A system including a means for receiving behavior record data input by a user through a communication interface, a generation AI means for analyzing the behavior record data and classifying the behavior record data into appropriate categories based on an analysis result, a means for storing the data classified into the categories in a database, a means for evaluating a progress status of the user based on the stored data and feeding back an evaluation result, and a means for analyzing the stored data and providing an improvement plan to the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional behavioral record management systems have the problem that users must use multiple applications, which requires a great deal of effort to input data according to the format of each application. This makes it difficult for users to record data consistently, resulting in reduced efficiency in time management and goal achievement. Furthermore, progress feedback and suggestions for improvement are provided separately by each application, making it difficult for users to comprehensively understand their own behavior. The present invention solves these problems. [Means for solving the problem]

[0005] The present invention is a system including: means for receiving behavior record data entered by a user through a communication interface; generating AI means for analyzing the behavior record data and classifying the data into appropriate categories based on the analysis results; means for storing the categorized data in a database; means for evaluating the user's progress based on the stored data and providing feedback on the evaluation results; and means for analyzing the stored data and providing the user with improvement suggestions. According to the present invention, a user can easily record and manage their behavior through a single communication interface and receive support for comprehensive time management and goal achievement.

[0006] A "communication interface" is a means or protocol for a user to input behavior record data and transmit the data to a server.

[0007] "Behavioral record data" refers to information about activities such as studying and training that users record in chat format.

[0008] "Generative AI" is an artificial intelligence system that analyzes behavioral record data entered by the user and classifies it into appropriate categories.

[0009] A "category" refers to a group or classification item for classifying behavioral record data, and includes, for example, "study" and "training."

[0010] A "database" is a data management system for structuring and storing analyzed data.

[0011] "Progress" refers to the achievements and progress calculated based on the user's behavioral records.

[0012] "Feedback" is a means of providing information to the user to notify them of progress and evaluation results.

[0013] "Improvement proposals" refer to suggestions and advice for improving efficiency and results that are generated by analyzing the user's behavioral record data.

[0014] "Real-time communication protocol" refers to a communication technology for sending and receiving user behavior record data to a server in real time.

[0015] "Natural language processing technology" is a technology for analyzing text data entered by users and understanding and classifying its content. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] This invention is a system that allows users to input behavioral record data through a communication interface they use on a daily basis, and then analyzes, classifies, saves, provides feedback, and suggests improvements to that data. Below, we will explain in detail the program processing of this system in natural language, and provide examples.

[0038] Overall system overview

[0039] When a user enters behavioral record data using a communication interface such as a chat app, the device sends the data to a server. The server analyzes the received data using generative AI and classifies it into appropriate categories. The classified data is then stored in a database, and the user's progress is evaluated. Feedback is provided to the user along with the evaluation results, and improvement plans are also proposed based on past data.

[0040] Program processing overview

[0041] 1. Receiving user input

[0042] Users enter activity records such as study and training into a chat app.

[0043] For example, a user types "15 minutes of English vocabulary study" into the chat.

[0044] 2. Sending input data

[0045] The device sends the data entered from the chat app to the server.

[0046] This data also includes the user ID and timestamp.

[0047] 3. Data Analysis and Categorization

[0048] The server passes the received data to the generation AI for analysis.

[0049] The generative AI uses natural language processing technology to analyze the data and classify it into categories such as "study" and "training."

[0050] As a concrete example, "15 minutes of studying English vocabulary" is classified into the category "study" and is recognized as a time period of 15 minutes.

[0051] 4. Saving to the database

[0052] The server stores the analysis results in a database.

[0053] The information stored includes details such as user ID, category, activity, time, and timestamp.

[0054] 5. Progress Feedback

[0055] The server aggregates the user's past data and calculates their progress.

[0056] For example, if the total study time this week is 3 hours and 15 minutes, that information is notified to the user as feedback.

[0057] 6. Proposal for improvement

[0058] The server analyzes past data stored in the database to understand user behavior patterns.

[0059] Based on the results, effective improvement proposals are generated and provided to the user.

[0060] For example, advice like, "If you study for 30 minutes every day, you're two weeks away from achieving your goal."

[0061] Specific examples

[0062] 1. User Input

[0063] A user types "3 sets of 10 bench presses at the gym" into a chat app.

[0064] 2. Transmission and Analysis

[0065] The terminal sends user input to the server.

[0066] The server uses a generative AI to analyze the data of "Bench press at the gym, 10 repetitions, 3 sets" and classifies it into the category of "Training." The content is recognized as "Bench press," the number of repetitions is recognized as "10 repetitions," and the number of sets is recognized as "3 sets."

[0067] 3. Database storage

[0068] The server stores the analysis results in a database, for example, in the format {User ID: 12345, Category: Training, Content: Bench Press, Repetitions: 10, Sets: 3, Time: [Automatically estimated time], Timestamp: [Recorded time]}.

[0069] 4. Feedback

[0070] The server provides feedback to the user saying, "Today's training sessions have been recorded. The total number of sets is three."

[0071] 5. Proposal for improvement

[0072] The server suggests to the user an improvement suggestion such as, "To increase the number of bench presses, it would be effective to shorten the rest time a little."

[0073] In this way, users can efficiently record their actions and receive feedback on their progress and suggestions for improvement through a single communication interface.

[0074] The processing flow will be explained below.

[0075] Step 1: Receiving User Input

[0076] The user inputs behavioral record data through a communication interface (e.g., a chat app). For example, the user inputs "Study English vocabulary for 15 minutes" into the chat.

[0077] The terminal receives this input and prepares to transmit.

[0078] Step 2: Submitting input data

[0079] The terminal sends the received input data to the server, including the user ID, input text, and timestamp.

[0080] The protocol used is HTTP POST request and real-time communication protocol.

[0081] Step 3: Receiving the data

[0082] The server receives the input data, converts it into a format suitable for analysis, and passes it on to the next processing step.

[0083] Step 4: Analysis by generative AI

[0084] The server passes the received data to the generation AI for analysis.

[0085] Generative AI uses natural language processing technology to analyze input text. For example, if the input data is "Study English vocabulary for 15 minutes," it will classify it into the category of "study" and recognize the time as 15 minutes.

[0086] Step 5: Receiving categorization results

[0087] The server receives the analysis results (category classification results) from the generation AI.

[0088] The analysis results are received as structured data, such as category (e.g., studying), activity (e.g., studying English vocabulary), and time (e.g., 15 minutes).

[0089] Step 6: Saving to the Database

[0090] The server stores the analysis results in a database, including the user ID, category, activity, time, and timestamp.

[0091] Check whether saving was successful and perform error handling.

[0092] Step 7: Assess your progress

[0093] The server evaluates the user's progress based on the data stored in the database, aggregating past data and calculating the duration and number of activities within a specific period.

[0094] For example, it generates an evaluation result such as "Your total study time this week is 3 hours and 15 minutes."

[0095] Step 8: Generate feedback

[0096] The server generates a feedback message based on the evaluation results.

[0097] Prepare the generated feedback message for sending to the user.

[0098] Step 9: Submit your feedback

[0099] The server sends a feedback message to the device, such as a notification that includes a message like "Your total study time this week is 3 hours and 15 minutes."

[0100] The device receives this feedback message and displays it to the user, either as a push notification or a chat message.

[0101] Step 10: Propose improvements

[0102] The server analyzes past data stored in the database to understand user behavior patterns and trends.

[0103] Based on the results, it generates effective improvement suggestions for the user, such as "If you study for 30 minutes every day, you'll be two weeks away from achieving your goal."

[0104] Step 11: Submit your improvement proposal

[0105] The server sends the generated improvement proposal to the terminal, and sends a message containing useful advice to the user.

[0106] The device receives the improvement suggestion message and displays it to the user. Notification methods include push notifications and chat messages.

[0107] In this way, the behavioral record data entered by the user is managed and supported in an integrated manner, going through the steps of analysis, classification, storage, feedback, and improvement suggestions.

[0108] Example 1

[0109] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0110] Conventional behavioral recording systems make it difficult for users to efficiently record their daily activities and appropriately evaluate their progress. Furthermore, they do not provide sufficient feedback or suggestions for improvement, making it difficult to promote behavioral improvement. This makes it difficult for users to understand their own behavioral patterns and take effective measures to improve.

[0111] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0112] In this invention, the server includes means for receiving behavior record data entered by a user through a communication interface, artificial intelligence (AI) means for analyzing the behavior record data and classifying it into appropriate categories based on the analysis results, means for saving the categorized data in a database, means for evaluating the user's progress based on the saved data and providing feedback on the evaluation results, means for analyzing the saved data and providing the user with improvement suggestions, means for assigning a user ID and a timestamp to the behavior record data, means for generating prompt sentences for analysis by the AI, and means for aggregating past data and evaluating behavior records over a certain period of time. This allows users to efficiently record their own behavior, making it easier to understand their progress, and also enabling them to receive specific improvement suggestions.

[0113] A "communications interface" is the means by which a user inputs data and interacts with a system.

[0114] "Behavior record data" is data that includes specific information about the user's behavior, such as studying or training.

[0115] A "generative AI means" is a means that uses generative artificial intelligence technology to analyze input data and classify it into appropriate categories.

[0116] A "database" is a system for systematically storing analyzed data and enabling quick retrieval of necessary information.

[0117] The "means for evaluating progress" is a means for measuring and evaluating the progress of a user's activities based on the data stored in the database.

[0118] The "feedback means" is a means for notifying the user of the progress evaluation results and providing useful information regarding the user's actions.

[0119] The "means for providing improvement proposals" is a means for analyzing past data stored in a database and generating and providing effective improvement proposals to users.

[0120] A "user ID" is an identifier that uniquely identifies each user.

[0121] The "timestamp" indicates the date and time when the behavior record data was input.

[0122] A "prompt sentence" is an instruction sentence used by the generative artificial intelligence when analyzing data.

[0123] The "means for aggregating past data" is a means for generating statistical information based on behavioral records within a certain period of time and understanding the activity patterns of users.

[0124] This invention is a system in which a user inputs behavioral record data through a communication interface, and the data is analyzed using a generating AI means, and then classified, saved, evaluated, and provided feedback. The specific implementation method of this system is described below.

[0125] Overall overview

[0126] The user uses a device such as a smartphone or PC to input behavioral record data through a communication interface such as a chat app. The device receives the user's input data, assigns a user ID and timestamp to it, and sends it to a server. The server analyzes the received data using generative AI and classifies it into appropriate categories. The server then stores the classified data in a database and evaluates the user's progress. The evaluation results are fed back to the user, and further improvement suggestions are provided based on the data.

[0127] Hardware and software used

[0128] Hardware:

[0129] Devices such as smartphones and computers.

[0130] Server equipment that operates the server.

[0131] software:

[0132] Communication interfaces such as chat apps.

[0133] A database system (e.g., MySQL, PostgreSQL) for storing and managing data.

[0134] APIs and libraries for using generative AI models (e.g., GPT-4).

[0135] Details of data processing and calculation

[0136] 1. Receiving user input:

[0137] A user enters an action record such as "Study English vocabulary for 15 minutes" into a chat app.

[0138] 2. Send input data:

[0139] The terminal sends the input data to the server along with the user ID and a timestamp.

[0140] Sending format example:

[0141] {

[0142] User ID: 12345,

[0143] Record: "15 minutes of English vocabulary study",

[0144] Timestamp: "2023-10-10 10:00:00"

[0145] }

[0146] 3. Data Analysis and Categorization:

[0147] The server passes the received data to the generation AI, which generates a prompt for analysis.

[0148] An example of a generated prompt is: "The user entered 'Study English vocabulary for 15 minutes.' Please analyze this data to extract categories and detailed information."

[0149] The generating AI categorizes it as "study" and recognizes the time as "15 minutes."

[0150] 4. Save to database:

[0151] The server stores the analysis results in a database.

[0152] Examples of data that may be saved:

[0153] {

[0154] User ID: 12345,

[0155] Category: "Study",

[0156] Activity: "Study English vocabulary",

[0157] Time: "15 minutes",

[0158] Timestamp: "2023-10-10 10:00:00"

[0159] }

[0160] 5. Progress feedback:

[0161] The server aggregates data from the database over a period of time and evaluates the user's progress.

[0162] Example: If the total study time this week is 3 hours and 15 minutes, send the user a feedback message saying "Your total study time this week is 3 hours and 15 minutes."

[0163] 6. Providing suggestions for improvement:

[0164] The server sends a prompt to the generation AI, which generates improvement suggestions based on past data.

[0165] An example of a generated improvement suggestion: "If you study for 30 minutes every day, you're two weeks away from achieving your goal."

[0166] Specific examples

[0167] If a user types "Bench press at the gym, 10 reps, 3 sets" into a chat app, the device receives this, assigns a user ID and timestamp, and sends it to the server. The server then uses the generated AI to analyze the data, generating a prompt that reads, "The user entered 'Bench press at the gym, 10 reps, 3 sets'. Please analyze this data and extract the category and detailed information." The generated AI classifies this as a "Training" category, and recognizes that the content is "Bench press," the number of repetitions is "10," and the number of sets is "3 sets." These analysis results are stored in a database and later used to provide progress feedback and improvement suggestions.

[0168] In this way, the system allows users to efficiently log their actions and receive feedback on their progress and suggestions for improvement.

[0169] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0170] Step 1:

[0171] Receiving User Input

[0172] The user inputs behavior record data into the chat application.

[0173] Example: Enter "Study English vocabulary for 15 minutes."

[0174] The terminal receives user input in real time.

[0175] The terminal assigns a user ID and a timestamp to the received data.

[0176] Input: User behavior record data (e.g., "Study English vocabulary for 15 minutes").

[0177] Output: Data containing user ID, behavior record data, and timestamp (e.g., {User ID: 12345, Record: "Study English vocabulary for 15 minutes", Timestamp: "2023-10-10 10:00:00"}).

[0178] Step 2:

[0179] Sending input data

[0180] The terminal transmits the received behavior record data to the server.

[0181] The data is sent as an HTTP POST request.

[0182] Data to be sent: Data including user ID, behavioral record data, and timestamp.

[0183] Input: Data including user ID, behavior record data, and timestamp.

[0184] Output: Data sent to the server (e.g., {User ID: 12345, Record: "Study English vocabulary for 15 minutes", Timestamp: "2023-10-10 10:00:00"}).

[0185] Step 3:

[0186] Data analysis and categorization

[0187] The server takes the received data and generates a prompt for analysis.

[0188] The server sends a prompt to the generative AI model.

[0189] A generative AI model analyzes the data and classifies it into appropriate categories.

[0190] Example prompt: "The user entered 'Study English vocabulary for 15 minutes.' Please parse this data to extract categories and detailed information."

[0191] Input: Data including user ID, behavior record data, and timestamp.

[0192] Output: Analysis results (category: "Study", activity: "Study English vocabulary", time: "15 minutes").

[0193] Step 4:

[0194] Saving to a database

[0195] The server obtains the analysis results and stores them in a database.

[0196] Save format: {User ID: 12345, Category: "Study", Activity: "Study English vocabulary", Time: "15 minutes", Timestamp: "2023-10-10 10:00:00"}

[0197] Input: Analysis results (category: "Study", activity: "Study English vocabulary", time: "15 minutes").

[0198] Output: Data stored in the database.

[0199] Step 5:

[0200] Progress feedback

[0201] The server compiles user behavior records for a specified period from the database.

[0202] Example: Calculate the total time spent studying this week and arrive at "3 hours and 15 minutes."

[0203] The server generates progress feedback based on the aggregated results and sends it to the user via a chat app.

[0204] Example feedback: "Your total study time this week is 3 hours and 15 minutes."

[0205] Input: Data stored in a database.

[0206] Output: Progress feedback sent to the user.

[0207] Step 6:

[0208] Providing improvement suggestions

[0209] Enter a prompt statement that the server will use to pass past data stored in the database to the generative AI model.

[0210] The server generates a prompt sentence and sends it to the generation AI, which then generates an improvement proposal.

[0211] Example of an improvement idea: "If I study 30 minutes each day, I'll be two weeks away from achieving my goal."

[0212] The server sends the generated improvement proposal to the user.

[0213] Input: Data stored in the database, prompts for analysis.

[0214] Output: The improvement suggestions sent to the user.

[0215] (Application example 1)

[0216] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0217] Improving driving skills and maintaining safe driving are becoming increasingly important with the spread of autonomous vehicles. However, conventional driver training and feedback systems lack real-time capabilities and individual optimization, making it difficult to quickly respond to the individual needs of each driver. Furthermore, there is a lack of a system that allows drivers to easily and intuitively input their driving records and receive instant feedback. Given this background, there is a need for a system that provides effective feedback and improvement suggestions to improve driving skills.

[0218] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0219] In this invention, the server includes: means for receiving behavior record data entered by the user through a communication interface; a generating AI means for analyzing the behavior record data and classifying it into appropriate categories based on the analysis results; means for storing the categorized data in a database; means for evaluating the user's progress based on the stored data and providing feedback on the evaluation results; means for analyzing the stored data and providing the user with improvement suggestions; means for inputting the user's behavior record data through a head-mounted display; and means for providing feedback and improvement suggestions for improving driving skills. This allows drivers to easily enter their own driving records in real time and receive immediate feedback and specific improvement suggestions based on the input. This makes it possible to maintain safe driving and improve driving skills simultaneously.

[0220] A "communication interface" is an interactive device for a user to input behavior record data, and is a device having means for transmitting and receiving data.

[0221] "Behavior record data" is information in which a user records daily behavior, exercise, work content, and the like.

[0222] "Generative AI" is a technology that uses artificial intelligence to analyze incoming data and classify it into appropriate categories.

[0223] A "database" is an information collection system for efficiently storing and managing analyzed and classified data.

[0224] "User progress" is information indicating the degree of progress or achievement of a specific action or task performed by a user.

[0225] "Feedback" refers to information and advice provided to the User based on the analyzed and evaluated progress.

[0226] "Improvement Suggestions" are suggestions and advice to help users achieve better results.

[0227] A "head-mounted display" is a display device worn by a user on the head, and is a device for providing visual information.

[0228] "Feedback for improving driving skills" refers to evaluations and advice provided to users regarding specific operations and actions they perform while driving.

[0229] "Improvement suggestions for improving driving skills" are practical suggestions and advice provided to users to improve their driving skills.

[0230] This invention is a system that allows users to improve their driving skills in autonomous vehicles using a head-mounted display (HMD). The system inputs recorded user behavior data, analyzes and classifies it, and provides the driver with immediate feedback and suggestions for improvement. This section describes the detailed configuration and operation of the system.

[0231] Hardware and software used

[0232] Hardware:

[0233] Head-mounted display (HMD): Using Oculus Quest 2 as an example.

[0234] software:

[0235] Development environment: Unity3D

[0236] Communication API: WebSocket or HTTP REST API

[0237] Generation AI: OpenAI GPT-4

[0238] Database: Firebase Firestore

[0239] System configuration

[0240] The system consists of the following main components:

[0241] Receiving user input:

[0242] The user can input behavioral data through the HMD. For example, they can input "Today's driving was smooth."

[0243] Sending input data:

[0244] The HMD sends the input data, including the user ID, timestamp, and location information, to the server in real time.

[0245] Data analysis and classification:

[0246] The server passes the input data to the generative AI (GPT-4), which analyzes it using natural language processing technology and classifies it into an appropriate category. For example, "Lane change was delayed" is classified as "driving skill improvement area."

[0247] Save to database:

[0248] The server stores the parsed and categorized data in Firebase Firestore, where detailed information (user ID, category, content, timestamp, etc.) is stored.

[0249] Progress assessment and feedback:

[0250] The server aggregates past data and evaluates the user's progress, and the evaluation results are communicated to the user via the HMD as feedback.

[0251] Suggested improvements:

[0252] The server analyzes the stored data and uses AI to provide users with specific suggestions for improving their driving skills, such as "Keep a safe distance to reduce sudden braking."

[0253] Specific examples

[0254] 1. User Input:

[0255] The user inputs their observations while driving into the HMD's chat interface.

[0256] For example, enter "There were a lot of sudden braking today."

[0257] 2. Transmission and Analysis:

[0258] The HMD sends the input data to the server.

[0259] The server uses a generative AI (GPT-4) to analyze the data that shows "frequent sudden braking" and classifies it as "an area for improvement in driving technique."

[0260] 3. Database storage:

[0261] The server saves the analysis results in a database (Firebase Firestore).

[0262] 4. Feedback:

[0263] The server provides feedback such as, "You braked suddenly five times today. Next time, try to maintain a safe distance while driving."

[0264] 5. Suggestions for improvement:

[0265] An improvement suggestion was displayed on the HMD: "To reduce sudden braking, it is safer to try to maintain a constant distance between vehicles."

[0266] Example prompts for generative AI models

[0267] Below are some example prompts that can be used to properly analyze and categorize user input data and provide specific feedback and suggestions for improvement:

[0268] Analyze the user's driving record data and classify it into appropriate categories. Then, provide advice to improve the user's driving skills based on that data. For example, if the input is "I braked a lot today," classify it as "Something to improve in driving skills" and output the advice "Keep a safe distance."

[0269] These prompts allow generative AI models (such as GPT-4) to efficiently analyze user input data and provide appropriate feedback and suggestions for improvement.

[0270] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0271] Step 1:

[0272] The user wears the HMD and inputs behavioral record data through the chat interface. For example, the user inputs "I was late changing lanes." This input data includes the user ID and timestamp.

[0273] Input: Activity record data, user ID, timestamp

[0274] Output: Formatted behavioral record data

[0275] Step 2:

[0276] The device (HMD) sends the behavioral record data entered by the user to the server in real time using WebSocket and HTTP REST API.

[0277] Input: Formatted behavioral record data

[0278] Output: Activity record data sent to the server

[0279] Step 3:

[0280] The server passes the received behavioral record data to a generative AI model (GPT-4) for analysis. A prompt sentence is used during this process. The generative AI model uses natural language processing technology to analyze the input data and classify it into appropriate categories. For example, "Lane change was delayed" is classified as "Areas for improvement in driving technique."

[0281] Input: Activity record data sent to the server

[0282] Output: Classified data (with categories)

[0283] Step 4:

[0284] The server stores the analyzed data in a database (Firebase Firestore), including information such as user ID, category, content, and timestamp.

[0285] Input: Classified data (with categories)

[0286] Output: Data stored in the database

[0287] Step 5:

[0288] The server evaluates the user's progress based on the data stored in the database, and then provides the evaluation results to the user as real-time feedback. For example, the HMD displays, "You braked suddenly five times today."

[0289] Input: Data stored in the database

[0290] Output: Feedback information

[0291] Step 6:

[0292] The server analyzes past data stored in the database and generates and provides improvement suggestions to the user using a generative AI model. For example, an improvement suggestion such as "Maintain a safe distance to reduce sudden braking" is generated. This improvement suggestion is also displayed on the HMD.

[0293] Input: Data stored in database, historical data

[0294] Output: Improvement plan

[0295] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0296] This invention is a system that analyzes and classifies behavioral record data entered by the user through a communication interface, and also recognizes the user's emotions and provides feedback and suggestions for improvement. Below, we will explain in detail the program processing of this system in natural language, and provide examples.

[0297] Overall system overview

[0298] When a user enters behavioral record data using a communication interface such as a chat app, the device sends the data to a server. The server analyzes the received data using a generative AI and emotion engine, classifying it into appropriate categories and determining the user's emotions. The classified data and emotion information are then stored in a database, and the user's progress is evaluated. Along with the evaluation results, the user is provided with feedback based on their emotions, and improvement suggestions are also proposed based on past data.

[0299] Program processing overview

[0300] 1. Receiving user input

[0301] Users enter activity records such as study and training into a chat app.

[0302] For example, a user types "15 minutes of English vocabulary study" into the chat.

[0303] The terminal receives this input and prepares to transmit.

[0304] 2. Sending input data

[0305] The terminal sends the received input data to the server, including the user ID, input text, and timestamp.

[0306] 3. Data Receipt and Initial Analysis

[0307] The server receives the input data, converts it into a format for analysis, and passes it on to the next processing step.

[0308] 4. Analysis by generative AI and emotion engine

[0309] The server passes the received data to the generation AI and emotion engine for analysis.

[0310] Generative AI uses natural language processing technology to analyze input text and classify it into categories such as "study," "training," etc. For example, input data such as "15 minutes of English vocabulary study" will be classified as "study" and the time will be recognized as 15 minutes.

[0311] The emotion engine recognizes emotions from the user's input text, such as happiness, sadness, excitement, and stress.

[0312] 5. Receiving categorization and emotion determination results

[0313] The server receives the analysis results from the generation AI and emotion engine. The analysis results are received as structured data, including category (e.g., studying), activity (e.g., studying English vocabulary), time (e.g., 15 minutes), and emotion (e.g., joy).

[0314] 6. Saving to the database

[0315] The server stores the analysis and emotion determination results in a database, including the user ID, category, activity, time, emotion, and timestamp.

[0316] Check whether saving was successful and perform error handling.

[0317] 7. Evaluating progress and generating emotional feedback

[0318] The server evaluates the user's progress based on the data stored in the database, aggregating past data and calculating the duration and number of activities within a specific period.

[0319] A feedback message is generated according to the user's emotion. For example, if the user is judged to be "sad," encouraging feedback is generated.

[0320] 8. Submitting Feedback

[0321] The server sends a feedback message to the device, such as a notification that includes "Your total study time this week is 3 hours and 15 minutes. Great!"

[0322] The device receives this feedback message and displays it to the user, either as a push notification or a chat message.

[0323] 9. Proposal for improvement

[0324] The server analyzes past data stored in the database to understand the user's behavioral patterns and emotional trends.

[0325] Based on the results, the system generates effective improvement suggestions for the user. For example, it creates advice such as, "If you study for 30 minutes every day, you will have two weeks to reach your goal. It would also be effective to take breaks to relax."

[0326] 10. Submitting Improvement Suggestions

[0327] The server sends the generated improvement proposal to the terminal, and sends a message containing useful advice to the user.

[0328] The device receives the improvement suggestion message and displays it to the user. Notification methods include push notifications and chat messages.

[0329] In this way, the behavioral record data and emotional information entered by the user are managed and supported in an integrated manner through the steps of analysis, classification, storage, feedback, and improvement suggestions.

[0330] The processing flow will be explained below.

[0331] Step 1: Receiving User Input

[0332] The user inputs behavioral record data through a communication interface (e.g., a chat app). For example, the user inputs "Study English vocabulary for 15 minutes" into the chat.

[0333] The terminal receives this input and prepares to transmit.

[0334] Step 2: Submitting input data

[0335] The terminal sends the received input data to the server, including the user ID, input text, and timestamp.

[0336] The protocol used is HTTP POST request and real-time communication protocol.

[0337] Step 3: Receiving the data

[0338] The server receives the input data, converts it into a format suitable for analysis, and passes it on to the next processing step.

[0339] Step 4: Analysis by generative AI

[0340] The server passes the received data to the generation AI for analysis.

[0341] Generative AI uses natural language processing technology to analyze input text. For example, if the input data is "Study English vocabulary for 15 minutes," it will classify it into the category of "study" and recognize the time as 15 minutes.

[0342] Step 5: Emotion determination by the emotion engine

[0343] The server passes the received data to the emotion engine for analysis.

[0344] The emotion engine analyzes the input text and determines the user's emotions. For example, if the input data is "15 minutes of English vocabulary study," it will determine emotions such as "motivation" or "joy."

[0345] Step 6: Receiving categorization and emotion determination results

[0346] The server receives the analysis results from the generation AI and emotion engine. The analysis results are received as structured data, including category (e.g., studying), activity (e.g., studying English vocabulary), time (e.g., 15 minutes), and emotion (e.g., joy).

[0347] Step 7: Saving to the Database

[0348] The server stores the analysis and emotion determination results in a database, including the user ID, category, activity, time, emotion, and timestamp.

[0349] Check whether saving was successful and perform error handling.

[0350] Step 8: Evaluate your progress

[0351] The server evaluates the user's progress based on the data stored in the database, aggregating past data and calculating the duration and number of activities within a specific period.

[0352] For example, it generates an evaluation result such as "Your total study time this week is 3 hours and 15 minutes."

[0353] Step 9: Generate emotional feedback

[0354] The server generates feedback messages based on the evaluation results, and adjusts the content according to the user's emotions.

[0355] For example, if the user is judged to be "happy," feedback such as "Great! Let's keep up the good work to achieve our goal" is generated.

[0356] Step 10: Submit your feedback

[0357] The server sends a feedback message to the device, such as a notification that includes "Your total study time this week is 3 hours and 15 minutes. Great!"

[0358] The device receives this feedback message and displays it to the user, either as a push notification or a chat message.

[0359] Step 11: Propose improvements

[0360] The server analyzes past data stored in the database to understand the user's behavioral patterns and emotional trends.

[0361] Based on the results, the system generates effective improvement suggestions for the user. For example, it creates advice such as, "If you study for 30 minutes every day, you will have two weeks to reach your goal. It would also be effective to take breaks to relax."

[0362] Step 12: Submit your improvement proposal

[0363] The server sends the generated improvement proposal to the terminal, and sends a message containing useful advice to the user.

[0364] The device receives the improvement suggestion message and displays it to the user. Notification methods include push notifications and chat messages.

[0365] In this way, the behavioral record data and emotional information entered by the user are managed and supported in an integrated manner through the steps of analysis, classification, storage, feedback, and improvement suggestions.

[0366] Example 2

[0367] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0368] Conventional systems do not adequately analyze the behavioral record data entered by users through a communication interface or provide feedback on that data. Furthermore, they are also inadequate in providing appropriate feedback and improvement suggestions based on the user's emotions, making it difficult to maintain the user's motivation or provide appropriate improvement measures. To solve these issues, it is necessary to provide a system that comprehensively analyzes and manages the user's behavioral record data and emotions, and provides appropriate feedback and improvement suggestions.

[0369] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for receiving behavior record data input by a user through a communication interface; generative AI model means for analyzing the behavior record data and classifying the data into appropriate categories based on the analysis results; emotion engine means for determining the user's emotions from the data analyzed by the generative AI model means; means for storing the categorized data and the emotion determination results in a database; means for evaluating the user's progress based on the stored data and generating feedback according to the evaluation results and the emotions; and means for analyzing the stored data and providing the user with improvement suggestions. This makes it possible to analyze and manage the user's behavior record data and emotion information in an integrated manner and provide appropriate feedback and improvement suggestions.

[0370] A "user" is an individual or entity that inputs behavioral data through a communications interface.

[0371] A "communication interface" is software or an application that allows a user to input behavior record data and communicate with a terminal.

[0372] "Behavior record data" is data relating to activities and behaviors that a user inputs through a communication interface, and includes, for example, information relating to study and training.

[0373] A "terminal" is an electronic device used to receive user input and send it to a server, and includes smartphones, tablets, personal computers, etc.

[0374] A "server" is a central processing unit that receives data sent from a terminal, analyzes and stores it, and generates feedback.

[0375] A "generative AI model" is an artificial intelligence method for analyzing behavioral record data and classifying it into appropriate categories.

[0376] An "emotion engine" is a technology that determines a user's emotions from data analyzed by a generative AI model.

[0377] A "category" is the type of behavioral record data classified by the generative AI model, such as "study" or "training."

[0378] A "database" is a recording medium for storing data handled by the system, such as analysis results and emotion determination results.

[0379] "Progress" refers to the progress of an activity evaluated based on the user's behavior record data.

[0380] "Feedback" refers to response messages and advice provided based on the user's progress and emotions.

[0381] "Improvement proposals" are suggestions for improving user behavior that are generated by analyzing past data stored in the database.

[0382] This invention is a system that analyzes and classifies behavioral record data entered by a user through a communication interface, recognizes the user's emotions, and provides feedback and suggestions for improvement. A specific embodiment of this system is described below.

[0383] Hardware and software used

[0384] Hardware

[0385] Devices: smartphones, tablets, computers, etc.

[0386] Server: Central processing unit (cloud server, on-premise server)

[0387] software

[0388] Communication Interface: Chat App, Web Application

[0389] Generative AI models: AI models that use natural language processing techniques (e.g., GPT series)

[0390] Sentiment engine: Software that performs sentiment analysis (e.g., sentiment analysis library)

[0391] Database: Data storage and management system (e.g., MySQL, PostgreSQL)

[0392] Explanation of program processing

[0393] 1. Receiving user input

[0394] The user enters the activity record of study, training, etc. into the chat app. For example, the user enters "Studying English vocabulary for 15 minutes."

[0395] The terminal receives this input and prepares to send it. The received data includes the user ID, the input text, and a timestamp.

[0396] 2. Sending input data

[0397] The device sends the received data to the server, where it is transferred over the network using an API.

[0398] 3. Data Receipt and Initial Analysis

[0399] The server receives the input data and converts it into a format for parsing. The data is parsed in JSON format.

[0400] 4. Analysis by generative AI and emotion engine

[0401] The server passes the received data to the generative AI model and emotion engine for analysis. The generative AI model analyzes the input text and classifies it into categories such as "study" and "training."

[0402] For example, data entered as "15 minutes of studying English vocabulary" will be classified into the category "study" and the time will be recognized as 15 minutes.

[0403] The emotion engine recognizes emotions from the input text and determines the user's emotions as "joy," "sadness," "excitement," "stress," etc.

[0404] 5. Receiving analysis results

[0405] The server receives the analysis results from the generative AI model and the emotion engine, which include structured data such as category, activity, time, and emotion.

[0406] 6. Saving to the database

[0407] The server stores the analysis and emotion determination results in a database, including the user ID, category, activity, time, emotion, and timestamp.

[0408] 7. Evaluating progress and generating feedback

[0409] The server evaluates the user's progress based on the data in the database, for example, by aggregating past data and calculating the duration and number of activities within a specific period.

[0410] Generate feedback messages based on the user's emotions, such as "You've studied a total of 3 hours and 15 minutes this week. Great!"

[0411] 8. Submitting Feedback

[0412] The server generates a feedback message and sends it to the device, which receives it and displays it to the user, either as a push notification or a chat message.

[0413] 9. Proposal for improvement

[0414] The server analyzes past data stored in the database to understand the user's behavioral patterns and emotional trends.

[0415] Based on the results, the system generates effective improvement suggestions for the user. For example, it creates advice such as, "If you study for 30 minutes every day, you will be two weeks away from achieving your goal. It would also be effective to take breaks to relax."

[0416] 10. Submitting Improvement Suggestions

[0417] The server sends the generated improvement proposal to the device. The device receives the improvement proposal message and displays it to the user. Notification methods include push notifications and chat messages.

[0418] Examples and prompts

[0419] Specific examples

[0420] User: "Running for 30 minutes"

[0421] Terminal: Receives this input and sends it to the server.

[0422] Server: Analyzes the received data, classifies it into the "Training" category, and determines the emotion as "Excitement."

[0423] Server: Stores the data in a database, evaluates progress, and sends feedback to the user, such as "You've trained a total of 2 hours this week. Great!"

[0424] Server: Send the following improvement suggestion: "Your health will improve even more if you increase your daily training by 15 minutes."

[0425] Prompt Sentence Examples

[0426] "The user entered 'Study English vocabulary for 20 minutes.' Please analyze this data."

[0427] "The user's emotion was determined to be sadness. Please generate appropriate feedback."

[0428] "Analyze user behavior patterns based on data from the past three months and propose improvements."

[0429] The above is a specific example of how to implement the invention. By using this system, it is possible to manage a user's behavioral record data and emotional information in an integrated manner, and provide appropriate feedback and suggestions for improvement.

[0430] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0431] Program processing steps

[0432] Step 1: Receiving User Input

[0433] A user enters behavioral data into a chat app. For example, the user enters "Study English vocabulary for 15 minutes."

[0434] The terminal receives this input and generates data containing the user ID, the input text, and a timestamp.

[0435] The input is "Study English vocabulary for 15 minutes" and the output is the user ID, the input text, and a timestamp.

[0436] Step 2: Submitting input data

[0437] The device sends the received data to the server, and then sends the data via the network using an API.

[0438] The inputs are the user ID, the input text, and a timestamp, and the output is the data sent to the server.

[0439] Step 3: Data reception and initial analysis

[0440] The server receives the input data and converts it into a format for analysis.

[0441] The input is a user ID, input text, and a timestamp, and the output is JSON format data. As a concrete example, the received data is mapped to key-value pairs.

[0442] Step 4: Analysis by generative AI and emotion engine

[0443] The server passes the data to the generative AI model and emotion engine to begin analysis.

[0444] The generative AI model analyzes the input text and classifies it into categories such as "study," "training," etc. For example, the input data "15 minutes of English vocabulary study" will be classified as "study" and the time will be recognized as 15 minutes.

[0445] The emotion engine identifies emotions from the input text, determining, for example, "joy."

[0446] The input is JSON formatted data and the output is data on category, activity, time, and emotion.

[0447] Step 5: Receive the analysis results

[0448] The server receives the analysis results from the generative AI model and the emotion engine.

[0449] The input is the analysis results of the generative AI model and emotion engine, and the output is structured data on category (e.g., studying), activity (e.g., studying English vocabulary), time (e.g., 15 minutes), and emotion (e.g., joy).

[0450] Step 6: Saving to the Database

[0451] The server stores the analysis and emotion determination results in a database.

[0452] The inputs are category, activity, time, emotion, user ID, and timestamp, and the output is a record stored in a database.

[0453] Step 7: Assess progress and generate feedback

[0454] The server evaluates the user's progress based on the stored data, for example by aggregating past data and calculating the duration and number of activities within a specific period.

[0455] Generate feedback messages based on the user's emotions, such as "Your total study time this week is 3 hours and 15 minutes. Great!"

[0456] The input is historical data retrieved from the database, and the output is progress assessments and feedback messages.

[0457] Step 8: Submit your feedback

[0458] The server sends the generated feedback message to the terminal, which receives the feedback message and displays it to the user.

[0459] The input is the feedback message and the output is the feedback notification displayed on the terminal.

[0460] Step 9: Propose improvements

[0461] The server analyzes past data stored in the database to understand the user's behavioral patterns and emotional trends.

[0462] Based on the results, the system generates effective improvement suggestions for the user. For example, it creates advice such as, "If you study for 30 minutes every day, you will be two weeks away from achieving your goal. It would also be effective to take breaks to relax."

[0463] The input is past data and the output is a message of proposed improvements.

[0464] Step 10: Submit your improvement proposal

[0465] The server sends the generated improvement proposal to the terminal, which receives the improvement proposal message and displays it to the user.

[0466] The input is the improvement proposal message, and the output is the improvement proposal notification displayed on the terminal.

[0467] (Application example 2)

[0468] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0469] Conventional advertising display systems typically provide uniform advertisements based on a user's behavioral history, but this often results in advertisements that ignore the user's emotional state, which can reduce user engagement. Furthermore, because advertisements are not optimized in response to emotional fluctuations, the user experience is not consistently high quality. The present invention aims to solve these problems by proposing a system that provides appropriate advertisements based on a user's behavioral history and emotional state.

[0470] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving behavior record data input by a user through a communication interface, generation AI means for analyzing the behavior record data and classifying the data into appropriate categories based on the analysis results, means for saving the data classified into the categories in a database, means for evaluating the user's progress based on the saved data and feeding back the evaluation results, means for analyzing the saved data and providing the user with improvement suggestions, emotion analysis means for analyzing the user's emotions, and means for suggesting advertisements based on the emotion analysis results. This makes it possible to display personalized advertisements according to the user's emotional state.

[0471] "User" means any individual or organization that uses this system.

[0472] A "communication interface" is a communication means such as a chat application or web form that allows a user to input behavior record data.

[0473] "Behavior record data" refers to data entered in text format by a user about their daily activities and work.

[0474] "Generative AI means" is an artificial intelligence technology that analyzes input behavioral record data and classifies its contents into appropriate categories.

[0475] "Category" refers to the classification of activity record data by type, such as "study," "training," and "relaxation."

[0476] The "database" is a data storage system for storing analyzed behavioral record data and emotional information.

[0477] "Means for assessing progress" refers to means for assessing a user's efforts and achievements based on stored data.

[0478] "Feedback" refers to messages and notifications that communicate progress assessment results to the user.

[0479] The "means for providing improvement suggestions" is a means for analyzing stored data and making suggestions to improve user behavior.

[0480] "Emotion analysis means" is a technology for identifying and analyzing emotions from user behavior record data.

[0481] The "means for suggesting advertisements" refers to a means for selecting and presenting the most suitable advertisement to the user based on emotion analysis and behavioral record data.

[0482] The system according to the present invention analyzes the behavioral record data entered by the user and personalizes advertisements based on the analysis results. The program processing of this system will be described in detail below.

[0483] First, a user uses a communication interface such as a chat application to input information about their daily activities and tasks. For example, they input text such as, "I had a great lunch with my friends today." This input data is received by the user's device and sent to the server.

[0484] The server analyzes the received data using a generation AI and a sentiment analysis engine. The generation AI uses natural language processing technology to analyze the input text and classify it into an appropriate category. For example, the activity "lunch" is classified as "relaxation." The sentiment analysis engine determines the emotion from the user's input text. For example, the emotion "fun" is determined.

[0485] The analysis results are passed to the server as structured data, including categories, activity details, time, and emotions. This data is stored in a database. The stored data is used to evaluate the user's progress. Based on the evaluation results, feedback is generated according to the user's emotions. For example, feedback such as "You had a great day today!" is sent.

[0486] Furthermore, the server will suggest advertisements based on the results of the sentiment analysis. For example, if the user is judged to be "happy," advertisements for relaxation-related products and services will be suggested. The advertisements will be sent to the user as push notifications or chat messages.

[0487] Hardware and software used:

[0488] User device: smartphone, tablet, etc.

[0489] Communication interfaces: chat applications, web forms

[0490] Server: Cloud server, on-premise server, etc.

[0491] Generative AI: AI models using natural language processing techniques (e.g., TextBlob)

[0492] Sentiment analysis engine: An engine that uses emotion recognition technology

[0493] Database: Data storage system (e.g. RDBMS, NoSQL)

[0494] Notification System: Push Notification Library

[0495] Examples and prompts:

[0496] If a user types, "I had a fun lunch with a friend today," the system analyzes this data and determines the category "relaxed" and the emotion "fun," then sends appropriate feedback and advertisements to the user.

[0497] Example prompt sentence:

[0498] "I had a great lunch with a friend today. I want to determine the sentiment and suggest an appropriate ad."

[0499] In this way, users receive feedback and advertising based on their behavior and emotions, resulting in a more consistent and high-quality experience.

[0500] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0501] Step 1:

[0502] A user inputs behavior record data using a communication interface. For example, the user inputs "I had a great lunch with my friends today" into a chat application. This input data is received by the user's terminal.

[0503] Step 2:

[0504] The terminal sends the received input data, including the user ID, text content, and timestamp, to the server. The server receives this data and passes it on to the next processing step.

[0505] Step 3:

[0506] The server converts the incoming data into a format for analysis and passes it to the generative AI and sentiment analysis engine, for example by tokenizing the input text and preprocessing it for contextual analysis.

[0507] Step 4:

[0508] The generative AI uses natural language processing techniques to analyze the input text and classify it into a category. For example, the activity "lunch" is classified as "relaxation." The sentiment analysis engine determines the emotion from the input text. For example, the emotion "fun" is determined.

[0509] Step 5:

[0510] The server receives the analysis results and formats them as structured data, including category, activity, emotion, and time. For example, the data could be that the category is "relaxing," the emotion is "fun," the activity is "having lunch with a friend," and the time is "1 hour."

[0511] Step 6:

[0512] The server saves the formatted data to the database, including the user ID, category, activity, emotion, time, and timestamp. It checks whether the save was successful and handles any errors.

[0513] Step 7:

[0514] The server evaluates the user's progress based on the stored data. For example, it may compile the time spent on relaxation activities over the past week and store this data as progress data. This data is used to evaluate the user's progress.

[0515] Step 8:

[0516] Generates feedback messages based on the results of sentiment analysis. For example, if the user is judged to be "happy," the system generates feedback such as "You had a great day today!"

[0517] Step 9:

[0518] The server sends a feedback message to the user's device, which receives the message and displays it to the user as a push notification or chat message.

[0519] Step 10:

[0520] The server proposes advertisements based on the results of the emotion analysis. For example, if the emotion is determined to be "fun," the server selects advertisements for products related to relaxation and proposes them to the user.

[0521] Step 11:

[0522] The server sends the selected advertisement to the user's device, which receives it and displays it to the user. The advertisement is displayed as a push notification or chat message.

[0523] Through the above process, personalized advertisements are provided based on the user's behavioral record data and emotional state.

[0524] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0525] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0526] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0527] [Second embodiment]

[0528] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0529] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0530] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0531] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0532] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0533] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0534] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0535] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0536] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0537] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0538] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0539] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0540] This invention is a system that allows users to input behavioral record data through a communication interface they use on a daily basis, and then analyzes, classifies, saves, provides feedback, and suggests improvements to that data. Below, we will explain in detail the program processing of this system in natural language, and provide examples.

[0541] Overall system overview

[0542] When a user enters behavioral record data using a communication interface such as a chat app, the device sends the data to a server. The server analyzes the received data using generative AI and classifies it into appropriate categories. The classified data is then stored in a database, and the user's progress is evaluated. Feedback is provided to the user along with the evaluation results, and improvement plans are also proposed based on past data.

[0543] Program processing overview

[0544] 1. Receiving user input

[0545] Users enter activity records such as study and training into a chat app.

[0546] For example, a user types "15 minutes of English vocabulary study" into the chat.

[0547] 2. Sending input data

[0548] The device sends the data entered from the chat app to the server.

[0549] This data also includes the user ID and timestamp.

[0550] 3. Data Analysis and Categorization

[0551] The server passes the received data to the generation AI for analysis.

[0552] The generative AI uses natural language processing technology to analyze the data and classify it into categories such as "study" and "training."

[0553] As a concrete example, "15 minutes of studying English vocabulary" is classified into the category "study" and is recognized as a time period of 15 minutes.

[0554] 4. Saving to the database

[0555] The server stores the analysis results in a database.

[0556] The information stored includes details such as user ID, category, activity, time, and timestamp.

[0557] 5. Progress Feedback

[0558] The server aggregates the user's past data and calculates their progress.

[0559] For example, if the total study time this week is 3 hours and 15 minutes, that information is notified to the user as feedback.

[0560] 6. Proposal for improvement

[0561] The server analyzes past data stored in the database to understand user behavior patterns.

[0562] Based on the results, effective improvement proposals are generated and provided to the user.

[0563] For example, advice like, "If you study for 30 minutes every day, you're two weeks away from achieving your goal."

[0564] Specific examples

[0565] 1. User Input

[0566] A user types "3 sets of 10 bench presses at the gym" into a chat app.

[0567] 2. Transmission and Analysis

[0568] The terminal sends user input to the server.

[0569] The server uses a generative AI to analyze the data of "Bench press at the gym, 10 repetitions, 3 sets" and classifies it into the category of "Training." The content is recognized as "Bench press," the number of repetitions is recognized as "10 repetitions," and the number of sets is recognized as "3 sets."

[0570] 3. Database storage

[0571] The server stores the analysis results in a database, for example, in the format {User ID: 12345, Category: Training, Content: Bench Press, Repetitions: 10, Sets: 3, Time: [Automatically estimated time], Timestamp: [Recorded time]}.

[0572] 4. Feedback

[0573] The server provides feedback to the user saying, "Today's training sessions have been recorded. The total number of sets is three."

[0574] 5. Proposal for improvement

[0575] The server suggests to the user an improvement suggestion such as, "To increase the number of bench presses, it would be effective to shorten the rest time a little."

[0576] In this way, users can efficiently record their actions and receive feedback on their progress and suggestions for improvement through a single communication interface.

[0577] The processing flow will be explained below.

[0578] Step 1: Receiving User Input

[0579] The user inputs behavioral record data through a communication interface (e.g., a chat app). For example, the user inputs "Study English vocabulary for 15 minutes" into the chat.

[0580] The terminal receives this input and prepares to transmit.

[0581] Step 2: Submitting input data

[0582] The terminal sends the received input data to the server, including the user ID, input text, and timestamp.

[0583] The protocol used is HTTP POST request and real-time communication protocol.

[0584] Step 3: Receiving the data

[0585] The server receives the input data, converts it into a format suitable for analysis, and passes it on to the next processing step.

[0586] Step 4: Analysis by generative AI

[0587] The server passes the received data to the generation AI for analysis.

[0588] Generative AI uses natural language processing technology to analyze input text. For example, if the input data is "Study English vocabulary for 15 minutes," it will classify it into the category of "study" and recognize the time as 15 minutes.

[0589] Step 5: Receiving categorization results

[0590] The server receives the analysis results (category classification results) from the generation AI.

[0591] The analysis results are received as structured data, such as category (e.g., studying), activity (e.g., studying English vocabulary), and time (e.g., 15 minutes).

[0592] Step 6: Saving to the Database

[0593] The server stores the analysis results in a database, including the user ID, category, activity, time, and timestamp.

[0594] Check whether saving was successful and perform error handling.

[0595] Step 7: Assess your progress

[0596] The server evaluates the user's progress based on the data stored in the database, aggregating past data and calculating the duration and number of activities within a specific period.

[0597] For example, it generates an evaluation result such as "Your total study time this week is 3 hours and 15 minutes."

[0598] Step 8: Generate feedback

[0599] The server generates a feedback message based on the evaluation results.

[0600] Prepare the generated feedback message for sending to the user.

[0601] Step 9: Submit your feedback

[0602] The server sends a feedback message to the device, such as a notification that includes a message like "Your total study time this week is 3 hours and 15 minutes."

[0603] The device receives this feedback message and displays it to the user, either as a push notification or a chat message.

[0604] Step 10: Propose improvements

[0605] The server analyzes past data stored in the database to understand user behavior patterns and trends.

[0606] Based on the results, it generates effective improvement suggestions for the user, such as "If you study for 30 minutes every day, you'll be two weeks away from achieving your goal."

[0607] Step 11: Submit your improvement proposal

[0608] The server sends the generated improvement proposal to the terminal, and sends a message containing useful advice to the user.

[0609] The device receives the improvement suggestion message and displays it to the user. Notification methods include push notifications and chat messages.

[0610] In this way, the behavioral record data entered by the user is managed and supported in an integrated manner, going through the steps of analysis, classification, storage, feedback, and improvement suggestions.

[0611] Example 1

[0612] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0613] Conventional behavioral recording systems make it difficult for users to efficiently record their daily activities and appropriately evaluate their progress. Furthermore, they do not provide sufficient feedback or suggestions for improvement, making it difficult to promote behavioral improvement. This makes it difficult for users to understand their own behavioral patterns and take effective measures to improve.

[0614] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0615] In this invention, the server includes means for receiving behavior record data entered by a user through a communication interface, artificial intelligence (AI) means for analyzing the behavior record data and classifying it into appropriate categories based on the analysis results, means for saving the categorized data in a database, means for evaluating the user's progress based on the saved data and providing feedback on the evaluation results, means for analyzing the saved data and providing the user with improvement suggestions, means for assigning a user ID and a timestamp to the behavior record data, means for generating prompt sentences for analysis by the AI, and means for aggregating past data and evaluating behavior records over a certain period of time. This allows users to efficiently record their own behavior, making it easier to understand their progress, and also enabling them to receive specific improvement suggestions.

[0616] A "communications interface" is the means by which a user inputs data and interacts with a system.

[0617] "Behavior record data" is data that includes specific information about the user's behavior, such as studying or training.

[0618] A "generative AI means" is a means that uses generative artificial intelligence technology to analyze input data and classify it into appropriate categories.

[0619] A "database" is a system for systematically storing analyzed data and enabling quick retrieval of necessary information.

[0620] The "means for evaluating progress" is a means for measuring and evaluating the progress of a user's activities based on the data stored in the database.

[0621] The "feedback means" is a means for notifying the user of the progress evaluation results and providing useful information regarding the user's actions.

[0622] The "means for providing improvement proposals" is a means for analyzing past data stored in a database and generating and providing effective improvement proposals to users.

[0623] A "user ID" is an identifier that uniquely identifies each user.

[0624] The "timestamp" indicates the date and time when the behavior record data was input.

[0625] A "prompt sentence" is an instruction sentence used by the generative artificial intelligence when analyzing data.

[0626] The "means for aggregating past data" is a means for generating statistical information based on behavioral records within a certain period of time and understanding the activity patterns of users.

[0627] This invention is a system in which a user inputs behavioral record data through a communication interface, and the data is analyzed using a generating AI means, and then classified, saved, evaluated, and provided feedback. The specific implementation method of this system is described below.

[0628] Overall overview

[0629] The user uses a device such as a smartphone or PC to input behavioral record data through a communication interface such as a chat app. The device receives the user's input data, assigns a user ID and timestamp to it, and sends it to a server. The server analyzes the received data using generative AI and classifies it into appropriate categories. The server then stores the classified data in a database and evaluates the user's progress. The evaluation results are fed back to the user, and further improvement suggestions are provided based on the data.

[0630] Hardware and software used

[0631] Hardware:

[0632] Devices such as smartphones and computers.

[0633] Server equipment that operates the server.

[0634] software:

[0635] Communication interfaces such as chat apps.

[0636] A database system (e.g., MySQL, PostgreSQL) for storing and managing data.

[0637] APIs and libraries for using generative AI models (e.g., GPT-4).

[0638] Details of data processing and calculation

[0639] 1. Receiving user input:

[0640] A user enters an action record such as "Study English vocabulary for 15 minutes" into a chat app.

[0641] 2. Send input data:

[0642] The terminal sends the input data to the server along with the user ID and a timestamp.

[0643] Sending format example:

[0644] {

[0645] User ID: 12345,

[0646] Record: "15 minutes of English vocabulary study",

[0647] Timestamp: "2023-10-10 10:00:00"

[0648] }

[0649] 3. Data Analysis and Categorization:

[0650] The server passes the received data to the generation AI, which generates a prompt for analysis.

[0651] An example of a generated prompt is: "The user entered 'Study English vocabulary for 15 minutes.' Please analyze this data to extract categories and detailed information."

[0652] The generating AI categorizes it as "study" and recognizes the time as "15 minutes."

[0653] 4. Save to database:

[0654] The server stores the analysis results in a database.

[0655] Examples of data that may be saved:

[0656] {

[0657] User ID: 12345,

[0658] Category: "Study",

[0659] Activity: "Study English vocabulary",

[0660] Time: "15 minutes",

[0661] Timestamp: "2023-10-10 10:00:00"

[0662] }

[0663] 5. Progress feedback:

[0664] The server aggregates data from the database over a period of time and evaluates the user's progress.

[0665] Example: If the total study time this week is 3 hours and 15 minutes, send the user a feedback message saying "Your total study time this week is 3 hours and 15 minutes."

[0666] 6. Providing suggestions for improvement:

[0667] The server sends a prompt to the generation AI, which generates improvement suggestions based on past data.

[0668] An example of a generated improvement suggestion: "If you study for 30 minutes every day, you're two weeks away from achieving your goal."

[0669] Specific examples

[0670] If a user types "Bench press at the gym, 10 reps, 3 sets" into a chat app, the device receives this, assigns a user ID and timestamp, and sends it to the server. The server then uses the generated AI to analyze the data, generating a prompt that reads, "The user entered 'Bench press at the gym, 10 reps, 3 sets'. Please analyze this data and extract the category and detailed information." The generated AI classifies this as a "Training" category, and recognizes that the content is "Bench press," the number of repetitions is "10," and the number of sets is "3 sets." These analysis results are stored in a database and later used to provide progress feedback and improvement suggestions.

[0671] In this way, the system allows users to efficiently log their actions and receive feedback on their progress and suggestions for improvement.

[0672] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0673] Step 1:

[0674] Receiving User Input

[0675] The user inputs behavior record data into the chat application.

[0676] Example: Enter "Study English vocabulary for 15 minutes."

[0677] The terminal receives user input in real time.

[0678] The terminal assigns a user ID and a timestamp to the received data.

[0679] Input: User behavior record data (e.g., "Study English vocabulary for 15 minutes").

[0680] Output: Data containing user ID, behavior record data, and timestamp (e.g., {User ID: 12345, Record: "Study English vocabulary for 15 minutes", Timestamp: "2023-10-10 10:00:00"}).

[0681] Step 2:

[0682] Sending input data

[0683] The terminal transmits the received behavior record data to the server.

[0684] The data is sent as an HTTP POST request.

[0685] Data to be sent: Data including user ID, behavioral record data, and timestamp.

[0686] Input: Data including user ID, behavior record data, and timestamp.

[0687] Output: Data sent to the server (e.g., {User ID: 12345, Record: "Study English vocabulary for 15 minutes", Timestamp: "2023-10-10 10:00:00"}).

[0688] Step 3:

[0689] Data analysis and categorization

[0690] The server takes the received data and generates a prompt for analysis.

[0691] The server sends a prompt to the generative AI model.

[0692] A generative AI model analyzes the data and classifies it into appropriate categories.

[0693] Example prompt: "The user entered 'Study English vocabulary for 15 minutes.' Please parse this data to extract categories and detailed information."

[0694] Input: Data including user ID, behavior record data, and timestamp.

[0695] Output: Analysis results (category: "Study", activity: "Study English vocabulary", time: "15 minutes").

[0696] Step 4:

[0697] Saving to a database

[0698] The server obtains the analysis results and stores them in a database.

[0699] Save format: {User ID: 12345, Category: "Study", Activity: "Study English vocabulary", Time: "15 minutes", Timestamp: "2023-10-10 10:00:00"}

[0700] Input: Analysis results (category: "Study", activity: "Study English vocabulary", time: "15 minutes").

[0701] Output: Data stored in the database.

[0702] Step 5:

[0703] Progress feedback

[0704] The server compiles user behavior records for a specified period from the database.

[0705] Example: Calculate the total time spent studying this week and arrive at "3 hours and 15 minutes."

[0706] The server generates progress feedback based on the aggregated results and sends it to the user via a chat app.

[0707] Example feedback: "Your total study time this week is 3 hours and 15 minutes."

[0708] Input: Data stored in a database.

[0709] Output: Progress feedback sent to the user.

[0710] Step 6:

[0711] Providing improvement suggestions

[0712] Enter a prompt statement that the server will use to pass past data stored in the database to the generative AI model.

[0713] The server generates a prompt sentence and sends it to the generation AI, which then generates an improvement proposal.

[0714] Example of an improvement idea: "If I study 30 minutes each day, I'll be two weeks away from achieving my goal."

[0715] The server sends the generated improvement proposal to the user.

[0716] Input: Data stored in the database, prompts for analysis.

[0717] Output: The improvement suggestions sent to the user.

[0718] (Application example 1)

[0719] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0720] Improving driving skills and maintaining safe driving are becoming increasingly important with the spread of autonomous vehicles. However, conventional driver training and feedback systems lack real-time capabilities and individual optimization, making it difficult to quickly respond to the individual needs of each driver. Furthermore, there is a lack of a system that allows drivers to easily and intuitively input their driving records and receive instant feedback. Given this background, there is a need for a system that provides effective feedback and improvement suggestions to improve driving skills.

[0721] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0722] In this invention, the server includes: means for receiving behavior record data entered by the user through a communication interface; a generating AI means for analyzing the behavior record data and classifying it into appropriate categories based on the analysis results; means for storing the categorized data in a database; means for evaluating the user's progress based on the stored data and providing feedback on the evaluation results; means for analyzing the stored data and providing the user with improvement suggestions; means for inputting the user's behavior record data through a head-mounted display; and means for providing feedback and improvement suggestions for improving driving skills. This allows drivers to easily enter their own driving records in real time and receive immediate feedback and specific improvement suggestions based on the input. This makes it possible to maintain safe driving and improve driving skills simultaneously.

[0723] A "communication interface" is an interactive device for a user to input behavior record data, and is a device having means for transmitting and receiving data.

[0724] "Behavior record data" is information in which a user records daily behavior, exercise, work content, and the like.

[0725] "Generative AI" is a technology that uses artificial intelligence to analyze incoming data and classify it into appropriate categories.

[0726] A "database" is an information collection system for efficiently storing and managing analyzed and classified data.

[0727] "User progress" is information indicating the degree of progress or achievement of a specific action or task performed by a user.

[0728] "Feedback" refers to information and advice provided to the User based on the analyzed and evaluated progress.

[0729] "Improvement Suggestions" are suggestions and advice to help users achieve better results.

[0730] A "head-mounted display" is a display device worn by a user on the head, and is a device for providing visual information.

[0731] "Feedback for improving driving skills" refers to evaluations and advice provided to users regarding specific operations and actions they perform while driving.

[0732] "Improvement suggestions for improving driving skills" are practical suggestions and advice provided to users to improve their driving skills.

[0733] This invention is a system that allows users to improve their driving skills in autonomous vehicles using a head-mounted display (HMD). The system inputs recorded user behavior data, analyzes and classifies it, and provides the driver with immediate feedback and suggestions for improvement. This section describes the detailed configuration and operation of the system.

[0734] Hardware and software used

[0735] Hardware:

[0736] Head-mounted display (HMD): Using Oculus Quest 2 as an example.

[0737] software:

[0738] Development environment: Unity3D

[0739] Communication API: WebSocket or HTTP REST API

[0740] Generation AI: OpenAI GPT-4

[0741] Database: Firebase Firestore

[0742] System configuration

[0743] The system consists of the following main components:

[0744] Receiving user input:

[0745] The user can input behavioral data through the HMD. For example, they can input "Today's driving was smooth."

[0746] Sending input data:

[0747] The HMD sends the input data, including the user ID, timestamp, and location information, to the server in real time.

[0748] Data analysis and classification:

[0749] The server passes the input data to the generative AI (GPT-4), which analyzes it using natural language processing technology and classifies it into an appropriate category. For example, "Lane change was delayed" is classified as "driving skill improvement area."

[0750] Save to database:

[0751] The server stores the parsed and categorized data in Firebase Firestore, where detailed information (user ID, category, content, timestamp, etc.) is stored.

[0752] Progress assessment and feedback:

[0753] The server aggregates past data and evaluates the user's progress, and the evaluation results are communicated to the user via the HMD as feedback.

[0754] Suggested improvements:

[0755] The server analyzes the stored data and uses AI to provide users with specific suggestions for improving their driving skills, such as "Keep a safe distance to reduce sudden braking."

[0756] Specific examples

[0757] 1. User Input:

[0758] The user inputs their observations while driving into the HMD's chat interface.

[0759] For example, enter "There were a lot of sudden braking today."

[0760] 2. Transmission and Analysis:

[0761] The HMD sends the input data to the server.

[0762] The server uses a generative AI (GPT-4) to analyze the data that shows "frequent sudden braking" and classifies it as "an area for improvement in driving technique."

[0763] 3. Database storage:

[0764] The server saves the analysis results in a database (Firebase Firestore).

[0765] 4. Feedback:

[0766] The server provides feedback such as, "You braked suddenly five times today. Next time, try to maintain a safe distance while driving."

[0767] 5. Suggestions for improvement:

[0768] An improvement suggestion was displayed on the HMD: "To reduce sudden braking, it is safer to try to maintain a constant distance between vehicles."

[0769] Example prompts for generative AI models

[0770] Below are some example prompts that can be used to properly analyze and categorize user input data and provide specific feedback and suggestions for improvement:

[0771] Analyze the user's driving record data and classify it into appropriate categories. Then, provide advice to improve the user's driving skills based on that data. For example, if the input is "I braked a lot today," classify it as "Something to improve in driving skills" and output the advice "Keep a safe distance."

[0772] These prompts allow generative AI models (such as GPT-4) to efficiently analyze user input data and provide appropriate feedback and suggestions for improvement.

[0773] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0774] Step 1:

[0775] The user wears the HMD and inputs behavioral record data through the chat interface. For example, the user inputs "I was late changing lanes." This input data includes the user ID and timestamp.

[0776] Input: Activity record data, user ID, timestamp

[0777] Output: Formatted behavioral record data

[0778] Step 2:

[0779] The device (HMD) sends the behavioral record data entered by the user to the server in real time using WebSocket and HTTP REST API.

[0780] Input: Formatted behavioral record data

[0781] Output: Activity record data sent to the server

[0782] Step 3:

[0783] The server passes the received behavioral record data to a generative AI model (GPT-4) for analysis. A prompt sentence is used during this process. The generative AI model uses natural language processing technology to analyze the input data and classify it into appropriate categories. For example, "Lane change was delayed" is classified as "Areas for improvement in driving technique."

[0784] Input: Activity record data sent to the server

[0785] Output: Classified data (with categories)

[0786] Step 4:

[0787] The server stores the analyzed data in a database (Firebase Firestore), including information such as user ID, category, content, and timestamp.

[0788] Input: Classified data (with categories)

[0789] Output: Data stored in the database

[0790] Step 5:

[0791] The server evaluates the user's progress based on the data stored in the database, and then provides the evaluation results to the user as real-time feedback. For example, the HMD displays, "You braked suddenly five times today."

[0792] Input: Data stored in the database

[0793] Output: Feedback information

[0794] Step 6:

[0795] The server analyzes past data stored in the database and generates and provides improvement suggestions to the user using a generative AI model. For example, an improvement suggestion such as "Maintain a safe distance to reduce sudden braking" is generated. This improvement suggestion is also displayed on the HMD.

[0796] Input: Data stored in database, historical data

[0797] Output: Improvement plan

[0798] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0799] This invention is a system that analyzes and classifies behavioral record data entered by the user through a communication interface, and also recognizes the user's emotions and provides feedback and suggestions for improvement. Below, we will explain in detail the program processing of this system in natural language, and provide examples.

[0800] Overall system overview

[0801] When a user enters behavioral record data using a communication interface such as a chat app, the device sends the data to a server. The server analyzes the received data using a generative AI and emotion engine, classifying it into appropriate categories and determining the user's emotions. The classified data and emotion information are then stored in a database, and the user's progress is evaluated. Along with the evaluation results, the user is provided with feedback based on their emotions, and improvement suggestions are also proposed based on past data.

[0802] Program processing overview

[0803] 1. Receiving user input

[0804] Users enter activity records such as study and training into a chat app.

[0805] For example, a user types "15 minutes of English vocabulary study" into the chat.

[0806] The terminal receives this input and prepares to transmit.

[0807] 2. Sending input data

[0808] The terminal sends the received input data to the server, including the user ID, input text, and timestamp.

[0809] 3. Data Receipt and Initial Analysis

[0810] The server receives the input data, converts it into a format for analysis, and passes it on to the next processing step.

[0811] 4. Analysis by generative AI and emotion engine

[0812] The server passes the received data to the generation AI and emotion engine for analysis.

[0813] Generative AI uses natural language processing technology to analyze input text and classify it into categories such as "study," "training," etc. For example, input data such as "15 minutes of English vocabulary study" will be classified as "study" and the time will be recognized as 15 minutes.

[0814] The emotion engine recognizes emotions from the user's input text, such as happiness, sadness, excitement, and stress.

[0815] 5. Receiving categorization and emotion determination results

[0816] The server receives the analysis results from the generation AI and emotion engine. The analysis results are received as structured data, including category (e.g., studying), activity (e.g., studying English vocabulary), time (e.g., 15 minutes), and emotion (e.g., joy).

[0817] 6. Saving to the database

[0818] The server stores the analysis and emotion determination results in a database, including the user ID, category, activity, time, emotion, and timestamp.

[0819] Check whether saving was successful and perform error handling.

[0820] 7. Evaluating progress and generating emotional feedback

[0821] The server evaluates the user's progress based on the data stored in the database, aggregating past data and calculating the duration and number of activities within a specific period.

[0822] A feedback message is generated according to the user's emotion. For example, if the user is judged to be "sad," encouraging feedback is generated.

[0823] 8. Submitting Feedback

[0824] The server sends a feedback message to the device, such as a notification that includes "Your total study time this week is 3 hours and 15 minutes. Great!"

[0825] The device receives this feedback message and displays it to the user, either as a push notification or a chat message.

[0826] 9. Proposal for improvement

[0827] The server analyzes past data stored in the database to understand the user's behavioral patterns and emotional trends.

[0828] Based on the results, the system generates effective improvement suggestions for the user. For example, it creates advice such as, "If you study for 30 minutes every day, you will have two weeks to reach your goal. It would also be effective to take breaks to relax."

[0829] 10. Submitting Improvement Suggestions

[0830] The server sends the generated improvement proposal to the terminal, and sends a message containing useful advice to the user.

[0831] The device receives the improvement suggestion message and displays it to the user. Notification methods include push notifications and chat messages.

[0832] In this way, the behavioral record data and emotional information entered by the user are managed and supported in an integrated manner through the steps of analysis, classification, storage, feedback, and improvement suggestions.

[0833] The processing flow will be explained below.

[0834] Step 1: Receiving User Input

[0835] The user inputs behavioral record data through a communication interface (e.g., a chat app). For example, the user inputs "Study English vocabulary for 15 minutes" into the chat.

[0836] The terminal receives this input and prepares to transmit.

[0837] Step 2: Submitting input data

[0838] The terminal sends the received input data to the server, including the user ID, input text, and timestamp.

[0839] The protocol used is HTTP POST request and real-time communication protocol.

[0840] Step 3: Receiving the data

[0841] The server receives the input data, converts it into a format suitable for analysis, and passes it on to the next processing step.

[0842] Step 4: Analysis by generative AI

[0843] The server passes the received data to the generation AI for analysis.

[0844] Generative AI uses natural language processing technology to analyze input text. For example, if the input data is "Study English vocabulary for 15 minutes," it will classify it into the category of "study" and recognize the time as 15 minutes.

[0845] Step 5: Emotion determination by the emotion engine

[0846] The server passes the received data to the emotion engine for analysis.

[0847] The emotion engine analyzes the input text and determines the user's emotions. For example, if the input data is "15 minutes of English vocabulary study," it will determine emotions such as "motivation" or "joy."

[0848] Step 6: Receiving categorization and emotion determination results

[0849] The server receives the analysis results from the generation AI and emotion engine. The analysis results are received as structured data, including category (e.g., studying), activity (e.g., studying English vocabulary), time (e.g., 15 minutes), and emotion (e.g., joy).

[0850] Step 7: Saving to the Database

[0851] The server stores the analysis and emotion determination results in a database, including the user ID, category, activity, time, emotion, and timestamp.

[0852] Check whether saving was successful and perform error handling.

[0853] Step 8: Evaluate your progress

[0854] The server evaluates the user's progress based on the data stored in the database, aggregating past data and calculating the duration and number of activities within a specific period.

[0855] For example, it generates an evaluation result such as "Your total study time this week is 3 hours and 15 minutes."

[0856] Step 9: Generate emotional feedback

[0857] The server generates feedback messages based on the evaluation results, and adjusts the content according to the user's emotions.

[0858] For example, if the user is judged to be "happy," feedback such as "Great! Let's keep up the good work to achieve our goal" is generated.

[0859] Step 10: Submit your feedback

[0860] The server sends a feedback message to the device, such as a notification that includes "Your total study time this week is 3 hours and 15 minutes. Great!"

[0861] The device receives this feedback message and displays it to the user, either as a push notification or a chat message.

[0862] Step 11: Propose improvements

[0863] The server analyzes past data stored in the database to understand the user's behavioral patterns and emotional trends.

[0864] Based on the results, the system generates effective improvement suggestions for the user. For example, it creates advice such as, "If you study for 30 minutes every day, you will have two weeks to reach your goal. It would also be effective to take breaks to relax."

[0865] Step 12: Submit your improvement proposal

[0866] The server sends the generated improvement proposal to the terminal, and sends a message containing useful advice to the user.

[0867] The device receives the improvement suggestion message and displays it to the user. Notification methods include push notifications and chat messages.

[0868] In this way, the behavioral record data and emotional information entered by the user are managed and supported in an integrated manner through the steps of analysis, classification, storage, feedback, and improvement suggestions.

[0869] Example 2

[0870] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0871] Conventional systems do not adequately analyze the behavioral record data entered by users through a communication interface or provide feedback on that data. Furthermore, they are also inadequate in providing appropriate feedback and improvement suggestions based on the user's emotions, making it difficult to maintain the user's motivation or provide appropriate improvement measures. To solve these issues, it is necessary to provide a system that comprehensively analyzes and manages the user's behavioral record data and emotions, and provides appropriate feedback and improvement suggestions.

[0872] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for receiving behavior record data input by a user through a communication interface; generative AI model means for analyzing the behavior record data and classifying the data into appropriate categories based on the analysis results; emotion engine means for determining the user's emotions from the data analyzed by the generative AI model means; means for storing the categorized data and the emotion determination results in a database; means for evaluating the user's progress based on the stored data and generating feedback according to the evaluation results and the emotions; and means for analyzing the stored data and providing the user with improvement suggestions. This makes it possible to analyze and manage the user's behavior record data and emotion information in an integrated manner and provide appropriate feedback and improvement suggestions.

[0873] A "user" is an individual or entity that inputs behavioral data through a communications interface.

[0874] A "communication interface" is software or an application that allows a user to input behavior record data and communicate with a terminal.

[0875] "Behavior record data" is data relating to activities and behaviors that a user inputs through a communication interface, and includes, for example, information relating to study and training.

[0876] A "terminal" is an electronic device used to receive user input and send it to a server, and includes smartphones, tablets, personal computers, etc.

[0877] A "server" is a central processing unit that receives data sent from a terminal, analyzes and stores it, and generates feedback.

[0878] A "generative AI model" is an artificial intelligence method for analyzing behavioral record data and classifying it into appropriate categories.

[0879] An "emotion engine" is a technology that determines a user's emotions from data analyzed by a generative AI model.

[0880] A "category" is the type of behavioral record data classified by the generative AI model, such as "study" or "training."

[0881] A "database" is a recording medium for storing data handled by the system, such as analysis results and emotion determination results.

[0882] "Progress" refers to the progress of an activity evaluated based on the user's behavior record data.

[0883] "Feedback" refers to response messages and advice provided based on the user's progress and emotions.

[0884] "Improvement proposals" are suggestions for improving user behavior that are generated by analyzing past data stored in the database.

[0885] This invention is a system that analyzes and classifies behavioral record data entered by a user through a communication interface, recognizes the user's emotions, and provides feedback and suggestions for improvement. A specific embodiment of this system is described below.

[0886] Hardware and software used

[0887] Hardware

[0888] Devices: smartphones, tablets, computers, etc.

[0889] Server: Central processing unit (cloud server, on-premise server)

[0890] software

[0891] Communication Interface: Chat App, Web Application

[0892] Generative AI models: AI models that use natural language processing techniques (e.g., GPT series)

[0893] Sentiment engine: Software that performs sentiment analysis (e.g., sentiment analysis library)

[0894] Database: Data storage and management system (e.g., MySQL, PostgreSQL)

[0895] Explanation of program processing

[0896] 1. Receiving user input

[0897] The user enters the activity record of study, training, etc. into the chat app. For example, the user enters "Studying English vocabulary for 15 minutes."

[0898] The terminal receives this input and prepares to send it. The received data includes the user ID, the input text, and a timestamp.

[0899] 2. Sending input data

[0900] The device sends the received data to the server, where it is transferred over the network using an API.

[0901] 3. Data Receipt and Initial Analysis

[0902] The server receives the input data and converts it into a format for parsing. The data is parsed in JSON format.

[0903] 4. Analysis by generative AI and emotion engine

[0904] The server passes the received data to the generative AI model and emotion engine for analysis. The generative AI model analyzes the input text and classifies it into categories such as "study" and "training."

[0905] For example, data entered as "15 minutes of studying English vocabulary" will be classified into the category "study" and the time will be recognized as 15 minutes.

[0906] The emotion engine recognizes emotions from the input text and determines the user's emotions as "joy," "sadness," "excitement," "stress," etc.

[0907] 5. Receiving analysis results

[0908] The server receives the analysis results from the generative AI model and the emotion engine, which include structured data such as category, activity, time, and emotion.

[0909] 6. Saving to the database

[0910] The server stores the analysis and emotion determination results in a database, including the user ID, category, activity, time, emotion, and timestamp.

[0911] 7. Evaluating progress and generating feedback

[0912] The server evaluates the user's progress based on the data in the database, for example, by aggregating past data and calculating the duration and number of activities within a specific period.

[0913] Generate feedback messages based on the user's emotions, such as "You've studied a total of 3 hours and 15 minutes this week. Great!"

[0914] 8. Submitting Feedback

[0915] The server generates a feedback message and sends it to the device, which receives it and displays it to the user, either as a push notification or a chat message.

[0916] 9. Proposal for improvement

[0917] The server analyzes past data stored in the database to understand the user's behavioral patterns and emotional trends.

[0918] Based on the results, the system generates effective improvement suggestions for the user. For example, it creates advice such as, "If you study for 30 minutes every day, you will be two weeks away from achieving your goal. It would also be effective to take breaks to relax."

[0919] 10. Submitting Improvement Suggestions

[0920] The server sends the generated improvement proposal to the device. The device receives the improvement proposal message and displays it to the user. Notification methods include push notifications and chat messages.

[0921] Examples and prompts

[0922] Specific examples

[0923] User: "Running for 30 minutes"

[0924] Terminal: Receives this input and sends it to the server.

[0925] Server: Analyzes the received data, classifies it into the "Training" category, and determines the emotion as "Excitement."

[0926] Server: Stores the data in a database, evaluates progress, and sends feedback to the user, such as "You've trained a total of 2 hours this week. Great!"

[0927] Server: Send the following improvement suggestion: "Your health will improve even more if you increase your daily training by 15 minutes."

[0928] Prompt Sentence Examples

[0929] "The user entered 'Study English vocabulary for 20 minutes.' Please analyze this data."

[0930] "The user's emotion was determined to be sadness. Please generate appropriate feedback."

[0931] "Analyze user behavior patterns based on data from the past three months and propose improvements."

[0932] The above is a specific example of how to implement the invention. By using this system, it is possible to manage a user's behavioral record data and emotional information in an integrated manner, and provide appropriate feedback and suggestions for improvement.

[0933] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0934] Program processing steps

[0935] Step 1: Receiving User Input

[0936] A user enters behavioral data into a chat app. For example, the user enters "Study English vocabulary for 15 minutes."

[0937] The terminal receives this input and generates data containing the user ID, the input text, and a timestamp.

[0938] The input is "Study English vocabulary for 15 minutes" and the output is the user ID, the input text, and a timestamp.

[0939] Step 2: Submitting input data

[0940] The device sends the received data to the server, and then sends the data via the network using an API.

[0941] The inputs are the user ID, the input text, and a timestamp, and the output is the data sent to the server.

[0942] Step 3: Data reception and initial analysis

[0943] The server receives the input data and converts it into a format for analysis.

[0944] The input is a user ID, input text, and a timestamp, and the output is JSON format data. As a concrete example, the received data is mapped to key-value pairs.

[0945] Step 4: Analysis by generative AI and emotion engine

[0946] The server passes the data to the generative AI model and emotion engine to begin analysis.

[0947] The generative AI model analyzes the input text and classifies it into categories such as "study," "training," etc. For example, the input data "15 minutes of English vocabulary study" will be classified as "study" and the time will be recognized as 15 minutes.

[0948] The emotion engine identifies emotions from the input text, determining, for example, "joy."

[0949] The input is JSON formatted data and the output is data on category, activity, time, and emotion.

[0950] Step 5: Receive the analysis results

[0951] The server receives the analysis results from the generative AI model and the emotion engine.

[0952] The input is the analysis results of the generative AI model and emotion engine, and the output is structured data on category (e.g., studying), activity (e.g., studying English vocabulary), time (e.g., 15 minutes), and emotion (e.g., joy).

[0953] Step 6: Saving to the Database

[0954] The server stores the analysis and emotion determination results in a database.

[0955] The inputs are category, activity, time, emotion, user ID, and timestamp, and the output is a record stored in a database.

[0956] Step 7: Assess progress and generate feedback

[0957] The server evaluates the user's progress based on the stored data, for example by aggregating past data and calculating the duration and number of activities within a specific period.

[0958] Generate feedback messages based on the user's emotions, such as "Your total study time this week is 3 hours and 15 minutes. Great!"

[0959] The input is historical data retrieved from the database, and the output is progress assessments and feedback messages.

[0960] Step 8: Submit your feedback

[0961] The server sends the generated feedback message to the terminal, which receives the feedback message and displays it to the user.

[0962] The input is the feedback message and the output is the feedback notification displayed on the terminal.

[0963] Step 9: Propose improvements

[0964] The server analyzes past data stored in the database to understand the user's behavioral patterns and emotional trends.

[0965] Based on the results, the system generates effective improvement suggestions for the user. For example, it creates advice such as, "If you study for 30 minutes every day, you will be two weeks away from achieving your goal. It would also be effective to take breaks to relax."

[0966] The input is past data and the output is a message of proposed improvements.

[0967] Step 10: Submit your improvement proposal

[0968] The server sends the generated improvement proposal to the terminal, which receives the improvement proposal message and displays it to the user.

[0969] The input is the improvement proposal message, and the output is the improvement proposal notification displayed on the terminal.

[0970] (Application example 2)

[0971] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0972] Conventional advertising display systems typically provide uniform advertisements based on a user's behavioral history, but this often results in advertisements that ignore the user's emotional state, which can reduce user engagement. Furthermore, because advertisements are not optimized in response to emotional fluctuations, the user experience is not consistently high quality. The present invention aims to solve these problems by proposing a system that provides appropriate advertisements based on a user's behavioral history and emotional state.

[0973] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving behavior record data input by a user through a communication interface, generation AI means for analyzing the behavior record data and classifying the data into appropriate categories based on the analysis results, means for saving the data classified into the categories in a database, means for evaluating the user's progress based on the saved data and feeding back the evaluation results, means for analyzing the saved data and providing the user with improvement suggestions, emotion analysis means for analyzing the user's emotions, and means for suggesting advertisements based on the emotion analysis results. This makes it possible to display personalized advertisements according to the user's emotional state.

[0974] "User" means any individual or organization that uses this system.

[0975] A "communication interface" is a communication means such as a chat application or web form that allows a user to input behavior record data.

[0976] "Behavior record data" refers to data entered in text format by a user about their daily activities and work.

[0977] "Generative AI means" is an artificial intelligence technology that analyzes input behavioral record data and classifies its contents into appropriate categories.

[0978] "Category" refers to the classification of activity record data by type, such as "study," "training," and "relaxation."

[0979] The "database" is a data storage system for storing analyzed behavioral record data and emotional information.

[0980] "Means for assessing progress" refers to means for assessing a user's efforts and achievements based on stored data.

[0981] "Feedback" refers to messages and notifications that communicate progress assessment results to the user.

[0982] The "means for providing improvement suggestions" is a means for analyzing stored data and making suggestions to improve user behavior.

[0983] "Emotion analysis means" is a technology for identifying and analyzing emotions from user behavior record data.

[0984] The "means for suggesting advertisements" refers to a means for selecting and presenting the most suitable advertisement to the user based on emotion analysis and behavioral record data.

[0985] The system according to the present invention analyzes the behavioral record data entered by the user and personalizes advertisements based on the analysis results. The program processing of this system will be described in detail below.

[0986] First, a user uses a communication interface such as a chat application to input information about their daily activities and tasks. For example, they input text such as, "I had a great lunch with my friends today." This input data is received by the user's device and sent to the server.

[0987] The server analyzes the received data using a generation AI and a sentiment analysis engine. The generation AI uses natural language processing technology to analyze the input text and classify it into an appropriate category. For example, the activity "lunch" is classified as "relaxation." The sentiment analysis engine determines the emotion from the user's input text. For example, the emotion "fun" is determined.

[0988] The analysis results are passed to the server as structured data, including categories, activity details, time, and emotions. This data is stored in a database. The stored data is used to evaluate the user's progress. Based on the evaluation results, feedback is generated according to the user's emotions. For example, feedback such as "You had a great day today!" is sent.

[0989] Furthermore, the server will suggest advertisements based on the results of the sentiment analysis. For example, if the user is judged to be "happy," advertisements for relaxation-related products and services will be suggested. The advertisements will be sent to the user as push notifications or chat messages.

[0990] Hardware and software used:

[0991] User device: smartphone, tablet, etc.

[0992] Communication interfaces: chat applications, web forms

[0993] Server: Cloud server, on-premise server, etc.

[0994] Generative AI: AI models using natural language processing techniques (e.g., TextBlob)

[0995] Sentiment analysis engine: An engine that uses emotion recognition technology

[0996] Database: Data storage system (e.g. RDBMS, NoSQL)

[0997] Notification System: Push Notification Library

[0998] Examples and prompts:

[0999] If a user types, "I had a fun lunch with a friend today," the system analyzes this data and determines the category "relaxed" and the emotion "fun," then sends appropriate feedback and advertisements to the user.

[1000] Example prompt sentence:

[1001] "I had a great lunch with a friend today. I want to determine the sentiment and suggest an appropriate ad."

[1002] In this way, users receive feedback and advertising based on their behavior and emotions, resulting in a more consistent and high-quality experience.

[1003] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1004] Step 1:

[1005] A user inputs behavior record data using a communication interface. For example, the user inputs "I had a great lunch with my friends today" into a chat application. This input data is received by the user's terminal.

[1006] Step 2:

[1007] The terminal sends the received input data, including the user ID, text content, and timestamp, to the server. The server receives this data and passes it on to the next processing step.

[1008] Step 3:

[1009] The server converts the incoming data into a format for analysis and passes it to the generative AI and sentiment analysis engine, for example by tokenizing the input text and preprocessing it for contextual analysis.

[1010] Step 4:

[1011] The generative AI uses natural language processing techniques to analyze the input text and classify it into a category. For example, the activity "lunch" is classified as "relaxation." The sentiment analysis engine determines the emotion from the input text. For example, the emotion "fun" is determined.

[1012] Step 5:

[1013] The server receives the analysis results and formats them as structured data, including category, activity, emotion, and time. For example, the data could be that the category is "relaxing," the emotion is "fun," the activity is "having lunch with a friend," and the time is "1 hour."

[1014] Step 6:

[1015] The server saves the formatted data to the database, including the user ID, category, activity, emotion, time, and timestamp. It checks whether the save was successful and handles any errors.

[1016] Step 7:

[1017] The server evaluates the user's progress based on the stored data. For example, it may compile the time spent on relaxation activities over the past week and store this data as progress data. This data is used to evaluate the user's progress.

[1018] Step 8:

[1019] Generates feedback messages based on the results of sentiment analysis. For example, if the user is judged to be "happy," the system generates feedback such as "You had a great day today!"

[1020] Step 9:

[1021] The server sends a feedback message to the user's device, which receives the message and displays it to the user as a push notification or chat message.

[1022] Step 10:

[1023] The server proposes advertisements based on the results of the emotion analysis. For example, if the emotion is determined to be "fun," the server selects advertisements for products related to relaxation and proposes them to the user.

[1024] Step 11:

[1025] The server sends the selected advertisement to the user's device, which receives it and displays it to the user. The advertisement is displayed as a push notification or chat message.

[1026] Through the above process, personalized advertisements are provided based on the user's behavioral record data and emotional state.

[1027] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1028] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1029] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1030] [Third embodiment]

[1031] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1032] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1033] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1034] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1035] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1036] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1037] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1038] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1039] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1040] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1041] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1042] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1043] This invention is a system that allows users to input behavioral record data through a communication interface they use on a daily basis, and then analyzes, classifies, saves, provides feedback, and suggests improvements to that data. Below, we will explain in detail the program processing of this system in natural language, and provide examples.

[1044] Overall system overview

[1045] When a user enters behavioral record data using a communication interface such as a chat app, the device sends the data to a server. The server analyzes the received data using generative AI and classifies it into appropriate categories. The classified data is then stored in a database, and the user's progress is evaluated. Feedback is provided to the user along with the evaluation results, and improvement plans are also proposed based on past data.

[1046] Program processing overview

[1047] 1. Receiving user input

[1048] Users enter activity records such as study and training into a chat app.

[1049] For example, a user types "15 minutes of English vocabulary study" into the chat.

[1050] 2. Sending input data

[1051] The device sends the data entered from the chat app to the server.

[1052] This data also includes the user ID and timestamp.

[1053] 3. Data Analysis and Categorization

[1054] The server passes the received data to the generation AI for analysis.

[1055] The generative AI uses natural language processing technology to analyze the data and classify it into categories such as "study" and "training."

[1056] As a concrete example, "15 minutes of studying English vocabulary" is classified into the category "study" and is recognized as a time period of 15 minutes.

[1057] 4. Saving to the database

[1058] The server stores the analysis results in a database.

[1059] The information stored includes details such as user ID, category, activity, time, and timestamp.

[1060] 5. Progress Feedback

[1061] The server aggregates the user's past data and calculates their progress.

[1062] For example, if the total study time this week is 3 hours and 15 minutes, that information is notified to the user as feedback.

[1063] 6. Proposal for improvement

[1064] The server analyzes past data stored in the database to understand user behavior patterns.

[1065] Based on the results, effective improvement proposals are generated and provided to the user.

[1066] For example, advice like, "If you study for 30 minutes every day, you're two weeks away from achieving your goal."

[1067] Specific examples

[1068] 1. User Input

[1069] A user types "3 sets of 10 bench presses at the gym" into a chat app.

[1070] 2. Transmission and Analysis

[1071] The terminal sends user input to the server.

[1072] The server uses a generative AI to analyze the data of "Bench press at the gym, 10 repetitions, 3 sets" and classifies it into the category of "Training." The content is recognized as "Bench press," the number of repetitions is recognized as "10 repetitions," and the number of sets is recognized as "3 sets."

[1073] 3. Database storage

[1074] The server stores the analysis results in a database, for example, in the format {User ID: 12345, Category: Training, Content: Bench Press, Repetitions: 10, Sets: 3, Time: [Automatically estimated time], Timestamp: [Recorded time]}.

[1075] 4. Feedback

[1076] The server provides feedback to the user saying, "Today's training sessions have been recorded. The total number of sets is three."

[1077] 5. Proposal for improvement

[1078] The server suggests to the user an improvement suggestion such as, "To increase the number of times you can bench press, it would be effective to shorten the rest time a little."

[1079] In this way, users can efficiently record their actions and receive feedback on their progress and suggestions for improvement through a single communication interface.

[1080] The processing flow will be explained below.

[1081] Step 1: Receiving User Input

[1082] The user inputs behavioral record data through a communication interface (e.g., a chat app). For example, the user inputs "Study English vocabulary for 15 minutes" into the chat.

[1083] The terminal receives this input and prepares to transmit.

[1084] Step 2: Submitting input data

[1085] The terminal sends the received input data to the server, including the user ID, input text, and timestamp.

[1086] The protocol used is HTTP POST request and real-time communication protocol.

[1087] Step 3: Receiving the data

[1088] The server receives the input data, converts it into a format suitable for analysis, and passes it on to the next processing step.

[1089] Step 4: Analysis by generative AI

[1090] The server passes the received data to the generation AI for analysis.

[1091] Generative AI uses natural language processing technology to analyze input text. For example, if the input data is "Study English vocabulary for 15 minutes," it will classify it into the category of "study" and recognize the time as 15 minutes.

[1092] Step 5: Receiving categorization results

[1093] The server receives the analysis results (category classification results) from the generation AI.

[1094] The analysis results are received as structured data, such as category (e.g., studying), activity (e.g., studying English vocabulary), and time (e.g., 15 minutes).

[1095] Step 6: Saving to the Database

[1096] The server stores the analysis results in a database, including the user ID, category, activity, time, and timestamp.

[1097] Check whether saving was successful and perform error handling.

[1098] Step 7: Assess your progress

[1099] The server evaluates the user's progress based on the data stored in the database, aggregating past data and calculating the duration and number of activities within a specific period.

[1100] For example, it generates an evaluation result such as "Your total study time this week is 3 hours and 15 minutes."

[1101] Step 8: Generate feedback

[1102] The server generates a feedback message based on the evaluation results.

[1103] Prepare the generated feedback message for sending to the user.

[1104] Step 9: Submit your feedback

[1105] The server sends a feedback message to the device, such as a notification that includes a message like "Your total study time this week is 3 hours and 15 minutes."

[1106] The device receives this feedback message and displays it to the user, either as a push notification or a chat message.

[1107] Step 10: Propose improvements

[1108] The server analyzes past data stored in the database to understand user behavior patterns and trends.

[1109] Based on the results, it generates effective improvement suggestions for the user, such as "If you study for 30 minutes every day, you'll be two weeks away from achieving your goal."

[1110] Step 11: Submit your improvement proposal

[1111] The server sends the generated improvement proposal to the terminal, and sends a message containing useful advice to the user.

[1112] The device receives the improvement suggestion message and displays it to the user. Notification methods include push notifications and chat messages.

[1113] In this way, the behavioral record data entered by the user is managed and supported in an integrated manner, going through the steps of analysis, classification, storage, feedback, and improvement suggestions.

[1114] Example 1

[1115] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1116] Conventional behavioral recording systems make it difficult for users to efficiently record their daily activities and appropriately evaluate their progress. Furthermore, they do not provide sufficient feedback or suggestions for improvement, making it difficult to promote behavioral improvement. This makes it difficult for users to understand their own behavioral patterns and take effective measures to improve.

[1117] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1118] In this invention, the server includes means for receiving behavior record data entered by a user through a communication interface, artificial intelligence (AI) means for analyzing the behavior record data and classifying it into appropriate categories based on the analysis results, means for saving the categorized data in a database, means for evaluating the user's progress based on the saved data and providing feedback on the evaluation results, means for analyzing the saved data and providing the user with improvement suggestions, means for assigning a user ID and a timestamp to the behavior record data, means for generating prompt sentences for analysis by the AI, and means for aggregating past data and evaluating behavior records over a certain period of time. This allows users to efficiently record their own behavior, making it easier to understand their progress, and also enabling them to receive specific improvement suggestions.

[1119] A "communications interface" is the means by which a user inputs data and interacts with a system.

[1120] "Behavior record data" is data that includes specific information about the user's behavior, such as studying or training.

[1121] A "generative AI means" is a means that uses generative artificial intelligence technology to analyze input data and classify it into appropriate categories.

[1122] A "database" is a system for systematically storing analyzed data and enabling quick retrieval of necessary information.

[1123] The "means for evaluating progress" is a means for measuring and evaluating the progress of a user's activities based on the data stored in the database.

[1124] The "feedback means" is a means for notifying the user of the progress evaluation results and providing useful information regarding the user's actions.

[1125] The "means for providing improvement proposals" is a means for analyzing past data stored in a database and generating and providing effective improvement proposals to users.

[1126] A "user ID" is an identifier that uniquely identifies each user.

[1127] The "timestamp" indicates the date and time when the behavior record data was input.

[1128] A "prompt sentence" is an instruction sentence used by the generative artificial intelligence when analyzing data.

[1129] The "means for aggregating past data" is a means for generating statistical information based on behavioral records within a certain period of time and understanding the activity patterns of users.

[1130] This invention is a system in which a user inputs behavioral record data through a communication interface, and the data is analyzed using a generating AI means, and then classified, saved, evaluated, and provided feedback. The specific implementation method of this system is described below.

[1131] Overall overview

[1132] The user uses a device such as a smartphone or PC to input behavioral record data through a communication interface such as a chat app. The device receives the user's input data, assigns a user ID and timestamp to it, and sends it to a server. The server analyzes the received data using generative AI and classifies it into appropriate categories. The server then stores the classified data in a database and evaluates the user's progress. The evaluation results are fed back to the user, and further improvement suggestions are provided based on the data.

[1133] Hardware and software used

[1134] Hardware:

[1135] Devices such as smartphones and computers.

[1136] Server equipment that operates the server.

[1137] software:

[1138] Communication interfaces such as chat apps.

[1139] A database system (e.g., MySQL, PostgreSQL) for storing and managing data.

[1140] APIs and libraries for using generative AI models (e.g., GPT-4).

[1141] Details of data processing and calculation

[1142] 1. Receiving user input:

[1143] A user enters an action record such as "Study English vocabulary for 15 minutes" into a chat app.

[1144] 2. Send input data:

[1145] The terminal sends the input data to the server along with the user ID and a timestamp.

[1146] Sending format example:

[1147] {

[1148] User ID: 12345,

[1149] Record: "15 minutes of English vocabulary study",

[1150] Timestamp: "2023-10-10 10:00:00"

[1151] }

[1152] 3. Data Analysis and Categorization:

[1153] The server passes the received data to the generation AI, which generates a prompt for analysis.

[1154] An example of a generated prompt is: "The user entered 'Study English vocabulary for 15 minutes.' Please analyze this data to extract categories and detailed information."

[1155] The generating AI categorizes it as "study" and recognizes the time as "15 minutes."

[1156] 4. Save to database:

[1157] The server stores the analysis results in a database.

[1158] Examples of data that may be saved:

[1159] {

[1160] User ID: 12345,

[1161] Category: "Study",

[1162] Activity: "Study English vocabulary",

[1163] Time: "15 minutes",

[1164] Timestamp: "2023-10-10 10:00:00"

[1165] }

[1166] 5. Progress feedback:

[1167] The server aggregates data from the database over a period of time and evaluates the user's progress.

[1168] Example: If the total study time this week is 3 hours and 15 minutes, send the user a feedback message saying "Your total study time this week is 3 hours and 15 minutes."

[1169] 6. Providing suggestions for improvement:

[1170] The server sends a prompt to the generation AI, which generates improvement suggestions based on past data.

[1171] An example of a generated improvement suggestion: "If you study for 30 minutes every day, you're two weeks away from achieving your goal."

[1172] Specific examples

[1173] If a user types "Bench press at the gym, 10 reps, 3 sets" into a chat app, the device receives this, assigns a user ID and timestamp, and sends it to the server. The server then uses the generated AI to analyze the data, generating a prompt that reads, "The user entered 'Bench press at the gym, 10 reps, 3 sets'. Please analyze this data and extract the category and detailed information." The generated AI classifies this as a "Training" category, and recognizes that the content is "Bench press," the number of repetitions is "10," and the number of sets is "3 sets." These analysis results are stored in a database and later used to provide progress feedback and improvement suggestions.

[1174] In this way, the system allows users to efficiently log their actions and receive feedback on their progress and suggestions for improvement.

[1175] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1176] Step 1:

[1177] Receiving User Input

[1178] The user inputs behavior record data into the chat application.

[1179] Example: Enter "Study English vocabulary for 15 minutes."

[1180] The terminal receives user input in real time.

[1181] The terminal assigns a user ID and a timestamp to the received data.

[1182] Input: User behavior record data (e.g., "Study English vocabulary for 15 minutes").

[1183] Output: Data containing user ID, behavior record data, and timestamp (e.g., {User ID: 12345, Record: "Study English vocabulary for 15 minutes", Timestamp: "2023-10-10 10:00:00"}).

[1184] Step 2:

[1185] Sending input data

[1186] The terminal transmits the received behavior record data to the server.

[1187] The data is sent as an HTTP POST request.

[1188] Data to be sent: Data including user ID, behavioral record data, and timestamp.

[1189] Input: Data including user ID, behavior record data, and timestamp.

[1190] Output: Data sent to the server (e.g., {User ID: 12345, Record: "Study English vocabulary for 15 minutes", Timestamp: "2023-10-10 10:00:00"}).

[1191] Step 3:

[1192] Data analysis and categorization

[1193] The server takes the received data and generates a prompt for analysis.

[1194] The server sends a prompt to the generative AI model.

[1195] A generative AI model analyzes the data and classifies it into appropriate categories.

[1196] Example prompt: "The user entered 'Study English vocabulary for 15 minutes.' Please parse this data to extract categories and detailed information."

[1197] Input: Data including user ID, behavior record data, and timestamp.

[1198] Output: Analysis results (category: "Study", activity: "Study English vocabulary", time: "15 minutes").

[1199] Step 4:

[1200] Saving to a database

[1201] The server obtains the analysis results and stores them in a database.

[1202] Save format: {User ID: 12345, Category: "Study", Activity: "Study English vocabulary", Time: "15 minutes", Timestamp: "2023-10-10 10:00:00"}

[1203] Input: Analysis results (category: "Study", activity: "Study English vocabulary", time: "15 minutes").

[1204] Output: Data stored in the database.

[1205] Step 5:

[1206] Progress feedback

[1207] The server compiles user behavior records for a specified period from the database.

[1208] Example: Calculate the total time spent studying this week and arrive at "3 hours and 15 minutes."

[1209] The server generates progress feedback based on the aggregated results and sends it to the user via a chat app.

[1210] Example feedback: "Your total study time this week is 3 hours and 15 minutes."

[1211] Input: Data stored in a database.

[1212] Output: Progress feedback sent to the user.

[1213] Step 6:

[1214] Providing improvement suggestions

[1215] Enter a prompt statement that the server will use to pass past data stored in the database to the generative AI model.

[1216] The server generates a prompt sentence and sends it to the generation AI, which then generates an improvement proposal.

[1217] Example of an improvement idea: "If I study 30 minutes each day, I'll be two weeks away from achieving my goal."

[1218] The server sends the generated improvement proposal to the user.

[1219] Input: Data stored in the database, prompts for analysis.

[1220] Output: The improvement suggestions sent to the user.

[1221] (Application example 1)

[1222] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1223] Improving driving skills and maintaining safe driving are becoming increasingly important with the spread of autonomous vehicles. However, conventional driver training and feedback systems lack real-time capabilities and individual optimization, making it difficult to quickly respond to the individual needs of each driver. Furthermore, there is a lack of a system that allows drivers to easily and intuitively input their driving records and receive instant feedback. Given this background, there is a need for a system that provides effective feedback and improvement suggestions to improve driving skills.

[1224] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1225] In this invention, the server includes: means for receiving behavior record data entered by the user through a communication interface; a generating AI means for analyzing the behavior record data and classifying it into appropriate categories based on the analysis results; means for storing the categorized data in a database; means for evaluating the user's progress based on the stored data and providing feedback on the evaluation results; means for analyzing the stored data and providing the user with improvement suggestions; means for inputting the user's behavior record data through a head-mounted display; and means for providing feedback and improvement suggestions for improving driving skills. This allows drivers to easily enter their own driving records in real time and receive immediate feedback and specific improvement suggestions based on the input. This makes it possible to maintain safe driving and improve driving skills simultaneously.

[1226] A "communication interface" is an interactive device for a user to input behavior record data, and is a device having means for transmitting and receiving data.

[1227] "Behavior record data" is information in which a user records daily behavior, exercise, work content, and the like.

[1228] "Generative AI" is a technology that uses artificial intelligence to analyze incoming data and classify it into appropriate categories.

[1229] A "database" is an information collection system for efficiently storing and managing analyzed and classified data.

[1230] "User progress" is information indicating the degree of progress or achievement of a specific action or task performed by a user.

[1231] "Feedback" refers to information and advice provided to the User based on the analyzed and evaluated progress.

[1232] "Improvement Suggestions" are suggestions and advice to help users achieve better results.

[1233] A "head-mounted display" is a display device worn by a user on the head, and is a device for providing visual information.

[1234] "Feedback for improving driving skills" refers to evaluations and advice provided to users regarding specific operations and actions they perform while driving.

[1235] "Improvement suggestions for improving driving skills" are practical suggestions and advice provided to users to improve their driving skills.

[1236] This invention is a system that allows users to improve their driving skills in autonomous vehicles using a head-mounted display (HMD). The system inputs recorded user behavior data, analyzes and classifies it, and provides the driver with immediate feedback and suggestions for improvement. This section describes the detailed configuration and operation of the system.

[1237] Hardware and software used

[1238] Hardware:

[1239] Head-mounted display (HMD): Using Oculus Quest 2 as an example.

[1240] software:

[1241] Development environment: Unity3D

[1242] Communication API: WebSocket or HTTP REST API

[1243] Generation AI: OpenAI GPT-4

[1244] Database: Firebase Firestore

[1245] System configuration

[1246] The system consists of the following main components:

[1247] Receiving user input:

[1248] The user can input behavioral data through the HMD. For example, they can input "Today's driving was smooth."

[1249] Sending input data:

[1250] The HMD sends the input data, including the user ID, timestamp, and location information, to the server in real time.

[1251] Data analysis and classification:

[1252] The server passes the input data to the generative AI (GPT-4), which analyzes it using natural language processing technology and classifies it into an appropriate category. For example, "Lane change was delayed" is classified as "driving skill improvement area."

[1253] Save to database:

[1254] The server stores the parsed and categorized data in Firebase Firestore, where detailed information (user ID, category, content, timestamp, etc.) is stored.

[1255] Progress assessment and feedback:

[1256] The server aggregates past data and evaluates the user's progress, and the evaluation results are communicated to the user via the HMD as feedback.

[1257] Suggested improvements:

[1258] The server analyzes the stored data and uses AI to provide users with specific suggestions for improving their driving skills, such as "Keep a safe distance to reduce sudden braking."

[1259] Specific examples

[1260] 1. User Input:

[1261] The user inputs their observations while driving into the HMD's chat interface.

[1262] For example, enter "There were a lot of sudden braking today."

[1263] 2. Transmission and Analysis:

[1264] The HMD sends the input data to the server.

[1265] The server uses a generative AI (GPT-4) to analyze the data that shows "frequent sudden braking" and classifies it as "an area for improvement in driving technique."

[1266] 3. Database storage:

[1267] The server saves the analysis results in a database (Firebase Firestore).

[1268] 4. Feedback:

[1269] The server provides feedback such as, "You braked suddenly five times today. Next time, try to maintain a safe distance while driving."

[1270] 5. Suggestions for improvement:

[1271] An improvement suggestion was displayed on the HMD: "To reduce sudden braking, it is safer to try to maintain a constant distance between vehicles."

[1272] Example prompts for generative AI models

[1273] Below are some example prompts that can be used to properly analyze and categorize user input data and provide specific feedback and suggestions for improvement:

[1274] Analyze the user's driving record data and classify it into appropriate categories. Then, provide advice to improve the user's driving skills based on that data. For example, if the input is "I braked a lot today," classify it as "Something to improve in driving skills" and output the advice "Keep a safe distance."

[1275] These prompts allow generative AI models (such as GPT-4) to efficiently analyze user input data and provide appropriate feedback and suggestions for improvement.

[1276] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1277] Step 1:

[1278] The user wears the HMD and inputs behavioral record data through the chat interface. For example, the user inputs "I was late changing lanes." This input data includes the user ID and timestamp.

[1279] Input: Activity record data, user ID, timestamp

[1280] Output: Formatted behavioral record data

[1281] Step 2:

[1282] The device (HMD) sends the behavioral record data entered by the user to the server in real time using WebSocket and HTTP REST API.

[1283] Input: Formatted behavioral record data

[1284] Output: Activity record data sent to the server

[1285] Step 3:

[1286] The server passes the received behavioral record data to a generative AI model (GPT-4) for analysis. A prompt sentence is used during this process. The generative AI model uses natural language processing technology to analyze the input data and classify it into appropriate categories. For example, "Lane change was delayed" is classified as "Areas for improvement in driving technique."

[1287] Input: Activity record data sent to the server

[1288] Output: Classified data (with categories)

[1289] Step 4:

[1290] The server stores the analyzed data in a database (Firebase Firestore), including information such as user ID, category, content, and timestamp.

[1291] Input: Classified data (with categories)

[1292] Output: Data stored in the database

[1293] Step 5:

[1294] The server evaluates the user's progress based on the data stored in the database, and then provides the evaluation results to the user as real-time feedback. For example, the HMD displays, "You braked suddenly five times today."

[1295] Input: Data stored in the database

[1296] Output: Feedback information

[1297] Step 6:

[1298] The server analyzes past data stored in the database and generates and provides improvement suggestions to the user using a generative AI model. For example, an improvement suggestion such as "Maintain a safe distance to reduce sudden braking" is generated. This improvement suggestion is also displayed on the HMD.

[1299] Input: Data stored in database, historical data

[1300] Output: Improvement plan

[1301] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1302] This invention is a system that analyzes and classifies behavioral record data entered by the user through a communication interface, and also recognizes the user's emotions and provides feedback and suggestions for improvement. Below, we will explain in detail the program processing of this system in natural language, and provide examples.

[1303] Overall system overview

[1304] When a user enters behavioral record data using a communication interface such as a chat app, the device sends the data to a server. The server analyzes the received data using a generative AI and emotion engine, classifying it into appropriate categories and determining the user's emotions. The classified data and emotion information are then stored in a database, and the user's progress is evaluated. Along with the evaluation results, the user is provided with feedback based on their emotions, and improvement suggestions are also proposed based on past data.

[1305] Program processing overview

[1306] 1. Receiving user input

[1307] Users enter activity records such as study and training into a chat app.

[1308] For example, a user types "15 minutes of English vocabulary study" into the chat.

[1309] The terminal receives this input and prepares to transmit.

[1310] 2. Sending input data

[1311] The terminal sends the received input data to the server, including the user ID, input text, and timestamp.

[1312] 3. Data Receipt and Initial Analysis

[1313] The server receives the input data, converts it into a format for analysis, and passes it on to the next processing step.

[1314] 4. Analysis by generative AI and emotion engine

[1315] The server passes the received data to the generation AI and emotion engine for analysis.

[1316] Generative AI uses natural language processing technology to analyze input text and classify it into categories such as "study," "training," etc. For example, input data such as "15 minutes of English vocabulary study" will be classified as "study" and the time will be recognized as 15 minutes.

[1317] The emotion engine recognizes emotions from the user's input text, such as happiness, sadness, excitement, and stress.

[1318] 5. Receiving categorization and emotion determination results

[1319] The server receives the analysis results from the generation AI and emotion engine. The analysis results are received as structured data, including category (e.g., studying), activity (e.g., studying English vocabulary), time (e.g., 15 minutes), and emotion (e.g., joy).

[1320] 6. Saving to the database

[1321] The server stores the analysis and emotion determination results in a database, including the user ID, category, activity, time, emotion, and timestamp.

[1322] Check whether saving was successful and perform error handling.

[1323] 7. Evaluating progress and generating emotional feedback

[1324] The server evaluates the user's progress based on the data stored in the database, aggregating past data and calculating the duration and number of activities within a specific period.

[1325] A feedback message is generated according to the user's emotion. For example, if the user is judged to be "sad," encouraging feedback is generated.

[1326] 8. Submitting Feedback

[1327] The server sends a feedback message to the device, such as a notification that includes "Your total study time this week is 3 hours and 15 minutes. Great!"

[1328] The device receives this feedback message and displays it to the user, either as a push notification or a chat message.

[1329] 9. Proposal for improvement

[1330] The server analyzes past data stored in the database to understand the user's behavioral patterns and emotional trends.

[1331] Based on the results, the system generates effective improvement suggestions for the user. For example, it creates advice such as, "If you study for 30 minutes every day, you will have two weeks to reach your goal. It would also be effective to take breaks to relax."

[1332] 10. Submitting Improvement Suggestions

[1333] The server sends the generated improvement proposal to the terminal, and sends a message containing useful advice to the user.

[1334] The device receives the improvement suggestion message and displays it to the user. Notification methods include push notifications and chat messages.

[1335] In this way, the behavioral record data and emotional information entered by the user are managed and supported in an integrated manner through the steps of analysis, classification, storage, feedback, and improvement suggestions.

[1336] The processing flow will be explained below.

[1337] Step 1: Receiving User Input

[1338] The user inputs behavioral record data through a communication interface (e.g., a chat app). For example, the user inputs "Study English vocabulary for 15 minutes" into the chat.

[1339] The terminal receives this input and prepares to transmit.

[1340] Step 2: Submitting input data

[1341] The terminal sends the received input data to the server, including the user ID, input text, and timestamp.

[1342] The protocol used is HTTP POST request and real-time communication protocol.

[1343] Step 3: Receiving the data

[1344] The server receives the input data, converts it into a format suitable for analysis, and passes it on to the next processing step.

[1345] Step 4: Analysis by generative AI

[1346] The server passes the received data to the generation AI for analysis.

[1347] Generative AI uses natural language processing technology to analyze input text. For example, if the input data is "Study English vocabulary for 15 minutes," it will classify it into the category of "study" and recognize the time as 15 minutes.

[1348] Step 5: Emotion determination by the emotion engine

[1349] The server passes the received data to the emotion engine for analysis.

[1350] The emotion engine analyzes the input text and determines the user's emotions. For example, if the input data is "15 minutes of English vocabulary study," it will determine emotions such as "motivation" or "joy."

[1351] Step 6: Receiving categorization and emotion determination results

[1352] The server receives the analysis results from the generation AI and emotion engine. The analysis results are received as structured data, including category (e.g., studying), activity (e.g., studying English vocabulary), time (e.g., 15 minutes), and emotion (e.g., joy).

[1353] Step 7: Saving to the Database

[1354] The server stores the analysis and emotion determination results in a database, including the user ID, category, activity, time, emotion, and timestamp.

[1355] Check whether saving was successful and perform error handling.

[1356] Step 8: Evaluate your progress

[1357] The server evaluates the user's progress based on the data stored in the database, aggregating past data and calculating the duration and number of activities within a specific period.

[1358] For example, it generates an evaluation result such as "Your total study time this week is 3 hours and 15 minutes."

[1359] Step 9: Generate emotional feedback

[1360] The server generates feedback messages based on the evaluation results, and adjusts the content according to the user's emotions.

[1361] For example, if the user is judged to be "happy," feedback such as "Great! Let's keep up the good work to achieve our goal" is generated.

[1362] Step 10: Submit your feedback

[1363] The server sends a feedback message to the device, such as a notification that includes "Your total study time this week is 3 hours and 15 minutes. Great!"

[1364] The device receives this feedback message and displays it to the user, either as a push notification or a chat message.

[1365] Step 11: Propose improvements

[1366] The server analyzes past data stored in the database to understand the user's behavioral patterns and emotional trends.

[1367] Based on the results, the system generates effective improvement suggestions for the user. For example, it creates advice such as, "If you study for 30 minutes every day, you will have two weeks to reach your goal. It would also be effective to take breaks to relax."

[1368] Step 12: Submit your improvement proposal

[1369] The server sends the generated improvement proposal to the terminal, and sends a message containing useful advice to the user.

[1370] The device receives the improvement suggestion message and displays it to the user. Notification methods include push notifications and chat messages.

[1371] In this way, the behavioral record data and emotional information entered by the user are managed and supported in an integrated manner through the steps of analysis, classification, storage, feedback, and improvement suggestions.

[1372] Example 2

[1373] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1374] Conventional systems do not adequately analyze the behavioral record data entered by users through a communication interface or provide feedback on that data. Furthermore, they are also inadequate in providing appropriate feedback and improvement suggestions based on the user's emotions, making it difficult to maintain the user's motivation or provide appropriate improvement measures. To solve these issues, it is necessary to provide a system that comprehensively analyzes and manages the user's behavioral record data and emotions, and provides appropriate feedback and improvement suggestions.

[1375] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for receiving behavior record data input by a user through a communication interface; generative AI model means for analyzing the behavior record data and classifying the data into appropriate categories based on the analysis results; emotion engine means for determining the user's emotions from the data analyzed by the generative AI model means; means for storing the categorized data and the emotion determination results in a database; means for evaluating the user's progress based on the stored data and generating feedback according to the evaluation results and the emotions; and means for analyzing the stored data and providing the user with improvement suggestions. This makes it possible to analyze and manage the user's behavior record data and emotion information in an integrated manner and provide appropriate feedback and improvement suggestions.

[1376] A "user" is an individual or entity that inputs behavioral data through a communications interface.

[1377] A "communication interface" is software or an application that allows a user to input behavior record data and communicate with a terminal.

[1378] "Behavior record data" is data relating to activities and behaviors that a user inputs through a communication interface, and includes, for example, information relating to study and training.

[1379] A "terminal" is an electronic device used to receive user input and send it to a server, and includes smartphones, tablets, personal computers, etc.

[1380] A "server" is a central processing unit that receives data sent from a terminal, analyzes and stores it, and generates feedback.

[1381] A "generative AI model" is an artificial intelligence method for analyzing behavioral record data and classifying it into appropriate categories.

[1382] An "emotion engine" is a technology that determines a user's emotions from data analyzed by a generative AI model.

[1383] A "category" is the type of behavioral record data classified by the generative AI model, such as "study" or "training."

[1384] A "database" is a recording medium for storing data handled by the system, such as analysis results and emotion determination results.

[1385] "Progress" refers to the progress of an activity evaluated based on the user's behavior record data.

[1386] "Feedback" refers to response messages and advice provided based on the user's progress and emotions.

[1387] "Improvement proposals" are suggestions for improving user behavior that are generated by analyzing past data stored in the database.

[1388] This invention is a system that analyzes and classifies behavioral record data entered by a user through a communication interface, recognizes the user's emotions, and provides feedback and suggestions for improvement. A specific embodiment of this system is described below.

[1389] Hardware and software used

[1390] Hardware

[1391] Devices: smartphones, tablets, computers, etc.

[1392] Server: Central processing unit (cloud server, on-premise server)

[1393] software

[1394] Communication Interface: Chat App, Web Application

[1395] Generative AI models: AI models that use natural language processing techniques (e.g., GPT series)

[1396] Sentiment engine: Software that performs sentiment analysis (e.g., sentiment analysis library)

[1397] Database: Data storage and management system (e.g., MySQL, PostgreSQL)

[1398] Explanation of program processing

[1399] 1. Receiving user input

[1400] The user enters the activity record of study, training, etc. into the chat app. For example, the user enters "Studying English vocabulary for 15 minutes."

[1401] The terminal receives this input and prepares to send it. The received data includes the user ID, the input text, and a timestamp.

[1402] 2. Sending input data

[1403] The device sends the received data to the server, where it is transferred over the network using an API.

[1404] 3. Data Receipt and Initial Analysis

[1405] The server receives the input data and converts it into a format for parsing. The data is parsed in JSON format.

[1406] 4. Analysis by generative AI and emotion engine

[1407] The server passes the received data to the generative AI model and emotion engine for analysis. The generative AI model analyzes the input text and classifies it into categories such as "study" and "training."

[1408] For example, data entered as "15 minutes of studying English vocabulary" will be classified into the category "study" and the time will be recognized as 15 minutes.

[1409] The emotion engine recognizes emotions from the input text and determines the user's emotions as "joy," "sadness," "excitement," "stress," etc.

[1410] 5. Receiving analysis results

[1411] The server receives the analysis results from the generative AI model and the emotion engine, which include structured data such as category, activity, time, and emotion.

[1412] 6. Saving to the database

[1413] The server stores the analysis and emotion determination results in a database, including the user ID, category, activity, time, emotion, and timestamp.

[1414] 7. Evaluating progress and generating feedback

[1415] The server evaluates the user's progress based on the data in the database, for example, by aggregating past data and calculating the duration and number of activities within a specific period.

[1416] Generate feedback messages based on the user's emotions, such as "You've studied a total of 3 hours and 15 minutes this week. Great!"

[1417] 8. Submitting Feedback

[1418] The server generates a feedback message and sends it to the device, which receives it and displays it to the user, either as a push notification or a chat message.

[1419] 9. Proposal for improvement

[1420] The server analyzes past data stored in the database to understand the user's behavioral patterns and emotional trends.

[1421] Based on the results, the system generates effective improvement suggestions for the user. For example, it creates advice such as, "If you study for 30 minutes every day, you will be two weeks away from achieving your goal. It would also be effective to take breaks to relax."

[1422] 10. Submitting Improvement Suggestions

[1423] The server sends the generated improvement proposal to the device. The device receives the improvement proposal message and displays it to the user. Notification methods include push notifications and chat messages.

[1424] Examples and prompts

[1425] Specific examples

[1426] User: "Running for 30 minutes"

[1427] Terminal: Receives this input and sends it to the server.

[1428] Server: Analyzes the received data, classifies it into the "Training" category, and determines the emotion as "Excitement."

[1429] Server: Stores the data in a database, evaluates progress, and sends feedback to the user, such as "You've trained a total of 2 hours this week. Great!"

[1430] Server: Send the following improvement suggestion: "Your health will improve even more if you increase your daily training by 15 minutes."

[1431] Prompt Sentence Examples

[1432] "The user entered 'Study English vocabulary for 20 minutes.' Please analyze this data."

[1433] "The user's emotion was determined to be sadness. Please generate appropriate feedback."

[1434] "Analyze user behavior patterns based on data from the past three months and propose improvements."

[1435] The above is a specific example of how to implement the invention. By using this system, it is possible to manage a user's behavioral record data and emotional information in an integrated manner, and provide appropriate feedback and suggestions for improvement.

[1436] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1437] Program processing steps

[1438] Step 1: Receiving User Input

[1439] A user enters behavioral data into a chat app. For example, the user enters "Study English vocabulary for 15 minutes."

[1440] The terminal receives this input and generates data containing the user ID, the input text, and a timestamp.

[1441] The input is "Study English vocabulary for 15 minutes" and the output is the user ID, the input text, and a timestamp.

[1442] Step 2: Submitting input data

[1443] The device sends the received data to the server, and then sends the data via the network using an API.

[1444] The inputs are the user ID, the input text, and a timestamp, and the output is the data sent to the server.

[1445] Step 3: Data reception and initial analysis

[1446] The server receives the input data and converts it into a format for analysis.

[1447] The input is a user ID, input text, and a timestamp, and the output is JSON format data. As a concrete example, the received data is mapped to key-value pairs.

[1448] Step 4: Analysis by generative AI and emotion engine

[1449] The server passes the data to the generative AI model and emotion engine to begin analysis.

[1450] The generative AI model analyzes the input text and classifies it into categories such as "study," "training," etc. For example, the input data "15 minutes of English vocabulary study" will be classified as "study" and the time will be recognized as 15 minutes.

[1451] The emotion engine identifies emotions from the input text, determining, for example, "joy."

[1452] The input is JSON formatted data and the output is data on category, activity, time, and emotion.

[1453] Step 5: Receive the analysis results

[1454] The server receives the analysis results from the generative AI model and the emotion engine.

[1455] The input is the analysis results of the generative AI model and emotion engine, and the output is structured data on category (e.g., studying), activity (e.g., studying English vocabulary), time (e.g., 15 minutes), and emotion (e.g., joy).

[1456] Step 6: Saving to the Database

[1457] The server stores the analysis and emotion determination results in a database.

[1458] The inputs are category, activity, time, emotion, user ID, and timestamp, and the output is a record stored in a database.

[1459] Step 7: Assess progress and generate feedback

[1460] The server evaluates the user's progress based on the stored data, for example by aggregating past data and calculating the duration and number of activities within a specific period.

[1461] Generate feedback messages based on the user's emotions, such as "Your total study time this week is 3 hours and 15 minutes. Great!"

[1462] The input is historical data retrieved from the database, and the output is progress assessments and feedback messages.

[1463] Step 8: Submit your feedback

[1464] The server sends the generated feedback message to the terminal, which receives the feedback message and displays it to the user.

[1465] The input is the feedback message and the output is the feedback notification displayed on the terminal.

[1466] Step 9: Propose improvements

[1467] The server analyzes past data stored in the database to understand the user's behavioral patterns and emotional trends.

[1468] Based on the results, the system generates effective improvement suggestions for the user. For example, it creates advice such as, "If you study for 30 minutes every day, you will be two weeks away from achieving your goal. It would also be effective to take breaks to relax."

[1469] The input is past data and the output is a message of proposed improvements.

[1470] Step 10: Submit your improvement proposal

[1471] The server sends the generated improvement proposal to the terminal, which receives the improvement proposal message and displays it to the user.

[1472] The input is the improvement proposal message, and the output is the improvement proposal notification displayed on the terminal.

[1473] (Application example 2)

[1474] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1475] Conventional advertising display systems typically provide uniform advertisements based on a user's behavioral history, but this often results in advertisements that ignore the user's emotional state, which can reduce user engagement. Furthermore, because advertisements are not optimized in response to emotional fluctuations, the user experience is not consistently high quality. The present invention aims to solve these problems by proposing a system that provides appropriate advertisements based on a user's behavioral history and emotional state.

[1476] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving behavior record data input by a user through a communication interface, generation AI means for analyzing the behavior record data and classifying the data into appropriate categories based on the analysis results, means for saving the data classified into the categories in a database, means for evaluating the user's progress based on the saved data and feeding back the evaluation results, means for analyzing the saved data and providing the user with improvement suggestions, emotion analysis means for analyzing the user's emotions, and means for suggesting advertisements based on the emotion analysis results. This makes it possible to display personalized advertisements according to the user's emotional state.

[1477] "User" means any individual or organization that uses this system.

[1478] A "communication interface" is a communication means such as a chat application or web form that allows a user to input behavior record data.

[1479] "Behavior record data" refers to data entered in text format by a user about their daily activities and work.

[1480] "Generative AI means" is an artificial intelligence technology that analyzes input behavioral record data and classifies its contents into appropriate categories.

[1481] "Category" refers to the classification of activity record data by type, such as "study," "training," and "relaxation."

[1482] The "database" is a data storage system for storing analyzed behavioral record data and emotional information.

[1483] "Means for assessing progress" refers to means for assessing a user's efforts and achievements based on stored data.

[1484] "Feedback" refers to messages and notifications that communicate progress assessment results to the user.

[1485] The "means for providing improvement suggestions" is a means for analyzing stored data and making suggestions to improve user behavior.

[1486] "Emotion analysis means" is a technology for identifying and analyzing emotions from user behavior record data.

[1487] The "means for suggesting advertisements" refers to a means for selecting and presenting the most suitable advertisement to the user based on emotion analysis and behavioral record data.

[1488] The system according to the present invention analyzes the behavioral record data entered by the user and personalizes advertisements based on the analysis results. The program processing of this system will be described in detail below.

[1489] First, a user uses a communication interface such as a chat application to input information about their daily activities and tasks. For example, they input text such as, "I had a great lunch with my friends today." This input data is received by the user's device and sent to the server.

[1490] The server analyzes the received data using a generation AI and a sentiment analysis engine. The generation AI uses natural language processing technology to analyze the input text and classify it into an appropriate category. For example, the activity "lunch" is classified as "relaxation." The sentiment analysis engine determines the emotion from the user's input text. For example, the emotion "fun" is determined.

[1491] The analysis results are passed to the server as structured data, including categories, activity details, time, and emotions. This data is stored in a database. The stored data is used to evaluate the user's progress. Based on the evaluation results, feedback is generated according to the user's emotions. For example, feedback such as "You had a great day today!" is sent.

[1492] Furthermore, the server will suggest advertisements based on the results of the sentiment analysis. For example, if the user is judged to be "happy," advertisements for relaxation-related products and services will be suggested. The advertisements will be sent to the user as push notifications or chat messages.

[1493] Hardware and software used:

[1494] User device: smartphone, tablet, etc.

[1495] Communication interfaces: chat applications, web forms

[1496] Server: Cloud server, on-premise server, etc.

[1497] Generative AI: AI models using natural language processing techniques (e.g., TextBlob)

[1498] Sentiment analysis engine: An engine that uses emotion recognition technology

[1499] Database: Data storage system (e.g. RDBMS, NoSQL)

[1500] Notification System: Push Notification Library

[1501] Examples and prompts:

[1502] If a user types, "I had a fun lunch with a friend today," the system analyzes this data and determines the category "relaxed" and the emotion "fun," then sends appropriate feedback and advertisements to the user.

[1503] Example prompt sentence:

[1504] "I had a great lunch with a friend today. I want to determine the sentiment and suggest an appropriate ad."

[1505] In this way, users receive feedback and advertising based on their behavior and emotions, resulting in a more consistent and high-quality experience.

[1506] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1507] Step 1:

[1508] A user inputs behavior record data using a communication interface. For example, the user inputs "I had a great lunch with my friends today" into a chat application. This input data is received by the user's terminal.

[1509] Step 2:

[1510] The terminal sends the received input data, including the user ID, text content, and timestamp, to the server. The server receives this data and passes it on to the next processing step.

[1511] Step 3:

[1512] The server converts the incoming data into a format for analysis and passes it to the generative AI and sentiment analysis engine, for example by tokenizing the input text and preprocessing it for contextual analysis.

[1513] Step 4:

[1514] The generative AI uses natural language processing techniques to analyze the input text and classify it into a category. For example, the activity "lunch" is classified as "relaxation." The sentiment analysis engine determines the emotion from the input text. For example, the emotion "fun" is determined.

[1515] Step 5:

[1516] The server receives the analysis results and formats them as structured data, including category, activity, emotion, and time. For example, the data could be that the category is "relaxing," the emotion is "fun," the activity is "having lunch with a friend," and the time is "1 hour."

[1517] Step 6:

[1518] The server saves the formatted data to the database, including the user ID, category, activity, emotion, time, and timestamp. It checks whether the save was successful and handles any errors.

[1519] Step 7:

[1520] The server evaluates the user's progress based on the stored data. For example, it may compile the time spent on relaxation activities over the past week and store this data as progress data. This data is used to evaluate the user's progress.

[1521] Step 8:

[1522] Generates feedback messages based on the results of sentiment analysis. For example, if the user is judged to be "happy," the system generates feedback such as "You had a great day today!"

[1523] Step 9:

[1524] The server sends a feedback message to the user's device, which receives the message and displays it to the user as a push notification or chat message.

[1525] Step 10:

[1526] The server proposes advertisements based on the results of the emotion analysis. For example, if the emotion is determined to be "fun," the server selects advertisements for products related to relaxation and proposes them to the user.

[1527] Step 11:

[1528] The server sends the selected advertisement to the user's device, which receives it and displays it to the user. The advertisement is displayed as a push notification or chat message.

[1529] Through the above process, personalized advertisements are provided based on the user's behavioral record data and emotional state.

[1530] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1531] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1532] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1533] [Fourth embodiment]

[1534] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1535] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1536] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1537] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1538] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1539] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1540] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1541] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1542] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1543] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1544] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1545] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1546] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1547] This invention is a system that allows users to input behavioral record data through a communication interface they use on a daily basis, and then analyzes, classifies, saves, provides feedback, and suggests improvements to that data. Below, we will explain in detail the program processing of this system in natural language, and provide examples.

[1548] Overall system overview

[1549] When a user enters behavioral record data using a communication interface such as a chat app, the device sends the data to a server. The server analyzes the received data using generative AI and classifies it into appropriate categories. The classified data is then stored in a database, and the user's progress is evaluated. Feedback is provided to the user along with the evaluation results, and improvement plans are also proposed based on past data.

[1550] Program processing overview

[1551] 1. Receiving user input

[1552] Users enter activity records such as study and training into a chat app.

[1553] For example, a user types "15 minutes of English vocabulary study" into the chat.

[1554] 2. Sending input data

[1555] The device sends the data entered from the chat app to the server.

[1556] This data also includes the user ID and timestamp.

[1557] 3. Data Analysis and Categorization

[1558] The server passes the received data to the generation AI for analysis.

[1559] The generative AI uses natural language processing technology to analyze the data and classify it into categories such as "study" and "training."

[1560] As a concrete example, "15 minutes of studying English vocabulary" is classified into the category "study" and is recognized as a time period of 15 minutes.

[1561] 4. Saving to the database

[1562] The server stores the analysis results in a database.

[1563] The information stored includes details such as user ID, category, activity, time, and timestamp.

[1564] 5. Progress Feedback

[1565] The server aggregates the user's past data and calculates their progress.

[1566] For example, if the total study time this week is 3 hours and 15 minutes, that information is notified to the user as feedback.

[1567] 6. Proposal for improvement

[1568] The server analyzes past data stored in the database to understand user behavior patterns.

[1569] Based on the results, effective improvement proposals are generated and provided to the user.

[1570] For example, advice like, "If you study for 30 minutes every day, you're two weeks away from achieving your goal."

[1571] Specific examples

[1572] 1. User Input

[1573] A user types "3 sets of 10 bench presses at the gym" into a chat app.

[1574] 2. Transmission and Analysis

[1575] The terminal sends user input to the server.

[1576] The server uses a generative AI to analyze the data of "Bench press at the gym, 10 repetitions, 3 sets" and classifies it into the category of "Training." The content is recognized as "Bench press," the number of repetitions is recognized as "10 repetitions," and the number of sets is recognized as "3 sets."

[1577] 3. Database storage

[1578] The server stores the analysis results in a database, for example, in the format {User ID: 12345, Category: Training, Content: Bench Press, Repetitions: 10, Sets: 3, Time: [Automatically estimated time], Timestamp: [Recorded time]}.

[1579] 4. Feedback

[1580] The server provides feedback to the user saying, "Today's training sessions have been recorded. The total number of sets is three."

[1581] 5. Proposal for improvement

[1582] The server suggests to the user an improvement suggestion such as, "To increase the number of bench presses, it would be effective to shorten the rest time a little."

[1583] In this way, users can efficiently record their actions and receive feedback on their progress and suggestions for improvement through a single communication interface.

[1584] The processing flow will be explained below.

[1585] Step 1: Receiving User Input

[1586] The user inputs behavioral record data through a communication interface (e.g., a chat app). For example, the user inputs "Study English vocabulary for 15 minutes" into the chat.

[1587] The terminal receives this input and prepares to transmit.

[1588] Step 2: Submitting input data

[1589] The terminal sends the received input data to the server, including the user ID, input text, and timestamp.

[1590] The protocol used is HTTP POST request and real-time communication protocol.

[1591] Step 3: Receiving the data

[1592] The server receives the input data, converts it into a format suitable for analysis, and passes it on to the next processing step.

[1593] Step 4: Analysis by generative AI

[1594] The server passes the received data to the generation AI for analysis.

[1595] Generative AI uses natural language processing technology to analyze input text. For example, if the input data is "Study English vocabulary for 15 minutes," it will classify it into the category of "study" and recognize the time as 15 minutes.

[1596] Step 5: Receiving categorization results

[1597] The server receives the analysis results (category classification results) from the generation AI.

[1598] The analysis results are received as structured data, such as category (e.g., studying), activity (e.g., studying English vocabulary), and time (e.g., 15 minutes).

[1599] Step 6: Saving to the Database

[1600] The server stores the analysis results in a database, including the user ID, category, activity, time, and timestamp.

[1601] Check whether saving was successful and perform error handling.

[1602] Step 7: Assess your progress

[1603] The server evaluates the user's progress based on the data stored in the database, aggregating past data and calculating the duration and number of activities within a specific period.

[1604] For example, it generates an evaluation result such as "Your total study time this week is 3 hours and 15 minutes."

[1605] Step 8: Generate feedback

[1606] The server generates a feedback message based on the evaluation results.

[1607] Prepare the generated feedback message for sending to the user.

[1608] Step 9: Submit your feedback

[1609] The server sends a feedback message to the device, such as a notification that includes a message like "Your total study time this week is 3 hours and 15 minutes."

[1610] The device receives this feedback message and displays it to the user, either as a push notification or a chat message.

[1611] Step 10: Propose improvements

[1612] The server analyzes past data stored in the database to understand user behavior patterns and trends.

[1613] Based on the results, it generates effective improvement suggestions for the user, such as "If you study for 30 minutes every day, you'll be two weeks away from achieving your goal."

[1614] Step 11: Submit your improvement proposal

[1615] The server sends the generated improvement proposal to the terminal, and sends a message containing useful advice to the user.

[1616] The device receives the improvement suggestion message and displays it to the user. Notification methods include push notifications and chat messages.

[1617] In this way, the behavioral record data entered by the user is managed and supported in an integrated manner, going through the steps of analysis, classification, storage, feedback, and improvement suggestions.

[1618] Example 1

[1619] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1620] Conventional behavioral recording systems make it difficult for users to efficiently record their daily activities and appropriately evaluate their progress. Furthermore, they do not provide sufficient feedback or suggestions for improvement, making it difficult to promote behavioral improvement. This makes it difficult for users to understand their own behavioral patterns and take effective measures to improve.

[1621] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1622] In this invention, the server includes means for receiving behavior record data entered by a user through a communication interface, artificial intelligence (AI) means for analyzing the behavior record data and classifying it into appropriate categories based on the analysis results, means for saving the categorized data in a database, means for evaluating the user's progress based on the saved data and providing feedback on the evaluation results, means for analyzing the saved data and providing the user with improvement suggestions, means for assigning a user ID and a timestamp to the behavior record data, means for generating prompt sentences for analysis by the AI, and means for aggregating past data and evaluating behavior records over a certain period of time. This allows users to efficiently record their own behavior, making it easier to understand their progress, and also enabling them to receive specific improvement suggestions.

[1623] A "communications interface" is the means by which a user inputs data and interacts with a system.

[1624] "Behavior record data" is data that includes specific information about the user's behavior, such as studying or training.

[1625] A "generative AI means" is a means that uses generative artificial intelligence technology to analyze input data and classify it into appropriate categories.

[1626] A "database" is a system for systematically storing analyzed data and enabling quick retrieval of necessary information.

[1627] The "means for evaluating progress" is a means for measuring and evaluating the progress of a user's activities based on the data stored in the database.

[1628] The "feedback means" is a means for notifying the user of the progress evaluation results and providing useful information regarding the user's actions.

[1629] The "means for providing improvement proposals" is a means for analyzing past data stored in a database and generating and providing effective improvement proposals to users.

[1630] A "user ID" is an identifier that uniquely identifies each user.

[1631] The "timestamp" indicates the date and time when the behavior record data was input.

[1632] A "prompt sentence" is an instruction sentence used by the generative artificial intelligence when analyzing data.

[1633] The "means for aggregating past data" is a means for generating statistical information based on behavioral records within a certain period of time and understanding the activity patterns of users.

[1634] This invention is a system in which a user inputs behavioral record data through a communication interface, and the data is analyzed using a generating AI means, and then classified, saved, evaluated, and provided feedback. The specific implementation method of this system is described below.

[1635] Overall overview

[1636] The user uses a device such as a smartphone or PC to input behavioral record data through a communication interface such as a chat app. The device receives the user's input data, assigns a user ID and timestamp to it, and sends it to a server. The server analyzes the received data using generative AI and classifies it into appropriate categories. The server then stores the classified data in a database and evaluates the user's progress. The evaluation results are fed back to the user, and further improvement suggestions are provided based on the data.

[1637] Hardware and software used

[1638] Hardware:

[1639] Devices such as smartphones and computers.

[1640] Server equipment that operates the server.

[1641] software:

[1642] Communication interfaces such as chat apps.

[1643] A database system (e.g., MySQL, PostgreSQL) for storing and managing data.

[1644] APIs and libraries for using generative AI models (e.g., GPT-4).

[1645] Details of data processing and calculation

[1646] 1. Receiving user input:

[1647] A user enters an action record such as "Study English vocabulary for 15 minutes" into a chat app.

[1648] 2. Send input data:

[1649] The terminal sends the input data to the server along with the user ID and a timestamp.

[1650] Sending format example:

[1651] {

[1652] User ID: 12345,

[1653] Record: "15 minutes of English vocabulary study",

[1654] Timestamp: "2023-10-10 10:00:00"

[1655] }

[1656] 3. Data Analysis and Categorization:

[1657] The server passes the received data to the generation AI, which generates a prompt for analysis.

[1658] An example of a generated prompt is: "The user entered 'Study English vocabulary for 15 minutes.' Please analyze this data to extract categories and detailed information."

[1659] The generating AI categorizes it as "study" and recognizes the time as "15 minutes."

[1660] 4. Save to database:

[1661] The server stores the analysis results in a database.

[1662] Examples of data that may be saved:

[1663] {

[1664] User ID: 12345,

[1665] Category: "Study",

[1666] Activity: "Study English vocabulary",

[1667] Time: "15 minutes",

[1668] Timestamp: "2023-10-10 10:00:00"

[1669] }

[1670] 5. Progress feedback:

[1671] The server aggregates data from the database over a period of time and evaluates the user's progress.

[1672] Example: If the total study time this week is 3 hours and 15 minutes, send the user a feedback message saying "Your total study time this week is 3 hours and 15 minutes."

[1673] 6. Providing suggestions for improvement:

[1674] The server sends a prompt to the generation AI, which generates improvement suggestions based on past data.

[1675] An example of a generated improvement suggestion: "If you study for 30 minutes every day, you're two weeks away from achieving your goal."

[1676] Specific examples

[1677] If a user types "Bench press at the gym, 10 reps, 3 sets" into a chat app, the device receives this, assigns a user ID and timestamp, and sends it to the server. The server then uses the generated AI to analyze the data, generating a prompt that reads, "The user entered 'Bench press at the gym, 10 reps, 3 sets'. Please analyze this data and extract the category and detailed information." The generated AI classifies this as a "Training" category, and recognizes that the content is "Bench press," the number of repetitions is "10," and the number of sets is "3 sets." These analysis results are stored in a database and later used to provide progress feedback and improvement suggestions.

[1678] In this way, the system allows users to efficiently log their actions and receive feedback on their progress and suggestions for improvement.

[1679] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1680] Step 1:

[1681] Receiving User Input

[1682] The user inputs behavior record data into the chat application.

[1683] Example: Enter "Study English vocabulary for 15 minutes."

[1684] The terminal receives user input in real time.

[1685] The terminal assigns a user ID and a timestamp to the received data.

[1686] Input: User behavior record data (e.g., "Study English vocabulary for 15 minutes").

[1687] Output: Data containing user ID, behavior record data, and timestamp (e.g., {User ID: 12345, Record: "Study English vocabulary for 15 minutes", Timestamp: "2023-10-10 10:00:00"}).

[1688] Step 2:

[1689] Sending input data

[1690] The terminal transmits the received behavior record data to the server.

[1691] The data is sent as an HTTP POST request.

[1692] Data to be sent: Data including user ID, behavioral record data, and timestamp.

[1693] Input: Data including user ID, behavior record data, and timestamp.

[1694] Output: Data sent to the server (e.g., {User ID: 12345, Record: "Study English vocabulary for 15 minutes", Timestamp: "2023-10-10 10:00:00"}).

[1695] Step 3:

[1696] Data analysis and categorization

[1697] The server takes the received data and generates a prompt for analysis.

[1698] The server sends a prompt to the generative AI model.

[1699] A generative AI model analyzes the data and classifies it into appropriate categories.

[1700] Example prompt: "The user entered 'Study English vocabulary for 15 minutes.' Please parse this data to extract categories and detailed information."

[1701] Input: Data including user ID, behavior record data, and timestamp.

[1702] Output: Analysis results (category: "Study", activity: "Study English vocabulary", time: "15 minutes").

[1703] Step 4:

[1704] Saving to a database

[1705] The server obtains the analysis results and stores them in a database.

[1706] Save format: {User ID: 12345, Category: "Study", Activity: "Study English vocabulary", Time: "15 minutes", Timestamp: "2023-10-10 10:00:00"}

[1707] Input: Analysis results (category: "Study", activity: "Study English vocabulary", time: "15 minutes").

[1708] Output: Data stored in the database.

[1709] Step 5:

[1710] Progress feedback

[1711] The server compiles user behavior records for a specified period from the database.

[1712] Example: Calculate the total time spent studying this week and arrive at "3 hours and 15 minutes."

[1713] The server generates progress feedback based on the aggregated results and sends it to the user via a chat app.

[1714] Example feedback: "Your total study time this week is 3 hours and 15 minutes."

[1715] Input: Data stored in a database.

[1716] Output: Progress feedback sent to the user.

[1717] Step 6:

[1718] Providing improvement suggestions

[1719] Enter a prompt statement that the server will use to pass past data stored in the database to the generative AI model.

[1720] The server generates a prompt sentence and sends it to the generation AI, which then generates an improvement proposal.

[1721] Example of an improvement idea: "If I study 30 minutes each day, I'll be two weeks away from achieving my goal."

[1722] The server sends the generated improvement proposal to the user.

[1723] Input: Data stored in the database, prompts for analysis.

[1724] Output: The improvement suggestions sent to the user.

[1725] (Application example 1)

[1726] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1727] Improving driving skills and maintaining safe driving are becoming increasingly important with the spread of autonomous vehicles. However, conventional driver training and feedback systems lack real-time capabilities and individual optimization, making it difficult to quickly respond to the individual needs of each driver. Furthermore, there is a lack of a system that allows drivers to easily and intuitively input their driving records and receive instant feedback. Given this background, there is a need for a system that provides effective feedback and improvement suggestions to improve driving skills.

[1728] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1729] In this invention, the server includes: means for receiving behavior record data entered by the user through a communication interface; a generating AI means for analyzing the behavior record data and classifying it into appropriate categories based on the analysis results; means for storing the categorized data in a database; means for evaluating the user's progress based on the stored data and providing feedback on the evaluation results; means for analyzing the stored data and providing the user with improvement suggestions; means for inputting the user's behavior record data through a head-mounted display; and means for providing feedback and improvement suggestions for improving driving skills. This allows drivers to easily enter their own driving records in real time and receive immediate feedback and specific improvement suggestions based on the input. This makes it possible to maintain safe driving and improve driving skills simultaneously.

[1730] A "communication interface" is an interactive device for a user to input behavior record data, and is a device having means for transmitting and receiving data.

[1731] "Behavior record data" is information in which a user records daily behavior, exercise, work content, and the like.

[1732] "Generative AI" is a technology that uses artificial intelligence to analyze incoming data and classify it into appropriate categories.

[1733] A "database" is an information collection system for efficiently storing and managing analyzed and classified data.

[1734] "User progress" is information indicating the degree of progress or achievement of a specific action or task performed by a user.

[1735] "Feedback" refers to information and advice provided to the User based on the analyzed and evaluated progress.

[1736] "Improvement Suggestions" are suggestions and advice to help users achieve better results.

[1737] A "head-mounted display" is a display device worn by a user on the head, and is a device for providing visual information.

[1738] "Feedback for improving driving skills" refers to evaluations and advice provided to users regarding specific operations and actions they perform while driving.

[1739] "Improvement suggestions for improving driving skills" are practical suggestions and advice provided to users to improve their driving skills.

[1740] This invention is a system that allows users to improve their driving skills in autonomous vehicles using a head-mounted display (HMD). The system inputs recorded user behavior data, analyzes and classifies it, and provides the driver with immediate feedback and suggestions for improvement. This section describes the detailed configuration and operation of the system.

[1741] Hardware and software used

[1742] Hardware:

[1743] Head-mounted display (HMD): Using Oculus Quest 2 as an example.

[1744] software:

[1745] Development environment: Unity3D

[1746] Communication API: WebSocket or HTTP REST API

[1747] Generation AI: OpenAI GPT-4

[1748] Database: Firebase Firestore

[1749] System configuration

[1750] The system consists of the following main components:

[1751] Receiving user input:

[1752] The user can input behavioral data through the HMD. For example, they can input "Today's driving was smooth."

[1753] Sending input data:

[1754] The HMD sends the input data, including the user ID, timestamp, and location information, to the server in real time.

[1755] Data analysis and classification:

[1756] The server passes the input data to the generative AI (GPT-4), which analyzes it using natural language processing technology and classifies it into an appropriate category. For example, "Lane change was delayed" is classified as "driving skill improvement area."

[1757] Save to database:

[1758] The server stores the parsed and categorized data in Firebase Firestore, where detailed information (user ID, category, content, timestamp, etc.) is stored.

[1759] Progress assessment and feedback:

[1760] The server aggregates past data and evaluates the user's progress, and the evaluation results are communicated to the user via the HMD as feedback.

[1761] Suggested improvements:

[1762] The server analyzes the stored data and uses AI to provide users with specific suggestions for improving their driving skills, such as "Keep a safe distance to reduce sudden braking."

[1763] Specific examples

[1764] 1. User Input:

[1765] The user inputs their observations while driving into the HMD's chat interface.

[1766] For example, enter "There were a lot of sudden braking today."

[1767] 2. Transmission and Analysis:

[1768] The HMD sends the input data to the server.

[1769] The server uses a generative AI (GPT-4) to analyze the data that shows "frequent sudden braking" and classifies it as "an area for improvement in driving technique."

[1770] 3. Database storage:

[1771] The server saves the analysis results in a database (Firebase Firestore).

[1772] 4. Feedback:

[1773] The server provides feedback such as, "You braked suddenly five times today. Next time, try to maintain a safe distance while driving."

[1774] 5. Suggestions for improvement:

[1775] An improvement suggestion was displayed on the HMD: "To reduce sudden braking, it is safer to try to maintain a constant distance between vehicles."

[1776] Example prompts for generative AI models

[1777] Below are some example prompts that can be used to properly analyze and categorize user input data and provide specific feedback and suggestions for improvement:

[1778] Analyze the user's driving record data and classify it into appropriate categories. Then, provide advice to improve the user's driving skills based on that data. For example, if the input is "I braked a lot today," classify it as "Something to improve in driving skills" and output the advice "Keep a safe distance."

[1779] These prompts allow generative AI models (such as GPT-4) to efficiently analyze user input data and provide appropriate feedback and suggestions for improvement.

[1780] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1781] Step 1:

[1782] The user wears the HMD and inputs behavioral record data through the chat interface. For example, the user inputs "I was late changing lanes." This input data includes the user ID and timestamp.

[1783] Input: Activity record data, user ID, timestamp

[1784] Output: Formatted behavioral record data

[1785] Step 2:

[1786] The device (HMD) sends the behavioral record data entered by the user to the server in real time using WebSocket and HTTP REST API.

[1787] Input: Formatted behavioral record data

[1788] Output: Activity record data sent to the server

[1789] Step 3:

[1790] The server passes the received behavioral record data to a generative AI model (GPT-4) for analysis. A prompt sentence is used during this process. The generative AI model uses natural language processing technology to analyze the input data and classify it into appropriate categories. For example, "Lane change was delayed" is classified as "Areas for improvement in driving technique."

[1791] Input: Activity record data sent to the server

[1792] Output: Classified data (with categories)

[1793] Step 4:

[1794] The server stores the analyzed data in a database (Firebase Firestore), including information such as user ID, category, content, and timestamp.

[1795] Input: Classified data (with categories)

[1796] Output: Data stored in the database

[1797] Step 5:

[1798] The server evaluates the user's progress based on the data stored in the database, and then provides the evaluation results to the user as real-time feedback. For example, the HMD displays, "You braked suddenly five times today."

[1799] Input: Data stored in the database

[1800] Output: Feedback information

[1801] Step 6:

[1802] The server analyzes past data stored in the database and generates and provides improvement suggestions to the user using a generative AI model. For example, an improvement suggestion such as "Maintain a safe distance to reduce sudden braking" is generated. This improvement suggestion is also displayed on the HMD.

[1803] Input: Data stored in database, historical data

[1804] Output: Improvement plan

[1805] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1806] This invention is a system that analyzes and classifies behavioral record data entered by the user through a communication interface, and also recognizes the user's emotions and provides feedback and suggestions for improvement. Below, we will explain in detail the program processing of this system in natural language, and provide examples.

[1807] Overall system overview

[1808] When a user enters behavioral record data using a communication interface such as a chat app, the device sends the data to a server. The server analyzes the received data using a generative AI and emotion engine, classifying it into appropriate categories and determining the user's emotions. The classified data and emotion information are then stored in a database, and the user's progress is evaluated. Along with the evaluation results, the user is provided with feedback based on their emotions, and improvement suggestions are also proposed based on past data.

[1809] Program processing overview

[1810] 1. Receiving user input

[1811] Users enter activity records such as study and training into a chat app.

[1812] For example, a user types "15 minutes of English vocabulary study" into the chat.

[1813] The terminal receives this input and prepares to transmit.

[1814] 2. Sending input data

[1815] The terminal sends the received input data to the server, including the user ID, input text, and timestamp.

[1816] 3. Data Receipt and Initial Analysis

[1817] The server receives the input data, converts it into a format for analysis, and passes it on to the next processing step.

[1818] 4. Analysis by generative AI and emotion engine

[1819] The server passes the received data to the generation AI and emotion engine for analysis.

[1820] Generative AI uses natural language processing technology to analyze input text and classify it into categories such as "study," "training," etc. For example, input data such as "15 minutes of English vocabulary study" will be classified as "study" and the time will be recognized as 15 minutes.

[1821] The emotion engine recognizes emotions from the user's input text, such as happiness, sadness, excitement, and stress.

[1822] 5. Receiving categorization and emotion determination results

[1823] The server receives the analysis results from the generation AI and emotion engine. The analysis results are received as structured data, including category (e.g., studying), activity (e.g., studying English vocabulary), time (e.g., 15 minutes), and emotion (e.g., joy).

[1824] 6. Saving to the database

[1825] The server stores the analysis and emotion determination results in a database, including the user ID, category, activity, time, emotion, and timestamp.

[1826] Check whether saving was successful and perform error handling.

[1827] 7. Evaluating progress and generating emotional feedback

[1828] The server evaluates the user's progress based on the data stored in the database, aggregating past data and calculating the duration and number of activities within a specific period.

[1829] A feedback message is generated according to the user's emotion. For example, if the user is judged to be "sad," encouraging feedback is generated.

[1830] 8. Submitting Feedback

[1831] The server sends a feedback message to the device, such as a notification that includes "Your total study time this week is 3 hours and 15 minutes. Great!"

[1832] The device receives this feedback message and displays it to the user, either as a push notification or a chat message.

[1833] 9. Proposal for improvement

[1834] The server analyzes past data stored in the database to understand the user's behavioral patterns and emotional trends.

[1835] Based on the results, the system generates effective improvement suggestions for the user. For example, it creates advice such as, "If you study for 30 minutes every day, you will have two weeks to reach your goal. It would also be effective to take breaks to relax."

[1836] 10. Submitting Improvement Suggestions

[1837] The server sends the generated improvement proposal to the terminal, and sends a message containing useful advice to the user.

[1838] The device receives the improvement suggestion message and displays it to the user. Notification methods include push notifications and chat messages.

[1839] In this way, the behavioral record data and emotional information entered by the user are managed and supported in an integrated manner through the steps of analysis, classification, storage, feedback, and improvement suggestions.

[1840] The processing flow will be explained below.

[1841] Step 1: Receiving User Input

[1842] The user inputs behavioral record data through a communication interface (e.g., a chat app). For example, the user inputs "Study English vocabulary for 15 minutes" into the chat.

[1843] The terminal receives this input and prepares to transmit.

[1844] Step 2: Submitting input data

[1845] The terminal sends the received input data to the server, including the user ID, input text, and timestamp.

[1846] The protocol used is HTTP POST request and real-time communication protocol.

[1847] Step 3: Receiving the data

[1848] The server receives the input data, converts it into a format suitable for analysis, and passes it on to the next processing step.

[1849] Step 4: Analysis by generative AI

[1850] The server passes the received data to the generation AI for analysis.

[1851] Generative AI uses natural language processing technology to analyze input text. For example, if the input data is "Study English vocabulary for 15 minutes," it will classify it into the category of "study" and recognize the time as 15 minutes.

[1852] Step 5: Emotion determination by the emotion engine

[1853] The server passes the received data to the emotion engine for analysis.

[1854] The emotion engine analyzes the input text and determines the user's emotions. For example, if the input data is "15 minutes of English vocabulary study," it will determine emotions such as "motivation" or "joy."

[1855] Step 6: Receiving categorization and emotion determination results

[1856] The server receives the analysis results from the generation AI and emotion engine. The analysis results are received as structured data, including category (e.g., studying), activity (e.g., studying English vocabulary), time (e.g., 15 minutes), and emotion (e.g., joy).

[1857] Step 7: Saving to the Database

[1858] The server stores the analysis and emotion determination results in a database, including the user ID, category, activity, time, emotion, and timestamp.

[1859] Check whether saving was successful and perform error handling.

[1860] Step 8: Evaluate your progress

[1861] The server evaluates the user's progress based on the data stored in the database, aggregating past data and calculating the duration and number of activities within a specific period.

[1862] For example, it generates an evaluation result such as "Your total study time this week is 3 hours and 15 minutes."

[1863] Step 9: Generate emotional feedback

[1864] The server generates feedback messages based on the evaluation results, and adjusts the content according to the user's emotions.

[1865] For example, if the user is judged to be "happy," feedback such as "Great! Let's keep up the good work to achieve our goal" is generated.

[1866] Step 10: Submit your feedback

[1867] The server sends a feedback message to the device, such as a notification that includes "Your total study time this week is 3 hours and 15 minutes. Great!"

[1868] The device receives this feedback message and displays it to the user, either as a push notification or a chat message.

[1869] Step 11: Propose improvements

[1870] The server analyzes past data stored in the database to understand the user's behavioral patterns and emotional trends.

[1871] Based on the results, the system generates effective improvement suggestions for the user. For example, it creates advice such as, "If you study for 30 minutes every day, you will have two weeks to reach your goal. It would also be effective to take breaks to relax."

[1872] Step 12: Submit your improvement proposal

[1873] The server sends the generated improvement proposal to the terminal, and sends a message containing useful advice to the user.

[1874] The device receives the improvement suggestion message and displays it to the user. Notification methods include push notifications and chat messages.

[1875] In this way, the behavioral record data and emotional information entered by the user are managed and supported in an integrated manner through the steps of analysis, classification, storage, feedback, and improvement suggestions.

[1876] Example 2

[1877] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1878] Conventional systems do not adequately analyze the behavioral record data entered by users through a communication interface or provide feedback on that data. Furthermore, they are also inadequate in providing appropriate feedback and improvement suggestions based on the user's emotions, making it difficult to maintain the user's motivation or provide appropriate improvement measures. To solve these issues, it is necessary to provide a system that comprehensively analyzes and manages the user's behavioral record data and emotions, and provides appropriate feedback and improvement suggestions.

[1879] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for receiving behavior record data input by a user through a communication interface; generative AI model means for analyzing the behavior record data and classifying the data into appropriate categories based on the analysis results; emotion engine means for determining the user's emotions from the data analyzed by the generative AI model means; means for storing the categorized data and the emotion determination results in a database; means for evaluating the user's progress based on the stored data and generating feedback according to the evaluation results and the emotions; and means for analyzing the stored data and providing the user with improvement suggestions. This makes it possible to analyze and manage the user's behavior record data and emotion information in an integrated manner and provide appropriate feedback and improvement suggestions.

[1880] A "user" is an individual or entity that inputs behavioral data through a communications interface.

[1881] A "communication interface" is software or an application that allows a user to input behavior record data and communicate with a terminal.

[1882] "Behavior record data" is data relating to activities and behaviors that a user inputs through a communication interface, and includes, for example, information relating to study and training.

[1883] A "terminal" is an electronic device used to receive user input and send it to a server, and includes smartphones, tablets, personal computers, etc.

[1884] A "server" is a central processing unit that receives data sent from a terminal, analyzes and stores it, and generates feedback.

[1885] A "generative AI model" is an artificial intelligence method for analyzing behavioral record data and classifying it into appropriate categories.

[1886] An "emotion engine" is a technology that determines a user's emotions from data analyzed by a generative AI model.

[1887] A "category" is the type of behavioral record data classified by the generative AI model, such as "study" or "training."

[1888] A "database" is a recording medium for storing data handled by the system, such as analysis results and emotion determination results.

[1889] "Progress" refers to the progress of an activity evaluated based on the user's behavior record data.

[1890] "Feedback" refers to response messages and advice provided based on the user's progress and emotions.

[1891] "Improvement proposals" are suggestions for improving user behavior that are generated by analyzing past data stored in the database.

[1892] This invention is a system that analyzes and classifies behavioral record data entered by a user through a communication interface, recognizes the user's emotions, and provides feedback and suggestions for improvement. A specific embodiment of this system is described below.

[1893] Hardware and software used

[1894] Hardware

[1895] Devices: smartphones, tablets, computers, etc.

[1896] Server: Central processing unit (cloud server, on-premise server)

[1897] software

[1898] Communication Interface: Chat App, Web Application

[1899] Generative AI models: AI models that use natural language processing techniques (e.g., GPT series)

[1900] Sentiment engine: Software that performs sentiment analysis (e.g., sentiment analysis library)

[1901] Database: Data storage and management system (e.g., MySQL, PostgreSQL)

[1902] Explanation of program processing

[1903] 1. Receiving user input

[1904] The user enters the activity record of study, training, etc. into the chat app. For example, the user enters "Studying English vocabulary for 15 minutes."

[1905] The terminal receives this input and prepares to send it. The received data includes the user ID, the input text, and a timestamp.

[1906] 2. Sending input data

[1907] The device sends the received data to the server, where it is transferred over the network using an API.

[1908] 3. Data Receipt and Initial Analysis

[1909] The server receives the input data and converts it into a format for parsing. The data is parsed in JSON format.

[1910] 4. Analysis by generative AI and emotion engine

[1911] The server passes the received data to the generative AI model and emotion engine for analysis. The generative AI model analyzes the input text and classifies it into categories such as "study" and "training."

[1912] For example, data entered as "15 minutes of studying English vocabulary" will be classified into the category "study" and the time will be recognized as 15 minutes.

[1913] The emotion engine recognizes emotions from the input text and determines the user's emotions as "joy," "sadness," "excitement," "stress," etc.

[1914] 5. Receiving analysis results

[1915] The server receives the analysis results from the generative AI model and the emotion engine, which include structured data such as category, activity, time, and emotion.

[1916] 6. Saving to the database

[1917] The server stores the analysis and emotion determination results in a database, including the user ID, category, activity, time, emotion, and timestamp.

[1918] 7. Evaluating progress and generating feedback

[1919] The server evaluates the user's progress based on the data in the database, for example, by aggregating past data and calculating the duration and number of activities within a specific period.

[1920] Generate feedback messages based on the user's emotions, such as "You've studied a total of 3 hours and 15 minutes this week. Great!"

[1921] 8. Submitting Feedback

[1922] The server generates a feedback message and sends it to the device, which receives it and displays it to the user, either as a push notification or a chat message.

[1923] 9. Proposal for improvement

[1924] The server analyzes past data stored in the database to understand the user's behavioral patterns and emotional trends.

[1925] Based on the results, the system generates effective improvement suggestions for the user. For example, it creates advice such as, "If you study for 30 minutes every day, you will be two weeks away from achieving your goal. It would also be effective to take breaks to relax."

[1926] 10. Submitting Improvement Suggestions

[1927] The server sends the generated improvement proposal to the device. The device receives the improvement proposal message and displays it to the user. Notification methods include push notifications and chat messages.

[1928] Examples and prompts

[1929] Specific examples

[1930] User: "Running for 30 minutes"

[1931] Terminal: Receives this input and sends it to the server.

[1932] Server: Analyzes the received data, classifies it into the "Training" category, and determines the emotion as "Excitement."

[1933] Server: Stores the data in a database, evaluates progress, and sends feedback to the user, such as "You've trained a total of 2 hours this week. Great!"

[1934] Server: Send the following improvement suggestion: "Your health will improve even more if you increase your daily training by 15 minutes."

[1935] Prompt Sentence Examples

[1936] "The user entered 'Study English vocabulary for 20 minutes.' Please analyze this data."

[1937] "The user's emotion was determined to be sadness. Please generate appropriate feedback."

[1938] "Analyze user behavior patterns based on data from the past three months and propose improvements."

[1939] The above is a specific example of how to implement the invention. By using this system, it is possible to manage a user's behavioral record data and emotional information in an integrated manner, and provide appropriate feedback and suggestions for improvement.

[1940] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1941] Program processing steps

[1942] Step 1: Receiving User Input

[1943] A user enters behavioral data into a chat app. For example, the user enters "Study English vocabulary for 15 minutes."

[1944] The terminal receives this input and generates data containing the user ID, the input text, and a timestamp.

[1945] The input is "Study English vocabulary for 15 minutes" and the output is the user ID, the input text, and a timestamp.

[1946] Step 2: Submitting input data

[1947] The device sends the received data to the server, and then sends the data via the network using an API.

[1948] The inputs are the user ID, the input text, and a timestamp, and the output is the data sent to the server.

[1949] Step 3: Data reception and initial analysis

[1950] The server receives the input data and converts it into a format for analysis.

[1951] The input is a user ID, input text, and a timestamp, and the output is JSON format data. As a concrete example, the received data is mapped to key-value pairs.

[1952] Step 4: Analysis by generative AI and emotion engine

[1953] The server passes the data to the generative AI model and emotion engine to begin analysis.

[1954] The generative AI model analyzes the input text and classifies it into categories such as "study," "training," etc. For example, the input data "15 minutes of English vocabulary study" will be classified as "study" and the time will be recognized as 15 minutes.

[1955] The emotion engine identifies emotions from the input text, determining, for example, "joy."

[1956] The input is JSON formatted data and the output is data on category, activity, time, and emotion.

[1957] Step 5: Receive the analysis results

[1958] The server receives the analysis results from the generative AI model and the emotion engine.

[1959] The input is the analysis results of the generative AI model and emotion engine, and the output is structured data on category (e.g., studying), activity (e.g., studying English vocabulary), time (e.g., 15 minutes), and emotion (e.g., joy).

[1960] Step 6: Saving to the Database

[1961] The server stores the analysis and emotion determination results in a database.

[1962] The inputs are category, activity, time, emotion, user ID, and timestamp, and the output is a record stored in a database.

[1963] Step 7: Assess progress and generate feedback

[1964] The server evaluates the user's progress based on the stored data, for example by aggregating past data and calculating the duration and number of activities within a specific period.

[1965] Generate feedback messages based on the user's emotions, such as "Your total study time this week is 3 hours and 15 minutes. Great!"

[1966] The input is historical data retrieved from the database, and the output is progress assessments and feedback messages.

[1967] Step 8: Submit your feedback

[1968] The server sends the generated feedback message to the terminal, which receives the feedback message and displays it to the user.

[1969] The input is the feedback message and the output is the feedback notification displayed on the terminal.

[1970] Step 9: Propose improvements

[1971] The server analyzes past data stored in the database to understand the user's behavioral patterns and emotional trends.

[1972] Based on the results, the system generates effective improvement suggestions for the user. For example, it creates advice such as, "If you study for 30 minutes every day, you will be two weeks away from achieving your goal. It would also be effective to take breaks to relax."

[1973] The input is past data and the output is a message of proposed improvements.

[1974] Step 10: Submit your improvement proposal

[1975] The server sends the generated improvement proposal to the terminal, which receives the improvement proposal message and displays it to the user.

[1976] The input is the improvement proposal message, and the output is the improvement proposal notification displayed on the terminal.

[1977] (Application example 2)

[1978] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1979] Conventional advertising display systems typically provide uniform advertisements based on a user's behavioral history, but this often results in advertisements that ignore the user's emotional state, which can reduce user engagement. Furthermore, because advertisements are not optimized in response to emotional fluctuations, the user experience is not consistently high quality. The present invention aims to solve these problems by proposing a system that provides appropriate advertisements based on a user's behavioral history and emotional state.

[1980] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving behavior record data input by a user through a communication interface, generation AI means for analyzing the behavior record data and classifying the data into appropriate categories based on the analysis results, means for saving the data classified into the categories in a database, means for evaluating the user's progress based on the saved data and feeding back the evaluation results, means for analyzing the saved data and providing the user with improvement suggestions, emotion analysis means for analyzing the user's emotions, and means for suggesting advertisements based on the emotion analysis results. This makes it possible to display personalized advertisements according to the user's emotional state.

[1981] "User" means any individual or organization that uses this system.

[1982] A "communication interface" is a communication means such as a chat application or web form that allows a user to input behavior record data.

[1983] "Behavior record data" refers to data entered in text format by a user about their daily activities and work.

[1984] "Generative AI means" is an artificial intelligence technology that analyzes input behavioral record data and classifies its contents into appropriate categories.

[1985] "Category" refers to the classification of activity record data by type, such as "study," "training," and "relaxation."

[1986] The "database" is a data storage system for storing analyzed behavioral record data and emotional information.

[1987] "Means for assessing progress" refers to means for assessing a user's efforts and achievements based on stored data.

[1988] "Feedback" refers to messages and notifications that communicate progress assessment results to the user.

[1989] The "means for providing improvement suggestions" is a means for analyzing stored data and making suggestions to improve user behavior.

[1990] "Emotion analysis means" is a technology for identifying and analyzing emotions from user behavior record data.

[1991] The "means for suggesting advertisements" refers to a means for selecting and presenting the most suitable advertisement to the user based on emotion analysis and behavioral record data.

[1992] The system according to the present invention analyzes the behavioral record data entered by the user and personalizes advertisements based on the analysis results. The program processing of this system will be described in detail below.

[1993] First, a user uses a communication interface such as a chat application to input information about their daily activities and tasks. For example, they input text such as, "I had a great lunch with my friends today." This input data is received by the user's device and sent to the server.

[1994] The server analyzes the received data using a generation AI and a sentiment analysis engine. The generation AI uses natural language processing technology to analyze the input text and classify it into an appropriate category. For example, the activity "lunch" is classified as "relaxation." The sentiment analysis engine determines the emotion from the user's input text. For example, the emotion "fun" is determined.

[1995] The analysis results are passed to the server as structured data, including categories, activity details, time, and emotions. This data is stored in a database. The stored data is used to evaluate the user's progress. Based on the evaluation results, feedback is generated according to the user's emotions. For example, feedback such as "You had a great day today!" is sent.

[1996] Furthermore, the server will suggest advertisements based on the results of the sentiment analysis. For example, if the user is judged to be "happy," advertisements for relaxation-related products and services will be suggested. The advertisements will be sent to the user as push notifications or chat messages.

[1997] Hardware and software used:

[1998] User device: smartphone, tablet, etc.

[1999] Communication interfaces: chat applications, web forms

[2000] Server: Cloud server, on-premise server, etc.

[2001] Generative AI: AI models using natural language processing techniques (e.g., TextBlob)

[2002] Sentiment analysis engine: An engine that uses emotion recognition technology

[2003] Database: Data storage system (e.g. RDBMS, NoSQL)

[2004] Notification System: Push Notification Library

[2005] Examples and prompts:

[2006] If a user types, "I had a fun lunch with a friend today," the system analyzes this data and determines the category "relaxed" and the emotion "fun," then sends appropriate feedback and advertisements to the user.

[2007] Example prompt sentence:

[2008] "I had a great lunch with a friend today. I want to determine the sentiment and suggest an appropriate ad."

[2009] In this way, users receive feedback and advertising based on their behavior and emotions, resulting in a more consistent and high-quality experience.

[2010] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2011] Step 1:

[2012] A user inputs behavior record data using a communication interface. For example, the user inputs "I had a great lunch with my friends today" into a chat application. This input data is received by the user's terminal.

[2013] Step 2:

[2014] The terminal sends the received input data, including the user ID, text content, and timestamp, to the server. The server receives this data and passes it on to the next processing step.

[2015] Step 3:

[2016] The server converts the incoming data into a format for analysis and passes it to the generative AI and sentiment analysis engine, for example by tokenizing the input text and preprocessing it for contextual analysis.

[2017] Step 4:

[2018] The generative AI uses natural language processing techniques to analyze the input text and classify it into a category. For example, the activity "lunch" is classified as "relaxation." The sentiment analysis engine determines the emotion from the input text. For example, the emotion "fun" is determined.

[2019] Step 5:

[2020] The server receives the analysis results and formats them as structured data, including category, activity, emotion, and time. For example, the data could be that the category is "relaxing," the emotion is "fun," the activity is "having lunch with a friend," and the time is "1 hour."

[2021] Step 6:

[2022] The server saves the formatted data to the database, including the user ID, category, activity, emotion, time, and timestamp. It checks whether the save was successful and handles any errors.

[2023] Step 7:

[2024] The server evaluates the user's progress based on the stored data. For example, it may compile the time spent on relaxation activities over the past week and store this data as progress data. This data is used to evaluate the user's progress.

[2025] Step 8:

[2026] Generates feedback messages based on the results of sentiment analysis. For example, if the user is judged to be "happy," the system generates feedback such as "You had a great day today!"

[2027] Step 9:

[2028] The server sends a feedback message to the user's device, which receives the message and displays it to the user as a push notification or chat message.

[2029] Step 10:

[2030] The server proposes advertisements based on the results of the emotion analysis. For example, if the emotion is determined to be "fun," the server selects advertisements for products related to relaxation and proposes them to the user.

[2031] Step 11:

[2032] The server sends the selected advertisement to the user's device, which receives it and displays it to the user. The advertisement is displayed as a push notification or chat message.

[2033] Through the above process, personalized advertisements are provided based on the user's behavioral record data and emotional state.

[2034] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2035] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2036] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2037] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2038] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2039] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2040] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2041] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2042] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2043] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2044] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2045] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2046] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2047] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2048] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2049] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2050] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2051] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2052] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2053] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2054] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2055] The following is further disclosed regarding the above embodiment.

[2056] (Claim 1)

[2057] A means for receiving behavior record data input by a user through a communication interface;

[2058] A generating AI means for analyzing the behavioral record data and classifying the behavioral record data into appropriate categories based on the analysis results;

[2059] means for storing the data classified into the categories in a database;

[2060] a means for evaluating the user's progress based on the stored data and providing feedback on the evaluation results;

[2061] means for analyzing the stored data and providing improvement suggestions to the user;

[2062] A system including:

[2063] (Claim 2)

[2064] 10. The system of claim 1, wherein the analyzing means uses natural language processing techniques.

[2065] (Claim 3)

[2066] The system of claim 1 , wherein the communication interface uses a real-time communication protocol.

[2067] "Example 1"

[2068] (Claim 1)

[2069] A means for receiving behavior record data input by a user through a communication interface;

[2070] a generating artificial intelligence means for analyzing the behavior record data and classifying the data into appropriate categories based on the analysis results;

[2071] means for storing the data classified into the categories in a database;

[2072] a means for evaluating the user's progress based on the stored data and providing feedback on the evaluation results;

[2073] means for analyzing the stored data and providing improvement suggestions to the user;

[2074] means for assigning a user ID and a timestamp to the behavior record data;

[2075] means for generating prompt sentences for analysis by the generative artificial intelligence;

[2076] A means of aggregating past data and evaluating behavioral records over a certain period of time;

[2077] A system including:

[2078] (Claim 2)

[2079] 10. The system of claim 1, wherein the analyzing means uses natural language processing techniques.

[2080] (Claim 3)

[2081] The system of claim 1 , wherein the communication interface uses a real-time communication protocol.

[2082] "Application Example 1"

[2083] (Claim 1)

[2084] A means for receiving behavior record data input by a user through a communication interface;

[2085] A generating AI means for analyzing the behavioral record data and classifying the behavioral record data into appropriate categories based on the analysis results;

[2086] means for storing the data classified into the categories in a database;

[2087] a means for evaluating the user's progress based on the stored data and providing feedback on the evaluation results;

[2088] means for analyzing the stored data and providing improvement suggestions to the user;

[2089] A means for inputting user behavior record data through a head-mounted display;

[2090] A means of providing feedback and suggestions for improving driving skills;

[2091] A system including:

[2092] (Claim 2)

[2093] 10. The system of claim 1, wherein the analyzing means uses natural language processing techniques.

[2094] (Claim 3)

[2095] The system of claim 1 , wherein the communication interface uses a real-time communication protocol.

[2096] "Example 2: Combining Emotion Engines"

[2097] (Claim 1)

[2098] A means for receiving behavior record data input by a user through a communication interface;

[2099] a generating AI model means for analyzing the behavioral record data and classifying the behavioral record data into appropriate categories based on the analysis results;

[2100] an emotion engine means for determining a user's emotion from the data analyzed by the generation AI model means;

[2101] means for storing the data classified into categories and emotion determination results in a database;

[2102] means for evaluating the user's progress based on the stored data and generating feedback according to the evaluation result and emotions;

[2103] means for analyzing the stored data and providing improvement suggestions to the user;

[2104] A system including:

[2105] (Claim 2)

[2106] 10. The system of claim 1, wherein the generative AI model means and emotion engine means utilize natural language processing techniques.

[2107] (Claim 3)

[2108] The system of claim 1 , wherein the communication interface uses a real-time communication protocol.

[2109] "Application example 2 when combining emotion engines"

[2110] (Claim 1)

[2111] A means for receiving behavior record data input by a user through a communication interface;

[2112] A generating AI means for analyzing the behavioral record data and classifying the behavioral record data into appropriate categories based on the analysis results;

[2113] means for storing the data classified into the categories in a database;

[2114] a means for evaluating the user's progress based on the stored data and providing feedback on the evaluation results;

[2115] means for analyzing the stored data and providing improvement suggestions to the user;

[2116] emotion analysis means for analyzing the emotions of a user;

[2117] means for suggesting advertisements based on the emotion analysis results;

[2118] A system including:

[2119] (Claim 2)

[2120] 10. The system of claim 1, wherein the analyzing means uses natural language processing techniques.

[2121] (Claim 3)

[2122] The system of claim 1 , wherein the communication interface uses a real-time communication protocol. [Explanation of symbols]

[2123] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for receiving behavior record data input by a user through a communication interface; A generating AI means for analyzing the behavioral record data and classifying the behavioral record data into appropriate categories based on the analysis results; means for storing the data classified into the categories in a database; a means for evaluating the user's progress based on the stored data and providing feedback on the evaluation results; means for analyzing the stored data and providing improvement suggestions to the user; A system including:

2. The system of claim 1 , wherein the analyzing means uses natural language processing techniques.

3. The system of claim 1 , wherein the communication interface uses a real-time communication protocol.

Images