System

Abstract

A system is provided.SOLUTION: A system comprising: means for inputting interest information, health information, and consumption information of a user; means for cooperating with past consumption data and external information of the user; means for generating an optimal life plan based on a budget and the number of days of the user; means for presenting the generated life plan to the user; and means for supporting store reservation or ticket purchase based on the life plan.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] For today's busy users with diverse tastes and hobbies, it is difficult to plan and execute an optimal life plan within a limited budget. Furthermore, planning effective consumption activities within a budget is time-consuming and laborious, often resulting in stress. Furthermore, there is a lack of methods for automatically creating plans that take into account external information, weather, and other variables. To address these challenges, there is a need for a support system that helps users live an efficient and satisfying life within a budget. [Means for solving the problem]

[0005] In order to solve the above problems, the present invention provides the following means.

[0006] The system includes a means for inputting a user's interest information, health information, and consumption information, a means for linking with the user's past consumption data and external information, a means for generating an optimal life plan based on the user's budget and number of days, a means for presenting the generated life plan to the user, and a means for supporting store reservations or ticket purchases based on the life plan, allowing users to easily obtain and implement a life plan within their budget based on their preferences.

[0007] Furthermore, by providing a means for generating life plans that take weather information into account and a means for collecting user feedback and reflecting it in the next plan generation, it is possible to provide more accurate plans that are tailored to individual needs.

[0008] "User interest information" is information about hobbies, activities, and specific themes in which a user is interested.

[0009] "Health information" refers to information about the user's health condition, medical data, and daily health habits.

[0010] "Consumption information" refers to information relating to a user's purchase history, consumption patterns, and expenditure items.

[0011] "External information" refers to information that can be obtained from outside and is related to the user's place of residence, such as weather information, local event information, and store opening hours.

[0012] A "budget" is a monetary limit set by a user that can be used within a certain period of time.

[0013] "Days" refers to the remaining time available for the user to implement the plan.

[0014] "Life Plan" refers to a feasible activity and spending plan within a budget that is automatically generated by AI based on the user's interest information, health information, consumption information, and external information.

[0015] "Store reservation" refers to the act and procedure of a user making a reservation in advance at a store that the user plans to visit.

[0016] "Ticket purchase" refers to the act and procedure of purchasing an admission ticket or voucher in advance to use an event or service.

[0017] "Feedback" is information about evaluations, opinions, and points for improvement provided by the user regarding the life plan they have implemented. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0019] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0022] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0023] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0024] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0026] [First embodiment]

[0027] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0028] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0031] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0034] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0038] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0039] The present invention relates to an AI-based assistance system that helps users live an efficient and satisfying life within their budget, and can be embodied in the following forms.

[0040] Natural language description of the program

[0041] This system exchanges information between the user, the terminal, and the server, and generates an optimal life plan based on the user's preference information and past consumption data. The specific program processing is explained below.

[0042] 1. Entering user data

[0043] User: First, the user opens the dedicated application and enters their interests (hobbies, favorite foods, etc.), health information (current health condition), and consumption information (obtained through collaboration with external services such as PayPay) into the device.

[0044] Terminal: Receives input data and sends it to the server.

[0045] 2. Receipt and storage of data

[0046] Server: Receives user interest information, health information, and consumption information sent from the device and stores it in a database.

[0047] 3. AI-based data analysis

[0048] Server: Collects user interest, health, and consumption information from the database, and also obtains external information (such as weather and local event information) from the API.

[0049] Server: Analyzes all collected data using an AI model. Based on the analysis results, a life plan optimized for the user's budget and number of days is generated.

[0050] 4. Creating a life plan

[0051] Server: The life plan generated by the AI ​​model may include suggestions such as:

[0052] Planning a movie and the best time to go

[0053] Jogging schedule at a nearby park

[0054] Proposal for a Japanese restaurant that takes health into consideration

[0055] Server: The generated life plan includes detailed information about each activity (cost, location, and planned time).

[0056] 5. Presenting the plan

[0057] Server: Sends the generated life plan to the device.

[0058] Device: Displays the received plan to the user. The screen shows detailed information about each activity (cost, location, and planned time).

[0059] 6. Reservations and Purchases

[0060] User: Choose the activity they want to do from the proposed plan and make store reservations or purchase tickets as necessary.

[0061] Terminal: Sends reservation and purchase requests to the server.

[0062] Server: Receives reservation and purchase requests, coordinates with affiliated cinema and restaurant systems to complete reservations and purchases, and sends confirmation information to the device upon completion.

[0063] On your device: Display reservation confirmation information to the user and automatically add it to your calendar and reminders.

[0064] 7. Activity execution and feedback

[0065] User: Implements the proposed plan and actually carries out the activities.

[0066] Users: Provide ratings and feedback for each activity.

[0067] Terminal: Sends feedback information to the server.

[0068] Server: The received feedback is stored in a database and used as training data for the AI ​​model to improve the accuracy of future plan suggestions.

[0069] Specific examples

[0070] An embodiment of this system will be explained using a specific scenario. For example, suppose a user inputs information such as "watching movies," "jogging," and "likes Japanese food," and has a budget of 20,000 yen and 10 days remaining. The AI ​​analyzes this data and external weather information to generate the following life plan:

[0071] 1. "I'm planning to see two movies at the cinema this weekend."

[0072] 2. "Jogging in a nearby park on a sunny day."

[0073] 3. "I recommend eating at a Japanese restaurant after jogging."

[0074] The user confirms the proposed plan and reserves the movie ticket. The server completes the reservation through the connected cinema system and sends the confirmation information to the terminal. After watching the movie, the user provides feedback, which is used to generate the next plan.

[0075] In this way, the system of the present invention automatically generates an optimal life plan that takes into account the user's budget and time, and supports the execution of the plan.

[0076] The processing flow will be explained below.

[0077] Step 1:

[0078] User: Opens the application and enters their interests (hobbies, favorite foods, etc.) and health information (current health condition). They also set up the application to connect to external services (e.g., PayPay) to obtain past consumption data.

[0079] Step 2:

[0080] Terminal: Receives interest information, health information, and consumption information entered by the user and sends this data to the server.

[0081] Step 3:

[0082] Server: Receives user information (interest information, health information, consumption information) sent from the device and stores it in a database.

[0083] Step 4:

[0084] Server: Collects user interest, health, and consumption information from a database, and also uses external APIs to obtain real-time weather information and local event information.

[0085] Step 5:

[0086] Server: The collected data is input into the AI ​​model, which analyzes and generates an optimal life plan based on the user's budget and number of days. The AI ​​model takes into consideration the user's past consumption patterns, preferences, current weather information, etc.

[0087] Step 6:

[0088] Server: The generated life plan includes specific activity suggestions (e.g., watching a movie, jogging, or eating at a Japanese restaurant), along with detailed information on costs, locations, and planned times.

[0089] Step 7:

[0090] Server: Sends the generated life plan to the device.

[0091] Step 8:

[0092] Device: Displays the received life plan to the user. The screen shows detailed information about each activity (cost, location, and planned time).

[0093] Step 9:

[0094] User: Selects the activities they want to carry out from the proposed life plan and makes store reservations or purchases tickets as necessary.

[0095] Step 10:

[0096] Terminal: Sends reservation and purchase requests to the server.

[0097] Step 11:

[0098] Server: Receives reservation and purchase requests, coordinates with affiliated cinema and restaurant systems to complete reservations and purchases, and sends confirmation information to the device upon completion.

[0099] Step 12:

[0100] On your device: Display reservation confirmation information to the user and automatically add it to your calendar and reminders.

[0101] Step 13:

[0102] User: Carry out the proposed plan, for example, watch a movie at a cinema, go jogging in a park, or enjoy a meal at a Japanese restaurant.

[0103] Step 14:

[0104] Users: Provide ratings and feedback for each activity.

[0105] Step 15:

[0106] Terminal: Sends user feedback information to the server.

[0107] Step 16:

[0108] Server: The received feedback is stored in a database and used as training data for the AI ​​model to propose future plans, improving the accuracy of subsequent plan generation.

[0109] Example 1

[0110] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0111] In today's busy lifestyles, it is difficult for users to create a satisfying life plan that uses time efficiently and stays within a budget. In particular, plans must be created that take into account individual preferences, health status, and past consumption data. Weather information and local event information also need to be appropriately reflected, but collecting these individually and creating the optimal plan requires a huge amount of effort. To solve these problems, a system is needed that can automatically generate the optimal plan tailored to the user's situation and provide consistent support, from reservation to purchase procedures.

[0112] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0113] In this invention, the server includes means for inputting user interest information, health information, and consumption information, means for linking with the user's past consumption data and external information, means for generating an optimal life plan based on the user's budget and number of days, means for displaying detailed information of the generated life plan on a screen, means for collecting user feedback and saving it in a database, and means for training an AI model using the saved feedback and reflecting it in the generation of the next plan. This makes it possible to automatically generate an optimal life plan tailored to the user's individual circumstances and support reservations, purchases, and execution.

[0114] "User interest information" is information about activities and preferences in which the user is interested (for example, hobbies, favorite foods, events they want to enjoy, etc.).

[0115] "Health information" refers to the user's current health condition and health-related information (e.g., medical history, daily activity level, diet, etc.).

[0116] "Consumption information" refers to information about a user's past consumption data and purchase history (for example, expenditure details, services used, types and quantities of purchased products, etc.).

[0117] "Past consumption data" refers to a history of information about purchases and expenditures made by a user.

[0118] "External information" is external data related to the creation of the user's life plan (for example, weather information, local event information, etc.).

[0119] A "budget" is a limit on the amount of money available to a user to plan their spending within a particular time period.

[0120] "Number of days" is the unit of time period set by the user for carrying out a particular life plan.

[0121] A "life plan" is a plan of activities and schedules optimized within a user's budget and number of days based on the user's interest information, health information, consumption information, and external information.

[0122] "Store reservation" is a process in which a user makes a reservation in advance to visit or use a specific store.

[0123] "Ticket purchase" refers to the act of a user obtaining a ticket necessary for admission to a movie theater, event, or the like.

[0124] "Feedback" is information about evaluations and impressions provided by users regarding activities they have performed.

[0125] "Database" means an electronic information management system for organizing and storing collected user information and feedback.

[0126] An "AI model" is an artificial intelligence computational model that analyzes collected data and generates the optimal life plan for the user.

[0127] This invention relates to an AI-based support system that helps users live an efficient and satisfying life within their budget. The system of this invention exchanges information between the user, terminal, and server, and generates an optimal life plan based on the user's preference information and past consumption data.

[0128] Entering User Data

[0129] The user opens the dedicated application and inputs their interests (hobbies, favorite foods, etc.), health information (current health condition), and consumption information (obtained, for example, through collaboration with external services such as payment services). The device receives the input data and sends it to the server.

[0130] Receiving and storing data

[0131] The server receives the user's interest, health, and consumption information sent from the device and stores it in a database. The stored data undergoes error checking to ensure it is properly formatted.

[0132] AI-based data analysis

[0133] The server collects information on the user's interests, health, and consumption from a database, and also obtains external information (such as weather information and local event information) from an API. All collected data is analyzed using an AI model (such as a model using TensorFlow or Keras). Based on the analysis results, a life plan optimized for the user's budget and number of days is generated.

[0134] Creating a life plan

[0135] Based on the analysis results of the AI ​​model, the server generates an optimal life plan for the user. For example, it may include "movie viewing plans and the best time to do so," "a jogging schedule in a nearby park," and "recommendations for Japanese restaurants that take health into consideration." The generated life plan includes detailed information for each activity (cost, location, and planned time).

[0136] Presenting the plan

[0137] The server sends the generated life plan to the device, which then displays the plan to the user, showing detailed information about each activity (cost, location, and planned time).

[0138] Pre-order and purchase

[0139] The user selects the activity they want to carry out from the proposed plan and makes a store reservation or purchase a ticket. The device sends the reservation or purchase request to the server. The server receives the reservation or purchase request and connects with the systems of affiliated movie theaters and restaurants to make the reservation or purchase. Once completed, confirmation information is sent to the device. The device displays the reservation confirmation information to the user and automatically adds it to their calendar or reminders.

[0140] Activity execution and feedback

[0141] The user executes the proposed plan and performs the actual activities. The user generates ratings and feedback for each activity. The device sends the feedback information to the server. The server stores the received feedback in a database and uses it as training data for the AI ​​model to improve the accuracy of future plan suggestions.

[0142] Specific examples

[0143] An embodiment of this system will be explained using a specific scenario. For example, suppose a user inputs information such as "watching movies," "jogging," and "likes Japanese food," and has a budget of 20,000 yen and 10 days remaining. In this case, the AI ​​analyzes this data and external weather information to generate the following life plan:

[0144] 1. "I'm planning to see two movies at the cinema this weekend."

[0145] 2. "Jogging in a nearby park on a sunny day."

[0146] 3. "I recommend eating at a Japanese restaurant after jogging."

[0147] The user confirms the proposed plan and reserves the movie ticket. The server completes the reservation through the connected cinema system and sends the confirmation information to the terminal. After watching the movie, the user provides feedback, which is used to generate the next plan.

[0148] Example prompts for generative AI models

[0149] Below are some examples of prompts to input to an AI model:

[0150] "If the user likes watching movies, jogging, and eating Japanese food, generate the optimal life plan for a 10-day period with a budget of 20,000 yen."

[0151] "Please provide a recommended activity plan that takes into account the user's preferences (e.g., watching movies, jogging, Japanese food), current budget, and duration."

[0152] In this way, the system of the present invention automatically generates an optimal life plan that takes into account the user's budget and time, and supports the execution of that plan.

[0153] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0154] Step 1: Enter user data

[0155] User: Opens the dedicated application and enters information about their interests, health, and consumption. Examples of input data include a preference for watching movies, jogging, and Japanese food.

[0156] Terminal: Receives data entered by the user and sends this data to the server. The entered data is structured in JSON or XML format.

[0157] Step 2: Receiving and storing data

[0158] Server: Receives data sent from the device. The received content includes user interest information, health information, and consumption information.

[0159] Server: The received data is checked for errors to ensure it is properly formatted, and then saved to a database. The saved data is indexed and managed for each user.

[0160] Step 3: Gathering external information

[0161] Server: Calls external APIs to obtain external data such as weather information and local event information. The data obtained from the API is provided in the form of a JSON object containing weather forecasts and event information.

[0162] Server: The acquired external information is temporarily stored in memory and used for subsequent data analysis.

[0163] Step 4: Data analysis using AI

[0164] Server: Collects user interest information, health information, and consumption information from the database, and creates a dataset that also includes external information.

[0165] Server: Runs AI models using libraries such as TensorFlow and Keras to analyze the created datasets, using clustering algorithms and predictive models.

[0166] Server: As a result of the analysis, it generates an optimal life plan based on the user's budget and number of days. This plan includes detailed information about each activity (e.g., cost, location, and planned time).

[0167] Step 5: Create a life plan

[0168] Server: Based on the analysis results of the AI ​​model, the server generates an optimal life plan for the user, such as movie-going plans and the best times to do so, a jogging schedule in a nearby park, and recommendations for Japanese restaurants that take health into consideration.

[0169] Server: Formats the generated life plan and prepares it for transmission to the terminal.

[0170] Step 6: Present your plan

[0171] Server: Sends the generated life plan to the device. Push notifications may be used as a notification mechanism.

[0172] Terminal: Displays the user's life plan, with detailed information for each activity (cost, location, and planned time) presented in a user-friendly interface.

[0173] Step 7: Pre-order and purchase

[0174] User: Selects the activity they want to perform from the displayed list and makes a store reservation or purchases a ticket. For example, they click the "Go see this movie" button.

[0175] Terminal: Sends reservation or purchase requests to the server based on the user's selection.

[0176] Server: Processes the received request and executes the reservation or purchase in cooperation with the systems of affiliated stores and service providers. After the process is complete, it sends confirmation information to the terminal.

[0177] On the device: Display the reservation confirmation to the user and automatically add it to their calendar or reminders as appropriate.

[0178] Step 8: Activity execution and feedback

[0179] User: Executes the proposed plan and performs an actual activity, e.g., watches a movie at a cinema.

[0180] Users: Provide ratings and feedback for each activity, such as "The quality of the movie was great" or "The service at the Japanese restaurant was good."

[0181] Terminal: Sends feedback information to the server.

[0182] Server: The received feedback is stored in a database and used as learning data for the AI ​​model to improve the accuracy of the next plan proposal.

[0183] (Application example 1)

[0184] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0185] There is a lack of a means to automatically generate and present plans that take into account individual interests, health conditions, and consumption trends when users are trying to live an efficient and satisfying life within their budget. There is also a lack of support systems that allow users to easily make reservations and purchases at physical stores based on those plans. Furthermore, there is a need for a means to provide users with intuitive information in real time by displaying the generated plans on smart glasses or head-mounted displays.

[0186] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0187] In this invention, the server includes means for inputting user interest information, health information, and consumption information, means for linking with the user's past consumption data and external information, means for generating an optimal life plan based on the user's budget and number of days, means for presenting the generated life plan to the user, means for supporting reservations or purchases at physical stores based on the life plan, and means for displaying the generated life plan on smart glasses or a head-mounted display. This makes it possible to automatically generate plans tailored to the user's individual needs, simplify the reservation and purchase process, and provide information intuitively in real time.

[0188] "User interest information" is information about the user's hobbies and preferences.

[0189] "Health information" is information relating to the user's current health condition and past health checkup results.

[0190] "Consumption information" is information relating to a user's purchase history and spending patterns.

[0191] "Historical Consumption Data" is data recorded by a user that includes details of previous purchases and expenditures.

[0192] "External information" is data obtained from external sources, such as weather information and local event information.

[0193] A "budget" is a spending limit set by a user.

[0194] "Number of days" refers to the date set by the user as the validity period of the life plan.

[0195] The "optimal life plan" is a proposal of an optimal life plan for the user, generated based on the user's interest information, health information, consumption information, and external information.

[0196] "Physical store" refers to a physical store, such as a clothing store, restaurant, or movie theater.

[0197] A "reservation" is the act of reserving in advance the use or purchase of a specific service or product.

[0198] "Purchase" is the act of acquiring a service or product with money.

[0199] "Smart glasses" are eyeglass-type devices that have the function of displaying information in the wearer's field of vision.

[0200] A "head-mounted display" is a display worn on a user's head that provides virtual or augmented reality visual information.

[0201] "User feedback" is information regarding opinions and evaluations provided by users regarding the proposed life plan.

[0202] "Real-time" refers to processing and information being provided immediately at that moment.

[0203] The system embodying the present invention supports users in living an efficient and satisfying life within their budget, and involves cooperation between users, terminals, and a server. Specific embodiments of the system are described below.

[0204] First, a user launches an application using a device such as a smartphone, smart glasses, or head-mounted display. The user then inputs their interest information (hobbies, favorite foods, etc.), health information (health status), and consumption information (purchase history, etc.) into the application. The input information is received by the device and sent to the server.

[0205] The server stores the interest, health, and consumption information sent by the user in a database. The server also obtains external information such as weather and local event information via API. The AI ​​model on the server analyzes this data and generates an optimal life plan based on the user's budget and number of days.

[0206] The generated life plan includes specific activity suggestions, such as watching a movie, jogging, or eating at a Japanese restaurant, along with detailed information such as the cost, location, and estimated time for each activity. This life plan is then sent by the server to the device and presented to the user.

[0207] Users can select the desired activity from the proposed life plan and make a reservation or purchase a ticket at a physical store. The device sends the reservation or purchase request to the server, which then completes the reservation or purchase by linking with the systems of affiliated stores and services. Once completed, confirmation information is sent to the device and displayed to the user. If using smart glasses or a head-mounted display, the information is displayed in real time, allowing the user to receive the information intuitively.

[0208] In addition, users can provide ratings and feedback on the activities they have performed. This feedback information is also sent to the server and stored in a database. The server uses this feedback as learning data for the AI ​​model and reflects it in the next life plan generation, thereby improving the accuracy of suggestions.

[0209] For example, if a user inputs information such as "watching movies," "jogging," and "likes Japanese food," along with a budget of 20,000 yen and 10 days remaining, the system will generate the following life plan:

[0210] 1. "I'm planning to see two movies at the cinema this weekend."

[0211] 2. "Jogging in a nearby park on a sunny day."

[0212] 3. "I recommend eating at a Japanese restaurant after jogging."

[0213] An example of a prompt sentence in this case is: "The user likes watching movies, jogging, and Japanese food. The budget is 20,000 yen and there are 10 days left. The weather is sunny on weekends and cloudy or rainy on other days. Please generate the optimal activity plan."

[0214] These steps allow users to automatically generate personalized plans, simplify the booking and purchasing process, and provide intuitive information in real time.

[0215] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0216] Step 1:

[0217] A user launches an application using a smartphone or smart glasses and inputs information about their interests, health, and consumption. This information is received by the device as user input. For example, the user inputs information such as "watching movies," "jogging," and "I like Japanese food." The device then sends the input data to the server.

[0218] Step 2:

[0219] The server receives the user's interest, health, and consumption information sent from the device and stores it in a database. At the same time, the server uses an external API to obtain weather information and local event information. This external information is also stored in the database. Input data includes the user's preferences and health status, past consumption data, weather information, and event information.

[0220] Step 3:

[0221] The AI ​​model on the server collects and analyzes the user's interest, health, consumption, and external information from a database. This analysis generates an optimal life plan based on the user's budget and number of days. For example, given a user's budget of 20,000 yen and 10 days remaining, the AI ​​model suggests activities such as watching a movie, jogging, and eating at a Japanese restaurant.

[0222] Step 4:

[0223] The server sends the generated life plan to the device, which receives it and displays it to the user. The displayed content includes detailed information about each activity (e.g., movie viewing plans, jogging time, location and cost of Japanese restaurants, etc.).

[0224] Step 5:

[0225] The user selects the desired activity from the proposed life plan. Based on the selected activity, the user makes a reservation or purchase request at a physical store. The device sends these requests to the server.

[0226] Step 6:

[0227] The server receives reservation and purchase requests sent by users and completes the reservation or purchase in cooperation with the systems of affiliated stores and services. After completion, reservation confirmation information is sent to the device. The device receives this information and displays it to the user. If smart glasses or a head-mounted display are used, this confirmation information is also displayed in real time.

[0228] Step 7:

[0229] Users provide ratings and feedback for the activities they perform. The device collects this feedback information and sends it to the server. The server stores the feedback information in a database and uses it as learning data for the AI ​​model to improve the accuracy of future life plan suggestions.

[0230] In this way, all steps work together to enable users to realize the optimal life plan while responding to their individual needs and using their budget and time efficiently.

[0231] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0232] The present invention relates to an AI-based support system for helping users live an efficient and satisfying life within their budget, and in particular to improving the accuracy of life plans by incorporating an emotion engine. Hereinafter, embodiments of the present invention will be described in detail.

[0233] Natural language description of the program

[0234] This system exchanges information between the user, device, and server, and not only generates an optimal life plan based on the user's preferences and past consumption data, but also has the ability to recognize the user's emotions in real time and adjust the plan accordingly.

[0235] 1. Entering user data

[0236] User: Opens the application and enters their interests (hobbies, favorite foods, etc.) and health information (current health condition). For emotion recognition, the application also allows voice input and facial recognition. The application also allows users to connect to external services (e.g., PayPay) to obtain past consumption data.

[0237] Terminal: Receives input interest information, health information, and consumption information and sends it to the server.

[0238] 2. Receipt and storage of data

[0239] Server: Receives user information (interest information, health information, consumption information) and emotion data sent from the device and stores it in a database.

[0240] 3. AI-based data analysis

[0241] Server: Collects user interest, health, and consumption information from a database, and also uses external APIs to obtain real-time weather information and local event information.

[0242] Server: Using an emotion engine, the server analyzes the voice, facial expressions, and text data sent from the device in real time to recognize the user's emotional state.

[0243] 4. Creating a life plan

[0244] Server: The collected data and the analysis results of the emotion engine are input into the AI ​​model to generate an optimal life plan based on the user's budget and number of days. The AI ​​model takes into consideration the user's past consumption patterns, preferences, current emotional state, weather information, etc.

[0245] 5. Plan generation and adjustment

[0246] Server: The life plan generated by the AI ​​model includes specific activity suggestions (e.g., watching a movie, jogging, or eating at a Japanese restaurant) along with detailed information about costs, locations, and planned times.

[0247] Server: Reflects the user's real-time emotional state and adjusts the plan, for example, by suggesting a plan that emphasizes relaxation activities if the user is under high stress, or by suggesting more active activities if the user is in a good mood.

[0248] 6. Presenting the plan

[0249] Server: Sends the generated life plan to the device.

[0250] Device: Displays the received life plan to the user. The screen shows detailed information about each activity (cost, location, planned time) and adjustment points based on the user's emotional state.

[0251] 7. Reservations and Purchases

[0252] User: Selects the activities they want to carry out from the proposed life plan and makes store reservations or purchases tickets as necessary.

[0253] Terminal: Sends reservation and purchase requests to the server.

[0254] Server: Receives reservation and purchase requests, coordinates with affiliated cinema and restaurant systems to complete reservations and purchases, and sends confirmation information to the device upon completion.

[0255] On your device: Display reservation confirmation information to the user and automatically add it to your calendar and reminders.

[0256] 8. Activity execution and feedback

[0257] User: Carry out the proposed plan, for example, watch a movie at a cinema, go jogging in a park, or enjoy a meal at a Japanese restaurant.

[0258] Users: Provide ratings and feedback for each activity.

[0259] Terminal: Sends user feedback information to the server.

[0260] Server: The received feedback is stored in a database and used as training data for the AI ​​model to propose future plans, improving the accuracy of subsequent plan generation.

[0261] Specific examples

[0262] An embodiment of this system will be explained using a specific scenario. For example, suppose a user inputs information such as "watching movies," "jogging," and "likes Japanese food," has a budget of 20,000 yen, and has 10 days left. The AI ​​analyzes this data and external weather information, and the emotion engine recognizes that the user's current stress level is high, and generates the following life plan:

[0263] 1. "I'm planning to see two movies at the cinema this weekend."

[0264] 2. "Jogging in a nearby park on a sunny day."

[0265] 3. "I recommend eating at a Japanese restaurant after jogging."

[0266] 4. "We also suggest visiting a relaxing hot spring facility to reduce stress."

[0267] The user checks the proposed plan and reserves movie tickets. The server completes the reservation through the connected cinema system and sends the confirmation information to the terminal. After watching the movie, the user provides feedback, which is used to generate the next plan. The emotion engine analyzes the user's emotions in real time, and the system always provides a plan optimized for the user's emotional state.

[0268] In this way, the system of the present invention takes into consideration the user's budget and time, and also analyzes and reflects their emotional state in real time to automatically generate an optimal life plan and support its implementation.

[0269] The processing flow will be explained below.

[0270] Step 1:

[0271] User: Opens the application and enters their interests (hobbies, favorite foods, etc.) and health information (current health condition). In addition, they authorize voice input and face recognition functions, and set up the application to connect to external services (e.g., PayPay) to obtain past consumption data.

[0272] Step 2:

[0273] Terminal: Receives interest information, health information, and consumption information entered by the user and sends this data to the server.

[0274] Step 3:

[0275] Server: Receives user information (interest information, health information, consumption information) and emotion data sent from the device and stores it in a database.

[0276] Step 4:

[0277] Server: Collects user interest, health, and consumption information from a database, and also uses external APIs to obtain real-time weather information and local event information.

[0278] Step 5:

[0279] Server: Using an emotion engine, the server analyzes the voice, facial expressions, and text data sent from the device in real time to recognize the user's emotional state.

[0280] Step 6:

[0281] Server: The collected data and the analysis results of the emotion engine are input into the AI ​​model to generate an optimal life plan based on the user's budget and number of days. The AI ​​model takes into consideration the user's past consumption patterns, preferences, current emotional state, weather information, etc.

[0282] Step 7:

[0283] Server: The generated life plan includes specific activity suggestions (e.g., watching a movie, jogging, or eating at a Japanese restaurant) with detailed information about costs, locations, and estimated times.

[0284] Step 8:

[0285] Server: Reflects the user's real-time emotional state and adjusts the plan, for example, by suggesting a plan that emphasizes relaxation activities if the user is under high stress, or by suggesting more active activities if the user is in a good mood.

[0286] Step 9:

[0287] Server: Sends the generated life plan to the device.

[0288] Step 10:

[0289] Device: Displays the received life plan to the user. The screen shows detailed information about each activity (cost, location, planned time) and adjustment points based on the user's emotional state.

[0290] Step 11:

[0291] User: Selects the activities they want to carry out from the proposed life plan and makes store reservations or purchases tickets as necessary.

[0292] Step 12:

[0293] Terminal: Sends reservation and purchase requests to the server.

[0294] Step 13:

[0295] Server: Receives reservation and purchase requests, coordinates with affiliated cinema and restaurant systems to complete reservations and purchases, and sends confirmation information to the device upon completion.

[0296] Step 14:

[0297] On your device: Display reservation confirmation information to the user and automatically add it to your calendar and reminders.

[0298] Step 15:

[0299] User: Carry out the proposed plan, for example, watch a movie at a cinema, go jogging in a park, or enjoy a meal at a Japanese restaurant.

[0300] Step 16:

[0301] Users: Provide ratings and feedback for each activity.

[0302] Step 17:

[0303] Terminal: Sends user feedback information to the server.

[0304] Step 18:

[0305] Server: The received feedback is stored in a database and used as training data for the AI ​​model to propose future plans, improving the accuracy of subsequent plan generation.

[0306] Example 2

[0307] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0308] When generating an efficient and satisfying life plan within a user-specified budget and number of days, there was a problem in that the system was unable to recognize the user's emotional state in real time and adjust the plan accordingly. It was also difficult to dynamically generate plans that incorporated weather information and local event information, and it was also a challenge to efficiently utilize the user's personal preferences and past consumption data. Another problem was the lack of a feedback loop to collect user feedback and reflect it in the next plan.

[0309] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0310] In this invention, the server includes: means for inputting user interest information, health information, and consumption information; means for linking with the user's past consumption data and external information; means for allowing voice input and facial recognition for emotion recognition; means for generating an optimal life plan based on the user's budget and number of days; means for analyzing the user's emotional state in real time and adjusting the life plan based on the analysis; means for presenting the generated life plan to the user; means for assisting with store reservations or ticket purchases based on the life plan; and means for collecting user feedback and reflecting it in the generation of the next plan. This allows for the generation of an optimized life plan in real time, taking the user's emotional state into consideration. Furthermore, incorporating weather information and local event information allows for the provision of more dynamic and personalized plans. Furthermore, by creating a feedback loop, the accuracy of subsequent plans can be improved, resulting in an overall improved user experience.

[0311] "Interest information" is information about the user's hobbies, preferences, and tastes.

[0312] "Health information" refers to information related to health, such as the user's current health condition and medical history.

[0313] "Consumption information" is data related to consumption activities that a user has engaged in in the past.

[0314] "External information" refers to information obtained from external databases or APIs, such as weather information and local event information.

[0315] "Emotion recognition" is a technology that analyzes a user's voice and facial expressions to determine their current emotional state.

[0316] "Voice input" is a means of capturing user voice data as input.

[0317] "Facial recognition" is a technology that analyzes a user's facial features and identifies them as an individual.

[0318] A "life plan" is a specific action plan proposed based on the user's budget and number of days.

[0319] "Real-time" means that data is analyzed immediately and necessary processing is carried out at that moment.

[0320] "Analysis" is the process of examining data in detail and deriving patterns and trends.

[0321] "Store reservation" refers to making a reservation in advance at a store that the user plans to visit.

[0322] "Ticket purchase" means the purchase of an admission ticket to participate in an event or activity.

[0323] "Feedback" refers to opinions and evaluations provided by users, and is information that can be used to improve services next time.

[0324] This invention is an AI-based support system that allows users to generate and adjust an optimal life plan within their budget and time frame. This system consists of three main components: the user, the device, and the server. It analyzes the user's emotional state in real time based on their interest, health, and consumption information, and provides a life plan that reflects that information.

[0325] Entering User Data

[0326] User: The user opens the application and enters their interest information (such as hobbies and favorite dishes), health information, and consumption information. They also configure the application to enable voice input and facial recognition, which provides basic data for emotion recognition. They also configure the application to connect to external services (e.g., electronic payment systems) and grant permission to obtain past consumption data.

[0327] Receiving and storing data

[0328] Terminal: The terminal sends all data entered by the user to the server. It also validates the user's input to ensure that the data is accurate.

[0329] Server: The server receives the data sent from the device and stores it in a database. The stored data is used for further analysis and calculation.

[0330] AI-based data analysis

[0331] Server: The server collects user information from a database and, if necessary, uses external APIs to obtain real-time weather information and local event information. It also uses an emotion engine to analyze voice, facial expressions, and text data to recognize the user's emotional state. Emotion recognition uses voice recognition algorithms and image recognition algorithms.

[0332] Creating and adjusting life plans

[0333] Server: Based on the collected data and the analysis results of the emotion engine, the data is input into the AI ​​model to generate an optimal life plan based on the user's budget and number of days. The AI ​​model takes into account past spending patterns, interests, current emotional state, weather information, etc. The generated plan includes specific activities such as going to the movies, jogging, and eating at a Japanese restaurant, along with the cost, location, and planned time for each.

[0334] Presenting and adjusting plans

[0335] Server: Sends the generated life plan to the device. The plan reflects the user's real-time emotional state and is dynamically adjusted according to the specific situation.

[0336] Device: Displays the received life plan to the user, along with detailed information on each activity and adjustment points.

[0337] Reservation and purchase assistance

[0338] User: The user selects the activities they want to carry out from the proposed life plan and makes store reservations or purchases tickets as necessary.

[0339] Terminal: Sends reservation and purchase requests to the server.

[0340] Server: The server receives reservation and purchase requests, executes them in cooperation with partner systems, and sends confirmation information to the terminal once completed.

[0341] On the device: Display reservation confirmation information to the user and automatically add it to their calendar or reminders.

[0342] Activity execution and feedback

[0343] User: Carry out the proposed plan and enjoy watching a movie, jogging, and eating at a Japanese restaurant.

[0344] Users: Provide ratings and feedback for each activity.

[0345] Terminal: Sends feedback information to the server.

[0346] Server: The received feedback is stored in a database and used as training data for the AI ​​model to propose future plans, improving the accuracy of subsequent plan generation.

[0347] Specific examples

[0348] For example, suppose a user inputs information such as "watching movies," "jogging," and "likes Japanese food," and requests a 10-day plan with a budget of ¥20,000. The system takes this data into account along with real-time weather information, and the emotion engine recognizes that the user's current stress level is high, generating the following life plan:

[0349] 1. "I'm going to see two movies at the cinema this weekend."

[0350] 2. "Jogging in a nearby park on a sunny day."

[0351] 3. "Recommend a meal at a Japanese restaurant after jogging."

[0352] 4. "We also suggest visiting a relaxing hot spring facility to reduce stress."

[0353] The user checks the proposed plan and reserves movie tickets. The server completes the reservation with the affiliated cinema system and sends the confirmation information to the terminal. After watching the movie, the user provides feedback, which is reflected in the next plan generation. The emotion engine analyzes the user's emotions in real time, so that the system always provides the most optimized plan.

[0354] Prompt Sentence Examples

[0355] "Watching movies," "Jogging," "Like Japanese food," "Budget is 20,000 yen," "Prefer a 10-day plan," "Recognizes and adjusts emotions in real time," "Includes relaxing activities."

[0356] In this way, the present invention automatically generates an optimal life plan that takes into account the user's budget and time, and also analyzes and reflects their emotional state in real time, and supports their execution.

[0357] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0358] Step 1: Enter user data

[0359] The user opens the application and inputs interest, health, and consumption information. The user also sets permissions for voice input and facial recognition. Furthermore, the user sets up connections with external services (e.g., electronic payment systems) and gives permission to obtain past consumption data.

[0360] Input: Interest information (hobbies, preferences), health information, consumption information, voice input permission, face recognition permission, external service linkage permission

[0361] Output: User dataset to send to terminal

[0362] What happens: A user enters information into an application form, and after completing each field, presses a confirmation button. The form data is packaged in JSON format and ready to be sent to the device.

[0363] Step 2: Receiving and storing data

[0364] The terminal transmits the data entered by the user to the server, where it verifies the consistency and integrity of the data and performs error checking before transmission.

[0365] The server receives the data sent from the terminal, verifies it, and then stores it in a database.

[0366] Input: A user-entered dataset

[0367] Output: User data stored in the database

[0368] Specific operation: The terminal sends an HTTP POST request to the server, the server receives the data, parses it, checks for errors and validates the data format, and then stores it in the database.

[0369] Step 3: Data analysis using AI

[0370] The server collects user information from a database and also uses external APIs to obtain current weather information and local event information.

[0371] The server uses an emotion engine to analyze the voice data, facial expression data, and text data sent from the terminal in real time and identify the user's emotional state.

[0372] Input: User information collected from the database, weather information obtained from external APIs, local event information, emotion data

[0373] Output: User's emotional state as an analysis result and supplementary information

[0374] Specific operation: The server executes a scheduled task to obtain user information and sends a request to an external API (e.g., weather API). The obtained data is input into the emotion engine for real-time analysis.

[0375] Step 4: Create a life plan

[0376] The server inputs the collected data and the results of the emotion engine's analysis into an AI model to generate an optimal life plan based on the user's budget and number of days.

[0377] Input: Collected user data, emotion engine analysis results, external API data

[0378] Output: Generated life plan

[0379] Specific actions: Input data into the AI ​​model and run inference. The generated life plan includes specific activity suggestions, with details (cost, location, and planned time) automatically calculated and entered.

[0380] Step 5: Present and adjust the plan

[0381] The server sends the generated life plan to the device, which adjusts the plan as needed to reflect the user's real-time emotional state.

[0382] The terminal displays the received life plan to the user.

[0383] Input: Generated life plan

[0384] Output: The adjusted life plan displayed to the user

[0385] Specific operation: The server sends plan data in JSON format to the device, which receives the data and displays it on the interface. Plan adjustments are made based on the results of real-time sentiment analysis, and adjustment points are presented to the user.

[0386] Step 6: Reservation and purchase assistance

[0387] Users can select the activities they want to carry out from the proposed life plan and make reservations or purchase tickets.

[0388] The terminal sends a reservation request or a purchase request to the server.

[0389] The server cooperates with the affiliated system to execute the reservation or purchase and transmits confirmation information to the terminal.

[0390] The device will display the reservation confirmation information to the user and automatically add it to their calendar or reminders.

[0391] Input: User-selected activity booking / purchase request

[0392] Output: Confirmation information and automatic addition to calendar and reminders

[0393] Specific operation: The user selects an activity and enters the required information in the reservation form. The device sends a request to the server, and the server returns the execution result to the device. The device displays a confirmation and automatically adds it to the calendar and reminders.

[0394] Step 7: Activity execution and feedback

[0395] The user carries out the proposed plan and enjoys watching a movie, jogging, and eating at a Japanese restaurant.

[0396] Users provide ratings and feedback for each activity.

[0397] The terminal transmits the feedback information to the server.

[0398] The server stores the received feedback in a database and uses it as training data for the AI ​​model to propose future plans.

[0399] Input: Feedback data after the user performs an activity

[0400] Output: Feedback stored in the database as training data

[0401] Specific operation: The user fills out a feedback form, which the device sends to the server. The server stores the feedback in a database and uses it to generate the next life plan.

[0402] (Application example 2)

[0403] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0404] In modern society, autonomous vehicles offer significant improvements in user comfort and efficiency, but their capabilities are not being fully utilized. Furthermore, there are many challenges in providing optimal life plans that take into account the user's emotional state and real-time information. Specifically, plans are not adequately generated based on the user's interests and health status, and real-time adjustments based on emotion recognition are insufficient. Furthermore, conventional systems have difficulty generating route proposals that take into account weather and event information. Therefore, effective means of improving user satisfaction and comfort are needed.

[0405] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0406] In this invention, the server includes means for recognizing the user's emotional state in real time and adjusting the life plan, means for generating and presenting route proposals based on the user's emotions and real-time information within the autonomous vehicle, and means for acquiring weather information and event information using an external API, thereby enabling the generation of a life plan and route proposals optimized for the user's real-time emotional state and external environment.

[0407] "User interest information" is data that indicates the user's preferred activities, hobbies, and interests.

[0408] "Health information" is data relating to the user's current health condition and past health history.

[0409] "Consumption information" is data indicating products and services purchased by a user in the past, the amount spent on those products and services, and so on.

[0410] "Past consumption data" refers to data including the user's past purchasing behavior and its history.

[0411] "External information" refers to information obtained from outside the system, such as weather, local events, and traffic conditions.

[0412] A "budget" is data that indicates a range of available amounts set by the user.

[0413] "Number of days" is data indicating the number of days within a specific period set by the user.

[0414] The "means for generating a life plan" is a process for creating a plan for the user to live an efficient and satisfying life based on collected user data and external information.

[0415] The "means for presenting a life plan to a user" refers to a method for visually or audibly displaying or notifying a user of a created life plan.

[0416] "Means for supporting store reservations or ticket purchases" are methods that allow users to easily make reservations or purchases based on the proposed life plan.

[0417] "Means for recognizing emotional state in real time" refers to technology that analyzes the user's voice and facial expressions to determine their current emotions.

[0418] The "means for generating and presenting route proposals" is a technology that plans an optimal travel route based on the user's emotional state and real-time environmental information, and displays it on the vehicle terminal.

[0419] "Means of obtaining weather and event information using external APIs" refers to a method of obtaining real-time weather and local event information by linking with external data provision services.

[0420] The system of this invention inputs a user's interest information, health information, and consumption information, and generates an optimal life plan based on the budget and number of days in conjunction with past consumption data and external information. The system also incorporates an emotion engine to recognize the user's real-time emotional state and dynamically adjust the life plan.

[0421] Hardware and Software Configuration

[0422] The server performs various data processing and calculations using a database, AI analysis engine, emotion engine, external API connection module, etc. These components will be explained in detail below.

[0423] 1. User Device:

[0424] Input devices (smartphones, tablets, etc.)

[0425] Emotion recognition devices (camera, microphone, etc.)

[0426] 2. Server:

[0427] Database (SQL, etc.)

[0428] AI analysis engine (TensorFlow, Keras, etc.)

[0429] Emotion Engine API

[0430] External API connection module (weather API, event API)

[0431] Data processing and calculation procedures

[0432] The server first receives interest, health, and consumption information from the user's device and stores it in a database. This information, along with real-time weather information and local event information obtained through external APIs, is then input into an AI analysis engine.

[0433] The server uses an emotion engine to analyze the user's voice, facial expressions, and input text in real time to recognize the user's emotional state. The recognized emotional data, along with other data, becomes the input data required to generate a life plan.

[0434] The generated life plan includes specific activity suggestions based on the user's schedule, activities, and budget. For example, the system suggests optimal routes based on the user's emotional state and real-time information in an autonomous vehicle. The system also supports users in making reservations and purchasing tickets, and the server handles these procedures.

[0435] Specific examples

[0436] For example, suppose a user enters information into the app such as "I want to relax" and "I like Japanese food," and sets a budget of 20,000 yen. The server generates the following life plan based on real-time weather information and local event information:

[0437] 1. "Plan a relaxing picnic in the park on a sunny day."

[0438] 2. "I suggest we go to a Japanese restaurant afterwards."

[0439] 3. "We recommend visiting a hot spring facility to relax in the evening."

[0440] If the emotion engine recognizes that the user's stress level is high, it will focus on suggesting more relaxing activities.

[0441] Prompt Sentence Examples

[0442] "The user wants to relax, so generate a route for stress reduction."

[0443] "Generate tonight's dinner plans for the user. My budget is 5000 yen and I'm in the mood to relax."

[0444] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0445] Step 1:

[0446] The user's device inputs information about the user's interests, health, and consumption. The user enters this data into the app from their smartphone or tablet and sends it to the server. Input may also be done using the user's voice or facial recognition function.

[0447] Input: User interest information, health information, consumption information

[0448] Output: Send data to the server

[0449] Step 2:

[0450] The server receives the information sent from the user terminal and stores it in a database. Based on this data, the server aggregates the data necessary to generate a life plan.

[0451] Input: User interest information, health information, consumption information

[0452] Output: Save data to a database

[0453] Step 3:

[0454] The server uses external APIs to obtain real-time weather and event information. For this purpose, it uses the weather API and event API.

[0455] Input: Real-time information request from an external API

[0456] Output: Get real-time weather and event information

[0457] Step 4:

[0458] The server uses an emotion engine to analyze the user's voice, facial expressions, and text in real time to recognize the user's emotional state, thereby determining the user's stress level and mood.

[0459] Input: User's voice data, facial expression data, text data

[0460] Output: Emotional state data

[0461] Step 5:

[0462] The server uses an AI analysis engine to generate an optimal life plan based on the user's data stored in the database, as well as real-time weather information, event information, and emotional state. The generated life plan includes the user's budget and details of each activity.

[0463] Input: User data, real-time weather information, event information, emotional state data

[0464] Output: Optimal Life Plan

[0465] Step 6:

[0466] The server transmits the generated life plan to the user terminal, which displays the plan and presents it to the user.

[0467] Enter: Optimal Life Plan

[0468] Output: Sending the life plan to the user's device

[0469] Step 7:

[0470] The user terminal supports store reservations and ticket purchase procedures based on the life plan. Reservation and purchase requests are sent to the server, which then cooperates with affiliated store systems to complete the procedures.

[0471] Input: User reservation request, purchase request

[0472] Output: Store reservation and ticket purchase confirmation information

[0473] Step 8:

[0474] The user executes the proposed life plan and provides feedback for each activity. The user terminal collects this feedback information and sends it to the server. The server stores the collected feedback in a database and uses it to generate the next plan.

[0475] Input: User feedback data

[0476] Output: Save feedback data to database

[0477] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0478] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0479] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0480] [Second embodiment]

[0481] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0482] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0483] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0484] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0485] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0486] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0487] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0488] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0489] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0490] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0491] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0492] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0493] The present invention relates to an AI-based assistance system that helps users live an efficient and satisfying life within their budget, and can be embodied in the following forms.

[0494] Natural language description of the program

[0495] This system exchanges information between the user, the terminal, and the server, and generates an optimal life plan based on the user's preference information and past consumption data. The specific program processing is explained below.

[0496] 1. Entering user data

[0497] User: First, the user opens the dedicated application and enters their interests (hobbies, favorite foods, etc.), health information (current health condition), and consumption information (obtained through collaboration with external services such as PayPay) into the device.

[0498] Terminal: Receives input data and sends it to the server.

[0499] 2. Receipt and storage of data

[0500] Server: Receives user interest information, health information, and consumption information sent from the device and stores it in a database.

[0501] 3. AI-based data analysis

[0502] Server: Collects user interest, health, and consumption information from the database, and also obtains external information (such as weather and local event information) from the API.

[0503] Server: Analyzes all collected data using an AI model. Based on the analysis results, a life plan optimized for the user's budget and number of days is generated.

[0504] 4. Creating a life plan

[0505] Server: The life plan generated by the AI ​​model may include suggestions such as:

[0506] Planning a movie and the best time to go

[0507] Jogging schedule at a nearby park

[0508] Proposal for a Japanese restaurant that takes health into consideration

[0509] Server: The generated life plan includes detailed information about each activity (cost, location, and planned time).

[0510] 5. Presenting the plan

[0511] Server: Sends the generated life plan to the device.

[0512] Device: Displays the received plan to the user. The screen shows detailed information about each activity (cost, location, and planned time).

[0513] 6. Reservations and Purchases

[0514] User: Choose the activity they want to do from the proposed plan and make store reservations or purchase tickets as necessary.

[0515] Terminal: Sends reservation and purchase requests to the server.

[0516] Server: Receives reservation and purchase requests, coordinates with affiliated cinema and restaurant systems to complete reservations and purchases, and sends confirmation information to the device upon completion.

[0517] On your device: Display reservation confirmation information to the user and automatically add it to your calendar and reminders.

[0518] 7. Activity execution and feedback

[0519] User: Implements the proposed plan and actually carries out the activities.

[0520] Users: Provide ratings and feedback for each activity.

[0521] Terminal: Sends feedback information to the server.

[0522] Server: The received feedback is stored in a database and used as training data for the AI ​​model to improve the accuracy of future plan suggestions.

[0523] Specific examples

[0524] An embodiment of this system will be explained using a specific scenario. For example, suppose a user inputs information such as "watching movies," "jogging," and "likes Japanese food," and has a budget of 20,000 yen and 10 days remaining. The AI ​​analyzes this data and external weather information to generate the following life plan:

[0525] 1. "I'm planning to see two movies at the cinema this weekend."

[0526] 2. "Jogging in a nearby park on a sunny day."

[0527] 3. "I recommend eating at a Japanese restaurant after jogging."

[0528] The user confirms the proposed plan and reserves the movie ticket. The server completes the reservation through the connected cinema system and sends the confirmation information to the terminal. After watching the movie, the user provides feedback, which is used to generate the next plan.

[0529] In this way, the system of the present invention automatically generates an optimal life plan that takes into account the user's budget and time, and supports the execution of the plan.

[0530] The processing flow will be explained below.

[0531] Step 1:

[0532] User: Opens the application and enters their interests (hobbies, favorite foods, etc.) and health information (current health condition). They also set up the application to connect to external services (e.g., PayPay) to obtain past consumption data.

[0533] Step 2:

[0534] Terminal: Receives interest information, health information, and consumption information entered by the user and sends this data to the server.

[0535] Step 3:

[0536] Server: Receives user information (interest information, health information, consumption information) sent from the device and stores it in a database.

[0537] Step 4:

[0538] Server: Collects user interest, health, and consumption information from a database, and also uses external APIs to obtain real-time weather information and local event information.

[0539] Step 5:

[0540] Server: The collected data is input into the AI ​​model, which analyzes and generates an optimal life plan based on the user's budget and number of days. The AI ​​model takes into consideration the user's past consumption patterns, preferences, current weather information, etc.

[0541] Step 6:

[0542] Server: The generated life plan includes specific activity suggestions (e.g., watching a movie, jogging, or eating at a Japanese restaurant), along with detailed information on costs, locations, and planned times.

[0543] Step 7:

[0544] Server: Sends the generated life plan to the device.

[0545] Step 8:

[0546] Device: Displays the received life plan to the user. The screen shows detailed information about each activity (cost, location, and planned time).

[0547] Step 9:

[0548] User: Selects the activities they want to carry out from the proposed life plan and makes store reservations or purchases tickets as necessary.

[0549] Step 10:

[0550] Terminal: Sends reservation and purchase requests to the server.

[0551] Step 11:

[0552] Server: Receives reservation and purchase requests, coordinates with affiliated cinema and restaurant systems to complete reservations and purchases, and sends confirmation information to the device upon completion.

[0553] Step 12:

[0554] On your device: Display reservation confirmation information to the user and automatically add it to your calendar and reminders.

[0555] Step 13:

[0556] User: Carry out the proposed plan, for example, watch a movie at a cinema, go jogging in a park, or enjoy a meal at a Japanese restaurant.

[0557] Step 14:

[0558] Users: Provide ratings and feedback for each activity.

[0559] Step 15:

[0560] Terminal: Sends user feedback information to the server.

[0561] Step 16:

[0562] Server: The received feedback is stored in a database and used as training data for the AI ​​model to propose future plans, improving the accuracy of subsequent plan generation.

[0563] Example 1

[0564] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0565] In today's busy lifestyles, it is difficult for users to create a satisfying life plan that uses time efficiently and stays within a budget. In particular, plans must be created that take into account individual preferences, health status, and past consumption data. Weather information and local event information also need to be appropriately reflected, but collecting these individually and creating the optimal plan requires a huge amount of effort. To solve these problems, a system is needed that can automatically generate the optimal plan tailored to the user's situation and provide consistent support, from reservation to purchase procedures.

[0566] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0567] In this invention, the server includes means for inputting user interest information, health information, and consumption information, means for linking with the user's past consumption data and external information, means for generating an optimal life plan based on the user's budget and number of days, means for displaying detailed information of the generated life plan on a screen, means for collecting user feedback and saving it in a database, and means for training an AI model using the saved feedback and reflecting it in the generation of the next plan. This makes it possible to automatically generate an optimal life plan tailored to the user's individual circumstances and support reservations, purchases, and execution.

[0568] "User interest information" is information about activities and preferences in which the user is interested (for example, hobbies, favorite foods, events they want to enjoy, etc.).

[0569] "Health information" refers to the user's current health condition and health-related information (e.g., medical history, daily activity level, diet, etc.).

[0570] "Consumption information" refers to information about a user's past consumption data and purchase history (for example, expenditure details, services used, types and quantities of purchased products, etc.).

[0571] "Past consumption data" refers to a history of information about purchases and expenditures made by a user.

[0572] "External information" is external data related to the creation of the user's life plan (for example, weather information, local event information, etc.).

[0573] A "budget" is a limit on the amount of money available to a user to plan their spending within a particular time period.

[0574] "Number of days" is the unit of time period set by the user for carrying out a particular life plan.

[0575] A "life plan" is a plan of activities and schedules optimized within a user's budget and number of days based on the user's interest information, health information, consumption information, and external information.

[0576] "Store reservation" is a process in which a user makes a reservation in advance to visit or use a specific store.

[0577] "Ticket purchase" refers to the act of a user obtaining a ticket necessary for admission to a movie theater, event, or the like.

[0578] "Feedback" is information about evaluations and impressions provided by users regarding activities they have performed.

[0579] "Database" means an electronic information management system for organizing and storing collected user information and feedback.

[0580] An "AI model" is an artificial intelligence computational model that analyzes collected data and generates the optimal life plan for the user.

[0581] This invention relates to an AI-based support system that helps users live an efficient and satisfying life within their budget. The system of this invention exchanges information between the user, terminal, and server, and generates an optimal life plan based on the user's preference information and past consumption data.

[0582] Entering User Data

[0583] The user opens the dedicated application and inputs their interests (hobbies, favorite foods, etc.), health information (current health condition), and consumption information (obtained, for example, through collaboration with external services such as payment services). The device receives the input data and sends it to the server.

[0584] Receiving and storing data

[0585] The server receives the user's interest, health, and consumption information sent from the device and stores it in a database. The stored data undergoes error checking to ensure it is properly formatted.

[0586] AI-based data analysis

[0587] The server collects information on the user's interests, health, and consumption from a database, and also obtains external information (such as weather information and local event information) from an API. All collected data is analyzed using an AI model (such as a model using TensorFlow or Keras). Based on the analysis results, a life plan optimized for the user's budget and number of days is generated.

[0588] Creating a life plan

[0589] Based on the analysis results of the AI ​​model, the server generates an optimal life plan for the user. For example, it may include "movie viewing plans and the best time to do so," "a jogging schedule in a nearby park," and "recommendations for Japanese restaurants that take health into consideration." The generated life plan includes detailed information for each activity (cost, location, and planned time).

[0590] Presenting the plan

[0591] The server sends the generated life plan to the device, which then displays the plan to the user, showing detailed information about each activity (cost, location, and planned time).

[0592] Pre-order and purchase

[0593] The user selects the activity they want to carry out from the proposed plan and makes a store reservation or purchase a ticket. The device sends the reservation or purchase request to the server. The server receives the reservation or purchase request and connects with the systems of affiliated movie theaters and restaurants to make the reservation or purchase. Once completed, confirmation information is sent to the device. The device displays the reservation confirmation information to the user and automatically adds it to their calendar or reminders.

[0594] Activity execution and feedback

[0595] The user executes the proposed plan and performs the actual activities. The user generates ratings and feedback for each activity. The device sends the feedback information to the server. The server stores the received feedback in a database and uses it as training data for the AI ​​model to improve the accuracy of future plan suggestions.

[0596] Specific examples

[0597] An embodiment of this system will be explained using a specific scenario. For example, suppose a user inputs information such as "watching movies," "jogging," and "likes Japanese food," and has a budget of 20,000 yen and 10 days remaining. In this case, the AI ​​analyzes this data and external weather information to generate the following life plan:

[0598] 1. "I'm planning to see two movies at the cinema this weekend."

[0599] 2. "Jogging in a nearby park on a sunny day."

[0600] 3. "I recommend eating at a Japanese restaurant after jogging."

[0601] The user confirms the proposed plan and reserves the movie ticket. The server completes the reservation through the connected cinema system and sends the confirmation information to the terminal. After watching the movie, the user provides feedback, which is used to generate the next plan.

[0602] Example prompts for generative AI models

[0603] Below are some examples of prompts to input to an AI model:

[0604] "If the user likes watching movies, jogging, and eating Japanese food, generate the optimal life plan for a 10-day period with a budget of 20,000 yen."

[0605] "Please provide a recommended activity plan that takes into account the user's preferences (e.g., watching movies, jogging, Japanese food), current budget, and duration."

[0606] In this way, the system of the present invention automatically generates an optimal life plan that takes into account the user's budget and time, and supports the execution of that plan.

[0607] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0608] Step 1: Enter user data

[0609] User: Opens the dedicated application and enters information about their interests, health, and consumption. Examples of input data include a preference for watching movies, jogging, and Japanese food.

[0610] Terminal: Receives data entered by the user and sends this data to the server. The entered data is structured in JSON or XML format.

[0611] Step 2: Receiving and storing data

[0612] Server: Receives data sent from the device. The received content includes user interest information, health information, and consumption information.

[0613] Server: The received data is checked for errors to ensure it is properly formatted, and then saved to a database. The saved data is indexed and managed for each user.

[0614] Step 3: Gathering external information

[0615] Server: Calls external APIs to obtain external data such as weather information and local event information. The data obtained from the API is provided in the form of a JSON object containing weather forecasts and event information.

[0616] Server: The acquired external information is temporarily stored in memory and used for subsequent data analysis.

[0617] Step 4: Data analysis using AI

[0618] Server: Collects user interest information, health information, and consumption information from the database, and creates a dataset that also includes external information.

[0619] Server: Runs AI models using libraries such as TensorFlow and Keras to analyze the created datasets, using clustering algorithms and predictive models.

[0620] Server: As a result of the analysis, it generates an optimal life plan based on the user's budget and number of days. This plan includes detailed information about each activity (e.g., cost, location, and planned time).

[0621] Step 5: Create a life plan

[0622] Server: Based on the analysis results of the AI ​​model, the server generates an optimal life plan for the user, such as movie-going plans and the best times to do so, a jogging schedule in a nearby park, and recommendations for Japanese restaurants that take health into consideration.

[0623] Server: Formats the generated life plan and prepares it for transmission to the terminal.

[0624] Step 6: Present your plan

[0625] Server: Sends the generated life plan to the device. Push notifications may be used as a notification mechanism.

[0626] Terminal: Displays the user's life plan, with detailed information for each activity (cost, location, and planned time) presented in a user-friendly interface.

[0627] Step 7: Pre-order and purchase

[0628] User: Selects the activity they want to perform from the displayed list and makes a store reservation or purchases a ticket. For example, they click the "Go see this movie" button.

[0629] Terminal: Sends reservation or purchase requests to the server based on the user's selection.

[0630] Server: Processes the received request and executes the reservation or purchase in cooperation with the systems of affiliated stores and service providers. After the process is complete, it sends confirmation information to the terminal.

[0631] On the device: Display the reservation confirmation to the user and automatically add it to their calendar or reminders as appropriate.

[0632] Step 8: Activity execution and feedback

[0633] User: Executes the proposed plan and performs an actual activity, e.g., watches a movie at a cinema.

[0634] Users: Provide ratings and feedback for each activity, such as "The quality of the movie was great" or "The service at the Japanese restaurant was good."

[0635] Terminal: Sends feedback information to the server.

[0636] Server: The received feedback is stored in a database and used as learning data for the AI ​​model to improve the accuracy of the next plan proposal.

[0637] (Application example 1)

[0638] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0639] There is a lack of a means to automatically generate and present plans that take into account individual interests, health conditions, and consumption trends when users are trying to live an efficient and satisfying life within their budget. There is also a lack of support systems that allow users to easily make reservations and purchases at physical stores based on those plans. Furthermore, there is a need for a means to provide users with intuitive information in real time by displaying the generated plans on smart glasses or head-mounted displays.

[0640] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0641] In this invention, the server includes means for inputting user interest information, health information, and consumption information, means for linking with the user's past consumption data and external information, means for generating an optimal life plan based on the user's budget and number of days, means for presenting the generated life plan to the user, means for supporting reservations or purchases at physical stores based on the life plan, and means for displaying the generated life plan on smart glasses or a head-mounted display. This makes it possible to automatically generate plans tailored to the user's individual needs, simplify the reservation and purchase process, and provide information intuitively in real time.

[0642] "User interest information" is information about the user's hobbies and preferences.

[0643] "Health information" is information relating to the user's current health condition and past health checkup results.

[0644] "Consumption information" is information relating to a user's purchase history and spending patterns.

[0645] "Historical Consumption Data" is data recorded by a user that includes details of previous purchases and expenditures.

[0646] "External information" is data obtained from external sources, such as weather information and local event information.

[0647] A "budget" is a spending limit set by a user.

[0648] "Number of days" refers to the date set by the user as the validity period of the life plan.

[0649] The "optimal life plan" is a proposal of an optimal life plan for the user, generated based on the user's interest information, health information, consumption information, and external information.

[0650] "Physical store" refers to a physical store, such as a clothing store, restaurant, or movie theater.

[0651] A "reservation" is the act of reserving in advance the use or purchase of a specific service or product.

[0652] "Purchase" is the act of acquiring a service or product with money.

[0653] "Smart glasses" are eyeglass-type devices that have the function of displaying information in the wearer's field of vision.

[0654] A "head-mounted display" is a display worn on a user's head that provides virtual or augmented reality visual information.

[0655] "User feedback" is information regarding opinions and evaluations provided by users regarding the proposed life plan.

[0656] "Real-time" refers to processing and information being provided immediately at that moment.

[0657] The system embodying the present invention supports users in living an efficient and satisfying life within their budget, and involves cooperation between users, terminals, and a server. Specific embodiments of the system are described below.

[0658] First, a user launches an application using a device such as a smartphone, smart glasses, or head-mounted display. The user then inputs their interest information (hobbies, favorite foods, etc.), health information (health status), and consumption information (purchase history, etc.) into the application. The input information is received by the device and sent to the server.

[0659] The server stores the interest, health, and consumption information sent by the user in a database. The server also obtains external information such as weather and local event information via API. The AI ​​model on the server analyzes this data and generates an optimal life plan based on the user's budget and number of days.

[0660] The generated life plan includes specific activity suggestions, such as watching a movie, jogging, or eating at a Japanese restaurant, along with detailed information such as the cost, location, and estimated time for each activity. This life plan is then sent by the server to the device and presented to the user.

[0661] Users can select the desired activity from the proposed life plan and make a reservation or purchase a ticket at a physical store. The device sends the reservation or purchase request to the server, which then completes the reservation or purchase by linking with the systems of affiliated stores and services. Once completed, confirmation information is sent to the device and displayed to the user. If using smart glasses or a head-mounted display, the information is displayed in real time, allowing the user to receive the information intuitively.

[0662] In addition, users can provide ratings and feedback on the activities they have performed. This feedback information is also sent to the server and stored in a database. The server uses this feedback as learning data for the AI ​​model and reflects it in the next life plan generation, thereby improving the accuracy of suggestions.

[0663] For example, if a user inputs information such as "watching movies," "jogging," and "likes Japanese food," along with a budget of 20,000 yen and 10 days remaining, the system will generate the following life plan:

[0664] 1. "I'm planning to see two movies at the cinema this weekend."

[0665] 2. "Jogging in a nearby park on a sunny day."

[0666] 3. "I recommend eating at a Japanese restaurant after jogging."

[0667] An example of a prompt sentence in this case is: "The user likes watching movies, jogging, and Japanese food. The budget is 20,000 yen and there are 10 days left. The weather is sunny on weekends and cloudy or rainy on other days. Please generate the optimal activity plan."

[0668] These steps allow users to automatically generate personalized plans, simplify the booking and purchasing process, and provide intuitive information in real time.

[0669] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0670] Step 1:

[0671] A user launches an application using a smartphone or smart glasses and inputs information about their interests, health, and consumption. This information is received by the device as user input. For example, the user inputs information such as "watching movies," "jogging," and "I like Japanese food." The device then sends the input data to the server.

[0672] Step 2:

[0673] The server receives the user's interest, health, and consumption information sent from the device and stores it in a database. At the same time, the server uses an external API to obtain weather information and local event information. This external information is also stored in the database. Input data includes the user's preferences and health status, past consumption data, weather information, and event information.

[0674] Step 3:

[0675] The AI ​​model on the server collects and analyzes the user's interest, health, consumption, and external information from a database. This analysis generates an optimal life plan based on the user's budget and number of days. For example, given a user's budget of 20,000 yen and 10 days remaining, the AI ​​model suggests activities such as watching a movie, jogging, and eating at a Japanese restaurant.

[0676] Step 4:

[0677] The server sends the generated life plan to the device, which receives it and displays it to the user. The displayed content includes detailed information about each activity (e.g., movie viewing plans, jogging time, location and cost of Japanese restaurants, etc.).

[0678] Step 5:

[0679] The user selects the desired activity from the proposed life plan. Based on the selected activity, the user makes a reservation or purchase request at a physical store. The device sends these requests to the server.

[0680] Step 6:

[0681] The server receives reservation and purchase requests sent by users and completes the reservation or purchase in cooperation with the systems of affiliated stores and services. After completion, reservation confirmation information is sent to the device. The device receives this information and displays it to the user. If smart glasses or a head-mounted display are used, this confirmation information is also displayed in real time.

[0682] Step 7:

[0683] Users provide ratings and feedback for the activities they perform. The device collects this feedback information and sends it to the server. The server stores the feedback information in a database and uses it as learning data for the AI ​​model to improve the accuracy of future life plan suggestions.

[0684] In this way, all steps work together to enable users to realize the optimal life plan while responding to their individual needs and using their budget and time efficiently.

[0685] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0686] The present invention relates to an AI-based support system for helping users live an efficient and satisfying life within their budget, and in particular to improving the accuracy of life plans by incorporating an emotion engine. Hereinafter, embodiments of the present invention will be described in detail.

[0687] Natural language description of the program

[0688] This system exchanges information between the user, device, and server, and not only generates an optimal life plan based on the user's preferences and past consumption data, but also has the ability to recognize the user's emotions in real time and adjust the plan accordingly.

[0689] 1. Entering user data

[0690] User: Opens the application and enters their interests (hobbies, favorite foods, etc.) and health information (current health condition). For emotion recognition, the application also allows voice input and facial recognition. The application also allows users to connect to external services (e.g., PayPay) to obtain past consumption data.

[0691] Terminal: Receives input interest information, health information, and consumption information and sends it to the server.

[0692] 2. Receipt and storage of data

[0693] Server: Receives user information (interest information, health information, consumption information) and emotion data sent from the device and stores it in a database.

[0694] 3. AI-based data analysis

[0695] Server: Collects user interest, health, and consumption information from a database, and also uses external APIs to obtain real-time weather information and local event information.

[0696] Server: Using an emotion engine, the server analyzes the voice, facial expressions, and text data sent from the device in real time to recognize the user's emotional state.

[0697] 4. Creating a life plan

[0698] Server: The collected data and the analysis results of the emotion engine are input into the AI ​​model to generate an optimal life plan based on the user's budget and number of days. The AI ​​model takes into consideration the user's past consumption patterns, preferences, current emotional state, weather information, etc.

[0699] 5. Plan generation and adjustment

[0700] Server: The life plan generated by the AI ​​model includes specific activity suggestions (e.g., watching a movie, jogging, or eating at a Japanese restaurant) along with detailed information about costs, locations, and planned times.

[0701] Server: Reflects the user's real-time emotional state and adjusts the plan, for example, by suggesting a plan that emphasizes relaxation activities if the user is under high stress, or by suggesting more active activities if the user is in a good mood.

[0702] 6. Presenting the plan

[0703] Server: Sends the generated life plan to the device.

[0704] Device: Displays the received life plan to the user. The screen shows detailed information about each activity (cost, location, planned time) and adjustment points based on the user's emotional state.

[0705] 7. Reservations and Purchases

[0706] User: Selects the activities they want to carry out from the proposed life plan and makes store reservations or purchases tickets as necessary.

[0707] Terminal: Sends reservation and purchase requests to the server.

[0708] Server: Receives reservation and purchase requests, coordinates with affiliated cinema and restaurant systems to complete reservations and purchases, and sends confirmation information to the device upon completion.

[0709] On your device: Display reservation confirmation information to the user and automatically add it to your calendar and reminders.

[0710] 8. Activity execution and feedback

[0711] User: Carry out the proposed plan, for example, watch a movie at a cinema, go jogging in a park, or enjoy a meal at a Japanese restaurant.

[0712] Users: Provide ratings and feedback for each activity.

[0713] Terminal: Sends user feedback information to the server.

[0714] Server: The received feedback is stored in a database and used as training data for the AI ​​model to propose future plans, improving the accuracy of subsequent plan generation.

[0715] Specific examples

[0716] An embodiment of this system will be explained using a specific scenario. For example, suppose a user inputs information such as "watching movies," "jogging," and "likes Japanese food," has a budget of 20,000 yen, and has 10 days left. The AI ​​analyzes this data and external weather information, and the emotion engine recognizes that the user's current stress level is high, and generates the following life plan:

[0717] 1. "I'm planning to see two movies at the cinema this weekend."

[0718] 2. "Jogging in a nearby park on a sunny day."

[0719] 3. "I recommend eating at a Japanese restaurant after jogging."

[0720] 4. "We also suggest visiting a relaxing hot spring facility to reduce stress."

[0721] The user checks the proposed plan and reserves movie tickets. The server completes the reservation through the connected cinema system and sends the confirmation information to the terminal. After watching the movie, the user provides feedback, which is used to generate the next plan. The emotion engine analyzes the user's emotions in real time, and the system always provides a plan optimized for the user's emotional state.

[0722] In this way, the system of the present invention takes into consideration the user's budget and time, and also analyzes and reflects their emotional state in real time to automatically generate an optimal life plan and support its implementation.

[0723] The processing flow will be explained below.

[0724] Step 1:

[0725] User: Opens the application and enters their interests (hobbies, favorite foods, etc.) and health information (current health condition). In addition, they authorize voice input and face recognition functions, and set up the application to connect to external services (e.g., PayPay) to obtain past consumption data.

[0726] Step 2:

[0727] Terminal: Receives interest information, health information, and consumption information entered by the user and sends this data to the server.

[0728] Step 3:

[0729] Server: Receives user information (interest information, health information, consumption information) and emotion data sent from the device and stores it in a database.

[0730] Step 4:

[0731] Server: Collects user interest, health, and consumption information from a database, and also uses external APIs to obtain real-time weather information and local event information.

[0732] Step 5:

[0733] Server: Using an emotion engine, the server analyzes the voice, facial expressions, and text data sent from the device in real time to recognize the user's emotional state.

[0734] Step 6:

[0735] Server: The collected data and the analysis results of the emotion engine are input into the AI ​​model to generate an optimal life plan based on the user's budget and number of days. The AI ​​model takes into consideration the user's past consumption patterns, preferences, current emotional state, weather information, etc.

[0736] Step 7:

[0737] Server: The generated life plan includes specific activity suggestions (e.g., watching a movie, jogging, or eating at a Japanese restaurant) with detailed information about costs, locations, and estimated times.

[0738] Step 8:

[0739] Server: Reflects the user's real-time emotional state and adjusts the plan, for example, by suggesting a plan that emphasizes relaxation activities if the user is under high stress, or by suggesting more active activities if the user is in a good mood.

[0740] Step 9:

[0741] Server: Sends the generated life plan to the device.

[0742] Step 10:

[0743] Device: Displays the received life plan to the user. The screen shows detailed information about each activity (cost, location, planned time) and adjustment points based on the user's emotional state.

[0744] Step 11:

[0745] User: Selects the activities they want to carry out from the proposed life plan and makes store reservations or purchases tickets as necessary.

[0746] Step 12:

[0747] Terminal: Sends reservation and purchase requests to the server.

[0748] Step 13:

[0749] Server: Receives reservation and purchase requests, coordinates with affiliated cinema and restaurant systems to complete reservations and purchases, and sends confirmation information to the device upon completion.

[0750] Step 14:

[0751] On your device: Display reservation confirmation information to the user and automatically add it to your calendar and reminders.

[0752] Step 15:

[0753] User: Carry out the proposed plan, for example, watch a movie at a cinema, go jogging in a park, or enjoy a meal at a Japanese restaurant.

[0754] Step 16:

[0755] Users: Provide ratings and feedback for each activity.

[0756] Step 17:

[0757] Terminal: Sends user feedback information to the server.

[0758] Step 18:

[0759] Server: The received feedback is stored in a database and used as training data for the AI ​​model to propose future plans, improving the accuracy of subsequent plan generation.

[0760] Example 2

[0761] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0762] When generating an efficient and satisfying life plan within a user-specified budget and number of days, there was a problem in that the system was unable to recognize the user's emotional state in real time and adjust the plan accordingly. It was also difficult to dynamically generate plans that incorporated weather information and local event information, and it was also a challenge to efficiently utilize the user's personal preferences and past consumption data. Another problem was the lack of a feedback loop to collect user feedback and reflect it in the next plan.

[0763] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0764] In this invention, the server includes: means for inputting user interest information, health information, and consumption information; means for linking with the user's past consumption data and external information; means for allowing voice input and facial recognition for emotion recognition; means for generating an optimal life plan based on the user's budget and number of days; means for analyzing the user's emotional state in real time and adjusting the life plan based on the analysis; means for presenting the generated life plan to the user; means for assisting with store reservations or ticket purchases based on the life plan; and means for collecting user feedback and reflecting it in the generation of the next plan. This allows for the generation of an optimized life plan in real time, taking the user's emotional state into consideration. Furthermore, incorporating weather information and local event information allows for the provision of more dynamic and personalized plans. Furthermore, by creating a feedback loop, the accuracy of subsequent plans can be improved, resulting in an overall improved user experience.

[0765] "Interest information" is information about the user's hobbies, preferences, and tastes.

[0766] "Health information" refers to information related to health, such as the user's current health condition and medical history.

[0767] "Consumption information" is data related to consumption activities that a user has engaged in in the past.

[0768] "External information" refers to information obtained from external databases or APIs, such as weather information and local event information.

[0769] "Emotion recognition" is a technology that analyzes a user's voice and facial expressions to determine their current emotional state.

[0770] "Voice input" is a means of capturing user voice data as input.

[0771] "Facial recognition" is a technology that analyzes a user's facial features and identifies them as an individual.

[0772] A "life plan" is a specific action plan proposed based on the user's budget and number of days.

[0773] "Real-time" means that data is analyzed immediately and necessary processing is carried out at that moment.

[0774] "Analysis" is the process of examining data in detail and deriving patterns and trends.

[0775] "Store reservation" refers to making a reservation in advance at a store that the user plans to visit.

[0776] "Ticket purchase" means the purchase of an admission ticket to participate in an event or activity.

[0777] "Feedback" refers to opinions and evaluations provided by users, and is information that can be used to improve services next time.

[0778] This invention is an AI-based support system that allows users to generate and adjust an optimal life plan within their budget and time frame. This system consists of three main components: the user, the device, and the server. It analyzes the user's emotional state in real time based on their interest, health, and consumption information, and provides a life plan that reflects that information.

[0779] Entering User Data

[0780] User: The user opens the application and enters their interest information (such as hobbies and favorite dishes), health information, and consumption information. They also configure the application to enable voice input and facial recognition, which provides basic data for emotion recognition. They also configure the application to connect to external services (e.g., electronic payment systems) and grant permission to obtain past consumption data.

[0781] Receiving and storing data

[0782] Terminal: The terminal sends all data entered by the user to the server. It also validates the user's input to ensure that the data is accurate.

[0783] Server: The server receives the data sent from the device and stores it in a database. The stored data is used for further analysis and calculation.

[0784] AI-based data analysis

[0785] Server: The server collects user information from a database and, if necessary, uses external APIs to obtain real-time weather information and local event information. It also uses an emotion engine to analyze voice, facial expressions, and text data to recognize the user's emotional state. Emotion recognition uses voice recognition algorithms and image recognition algorithms.

[0786] Creating and adjusting life plans

[0787] Server: Based on the collected data and the analysis results of the emotion engine, the data is input into the AI ​​model to generate an optimal life plan based on the user's budget and number of days. The AI ​​model takes into account past spending patterns, interests, current emotional state, weather information, etc. The generated plan includes specific activities such as going to the movies, jogging, and eating at a Japanese restaurant, along with the cost, location, and planned time for each.

[0788] Presenting and adjusting plans

[0789] Server: Sends the generated life plan to the device. The plan reflects the user's real-time emotional state and is dynamically adjusted according to the specific situation.

[0790] Device: Displays the received life plan to the user, along with detailed information on each activity and adjustment points.

[0791] Reservation and purchase assistance

[0792] User: The user selects the activities they want to carry out from the proposed life plan and makes store reservations or purchases tickets as necessary.

[0793] Terminal: Sends reservation and purchase requests to the server.

[0794] Server: The server receives reservation and purchase requests, executes them in cooperation with partner systems, and sends confirmation information to the terminal once completed.

[0795] On the device: Display reservation confirmation information to the user and automatically add it to their calendar or reminders.

[0796] Activity execution and feedback

[0797] User: Carry out the proposed plan and enjoy watching a movie, jogging, and eating at a Japanese restaurant.

[0798] Users: Provide ratings and feedback for each activity.

[0799] Terminal: Sends feedback information to the server.

[0800] Server: The received feedback is stored in a database and used as training data for the AI ​​model to propose future plans, improving the accuracy of subsequent plan generation.

[0801] Specific examples

[0802] For example, suppose a user inputs information such as "watching movies," "jogging," and "likes Japanese food," and requests a 10-day plan with a budget of ¥20,000. The system takes this data into account along with real-time weather information, and the emotion engine recognizes that the user's current stress level is high, generating the following life plan:

[0803] 1. "I'm going to see two movies at the cinema this weekend."

[0804] 2. "Jogging in a nearby park on a sunny day."

[0805] 3. "Recommend a meal at a Japanese restaurant after jogging."

[0806] 4. "We also suggest visiting a relaxing hot spring facility to reduce stress."

[0807] The user checks the proposed plan and reserves movie tickets. The server completes the reservation with the affiliated cinema system and sends the confirmation information to the terminal. After watching the movie, the user provides feedback, which is reflected in the next plan generation. The emotion engine analyzes the user's emotions in real time, so that the system always provides the most optimized plan.

[0808] Prompt Sentence Examples

[0809] "Watching movies," "Jogging," "Like Japanese food," "Budget is 20,000 yen," "Prefer a 10-day plan," "Recognizes and adjusts emotions in real time," "Includes relaxing activities."

[0810] In this way, the present invention automatically generates an optimal life plan that takes into account the user's budget and time, and also analyzes and reflects their emotional state in real time, and supports their execution.

[0811] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0812] Step 1: Enter user data

[0813] The user opens the application and inputs interest, health, and consumption information. The user also sets permissions for voice input and facial recognition. Furthermore, the user sets up connections with external services (e.g., electronic payment systems) and gives permission to obtain past consumption data.

[0814] Input: Interest information (hobbies, preferences), health information, consumption information, voice input permission, face recognition permission, external service linkage permission

[0815] Output: User dataset to send to terminal

[0816] What happens: A user enters information into an application form, and after completing each field, presses a confirmation button. The form data is packaged in JSON format and ready to be sent to the device.

[0817] Step 2: Receiving and storing data

[0818] The terminal transmits the data entered by the user to the server, where it verifies the consistency and integrity of the data and performs error checking before transmission.

[0819] The server receives the data sent from the terminal, verifies it, and then stores it in a database.

[0820] Input: A user-entered dataset

[0821] Output: User data stored in the database

[0822] Specific operation: The terminal sends an HTTP POST request to the server, the server receives the data, parses it, checks for errors and validates the data format, and then stores it in the database.

[0823] Step 3: Data analysis using AI

[0824] The server collects user information from a database and also uses external APIs to obtain current weather information and local event information.

[0825] The server uses an emotion engine to analyze the voice data, facial expression data, and text data sent from the terminal in real time and identify the user's emotional state.

[0826] Input: User information collected from the database, weather information obtained from external APIs, local event information, emotion data

[0827] Output: User's emotional state as an analysis result and supplementary information

[0828] Specific operation: The server executes a scheduled task to obtain user information and sends a request to an external API (e.g., weather API). The obtained data is input into the emotion engine for real-time analysis.

[0829] Step 4: Create a life plan

[0830] The server inputs the collected data and the results of the emotion engine's analysis into an AI model to generate an optimal life plan based on the user's budget and number of days.

[0831] Input: Collected user data, emotion engine analysis results, external API data

[0832] Output: Generated life plan

[0833] Specific actions: Input data into the AI ​​model and run inference. The generated life plan includes specific activity suggestions, with details (cost, location, and planned time) automatically calculated and entered.

[0834] Step 5: Present and adjust the plan

[0835] The server sends the generated life plan to the device, which adjusts the plan as needed to reflect the user's real-time emotional state.

[0836] The terminal displays the received life plan to the user.

[0837] Input: Generated life plan

[0838] Output: The adjusted life plan displayed to the user

[0839] Specific operation: The server sends plan data in JSON format to the device, which receives the data and displays it on the interface. Plan adjustments are made based on the results of real-time sentiment analysis, and adjustment points are presented to the user.

[0840] Step 6: Reservation and purchase assistance

[0841] Users can select the activities they want to carry out from the proposed life plan and make reservations or purchase tickets.

[0842] The terminal sends a reservation request or a purchase request to the server.

[0843] The server cooperates with the affiliated system to execute the reservation or purchase and transmits confirmation information to the terminal.

[0844] The device will display the reservation confirmation information to the user and automatically add it to their calendar or reminders.

[0845] Input: User-selected activity booking / purchase request

[0846] Output: Confirmation information and automatic addition to calendar and reminders

[0847] Specific operation: The user selects an activity and enters the required information in the reservation form. The device sends a request to the server, and the server returns the execution result to the device. The device displays a confirmation and automatically adds it to the calendar and reminders.

[0848] Step 7: Activity execution and feedback

[0849] The user carries out the proposed plan and enjoys watching a movie, jogging, and eating at a Japanese restaurant.

[0850] Users provide ratings and feedback for each activity.

[0851] The terminal transmits the feedback information to the server.

[0852] The server stores the received feedback in a database and uses it as training data for the AI ​​model to propose future plans.

[0853] Input: Feedback data after the user performs an activity

[0854] Output: Feedback stored in the database as training data

[0855] Specific operation: The user fills out a feedback form, which the device sends to the server. The server stores the feedback in a database and uses it to generate the next life plan.

[0856] (Application example 2)

[0857] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0858] In modern society, autonomous vehicles offer significant improvements in user comfort and efficiency, but their capabilities are not being fully utilized. Furthermore, there are many challenges in providing optimal life plans that take into account the user's emotional state and real-time information. Specifically, plans are not adequately generated based on the user's interests and health status, and real-time adjustments based on emotion recognition are insufficient. Furthermore, conventional systems have difficulty generating route proposals that take into account weather and event information. Therefore, effective means of improving user satisfaction and comfort are needed.

[0859] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0860] In this invention, the server includes means for recognizing the user's emotional state in real time and adjusting the life plan, means for generating and presenting route proposals based on the user's emotions and real-time information within the autonomous vehicle, and means for acquiring weather information and event information using an external API, thereby enabling the generation of a life plan and route proposals optimized for the user's real-time emotional state and external environment.

[0861] "User interest information" is data that indicates the user's preferred activities, hobbies, and interests.

[0862] "Health information" is data relating to the user's current health condition and past health history.

[0863] "Consumption information" is data indicating products and services purchased by a user in the past, the amount spent on those products and services, and so on.

[0864] "Past consumption data" refers to data including the user's past purchasing behavior and its history.

[0865] "External information" refers to information obtained from outside the system, such as weather, local events, and traffic conditions.

[0866] A "budget" is data that indicates a range of available amounts set by the user.

[0867] "Number of days" is data indicating the number of days within a specific period set by the user.

[0868] The "means for generating a life plan" is a process for creating a plan for the user to live an efficient and satisfying life based on collected user data and external information.

[0869] The "means for presenting a life plan to a user" refers to a method for visually or audibly displaying or notifying a user of a created life plan.

[0870] "Means for supporting store reservations or ticket purchases" are methods that allow users to easily make reservations or purchases based on the proposed life plan.

[0871] "Means for recognizing emotional state in real time" refers to technology that analyzes the user's voice and facial expressions to determine their current emotions.

[0872] The "means for generating and presenting route proposals" is a technology that plans an optimal travel route based on the user's emotional state and real-time environmental information, and displays it on the vehicle terminal.

[0873] "Means of obtaining weather and event information using external APIs" refers to a method of obtaining real-time weather and local event information by linking with external data provision services.

[0874] The system of this invention inputs a user's interest information, health information, and consumption information, and generates an optimal life plan based on the budget and number of days in conjunction with past consumption data and external information. The system also incorporates an emotion engine to recognize the user's real-time emotional state and dynamically adjust the life plan.

[0875] Hardware and Software Configuration

[0876] The server performs various data processing and calculations using a database, AI analysis engine, emotion engine, external API connection module, etc. These components will be explained in detail below.

[0877] 1. User Device:

[0878] Input devices (smartphones, tablets, etc.)

[0879] Emotion recognition devices (camera, microphone, etc.)

[0880] 2. Server:

[0881] Database (SQL, etc.)

[0882] AI analysis engine (TensorFlow, Keras, etc.)

[0883] Emotion Engine API

[0884] External API connection module (weather API, event API)

[0885] Data processing and calculation procedures

[0886] The server first receives interest, health, and consumption information from the user's device and stores it in a database. This information, along with real-time weather information and local event information obtained through external APIs, is then input into an AI analysis engine.

[0887] The server uses an emotion engine to analyze the user's voice, facial expressions, and input text in real time to recognize the user's emotional state. The recognized emotional data, along with other data, becomes the input data required to generate a life plan.

[0888] The generated life plan includes specific activity suggestions based on the user's schedule, activities, and budget. For example, the system suggests optimal routes based on the user's emotional state and real-time information in an autonomous vehicle. The system also supports users in making reservations and purchasing tickets, and the server handles these procedures.

[0889] Specific examples

[0890] For example, suppose a user enters information into the app such as "I want to relax" and "I like Japanese food," and sets a budget of 20,000 yen. The server generates the following life plan based on real-time weather information and local event information:

[0891] 1. "Plan a relaxing picnic in the park on a sunny day."

[0892] 2. "I suggest we go to a Japanese restaurant afterwards."

[0893] 3. "We recommend visiting a hot spring facility to relax in the evening."

[0894] If the emotion engine recognizes that the user's stress level is high, it will focus on suggesting more relaxing activities.

[0895] Prompt Sentence Examples

[0896] "The user wants to relax, so generate a route for stress reduction."

[0897] "Generate tonight's dinner plans for the user. My budget is 5000 yen and I'm in the mood to relax."

[0898] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0899] Step 1:

[0900] The user's device inputs information about the user's interests, health, and consumption. The user enters this data into the app from their smartphone or tablet and sends it to the server. Input may also be done using the user's voice or facial recognition function.

[0901] Input: User interest information, health information, consumption information

[0902] Output: Send data to the server

[0903] Step 2:

[0904] The server receives the information sent from the user terminal and stores it in a database. Based on this data, the server aggregates the data necessary to generate a life plan.

[0905] Input: User interest information, health information, consumption information

[0906] Output: Save data to a database

[0907] Step 3:

[0908] The server uses external APIs to obtain real-time weather and event information. For this purpose, it uses the weather API and event API.

[0909] Input: Real-time information request from an external API

[0910] Output: Get real-time weather and event information

[0911] Step 4:

[0912] The server uses an emotion engine to analyze the user's voice, facial expressions, and text in real time to recognize the user's emotional state, thereby determining the user's stress level and mood.

[0913] Input: User's voice data, facial expression data, text data

[0914] Output: Emotional state data

[0915] Step 5:

[0916] The server uses an AI analysis engine to generate an optimal life plan based on the user's data stored in the database, as well as real-time weather information, event information, and emotional state. The generated life plan includes the user's budget and details of each activity.

[0917] Input: User data, real-time weather information, event information, emotional state data

[0918] Output: Optimal Life Plan

[0919] Step 6:

[0920] The server transmits the generated life plan to the user terminal, which displays the plan and presents it to the user.

[0921] Enter: Optimal Life Plan

[0922] Output: Sending the life plan to the user's device

[0923] Step 7:

[0924] The user terminal supports store reservations and ticket purchase procedures based on the life plan. Reservation and purchase requests are sent to the server, which then cooperates with affiliated store systems to complete the procedures.

[0925] Input: User reservation request, purchase request

[0926] Output: Store reservation and ticket purchase confirmation information

[0927] Step 8:

[0928] The user executes the proposed life plan and provides feedback for each activity. The user terminal collects this feedback information and sends it to the server. The server stores the collected feedback in a database and uses it to generate the next plan.

[0929] Input: User feedback data

[0930] Output: Save feedback data to database

[0931] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0932] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0933] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0934] [Third embodiment]

[0935] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0936] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0937] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0938] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0939] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0940] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0941] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0942] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0943] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0944] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0945] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0946] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0947] The present invention relates to an AI-based assistance system that helps users live an efficient and satisfying life within their budget, and can be embodied in the following forms.

[0948] Natural language description of the program

[0949] This system exchanges information between the user, the terminal, and the server, and generates an optimal life plan based on the user's preference information and past consumption data. The specific program processing is explained below.

[0950] 1. Entering user data

[0951] User: First, the user opens the dedicated application and enters their interests (hobbies, favorite foods, etc.), health information (current health condition), and consumption information (obtained through collaboration with external services such as PayPay) into the device.

[0952] Terminal: Receives input data and sends it to the server.

[0953] 2. Receipt and storage of data

[0954] Server: Receives user interest information, health information, and consumption information sent from the device and stores it in a database.

[0955] 3. AI-based data analysis

[0956] Server: Collects user interest, health, and consumption information from the database, and also obtains external information (such as weather and local event information) from the API.

[0957] Server: Analyzes all collected data using an AI model. Based on the analysis results, a life plan optimized for the user's budget and number of days is generated.

[0958] 4. Creating a life plan

[0959] Server: The life plan generated by the AI ​​model may include suggestions such as:

[0960] Planning a movie and the best time to go

[0961] Jogging schedule at a nearby park

[0962] Proposal for a Japanese restaurant that takes health into consideration

[0963] Server: The generated life plan includes detailed information about each activity (cost, location, and planned time).

[0964] 5. Presenting the plan

[0965] Server: Sends the generated life plan to the device.

[0966] Device: Displays the received plan to the user. The screen shows detailed information about each activity (cost, location, and planned time).

[0967] 6. Reservations and Purchases

[0968] User: Choose the activity they want to do from the proposed plan and make store reservations or purchase tickets as necessary.

[0969] Terminal: Sends reservation and purchase requests to the server.

[0970] Server: Receives reservation and purchase requests, coordinates with affiliated cinema and restaurant systems to complete reservations and purchases, and sends confirmation information to the device upon completion.

[0971] On your device: Display reservation confirmation information to the user and automatically add it to your calendar and reminders.

[0972] 7. Activity execution and feedback

[0973] User: Implements the proposed plan and actually carries out the activities.

[0974] Users: Provide ratings and feedback for each activity.

[0975] Terminal: Sends feedback information to the server.

[0976] Server: The received feedback is stored in a database and used as training data for the AI ​​model to improve the accuracy of future plan suggestions.

[0977] Specific examples

[0978] An embodiment of this system will be explained using a specific scenario. For example, suppose a user inputs information such as "watching movies," "jogging," and "likes Japanese food," and has a budget of 20,000 yen and 10 days remaining. The AI ​​analyzes this data and external weather information to generate the following life plan:

[0979] 1. "I'm planning to see two movies at the cinema this weekend."

[0980] 2. "Jogging in a nearby park on a sunny day."

[0981] 3. "I recommend eating at a Japanese restaurant after jogging."

[0982] The user confirms the proposed plan and reserves the movie ticket. The server completes the reservation through the connected cinema system and sends the confirmation information to the terminal. After watching the movie, the user provides feedback, which is used to generate the next plan.

[0983] In this way, the system of the present invention automatically generates an optimal life plan that takes into account the user's budget and time, and supports the execution of the plan.

[0984] The processing flow will be explained below.

[0985] Step 1:

[0986] User: Opens the application and enters their interests (hobbies, favorite foods, etc.) and health information (current health condition). They also set up the application to connect to external services (e.g., PayPay) to obtain past consumption data.

[0987] Step 2:

[0988] Terminal: Receives interest information, health information, and consumption information entered by the user and sends this data to the server.

[0989] Step 3:

[0990] Server: Receives user information (interest information, health information, consumption information) sent from the device and stores it in a database.

[0991] Step 4:

[0992] Server: Collects user interest, health, and consumption information from a database, and also uses external APIs to obtain real-time weather information and local event information.

[0993] Step 5:

[0994] Server: The collected data is input into the AI ​​model, which analyzes and generates an optimal life plan based on the user's budget and number of days. The AI ​​model takes into consideration the user's past consumption patterns, preferences, current weather information, etc.

[0995] Step 6:

[0996] Server: The generated life plan includes specific activity suggestions (e.g., watching a movie, jogging, or eating at a Japanese restaurant), along with detailed information on costs, locations, and planned times.

[0997] Step 7:

[0998] Server: Sends the generated life plan to the device.

[0999] Step 8:

[1000] Device: Displays the received life plan to the user. The screen shows detailed information about each activity (cost, location, and planned time).

[1001] Step 9:

[1002] User: Selects the activities they want to carry out from the proposed life plan and makes store reservations or purchases tickets as necessary.

[1003] Step 10:

[1004] Terminal: Sends reservation and purchase requests to the server.

[1005] Step 11:

[1006] Server: Receives reservation and purchase requests, coordinates with affiliated cinema and restaurant systems to complete reservations and purchases, and sends confirmation information to the device upon completion.

[1007] Step 12:

[1008] On your device: Display reservation confirmation information to the user and automatically add it to your calendar and reminders.

[1009] Step 13:

[1010] User: Carry out the proposed plan, for example, watch a movie at a cinema, go jogging in a park, or enjoy a meal at a Japanese restaurant.

[1011] Step 14:

[1012] Users: Provide ratings and feedback for each activity.

[1013] Step 15:

[1014] Terminal: Sends user feedback information to the server.

[1015] Step 16:

[1016] Server: The received feedback is stored in a database and used as training data for the AI ​​model to propose future plans, improving the accuracy of subsequent plan generation.

[1017] Example 1

[1018] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1019] In today's busy lifestyles, it is difficult for users to create a satisfying life plan that uses time efficiently and stays within a budget. In particular, plans must be created that take into account individual preferences, health status, and past consumption data. Weather information and local event information also need to be appropriately reflected, but collecting these individually and creating the optimal plan requires a huge amount of effort. To solve these problems, a system is needed that can automatically generate the optimal plan tailored to the user's situation and provide consistent support, from reservation to purchase procedures.

[1020] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1021] In this invention, the server includes means for inputting user interest information, health information, and consumption information, means for linking with the user's past consumption data and external information, means for generating an optimal life plan based on the user's budget and number of days, means for displaying detailed information of the generated life plan on a screen, means for collecting user feedback and saving it in a database, and means for training an AI model using the saved feedback and reflecting it in the generation of the next plan. This makes it possible to automatically generate an optimal life plan tailored to the user's individual circumstances and support reservations, purchases, and execution.

[1022] "User interest information" is information about activities and preferences in which the user is interested (for example, hobbies, favorite foods, events they want to enjoy, etc.).

[1023] "Health information" refers to the user's current health condition and health-related information (e.g., medical history, daily activity level, diet, etc.).

[1024] "Consumption information" refers to information about a user's past consumption data and purchase history (for example, expenditure details, services used, types and quantities of purchased products, etc.).

[1025] "Past consumption data" refers to a history of information about purchases and expenditures made by a user.

[1026] "External information" is external data related to the creation of the user's life plan (for example, weather information, local event information, etc.).

[1027] A "budget" is a limit on the amount of money available to a user to plan their spending within a particular time period.

[1028] "Number of days" is the unit of time period set by the user for carrying out a particular life plan.

[1029] A "life plan" is a plan of activities and schedules optimized within a user's budget and number of days based on the user's interest information, health information, consumption information, and external information.

[1030] "Store reservation" is a process in which a user makes a reservation in advance to visit or use a specific store.

[1031] "Ticket purchase" refers to the act of a user obtaining a ticket necessary for admission to a movie theater, event, or the like.

[1032] "Feedback" is information about evaluations and impressions provided by users regarding activities they have performed.

[1033] "Database" means an electronic information management system for organizing and storing collected user information and feedback.

[1034] An "AI model" is an artificial intelligence computational model that analyzes collected data and generates the optimal life plan for the user.

[1035] This invention relates to an AI-based support system that helps users live an efficient and satisfying life within their budget. The system of this invention exchanges information between the user, terminal, and server, and generates an optimal life plan based on the user's preference information and past consumption data.

[1036] Entering User Data

[1037] The user opens the dedicated application and inputs their interests (hobbies, favorite foods, etc.), health information (current health condition), and consumption information (obtained, for example, through collaboration with external services such as payment services). The device receives the input data and sends it to the server.

[1038] Receiving and storing data

[1039] The server receives the user's interest, health, and consumption information sent from the device and stores it in a database. The stored data undergoes error checking to ensure it is properly formatted.

[1040] AI-based data analysis

[1041] The server collects information on the user's interests, health, and consumption from a database, and also obtains external information (such as weather information and local event information) from an API. All collected data is analyzed using an AI model (such as a model using TensorFlow or Keras). Based on the analysis results, a life plan optimized for the user's budget and number of days is generated.

[1042] Creating a life plan

[1043] Based on the analysis results of the AI ​​model, the server generates an optimal life plan for the user. For example, it may include "movie viewing plans and the best time to do so," "a jogging schedule in a nearby park," and "recommendations for Japanese restaurants that take health into consideration." The generated life plan includes detailed information for each activity (cost, location, and planned time).

[1044] Presenting the plan

[1045] The server sends the generated life plan to the device, which then displays the plan to the user, showing detailed information about each activity (cost, location, and planned time).

[1046] Pre-order and purchase

[1047] The user selects the activity they want to carry out from the proposed plan and makes a store reservation or purchase a ticket. The device sends the reservation or purchase request to the server. The server receives the reservation or purchase request and connects with the systems of affiliated movie theaters and restaurants to make the reservation or purchase. Once completed, confirmation information is sent to the device. The device displays the reservation confirmation information to the user and automatically adds it to their calendar or reminders.

[1048] Activity execution and feedback

[1049] The user executes the proposed plan and performs the actual activities. The user generates ratings and feedback for each activity. The device sends the feedback information to the server. The server stores the received feedback in a database and uses it as training data for the AI ​​model to improve the accuracy of future plan suggestions.

[1050] Specific examples

[1051] An embodiment of this system will be explained using a specific scenario. For example, suppose a user inputs information such as "watching movies," "jogging," and "likes Japanese food," and has a budget of 20,000 yen and 10 days remaining. In this case, the AI ​​analyzes this data and external weather information to generate the following life plan:

[1052] 1. "I'm planning to see two movies at the cinema this weekend."

[1053] 2. "Jogging in a nearby park on a sunny day."

[1054] 3. "I recommend eating at a Japanese restaurant after jogging."

[1055] The user confirms the proposed plan and reserves the movie ticket. The server completes the reservation through the connected cinema system and sends the confirmation information to the terminal. After watching the movie, the user provides feedback, which is used to generate the next plan.

[1056] Example prompts for generative AI models

[1057] Below are some examples of prompts to input to an AI model:

[1058] "If the user likes watching movies, jogging, and eating Japanese food, generate the optimal life plan for a 10-day period with a budget of 20,000 yen."

[1059] "Please provide a recommended activity plan that takes into account the user's preferences (e.g., watching movies, jogging, Japanese food), current budget, and duration."

[1060] In this way, the system of the present invention automatically generates an optimal life plan that takes into account the user's budget and time, and supports the execution of that plan.

[1061] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1062] Step 1: Enter user data

[1063] User: Opens the dedicated application and enters information about their interests, health, and consumption. Examples of input data include a preference for watching movies, jogging, and Japanese food.

[1064] Terminal: Receives data entered by the user and sends this data to the server. The entered data is structured in JSON or XML format.

[1065] Step 2: Receiving and storing data

[1066] Server: Receives data sent from the device. The received content includes user interest information, health information, and consumption information.

[1067] Server: The received data is checked for errors to ensure it is properly formatted, and then saved to a database. The saved data is indexed and managed for each user.

[1068] Step 3: Gathering external information

[1069] Server: Calls external APIs to obtain external data such as weather information and local event information. The data obtained from the API is provided in the form of a JSON object containing weather forecasts and event information.

[1070] Server: The acquired external information is temporarily stored in memory and used for subsequent data analysis.

[1071] Step 4: Data analysis using AI

[1072] Server: Collects user interest information, health information, and consumption information from the database, and creates a dataset that also includes external information.

[1073] Server: Runs AI models using libraries such as TensorFlow and Keras to analyze the created datasets, using clustering algorithms and predictive models.

[1074] Server: As a result of the analysis, it generates an optimal life plan based on the user's budget and number of days. This plan includes detailed information about each activity (e.g., cost, location, and planned time).

[1075] Step 5: Create a life plan

[1076] Server: Based on the analysis results of the AI ​​model, the server generates an optimal life plan for the user, such as movie-going plans and the best times to do so, a jogging schedule in a nearby park, and recommendations for Japanese restaurants that take health into consideration.

[1077] Server: Formats the generated life plan and prepares it for transmission to the terminal.

[1078] Step 6: Present your plan

[1079] Server: Sends the generated life plan to the device. Push notifications may be used as a notification mechanism.

[1080] Terminal: Displays the user's life plan, with detailed information for each activity (cost, location, and planned time) presented in a user-friendly interface.

[1081] Step 7: Pre-order and purchase

[1082] User: Selects the activity they want to perform from the displayed list and makes a store reservation or purchases a ticket. For example, they click the "Go see this movie" button.

[1083] Terminal: Sends reservation or purchase requests to the server based on the user's selection.

[1084] Server: Processes the received request and executes the reservation or purchase in cooperation with the systems of affiliated stores and service providers. After the process is complete, it sends confirmation information to the terminal.

[1085] On the device: Display the reservation confirmation to the user and automatically add it to their calendar or reminders as appropriate.

[1086] Step 8: Activity execution and feedback

[1087] User: Executes the proposed plan and performs an actual activity, e.g., watches a movie at a cinema.

[1088] Users: Provide ratings and feedback for each activity, such as "The quality of the movie was great" or "The service at the Japanese restaurant was good."

[1089] Terminal: Sends feedback information to the server.

[1090] Server: The received feedback is stored in a database and used as learning data for the AI ​​model to improve the accuracy of the next plan proposal.

[1091] (Application example 1)

[1092] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1093] There is a lack of a means to automatically generate and present plans that take into account individual interests, health conditions, and consumption trends when users are trying to live an efficient and satisfying life within their budget. There is also a lack of support systems that allow users to easily make reservations and purchases at physical stores based on those plans. Furthermore, there is a need for a means to provide users with intuitive information in real time by displaying the generated plans on smart glasses or head-mounted displays.

[1094] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1095] In this invention, the server includes means for inputting user interest information, health information, and consumption information, means for linking with the user's past consumption data and external information, means for generating an optimal life plan based on the user's budget and number of days, means for presenting the generated life plan to the user, means for supporting reservations or purchases at physical stores based on the life plan, and means for displaying the generated life plan on smart glasses or a head-mounted display. This makes it possible to automatically generate plans tailored to the user's individual needs, simplify the reservation and purchase process, and provide information intuitively in real time.

[1096] "User interest information" is information about the user's hobbies and preferences.

[1097] "Health information" is information relating to the user's current health condition and past health checkup results.

[1098] "Consumption information" is information relating to a user's purchase history and spending patterns.

[1099] "Historical Consumption Data" is data recorded by a user that includes details of previous purchases and expenditures.

[1100] "External information" is data obtained from external sources, such as weather information and local event information.

[1101] A "budget" is a spending limit set by a user.

[1102] "Number of days" refers to the date set by the user as the validity period of the life plan.

[1103] The "optimal life plan" is a proposal of an optimal life plan for the user, generated based on the user's interest information, health information, consumption information, and external information.

[1104] "Physical store" refers to a physical store, such as a clothing store, restaurant, or movie theater.

[1105] A "reservation" is the act of reserving in advance the use or purchase of a specific service or product.

[1106] "Purchase" is the act of acquiring a service or product with money.

[1107] "Smart glasses" are eyeglass-type devices that have the function of displaying information in the wearer's field of vision.

[1108] A "head-mounted display" is a display worn on a user's head that provides virtual or augmented reality visual information.

[1109] "User feedback" is information regarding opinions and evaluations provided by users regarding the proposed life plan.

[1110] "Real-time" refers to processing and information being provided immediately at that moment.

[1111] The system embodying the present invention supports users in living an efficient and satisfying life within their budget, and involves cooperation between users, terminals, and a server. Specific embodiments of the system are described below.

[1112] First, a user launches an application using a device such as a smartphone, smart glasses, or head-mounted display. The user then inputs their interest information (hobbies, favorite foods, etc.), health information (health status), and consumption information (purchase history, etc.) into the application. The input information is received by the device and sent to the server.

[1113] The server stores the interest, health, and consumption information sent by the user in a database. The server also obtains external information such as weather and local event information via API. The AI ​​model on the server analyzes this data and generates an optimal life plan based on the user's budget and number of days.

[1114] The generated life plan includes specific activity suggestions, such as watching a movie, jogging, or eating at a Japanese restaurant, along with detailed information such as the cost, location, and estimated time for each activity. This life plan is then sent by the server to the device and presented to the user.

[1115] Users can select the desired activity from the proposed life plan and make a reservation or purchase a ticket at a physical store. The device sends the reservation or purchase request to the server, which then completes the reservation or purchase by linking with the systems of affiliated stores and services. Once completed, confirmation information is sent to the device and displayed to the user. If using smart glasses or a head-mounted display, the information is displayed in real time, allowing the user to receive the information intuitively.

[1116] In addition, users can provide ratings and feedback on the activities they have performed. This feedback information is also sent to the server and stored in a database. The server uses this feedback as learning data for the AI ​​model and reflects it in the next life plan generation, thereby improving the accuracy of suggestions.

[1117] For example, if a user inputs information such as "watching movies," "jogging," and "likes Japanese food," along with a budget of 20,000 yen and 10 days remaining, the system will generate the following life plan:

[1118] 1. "I'm planning to see two movies at the cinema this weekend."

[1119] 2. "Jogging in a nearby park on a sunny day."

[1120] 3. "I recommend eating at a Japanese restaurant after jogging."

[1121] An example of a prompt sentence in this case is: "The user likes watching movies, jogging, and Japanese food. The budget is 20,000 yen and there are 10 days left. The weather is sunny on weekends and cloudy or rainy on other days. Please generate the optimal activity plan."

[1122] These steps allow users to automatically generate personalized plans, simplify the booking and purchasing process, and provide intuitive information in real time.

[1123] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1124] Step 1:

[1125] A user launches an application using a smartphone or smart glasses and inputs information about their interests, health, and consumption. This information is received by the device as user input. For example, the user inputs information such as "watching movies," "jogging," and "I like Japanese food." The device then sends the input data to the server.

[1126] Step 2:

[1127] The server receives the user's interest, health, and consumption information sent from the device and stores it in a database. At the same time, the server uses an external API to obtain weather information and local event information. This external information is also stored in the database. Input data includes the user's preferences and health status, past consumption data, weather information, and event information.

[1128] Step 3:

[1129] The AI ​​model on the server collects and analyzes the user's interest, health, consumption, and external information from a database. This analysis generates an optimal life plan based on the user's budget and number of days. For example, given a user's budget of 20,000 yen and 10 days remaining, the AI ​​model suggests activities such as watching a movie, jogging, and eating at a Japanese restaurant.

[1130] Step 4:

[1131] The server sends the generated life plan to the device, which receives it and displays it to the user. The displayed content includes detailed information about each activity (e.g., movie viewing plans, jogging time, location and cost of Japanese restaurants, etc.).

[1132] Step 5:

[1133] The user selects the desired activity from the proposed life plan. Based on the selected activity, the user makes a reservation or purchase request at a physical store. The device sends these requests to the server.

[1134] Step 6:

[1135] The server receives reservation and purchase requests sent by users and completes the reservation or purchase in cooperation with the systems of affiliated stores and services. After completion, reservation confirmation information is sent to the device. The device receives this information and displays it to the user. If smart glasses or a head-mounted display are used, this confirmation information is also displayed in real time.

[1136] Step 7:

[1137] Users provide ratings and feedback for the activities they perform. The device collects this feedback information and sends it to the server. The server stores the feedback information in a database and uses it as learning data for the AI ​​model to improve the accuracy of future life plan suggestions.

[1138] In this way, all steps work together to enable users to realize the optimal life plan while responding to their individual needs and using their budget and time efficiently.

[1139] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1140] The present invention relates to an AI-based support system for helping users live an efficient and satisfying life within their budget, and in particular to improving the accuracy of life plans by incorporating an emotion engine. Hereinafter, embodiments of the present invention will be described in detail.

[1141] Natural language description of the program

[1142] This system exchanges information between the user, device, and server, and not only generates an optimal life plan based on the user's preferences and past consumption data, but also has the ability to recognize the user's emotions in real time and adjust the plan accordingly.

[1143] 1. Entering user data

[1144] User: Opens the application and enters their interests (hobbies, favorite foods, etc.) and health information (current health condition). For emotion recognition, the application also allows voice input and facial recognition. The application also allows users to connect to external services (e.g., PayPay) to obtain past consumption data.

[1145] Terminal: Receives input interest information, health information, and consumption information and sends it to the server.

[1146] 2. Receipt and storage of data

[1147] Server: Receives user information (interest information, health information, consumption information) and emotion data sent from the device and stores it in a database.

[1148] 3. AI-based data analysis

[1149] Server: Collects user interest, health, and consumption information from a database, and also uses external APIs to obtain real-time weather information and local event information.

[1150] Server: Using an emotion engine, the server analyzes the voice, facial expressions, and text data sent from the device in real time to recognize the user's emotional state.

[1151] 4. Creating a life plan

[1152] Server: The collected data and the analysis results of the emotion engine are input into the AI ​​model to generate an optimal life plan based on the user's budget and number of days. The AI ​​model takes into consideration the user's past consumption patterns, preferences, current emotional state, weather information, etc.

[1153] 5. Plan generation and adjustment

[1154] Server: The life plan generated by the AI ​​model includes specific activity suggestions (e.g., watching a movie, jogging, or eating at a Japanese restaurant) along with detailed information about costs, locations, and planned times.

[1155] Server: Reflects the user's real-time emotional state and adjusts the plan, for example, by suggesting a plan that emphasizes relaxation activities if the user is under high stress, or by suggesting more active activities if the user is in a good mood.

[1156] 6. Presenting the plan

[1157] Server: Sends the generated life plan to the device.

[1158] Device: Displays the received life plan to the user. The screen shows detailed information about each activity (cost, location, planned time) and adjustment points based on the user's emotional state.

[1159] 7. Reservations and Purchases

[1160] User: Selects the activities they want to carry out from the proposed life plan and makes store reservations or purchases tickets as necessary.

[1161] Terminal: Sends reservation and purchase requests to the server.

[1162] Server: Receives reservation and purchase requests, coordinates with affiliated cinema and restaurant systems to complete reservations and purchases, and sends confirmation information to the device upon completion.

[1163] On your device: Display reservation confirmation information to the user and automatically add it to your calendar and reminders.

[1164] 8. Activity execution and feedback

[1165] User: Carry out the proposed plan, for example, watch a movie at a cinema, go jogging in a park, or enjoy a meal at a Japanese restaurant.

[1166] Users: Provide ratings and feedback for each activity.

[1167] Terminal: Sends user feedback information to the server.

[1168] Server: The received feedback is stored in a database and used as training data for the AI ​​model to propose future plans, improving the accuracy of subsequent plan generation.

[1169] Specific examples

[1170] An embodiment of this system will be explained using a specific scenario. For example, suppose a user inputs information such as "watching movies," "jogging," and "likes Japanese food," has a budget of 20,000 yen, and has 10 days left. The AI ​​analyzes this data and external weather information, and the emotion engine recognizes that the user's current stress level is high, and generates the following life plan:

[1171] 1. "I'm planning to see two movies at the cinema this weekend."

[1172] 2. "Jogging in a nearby park on a sunny day."

[1173] 3. "I recommend eating at a Japanese restaurant after jogging."

[1174] 4. "We also suggest visiting a relaxing hot spring facility to reduce stress."

[1175] The user checks the proposed plan and reserves movie tickets. The server completes the reservation through the connected cinema system and sends the confirmation information to the terminal. After watching the movie, the user provides feedback, which is used to generate the next plan. The emotion engine analyzes the user's emotions in real time, and the system always provides a plan optimized for the user's emotional state.

[1176] In this way, the system of the present invention takes into consideration the user's budget and time, and also analyzes and reflects their emotional state in real time to automatically generate an optimal life plan and support its implementation.

[1177] The processing flow will be explained below.

[1178] Step 1:

[1179] User: Opens the application and enters their interests (hobbies, favorite foods, etc.) and health information (current health condition). In addition, they authorize voice input and face recognition functions, and set up the application to connect to external services (e.g., PayPay) to obtain past consumption data.

[1180] Step 2:

[1181] Terminal: Receives interest information, health information, and consumption information entered by the user and sends this data to the server.

[1182] Step 3:

[1183] Server: Receives user information (interest information, health information, consumption information) and emotion data sent from the device and stores it in a database.

[1184] Step 4:

[1185] Server: Collects user interest, health, and consumption information from a database, and also uses external APIs to obtain real-time weather information and local event information.

[1186] Step 5:

[1187] Server: Using an emotion engine, the server analyzes the voice, facial expressions, and text data sent from the device in real time to recognize the user's emotional state.

[1188] Step 6:

[1189] Server: The collected data and the analysis results of the emotion engine are input into the AI ​​model to generate an optimal life plan based on the user's budget and number of days. The AI ​​model takes into consideration the user's past consumption patterns, preferences, current emotional state, weather information, etc.

[1190] Step 7:

[1191] Server: The generated life plan includes specific activity suggestions (e.g., watching a movie, jogging, or eating at a Japanese restaurant) with detailed information about costs, locations, and estimated times.

[1192] Step 8:

[1193] Server: Reflects the user's real-time emotional state and adjusts the plan, for example, by suggesting a plan that emphasizes relaxation activities if the user is under high stress, or by suggesting more active activities if the user is in a good mood.

[1194] Step 9:

[1195] Server: Sends the generated life plan to the device.

[1196] Step 10:

[1197] Device: Displays the received life plan to the user. The screen shows detailed information about each activity (cost, location, planned time) and adjustment points based on the user's emotional state.

[1198] Step 11:

[1199] User: Selects the activities they want to carry out from the proposed life plan and makes store reservations or purchases tickets as necessary.

[1200] Step 12:

[1201] Terminal: Sends reservation and purchase requests to the server.

[1202] Step 13:

[1203] Server: Receives reservation and purchase requests, coordinates with affiliated cinema and restaurant systems to complete reservations and purchases, and sends confirmation information to the device upon completion.

[1204] Step 14:

[1205] On your device: Display reservation confirmation information to the user and automatically add it to your calendar and reminders.

[1206] Step 15:

[1207] User: Carry out the proposed plan, for example, watch a movie at a cinema, go jogging in a park, or enjoy a meal at a Japanese restaurant.

[1208] Step 16:

[1209] Users: Provide ratings and feedback for each activity.

[1210] Step 17:

[1211] Terminal: Sends user feedback information to the server.

[1212] Step 18:

[1213] Server: The received feedback is stored in a database and used as training data for the AI ​​model to propose future plans, improving the accuracy of subsequent plan generation.

[1214] Example 2

[1215] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1216] When generating an efficient and satisfying life plan within a user-specified budget and number of days, there was a problem in that the system was unable to recognize the user's emotional state in real time and adjust the plan accordingly. It was also difficult to dynamically generate plans that incorporated weather information and local event information, and it was also a challenge to efficiently utilize the user's personal preferences and past consumption data. Another problem was the lack of a feedback loop to collect user feedback and reflect it in the next plan.

[1217] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1218] In this invention, the server includes: means for inputting user interest information, health information, and consumption information; means for linking with the user's past consumption data and external information; means for allowing voice input and facial recognition for emotion recognition; means for generating an optimal life plan based on the user's budget and number of days; means for analyzing the user's emotional state in real time and adjusting the life plan based on the analysis; means for presenting the generated life plan to the user; means for assisting with store reservations or ticket purchases based on the life plan; and means for collecting user feedback and reflecting it in the generation of the next plan. This allows for the generation of an optimized life plan in real time, taking the user's emotional state into consideration. Furthermore, incorporating weather information and local event information allows for the provision of more dynamic and personalized plans. Furthermore, by creating a feedback loop, the accuracy of subsequent plans can be improved, resulting in an overall improved user experience.

[1219] "Interest information" is information about the user's hobbies, preferences, and tastes.

[1220] "Health information" refers to information related to health, such as the user's current health condition and medical history.

[1221] "Consumption information" is data related to consumption activities that a user has engaged in in the past.

[1222] "External information" refers to information obtained from external databases or APIs, such as weather information and local event information.

[1223] "Emotion recognition" is a technology that analyzes a user's voice and facial expressions to determine their current emotional state.

[1224] "Voice input" is a means of capturing user voice data as input.

[1225] "Facial recognition" is a technology that analyzes a user's facial features and identifies them as an individual.

[1226] A "life plan" is a specific action plan proposed based on the user's budget and number of days.

[1227] "Real-time" means that data is analyzed immediately and necessary processing is carried out at that moment.

[1228] "Analysis" is the process of examining data in detail and deriving patterns and trends.

[1229] "Store reservation" refers to making a reservation in advance at a store that the user plans to visit.

[1230] "Ticket purchase" means the purchase of an admission ticket to participate in an event or activity.

[1231] "Feedback" refers to opinions and evaluations provided by users, and is information that can be used to improve services next time.

[1232] This invention is an AI-based support system that allows users to generate and adjust an optimal life plan within their budget and time frame. This system consists of three main components: the user, the device, and the server. It analyzes the user's emotional state in real time based on their interest, health, and consumption information, and provides a life plan that reflects that information.

[1233] Entering User Data

[1234] User: The user opens the application and enters their interest information (such as hobbies and favorite dishes), health information, and consumption information. They also configure the application to enable voice input and facial recognition, which provides basic data for emotion recognition. They also configure the application to connect to external services (e.g., electronic payment systems) and grant permission to obtain past consumption data.

[1235] Receiving and storing data

[1236] Terminal: The terminal sends all data entered by the user to the server. It also validates the user's input to ensure that the data is accurate.

[1237] Server: The server receives the data sent from the device and stores it in a database. The stored data is used for further analysis and calculation.

[1238] AI-based data analysis

[1239] Server: The server collects user information from a database and, if necessary, uses external APIs to obtain real-time weather information and local event information. It also uses an emotion engine to analyze voice, facial expressions, and text data to recognize the user's emotional state. Emotion recognition uses voice recognition algorithms and image recognition algorithms.

[1240] Creating and adjusting life plans

[1241] Server: Based on the collected data and the analysis results of the emotion engine, the data is input into the AI ​​model to generate an optimal life plan based on the user's budget and number of days. The AI ​​model takes into account past spending patterns, interests, current emotional state, weather information, etc. The generated plan includes specific activities such as going to the movies, jogging, and eating at a Japanese restaurant, along with the cost, location, and planned time for each.

[1242] Presenting and adjusting plans

[1243] Server: Sends the generated life plan to the device. The plan reflects the user's real-time emotional state and is dynamically adjusted according to the specific situation.

[1244] Device: Displays the received life plan to the user, along with detailed information on each activity and adjustment points.

[1245] Reservation and purchase assistance

[1246] User: The user selects the activities they want to carry out from the proposed life plan and makes store reservations or purchases tickets as necessary.

[1247] Terminal: Sends reservation and purchase requests to the server.

[1248] Server: The server receives reservation and purchase requests, executes them in cooperation with partner systems, and sends confirmation information to the terminal once completed.

[1249] On the device: Display reservation confirmation information to the user and automatically add it to their calendar or reminders.

[1250] Activity execution and feedback

[1251] User: Carry out the proposed plan and enjoy watching a movie, jogging, and eating at a Japanese restaurant.

[1252] Users: Provide ratings and feedback for each activity.

[1253] Terminal: Sends feedback information to the server.

[1254] Server: The received feedback is stored in a database and used as training data for the AI ​​model to propose future plans, improving the accuracy of subsequent plan generation.

[1255] Specific examples

[1256] For example, suppose a user inputs information such as "watching movies," "jogging," and "likes Japanese food," and requests a 10-day plan with a budget of ¥20,000. The system takes this data into account along with real-time weather information, and the emotion engine recognizes that the user's current stress level is high, generating the following life plan:

[1257] 1. "I'm going to see two movies at the cinema this weekend."

[1258] 2. "Jogging in a nearby park on a sunny day."

[1259] 3. "Recommend a meal at a Japanese restaurant after jogging."

[1260] 4. "We also suggest visiting a relaxing hot spring facility to reduce stress."

[1261] The user checks the proposed plan and reserves movie tickets. The server completes the reservation with the affiliated cinema system and sends the confirmation information to the terminal. After watching the movie, the user provides feedback, which is reflected in the next plan generation. The emotion engine analyzes the user's emotions in real time, so that the system always provides the most optimized plan.

[1262] Prompt Sentence Examples

[1263] "Watching movies," "Jogging," "Like Japanese food," "Budget is 20,000 yen," "Prefer a 10-day plan," "Recognizes and adjusts emotions in real time," "Includes relaxing activities."

[1264] In this way, the present invention automatically generates an optimal life plan that takes into account the user's budget and time, and also analyzes and reflects their emotional state in real time, and supports their execution.

[1265] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1266] Step 1: Enter user data

[1267] The user opens the application and inputs interest, health, and consumption information. The user also sets permissions for voice input and facial recognition. Furthermore, the user sets up connections with external services (e.g., electronic payment systems) and gives permission to obtain past consumption data.

[1268] Input: Interest information (hobbies, preferences), health information, consumption information, voice input permission, face recognition permission, external service linkage permission

[1269] Output: User dataset to send to terminal

[1270] What happens: A user enters information into an application form, and after completing each field, presses a confirmation button. The form data is packaged in JSON format and ready to be sent to the device.

[1271] Step 2: Receiving and storing data

[1272] The terminal transmits the data entered by the user to the server, where it verifies the consistency and integrity of the data and performs error checking before transmission.

[1273] The server receives the data sent from the terminal, verifies it, and then stores it in a database.

[1274] Input: A user-entered dataset

[1275] Output: User data stored in the database

[1276] Specific operation: The terminal sends an HTTP POST request to the server, the server receives the data, parses it, checks for errors and validates the data format, and then stores it in the database.

[1277] Step 3: Data analysis using AI

[1278] The server collects user information from a database and also uses external APIs to obtain current weather information and local event information.

[1279] The server uses an emotion engine to analyze the voice data, facial expression data, and text data sent from the terminal in real time and identify the user's emotional state.

[1280] Input: User information collected from the database, weather information obtained from external APIs, local event information, emotion data

[1281] Output: User's emotional state as an analysis result and supplementary information

[1282] Specific operation: The server executes a scheduled task to obtain user information and sends a request to an external API (e.g., weather API). The obtained data is input into the emotion engine for real-time analysis.

[1283] Step 4: Create a life plan

[1284] The server inputs the collected data and the results of the emotion engine's analysis into an AI model to generate an optimal life plan based on the user's budget and number of days.

[1285] Input: Collected user data, emotion engine analysis results, external API data

[1286] Output: Generated life plan

[1287] Specific actions: Input data into the AI ​​model and run inference. The generated life plan includes specific activity suggestions, with details (cost, location, and planned time) automatically calculated and entered.

[1288] Step 5: Present and adjust the plan

[1289] The server sends the generated life plan to the device, which adjusts the plan as needed to reflect the user's real-time emotional state.

[1290] The terminal displays the received life plan to the user.

[1291] Input: Generated life plan

[1292] Output: The adjusted life plan displayed to the user

[1293] Specific operation: The server sends plan data in JSON format to the device, which receives the data and displays it on the interface. Plan adjustments are made based on the results of real-time sentiment analysis, and adjustment points are presented to the user.

[1294] Step 6: Reservation and purchase assistance

[1295] Users can select the activities they want to carry out from the proposed life plan and make reservations or purchase tickets.

[1296] The terminal sends a reservation request or a purchase request to the server.

[1297] The server cooperates with the affiliated system to execute the reservation or purchase and transmits confirmation information to the terminal.

[1298] The device will display the reservation confirmation information to the user and automatically add it to their calendar or reminders.

[1299] Input: User-selected activity booking / purchase request

[1300] Output: Confirmation information and automatic addition to calendar and reminders

[1301] Specific operation: The user selects an activity and enters the required information in the reservation form. The device sends a request to the server, and the server returns the execution result to the device. The device displays a confirmation and automatically adds it to the calendar and reminders.

[1302] Step 7: Activity execution and feedback

[1303] The user carries out the proposed plan and enjoys watching a movie, jogging, and eating at a Japanese restaurant.

[1304] Users provide ratings and feedback for each activity.

[1305] The terminal transmits the feedback information to the server.

[1306] The server stores the received feedback in a database and uses it as training data for the AI ​​model to propose future plans.

[1307] Input: Feedback data after the user performs an activity

[1308] Output: Feedback stored in the database as training data

[1309] Specific operation: The user fills out a feedback form, which the device sends to the server. The server stores the feedback in a database and uses it to generate the next life plan.

[1310] (Application example 2)

[1311] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1312] In modern society, autonomous vehicles offer significant improvements in user comfort and efficiency, but their capabilities are not being fully utilized. Furthermore, there are many challenges in providing optimal life plans that take into account the user's emotional state and real-time information. Specifically, plans are not adequately generated based on the user's interests and health status, and real-time adjustments based on emotion recognition are insufficient. Furthermore, conventional systems have difficulty generating route proposals that take into account weather and event information. Therefore, effective means of improving user satisfaction and comfort are needed.

[1313] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1314] In this invention, the server includes means for recognizing the user's emotional state in real time and adjusting the life plan, means for generating and presenting route proposals based on the user's emotions and real-time information within the autonomous vehicle, and means for acquiring weather information and event information using an external API, thereby enabling the generation of a life plan and route proposals optimized for the user's real-time emotional state and external environment.

[1315] "User interest information" is data that indicates the user's preferred activities, hobbies, and interests.

[1316] "Health information" is data relating to the user's current health condition and past health history.

[1317] "Consumption information" is data indicating products and services purchased by a user in the past, the amount spent on those products and services, and so on.

[1318] "Past consumption data" refers to data including the user's past purchasing behavior and its history.

[1319] "External information" refers to information obtained from outside the system, such as weather, local events, and traffic conditions.

[1320] A "budget" is data that indicates a range of available amounts set by the user.

[1321] "Number of days" is data indicating the number of days within a specific period set by the user.

[1322] The "means for generating a life plan" is a process for creating a plan for the user to live an efficient and satisfying life based on collected user data and external information.

[1323] The "means for presenting a life plan to a user" refers to a method for visually or audibly displaying or notifying a user of a created life plan.

[1324] "Means for supporting store reservations or ticket purchases" are methods that allow users to easily make reservations or purchases based on the proposed life plan.

[1325] "Means for recognizing emotional state in real time" refers to technology that analyzes the user's voice and facial expressions to determine their current emotions.

[1326] The "means for generating and presenting route proposals" is a technology that plans an optimal travel route based on the user's emotional state and real-time environmental information, and displays it on the vehicle terminal.

[1327] "Means of obtaining weather and event information using external APIs" refers to a method of obtaining real-time weather and local event information by linking with external data provision services.

[1328] The system of this invention inputs a user's interest information, health information, and consumption information, and generates an optimal life plan based on the budget and number of days in conjunction with past consumption data and external information. The system also incorporates an emotion engine to recognize the user's real-time emotional state and dynamically adjust the life plan.

[1329] Hardware and Software Configuration

[1330] The server performs various data processing and calculations using a database, AI analysis engine, emotion engine, external API connection module, etc. These components will be explained in detail below.

[1331] 1. User Device:

[1332] Input devices (smartphones, tablets, etc.)

[1333] Emotion recognition devices (camera, microphone, etc.)

[1334] 2. Server:

[1335] Database (SQL, etc.)

[1336] AI analysis engine (TensorFlow, Keras, etc.)

[1337] Emotion Engine API

[1338] External API connection module (weather API, event API)

[1339] Data processing and calculation procedures

[1340] The server first receives interest, health, and consumption information from the user's device and stores it in a database. This information, along with real-time weather information and local event information obtained through external APIs, is then input into an AI analysis engine.

[1341] The server uses an emotion engine to analyze the user's voice, facial expressions, and input text in real time to recognize the user's emotional state. The recognized emotional data, along with other data, becomes the input data required to generate a life plan.

[1342] The generated life plan includes specific activity suggestions based on the user's schedule, activities, and budget. For example, the system suggests optimal routes based on the user's emotional state and real-time information in an autonomous vehicle. The system also supports users in making reservations and purchasing tickets, and the server handles these procedures.

[1343] Specific examples

[1344] For example, suppose a user enters information into the app such as "I want to relax" and "I like Japanese food," and sets a budget of 20,000 yen. The server generates the following life plan based on real-time weather information and local event information:

[1345] 1. "Plan a relaxing picnic in the park on a sunny day."

[1346] 2. "I suggest we go to a Japanese restaurant afterwards."

[1347] 3. "We recommend visiting a hot spring facility to relax in the evening."

[1348] If the emotion engine recognizes that the user's stress level is high, it will focus on suggesting more relaxing activities.

[1349] Prompt Sentence Examples

[1350] "The user wants to relax, so generate a route for stress reduction."

[1351] "Generate tonight's dinner plans for the user. My budget is 5000 yen and I'm in the mood to relax."

[1352] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1353] Step 1:

[1354] The user's device inputs information about the user's interests, health, and consumption. The user enters this data into the app from their smartphone or tablet and sends it to the server. Input may also be done using the user's voice or facial recognition function.

[1355] Input: User interest information, health information, consumption information

[1356] Output: Send data to the server

[1357] Step 2:

[1358] The server receives the information sent from the user terminal and stores it in a database. Based on this data, the server aggregates the data necessary to generate a life plan.

[1359] Input: User interest information, health information, consumption information

[1360] Output: Save data to a database

[1361] Step 3:

[1362] The server uses external APIs to obtain real-time weather and event information. For this purpose, it uses the weather API and event API.

[1363] Input: Real-time information request from an external API

[1364] Output: Get real-time weather and event information

[1365] Step 4:

[1366] The server uses an emotion engine to analyze the user's voice, facial expressions, and text in real time to recognize the user's emotional state, thereby determining the user's stress level and mood.

[1367] Input: User's voice data, facial expression data, text data

[1368] Output: Emotional state data

[1369] Step 5:

[1370] The server uses an AI analysis engine to generate an optimal life plan based on the user's data stored in the database, as well as real-time weather information, event information, and emotional state. The generated life plan includes the user's budget and details of each activity.

[1371] Input: User data, real-time weather information, event information, emotional state data

[1372] Output: Optimal Life Plan

[1373] Step 6:

[1374] The server transmits the generated life plan to the user terminal, which displays the plan and presents it to the user.

[1375] Enter: Optimal Life Plan

[1376] Output: Sending the life plan to the user's device

[1377] Step 7:

[1378] The user terminal supports store reservations and ticket purchase procedures based on the life plan. Reservation and purchase requests are sent to the server, which then cooperates with affiliated store systems to complete the procedures.

[1379] Input: User reservation request, purchase request

[1380] Output: Store reservation and ticket purchase confirmation information

[1381] Step 8:

[1382] The user executes the proposed life plan and provides feedback for each activity. The user terminal collects this feedback information and sends it to the server. The server stores the collected feedback in a database and uses it to generate the next plan.

[1383] Input: User feedback data

[1384] Output: Save feedback data to database

[1385] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1386] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1387] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1388] [Fourth embodiment]

[1389] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1390] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1391] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1392] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1393] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1394] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1395] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1396] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1397] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1398] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1399] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1400] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1401] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1402] The present invention relates to an AI-based assistance system that helps users live an efficient and satisfying life within their budget, and can be embodied in the following forms.

[1403] Natural language description of the program

[1404] This system exchanges information between the user, the terminal, and the server, and generates an optimal life plan based on the user's preference information and past consumption data. The specific program processing is explained below.

[1405] 1. Entering user data

[1406] User: First, the user opens the dedicated application and enters their interests (hobbies, favorite foods, etc.), health information (current health condition), and consumption information (obtained through collaboration with external services such as PayPay) into the device.

[1407] Terminal: Receives input data and sends it to the server.

[1408] 2. Receipt and storage of data

[1409] Server: Receives user interest information, health information, and consumption information sent from the device and stores it in a database.

[1410] 3. AI-based data analysis

[1411] Server: Collects user interest, health, and consumption information from the database, and also obtains external information (such as weather and local event information) from the API.

[1412] Server: Analyzes all collected data using an AI model. Based on the analysis results, a life plan optimized for the user's budget and number of days is generated.

[1413] 4. Creating a life plan

[1414] Server: The life plan generated by the AI ​​model may include suggestions such as:

[1415] Planning a movie and the best time to go

[1416] Jogging schedule at a nearby park

[1417] Proposal for a Japanese restaurant that takes health into consideration

[1418] Server: The generated life plan includes detailed information about each activity (cost, location, and planned time).

[1419] 5. Presenting the plan

[1420] Server: Sends the generated life plan to the device.

[1421] Device: Displays the received plan to the user. The screen shows detailed information about each activity (cost, location, and planned time).

[1422] 6. Reservations and Purchases

[1423] User: Choose the activity they want to do from the proposed plan and make store reservations or purchase tickets as necessary.

[1424] Terminal: Sends reservation and purchase requests to the server.

[1425] Server: Receives reservation and purchase requests, coordinates with affiliated cinema and restaurant systems to complete reservations and purchases, and sends confirmation information to the device upon completion.

[1426] On your device: Display reservation confirmation information to the user and automatically add it to your calendar and reminders.

[1427] 7. Activity execution and feedback

[1428] User: Implements the proposed plan and actually carries out the activities.

[1429] Users: Provide ratings and feedback for each activity.

[1430] Terminal: Sends feedback information to the server.

[1431] Server: The received feedback is stored in a database and used as training data for the AI ​​model to improve the accuracy of future plan suggestions.

[1432] Specific examples

[1433] An embodiment of this system will be explained using a specific scenario. For example, suppose a user inputs information such as "watching movies," "jogging," and "likes Japanese food," and has a budget of 20,000 yen and 10 days remaining. The AI ​​analyzes this data and external weather information to generate the following life plan:

[1434] 1. "I'm planning to see two movies at the cinema this weekend."

[1435] 2. "Jogging in a nearby park on a sunny day."

[1436] 3. "I recommend eating at a Japanese restaurant after jogging."

[1437] The user confirms the proposed plan and reserves the movie ticket. The server completes the reservation through the connected cinema system and sends the confirmation information to the terminal. After watching the movie, the user provides feedback, which is used to generate the next plan.

[1438] In this way, the system of the present invention automatically generates an optimal life plan that takes into account the user's budget and time, and supports the execution of the plan.

[1439] The processing flow will be explained below.

[1440] Step 1:

[1441] User: Opens the application and enters their interests (hobbies, favorite foods, etc.) and health information (current health condition). They also set up the application to connect to external services (e.g., PayPay) to obtain past consumption data.

[1442] Step 2:

[1443] Terminal: Receives interest information, health information, and consumption information entered by the user and sends this data to the server.

[1444] Step 3:

[1445] Server: Receives user information (interest information, health information, consumption information) sent from the device and stores it in a database.

[1446] Step 4:

[1447] Server: Collects user interest, health, and consumption information from a database, and also uses external APIs to obtain real-time weather information and local event information.

[1448] Step 5:

[1449] Server: The collected data is input into the AI ​​model, which analyzes and generates an optimal life plan based on the user's budget and number of days. The AI ​​model takes into consideration the user's past consumption patterns, preferences, current weather information, etc.

[1450] Step 6:

[1451] Server: The generated life plan includes specific activity suggestions (e.g., watching a movie, jogging, or eating at a Japanese restaurant), along with detailed information on costs, locations, and planned times.

[1452] Step 7:

[1453] Server: Sends the generated life plan to the device.

[1454] Step 8:

[1455] Device: Displays the received life plan to the user. The screen shows detailed information about each activity (cost, location, and planned time).

[1456] Step 9:

[1457] User: Selects the activities they want to carry out from the proposed life plan and makes store reservations or purchases tickets as necessary.

[1458] Step 10:

[1459] Terminal: Sends reservation and purchase requests to the server.

[1460] Step 11:

[1461] Server: Receives reservation and purchase requests, coordinates with affiliated cinema and restaurant systems to complete reservations and purchases, and sends confirmation information to the device upon completion.

[1462] Step 12:

[1463] On your device: Display reservation confirmation information to the user and automatically add it to your calendar and reminders.

[1464] Step 13:

[1465] User: Carry out the proposed plan, for example, watch a movie at a cinema, go jogging in a park, or enjoy a meal at a Japanese restaurant.

[1466] Step 14:

[1467] Users: Provide ratings and feedback for each activity.

[1468] Step 15:

[1469] Terminal: Sends user feedback information to the server.

[1470] Step 16:

[1471] Server: The received feedback is stored in a database and used as training data for the AI ​​model to propose future plans, improving the accuracy of subsequent plan generation.

[1472] Example 1

[1473] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1474] In today's busy lifestyles, it is difficult for users to create a satisfying life plan that uses time efficiently and stays within a budget. In particular, plans must be created that take into account individual preferences, health status, and past consumption data. Weather information and local event information also need to be appropriately reflected, but collecting these individually and creating the optimal plan requires a huge amount of effort. To solve these problems, a system is needed that can automatically generate the optimal plan tailored to the user's situation and provide consistent support, from reservation to purchase procedures.

[1475] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1476] In this invention, the server includes means for inputting user interest information, health information, and consumption information, means for linking with the user's past consumption data and external information, means for generating an optimal life plan based on the user's budget and number of days, means for displaying detailed information of the generated life plan on a screen, means for collecting user feedback and saving it in a database, and means for training an AI model using the saved feedback and reflecting it in the generation of the next plan. This makes it possible to automatically generate an optimal life plan tailored to the user's individual circumstances and support reservations, purchases, and execution.

[1477] "User interest information" is information about activities and preferences in which the user is interested (for example, hobbies, favorite foods, events they want to enjoy, etc.).

[1478] "Health information" refers to the user's current health condition and health-related information (e.g., medical history, daily activity level, diet, etc.).

[1479] "Consumption information" refers to information about a user's past consumption data and purchase history (for example, expenditure details, services used, types and quantities of purchased products, etc.).

[1480] "Past consumption data" refers to a history of information about purchases and expenditures made by a user.

[1481] "External information" is external data related to the creation of the user's life plan (for example, weather information, local event information, etc.).

[1482] A "budget" is a limit on the amount of money available to a user to plan their spending within a particular time period.

[1483] "Number of days" is the unit of time period set by the user for carrying out a particular life plan.

[1484] A "life plan" is a plan of activities and schedules optimized within a user's budget and number of days based on the user's interest information, health information, consumption information, and external information.

[1485] "Store reservation" is a process in which a user makes a reservation in advance to visit or use a specific store.

[1486] "Ticket purchase" refers to the act of a user obtaining a ticket necessary for admission to a movie theater, event, or the like.

[1487] "Feedback" is information about evaluations and impressions provided by users regarding activities they have performed.

[1488] "Database" means an electronic information management system for organizing and storing collected user information and feedback.

[1489] An "AI model" is an artificial intelligence computational model that analyzes collected data and generates the optimal life plan for the user.

[1490] This invention relates to an AI-based support system that helps users live an efficient and satisfying life within their budget. The system of this invention exchanges information between the user, terminal, and server, and generates an optimal life plan based on the user's preference information and past consumption data.

[1491] Entering User Data

[1492] The user opens the dedicated application and inputs their interests (hobbies, favorite foods, etc.), health information (current health condition), and consumption information (obtained, for example, through collaboration with external services such as payment services). The device receives the input data and sends it to the server.

[1493] Receiving and storing data

[1494] The server receives the user's interest, health, and consumption information sent from the device and stores it in a database. The stored data undergoes error checking to ensure it is properly formatted.

[1495] AI-based data analysis

[1496] The server collects information on the user's interests, health, and consumption from a database, and also obtains external information (such as weather information and local event information) from an API. All collected data is analyzed using an AI model (such as a model using TensorFlow or Keras). Based on the analysis results, a life plan optimized for the user's budget and number of days is generated.

[1497] Creating a life plan

[1498] Based on the analysis results of the AI ​​model, the server generates an optimal life plan for the user. For example, it may include "movie viewing plans and the best time to do so," "a jogging schedule in a nearby park," and "recommendations for Japanese restaurants that take health into consideration." The generated life plan includes detailed information for each activity (cost, location, and planned time).

[1499] Presenting the plan

[1500] The server sends the generated life plan to the device, which then displays the plan to the user, showing detailed information about each activity (cost, location, and planned time).

[1501] Pre-order and purchase

[1502] The user selects the activity they want to carry out from the proposed plan and makes a store reservation or purchase a ticket. The device sends the reservation or purchase request to the server. The server receives the reservation or purchase request and connects with the systems of affiliated movie theaters and restaurants to make the reservation or purchase. Once completed, confirmation information is sent to the device. The device displays the reservation confirmation information to the user and automatically adds it to their calendar or reminders.

[1503] Activity execution and feedback

[1504] The user executes the proposed plan and performs the actual activities. The user generates ratings and feedback for each activity. The device sends the feedback information to the server. The server stores the received feedback in a database and uses it as training data for the AI ​​model to improve the accuracy of future plan suggestions.

[1505] Specific examples

[1506] An embodiment of this system will be explained using a specific scenario. For example, suppose a user inputs information such as "watching movies," "jogging," and "likes Japanese food," and has a budget of 20,000 yen and 10 days remaining. In this case, the AI ​​analyzes this data and external weather information to generate the following life plan:

[1507] 1. "I'm planning to see two movies at the cinema this weekend."

[1508] 2. "Jogging in a nearby park on a sunny day."

[1509] 3. "I recommend eating at a Japanese restaurant after jogging."

[1510] The user confirms the proposed plan and reserves the movie ticket. The server completes the reservation through the connected cinema system and sends the confirmation information to the terminal. After watching the movie, the user provides feedback, which is used to generate the next plan.

[1511] Example prompts for generative AI models

[1512] Below are some examples of prompts to input to an AI model:

[1513] "If the user likes watching movies, jogging, and eating Japanese food, generate the optimal life plan for a 10-day period with a budget of 20,000 yen."

[1514] "Please provide a recommended activity plan that takes into account the user's preferences (e.g., watching movies, jogging, Japanese food), current budget, and duration."

[1515] In this way, the system of the present invention automatically generates an optimal life plan that takes into account the user's budget and time, and supports the execution of that plan.

[1516] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1517] Step 1: Enter user data

[1518] User: Opens the dedicated application and enters information about their interests, health, and consumption. Examples of input data include a preference for watching movies, jogging, and Japanese food.

[1519] Terminal: Receives data entered by the user and sends this data to the server. The entered data is structured in JSON or XML format.

[1520] Step 2: Receiving and storing data

[1521] Server: Receives data sent from the device. The received content includes user interest information, health information, and consumption information.

[1522] Server: The received data is checked for errors to ensure it is properly formatted, and then saved to a database. The saved data is indexed and managed for each user.

[1523] Step 3: Gathering external information

[1524] Server: Calls external APIs to obtain external data such as weather information and local event information. The data obtained from the API is provided in the form of a JSON object containing weather forecasts and event information.

[1525] Server: The acquired external information is temporarily stored in memory and used for subsequent data analysis.

[1526] Step 4: Data analysis using AI

[1527] Server: Collects user interest information, health information, and consumption information from the database, and creates a dataset that also includes external information.

[1528] Server: Runs AI models using libraries such as TensorFlow and Keras to analyze the created datasets, using clustering algorithms and predictive models.

[1529] Server: As a result of the analysis, it generates an optimal life plan based on the user's budget and number of days. This plan includes detailed information about each activity (e.g., cost, location, and planned time).

[1530] Step 5: Create a life plan

[1531] Server: Based on the analysis results of the AI ​​model, the server generates an optimal life plan for the user, such as movie-going plans and the best times to do so, a jogging schedule in a nearby park, and recommendations for Japanese restaurants that take health into consideration.

[1532] Server: Formats the generated life plan and prepares it for transmission to the terminal.

[1533] Step 6: Present your plan

[1534] Server: Sends the generated life plan to the device. Push notifications may be used as a notification mechanism.

[1535] Terminal: Displays the user's life plan, with detailed information for each activity (cost, location, and planned time) presented in a user-friendly interface.

[1536] Step 7: Pre-order and purchase

[1537] User: Selects the activity they want to perform from the displayed list and makes a store reservation or purchases a ticket. For example, they click the "Go see this movie" button.

[1538] Terminal: Sends reservation or purchase requests to the server based on the user's selection.

[1539] Server: Processes the received request and executes the reservation or purchase in cooperation with the systems of affiliated stores and service providers. After the process is complete, it sends confirmation information to the terminal.

[1540] On the device: Display the reservation confirmation to the user and automatically add it to their calendar or reminders as appropriate.

[1541] Step 8: Activity execution and feedback

[1542] User: Executes the proposed plan and performs an actual activity, e.g., watches a movie at a cinema.

[1543] Users: Provide ratings and feedback for each activity, such as "The quality of the movie was great" or "The service at the Japanese restaurant was good."

[1544] Terminal: Sends feedback information to the server.

[1545] Server: The received feedback is stored in a database and used as learning data for the AI ​​model to improve the accuracy of the next plan proposal.

[1546] (Application example 1)

[1547] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1548] There is a lack of a means to automatically generate and present plans that take into account individual interests, health conditions, and consumption trends when users are trying to live an efficient and satisfying life within their budget. There is also a lack of support systems that allow users to easily make reservations and purchases at physical stores based on those plans. Furthermore, there is a need for a means to provide users with intuitive information in real time by displaying the generated plans on smart glasses or head-mounted displays.

[1549] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1550] In this invention, the server includes means for inputting user interest information, health information, and consumption information, means for linking with the user's past consumption data and external information, means for generating an optimal life plan based on the user's budget and number of days, means for presenting the generated life plan to the user, means for supporting reservations or purchases at physical stores based on the life plan, and means for displaying the generated life plan on smart glasses or a head-mounted display. This makes it possible to automatically generate plans tailored to the user's individual needs, simplify the reservation and purchase process, and provide information intuitively in real time.

[1551] "User interest information" is information about the user's hobbies and preferences.

[1552] "Health information" is information relating to the user's current health condition and past health checkup results.

[1553] "Consumption information" is information relating to a user's purchase history and spending patterns.

[1554] "Historical Consumption Data" is data recorded by a user that includes details of previous purchases and expenditures.

[1555] "External information" is data obtained from external sources, such as weather information and local event information.

[1556] A "budget" is a spending limit set by a user.

[1557] "Number of days" refers to the date set by the user as the validity period of the life plan.

[1558] The "optimal life plan" is a proposal of an optimal life plan for the user, generated based on the user's interest information, health information, consumption information, and external information.

[1559] "Physical store" refers to a physical store, such as a clothing store, restaurant, or movie theater.

[1560] A "reservation" is the act of reserving in advance the use or purchase of a specific service or product.

[1561] "Purchase" is the act of acquiring a service or product with money.

[1562] "Smart glasses" are eyeglass-type devices that have the function of displaying information in the wearer's field of vision.

[1563] A "head-mounted display" is a display worn on a user's head that provides virtual or augmented reality visual information.

[1564] "User feedback" is information regarding opinions and evaluations provided by users regarding the proposed life plan.

[1565] "Real-time" refers to processing and information being provided immediately at that moment.

[1566] The system embodying the present invention supports users in living an efficient and satisfying life within their budget, and involves cooperation between users, terminals, and a server. Specific embodiments of the system are described below.

[1567] First, a user launches an application using a device such as a smartphone, smart glasses, or head-mounted display. The user then inputs their interest information (hobbies, favorite foods, etc.), health information (health status), and consumption information (purchase history, etc.) into the application. The input information is received by the device and sent to the server.

[1568] The server stores the interest, health, and consumption information sent by the user in a database. The server also obtains external information such as weather and local event information via API. The AI ​​model on the server analyzes this data and generates an optimal life plan based on the user's budget and number of days.

[1569] The generated life plan includes specific activity suggestions, such as watching a movie, jogging, or eating at a Japanese restaurant, along with detailed information such as the cost, location, and estimated time for each activity. This life plan is then sent by the server to the device and presented to the user.

[1570] Users can select the desired activity from the proposed life plan and make a reservation or purchase a ticket at a physical store. The device sends the reservation or purchase request to the server, which then completes the reservation or purchase by linking with the systems of affiliated stores and services. Once completed, confirmation information is sent to the device and displayed to the user. If using smart glasses or a head-mounted display, the information is displayed in real time, allowing the user to receive the information intuitively.

[1571] In addition, users can provide ratings and feedback on the activities they have performed. This feedback information is also sent to the server and stored in a database. The server uses this feedback as learning data for the AI ​​model and reflects it in the next life plan generation, thereby improving the accuracy of suggestions.

[1572] For example, if a user inputs information such as "watching movies," "jogging," and "likes Japanese food," along with a budget of 20,000 yen and 10 days remaining, the system will generate the following life plan:

[1573] 1. "I'm planning to see two movies at the cinema this weekend."

[1574] 2. "Jogging in a nearby park on a sunny day."

[1575] 3. "I recommend eating at a Japanese restaurant after jogging."

[1576] An example of a prompt sentence in this case is: "The user likes watching movies, jogging, and Japanese food. The budget is 20,000 yen and there are 10 days left. The weather is sunny on weekends and cloudy or rainy on other days. Please generate the optimal activity plan."

[1577] These steps allow users to automatically generate personalized plans, simplify the booking and purchasing process, and provide intuitive information in real time.

[1578] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1579] Step 1:

[1580] A user launches an application using a smartphone or smart glasses and inputs information about their interests, health, and consumption. This information is received by the device as user input. For example, the user inputs information such as "watching movies," "jogging," and "I like Japanese food." The device then sends the input data to the server.

[1581] Step 2:

[1582] The server receives the user's interest, health, and consumption information sent from the device and stores it in a database. At the same time, the server uses an external API to obtain weather information and local event information. This external information is also stored in the database. Input data includes the user's preferences and health status, past consumption data, weather information, and event information.

[1583] Step 3:

[1584] The AI ​​model on the server collects and analyzes the user's interest, health, consumption, and external information from a database. This analysis generates an optimal life plan based on the user's budget and number of days. For example, given a user's budget of 20,000 yen and 10 days remaining, the AI ​​model suggests activities such as watching a movie, jogging, and eating at a Japanese restaurant.

[1585] Step 4:

[1586] The server sends the generated life plan to the device, which receives it and displays it to the user. The displayed content includes detailed information about each activity (e.g., movie viewing plans, jogging time, location and cost of Japanese restaurants, etc.).

[1587] Step 5:

[1588] The user selects the desired activity from the proposed life plan. Based on the selected activity, the user makes a reservation or purchase request at a physical store. The device sends these requests to the server.

[1589] Step 6:

[1590] The server receives reservation and purchase requests sent by users and completes the reservation or purchase in cooperation with the systems of affiliated stores and services. After completion, reservation confirmation information is sent to the device. The device receives this information and displays it to the user. If smart glasses or a head-mounted display are used, this confirmation information is also displayed in real time.

[1591] Step 7:

[1592] Users provide ratings and feedback for the activities they perform. The device collects this feedback information and sends it to the server. The server stores the feedback information in a database and uses it as learning data for the AI ​​model to improve the accuracy of future life plan suggestions.

[1593] In this way, all steps work together to enable users to realize the optimal life plan while responding to their individual needs and using their budget and time efficiently.

[1594] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1595] The present invention relates to an AI-based support system for helping users live an efficient and satisfying life within their budget, and in particular to improving the accuracy of life plans by incorporating an emotion engine. Hereinafter, embodiments of the present invention will be described in detail.

[1596] Natural language description of the program

[1597] This system exchanges information between the user, device, and server, and not only generates an optimal life plan based on the user's preferences and past consumption data, but also has the ability to recognize the user's emotions in real time and adjust the plan accordingly.

[1598] 1. Entering user data

[1599] User: Opens the application and enters their interests (hobbies, favorite foods, etc.) and health information (current health condition). For emotion recognition, the application also allows voice input and facial recognition. The application also allows users to connect to external services (e.g., PayPay) to obtain past consumption data.

[1600] Terminal: Receives input interest information, health information, and consumption information and sends it to the server.

[1601] 2. Receipt and storage of data

[1602] Server: Receives user information (interest information, health information, consumption information) and emotion data sent from the device and stores it in a database.

[1603] 3. AI-based data analysis

[1604] Server: Collects user interest, health, and consumption information from a database, and also uses external APIs to obtain real-time weather information and local event information.

[1605] Server: Using an emotion engine, the server analyzes the voice, facial expressions, and text data sent from the device in real time to recognize the user's emotional state.

[1606] 4. Creating a life plan

[1607] Server: The collected data and the analysis results of the emotion engine are input into the AI ​​model to generate an optimal life plan based on the user's budget and number of days. The AI ​​model takes into consideration the user's past consumption patterns, preferences, current emotional state, weather information, etc.

[1608] 5. Plan generation and adjustment

[1609] Server: The life plan generated by the AI ​​model includes specific activity suggestions (e.g., watching a movie, jogging, or eating at a Japanese restaurant) along with detailed information about costs, locations, and planned times.

[1610] Server: Reflects the user's real-time emotional state and adjusts the plan, for example, by suggesting a plan that emphasizes relaxation activities if the user is under high stress, or by suggesting more active activities if the user is in a good mood.

[1611] 6. Presenting the plan

[1612] Server: Sends the generated life plan to the device.

[1613] Device: Displays the received life plan to the user. The screen shows detailed information about each activity (cost, location, planned time) and adjustment points based on the user's emotional state.

[1614] 7. Reservations and Purchases

[1615] User: Selects the activities they want to carry out from the proposed life plan and makes store reservations or purchases tickets as necessary.

[1616] Terminal: Sends reservation and purchase requests to the server.

[1617] Server: Receives reservation and purchase requests, coordinates with affiliated cinema and restaurant systems to complete reservations and purchases, and sends confirmation information to the device upon completion.

[1618] On your device: Display reservation confirmation information to the user and automatically add it to your calendar and reminders.

[1619] 8. Activity execution and feedback

[1620] User: Carry out the proposed plan, for example, watch a movie at a cinema, go jogging in a park, or enjoy a meal at a Japanese restaurant.

[1621] Users: Provide ratings and feedback for each activity.

[1622] Terminal: Sends user feedback information to the server.

[1623] Server: The received feedback is stored in a database and used as training data for the AI ​​model to propose future plans, improving the accuracy of subsequent plan generation.

[1624] Specific examples

[1625] An embodiment of this system will be explained using a specific scenario. For example, suppose a user inputs information such as "watching movies," "jogging," and "likes Japanese food," has a budget of 20,000 yen, and has 10 days left. The AI ​​analyzes this data and external weather information, and the emotion engine recognizes that the user's current stress level is high, and generates the following life plan:

[1626] 1. "I'm planning to see two movies at the cinema this weekend."

[1627] 2. "Jogging in a nearby park on a sunny day."

[1628] 3. "I recommend eating at a Japanese restaurant after jogging."

[1629] 4. "We also suggest visiting a relaxing hot spring facility to reduce stress."

[1630] The user checks the proposed plan and reserves movie tickets. The server completes the reservation through the connected cinema system and sends the confirmation information to the terminal. After watching the movie, the user provides feedback, which is used to generate the next plan. The emotion engine analyzes the user's emotions in real time, and the system always provides a plan optimized for the user's emotional state.

[1631] In this way, the system of the present invention takes into consideration the user's budget and time, and also analyzes and reflects their emotional state in real time to automatically generate an optimal life plan and support its implementation.

[1632] The processing flow will be explained below.

[1633] Step 1:

[1634] User: Opens the application and enters their interests (hobbies, favorite foods, etc.) and health information (current health condition). In addition, they authorize voice input and face recognition functions, and set up the application to connect to external services (e.g., PayPay) to obtain past consumption data.

[1635] Step 2:

[1636] Terminal: Receives interest information, health information, and consumption information entered by the user and sends this data to the server.

[1637] Step 3:

[1638] Server: Receives user information (interest information, health information, consumption information) and emotion data sent from the device and stores it in a database.

[1639] Step 4:

[1640] Server: Collects user interest, health, and consumption information from a database, and also uses external APIs to obtain real-time weather information and local event information.

[1641] Step 5:

[1642] Server: Using an emotion engine, the server analyzes the voice, facial expressions, and text data sent from the device in real time to recognize the user's emotional state.

[1643] Step 6:

[1644] Server: The collected data and the analysis results of the emotion engine are input into the AI ​​model to generate an optimal life plan based on the user's budget and number of days. The AI ​​model takes into consideration the user's past consumption patterns, preferences, current emotional state, weather information, etc.

[1645] Step 7:

[1646] Server: The generated life plan includes specific activity suggestions (e.g., watching a movie, jogging, or eating at a Japanese restaurant) with detailed information about costs, locations, and estimated times.

[1647] Step 8:

[1648] Server: Reflects the user's real-time emotional state and adjusts the plan, for example, by suggesting a plan that emphasizes relaxation activities if the user is under high stress, or by suggesting more active activities if the user is in a good mood.

[1649] Step 9:

[1650] Server: Sends the generated life plan to the device.

[1651] Step 10:

[1652] Device: Displays the received life plan to the user. The screen shows detailed information about each activity (cost, location, planned time) and adjustment points based on the user's emotional state.

[1653] Step 11:

[1654] User: Selects the activities they want to carry out from the proposed life plan and makes store reservations or purchases tickets as necessary.

[1655] Step 12:

[1656] Terminal: Sends reservation and purchase requests to the server.

[1657] Step 13:

[1658] Server: Receives reservation and purchase requests, coordinates with affiliated cinema and restaurant systems to complete reservations and purchases, and sends confirmation information to the device upon completion.

[1659] Step 14:

[1660] On your device: Display reservation confirmation information to the user and automatically add it to your calendar and reminders.

[1661] Step 15:

[1662] User: Carry out the proposed plan, for example, watch a movie at a cinema, go jogging in a park, or enjoy a meal at a Japanese restaurant.

[1663] Step 16:

[1664] Users: Provide ratings and feedback for each activity.

[1665] Step 17:

[1666] Terminal: Sends user feedback information to the server.

[1667] Step 18:

[1668] Server: The received feedback is stored in a database and used as training data for the AI ​​model to propose future plans, improving the accuracy of subsequent plan generation.

[1669] Example 2

[1670] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1671] When generating an efficient and satisfying life plan within a user-specified budget and number of days, there was a problem in that the system was unable to recognize the user's emotional state in real time and adjust the plan accordingly. It was also difficult to dynamically generate plans that incorporated weather information and local event information, and it was also a challenge to efficiently utilize the user's personal preferences and past consumption data. Another problem was the lack of a feedback loop to collect user feedback and reflect it in the next plan.

[1672] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1673] In this invention, the server includes: means for inputting user interest information, health information, and consumption information; means for linking with the user's past consumption data and external information; means for allowing voice input and facial recognition for emotion recognition; means for generating an optimal life plan based on the user's budget and number of days; means for analyzing the user's emotional state in real time and adjusting the life plan based on the analysis; means for presenting the generated life plan to the user; means for assisting with store reservations or ticket purchases based on the life plan; and means for collecting user feedback and reflecting it in the generation of the next plan. This allows for the generation of an optimized life plan in real time, taking the user's emotional state into consideration. Furthermore, incorporating weather information and local event information allows for the provision of more dynamic and personalized plans. Furthermore, by creating a feedback loop, the accuracy of subsequent plans can be improved, resulting in an overall improved user experience.

[1674] "Interest information" is information about the user's hobbies, preferences, and tastes.

[1675] "Health information" refers to information related to health, such as the user's current health condition and medical history.

[1676] "Consumption information" is data related to consumption activities that a user has engaged in in the past.

[1677] "External information" refers to information obtained from external databases or APIs, such as weather information and local event information.

[1678] "Emotion recognition" is a technology that analyzes a user's voice and facial expressions to determine their current emotional state.

[1679] "Voice input" is a means of capturing user voice data as input.

[1680] "Facial recognition" is a technology that analyzes a user's facial features and identifies them as an individual.

[1681] A "life plan" is a specific action plan proposed based on the user's budget and number of days.

[1682] "Real-time" means that data is analyzed immediately and necessary processing is carried out at that moment.

[1683] "Analysis" is the process of examining data in detail and deriving patterns and trends.

[1684] "Store reservation" refers to making a reservation in advance at a store that the user plans to visit.

[1685] "Ticket purchase" means the purchase of an admission ticket to participate in an event or activity.

[1686] "Feedback" refers to opinions and evaluations provided by users, and is information that can be used to improve services next time.

[1687] This invention is an AI-based support system that allows users to generate and adjust an optimal life plan within their budget and time frame. This system consists of three main components: the user, the device, and the server. It analyzes the user's emotional state in real time based on their interest, health, and consumption information, and provides a life plan that reflects that information.

[1688] Entering User Data

[1689] User: The user opens the application and enters their interest information (such as hobbies and favorite dishes), health information, and consumption information. They also configure the application to enable voice input and facial recognition, which provides basic data for emotion recognition. They also configure the application to connect to external services (e.g., electronic payment systems) and grant permission to obtain past consumption data.

[1690] Receiving and storing data

[1691] Terminal: The terminal sends all data entered by the user to the server. It also validates the user's input to ensure that the data is accurate.

[1692] Server: The server receives the data sent from the device and stores it in a database. The stored data is used for further analysis and calculation.

[1693] AI-based data analysis

[1694] Server: The server collects user information from a database and, if necessary, uses external APIs to obtain real-time weather information and local event information. It also uses an emotion engine to analyze voice, facial expressions, and text data to recognize the user's emotional state. Emotion recognition uses voice recognition algorithms and image recognition algorithms.

[1695] Creating and adjusting life plans

[1696] Server: Based on the collected data and the analysis results of the emotion engine, the data is input into the AI ​​model to generate an optimal life plan based on the user's budget and number of days. The AI ​​model takes into account past spending patterns, interests, current emotional state, weather information, etc. The generated plan includes specific activities such as going to the movies, jogging, and eating at a Japanese restaurant, along with the cost, location, and planned time for each.

[1697] Presenting and adjusting plans

[1698] Server: Sends the generated life plan to the device. The plan reflects the user's real-time emotional state and is dynamically adjusted according to the specific situation.

[1699] Device: Displays the received life plan to the user, along with detailed information on each activity and adjustment points.

[1700] Reservation and purchase assistance

[1701] User: The user selects the activities they want to carry out from the proposed life plan and makes store reservations or purchases tickets as necessary.

[1702] Terminal: Sends reservation and purchase requests to the server.

[1703] Server: The server receives reservation and purchase requests, executes them in cooperation with partner systems, and sends confirmation information to the terminal once completed.

[1704] On the device: Display reservation confirmation information to the user and automatically add it to their calendar or reminders.

[1705] Activity execution and feedback

[1706] User: Carry out the proposed plan and enjoy watching a movie, jogging, and eating at a Japanese restaurant.

[1707] Users: Provide ratings and feedback for each activity.

[1708] Terminal: Sends feedback information to the server.

[1709] Server: The received feedback is stored in a database and used as training data for the AI ​​model to propose future plans, improving the accuracy of subsequent plan generation.

[1710] Specific examples

[1711] For example, suppose a user inputs information such as "watching movies," "jogging," and "likes Japanese food," and requests a 10-day plan with a budget of ¥20,000. The system takes this data into account along with real-time weather information, and the emotion engine recognizes that the user's current stress level is high, generating the following life plan:

[1712] 1. "I'm going to see two movies at the cinema this weekend."

[1713] 2. "Jogging in a nearby park on a sunny day."

[1714] 3. "Recommend a meal at a Japanese restaurant after jogging."

[1715] 4. "We also suggest visiting a relaxing hot spring facility to reduce stress."

[1716] The user checks the proposed plan and reserves movie tickets. The server completes the reservation with the affiliated cinema system and sends the confirmation information to the terminal. After watching the movie, the user provides feedback, which is reflected in the next plan generation. The emotion engine analyzes the user's emotions in real time, so that the system always provides the most optimized plan.

[1717] Prompt Sentence Examples

[1718] "Watching movies," "Jogging," "Like Japanese food," "Budget is 20,000 yen," "Prefer a 10-day plan," "Recognizes and adjusts emotions in real time," "Includes relaxing activities."

[1719] In this way, the present invention automatically generates an optimal life plan that takes into account the user's budget and time, and also analyzes and reflects their emotional state in real time, and supports their execution.

[1720] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1721] Step 1: Enter user data

[1722] The user opens the application and inputs interest, health, and consumption information. The user also sets permissions for voice input and facial recognition. Furthermore, the user sets up connections with external services (e.g., electronic payment systems) and gives permission to obtain past consumption data.

[1723] Input: Interest information (hobbies, preferences), health information, consumption information, voice input permission, face recognition permission, external service linkage permission

[1724] Output: User dataset to send to terminal

[1725] What happens: A user enters information into an application form, and after completing each field, presses a confirmation button. The form data is packaged in JSON format and ready to be sent to the device.

[1726] Step 2: Receiving and storing data

[1727] The terminal transmits the data entered by the user to the server, where it verifies the consistency and integrity of the data and performs error checking before transmission.

[1728] The server receives the data sent from the terminal, verifies it, and then stores it in a database.

[1729] Input: A user-entered dataset

[1730] Output: User data stored in the database

[1731] Specific operation: The terminal sends an HTTP POST request to the server, the server receives the data, parses it, checks for errors and validates the data format, and then stores it in the database.

[1732] Step 3: Data analysis using AI

[1733] The server collects user information from a database and also uses external APIs to obtain current weather information and local event information.

[1734] The server uses an emotion engine to analyze the voice data, facial expression data, and text data sent from the terminal in real time and identify the user's emotional state.

[1735] Input: User information collected from the database, weather information obtained from external APIs, local event information, emotion data

[1736] Output: User's emotional state as an analysis result and supplementary information

[1737] Specific operation: The server executes a scheduled task to obtain user information and sends a request to an external API (e.g., weather API). The obtained data is input into the emotion engine for real-time analysis.

[1738] Step 4: Create a life plan

[1739] The server inputs the collected data and the results of the emotion engine's analysis into an AI model to generate an optimal life plan based on the user's budget and number of days.

[1740] Input: Collected user data, emotion engine analysis results, external API data

[1741] Output: Generated life plan

[1742] Specific actions: Input data into the AI ​​model and run inference. The generated life plan includes specific activity suggestions, with details (cost, location, and planned time) automatically calculated and entered.

[1743] Step 5: Present and adjust the plan

[1744] The server sends the generated life plan to the device, which adjusts the plan as needed to reflect the user's real-time emotional state.

[1745] The terminal displays the received life plan to the user.

[1746] Input: Generated life plan

[1747] Output: The adjusted life plan displayed to the user

[1748] Specific operation: The server sends plan data in JSON format to the device, which receives the data and displays it on the interface. Plan adjustments are made based on the results of real-time sentiment analysis, and adjustment points are presented to the user.

[1749] Step 6: Reservation and purchase assistance

[1750] Users can select the activities they want to carry out from the proposed life plan and make reservations or purchase tickets.

[1751] The terminal sends a reservation request or a purchase request to the server.

[1752] The server cooperates with the affiliated system to execute the reservation or purchase and transmits confirmation information to the terminal.

[1753] The device will display the reservation confirmation information to the user and automatically add it to their calendar or reminders.

[1754] Input: User-selected activity booking / purchase request

[1755] Output: Confirmation information and automatic addition to calendar and reminders

[1756] Specific operation: The user selects an activity and enters the required information in the reservation form. The device sends a request to the server, and the server returns the execution result to the device. The device displays a confirmation and automatically adds it to the calendar and reminders.

[1757] Step 7: Activity execution and feedback

[1758] The user carries out the proposed plan and enjoys watching a movie, jogging, and eating at a Japanese restaurant.

[1759] Users provide ratings and feedback for each activity.

[1760] The terminal transmits the feedback information to the server.

[1761] The server stores the received feedback in a database and uses it as training data for the AI ​​model to propose future plans.

[1762] Input: Feedback data after the user performs an activity

[1763] Output: Feedback stored in the database as training data

[1764] Specific operation: The user fills out a feedback form, which the device sends to the server. The server stores the feedback in a database and uses it to generate the next life plan.

[1765] (Application example 2)

[1766] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1767] In modern society, autonomous vehicles offer significant improvements in user comfort and efficiency, but their capabilities are not being fully utilized. Furthermore, there are many challenges in providing optimal life plans that take into account the user's emotional state and real-time information. Specifically, plans are not adequately generated based on the user's interests and health status, and real-time adjustments based on emotion recognition are insufficient. Furthermore, conventional systems have difficulty generating route proposals that take into account weather and event information. Therefore, effective means of improving user satisfaction and comfort are needed.

[1768] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1769] In this invention, the server includes means for recognizing the user's emotional state in real time and adjusting the life plan, means for generating and presenting route proposals based on the user's emotions and real-time information within the autonomous vehicle, and means for acquiring weather information and event information using an external API, thereby enabling the generation of a life plan and route proposals optimized for the user's real-time emotional state and external environment.

[1770] "User interest information" is data that indicates the user's preferred activities, hobbies, and interests.

[1771] "Health information" is data relating to the user's current health condition and past health history.

[1772] "Consumption information" is data indicating products and services purchased by a user in the past, the amount spent on those products and services, and so on.

[1773] "Past consumption data" refers to data including the user's past purchasing behavior and its history.

[1774] "External information" refers to information obtained from outside the system, such as weather, local events, and traffic conditions.

[1775] A "budget" is data that indicates a range of available amounts set by the user.

[1776] "Number of days" is data indicating the number of days within a specific period set by the user.

[1777] The "means for generating a life plan" is a process for creating a plan for the user to live an efficient and satisfying life based on collected user data and external information.

[1778] The "means for presenting a life plan to a user" refers to a method for visually or audibly displaying or notifying a user of a created life plan.

[1779] "Means for supporting store reservations or ticket purchases" are methods that allow users to easily make reservations or purchases based on the proposed life plan.

[1780] "Means for recognizing emotional state in real time" refers to technology that analyzes the user's voice and facial expressions to determine their current emotions.

[1781] The "means for generating and presenting route proposals" is a technology that plans an optimal travel route based on the user's emotional state and real-time environmental information, and displays it on the vehicle terminal.

[1782] "Means of obtaining weather and event information using external APIs" refers to a method of obtaining real-time weather and local event information by linking with external data provision services.

[1783] The system of this invention inputs a user's interest information, health information, and consumption information, and generates an optimal life plan based on the budget and number of days in conjunction with past consumption data and external information. The system also incorporates an emotion engine to recognize the user's real-time emotional state and dynamically adjust the life plan.

[1784] Hardware and Software Configuration

[1785] The server performs various data processing and calculations using a database, AI analysis engine, emotion engine, external API connection module, etc. These components will be explained in detail below.

[1786] 1. User Device:

[1787] Input devices (smartphones, tablets, etc.)

[1788] Emotion recognition devices (camera, microphone, etc.)

[1789] 2. Server:

[1790] Database (SQL, etc.)

[1791] AI analysis engine (TensorFlow, Keras, etc.)

[1792] Emotion Engine API

[1793] External API connection module (weather API, event API)

[1794] Data processing and calculation procedures

[1795] The server first receives interest, health, and consumption information from the user's device and stores it in a database. This information, along with real-time weather information and local event information obtained through external APIs, is then input into an AI analysis engine.

[1796] The server uses an emotion engine to analyze the user's voice, facial expressions, and input text in real time to recognize the user's emotional state. The recognized emotional data, along with other data, becomes the input data required to generate a life plan.

[1797] The generated life plan includes specific activity suggestions based on the user's schedule, activities, and budget. For example, the system suggests optimal routes based on the user's emotional state and real-time information in an autonomous vehicle. The system also supports users in making reservations and purchasing tickets, and the server handles these procedures.

[1798] Specific examples

[1799] For example, suppose a user enters information into the app such as "I want to relax" and "I like Japanese food," and sets a budget of 20,000 yen. The server generates the following life plan based on real-time weather information and local event information:

[1800] 1. "Plan a relaxing picnic in the park on a sunny day."

[1801] 2. "I suggest we go to a Japanese restaurant afterwards."

[1802] 3. "We recommend visiting a hot spring facility to relax in the evening."

[1803] If the emotion engine recognizes that the user's stress level is high, it will focus on suggesting more relaxing activities.

[1804] Prompt Sentence Examples

[1805] "The user wants to relax, so generate a route for stress reduction."

[1806] "Generate tonight's dinner plans for the user. My budget is 5000 yen and I'm in the mood to relax."

[1807] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1808] Step 1:

[1809] The user's device inputs information about the user's interests, health, and consumption. The user enters this data into the app from their smartphone or tablet and sends it to the server. Input may also be done using the user's voice or facial recognition function.

[1810] Input: User interest information, health information, consumption information

[1811] Output: Send data to the server

[1812] Step 2:

[1813] The server receives the information sent from the user terminal and stores it in a database. Based on this data, the server aggregates the data necessary to generate a life plan.

[1814] Input: User interest information, health information, consumption information

[1815] Output: Save data to a database

[1816] Step 3:

[1817] The server uses external APIs to obtain real-time weather and event information. For this purpose, it uses the weather API and event API.

[1818] Input: Real-time information request from an external API

[1819] Output: Get real-time weather and event information

[1820] Step 4:

[1821] The server uses an emotion engine to analyze the user's voice, facial expressions, and text in real time to recognize the user's emotional state, thereby determining the user's stress level and mood.

[1822] Input: User's voice data, facial expression data, text data

[1823] Output: Emotional state data

[1824] Step 5:

[1825] The server uses an AI analysis engine to generate an optimal life plan based on the user's data stored in the database, as well as real-time weather information, event information, and emotional state. The generated life plan includes the user's budget and details of each activity.

[1826] Input: User data, real-time weather information, event information, emotional state data

[1827] Output: Optimal Life Plan

[1828] Step 6:

[1829] The server transmits the generated life plan to the user terminal, which displays the plan and presents it to the user.

[1830] Enter: Optimal Life Plan

[1831] Output: Sending the life plan to the user's device

[1832] Step 7:

[1833] The user terminal supports store reservations and ticket purchase procedures based on the life plan. Reservation and purchase requests are sent to the server, which then cooperates with affiliated store systems to complete the procedures.

[1834] Input: User reservation request, purchase request

[1835] Output: Store reservation and ticket purchase confirmation information

[1836] Step 8:

[1837] The user executes the proposed life plan and provides feedback for each activity. The user terminal collects this feedback information and sends it to the server. The server stores the collected feedback in a database and uses it to generate the next plan.

[1838] Input: User feedback data

[1839] Output: Save feedback data to database

[1840] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1841] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1842] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1843] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1844] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1845] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1846] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1847] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1848] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1849] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1850] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1851] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1852] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1853] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1854] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1855] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1856] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1857] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1858] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1859] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. ...

Claims

1. A means for inputting user interest information, health information, and consumption information; A means for linking the user's past consumption data and external information; means for generating an optimal life plan based on the user's budget and number of days; means for presenting the generated life plan to a user; A means for supporting store reservations or ticket purchases based on the life plan; A system including:

2. The system of claim 1 , further comprising means for taking weather information into account in generating the life plan.

3. The system according to claim 1 , further comprising means for collecting user feedback in generating the life plan and reflecting the feedback in the next plan generation.

Images