system

Abstract

We provide the system. [Solution] A means of storing personal attribute information received from a user in information storage, Means for obtaining weather information and congestion information from external sources, A means of generating an optimal activity plan for a user using a machine learning algorithm, A means for transmitting the generated activity plan to the user's communication device, A means for processing activity plan selection information received from the user, A means of analyzing user feedback information and storing it in information storage to reflect it in the generation of the next activity plan, A system that includes means of providing users with local information and route guidance in real time.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] It is difficult for many individuals to make their own plans to spend holidays meaningfully, and as a result, there is a problem that holidays pass by idly. Also, even if a plan is made, there is a lack of real-time information during execution, and there is also a problem that it is impossible to appropriately respond to changes such as weather and congestion conditions. Such a situation has become a factor in reducing the quality and satisfaction of an individual's life.

Means for Solving the Problems

[0005] This invention provides a system that generates an optimal plan for a user by acquiring external data such as weather information and congestion information based on personal attribute information from the user, and applying a machine learning model. The generated plan is transmitted to the user's communication device, and real-time location information and guidance information based on the plan selected by the user are also provided. Furthermore, the system includes means to improve the accuracy of the plan by receiving feedback from the user, accumulating information in a database used for generating the next plan, and updating the machine learning model.

[0006] A "user" is someone who uses the system to provide personal information or activity plan requests.

[0007] "Personal attribute information" refers to information such as a user's name, age, and hobbies, which is used to identify a user and provide personalized services.

[0008] A "database" is an information management system that systematically stores user personal attribute information, feedback, and other related data, making it accessible as needed.

[0009] "External data sources" refer to external information providers or APIs that the system accesses to provide additional information related to the user's activity plan, such as weather information and congestion information.

[0010] A "machine learning model" is a set of algorithms that generate the optimal activity plan based on a user's past data and real-time information.

[0011] "Communication equipment" refers to digital devices used by users to receive activity plans presented by a system or to transmit information.

[0012] "Feedback" refers to information that users report to the system regarding their satisfaction level and evaluation of the activity plan they experienced.

[0013] "Real-time location information" refers to instantly updated geographical information provided based on the user's current location.

[0014] "Guidance information" refers to support information such as traffic information and directions provided to assist users in carrying out their activity plans. [Brief explanation of the drawing]

[0015] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13]It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when the emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when the emotion engine is combined.

Mode for Carrying Out the Invention

[0016] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a processor with a reference numeral (hereinafter simply referred to as "processor") may be one arithmetic unit or a combination of a plurality of arithmetic units. Also, the processor may be one type of arithmetic unit or a combination of a plurality of types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0019] In the following embodiments, a RAM (Random Access Memory) with a reference numeral is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0020] In the following embodiments, a storage with a reference numeral is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0021] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0023] [First Embodiment]

[0024] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0025] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0028] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0031] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0035] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0036] This invention is a system that helps users have more meaningful holidays. This system presents an activity plan based on the user's personal attribute information and provides real-time support for that plan. Its specific operation is described below.

[0037] Users first register their personal information with the system using their device. This includes personal attribute information such as name, age, and hobbies. Once registration is complete, this information is stored in a database by the server and used to generate future activity plans.

[0038] On a holiday morning, users request an activity plan using the chat interface on their device. The server accepts this request, investigates the user's profile and past activity history, and then retrieves current weather information and congestion status from external data sources. Subsequently, considering this data and the user's preferences, a machine learning model is used to generate the optimal activity plan.

[0039] The generated plan is sent from the server to the user's device. The user selects an action plan from several presented options, and this selection is notified to the server. Once the user starts going out, the server provides the user with real-time location information and directions. This feature allows users to make appropriate decisions based on the situation and carry out their activities more systematically.

[0040] After the activity ends, users send feedback to the system regarding their satisfaction with the plan and their experience. The server stores this feedback in a database and uses it to continuously improve the machine learning model. This enables more accurate personalization in future plan generation, further improving user satisfaction.

[0041] As a concrete example, the personal information registered by user A includes preferences such as "I like nature" and "I enjoy the outdoors." Based on this information, the server suggests an activity plan that includes hiking in a nearby park and lunch at a popular local cafe on a day with good weather. User A selects this plan and can enjoy the day while receiving navigation and updates on congestion levels from their device. After the activity, user A provides feedback on their satisfaction level, and the server uses this information to improve the accuracy of future plans.

[0042] The following describes the processing flow.

[0043] Step 1:

[0044] Users use a device to enter personal information (name, age, hobbies, etc.) and register it in the system.

[0045] Step 2:

[0046] The terminal sends the entered personal attribute information to the server.

[0047] Step 3:

[0048] The server verifies the received personal attribute information and stores it in the database.

[0049] Step 4:

[0050] On holiday mornings, users use their device's chat interface to send activity plan requests to the system.

[0051] Step 5:

[0052] The server retrieves the user's profile information and past behavioral history, preparing the basic data for plan generation.

[0053] Step 6:

[0054] The server obtains weather and congestion information from external data sources and collects real-time information necessary for plan generation.

[0055] Step 7:

[0056] The server uses machine learning models to generate the optimal activity plan for the user. This process takes into account user preferences, weather, and congestion levels.

[0057] Step 8:

[0058] The server sends the generated activity plan to the user's device and prompts them to make a selection.

[0059] Step 9:

[0060] The user reviews the activity plan presented through their device and selects a plan to execute. The selection is then sent to the server.

[0061] Step 10:

[0062] The server receives the user's selection and prepares to provide real-time information while the user is away from their computer.

[0063] Step 11:

[0064] While the user is out, the server continues to provide location and navigation information to the user's device in real time.

[0065] Step 12:

[0066] After the activity ends, the user sends feedback about the plan to the server from their device.

[0067] Step 13:

[0068] The server stores the received feedback in a database and uses it to update the machine learning model for future plan generation.

[0069] (Example 1)

[0070] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0071] It is difficult to efficiently propose activity plans that match the individual preferences and circumstances of each user regarding how to spend their holidays. Furthermore, it is necessary to utilize environmental information and user feedback and effectively reflect this in future plans. In addition, a system is needed that provides real-time support and helps users make appropriate decisions based on their circumstances.

[0072] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0073] In this invention, the server includes means for storing personal characteristic information received from the user in a storage device, means for acquiring environmental information and situational information from external information sources, and means for generating an optimal activity plan for the user using a machine learning model. This makes it possible to present a personalized activity plan according to the user's characteristics and circumstances, and to improve user satisfaction through real-time guidance and support.

[0074] A "user" refers to a person who uses an information system, provides personal information, and receives an activity plan.

[0075] "Personal characteristics information" refers to attribute information such as the user's name, age, and hobbies, which is stored in a database and used to generate activity plans.

[0076] A "storage device" refers to a hardware or software system for storing data or programs.

[0077] "External information sources" refer to other systems or platforms that provide various data from the internet or APIs.

[0078] "Environmental information" refers to data about external conditions that affect user activities, such as weather information and congestion information.

[0079] "Status information" refers to data related to the user's situation at that time, such as real-time location information and route information.

[0080] A "machine learning model" refers to an algorithm that learns patterns based on large amounts of data and generates an optimal activity plan for the user.

[0081] "Communication device" refers to a device used to send and receive data between a server and a user, such as a smartphone or computer.

[0082] "Evaluation information" refers to feedback received from users, including data on satisfaction with activity plans and the content of their experiences.

[0083] This invention is a system that proposes and supports activity plans tailored to the user's preferences and circumstances in real time. Specific embodiments of this system are shown below.

[0084] Users access the system using their own devices and enter personal information, including their name, age, and hobbies. This information is received by the server and stored in a database. This stored personal information is later used to generate activity plans.

[0085] On a holiday morning, users request an activity plan by entering a prompt message into the system's chat interface via their terminal. For example, they might send a prompt message like, "What activities are recommended for today?" In response to this request, the server retrieves the user's profile information and past history data, and gathers environmental and situational information from external sources.

[0086] Based on this data, the server generates an optimal activity plan using a generative AI model. This AI model incorporates machine learning algorithms and is designed to provide personalized suggestions to the user. The generated activity plan is sent from the server to the user's communication device, where multiple options are presented.

[0087] The user selects an activity plan that interests them from several options presented, and this selection is notified to the server. Subsequently, as the user proceeds with the activity, the server provides the user with real-time location and route information, supporting the activity through the device.

[0088] After an activity is completed, users send feedback to the system evaluating their experience. The server stores this evaluation information in a database and updates the AI ​​model for generating activity plans, which can then be used to create more personalized and accurate activity plans for future activities.

[0089] For example, if user A has registered information such as "I like nature" and "I'm an outdoorsy person," the server will suggest an activity plan based on that information, including hiking and lunch at a local cafe on a day with good weather. User A can select this plan and enjoy the day using navigation and real-time information from their device.

[0090] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0091] Step 1:

[0092] The user uses a terminal to input and transmit personal information into the system. This information includes name, age, preferences, etc. The terminal sends this input to the server. The server receives the entered personal information and stores it in a database as storage. In this step, the user provides data to personalize the system, and the system processes and stores that data.

[0093] Step 2:

[0094] On a holiday morning, a user uses the chat interface from their device to send a prompt message to the system. For example, they might enter a request such as, "What activities do you recommend today?" The device sends this prompt message to the server. The server receives the prompt message and retrieves personal information from its database based on the user ID. This step involves processing to prepare appropriate information in response to the user's request.

[0095] Step 3:

[0096] The server calls external APIs to obtain environmental information (weather information, congestion information, etc.) from external sources. This input includes location information and date / time. Based on this data, the server retrieves data that affects the user's current situation. Then, it integrates the retrieved data and creates a dataset that combines it with the user's personal characteristics information. In this step, real-time external information is ingested and a customized dataset is created for each user.

[0097] Step 4:

[0098] The server uses a generative AI model to generate the optimal activity plan from the dataset. This AI model incorporates machine learning algorithms that process the input data to generate the most appropriate options for the user. The generated activity plan is based on the user's personal characteristics and environmental information. In this step, complex data analysis and model inference are used to generate a customized output for the user.

[0099] Step 5:

[0100] The server sends the generated activity plan to the user's device. The user's device displays the received plan in a chat interface and presents several options. The user selects an activity plan of interest from the presented options and sends their selection to the server via their device. This step provides an interface for the user to select an activity of interest and records that selection.

[0101] Step 6:

[0102] When a user executes their selected activity plan, the server provides real-time location and route information to the device. The device displays this information to help the user smoothly carry out the activity. Based on the entered location information, appropriate directions are calculated and presented to the user. In this step, real-time navigation and feedback enable the user to effectively complete the plan.

[0103] Step 7:

[0104] After the activity ends, users input their experience evaluation information into the system from their terminal and submit feedback. The server saves this evaluation information to a database and updates the AI ​​model for use in generating future activity plans. This step incorporates user feedback into the next plan generation, aiming for continuous system improvement.

[0105] (Application Example 1)

[0106] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0107] The challenge is to enable users to easily create activity plans necessary for a fulfilling holiday based on their individual hobbies and preferences. Furthermore, it is necessary to utilize real-time information during the planning process to enhance user satisfaction. Additionally, the accuracy of the information provided needs to be improved to offer users more meaningful options.

[0108] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0109] In this invention, the server includes means for storing personal attribute information received from the user in information storage, means for acquiring weather information and congestion information from external information sources, means for generating an optimal activity plan for the user using a machine learning algorithm, means for transmitting the generated activity plan to the user's communication device, means for processing activity plan selection information received from the user, means for analyzing user feedback information and storing it in information storage to reflect it in the generation of the next activity plan, means for providing the user with local information and route guidance in real time, and means for generating prompt sentences using a generative artificial intelligence model to suggest comfortable and enjoyable activities based on the user's preferences. This enables the provision of personalized activity plans to users and real-time support based on those plans.

[0110] "User" refers to the individual consumers or participants who use this system.

[0111] "Personal attribute information" refers to characteristic information related to the user, such as age, hobbies, and preferences.

[0112] "Information storage" refers to technologies or systems for securely and efficiently storing and managing data.

[0113] "External information sources" refer to data providers that exist outside the system and provide information such as weather information and congestion data.

[0114] A "machine learning algorithm" refers to a computational method that learns patterns from data and performs predictions and classifications.

[0115] "Communication equipment" refers to terminals and devices used for sending and receiving data.

[0116] "Feedback information" refers to data that shows the opinions and satisfaction levels that users provide after experiencing a product or service.

[0117] "Real-time information" refers to information that provides immediate updates on the ongoing situation.

[0118] "Local information" refers to data related to geographical locations and facilities.

[0119] "Route guidance" refers to an information service that shows the route to a destination.

[0120] A "generative artificial intelligence model" is an algorithm that generates new content or output through natural language processing and other methods.

[0121] A "prompt message" refers to the words or commands used to convey instructions to artificial intelligence.

[0122] To implement this invention, a user-owned terminal and a server for processing information are required. The user first inputs their personal attribute information through the terminal, and this data is securely stored in the server's information storage. The hardware is envisioned to be the user's smartphone or tablet, and the software used is a cloud-based database service.

[0123] The server acquires weather and congestion information from external sources in real time and uses machine learning algorithms to generate an optimal activity plan for the user. Generative AI technology is required here, and Google Cloud's machine learning platform is utilized.

[0124] The generated activity plan is transmitted to the user via a communication device. The user selects an activity based on the provided information, and this selection is fed back to the server. Based on this feedback, the server uses a generation artificial intelligence model to generate prompts to improve the accuracy of the activity plan and provides a new plan that reflects these prompts.

[0125] For example, if a user expresses a desire to participate in an art-related event, the server gathers weather and local cultural event information and proposes a plan that includes art appreciation and lunch at a cafe. Based on this, the user executes the plan and, as a result, can have a highly satisfying experience. An example of a prompt message would be, "My current interest is art. Please suggest some pleasant and enjoyable local art-related events."

[0126] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0127] Step 1:

[0128] The user enters personal attribute information using the device. This input includes data such as the user's age, hobbies, and preferences. The device collects this data and prepares to send it to information storage.

[0129] Step 2:

[0130] The server receives personal attribute information transmitted from the terminal and stores it in its information storage. This data serves as fundamental information necessary for generating future activity plans.

[0131] Step 3:

[0132] The server accesses external information sources to obtain weather and congestion information. This allows the latest weather data and congestion status to be collected in the database. This information is then used to plan subsequent activities.

[0133] Step 4:

[0134] The server uses stored user personal attribute information and data from external sources to execute machine learning algorithms and generate an optimal activity plan for the user. Using a generative AI model, it efficiently creates multiple prompt options, leading to the best plan for the user.

[0135] Step 5:

[0136] The generated activity plan is sent from the server to the user's terminal. The user reviews this plan and chooses the option that best suits their preferences from several choices.

[0137] Step 6:

[0138] The user sends their selected activity plan information to the server via their device. The server receives this information and prepares to provide relevant location information and route guidance in real time.

[0139] Step 7:

[0140] After the user completes an activity, they send feedback information to the server via their device. The server stores this feedback in its information storage and analyzes the data to incorporate it into future planning.

[0141] Step 8:

[0142] The server uses the acquired feedback information to update its machine learning algorithm, aiming to improve the accuracy of its activity plans. This will enable more personalized responses in future activity plan suggestions.

[0143] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0144] This invention is an activity planning system that incorporates a function to recognize the user's emotions. This system generates and provides activity plans based on the user's emotional state, enabling them to have a more satisfying holiday. Its specific operation is described below.

[0145] Users first register their personal attribute information with the system using their device. This includes basic profile information as well as data on the user's activity history and emotions. This information is stored in a database by the server and used to generate activity plans.

[0146] The emotion engine recognizes the user's emotional state and understands their current feelings (joy, sadness, stress, etc.). This allows for the generation of personalized activity plans tailored to the user's preferences and current emotions. This emotional data, like other data, is considered by the server during plan generation.

[0147] On a holiday morning, users send activity plan requests via their devices. The server generates the optimal activity plan based on the user's personal attributes, activity history, and emotional data, while also considering weather information and congestion levels from external data sources. By utilizing data from the emotional engine, the system can suggest activities best suited to the user's emotional state.

[0148] The generated activity plan is sent from the server to the user's terminal, and the user selects a plan to execute from the available options. Once the plan selection is complete, the server prepares to provide real-time information while the user is out and about, supporting their activities.

[0149] After the activity is completed, users use their devices to provide feedback on their evaluation of the plan and any newly arising emotions. The server stores this feedback information in a database and analyzes it, including the emotional data. This information is used to improve the accuracy of future activity plan generation.

[0150] As a concrete example, consider the case of User B using the system. User B has recently been feeling stressed and wants to relax. The emotion engine, recognizing User B's emotional state, suggests activities suitable for stress reduction, such as reading in a quiet park or listening to natural sounds. User B chooses one of the suggested plans and, by engaging in the activities based on the presented plan and navigation, can enjoy a refreshing holiday.

[0151] The following describes the processing flow.

[0152] Step 1:

[0153] Users use their devices to input profile information and emotion-related data (such as their current mood and emotion history) and register it with the system.

[0154] Step 2:

[0155] The terminal sends the entered information to the server.

[0156] Step 3:

[0157] The server verifies the received information and stores personal attribute information, sentiment data, and activity history in a database.

[0158] Step 4:

[0159] The emotion engine recognizes the user's current emotional state and provides the result to the server.

[0160] Step 5:

[0161] On a holiday morning, users request an activity plan via the chat interface on their device.

[0162] Step 6:

[0163] The server considers user profiles and sentiment data, and retrieves weather and congestion information from external data sources.

[0164] Step 7:

[0165] The server uses machine learning models to generate an optimal activity plan that takes into account the user's preferences, current emotional state, weather, and congestion levels.

[0166] Step 8:

[0167] The generated activity plan is sent from the server to the device, tailored to the user's emotional state.

[0168] Step 9:

[0169] The user reviews the activity plan presented on their device and selects a plan to execute. The selection is then sent to the server.

[0170] Step 10:

[0171] Based on the user's selections, the server prepares to provide real-time information (location information, guidance information, etc.) while the user is out and about.

[0172] Step 11:

[0173] While the user is out and about, the server provides real-time location information and navigation to the user's device.

[0174] Step 12:

[0175] After the activity ends, the user uses their device to send an evaluation of the activity, feedback, and changes in their emotional state during the activity to the server.

[0176] Step 13:

[0177] The server stores the received feedback and sentiment data in a database and uses it to generate future plans and update machine learning models.

[0178] (Example 2)

[0179] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0180] Conventional activity planning systems failed to adequately address users' emotional states and individual preferences, making it difficult to provide users with satisfying holiday plans. Furthermore, delays or inaccuracies in providing information based on external circumstances led to problems with the feasibility and effectiveness of the plans.

[0181] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0182] In this invention, the server includes means for storing personal attribute information and emotional data received from the user in a data storage means, means for acquiring weather data and congestion data from an external information source, and means for generating an optimal activity plan for the user by utilizing a model for analyzing the user's emotional state. This makes it possible to provide a more personalized activity plan in real time, taking into account the user's individual emotional state and external environment.

[0183] "Personal attribute information" refers to a collection of data that includes a user's basic profile information and past activity history.

[0184] "Emotional data" refers to information that indicates a user's current emotional state and is used to personalize activity plans.

[0185] "Data storage means" refers to elements such as databases and storage devices that store information and allow for quick access when needed.

[0186] "External information sources" refer to organizations or networks that provide data such as weather information and congestion information obtained from outside the system.

[0187] "Weather data" refers to information that indicates the environmental conditions of a location, such as weather, temperature, and humidity.

[0188] "Congestion data" refers to information about the congestion levels in a specific area or facility.

[0189] A "model" refers to a machine learning model or algorithm used to analyze a user's emotional state and generate an appropriate action plan.

[0190] "Terminal device" refers to an electronic device used by a user to send and receive information or to check activity plans.

[0191] This invention is a system that generates an activity plan based on the user's emotional state and provides the user with the optimal choice. This system mainly consists of a server and terminals and operates as follows.

[0192] Users use a device to input their personal attribute information and sentiment data and send it to the server. This device can be a smartphone, tablet, or personal computer—any electronic device capable of inputting, verifying, and sending / receiving such information. Upon receiving this information, the server records it in a database and uses it to generate subsequent action plans.

[0193] The server also has communication technologies to obtain weather and congestion data from external sources. This allows local weather and congestion conditions to be taken into consideration when planning.

[0194] User emotion data is analyzed by an emotion analysis engine located on the server to understand the user's current emotional state. This analysis utilizes specific models based on machine learning techniques. For example, an emotion recognition model analyzes the user's text and voice input to identify emotions such as joy, sadness, and stress.

[0195] The server uses a generated AI model based on the user's personal attribute information, emotional state, and external environment data to create an optimal activity plan for the user. This activity plan is sent to the terminal and presented to the user. The user can then select the most suitable plan from the presented options and act accordingly.

[0196] As a concrete example, consider a case where a user experiencing stress seeks relaxation through the system. The system can suggest activities such as reading or listening to music in a quiet park, and this suggestion is optimized based on the user's emotional analysis results and environmental data.

[0197] Example of a prompt:

[0198] "The user's current emotion is stress. Please propose an activity plan to reduce this stress."

[0199] This configuration allows users to receive activity plans that perfectly match their emotional state, resulting in a more fulfilling experience.

[0200] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0201] Step 1:

[0202] The user uses a terminal to input their personal attribute information and emotional data. Specifically, they enter information such as their name, age, past activities, and current emotions into a form. This information is processed by the terminal and sent to the server. The input data is in text format and stored in the server's data storage system. The output of this step is the personal attribute information and emotional data stored in the database.

[0203] Step 2:

[0204] The server retrieves weather and congestion data from external sources. In this process, the server uses external APIs to obtain local weather information and congestion data for specific locations, and records it in a database. As a result, the input is up-to-date information from external sources, and the output is this information neatly stored and passed on to the next step.

[0205] Step 3:

[0206] The server runs an emotion analysis engine to analyze the user's emotional data. This step uses machine learning algorithms to identify the user's emotional state (e.g., joy, sadness, stress) from the input emotional data. The output of the process is an analysis report containing the recognized emotional state, which is used to generate an action plan in the next step.

[0207] Step 4:

[0208] The server uses a generative AI model to generate an optimal activity plan for the user. The inputs here are personal attribute information, analyzed sentiment data, and external environment data. Based on this information, the AI ​​model lists potential activities and generates an optimal plan that meets the user's needs. The output is a specific activity plan, which is presented to the user in the next step.

[0209] Step 5:

[0210] The generated activity plan is sent to the terminal. The terminal displays the plan provided to the user, and the user selects the activities they wish to perform. Once the user has completed their selections on the interface, that information is sent from the terminal to the server as feedback. In this step, the selected activity plan and the user's feedback are output.

[0211] Step 6:

[0212] The server stores user feedback information in a database and performs analysis to help generate future activity plans. Based on the feedback input data, it analyzes the correlation with further sentiment data to improve the overall system accuracy. This output is the analyzed feedback data stored in the database.

[0213] (Application Example 2)

[0214] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0215] In today's busy lifestyle, users face the challenge of planning activities that align with their current emotional state. Furthermore, there is a lack of systems that provide personalized activity suggestions based on the user's emotional state. As a result, it is difficult for users to enjoy fulfilling leisure time.

[0216] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0217] In this invention, the server includes means for storing personal attribute information received from the user in a storage device, means for collecting environmental information and crowd information from external information sources, and means for utilizing an artificial intelligence module that analyzes the user's emotional state using an emotion recognition function. This makes it possible to provide activity suggestions based on the user's emotional state.

[0218] "Personal attribute information" refers to information that indicates individual characteristics of a user, such as age, gender, preferences, and activity history.

[0219] A "storage device" is a device that stores data over a long period of time and allows it to be retrieved and used as needed.

[0220] "External information sources" are data sources that exist outside the system and provide information related to activity planning, such as environmental information and crowd information.

[0221] "Environmental information" refers to physical information that affects the user's environment, such as weather conditions, temperature, and humidity.

[0222] "Crowd information" refers to information about collective behavior, such as the degree of crowding and traffic conditions in a specific area.

[0223] A "machine learning algorithm" is a computational processing method that learns patterns and rules from data and uses them to predict future situations and make decisions.

[0224] A "communication terminal" is a device used to send and receive information, and includes smartphones and tablets.

[0225] "Emotion recognition functionality" is a technology that analyzes input data such as the user's voice and facial expressions to identify their emotional state.

[0226] An "artificial intelligence module" is an artificial intelligence-related software component designed to handle specific tasks or problems.

[0227] An "activity suggestion" is a specific action or plan recommended based on the user's preferences and emotional state.

[0228] The system implementing this invention provides personalized activity suggestions based on the user's emotional state. The server implements this process using the following hardware and software.

[0229] The server first stores personal attribute information received from the user in a storage device. This process securely and efficiently collects and stores data such as the user's age, preferences, and activity history.

[0230] Next, the server collects environmental information (such as weather and temperature) and crowd information (such as congestion levels and traffic conditions) from external sources. This helps to make the user's activity plan realistic and feasible.

[0231] The server uses an artificial intelligence module with emotion recognition capabilities to analyze the user's emotional state from their voice and facial expression data. This utilizes general-purpose voice analysis software, facial recognition libraries, and emotion recognition APIs.

[0232] Based on the analysis results, the server utilizes machine learning algorithms to create an activity plan tailored to the user. The generated activity plan is immediately sent to the user's communication terminal.

[0233] For example, if a user wants to alleviate daily stress, the server uses emotion recognition to determine the user's emotional state as "stressed" and accordingly suggests activities with a relaxing effect, such as a quiet walk.

[0234] Example prompt for a generative AI model: "If the user's current emotional state is stress, what relaxation activities should be suggested? Please include a sentence indicating that the user's emotions are being taken into consideration."

[0235] This format allows for activity suggestions tailored to the individual needs of users, enabling them to have highly satisfying leisure experiences.

[0236] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0237] Step 1:

[0238] The server receives personal attribute information (age, preferences, activity history, etc.) from the user as input and stores it in a storage device. This storage process securely stores the data using a database management system. The output is a collection of personal attributes that can be used to generate future activity plans.

[0239] Step 2:

[0240] The server takes environmental information (weather and temperature) and crowd information (congestion level and traffic conditions) as input from external sources. This allows the server to collect contextual data around the user's current location and evaluate the feasibility of activity plans. The output is up-to-date and detailed environmental and crowd information.

[0241] Step 3:

[0242] The server uses an artificial intelligence module with emotion recognition capabilities to receive user voice and facial expression data as input and analyze their emotional state. Voice analysis software and a facial recognition library are utilized here. The output is data indicating the user's emotional state (e.g., joy, stress).

[0243] Step 4:

[0244] The server synthesizes personal attribute information, environmental / crowd information, and emotional state, and uses machine learning algorithms to generate an optimal activity plan. Pattern recognition and predictive models are applied to generate the activity plan, creating individual suggestions for each user. The output is a list of specific activity suggestions.

[0245] Step 5:

[0246] The generated activity proposals are sent by the server to the user's communication terminal. The terminal receives the proposals and displays them to the user. The displayed information includes details of the proposed activity and its background information. The output is an activity proposal presented visually to the user.

[0247] Step 6:

[0248] The user selects an activity from the received activity suggestions and sends this selection information to the server. The server records the selected activity as input and refers to it in subsequent feedback processing. The output is the specific activity information selected by the user.

[0249] Step 7:

[0250] After completing an activity, users provide feedback by entering information on a terminal. The server receives this feedback and records it in a storage device. This information includes satisfaction levels and areas for improvement. The output is feedback data used to improve the accuracy of future activity plans.

[0251] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0252] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0253] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0254] [Second Embodiment]

[0255] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0256] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0257] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0258] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0259] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0260] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0261] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0262] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0263] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0264] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0265] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0266] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0267] This invention is a system that helps users have more meaningful holidays. This system presents an activity plan based on the user's personal attribute information and provides real-time support for that plan. Its specific operation is described below.

[0268] Users first register their personal information with the system using their device. This includes personal attribute information such as name, age, and hobbies. Once registration is complete, this information is stored in a database by the server and used to generate future activity plans.

[0269] On a holiday morning, users request an activity plan using the chat interface on their device. The server accepts this request, investigates the user's profile and past activity history, and then retrieves current weather information and congestion status from external data sources. Subsequently, considering this data and the user's preferences, a machine learning model is used to generate the optimal activity plan.

[0270] The generated plan is sent from the server to the user's device. The user selects an action plan from several presented options, and this selection is notified to the server. Once the user starts going out, the server provides the user with real-time location information and directions. This feature allows users to make appropriate decisions based on the situation and carry out their activities more systematically.

[0271] After the activity ends, users send feedback to the system regarding their satisfaction with the plan and their experience. The server stores this feedback in a database and uses it to continuously improve the machine learning model. This enables more accurate personalization in future plan generation, further improving user satisfaction.

[0272] As a concrete example, the personal information registered by user A includes preferences such as "I like nature" and "I enjoy the outdoors." Based on this information, the server suggests an activity plan that includes hiking in a nearby park and lunch at a popular local cafe on a day with good weather. User A selects this plan and can enjoy the day while receiving navigation and updates on congestion levels from their device. After the activity, user A provides feedback on their satisfaction level, and the server uses this information to improve the accuracy of future plans.

[0273] The following describes the processing flow.

[0274] Step 1:

[0275] Users use a device to enter personal information (name, age, hobbies, etc.) and register it in the system.

[0276] Step 2:

[0277] The terminal sends the input personal attribute information to the server.

[0278] Step 3:

[0279] The server verifies the received personal attribute information and saves it in the database.

[0280] Step 4:

[0281] On a holiday morning, the user uses the chat interface of the terminal to send a request for an activity plan to the system.

[0282] Step 5:

[0283] The server obtains the user's profile information and past behavior history, and prepares the basic data for plan generation.

[0284] Step 6:

[0285] The server obtains weather information and congestion information from external data sources, and collects the real-time information required for plan generation.

[0286] Step 7:

[0287] The server uses a machine learning model to generate an optimal activity plan for the user. The user's preferences, weather, and congestion situation are considered in this process.

[0288] Step 8:

[0289] The server sends the generated activity plan to the user's terminal and prompts for selection.

[0290] Step 9:

[0291] The user reviews the activity plan presented through their device and selects a plan to execute. The selection is then sent to the server.

[0292] Step 10:

[0293] The server receives the user's selection and prepares to provide real-time information while the user is away from their computer.

[0294] Step 11:

[0295] While the user is out, the server continues to provide location and navigation information to the user's device in real time.

[0296] Step 12:

[0297] After the activity ends, the user sends feedback about the plan to the server from their device.

[0298] Step 13:

[0299] The server stores the received feedback in a database and uses it to update the machine learning model for future plan generation.

[0300] (Example 1)

[0301] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0302] It is difficult to efficiently propose activity plans that match the individual preferences and circumstances of each user regarding how to spend their holidays. Furthermore, it is necessary to utilize environmental information and user feedback and effectively reflect this in future plans. In addition, a system is needed that provides real-time support and helps users make appropriate decisions based on their circumstances.

[0303] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0304] In this invention, the server includes means for storing personal characteristic information received from a user in a storage device, means for acquiring environmental information and situation information from an external information source, and means for generating an optimal activity plan for the user using a machine learning model. Thereby, it is possible to present a personalized activity plan according to the characteristics and situation of the user, and improve the user's satisfaction through real-time guidance and support.

[0305] The "user" refers to a person who uses an information system and provides personal characteristic information to receive an activity plan.

[0306] The "personal characteristic information" refers to attribute information such as the user's name, age, and hobbies, and refers to data stored in a database and used for generating an activity plan.

[0307] The "storage device" refers to a hardware or software mechanism for storing data and programs.

[0308] The "external information source" refers to other systems or platforms that provide various data from the Internet or APIs.

[0309] The "environmental information" refers to data related to external conditions that affect the user's activities, such as weather information and congestion information.

[0310] The "situation information" refers to data related to the user's situation, such as real-time position information and route information at that time.

[0311] The "machine learning model" refers to an algorithm that learns patterns based on a large amount of data and generates an optimal activity plan for the user.

[0312] The "communication device" refers to a device for transmitting and receiving data between the server and the user, such as a smartphone or a computer.

[0313] "Evaluation information" refers to feedback received from users, including data on satisfaction with activity plans and the content of their experiences.

[0314] This invention is a system that proposes and supports activity plans tailored to the user's preferences and circumstances in real time. Specific embodiments of this system are shown below.

[0315] Users access the system using their own devices and enter personal information, including their name, age, and hobbies. This information is received by the server and stored in a database. This stored personal information is later used to generate activity plans.

[0316] On a holiday morning, users request an activity plan by entering a prompt message into the system's chat interface via their terminal. For example, they might send a prompt message like, "What activities are recommended for today?" In response to this request, the server retrieves the user's profile information and past history data, and gathers environmental and situational information from external sources.

[0317] Based on this data, the server generates an optimal activity plan using a generative AI model. This AI model incorporates machine learning algorithms and is designed to provide personalized suggestions to the user. The generated activity plan is sent from the server to the user's communication device, where multiple options are presented.

[0318] The user selects an activity plan that interests them from several options presented, and this selection is notified to the server. Subsequently, as the user proceeds with the activity, the server provides the user with real-time location and route information, supporting the activity through the device.

[0319] After an activity is completed, users send feedback to the system evaluating their experience. The server stores this evaluation information in a database and updates the AI ​​model for generating activity plans, which can then be used to create more personalized and accurate activity plans for future activities.

[0320] For example, if user A has registered information such as "I like nature" and "I'm an outdoorsy person," the server will suggest an activity plan based on that information, including hiking and lunch at a local cafe on a day with good weather. User A can select this plan and enjoy the day using navigation and real-time information from their device.

[0321] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0322] Step 1:

[0323] The user uses a terminal to input and transmit personal information into the system. This information includes name, age, preferences, etc. The terminal sends this input to the server. The server receives the entered personal information and stores it in a database as storage. In this step, the user provides data to personalize the system, and the system processes and stores that data.

[0324] Step 2:

[0325] On a holiday morning, a user uses the chat interface from their device to send a prompt message to the system. For example, they might enter a request such as, "What activities do you recommend today?" The device sends this prompt message to the server. The server receives the prompt message and retrieves personal information from its database based on the user ID. This step involves processing to prepare appropriate information in response to the user's request.

[0326] Step 3:

[0327] The server calls external APIs to obtain environmental information (weather information, congestion information, etc.) from external sources. This input includes location information and date / time. Based on this data, the server retrieves data that affects the user's current situation. Then, it integrates the retrieved data and creates a dataset that combines it with the user's personal characteristics information. In this step, real-time external information is ingested and a customized dataset is created for each user.

[0328] Step 4:

[0329] The server uses a generative AI model to generate the optimal activity plan from the dataset. This AI model incorporates machine learning algorithms that process the input data to generate the most appropriate options for the user. The generated activity plan is based on the user's personal characteristics and environmental information. In this step, complex data analysis and model inference are used to generate a customized output for the user.

[0330] Step 5:

[0331] The server sends the generated activity plan to the user's device. The user's device displays the received plan in a chat interface and presents several options. The user selects an activity plan of interest from the presented options and sends their selection to the server via their device. This step provides an interface for the user to select an activity of interest and records that selection.

[0332] Step 6:

[0333] When a user executes their selected activity plan, the server provides real-time location and route information to the device. The device displays this information to help the user smoothly carry out the activity. Based on the entered location information, appropriate directions are calculated and presented to the user. In this step, real-time navigation and feedback enable the user to effectively complete the plan.

[0334] Step 7:

[0335] After the activity ends, users input their experience evaluation information into the system from their terminal and submit feedback. The server saves this evaluation information to a database and updates the AI ​​model for use in generating future activity plans. This step incorporates user feedback into the next plan generation, aiming for continuous system improvement.

[0336] (Application Example 1)

[0337] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0338] The challenge is to enable users to easily create activity plans necessary for a fulfilling holiday based on their individual hobbies and preferences. Furthermore, it is necessary to utilize real-time information during the planning process to enhance user satisfaction. Additionally, the accuracy of the information provided needs to be improved to offer users more meaningful options.

[0339] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0340] In this invention, the server includes means for storing personal attribute information received from the user in information storage, means for acquiring weather information and congestion information from external information sources, means for generating an optimal activity plan for the user using a machine learning algorithm, means for transmitting the generated activity plan to the user's communication device, means for processing activity plan selection information received from the user, means for analyzing user feedback information and storing it in information storage to reflect it in the generation of the next activity plan, means for providing the user with local information and route guidance in real time, and means for generating prompt sentences using a generative artificial intelligence model to suggest comfortable and enjoyable activities based on the user's preferences. This enables the provision of personalized activity plans to users and real-time support based on those plans.

[0341] "User" refers to the individual consumers or participants who use this system.

[0342] "Personal attribute information" refers to characteristic information related to the user, such as age, hobbies, and preferences.

[0343] "Information storage" refers to technologies or systems for securely and efficiently storing and managing data.

[0344] "External information sources" refer to data providers that exist outside the system and provide information such as weather information and congestion data.

[0345] A "machine learning algorithm" refers to a computational method that learns patterns from data and performs predictions and classifications.

[0346] "Communication equipment" refers to terminals and devices used for sending and receiving data.

[0347] "Feedback information" refers to data that shows the opinions and satisfaction levels that users provide after experiencing a product or service.

[0348] "Real-time information" refers to information that provides immediate updates on the ongoing situation.

[0349] "Local information" refers to data related to geographical locations and facilities.

[0350] "Route guidance" refers to an information service that shows the route to a destination.

[0351] A "generative artificial intelligence model" is an algorithm that generates new content or output through natural language processing and other methods.

[0352] A "prompt message" refers to the words or commands used to convey instructions to artificial intelligence.

[0353] To implement this invention, a user-owned terminal and a server for processing information are required. The user first inputs their personal attribute information through the terminal, and this data is securely stored in the server's information storage. The hardware is envisioned to be the user's smartphone or tablet, and the software used is a cloud-based database service.

[0354] The server acquires weather and congestion information from external sources in real time and uses machine learning algorithms to generate an optimal activity plan for the user. Generative AI technology is required here, and Google Cloud's machine learning platform is utilized.

[0355] The generated activity plan is transmitted to the user via a communication device. The user selects an activity based on the provided information, and this selection is fed back to the server. Based on this feedback, the server uses a generation artificial intelligence model to generate prompts to improve the accuracy of the activity plan and provides a new plan that reflects these prompts.

[0356] For example, if a user expresses a desire to participate in an art-related event, the server gathers weather and local cultural event information and proposes a plan that includes art appreciation and lunch at a cafe. Based on this, the user executes the plan and, as a result, can have a highly satisfying experience. An example of a prompt message would be, "My current interest is art. Please suggest some pleasant and enjoyable local art-related events."

[0357] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0358] Step 1:

[0359] The user enters personal attribute information using the device. This input includes data such as the user's age, hobbies, and preferences. The device collects this data and prepares to send it to information storage.

[0360] Step 2:

[0361] The server receives personal attribute information transmitted from the terminal and stores it in its information storage. This data serves as fundamental information necessary for generating future activity plans.

[0362] Step 3:

[0363] The server accesses external information sources to obtain weather and congestion information. This allows the latest weather data and congestion status to be collected in the database. This information is then used to plan subsequent activities.

[0364] Step 4:

[0365] The server uses stored user personal attribute information and data from external sources to execute machine learning algorithms and generate an optimal activity plan for the user. Using a generative AI model, it efficiently creates multiple prompt options, leading to the best plan for the user.

[0366] Step 5:

[0367] The generated activity plan is sent from the server to the user's terminal. The user reviews this plan and chooses the option that best suits their preferences from several choices.

[0368] Step 6:

[0369] The user sends their selected activity plan information to the server via their device. The server receives this information and prepares to provide relevant location information and route guidance in real time.

[0370] Step 7:

[0371] After the user completes an activity, they send feedback information to the server via their device. The server stores this feedback in its information storage and analyzes the data to incorporate it into future planning.

[0372] Step 8:

[0373] The server uses the acquired feedback information to update its machine learning algorithm, aiming to improve the accuracy of its activity plans. This will enable more personalized responses in future activity plan suggestions.

[0374] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0375] This invention is an activity planning system that incorporates a function to recognize the user's emotions. This system generates and provides activity plans based on the user's emotional state, enabling them to have a more satisfying holiday. Its specific operation is described below.

[0376] Users first register their personal attribute information with the system using their device. This includes basic profile information as well as data on the user's activity history and emotions. This information is stored in a database by the server and used to generate activity plans.

[0377] The emotion engine recognizes the user's emotional state and understands their current feelings (joy, sadness, stress, etc.). This allows for the generation of personalized activity plans tailored to the user's preferences and current emotions. This emotional data, like other data, is considered by the server during plan generation.

[0378] On a holiday morning, users send activity plan requests via their devices. The server generates the optimal activity plan based on the user's personal attributes, activity history, and emotional data, while also considering weather information and congestion levels from external data sources. By utilizing data from the emotional engine, the system can suggest activities best suited to the user's emotional state.

[0379] The generated activity plan is sent from the server to the user's terminal, and the user selects a plan to execute from the available options. Once the plan selection is complete, the server prepares to provide real-time information while the user is out and about, supporting their activities.

[0380] After the activity is completed, users use their devices to provide feedback on their evaluation of the plan and any newly arising emotions. The server stores this feedback information in a database and analyzes it, including the emotional data. This information is used to improve the accuracy of future activity plan generation.

[0381] As a concrete example, consider the case of User B using the system. User B has recently been feeling stressed and wants to relax. The emotion engine, recognizing User B's emotional state, suggests activities suitable for stress reduction, such as reading in a quiet park or listening to natural sounds. User B chooses one of the suggested plans and, by engaging in the activities based on the presented plan and navigation, can enjoy a refreshing holiday.

[0382] The following describes the processing flow.

[0383] Step 1:

[0384] Users use their devices to input profile information and emotion-related data (such as their current mood and emotion history) and register it with the system.

[0385] Step 2:

[0386] The terminal sends the entered information to the server.

[0387] Step 3:

[0388] The server verifies the received information and stores personal attribute information, sentiment data, and activity history in a database.

[0389] Step 4:

[0390] The emotion engine recognizes the user's current emotional state and provides the result to the server.

[0391] Step 5:

[0392] On a holiday morning, users request an activity plan via the chat interface on their device.

[0393] Step 6:

[0394] The server considers user profiles and sentiment data, and retrieves weather and congestion information from external data sources.

[0395] Step 7:

[0396] The server uses machine learning models to generate an optimal activity plan that takes into account the user's preferences, current emotional state, weather, and congestion levels.

[0397] Step 8:

[0398] The generated activity plan is sent from the server to the device, tailored to the user's emotional state.

[0399] Step 9:

[0400] The user reviews the activity plan presented on their device and selects a plan to execute. The selection is then sent to the server.

[0401] Step 10:

[0402] Based on the user's selections, the server prepares to provide real-time information (location information, guidance information, etc.) while the user is out and about.

[0403] Step 11:

[0404] While the user is out and about, the server provides real-time location information and navigation to the user's device.

[0405] Step 12:

[0406] After the activity ends, the user uses their device to send an evaluation of the activity, feedback, and changes in their emotional state during the activity to the server.

[0407] Step 13:

[0408] The server stores the received feedback and sentiment data in a database and uses it to generate future plans and update machine learning models.

[0409] (Example 2)

[0410] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0411] Conventional activity planning systems failed to adequately address users' emotional states and individual preferences, making it difficult to provide users with satisfying holiday plans. Furthermore, delays or inaccuracies in providing information based on external circumstances led to problems with the feasibility and effectiveness of the plans.

[0412] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0413] In this invention, the server includes means for storing personal attribute information and emotional data received from the user in a data storage means, means for acquiring weather data and congestion data from an external information source, and means for generating an optimal activity plan for the user by utilizing a model for analyzing the user's emotional state. This makes it possible to provide a more personalized activity plan in real time, taking into account the user's individual emotional state and external environment.

[0414] "Personal attribute information" refers to a collection of data that includes a user's basic profile information and past activity history.

[0415] "Emotional data" refers to information that indicates a user's current emotional state and is used to personalize activity plans.

[0416] "Data storage means" refers to elements such as databases and storage devices that store information and allow for quick access when needed.

[0417] "External information sources" refer to organizations or networks that provide data such as weather information and congestion information obtained from outside the system.

[0418] "Weather data" refers to information that indicates the environmental conditions of a location, such as weather, temperature, and humidity.

[0419] "Congestion data" refers to information about the congestion levels in a specific area or facility.

[0420] A "model" refers to a machine learning model or algorithm used to analyze a user's emotional state and generate an appropriate action plan.

[0421] "Terminal device" refers to an electronic device used by a user to send and receive information or to check activity plans.

[0422] This invention is a system that generates an activity plan based on the user's emotional state and provides the user with the optimal choice. This system mainly consists of a server and terminals and operates as follows.

[0423] Users use a device to input their personal attribute information and sentiment data and send it to the server. This device can be a smartphone, tablet, or personal computer—any electronic device capable of inputting, verifying, and sending / receiving such information. Upon receiving this information, the server records it in a database and uses it to generate subsequent action plans.

[0424] The server also has communication technologies to obtain weather and congestion data from external sources. This allows local weather and congestion conditions to be taken into consideration when planning.

[0425] User emotion data is analyzed by an emotion analysis engine located on the server to understand the user's current emotional state. This analysis utilizes specific models based on machine learning techniques. For example, an emotion recognition model analyzes the user's text and voice input to identify emotions such as joy, sadness, and stress.

[0426] The server uses a generated AI model based on the user's personal attribute information, emotional state, and external environment data to create an optimal activity plan for the user. This activity plan is sent to the terminal and presented to the user. The user can then select the most suitable plan from the presented options and act accordingly.

[0427] As a concrete example, consider a case where a user experiencing stress seeks relaxation through the system. The system can suggest activities such as reading or listening to music in a quiet park, and this suggestion is optimized based on the user's emotional analysis results and environmental data.

[0428] Example of a prompt:

[0429] "The user's current emotion is stress. Please propose an activity plan to reduce this stress."

[0430] This configuration allows users to receive activity plans that perfectly match their emotional state, resulting in a more fulfilling experience.

[0431] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0432] Step 1:

[0433] The user uses a terminal to input their personal attribute information and emotional data. Specifically, they enter information such as their name, age, past activities, and current emotions into a form. This information is processed by the terminal and sent to the server. The input data is in text format and stored in the server's data storage system. The output of this step is the personal attribute information and emotional data stored in the database.

[0434] Step 2:

[0435] The server retrieves weather and congestion data from external sources. In this process, the server uses external APIs to obtain local weather information and congestion data for specific locations, and records it in a database. As a result, the input is up-to-date information from external sources, and the output is this information neatly stored and passed on to the next step.

[0436] Step 3:

[0437] The server runs an emotion analysis engine to analyze the user's emotional data. This step uses machine learning algorithms to identify the user's emotional state (e.g., joy, sadness, stress) from the input emotional data. The output of the process is an analysis report containing the recognized emotional state, which is used to generate an action plan in the next step.

[0438] Step 4:

[0439] The server uses a generative AI model to generate an optimal activity plan for the user. The inputs here are personal attribute information, analyzed sentiment data, and external environment data. Based on this information, the AI ​​model lists potential activities and generates an optimal plan that meets the user's needs. The output is a specific activity plan, which is presented to the user in the next step.

[0440] Step 5:

[0441] The generated activity plan is sent to the terminal. The terminal displays the plan provided to the user, and the user selects the activities they wish to perform. Once the user has completed their selections on the interface, that information is sent from the terminal to the server as feedback. In this step, the selected activity plan and the user's feedback are output.

[0442] Step 6:

[0443] The server stores user feedback information in a database and performs analysis to help generate future activity plans. Based on the feedback input data, it analyzes the correlation with further sentiment data to improve the overall system accuracy. This output is the analyzed feedback data stored in the database.

[0444] (Application Example 2)

[0445] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0446] In today's busy lifestyle, users face the challenge of planning activities that align with their current emotional state. Furthermore, there is a lack of systems that provide personalized activity suggestions based on the user's emotional state. As a result, it is difficult for users to enjoy fulfilling leisure time.

[0447] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0448] In this invention, the server includes means for storing personal attribute information received from the user in a storage device, means for collecting environmental information and crowd information from external information sources, and means for utilizing an artificial intelligence module that analyzes the user's emotional state using an emotion recognition function. This makes it possible to provide activity suggestions based on the user's emotional state.

[0449] "Personal attribute information" refers to information that indicates individual characteristics of a user, such as age, gender, preferences, and activity history.

[0450] A "storage device" is a device that stores data over a long period of time and allows it to be retrieved and used as needed.

[0451] An "external information source" is a data source that exists outside the system and provides information related to the activity plan, such as environmental information or crowd information.

[0452] "Environmental information" refers to physical information that affects the user's environment, such as weather conditions, temperature, and humidity.

[0453] "Crowd information" refers to information about collective behavior, such as the degree of crowding and traffic conditions in a specific area.

[0454] A "machine learning algorithm" is a computational processing method that learns patterns and rules from data and uses them to predict future situations and make decisions.

[0455] A "communication terminal" is a device used to send and receive information, and includes smartphones and tablets.

[0456] "Emotion recognition functionality" is a technology that analyzes input data such as the user's voice and facial expressions to identify their emotional state.

[0457] An "artificial intelligence module" is an artificial intelligence-related software component designed to handle specific tasks or problems.

[0458] An "activity suggestion" is a specific action or plan recommended based on the user's preferences and emotional state.

[0459] The system implementing this invention provides personalized activity suggestions based on the user's emotional state. The server implements this process using the following hardware and software.

[0460] The server first stores personal attribute information received from the user in a storage device. This process securely and efficiently collects and stores data such as the user's age, preferences, and activity history.

[0461] Next, the server collects environmental information (such as weather and temperature) and crowd information (such as congestion levels and traffic conditions) from external sources. This helps to make the user's activity plan realistic and feasible.

[0462] The server uses an artificial intelligence module with emotion recognition capabilities to analyze the user's emotional state from their voice and facial expression data. This utilizes general-purpose voice analysis software, facial recognition libraries, and emotion recognition APIs.

[0463] Based on the analysis results, the server utilizes machine learning algorithms to create an activity plan tailored to the user. The generated activity plan is immediately sent to the user's communication terminal.

[0464] For example, if a user wants to alleviate daily stress, the server uses emotion recognition to determine the user's emotional state as "stressed" and accordingly suggests activities with a relaxing effect, such as a quiet walk.

[0465] Example prompt for a generative AI model: "If the user's current emotional state is stress, what relaxation activities should be suggested? Please include a sentence indicating that the user's emotions are being taken into consideration."

[0466] This format allows for activity suggestions tailored to the individual needs of users, enabling them to have highly satisfying leisure experiences.

[0467] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0468] Step 1:

[0469] The server receives personal attribute information (age, preferences, activity history, etc.) from the user as input and stores it in a storage device. This storage process securely stores the data using a database management system. The output is a collection of personal attributes that can be used to generate future activity plans.

[0470] Step 2:

[0471] The server takes environmental information (weather and temperature) and crowd information (congestion level and traffic conditions) as input from external sources. This allows the server to collect contextual data around the user's current location and evaluate the feasibility of activity plans. The output is up-to-date and detailed environmental and crowd information.

[0472] Step 3:

[0473] The server uses an artificial intelligence module with emotion recognition capabilities to receive user voice and facial expression data as input and analyze their emotional state. Voice analysis software and a facial recognition library are utilized here. The output is data indicating the user's emotional state (e.g., joy, stress).

[0474] Step 4:

[0475] The server synthesizes personal attribute information, environmental / crowd information, and emotional state, and uses machine learning algorithms to generate an optimal activity plan. Pattern recognition and predictive models are applied to generate the activity plan, creating individual suggestions for each user. The output is a list of specific activity suggestions.

[0476] Step 5:

[0477] The generated activity proposals are sent by the server to the user's communication terminal. The terminal receives the proposals and displays them to the user. The displayed information includes details of the proposed activity and its background information. The output is an activity proposal presented visually to the user.

[0478] Step 6:

[0479] The user selects an activity from the received activity suggestions and sends this selection information to the server. The server records the selected activity as input and refers to it in subsequent feedback processing. The output is the specific activity information selected by the user.

[0480] Step 7:

[0481] After completing an activity, users provide feedback by entering information on a terminal. The server receives this feedback and records it in a storage device. This information includes satisfaction levels and areas for improvement. The output is feedback data used to improve the accuracy of future activity plans.

[0482] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0483] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0484] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0485] [Third Embodiment]

[0486] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0487] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0488] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0489] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0490] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0491] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0492] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0493] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0494] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0495] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0496] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0497] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0498] This invention is a system that helps users have more meaningful holidays. This system presents an activity plan based on the user's personal attribute information and provides real-time support for that plan. Its specific operation is described below.

[0499] Users first register their personal information with the system using their device. This includes personal attribute information such as name, age, and hobbies. Once registration is complete, this information is stored in a database by the server and used to generate future activity plans.

[0500] On a holiday morning, users request an activity plan using the chat interface on their device. The server accepts this request, investigates the user's profile and past activity history, and then retrieves current weather information and congestion status from external data sources. Subsequently, considering this data and the user's preferences, a machine learning model is used to generate the optimal activity plan.

[0501] The generated plan is sent from the server to the user's device. The user selects an action plan from several presented options, and this selection is notified to the server. Once the user starts going out, the server provides the user with real-time location information and directions. This feature allows users to make appropriate decisions based on the situation and carry out their activities more systematically.

[0502] After the activity ends, users send feedback to the system regarding their satisfaction with the plan and their experience. The server stores this feedback in a database and uses it to continuously improve the machine learning model. This enables more accurate personalization in future plan generation, further improving user satisfaction.

[0503] As a concrete example, the personal information registered by user A includes preferences such as "I like nature" and "I enjoy the outdoors." Based on this information, the server suggests an activity plan that includes hiking in a nearby park and lunch at a popular local cafe on a day with good weather. User A selects this plan and can enjoy the day while receiving navigation and updates on congestion levels from their device. After the activity, user A provides feedback on their satisfaction level, and the server uses this information to improve the accuracy of future plans.

[0504] The following describes the processing flow.

[0505] Step 1:

[0506] Users use a device to enter personal information (name, age, hobbies, etc.) and register it in the system.

[0507] Step 2:

[0508] The terminal sends the entered personal attribute information to the server.

[0509] Step 3:

[0510] The server verifies the received personal attribute information and stores it in the database.

[0511] Step 4:

[0512] On holiday mornings, users use their device's chat interface to send activity plan requests to the system.

[0513] Step 5:

[0514] The server retrieves the user's profile information and past behavioral history, preparing the basic data for plan generation.

[0515] Step 6:

[0516] The server obtains weather and congestion information from external data sources and collects real-time information necessary for plan generation.

[0517] Step 7:

[0518] The server uses machine learning models to generate the optimal activity plan for the user. This process takes into account user preferences, weather, and congestion levels.

[0519] Step 8:

[0520] The server sends the generated activity plan to the user's device and prompts them to make a selection.

[0521] Step 9:

[0522] The user reviews the activity plan presented through their device and selects a plan to execute. The selection is then sent to the server.

[0523] Step 10:

[0524] The server receives the user's selection and prepares to provide real-time information while the user is away from their computer.

[0525] Step 11:

[0526] While the user is out, the server continues to provide location and navigation information to the user's device in real time.

[0527] Step 12:

[0528] After the activity ends, the user sends feedback about the plan to the server from their device.

[0529] Step 13:

[0530] The server stores the received feedback in a database and uses it to update the machine learning model for future plan generation.

[0531] (Example 1)

[0532] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0533] It is difficult to efficiently propose activity plans that match the individual preferences and circumstances of each user regarding how to spend their holidays. Furthermore, it is necessary to utilize environmental information and user feedback and effectively reflect this in future plans. In addition, a system is needed that provides real-time support and helps users make appropriate decisions based on their circumstances.

[0534] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0535] In this invention, the server includes means for storing personal characteristic information received from the user in a storage device, means for acquiring environmental information and situational information from external information sources, and means for generating an optimal activity plan for the user using a machine learning model. This makes it possible to present a personalized activity plan according to the user's characteristics and circumstances, and to improve user satisfaction through real-time guidance and support.

[0536] A "user" refers to a person who uses an information system, provides personal information, and receives an activity plan.

[0537] "Personal characteristics information" refers to attribute information such as the user's name, age, and hobbies, which is stored in a database and used to generate activity plans.

[0538] A "storage device" refers to a hardware or software system for storing data or programs.

[0539] "External information sources" refer to other systems or platforms that provide various data from the internet or APIs.

[0540] "Environmental information" refers to data about external conditions that affect user activities, such as weather information and congestion information.

[0541] "Status information" refers to data related to the user's situation at that time, such as real-time location information and route information.

[0542] A "machine learning model" refers to an algorithm that learns patterns based on large amounts of data and generates an optimal activity plan for the user.

[0543] "Communication device" refers to a device used to send and receive data between a server and a user, such as a smartphone or computer.

[0544] "Evaluation information" refers to feedback received from users, including data on satisfaction with activity plans and the content of their experiences.

[0545] This invention is a system that proposes and supports activity plans tailored to the user's preferences and circumstances in real time. Specific embodiments of this system are shown below.

[0546] Users access the system using their own devices and enter personal information, including their name, age, and hobbies. This information is received by the server and stored in a database. This stored personal information is later used to generate activity plans.

[0547] On a holiday morning, users request an activity plan by entering a prompt message into the system's chat interface via their terminal. For example, they might send a prompt message like, "What activities are recommended for today?" In response to this request, the server retrieves the user's profile information and past history data, and gathers environmental and situational information from external sources.

[0548] Based on this data, the server generates an optimal activity plan using a generative AI model. This AI model incorporates machine learning algorithms and is designed to provide personalized suggestions to the user. The generated activity plan is sent from the server to the user's communication device, where multiple options are presented.

[0549] The user selects an activity plan that interests them from several options presented, and this selection is notified to the server. Subsequently, as the user proceeds with the activity, the server provides the user with real-time location and route information, supporting the activity through the device.

[0550] After an activity is completed, users send feedback to the system evaluating their experience. The server stores this evaluation information in a database and updates the AI ​​model for generating activity plans, which can then be used to create more personalized and accurate activity plans for future activities.

[0551] For example, if user A has registered information such as "I like nature" and "I'm an outdoorsy person," the server will suggest an activity plan based on that information, including hiking and lunch at a local cafe on a day with good weather. User A can select this plan and enjoy the day using navigation and real-time information from their device.

[0552] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0553] Step 1:

[0554] The user uses a terminal to input and transmit personal information into the system. This information includes name, age, preferences, etc. The terminal sends this input to the server. The server receives the entered personal information and stores it in a database as storage. In this step, the user provides data to personalize the system, and the system processes and stores that data.

[0555] Step 2:

[0556] On a holiday morning, a user uses the chat interface from their device to send a prompt message to the system. For example, they might enter a request such as, "What activities do you recommend today?" The device sends this prompt message to the server. The server receives the prompt message and retrieves personal information from its database based on the user ID. This step involves processing to prepare appropriate information in response to the user's request.

[0557] Step 3:

[0558] The server calls external APIs to obtain environmental information (weather information, congestion information, etc.) from external sources. This input includes location information and date / time. Based on this data, the server retrieves data that affects the user's current situation. Then, it integrates the retrieved data and creates a dataset that combines it with the user's personal characteristics information. In this step, real-time external information is ingested and a customized dataset is created for each user.

[0559] Step 4:

[0560] The server uses a generative AI model to generate the optimal activity plan from the dataset. This AI model incorporates machine learning algorithms that process the input data to generate the most appropriate options for the user. The generated activity plan is based on the user's personal characteristics and environmental information. In this step, complex data analysis and model inference are used to generate a customized output for the user.

[0561] Step 5:

[0562] The server sends the generated activity plan to the user's device. The user's device displays the received plan in a chat interface and presents several options. The user selects an activity plan of interest from the presented options and sends their selection to the server via their device. This step provides an interface for the user to select an activity of interest and records that selection.

[0563] Step 6:

[0564] When a user executes their selected activity plan, the server provides real-time location and route information to the device. The device displays this information to help the user smoothly carry out the activity. Based on the entered location information, appropriate directions are calculated and presented to the user. In this step, real-time navigation and feedback enable the user to effectively complete the plan.

[0565] Step 7:

[0566] After the activity ends, users input their experience evaluation information into the system from their terminal and submit feedback. The server saves this evaluation information to a database and updates the AI ​​model for use in generating future activity plans. This step incorporates user feedback into the next plan generation, aiming for continuous system improvement.

[0567] (Application Example 1)

[0568] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0569] The challenge is to enable users to easily create activity plans necessary for a fulfilling holiday based on their individual hobbies and preferences. Furthermore, it is necessary to utilize real-time information during the planning process to enhance user satisfaction. Additionally, the accuracy of the information provided needs to be improved to offer users more meaningful options.

[0570] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0571] In this invention, the server includes means for storing personal attribute information received from the user in information storage, means for acquiring weather information and congestion information from external information sources, means for generating an optimal activity plan for the user using a machine learning algorithm, means for transmitting the generated activity plan to the user's communication device, means for processing activity plan selection information received from the user, means for analyzing user feedback information and storing it in information storage to reflect it in the generation of the next activity plan, means for providing the user with local information and route guidance in real time, and means for generating prompt sentences using a generative artificial intelligence model to suggest comfortable and enjoyable activities based on the user's preferences. This enables the provision of personalized activity plans to users and real-time support based on those plans.

[0572] "User" refers to the individual consumers or participants who use this system.

[0573] "Personal attribute information" refers to characteristic information related to the user, such as age, hobbies, and preferences.

[0574] "Information storage" refers to technologies or systems for securely and efficiently storing and managing data.

[0575] "External information sources" refer to data providers that exist outside the system and provide information such as weather information and congestion data.

[0576] A "machine learning algorithm" refers to a computational method that learns patterns from data and performs predictions and classifications.

[0577] "Communication equipment" refers to terminals and devices used for sending and receiving data.

[0578] "Feedback information" refers to data that shows the opinions and satisfaction levels that users provide after experiencing a product or service.

[0579] "Real-time information" refers to information that provides immediate updates on the ongoing situation.

[0580] "Local information" refers to data related to geographical locations and facilities.

[0581] "Route guidance" refers to an information service that shows the route to a destination.

[0582] A "generative artificial intelligence model" is an algorithm that generates new content or output through natural language processing and other methods.

[0583] A "prompt message" refers to the words or commands used to convey instructions to artificial intelligence.

[0584] To implement this invention, a user-owned terminal and a server for processing information are required. The user first inputs their personal attribute information through the terminal, and this data is securely stored in the server's information storage. The hardware is envisioned to be the user's smartphone or tablet, and the software used is a cloud-based database service.

[0585] The server acquires weather and congestion information from external sources in real time and uses machine learning algorithms to generate an optimal activity plan for the user. Generative AI technology is required here, and Google Cloud's machine learning platform is utilized.

[0586] The generated activity plan is transmitted to the user via a communication device. The user selects an activity based on the provided information, and this selection is fed back to the server. Based on this feedback, the server uses a generation artificial intelligence model to generate prompts to improve the accuracy of the activity plan and provides a new plan that reflects these prompts.

[0587] For example, if a user expresses a desire to participate in an art-related event, the server gathers weather and local cultural event information and proposes a plan that includes art appreciation and lunch at a cafe. Based on this, the user executes the plan and, as a result, can have a highly satisfying experience. An example of a prompt message would be, "My current interest is art. Please suggest some pleasant and enjoyable local art-related events."

[0588] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0589] Step 1:

[0590] The user enters personal attribute information using the device. This input includes data such as the user's age, hobbies, and preferences. The device collects this data and prepares to send it to information storage.

[0591] Step 2:

[0592] The server receives personal attribute information transmitted from the terminal and stores it in its information storage. This data serves as fundamental information necessary for generating future activity plans.

[0593] Step 3:

[0594] The server accesses external information sources to obtain weather and congestion information. This allows the latest weather data and congestion status to be collected in the database. This information is then used to plan subsequent activities.

[0595] Step 4:

[0596] The server uses stored user personal attribute information and data from external sources to execute machine learning algorithms and generate an optimal activity plan for the user. Using a generative AI model, it efficiently creates multiple prompt options, leading to the best plan for the user.

[0597] Step 5:

[0598] The generated activity plan is sent from the server to the user's terminal. The user reviews this plan and chooses the option that best suits their preferences from several choices.

[0599] Step 6:

[0600] The user sends their selected activity plan information to the server via their device. The server receives this information and prepares to provide relevant location information and route guidance in real time.

[0601] Step 7:

[0602] After the user completes an activity, they send feedback information to the server via their device. The server stores this feedback in its information storage and analyzes the data to incorporate it into future planning.

[0603] Step 8:

[0604] The server uses the acquired feedback information to update its machine learning algorithm, aiming to improve the accuracy of its activity plans. This will enable more personalized responses in future activity plan suggestions.

[0605] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0606] This invention is an activity planning system that incorporates a function to recognize the user's emotions. This system generates and provides activity plans based on the user's emotional state, enabling them to have a more satisfying holiday. Its specific operation is described below.

[0607] Users first register their personal attribute information with the system using their device. This includes basic profile information as well as data on the user's activity history and emotions. This information is stored in a database by the server and used to generate activity plans.

[0608] The emotion engine recognizes the user's emotional state and understands their current feelings (joy, sadness, stress, etc.). This allows for the generation of personalized activity plans tailored to the user's preferences and current emotions. This emotional data, like other data, is considered by the server during plan generation.

[0609] On a holiday morning, users send activity plan requests via their devices. The server generates the optimal activity plan based on the user's personal attributes, activity history, and emotional data, while also considering weather information and congestion levels from external data sources. By utilizing data from the emotional engine, the system can suggest activities best suited to the user's emotional state.

[0610] The generated activity plan is sent from the server to the user's terminal, and the user selects a plan to execute from the available options. Once the plan selection is complete, the server prepares to provide real-time information while the user is out and about, supporting their activities.

[0611] After the activity is completed, users use their devices to provide feedback on their evaluation of the plan and any newly arising emotions. The server stores this feedback information in a database and analyzes it, including the emotional data. This information is used to improve the accuracy of future activity plan generation.

[0612] As a concrete example, consider the case of User B using the system. User B has recently been feeling stressed and wants to relax. The emotion engine, recognizing User B's emotional state, suggests activities suitable for stress reduction, such as reading in a quiet park or listening to natural sounds. User B chooses one of the suggested plans and, by engaging in the activities based on the presented plan and navigation, can enjoy a refreshing holiday.

[0613] The following describes the processing flow.

[0614] Step 1:

[0615] Users use their devices to input profile information and emotion-related data (such as their current mood and emotion history) and register it with the system.

[0616] Step 2:

[0617] The terminal sends the entered information to the server.

[0618] Step 3:

[0619] The server verifies the received information and stores personal attribute information, sentiment data, and activity history in a database.

[0620] Step 4:

[0621] The emotion engine recognizes the user's current emotional state and provides the result to the server.

[0622] Step 5:

[0623] On a holiday morning, users request an activity plan via the chat interface on their device.

[0624] Step 6:

[0625] The server considers user profiles and sentiment data, and retrieves weather and congestion information from external data sources.

[0626] Step 7:

[0627] The server uses machine learning models to generate an optimal activity plan that takes into account the user's preferences, current emotional state, weather, and congestion levels.

[0628] Step 8:

[0629] The generated activity plan is sent from the server to the device, tailored to the user's emotional state.

[0630] Step 9:

[0631] The user reviews the activity plan presented on their device and selects a plan to execute. The selection is then sent to the server.

[0632] Step 10:

[0633] Based on the user's selections, the server prepares to provide real-time information (location information, guidance information, etc.) while the user is out and about.

[0634] Step 11:

[0635] While the user is out and about, the server provides real-time location information and navigation to the user's device.

[0636] Step 12:

[0637] After the activity ends, the user uses their device to send an evaluation of the activity, feedback, and changes in their emotional state during the activity to the server.

[0638] Step 13:

[0639] The server stores the received feedback and sentiment data in a database and uses it to generate future plans and update machine learning models.

[0640] (Example 2)

[0641] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0642] Conventional activity planning systems failed to adequately address users' emotional states and individual preferences, making it difficult to provide users with satisfying holiday plans. Furthermore, delays or inaccuracies in providing information based on external circumstances led to problems with the feasibility and effectiveness of the plans.

[0643] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0644] In this invention, the server includes means for storing personal attribute information and emotional data received from the user in a data storage means, means for acquiring weather data and congestion data from an external information source, and means for generating an optimal activity plan for the user by utilizing a model for analyzing the user's emotional state. This makes it possible to provide a more personalized activity plan in real time, taking into account the user's individual emotional state and external environment.

[0645] "Personal attribute information" refers to a collection of data that includes a user's basic profile information and past activity history.

[0646] "Emotional data" refers to information that indicates a user's current emotional state and is used to personalize activity plans.

[0647] "Data storage means" refers to elements such as databases and storage devices that store information and allow for quick access when needed.

[0648] "External information sources" refer to organizations or networks that provide data such as weather information and congestion information obtained from outside the system.

[0649] "Weather data" refers to information that indicates the environmental conditions of a location, such as weather, temperature, and humidity.

[0650] "Congestion data" refers to information about the congestion levels in a specific area or facility.

[0651] A "model" refers to a machine learning model or algorithm used to analyze a user's emotional state and generate an appropriate action plan.

[0652] "Terminal device" refers to an electronic device used by a user to send and receive information or to check activity plans.

[0653] This invention is a system that generates an activity plan based on the user's emotional state and provides the user with the optimal choice. This system mainly consists of a server and terminals and operates as follows.

[0654] Users use a device to input their personal attribute information and sentiment data and send it to the server. This device can be a smartphone, tablet, or personal computer—any electronic device capable of inputting, verifying, and sending / receiving such information. Upon receiving this information, the server records it in a database and uses it to generate subsequent action plans.

[0655] The server also has communication technologies to obtain weather and congestion data from external sources. This allows local weather and congestion conditions to be taken into consideration when planning.

[0656] User emotion data is analyzed by an emotion analysis engine located on the server to understand the user's current emotional state. This analysis utilizes specific models based on machine learning techniques. For example, an emotion recognition model analyzes the user's text and voice input to identify emotions such as joy, sadness, and stress.

[0657] The server uses a generated AI model based on the user's personal attribute information, emotional state, and external environment data to create an optimal activity plan for the user. This activity plan is sent to the terminal and presented to the user. The user can then select the most suitable plan from the presented options and act accordingly.

[0658] As a concrete example, consider a case where a user experiencing stress seeks relaxation through the system. The system can suggest activities such as reading or listening to music in a quiet park, and this suggestion is optimized based on the user's emotional analysis results and environmental data.

[0659] Example of a prompt:

[0660] "The user's current emotion is stress. Please propose an activity plan to reduce this stress."

[0661] This configuration allows users to receive activity plans that perfectly match their emotional state, resulting in a more fulfilling experience.

[0662] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0663] Step 1:

[0664] The user uses a terminal to input their personal attribute information and emotional data. Specifically, they enter information such as their name, age, past activities, and current emotions into a form. This information is processed by the terminal and sent to the server. The input data is in text format and stored in the server's data storage system. The output of this step is the personal attribute information and emotional data stored in the database.

[0665] Step 2:

[0666] The server retrieves weather and congestion data from external sources. In this process, the server uses external APIs to obtain local weather information and congestion data for specific locations, and records it in a database. As a result, the input is up-to-date information from external sources, and the output is this information neatly stored and passed on to the next step.

[0667] Step 3:

[0668] The server runs an emotion analysis engine to analyze the user's emotional data. This step uses machine learning algorithms to identify the user's emotional state (e.g., joy, sadness, stress) from the input emotional data. The output of the process is an analysis report containing the recognized emotional state, which is used to generate an action plan in the next step.

[0669] Step 4:

[0670] The server uses a generative AI model to generate an optimal activity plan for the user. The inputs here are personal attribute information, analyzed sentiment data, and external environment data. Based on this information, the AI ​​model lists potential activities and generates an optimal plan that meets the user's needs. The output is a specific activity plan, which is presented to the user in the next step.

[0671] Step 5:

[0672] The generated activity plan is sent to the terminal. The terminal displays the plan provided to the user, and the user selects the activities they wish to perform. Once the user has completed their selections on the interface, that information is sent from the terminal to the server as feedback. In this step, the selected activity plan and the user's feedback are output.

[0673] Step 6:

[0674] The server stores user feedback information in a database and performs analysis to help generate future activity plans. Based on the feedback input data, it analyzes the correlation with further sentiment data to improve the overall system accuracy. This output is the analyzed feedback data stored in the database.

[0675] (Application Example 2)

[0676] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0677] In today's busy lifestyle, users face the challenge of planning activities that align with their current emotional state. Furthermore, there is a lack of systems that provide personalized activity suggestions based on the user's emotional state. As a result, it is difficult for users to enjoy fulfilling leisure time.

[0678] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0679] In this invention, the server includes means for storing personal attribute information received from the user in a storage device, means for collecting environmental information and crowd information from external information sources, and means for utilizing an artificial intelligence module that analyzes the user's emotional state using an emotion recognition function. This makes it possible to provide activity suggestions based on the user's emotional state.

[0680] "Personal attribute information" refers to information that indicates individual characteristics of a user, such as age, gender, preferences, and activity history.

[0681] A "storage device" is a device that stores data over a long period of time and allows it to be retrieved and used as needed.

[0682] An "external information source" is a data source that exists outside the system and provides information related to the activity plan, such as environmental information or crowd information.

[0683] "Environmental information" refers to physical information that affects the user's environment, such as weather conditions, temperature, and humidity.

[0684] "Crowd information" refers to information about collective behavior, such as the degree of crowding and traffic conditions in a specific area.

[0685] A "machine learning algorithm" is a computational processing method that learns patterns and rules from data and uses them to predict future situations and make decisions.

[0686] A "communication terminal" is a device used to send and receive information, and includes smartphones and tablets.

[0687] "Emotion recognition functionality" is a technology that analyzes input data such as the user's voice and facial expressions to identify their emotional state.

[0688] An "artificial intelligence module" is an artificial intelligence-related software component designed to handle specific tasks or problems.

[0689] An "activity suggestion" is a specific action or plan recommended based on the user's preferences and emotional state.

[0690] The system implementing this invention provides personalized activity suggestions based on the user's emotional state. The server implements this process using the following hardware and software.

[0691] The server first stores personal attribute information received from the user in a storage device. This process securely and efficiently collects and stores data such as the user's age, preferences, and activity history.

[0692] Next, the server collects environmental information (such as weather and temperature) and crowd information (such as congestion levels and traffic conditions) from external sources. This helps to make the user's activity plan realistic and feasible.

[0693] The server uses an artificial intelligence module with emotion recognition capabilities to analyze the user's emotional state from their voice and facial expression data. This utilizes general-purpose voice analysis software, facial recognition libraries, and emotion recognition APIs.

[0694] Based on the analysis results, the server utilizes machine learning algorithms to create an activity plan tailored to the user. The generated activity plan is immediately sent to the user's communication terminal.

[0695] For example, if a user wants to alleviate daily stress, the server uses emotion recognition to determine the user's emotional state as "stressed" and accordingly suggests activities with a relaxing effect, such as a quiet walk.

[0696] Example prompt for a generative AI model: "If the user's current emotional state is stress, what relaxation activities should be suggested? Please include a sentence indicating that the user's emotions are being taken into consideration."

[0697] This format allows for activity suggestions tailored to the individual needs of users, enabling them to have highly satisfying leisure experiences.

[0698] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0699] Step 1:

[0700] The server receives personal attribute information (age, preferences, activity history, etc.) from the user as input and stores it in a storage device. This storage process securely stores the data using a database management system. The output is a collection of personal attributes that can be used to generate future activity plans.

[0701] Step 2:

[0702] The server takes environmental information (weather and temperature) and crowd information (congestion level and traffic conditions) as input from external sources. This allows the server to collect contextual data around the user's current location and evaluate the feasibility of activity plans. The output is up-to-date and detailed environmental and crowd information.

[0703] Step 3:

[0704] The server uses an artificial intelligence module with emotion recognition capabilities to receive user voice and facial expression data as input and analyze their emotional state. Voice analysis software and a facial recognition library are utilized here. The output is data indicating the user's emotional state (e.g., joy, stress).

[0705] Step 4:

[0706] The server synthesizes personal attribute information, environmental / crowd information, and emotional state, and uses machine learning algorithms to generate an optimal activity plan. Pattern recognition and predictive models are applied to generate the activity plan, creating individual suggestions for each user. The output is a list of specific activity suggestions.

[0707] Step 5:

[0708] The generated activity proposals are sent by the server to the user's communication terminal. The terminal receives the proposals and displays them to the user. The displayed information includes details of the proposed activity and its background information. The output is an activity proposal presented visually to the user.

[0709] Step 6:

[0710] The user selects an activity from the received activity suggestions and sends this selection information to the server. The server records the selected activity as input and refers to it in subsequent feedback processing. The output is the specific activity information selected by the user.

[0711] Step 7:

[0712] After completing an activity, users provide feedback by entering information on a terminal. The server receives this feedback and records it in a storage device. This information includes satisfaction levels and areas for improvement. The output is feedback data used to improve the accuracy of future activity plans.

[0713] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0714] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0715] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0716] [Fourth Embodiment]

[0717] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0718] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0719] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0720] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0721] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0722] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0723] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0724] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0725] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0726] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0727] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0728] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0729] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0730] This invention is a system that helps users have more meaningful holidays. This system presents an activity plan based on the user's personal attribute information and provides real-time support for that plan. Its specific operation is described below.

[0731] Users first register their personal information with the system using their device. This includes personal attribute information such as name, age, and hobbies. Once registration is complete, this information is stored in a database by the server and used to generate future activity plans.

[0732] On a holiday morning, users request an activity plan using the chat interface on their device. The server accepts this request, investigates the user's profile and past activity history, and then retrieves current weather information and congestion status from external data sources. Subsequently, considering this data and the user's preferences, a machine learning model is used to generate the optimal activity plan.

[0733] The generated plan is sent from the server to the user's device. The user selects an action plan from several presented options, and this selection is notified to the server. Once the user starts going out, the server provides the user with real-time location information and directions. This feature allows users to make appropriate decisions based on the situation and carry out their activities more systematically.

[0734] After the activity ends, users send feedback to the system regarding their satisfaction with the plan and their experience. The server stores this feedback in a database and uses it to continuously improve the machine learning model. This enables more accurate personalization in future plan generation, further improving user satisfaction.

[0735] As a concrete example, the personal information registered by user A includes preferences such as "I like nature" and "I enjoy the outdoors." Based on this information, the server suggests an activity plan that includes hiking in a nearby park and lunch at a popular local cafe on a day with good weather. User A selects this plan and can enjoy the day while receiving navigation and updates on congestion levels from their device. After the activity, user A provides feedback on their satisfaction level, and the server uses this information to improve the accuracy of future plans.

[0736] The following describes the processing flow.

[0737] Step 1:

[0738] Users use a device to enter personal information (name, age, hobbies, etc.) and register it in the system.

[0739] Step 2:

[0740] The terminal sends the entered personal attribute information to the server.

[0741] Step 3:

[0742] The server verifies the received personal attribute information and stores it in the database.

[0743] Step 4:

[0744] On holiday mornings, users use their device's chat interface to send activity plan requests to the system.

[0745] Step 5:

[0746] The server retrieves the user's profile information and past behavioral history, preparing the basic data for plan generation.

[0747] Step 6:

[0748] The server obtains weather and congestion information from external data sources and collects real-time information necessary for plan generation.

[0749] Step 7:

[0750] The server uses machine learning models to generate the optimal activity plan for the user. This process takes into account user preferences, weather, and congestion levels.

[0751] Step 8:

[0752] The server sends the generated activity plan to the user's device and prompts them to make a selection.

[0753] Step 9:

[0754] The user reviews the activity plan presented through their device and selects a plan to execute. The selection is then sent to the server.

[0755] Step 10:

[0756] The server receives the user's selection and prepares to provide real-time information while the user is away from their computer.

[0757] Step 11:

[0758] While the user is out, the server continues to provide location and navigation information to the user's device in real time.

[0759] Step 12:

[0760] After the activity ends, the user sends feedback about the plan to the server from their device.

[0761] Step 13:

[0762] The server stores the received feedback in a database and uses it to update the machine learning model for future plan generation.

[0763] (Example 1)

[0764] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0765] It is difficult to efficiently propose activity plans that match the individual preferences and circumstances of each user regarding how to spend their holidays. Furthermore, it is necessary to utilize environmental information and user feedback and effectively reflect this in future plans. In addition, a system is needed that provides real-time support and helps users make appropriate decisions based on their circumstances.

[0766] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0767] In this invention, the server includes means for storing personal characteristic information received from the user in a storage device, means for acquiring environmental information and situational information from external information sources, and means for generating an optimal activity plan for the user using a machine learning model. This makes it possible to present a personalized activity plan according to the user's characteristics and circumstances, and to improve user satisfaction through real-time guidance and support.

[0768] A "user" refers to a person who uses an information system, provides personal information, and receives an activity plan.

[0769] "Personal characteristics information" refers to attribute information such as the user's name, age, and hobbies, which is stored in a database and used to generate activity plans.

[0770] A "storage device" refers to a hardware or software system for storing data or programs.

[0771] "External information sources" refer to other systems or platforms that provide various data from the internet or APIs.

[0772] "Environmental information" refers to data about external conditions that affect user activities, such as weather information and congestion information.

[0773] "Status information" refers to data related to the user's situation at that time, such as real-time location information and route information.

[0774] A "machine learning model" refers to an algorithm that learns patterns based on large amounts of data and generates an optimal activity plan for the user.

[0775] "Communication device" refers to a device used to send and receive data between a server and a user, such as a smartphone or computer.

[0776] "Evaluation information" refers to feedback received from users, including data on satisfaction with activity plans and the content of their experiences.

[0777] This invention is a system that proposes and supports activity plans tailored to the user's preferences and circumstances in real time. Specific embodiments of this system are shown below.

[0778] Users access the system using their own devices and enter personal information, including their name, age, and hobbies. This information is received by the server and stored in a database. This stored personal information is later used to generate activity plans.

[0779] On a holiday morning, users request an activity plan by entering a prompt message into the system's chat interface via their terminal. For example, they might send a prompt message like, "What activities are recommended for today?" In response to this request, the server retrieves the user's profile information and past history data, and gathers environmental and situational information from external sources.

[0780] Based on this data, the server generates an optimal activity plan using a generative AI model. This AI model incorporates machine learning algorithms and is designed to provide personalized suggestions to the user. The generated activity plan is sent from the server to the user's communication device, where multiple options are presented.

[0781] The user selects an activity plan that interests them from several options presented, and this selection is notified to the server. Subsequently, as the user proceeds with the activity, the server provides the user with real-time location and route information, supporting the activity through the device.

[0782] After an activity is completed, users send feedback to the system evaluating their experience. The server stores this evaluation information in a database and updates the AI ​​model for generating activity plans, which can then be used to create more personalized and accurate activity plans for future activities.

[0783] For example, if user A has registered information such as "I like nature" and "I'm an outdoorsy person," the server will suggest an activity plan based on that information, including hiking and lunch at a local cafe on a day with good weather. User A can select this plan and enjoy the day using navigation and real-time information from their device.

[0784] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0785] Step 1:

[0786] The user uses a terminal to input and transmit personal information into the system. This information includes name, age, preferences, etc. The terminal sends this input to the server. The server receives the entered personal information and stores it in a database as storage. In this step, the user provides data to personalize the system, and the system processes and stores that data.

[0787] Step 2:

[0788] On a holiday morning, a user uses the chat interface from their device to send a prompt message to the system. For example, they might enter a request such as, "What activities do you recommend today?" The device sends this prompt message to the server. The server receives the prompt message and retrieves personal information from its database based on the user ID. This step involves processing to prepare appropriate information in response to the user's request.

[0789] Step 3:

[0790] The server calls external APIs to obtain environmental information (weather information, congestion information, etc.) from external sources. This input includes location information and date / time. Based on this data, the server retrieves data that affects the user's current situation. Then, it integrates the retrieved data and creates a dataset that combines it with the user's personal characteristics information. In this step, real-time external information is ingested and a customized dataset is created for each user.

[0791] Step 4:

[0792] The server uses a generative AI model to generate the optimal activity plan from the dataset. This AI model incorporates machine learning algorithms that process the input data to generate the most appropriate options for the user. The generated activity plan is based on the user's personal characteristics and environmental information. In this step, complex data analysis and model inference are used to generate a customized output for the user.

[0793] Step 5:

[0794] The server sends the generated activity plan to the user's device. The user's device displays the received plan in a chat interface and presents several options. The user selects an activity plan of interest from the presented options and sends their selection to the server via their device. This step provides an interface for the user to select an activity of interest and records that selection.

[0795] Step 6:

[0796] When a user executes their selected activity plan, the server provides real-time location and route information to the device. The device displays this information to help the user smoothly carry out the activity. Based on the entered location information, appropriate directions are calculated and presented to the user. In this step, real-time navigation and feedback enable the user to effectively complete the plan.

[0797] Step 7:

[0798] After the activity ends, users input their experience evaluation information into the system from their terminal and submit feedback. The server saves this evaluation information to a database and updates the AI ​​model for use in generating future activity plans. This step incorporates user feedback into the next plan generation, aiming for continuous system improvement.

[0799] (Application Example 1)

[0800] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0801] The challenge is to enable users to easily create activity plans necessary for a fulfilling holiday based on their individual hobbies and preferences. Furthermore, it is necessary to utilize real-time information during the planning process to enhance user satisfaction. Additionally, the accuracy of the information provided needs to be improved to offer users more meaningful options.

[0802] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0803] In this invention, the server includes means for storing personal attribute information received from the user in information storage, means for acquiring weather information and congestion information from external information sources, means for generating an optimal activity plan for the user using a machine learning algorithm, means for transmitting the generated activity plan to the user's communication device, means for processing activity plan selection information received from the user, means for analyzing user feedback information and storing it in information storage to reflect it in the generation of the next activity plan, means for providing the user with local information and route guidance in real time, and means for generating prompt sentences using a generative artificial intelligence model to suggest comfortable and enjoyable activities based on the user's preferences. This enables the provision of personalized activity plans to users and real-time support based on those plans.

[0804] "User" refers to the individual consumers or participants who use this system.

[0805] "Personal attribute information" refers to characteristic information related to the user, such as age, hobbies, and preferences.

[0806] "Information storage" refers to technologies or systems for securely and efficiently storing and managing data.

[0807] "External information sources" refer to data providers that exist outside the system and provide information such as weather information and congestion data.

[0808] A "machine learning algorithm" refers to a computational method that learns patterns from data and performs predictions and classifications.

[0809] "Communication equipment" refers to terminals and devices used for sending and receiving data.

[0810] "Feedback information" refers to data that shows the opinions and satisfaction levels that users provide after experiencing a product or service.

[0811] "Real-time information" refers to information that provides immediate updates on the ongoing situation.

[0812] "Local information" refers to data related to geographical locations and facilities.

[0813] "Route guidance" refers to an information service that shows the route to a destination.

[0814] A "generative artificial intelligence model" is an algorithm that generates new content or output through natural language processing and other methods.

[0815] A "prompt message" refers to the words or commands used to convey instructions to artificial intelligence.

[0816] To implement this invention, a user-owned terminal and a server for processing information are required. The user first inputs their personal attribute information through the terminal, and this data is securely stored in the server's information storage. The hardware is envisioned to be the user's smartphone or tablet, and the software used is a cloud-based database service.

[0817] The server acquires weather and congestion information from external sources in real time and uses machine learning algorithms to generate an optimal activity plan for the user. Generative AI technology is required here, and Google Cloud's machine learning platform is utilized.

[0818] The generated activity plan is transmitted to the user via a communication device. The user selects an activity based on the provided information, and this selection is fed back to the server. Based on this feedback, the server uses a generation artificial intelligence model to generate prompts to improve the accuracy of the activity plan and provides a new plan that reflects these prompts.

[0819] For example, if a user expresses a desire to participate in an art-related event, the server gathers weather and local cultural event information and proposes a plan that includes art appreciation and lunch at a cafe. Based on this, the user executes the plan and, as a result, can have a highly satisfying experience. An example of a prompt message would be, "My current interest is art. Please suggest some pleasant and enjoyable local art-related events."

[0820] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0821] Step 1:

[0822] The user enters personal attribute information using the device. This input includes data such as the user's age, hobbies, and preferences. The device collects this data and prepares to send it to information storage.

[0823] Step 2:

[0824] The server receives personal attribute information transmitted from the terminal and stores it in its information storage. This data serves as fundamental information necessary for generating future activity plans.

[0825] Step 3:

[0826] The server accesses external information sources to obtain weather and congestion information. This allows the latest weather data and congestion status to be collected in the database. This information is then used to plan subsequent activities.

[0827] Step 4:

[0828] The server uses stored user personal attribute information and data from external sources to execute machine learning algorithms and generate an optimal activity plan for the user. Using a generative AI model, it efficiently creates multiple prompt options, leading to the best plan for the user.

[0829] Step 5:

[0830] The generated activity plan is sent from the server to the user's terminal. The user reviews this plan and chooses the option that best suits their preferences from several choices.

[0831] Step 6:

[0832] The user sends their selected activity plan information to the server via their device. The server receives this information and prepares to provide relevant location information and route guidance in real time.

[0833] Step 7:

[0834] After the user completes an activity, they send feedback information to the server via their device. The server stores this feedback in its information storage and analyzes the data to incorporate it into future planning.

[0835] Step 8:

[0836] The server uses the acquired feedback information to update its machine learning algorithm, aiming to improve the accuracy of its activity plans. This will enable more personalized responses in future activity plan suggestions.

[0837] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0838] This invention is an activity planning system that incorporates a function to recognize the user's emotions. This system generates and provides activity plans based on the user's emotional state, enabling them to have a more satisfying holiday. Its specific operation is described below.

[0839] Users first register their personal attribute information with the system using their device. This includes basic profile information as well as data on the user's activity history and emotions. This information is stored in a database by the server and used to generate activity plans.

[0840] The emotion engine recognizes the user's emotional state and understands their current feelings (joy, sadness, stress, etc.). This allows for the generation of personalized activity plans tailored to the user's preferences and current emotions. This emotional data, like other data, is considered by the server during plan generation.

[0841] On a holiday morning, users send activity plan requests via their devices. The server generates the optimal activity plan based on the user's personal attributes, activity history, and emotional data, while also considering weather information and congestion levels from external data sources. By utilizing data from the emotional engine, the system can suggest activities best suited to the user's emotional state.

[0842] The generated activity plan is sent from the server to the user's terminal, and the user selects a plan to execute from the available options. Once the plan selection is complete, the server prepares to provide real-time information while the user is out and about, supporting their activities.

[0843] After the activity is completed, users use their devices to provide feedback on their evaluation of the plan and any newly arising emotions. The server stores this feedback information in a database and analyzes it, including the emotional data. This information is used to improve the accuracy of future activity plan generation.

[0844] As a concrete example, consider the case of User B using the system. User B has recently been feeling stressed and wants to relax. The emotion engine, recognizing User B's emotional state, suggests activities suitable for stress reduction, such as reading in a quiet park or listening to natural sounds. User B chooses one of the suggested plans and, by engaging in the activities based on the presented plan and navigation, can enjoy a refreshing holiday.

[0845] The following describes the processing flow.

[0846] Step 1:

[0847] Users use their devices to input profile information and emotion-related data (such as their current mood and emotion history) and register it with the system.

[0848] Step 2:

[0849] The terminal sends the entered information to the server.

[0850] Step 3:

[0851] The server verifies the received information and stores personal attribute information, sentiment data, and activity history in a database.

[0852] Step 4:

[0853] The emotion engine recognizes the user's current emotional state and provides the result to the server.

[0854] Step 5:

[0855] On a holiday morning, users request an activity plan via the chat interface on their device.

[0856] Step 6:

[0857] The server considers user profiles and sentiment data, and retrieves weather and congestion information from external data sources.

[0858] Step 7:

[0859] The server uses machine learning models to generate an optimal activity plan that takes into account the user's preferences, current emotional state, weather, and congestion levels.

[0860] Step 8:

[0861] The generated activity plan is sent from the server to the device, tailored to the user's emotional state.

[0862] Step 9:

[0863] The user reviews the activity plan presented on their device and selects a plan to execute. The selection is then sent to the server.

[0864] Step 10:

[0865] Based on the user's selections, the server prepares to provide real-time information (location information, guidance information, etc.) while the user is out and about.

[0866] Step 11:

[0867] While the user is out and about, the server provides real-time location information and navigation to the user's device.

[0868] Step 12:

[0869] After the activity ends, the user uses their device to send an evaluation of the activity, feedback, and changes in their emotional state during the activity to the server.

[0870] Step 13:

[0871] The server stores the received feedback and sentiment data in a database and uses it to generate future plans and update machine learning models.

[0872] (Example 2)

[0873] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0874] Conventional activity planning systems failed to adequately address users' emotional states and individual preferences, making it difficult to provide users with satisfying holiday plans. Furthermore, delays or inaccuracies in providing information based on external circumstances led to problems with the feasibility and effectiveness of the plans.

[0875] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0876] In this invention, the server includes means for storing personal attribute information and emotional data received from the user in a data storage means, means for acquiring weather data and congestion data from an external information source, and means for generating an optimal activity plan for the user by utilizing a model for analyzing the user's emotional state. This makes it possible to provide a more personalized activity plan in real time, taking into account the user's individual emotional state and external environment.

[0877] "Personal attribute information" refers to a collection of data that includes a user's basic profile information and past activity history.

[0878] "Emotional data" refers to information that indicates a user's current emotional state and is used to personalize activity plans.

[0879] "Data storage means" refers to elements such as databases and storage devices that store information and allow for quick access when needed.

[0880] "External information sources" refer to organizations or networks that provide data such as weather information and congestion information obtained from outside the system.

[0881] "Weather data" refers to information that indicates the environmental conditions of a location, such as weather, temperature, and humidity.

[0882] "Congestion data" refers to information about the congestion levels in a specific area or facility.

[0883] A "model" refers to a machine learning model or algorithm used to analyze a user's emotional state and generate an appropriate action plan.

[0884] "Terminal device" refers to an electronic device used by a user to send and receive information or to check activity plans.

[0885] This invention is a system that generates an activity plan based on the user's emotional state and provides the user with the optimal choice. This system mainly consists of a server and terminals and operates as follows.

[0886] Users use a device to input their personal attribute information and sentiment data and send it to the server. This device can be a smartphone, tablet, or personal computer—any electronic device capable of inputting, verifying, and sending / receiving such information. Upon receiving this information, the server records it in a database and uses it to generate subsequent action plans.

[0887] The server also has communication technologies to obtain weather and congestion data from external sources. This allows local weather and congestion conditions to be taken into consideration when planning.

[0888] User emotion data is analyzed by an emotion analysis engine located on the server to understand the user's current emotional state. This analysis utilizes specific models based on machine learning techniques. For example, an emotion recognition model analyzes the user's text and voice input to identify emotions such as joy, sadness, and stress.

[0889] The server uses a generated AI model based on the user's personal attribute information, emotional state, and external environment data to create an optimal activity plan for the user. This activity plan is sent to the terminal and presented to the user. The user can then select the most suitable plan from the presented options and act accordingly.

[0890] As a concrete example, consider a case where a user experiencing stress seeks relaxation through the system. The system can suggest activities such as reading or listening to music in a quiet park, and this suggestion is optimized based on the user's emotional analysis results and environmental data.

[0891] Example of a prompt:

[0892] "The user's current emotion is stress. Please propose an activity plan to reduce this stress."

[0893] This configuration allows users to receive activity plans that perfectly match their emotional state, resulting in a more fulfilling experience.

[0894] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0895] Step 1:

[0896] The user uses a terminal to input their personal attribute information and emotional data. Specifically, they enter information such as their name, age, past activities, and current emotions into a form. This information is processed by the terminal and sent to the server. The input data is in text format and stored in the server's data storage system. The output of this step is the personal attribute information and emotional data stored in the database.

[0897] Step 2:

[0898] The server retrieves weather and congestion data from external sources. In this process, the server uses external APIs to obtain local weather information and congestion data for specific locations, and records it in a database. As a result, the input is up-to-date information from external sources, and the output is this information neatly stored and passed on to the next step.

[0899] Step 3:

[0900] The server runs an emotion analysis engine to analyze the user's emotional data. This step uses machine learning algorithms to identify the user's emotional state (e.g., joy, sadness, stress) from the input emotional data. The output of the process is an analysis report containing the recognized emotional state, which is used to generate an action plan in the next step.

[0901] Step 4:

[0902] The server uses a generative AI model to generate an optimal activity plan for the user. The inputs here are personal attribute information, analyzed sentiment data, and external environment data. Based on this information, the AI ​​model lists potential activities and generates an optimal plan that meets the user's needs. The output is a specific activity plan, which is presented to the user in the next step.

[0903] Step 5:

[0904] The generated activity plan is sent to the terminal. The terminal displays the plan provided to the user, and the user selects the activities they wish to perform. Once the user has completed their selections on the interface, that information is sent from the terminal to the server as feedback. In this step, the selected activity plan and the user's feedback are output.

[0905] Step 6:

[0906] The server stores user feedback information in a database and performs analysis to help generate future activity plans. Based on the feedback input data, it analyzes the correlation with further sentiment data to improve the overall system accuracy. This output is the analyzed feedback data stored in the database.

[0907] (Application Example 2)

[0908] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0909] In today's busy lifestyle, users face the challenge of planning activities that align with their current emotional state. Furthermore, there is a lack of systems that provide personalized activity suggestions based on the user's emotional state. As a result, it is difficult for users to enjoy fulfilling leisure time.

[0910] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0911] In this invention, the server includes means for storing personal attribute information received from the user in a storage device, means for collecting environmental information and crowd information from external information sources, and means for utilizing an artificial intelligence module that analyzes the user's emotional state using an emotion recognition function. This makes it possible to provide activity suggestions based on the user's emotional state.

[0912] "Personal attribute information" refers to information that indicates individual characteristics of a user, such as age, gender, preferences, and activity history.

[0913] A "storage device" is a device that stores data over a long period of time and allows it to be retrieved and used as needed.

[0914] An "external information source" is a data source that exists outside the system and provides information related to the activity plan, such as environmental information or crowd information.

[0915] "Environmental information" refers to physical information that affects the user's environment, such as weather conditions, temperature, and humidity.

[0916] "Crowd information" refers to information about collective behavior, such as the degree of crowding and traffic conditions in a specific area.

[0917] A "machine learning algorithm" is a computational processing method that learns patterns and rules from data and uses them to predict future situations and make decisions.

[0918] A "communication terminal" is a device used to send and receive information, and includes smartphones and tablets.

[0919] "Emotion recognition functionality" is a technology that analyzes input data such as the user's voice and facial expressions to identify their emotional state.

[0920] An "artificial intelligence module" is an artificial intelligence-related software component designed to handle specific tasks or problems.

[0921] An "activity suggestion" is a specific action or plan recommended based on the user's preferences and emotional state.

[0922] The system implementing this invention provides personalized activity suggestions based on the user's emotional state. The server implements this process using the following hardware and software.

[0923] The server first stores personal attribute information received from the user in a storage device. This process securely and efficiently collects and stores data such as the user's age, preferences, and activity history.

[0924] Next, the server collects environmental information (such as weather and temperature) and crowd information (such as congestion levels and traffic conditions) from external sources. This helps to make the user's activity plan realistic and feasible.

[0925] The server uses an artificial intelligence module with emotion recognition capabilities to analyze the user's emotional state from their voice and facial expression data. This utilizes general-purpose voice analysis software, facial recognition libraries, and emotion recognition APIs.

[0926] Based on the analysis results, the server utilizes machine learning algorithms to create an activity plan tailored to the user. The generated activity plan is immediately sent to the user's communication terminal.

[0927] For example, if a user wants to alleviate daily stress, the server uses emotion recognition to determine the user's emotional state as "stressed" and accordingly suggests activities with a relaxing effect, such as a quiet walk.

[0928] Example prompt for a generative AI model: "If the user's current emotional state is stress, what relaxation activities should be suggested? Please include a sentence indicating that the user's emotions are being taken into consideration."

[0929] This format allows for activity suggestions tailored to the individual needs of users, enabling them to have highly satisfying leisure experiences.

[0930] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0931] Step 1:

[0932] The server receives personal attribute information (age, preferences, activity history, etc.) from the user as input and stores it in a storage device. This storage process securely stores the data using a database management system. The output is a collection of personal attributes that can be used to generate future activity plans.

[0933] Step 2:

[0934] The server takes environmental information (weather and temperature) and crowd information (congestion level and traffic conditions) as input from external sources. This allows the server to collect contextual data around the user's current location and evaluate the feasibility of activity plans. The output is up-to-date and detailed environmental and crowd information.

[0935] Step 3:

[0936] The server uses an artificial intelligence module with emotion recognition capabilities to receive user voice and facial expression data as input and analyze their emotional state. Voice analysis software and a facial recognition library are utilized here. The output is data indicating the user's emotional state (e.g., joy, stress).

[0937] Step 4:

[0938] The server synthesizes personal attribute information, environmental / crowd information, and emotional state, and uses machine learning algorithms to generate an optimal activity plan. Pattern recognition and predictive models are applied to generate the activity plan, creating individual suggestions for each user. The output is a list of specific activity suggestions.

[0939] Step 5:

[0940] The generated activity proposals are sent by the server to the user's communication terminal. The terminal receives the proposals and displays them to the user. The displayed information includes details of the proposed activity and its background information. The output is an activity proposal presented visually to the user.

[0941] Step 6:

[0942] The user selects an activity from the received activity suggestions and sends this selection information to the server. The server records the selected activity as input and refers to it in subsequent feedback processing. The output is the specific activity information selected by the user.

[0943] Step 7:

[0944] After completing an activity, users provide feedback by entering information on a terminal. The server receives this feedback and records it in a storage device. This information includes satisfaction levels and areas for improvement. The output is feedback data used to improve the accuracy of future activity plans.

[0945] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0946] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0947] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0948] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0949] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0950] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0951] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0952] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0953] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0954] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0955] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0956] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0957] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0958] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0959] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0960] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0961] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0962] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0963] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0964] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0965] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0966] The following is further disclosed regarding the embodiments described above.

[0967] (Claim 1)

[0968] A means of storing personal attribute information received from users in a database,

[0969] A means of obtaining weather information and congestion information from an external data source,

[0970] A means of generating an optimal activity plan for a user using a machine learning model,

[0971] A means for transmitting the generated activity plan to the user's communication device,

[0972] A means for processing activity plan selection information received from the user,

[0973] A system that includes means for analyzing user feedback information and storing it in a database to reflect it in the generation of future activity plans.

[0974] (Claim 2)

[0975] The system according to claim 1, comprising means for providing location information and guidance information to a user in real time.

[0976] (Claim 3)

[0977] The system according to claim 1, comprising means for analyzing user activity history data and updating a machine learning model to improve the accuracy of activity planning.

[0978] "Example 1"

[0979] (Claim 1)

[0980] A means for storing personal characteristic information received from a user in a storage device,

[0981] Means for obtaining environmental information and situational information from external sources,

[0982] A means of generating an optimal activity plan for a user using a machine learning model,

[0983] A means for transmitting the generated activity plan to the user's communication device,

[0984] A means for processing activity plan selection information received from the user,

[0985] A means of analyzing user evaluation information and storing it in a storage device to reflect it in the generation of the next activity plan,

[0986] A system that includes means for controlling the acquisition of external data in accordance with user instructions.

[0987] (Claim 2)

[0988] The system according to claim 1, comprising means for providing location information and route information to a user in real time.

[0989] (Claim 3)

[0990] The system according to claim 1, comprising means for analyzing user behavior history data and updating a machine learning model to improve the accuracy of activity plans.

[0991] "Application Example 1"

[0992] (Claim 1)

[0993] A means of storing personal attribute information received from a user in information storage,

[0994] Means for obtaining weather information and congestion information from external sources,

[0995] A means of generating an optimal activity plan for a user using a machine learning algorithm,

[0996] A means for transmitting the generated activity plan to the user's communication device,

[0997] A means for processing activity plan selection information received from the user,

[0998] A means of analyzing user feedback information and storing it in information storage to reflect it in the generation of the next activity plan,

[0999] A system that includes means of providing users with local information and route guidance in real time.

[1000] (Claim 2)

[1001] The system according to claim 1, comprising means for analyzing user activity history data and updating machine learning algorithms to improve the accuracy of activity plans.

[1002] (Claim 3)

[1003] The system according to claim 1, comprising a prompt sentence generation means using a generative artificial intelligence model to suggest comfortable and enjoyable activities based on the user's preferences.

[1004] "Example 2 of combining an emotion engine"

[1005] (Claim 1)

[1006] A means for storing personal attribute information and emotional data received from a user in a data storage means,

[1007] Means for obtaining weather data and congestion data from external sources,

[1008] A means of generating an optimal activity plan for a user by utilizing a model to analyze the user's emotional state,

[1009] A means for transmitting the generated activity plan to the user's terminal device,

[1010] A means for processing activity plan selection information received from the user,

[1011] A system that includes means for storing user feedback information in a data storage device and analyzing it to reflect it in the generation of the next activity plan.

[1012] (Claim 2)

[1013] The system according to claim 1, comprising means for providing location data and guidance data to a user in real time.

[1014] (Claim 3)

[1015] The system according to claim 1, comprising means for analyzing user activity history information and emotional feedback data and updating a model to improve the accuracy of activity planning.

[1016] "Application example 2 when combining with an emotional engine"

[1017] (Claim 1)

[1018] A means for storing personal attribute information received from a user in a storage device,

[1019] Means for collecting environmental information and crowd information from external sources,

[1020] A means of creating a user-friendly action plan using machine learning algorithms,

[1021] A means for transmitting the created action plan to the user's communication terminal,

[1022] A means for processing action plan selection information received from the user,

[1023] A means for analyzing user evaluation information and storing it in a storage device to reflect it in the creation of the next action plan,

[1024] A means of using an artificial intelligence module that analyzes the user's emotional state using emotion recognition functionality,

[1025] A means of providing activity suggestions based on the user's emotional state,

[1026] A system that includes this.

[1027] (Claim 2)

[1028] The system according to claim 1, comprising means for providing spatial information and guidance information to a user in real time.

[1029] (Claim 3)

[1030] The system according to claim 1, comprising means for analyzing user activity history data and updating machine learning algorithms to improve the accuracy of action plans. [Explanation of Symbols]

[1031] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of storing personal attribute information received from a user in information storage, Means for obtaining weather information and congestion information from external sources, A means of generating an optimal activity plan for a user using a machine learning algorithm, A means for transmitting the generated activity plan to the user's communication device, A means for processing activity plan selection information received from the user, A means of analyzing user feedback information and storing it in information storage to reflect it in the generation of the next activity plan, A system that includes means of providing users with local information and route guidance in real time.

2. The system according to claim 1, further comprising means for analyzing user activity history data and updating machine learning algorithms to improve the accuracy of activity plans.

3. The system according to claim 1, comprising a prompt sentence generation means using a generative artificial intelligence model to suggest comfortable and enjoyable activities based on the user's preferences.

Images