system

Abstract

We provide the system. [Solution] An information processing device provides means for acquiring a user's social media data via a communication device, The information processing device includes means for generating an interest profile based on the data, The schedule management device provides a means for identifying the user's free time, The proposed device includes means for suggesting activities based on the interest profile and free time, The notification device includes means for notifying the user of the proposal, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In an information society, users seek to spend efficient and fulfilling time in their busy daily lives. However, sufficient methods for effectively incorporating activities that suit individual hobbies and preferences into free time have not been provided. In such an environment, there is a need for a system that enables users to effectively utilize their time and improve the quality of life.

Means for Solving the Problems

[0005] This invention provides a system that notifies users of highly satisfying activities tailored to their individual interests and schedules, by including an information processing device that acquires users' social media data and generates an interest profile based on it, a schedule management device that manages the user's schedule and identifies free time, and a suggestion device that proposes activities based on the interest profile and free time. As a result, users can use their time efficiently and improve their quality of life.

[0006] An "information processing device" is a device that acquires social media data and generates interest profiles from it.

[0007] A "communication device" is a device used by an information processing device to acquire social media data, and it is a device that sends and receives data over a network.

[0008] A "user" is an individual who uses this system to receive activity suggestions based on their interests.

[0009] "Social media data" refers to digital data such as posts, images, and location information that users share on social media.

[0010] An "interest profile" is a collection of information generated by analyzing a user's interests and preferences based on social media data.

[0011] A "schedule management device" is a device that manages users' schedule data and identifies their free time.

[0012] A "suggestion device" is a device that recommends activities to users based on their interest profile and free time.

[0013] A "notification device" is a device that informs users of suggested activities, and typically provides information via push notifications or digital displays.

[0014] "Feedback" refers to information regarding the evaluation and satisfaction level of users regarding the suggested activities.

[0015] A "generative model" is an algorithm used by the proposed device to select activities based on the user's interest profile and feedback data. [Brief explanation of the drawing]

[0016] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when combined with an emotion engine. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when combined with an emotion engine.

Embodiments for Carrying Out the Invention

[0017] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0018] First, the language used in the following description will be explained.

[0019] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0020] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0021] In the following embodiments, a numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0022] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0024] [First Embodiment]

[0025] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0026] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0029] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0032] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0036] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0037] The system for carrying out the present invention includes an information processing device, a communication device, a schedule management device, a suggestion device, and a notification device. By coordinating these, the system proposes the optimal activity to the user. The details are described below.

[0038] First, the device accesses the user's social media data via a communication device. With the user's permission, it uses social media APIs to retrieve past posts, images, and current location information. This enables real-time data collection.

[0039] Next, the information processing device on the server analyzes the acquired data and generates an interest profile that reflects the user's interests and preferences. Here, text analysis and image recognition technologies are used to extract relevant categories from the posted content. For example, "cafes," "travel," and "outdoors" are identified as the user's interests.

[0040] Next, the device retrieves the user's schedule information. It works in conjunction with the user's calendar app to identify periods of free time. At this time, it analyzes event information and reminders to calculate the exact time.

[0041] The server's suggestion system combines interest profiles and schedule information to propose the most suitable activities based on the user's current location. For example, if the user has two hours of free time, it might suggest visiting a popular nearby cafe. It can also prioritize places the user has never visited before, based on their past history.

[0042] The proposed content will be notified to the user in real time via the device's notification system. Using the smartphone's push notification function, the specific activity name, location, and summary will be provided to the user. This will result in a user-friendly interface.

[0043] After performing a suggested activity, users can input feedback on their device. This feedback includes information about satisfaction and the accuracy of the suggestion, and is used to improve the accuracy of subsequent suggestions. This feedback is also sent to the server and used to update the generative model. As a concrete example, a user might visit a suggested cafe and evaluate whether their experience met their expectations.

[0044] In this way, the present invention can effectively utilize users' free time and suggest activities that are best suited to their individual interests. This improves the quality of life for users and enables efficient use of their time.

[0045] The following describes the processing flow.

[0046] Step 1:

[0047] The device collects the user's past posts, images, and location information via social media APIs with the user's permission. This data includes the text content of posts, image metadata, and location information.

[0048] Step 2:

[0049] The device sends the collected social media data to the server. This transmission uses security protocols to protect the data.

[0050] Step 3:

[0051] The server uses a generative AI model to analyze the user's interests and preferences based on the data it receives. This extracts categories that indicate the user's interests (e.g., "cafes," "sports," "movies") and generates an interest profile.

[0052] Step 4:

[0053] The device connects with the user's calendar to retrieve schedule information. In this process, it analyzes the current and upcoming appointment times to identify free time.

[0054] Step 5:

[0055] The device obtains the user's current location information using GPS. This information is used to select activities to suggest during free time.

[0056] Step 6:

[0057] The server determines the most suitable activity for the user based on the generated interest profile, identified free time, and current location information. For example, if the user has two hours of free time, it might suggest visiting a nearby art gallery.

[0058] Step 7:

[0059] The device notifies the user of the suggested activity. This notification includes the activity's name, location, and detailed information. The information is displayed on the user's smartphone using push notifications or in-app notifications.

[0060] Step 8:

[0061] After the user completes the suggested activity, they enter feedback about their experience. This feedback includes satisfaction levels and an evaluation of the activity.

[0062] Step 9:

[0063] The device collects user feedback and sends that data to the server. This feedback is used to improve the quality of future suggestions.

[0064] Step 10:

[0065] The server updates its AI model based on the feedback it receives, improving the accuracy of future suggestions. This update enables more accurate activity suggestions based on the user's preferences and behavioral patterns.

[0066] (Example 1)

[0067] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0068] In modern society, there is a growing need for automated systems that offer suggestions for meaningfully utilizing limited free time based on an individual's interests and preferences. However, conventional systems have struggled to efficiently collect and analyze diverse user information and propose appropriate activities in real time. To address this challenge, it is necessary to provide a method that integrates user social media data, schedule information, and location information to provide highly accurate activity suggestions.

[0069] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0070] In this invention, the server includes means for generating an interest profile using text analysis and image recognition technology, means for suggesting activities based on the interest profile, free time, and location information, and means for updating the generated AI model using feedback. As a result, users can make meaningful use of their free time by being accurately suggested activities that match their individual interests.

[0071] A "terminal" is an information processing device that a user can directly operate and that has the function of receiving or transmitting external data via a communication device.

[0072] A "server" is a central information processing device that stores, processes, and manages various types of data over a network.

[0073] "Social media data" refers to digital data based on user activity on online platforms, including posts, comments, location information, and images.

[0074] An "interest profile" represents a user's preferences and interests, and is a collection of categorized information generated based on social media data.

[0075] A "calendar app" is software used to manage a user's schedule and events, allowing them to input and view schedule information.

[0076] "Push notifications" are information proactively sent from a server to a device, and are a means of communication that provides users with real-time information.

[0077] A "generative AI model" is an algorithm that utilizes artificial intelligence for data analysis and prediction, and is a model that is improved and updated through machine learning.

[0078] "Feedback" is evaluation information provided by users based on their experiences and impressions, and it is data that can be used to improve the system.

[0079] This invention provides an activity suggestion system that allows individuals to utilize their free time based on their interests. The system primarily consists of a terminal, server, communication device, schedule management device, suggestion device, and notification device. The detailed functions of each of these elements are described below.

[0080] A terminal is an information processing device operated by the user, such as a smartphone or tablet. The terminal accesses social media APIs via communication devices to obtain user-generated data and location information. This enables the terminal to collect data in real time.

[0081] The acquired data is sent to a server and analyzed by an information processing device. The server is equipped with software for text analysis and image recognition, specifically using natural language processing (NLP) and computer vision technologies. These technologies generate an interest profile that reflects the user's interests.

[0082] The schedule management device works in conjunction with the device's calendar app to analyze the user's schedule information. The device reads schedule data via an API to identify the user's free time and analyzes the importance of events.

[0083] The suggestion device resides on a server and proposes optimal activities based on the generated interest profile, free time, and current location information. This makes it possible to select the most suitable activity for the user. For example, if the user has two hours of free time, a visit to a nearby popular cafe might be suggested.

[0084] The suggested activities are notified to the user in real time via the device's notification system. These notifications are sent to the smartphone as push notifications, providing detailed activity information and resulting in a user-friendly interface.

[0085] For example, if a user frequently posts about cafes on social media, their interests profile will include "cafes," and they will be suggested to visit nearby cafes during their free time.

[0086] An example of a prompt for a generative AI model would be: "Based on the user's past social media posts, suggest an activity that would be suitable for them to perform. Consider the user's current location and calendar data to determine a place they should visit during their two-hour free time."

[0087] This invention enables the provision of activity suggestions based on the user's individual interests, thereby improving their quality of life and supporting efficient time management.

[0088] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0089] Step 1:

[0090] The device retrieves the user's social media data via a communication device. Specifically, the device sends a request to the social media API and, with the user's permission, downloads past posts, images, and location information. The input for this step is the API key and user ID, and the output is the set of retrieved social media data.

[0091] Step 2:

[0092] The server analyzes social media data sent from the terminal. The server applies text analysis and image recognition technologies to extract relevant categories from the user's posts and images. The input is social media data, and the output is an interest profile indicating the user's interests.

[0093] Step 3:

[0094] The device retrieves schedule information from the user's calendar app. The device reads the appointment data via an API and runs an algorithm to identify free time. The input is the calendar API key and user ID, and the output is a list of identified free time slots.

[0095] Step 4:

[0096] The server generates activities using a proposed device based on interest profiles and free time information. The server also considers location information and searches for relevant activities in the database. The inputs are interest profiles, free time, and location information, and the output is detailed information about the proposed activities.

[0097] Step 5:

[0098] The device notifies the user of activity suggestions received from the server. The device uses push notification functionality to send the activity name, location, and details to the user's smartphone. The input is the detailed data of the activity suggestion, and the output is the notification to the user.

[0099] Step 6:

[0100] The user performs the suggested activity and enters feedback about the experience into the device. The user records their thoughts and evaluations through a feedback form. The input is the user's feedback, and the output is an updated feedback dataset.

[0101] Step 7:

[0102] The server updates the generative AI model using user feedback. The server analyzes the feedback data and applies it to the learning algorithm to improve the accuracy of the next activity suggestion. The input is the updated feedback dataset, and the output is the improved generative AI model.

[0103] (Application Example 1)

[0104] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0105] In modern society, many users want to spend their free time efficiently based on their interests and passions. However, finding suitable activities is not easy and can be time-consuming and require effort. Furthermore, if activity suggestions do not match individual interests and schedules, users are less likely to take action, which can diminish the user's convenience.

[0106] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0107] In this invention, the server includes an information processing device that acquires the user's social media data via a communication device, a means for generating an interest profile, and a schedule management device that identifies free time. This makes it possible to suggest optimal activities in real time based on the user's individual interests, current location, and free time.

[0108] An "information processing device" is a device that acquires a user's social media data, location information, and schedule information, and generates an interest profile by analyzing this data.

[0109] A "communication device" is a device that connects a user's device to an information processing device, enabling real-time transmission and reception of data.

[0110] A "schedule management device" is a device that works in conjunction with the user's calendar app to understand the user's schedule and identify free time.

[0111] A "suggestion device" is a device that selects and suggests the most suitable activities to the user based on the interest profile and schedule information generated by an information processing device.

[0112] A "notification device" is a device that informs users of suggested activities, and it uses the push notification function of smartphones.

[0113] A "user interface device" is a device used to explain proposed content to the user via the robot's voice function.

[0114] To implement this invention, it is necessary to build a system in which three entities—a server, a terminal, and a user—work in cooperation. First, the server uses an information processing device to acquire the user's social media data via a communication device. This data is used to generate an interest profile that reflects the user's interests, using text analysis and image recognition technologies. For example, Amazon's AWS® Lambda or Google® Cloud Vision API can be used for the server.

[0115] Next, a schedule management device installed in the terminal checks the user's calendar app to confirm their schedule and identify free time. The suggestion device combines the interest profile and schedule information, and selects and suggests the most suitable activities while considering the user's current location. This suggestion is notified to the user in real time via a notification device, and the terminal uses a user interface device to explain the suggestion verbally. Smartphones and consumer robots can be used as terminals.

[0116] As a concrete example, a scenario could be envisioned where a user receives a suggestion from their smartphone or a consumer robot during their free time after work, such as "attending a music event being held at a nearby cafe." The user can provide satisfaction feedback on this suggestion, and the results are sent to a server and used to update the generated AI model.

[0117] Example of a prompt:

[0118] "The user is interested in cafes and music, is currently located in Tokyo, and has free time from 18:00 to 20:00. Based on these conditions, please suggest appropriate activities."

[0119] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0120] Step 1:

[0121] The server retrieves user social media data from the communication device via an information processing device. The input is the user's social media data obtained through the API, and the output is the raw data stored in the database. This process, with the user's permission, uses APIs such as Instagram and Twitter to collect past posts and location information.

[0122] Step 2:

[0123] The server analyzes acquired social media data and generates an interest profile that reflects the user's interests. The input is the raw data obtained in step 1, and the output is profile information indicating the user's interest categories. At this stage, text analysis technology is used to extract themes from the posts, and image recognition technology is used to identify visual interests.

[0124] Step 3:

[0125] The device uses a scheduling device to retrieve the user's calendar information and identify free time. The input is schedule data obtained from the user's calendar app, and the output is a list of available free time. In this process, appointments are read via a calendar API, and free time is calculated while avoiding overlaps.

[0126] Step 4:

[0127] The server's suggestion device combines the user's interest profile, free time, and current location information to select the most suitable activity. The input is the data and location information obtained in steps 2 and 3, and the output is a list of activities to suggest. Here, a geographical database is referenced to select events of interest near the user.

[0128] Step 5:

[0129] The terminal's notification device informs the user of the suggested activity and provides an audio explanation using the user interface device. The input is the activity information selected in step 4, and the output is the audio and text notification delivered to the user. A consumer robot provides detailed information via voice using the smartphone's push notification function.

[0130] Step 6:

[0131] After the user completes an activity, they provide feedback via their device. The input is feedback data provided by the user, and the output is feedback information stored on the server. This feedback evaluates satisfaction with the notified activity and the accuracy of the suggestions.

[0132] Step 7:

[0133] The server updates the AI ​​model based on user feedback to improve the accuracy of the suggestions. The input is the feedback data obtained in step 6, and the output is the improved suggestion algorithm. At this stage, machine learning techniques are used to tune the suggestion algorithm and incorporate the results into the next suggestion.

[0134] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0135] This invention is a system that proposes activities optimized for the user by combining social media data analysis, schedule management, activity suggestion, and an emotion engine. This system includes an information processing device, a communication device, a schedule management device, a suggestion device, a notification device, and an emotion engine.

[0136] First, the device uses a communication device to retrieve posted data, images, and location information from the user's social media API. The data obtained at this time is collected from the user's public profile.

[0137] Next, the server's information processing unit uses a generative AI model to generate a user interest profile based on the acquired data. This process involves text analysis and image recognition to extract user interest categories such as "movies," "hiking," and "cooking."

[0138] Furthermore, an emotion engine is used to analyze the user's past posts and photos to form an emotion profile. For example, if the content of a post indicates emotions such as "fun" or "satisfied," this will be reflected in the emotion profile.

[0139] The device syncs with the user's calendar to identify upcoming appointments and free time between appointments. This free time data is sent to the server via a scheduling device.

[0140] Subsequently, the server's suggestion device integrates the generated interest profile, schedule data, current location information, and emotion profile to select an activity. In this selection process, for example, if the emotion profile indicates that the user is "depressed," the server will suggest an activity that will cheer them up, such as "watch your favorite comedy movie."

[0141] Suggestions are notified to the user in real time via the device's notification system. User reactions and satisfaction with the suggestions are also monitored, and flexible suggestions are made in real time based on their emotions.

[0142] After performing a suggested activity, the user inputs the results into the terminal and provides feedback. This feedback is used in the server's generative model to enable more accurate suggestions. For example, after watching a suggested movie, the user inputs their thoughts and evaluation into the system.

[0143] Through the above operations, the present invention can support users in making better use of their time by suggesting activities that match their interests and emotional state.

[0144] The following describes the processing flow.

[0145] Step 1:

[0146] The device, with the user's permission, collects user-generated content, images, and location information via social media APIs. This data includes posts from the past few weeks.

[0147] Step 2:

[0148] The device transmits the collected data to the server via a secure communication method. Encryption is applied during this process to protect the confidentiality of the data.

[0149] Step 3:

[0150] The server analyzes the received data using text analysis and image recognition technologies to generate a user's interest profile. For example, it might extract categories such as "sports," "cafes," and "travel."

[0151] Step 4:

[0152] The server uses an emotion engine to analyze posted data and determine the user's emotions from their past posts. This creates an emotion profile, such as positive or negative.

[0153] Step 5:

[0154] The device uses a calendar app to retrieve the user's schedule information and identify available time slots. This data is obtained directly from the user's device.

[0155] Step 6:

[0156] The device uses GPS functionality to obtain the user's current location. This allows for geographical optimization of suggested activities.

[0157] Step 7:

[0158] The server combines interest profiles, emotional profiles, schedule information, and current location information, and uses a generative AI model to select the most suitable activity for the user. For example, if the user is feeling stressed, it might suggest a nearby relaxing cafe.

[0159] Step 8:

[0160] The device notifies the user of suggested activities via push notifications or in-app screens. This notification includes the activity name, location, and detailed information.

[0161] Step 9:

[0162] The user performs the suggested activity and enters feedback about the experience into the device. This feedback includes satisfaction levels and impressions.

[0163] Step 10:

[0164] The device sends user feedback to the server, updating the generative model. This feedback information is then incorporated into the next proposal, improving its accuracy.

[0165] (Example 2)

[0166] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0167] In modern society, despite individuals having access to vast amounts of data from diverse sources, there is a challenge in effectively utilizing that data based on individual behaviors and emotions. Therefore, there is a need for systems that appropriately analyze users' interests and emotional states and suggest activities tailored to their individual circumstances.

[0168] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0169] In this invention, the server includes means for acquiring social network data via a communication device, means for generating interest characteristics using a generative artificial intelligence model, means for forming emotional characteristics using emotion analysis means, means for identifying the user's free time through schedule management, means for suggesting appropriate activities based on this information, and means for collecting feedback and updating the generative model. This makes it possible to automatically and dynamically provide suggestions optimized for the user's individual interests and emotions.

[0170] An "information processing device" is an electronic device used for data collection, analysis, and information generation.

[0171] A "communication device" is a device used to send and receive information over a network.

[0172] A "generative artificial intelligence model" is an algorithm that learns patterns based on data and generates predictions and suggestions.

[0173] "Interest characteristics" refer to a collection of information that indicates the themes and topics that users are interested in.

[0174] "Emotional analysis means" refers to a method or technology for analyzing user data and determining their emotional state.

[0175] "Emotional characteristics" refer to a collection of information that indicates a user's emotional state.

[0176] A "schedule management device" is a device or function used to manage a user's schedule and identify available time slots.

[0177] "Activities" refer to actions or behaviors suggested to users, and include recreational and hobby activities.

[0178] "Feedback" refers to the user's comments and evaluations of the suggestions made by the system.

[0179] "Methods for updating generative models" refer to methods of modifying a model to improve its generative capabilities based on newly obtained data.

[0180] This system modernizes behavior based on individual interests and emotions by collecting and analyzing user data and suggesting optimized activities. The system primarily relies on the exchange of information and data between the server, the terminal, and the user.

[0181] The server is the core of this system, using an information processing device to collect publicly available user data via APIs of social networking services. The collected data is analyzed using a generative artificial intelligence model to generate user interest characteristics. Interest characteristics are data that reflects the user's hobbies and interests, such as movies, cooking, and travel.

[0182] Furthermore, the server uses sentiment analysis tools to generate sentiment characteristics from the user's past posts. This process analyzes the images and comments posted by the user and classifies their emotions at the time into categories such as positive or negative. As a result, it becomes possible to understand what kind of emotions the user experiences in different situations.

[0183] The device primarily works in conjunction with the user's calendar application, identifying the user's free time through a schedule management system. This free time data is used as important basic information when making suggestions.

[0184] The proposed device integrates this analyzed data and uses a generative artificial intelligence model to select the optimal activity. This selection takes into account the user's interests, emotional characteristics, free time, and location. For example, if the device determines that the user has recently been feeling stressed, it can suggest watching a relaxing movie.

[0185] Selected activities are notified to the user in real time via the device's notification system. The user can review the suggested activities and choose whether or not to perform them. After performing the suggested activity, the user provides feedback on the activity to the system by entering it into the device. This feedback is provided to the server's generative model as new training material, leading to improved accuracy.

[0186] For example, if a user is unsure what to do on their day off, this system will suggest a movie for that day. After enjoying the movie, the user can provide feedback to the system, which will then generate even more personalized suggestions for the next time.

[0187] Example of a prompt:

[0188] "Analyze past posts to assess user interest in the movie category. Also, suggest relaxation activities that are recommended when users are experiencing strong negative emotions."

[0189] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0190] Step 1:

[0191] The device connects to the user's social networking service API using a communication device to retrieve posted data, images, and location information. The input is the user's public profile information, and the output is a dataset of these. This retrieved data is used in subsequent analysis steps.

[0192] Step 2:

[0193] The server uses an information processing device to input the data acquired in step 1 into a generative artificial intelligence model. The input data consists of user-submitted data and images, and the output is the user's interest characteristics. Specifically, the categories that the user is interested in are identified through text analysis and image recognition. For example, the analysis may extract interests such as "travel" and "cooking."

[0194] Step 3:

[0195] The server uses sentiment analysis tools to generate sentiment characteristics based on the data acquired in Step 1. The input is the user's posts and photos, and the output is sentiment characteristics. Using the sentiment analysis engine, it classifies emotions such as positive and negative from the text and images contained in the user's past posts and forms an sentiment profile.

[0196] Step 4:

[0197] The device works in conjunction with the user's calendar application to identify the user's free time through its schedule management function. The input is the user's schedule information, and the output is a list of free time slots. This allows the user to reserve time between appointments and notify the server of this.

[0198] Step 5:

[0199] The server's suggestion device integrates the interest characteristics generated in step 2, the emotional characteristics from step 3, and the free time data from step 4. The input is these analyzed datasets, and the output is the suggested activities. A generative AI model is used to select the most suitable activity based on this data. At this stage, for example, if the system determines that the user is tired, it will suggest relaxing activities.

[0200] Step 6:

[0201] The device's notification system will notify the user in real time of the activity suggested in step 5. The input is information about the suggested activity, and the output is a notification message to the user. The user can review the suggestion and decide on their next action.

[0202] Step 7:

[0203] After performing a suggested activity, the user enters feedback into the device. The input is the user's evaluation and impressions of the activity, and the output is feedback data sent to the server. The server updates its generative model based on this feedback to improve the accuracy of future suggestions.

[0204] (Application Example 2)

[0205] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0206] In modern society, users are often extremely busy, making it difficult to find suitable activities to effectively utilize their free time. Furthermore, personalized suggestions based on users' interests and emotions are rare, and the sheer volume of information can make selection difficult. This creates a challenge for users in leading fulfilling lives.

[0207] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0208] In this invention, the server includes an information processing means for acquiring the user's social media information via a communication means, a means for the information processing means to generate interest information based on the information, and an emotion analysis engine for analyzing the user's emotions and generating emotion data. This enables personalized activity suggestions based on the user's interests and emotions to be provided in real time.

[0209] "Information processing means" refers to a device that acquires a user's social media information through communication means and generates interest information based on that information.

[0210] "Communication means" refers to a system for exchanging information between information processing means and social media.

[0211] "Interest information" refers to data that represents a user's interests and preferences, and is generated based on information from social media.

[0212] An "emotion analysis engine" is a device that analyzes a user's emotions and generates emotional data based on that analysis.

[0213] "Emotional data" refers to data that represents a user's emotional state using numerical values ​​or categories, and is generated by an emotion analysis engine.

[0214] A "schedule management system" is a system for managing users' schedules and identifying available time slots.

[0215] An "activity suggestion device" is a device that suggests the most suitable activity to a user based on their interests, emotional data, and free time.

[0216] A "notification system" is a system for informing users about suggested activities.

[0217] "User geographical location information" refers to data about where the user is located on Earth.

[0218] A "generative model" is a model that has algorithms for making predictions and suggestions based on data, and is updated based on feedback from users.

[0219] "Opinions" refer to feedback and evaluations from users, which are used to update the generative model.

[0220] To realize this invention, the server first collects user data from social media via communication means using information processing means. This data includes posts, images, location information, etc. Next, the server's information processing means generates user interest information using a generative AI model based on the acquired data. Here, technologies such as natural language processing and image recognition are utilized, and specifically, software such as TextBlob and OpenCV may be used.

[0221] In parallel, the server uses an emotion analysis engine to analyze the user's emotions from past posts and images, and generates emotion data. This process reveals the user's emotional state in real time.

[0222] The device manages calendar data using a scheduling tool and identifies the user's free time. This information is sent to the server and used to suggest activities.

[0223] Next, the server's activity suggestion system integrates interest information, emotional data, free time, and geographical location information to select and suggest the most suitable activity for the user. In this process, for example, an activity that energizes the user is provided according to their current state.

[0224] The suggested activity is notified to the user in real time via the device's notification system. The user performs the suggested activity and sends feedback about their experience to the server. This feedback information is reflected in the generative model and used to improve the accuracy of future suggestions.

[0225] As a concrete example, suppose the user enjoys watching movies and is currently feeling down. In this case, the system can consider the emotional profile and suggest "watching a favorite comedy movie." An example of the suggestion is as follows:

[0226] "User's latest social media posts:

[0227] Cooking tags

[0228] Happy feelings

[0229] Current free time: 2 hours

[0230] Please propose cooking-based activities. Choose activities that can be done at home and use ingredients that are readily available.

[0231] In this way, users can make more fulfilling use of their time.

[0232] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0233] Step 1:

[0234] The server uses APIs from social media to retrieve user posts, images, and location information through information processing tools. It takes data from social media APIs as input and outputs raw data in digital format based on that data. This process involves accessing API endpoints and collecting necessary data using authentication tokens.

[0235] Step 2:

[0236] The server's information processing mechanism uses the data acquired in Step 1 to generate interest information using a generative AI model. The input is social media data, and the data is analyzed using natural language processing libraries (e.g., TextBlob) and image recognition tools (e.g., OpenCV) to output data indicating the user's interests. Specifically, it identifies a particular interest category through text analysis.

[0237] Step 3:

[0238] The server uses an emotion analysis engine to further analyze the acquired data and generate emotion data. The input is also social media data, and it identifies emotional states such as positive and negative through natural language processing and outputs the analysis results. Here, the content of the user's text and images is evaluated to form an emotion profile.

[0239] Step 4:

[0240] The device uses a schedule management tool to collect digital calendar data from calendar applications and other sources to identify the user's free time. The input is calendar data, and the analysis result outputs a list of free time slots. This operation includes searching for available slots using a calendar API.

[0241] Step 5:

[0242] The server's activity suggestion mechanism integrates interest information and sentiment data obtained in steps 2 and 3, and free time in step 4, to suggest the most suitable activity for the user. The inputs are interest information, sentiment data, and free time. Based on this data, it selects relevant activities and outputs suggestions. The resulting output is an activity list based on conditions pre-set in the prompt message.

[0243] Step 6:

[0244] The proposed activity is notified to the user in real time via the device's notification system. Activity information is entered into the notification system, and the notification result is output. Here, an instant message is sent to the user using the device's push notification service.

[0245] Step 7:

[0246] The user performs the activity notified by the server, inputs their opinion about the experience as feedback on their device, and sends it to the server. The feedback includes an evaluation of the activity, and data necessary for adjusting the generative model is output. This feedback information is used to optimize future suggestions.

[0247] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0248] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0249] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0250] [Second Embodiment]

[0251] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0252] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0253] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0254] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0255] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0256] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0257] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0258] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0259] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0260] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0261] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0262] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0263] The system for carrying out the present invention includes an information processing device, a communication device, a schedule management device, a suggestion device, and a notification device. By coordinating these, the system proposes the optimal activity to the user. The details are described below.

[0264] First, the device accesses the user's social media data via a communication device. With the user's permission, it uses social media APIs to retrieve past posts, images, and current location information. This enables real-time data collection.

[0265] Next, the information processing device on the server analyzes the acquired data and generates an interest profile that reflects the user's interests and preferences. Here, text analysis and image recognition technologies are used to extract relevant categories from the posted content. For example, "cafes," "travel," and "outdoors" are identified as the user's interests.

[0266] Next, the device retrieves the user's schedule information. It works in conjunction with the user's calendar app to identify periods of free time. At this time, it analyzes event information and reminders to calculate the exact time.

[0267] The server's suggestion system combines interest profiles and schedule information to propose the most suitable activities based on the user's current location. For example, if the user has two hours of free time, it might suggest visiting a popular nearby cafe. It can also prioritize places the user has never visited before, based on their past history.

[0268] The proposed content will be notified to the user in real time via the device's notification system. Using the smartphone's push notification function, the specific activity name, location, and summary will be provided to the user. This will result in a user-friendly interface.

[0269] After performing a suggested activity, users can input feedback on their device. This feedback includes information about satisfaction and the accuracy of the suggestion, and is used to improve the accuracy of subsequent suggestions. This feedback is also sent to the server and used to update the generative model. As a concrete example, a user might visit a suggested cafe and evaluate whether their experience met their expectations.

[0270] In this way, the present invention can effectively utilize users' free time and suggest activities that are best suited to their individual interests. This improves the quality of life for users and enables efficient use of their time.

[0271] The following describes the processing flow.

[0272] Step 1:

[0273] The device collects the user's past posts, images, and location information via social media APIs with the user's permission. This data includes the text content of posts, image metadata, and location information.

[0274] Step 2:

[0275] The device sends the collected social media data to the server. This transmission uses security protocols to protect the data.

[0276] Step 3:

[0277] The server uses a generative AI model to analyze the user's interests and preferences based on the data it receives. This extracts categories that indicate the user's interests (e.g., "cafes," "sports," "movies") and generates an interest profile.

[0278] Step 4:

[0279] The device connects with the user's calendar to retrieve schedule information. In this process, it analyzes the current and upcoming appointment times to identify free time.

[0280] Step 5:

[0281] The device obtains the user's current location information using GPS. This information is used to select activities to suggest during free time.

[0282] Step 6:

[0283] The server determines the optimal activity for the user based on the generated interest profile, identified free time, and current location information. For example, if there is two hours of free time, a proposal to visit a nearby art gallery is made.

[0284] Step 7:

[0285] The terminal notifies the user of the proposed activity. This notification includes the name, location, and detailed information of the activity. Push notifications or in-app notifications are used to display the information on the user's smartphone.

[0286] Step 8:

[0287] After the user executes the presented activity, the user inputs feedback about the experience. This feedback includes satisfaction and an evaluation of the activity.

[0288] Step 9:

[0289] The terminal collects the feedback from the user and sends the data to the server. This feedback is utilized to improve the quality of future proposals.

[0290] Step 10:

[0291] The server updates the generated AI model based on the received feedback to improve the accuracy of future proposals. This update enables more accurate activity proposals based on the user's preferences and behavior patterns.

[0292] (Example 1)

[0293] Next, Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0294] In modern society, there is a growing need for automated systems that offer suggestions for meaningfully utilizing limited free time based on an individual's interests and preferences. However, conventional systems have struggled to efficiently collect and analyze diverse user information and propose appropriate activities in real time. To address this challenge, it is necessary to provide a method that integrates user social media data, schedule information, and location information to provide highly accurate activity suggestions.

[0295] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0296] In this invention, the server includes means for generating an interest profile using text analysis and image recognition technology, means for suggesting activities based on the interest profile, free time, and location information, and means for updating the generated AI model using feedback. As a result, users can make meaningful use of their free time by being accurately suggested activities that match their individual interests.

[0297] A "terminal" is an information processing device that a user can directly operate and that has the function of receiving or transmitting external data via a communication device.

[0298] A "server" is a central information processing device that stores, processes, and manages various types of data over a network.

[0299] "Social media data" refers to digital data based on user activity on online platforms, including posts, comments, location information, and images.

[0300] An "interest profile" represents a user's preferences and interests, and is a collection of categorized information generated based on social media data.

[0301] The "Calendar App" is software for managing a user's schedule and events, enabling the input and viewing of schedule information.

[0302] "Push notification" is information actively sent from a server to a terminal, serving as a communication means for providing real-time information to the user.

[0303] The "Generative AI Model" is an algorithm that utilizes artificial intelligence for data analysis and prediction, and is a model improved and updated through machine learning.

[0304] "Feedback" is evaluation information given based on the results and impressions experienced by the user, and is data useful for improving the system.

[0305] The present invention provides an activity proposal system that utilizes free time based on an individual's interests. The system mainly consists of a terminal, a server, a communication device, a schedule management device, a proposal device, and a notification device. The detailed functions of each of these elements will be described below.

[0306] The terminal is an information processing device operated by the user, such as a smartphone or a tablet. The terminal accesses the API of social media through the communication device to obtain the user's posted data and location information. Thereby, the terminal realizes real-time data collection.

[0307] The acquired data is sent to the server and analyzed by the information processing device. The server is equipped with software for text analysis and image recognition. Specifically, natural language processing (NLP) and computer vision technology are used. Through these technologies, an interest profile reflecting the user's interests is generated.

[0308] The schedule management device works in conjunction with the device's calendar app to analyze the user's schedule information. The device reads schedule data via an API to identify the user's free time and analyzes the importance of events.

[0309] The suggestion device resides on a server and proposes optimal activities based on the generated interest profile, free time, and current location information. This makes it possible to select the most suitable activity for the user. For example, if the user has two hours of free time, a visit to a nearby popular cafe might be suggested.

[0310] The suggested activities are notified to the user in real time via the device's notification system. These notifications are sent to the smartphone as push notifications, providing detailed activity information and resulting in a user-friendly interface.

[0311] For example, if a user frequently posts about cafes on social media, their interests profile will include "cafes," and they will be suggested to visit nearby cafes during their free time.

[0312] An example of a prompt for a generative AI model would be: "Based on the user's past social media posts, suggest an activity that would be suitable for them to perform. Consider the user's current location and calendar data to determine a place they should visit during their two-hour free time."

[0313] This invention enables the provision of activity suggestions based on the user's individual interests, thereby improving their quality of life and supporting efficient time management.

[0314] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0315] Step 1:

[0316] The device retrieves the user's social media data via a communication device. Specifically, the device sends a request to the social media API and, with the user's permission, downloads past posts, images, and location information. The input for this step is the API key and user ID, and the output is the set of retrieved social media data.

[0317] Step 2:

[0318] The server analyzes social media data sent from the terminal. The server applies text analysis and image recognition technologies to extract relevant categories from the user's posts and images. The input is social media data, and the output is an interest profile indicating the user's interests.

[0319] Step 3:

[0320] The device retrieves schedule information from the user's calendar app. The device reads the appointment data via an API and runs an algorithm to identify free time. The input is the calendar API key and user ID, and the output is a list of identified free time slots.

[0321] Step 4:

[0322] The server generates activities using a proposed device based on interest profiles and free time information. The server also considers location information and searches for relevant activities in the database. The inputs are interest profiles, free time, and location information, and the output is detailed information about the proposed activities.

[0323] Step 5:

[0324] The device notifies the user of activity suggestions received from the server. The device uses push notification functionality to send the activity name, location, and details to the user's smartphone. The input is the detailed data of the activity suggestion, and the output is the notification to the user.

[0325] Step 6:

[0326] The user performs the suggested activity and enters feedback about the experience into the device. The user records their thoughts and evaluations through a feedback form. The input is the user's feedback, and the output is an updated feedback dataset.

[0327] Step 7:

[0328] The server updates the generative AI model using user feedback. The server analyzes the feedback data and applies it to the learning algorithm to improve the accuracy of the next activity suggestion. The input is the updated feedback dataset, and the output is the improved generative AI model.

[0329] (Application Example 1)

[0330] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0331] In modern society, many users want to spend their free time efficiently based on their interests and passions. However, finding suitable activities is not easy and can be time-consuming and require effort. Furthermore, if activity suggestions do not match individual interests and schedules, users are less likely to take action, which can diminish the user's convenience.

[0332] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0333] In this invention, the server includes an information processing device that acquires the user's social media data via a communication device, a means for generating an interest profile, and a schedule management device that identifies free time. This makes it possible to suggest optimal activities in real time based on the user's individual interests, current location, and free time.

[0334] An "information processing device" is a device that acquires a user's social media data, location information, and schedule information, and generates an interest profile by analyzing this data.

[0335] A "communication device" is a device that connects a user's device to an information processing device, enabling real-time transmission and reception of data.

[0336] A "schedule management device" is a device that works in conjunction with the user's calendar app to understand the user's schedule and identify free time.

[0337] A "suggestion device" is a device that selects and suggests the most suitable activities to the user based on the interest profile and schedule information generated by an information processing device.

[0338] A "notification device" is a device that informs users of suggested activities, and it uses the push notification function of smartphones.

[0339] A "user interface device" is a device used to explain proposed content to the user via the robot's voice function.

[0340] To implement this invention, it is necessary to build a system in which three entities—a server, a terminal, and a user—work in cooperation. First, the server uses an information processing device to acquire the user's social media data via a communication device. This data is used to generate an interest profile that reflects the user's interests, using text analysis and image recognition technologies. For example, Amazon's AWS Lambda or Google Cloud Vision API can be used as the server.

[0341] Next, a schedule management device installed in the terminal checks the user's calendar app to confirm their schedule and identify free time. The suggestion device combines the interest profile and schedule information, and selects and suggests the most suitable activities while considering the user's current location. This suggestion is notified to the user in real time via a notification device, and the terminal uses a user interface device to explain the suggestion verbally. Smartphones and consumer robots can be used as terminals.

[0342] As a concrete example, a scenario could be envisioned where a user receives a suggestion from their smartphone or a consumer robot during their free time after work, such as "attending a music event being held at a nearby cafe." The user can provide satisfaction feedback on this suggestion, and the results are sent to a server and used to update the generated AI model.

[0343] Example of a prompt:

[0344] "The user is interested in cafes and music, is currently located in Tokyo, and has free time from 18:00 to 20:00. Based on these conditions, please suggest appropriate activities."

[0345] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0346] Step 1:

[0347] The server retrieves user social media data from the communication device via an information processing device. The input is the user's social media data obtained through the API, and the output is the raw data stored in the database. This process, with the user's permission, uses APIs such as Instagram and Twitter to collect past posts and location information.

[0348] Step 2:

[0349] The server analyzes acquired social media data and generates an interest profile that reflects the user's interests. The input is the raw data obtained in step 1, and the output is profile information indicating the user's interest categories. At this stage, text analysis technology is used to extract themes from the posts, and image recognition technology is used to identify visual interests.

[0350] Step 3:

[0351] The device uses a scheduling device to retrieve the user's calendar information and identify free time. The input is schedule data obtained from the user's calendar app, and the output is a list of available free time. In this process, appointments are read via a calendar API, and free time is calculated while avoiding overlaps.

[0352] Step 4:

[0353] The server's suggestion device combines the user's interest profile, free time, and current location information to select the most suitable activity. The input is the data and location information obtained in steps 2 and 3, and the output is a list of activities to suggest. Here, a geographical database is referenced to select events of interest near the user.

[0354] Step 5:

[0355] The terminal's notification device informs the user of the suggested activity and provides an audio explanation using the user interface device. The input is the activity information selected in step 4, and the output is the audio and text notification delivered to the user. A consumer robot provides detailed information via voice using the smartphone's push notification function.

[0356] Step 6:

[0357] After the user completes an activity, they provide feedback via their device. The input is feedback data provided by the user, and the output is feedback information stored on the server. This feedback evaluates satisfaction with the notified activity and the accuracy of the suggestions.

[0358] Step 7:

[0359] The server updates the AI ​​model based on user feedback to improve the accuracy of the suggestions. The input is the feedback data obtained in step 6, and the output is the improved suggestion algorithm. At this stage, machine learning techniques are used to tune the suggestion algorithm and incorporate the results into the next suggestion.

[0360] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0361] This invention is a system that proposes activities optimized for the user by combining social media data analysis, schedule management, activity suggestion, and an emotion engine. This system includes an information processing device, a communication device, a schedule management device, a suggestion device, a notification device, and an emotion engine.

[0362] First, the device uses a communication device to retrieve posted data, images, and location information from the user's social media API. The data obtained at this time is collected from the user's public profile.

[0363] Next, the server's information processing unit uses a generative AI model to generate a user interest profile based on the acquired data. This process involves text analysis and image recognition to extract user interest categories such as "movies," "hiking," and "cooking."

[0364] Furthermore, an emotion engine is used to analyze the user's past posts and photos to form an emotion profile. For example, if the content of a post indicates emotions such as "fun" or "satisfied," this will be reflected in the emotion profile.

[0365] The device syncs with the user's calendar to identify upcoming appointments and free time between appointments. This free time data is sent to the server via a scheduling device.

[0366] Subsequently, the server's suggestion device integrates the generated interest profile, schedule data, current location information, and emotion profile to select an activity. In this selection process, for example, if the emotion profile indicates that the user is "depressed," the server will suggest an activity that will cheer them up, such as "watch your favorite comedy movie."

[0367] Suggestions are notified to the user in real time via the device's notification system. User reactions and satisfaction with the suggestions are also monitored, and flexible suggestions are made in real time based on their emotions.

[0368] After performing a suggested activity, the user inputs the results into the terminal and provides feedback. This feedback is used in the server's generative model to enable more accurate suggestions. For example, after watching a suggested movie, the user inputs their thoughts and evaluation into the system.

[0369] Through the above operations, the present invention can support users in making better use of their time by suggesting activities that match their interests and emotional state.

[0370] The following describes the processing flow.

[0371] Step 1:

[0372] The device, with the user's permission, collects user-generated content, images, and location information via social media APIs. This data includes posts from the past few weeks.

[0373] Step 2:

[0374] The device transmits the collected data to the server via a secure communication method. Encryption is applied during this process to protect the confidentiality of the data.

[0375] Step 3:

[0376] The server analyzes the received data using text analysis and image recognition technologies to generate a user's interest profile. For example, it might extract categories such as "sports," "cafes," and "travel."

[0377] Step 4:

[0378] The server uses an emotion engine to analyze posted data and determine the user's emotions from their past posts. This creates an emotion profile, such as positive or negative.

[0379] Step 5:

[0380] The device uses a calendar app to retrieve the user's schedule information and identify available time slots. This data is obtained directly from the user's device.

[0381] Step 6:

[0382] The device uses GPS functionality to obtain the user's current location. This allows for geographical optimization of suggested activities.

[0383] Step 7:

[0384] The server combines interest profiles, emotional profiles, schedule information, and current location information, and uses a generative AI model to select the most suitable activity for the user. For example, if the user is feeling stressed, it might suggest a nearby relaxing cafe.

[0385] Step 8:

[0386] The device notifies the user of suggested activities via push notifications or in-app screens. This notification includes the activity name, location, and detailed information.

[0387] Step 9:

[0388] The user performs the suggested activity and enters feedback about the experience into the device. This feedback includes satisfaction levels and impressions.

[0389] Step 10:

[0390] The device sends user feedback to the server, updating the generative model. This feedback information is then incorporated into the next proposal, improving its accuracy.

[0391] (Example 2)

[0392] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0393] In modern society, despite individuals having access to vast amounts of data from diverse sources, there is a challenge in effectively utilizing that data based on individual behaviors and emotions. Therefore, there is a need for systems that appropriately analyze users' interests and emotional states and suggest activities tailored to their individual circumstances.

[0394] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0395] In this invention, the server includes means for acquiring social network data via a communication device, means for generating interest characteristics using a generative artificial intelligence model, means for forming emotional characteristics using emotion analysis means, means for identifying the user's free time through schedule management, means for suggesting appropriate activities based on this information, and means for collecting feedback and updating the generative model. This makes it possible to automatically and dynamically provide suggestions optimized for the user's individual interests and emotions.

[0396] An "information processing device" is an electronic device used for data collection, analysis, and information generation.

[0397] A "communication device" is a device used to send and receive information over a network.

[0398] A "generative artificial intelligence model" is an algorithm that learns patterns based on data and generates predictions and suggestions.

[0399] "Interest characteristics" refer to a collection of information that indicates the themes and topics that users are interested in.

[0400] "Emotional analysis means" refers to a method or technology for analyzing user data and determining their emotional state.

[0401] "Emotional characteristics" refer to a collection of information that indicates a user's emotional state.

[0402] A "schedule management device" is a device or function used to manage a user's schedule and identify available time slots.

[0403] "Activities" refer to actions or behaviors suggested to users, and include recreational and hobby activities.

[0404] "Feedback" refers to the user's comments and evaluations of the suggestions made by the system.

[0405] "Methods for updating generative models" refer to methods of modifying a model to improve its generative capabilities based on newly obtained data.

[0406] This system modernizes behavior based on individual interests and emotions by collecting and analyzing user data and suggesting optimized activities. The system primarily relies on the exchange of information and data between the server, the terminal, and the user.

[0407] The server is the core of this system, using an information processing device to collect publicly available user data via APIs of social networking services. The collected data is analyzed using a generative artificial intelligence model to generate user interest characteristics. Interest characteristics are data that reflects the user's hobbies and interests, such as movies, cooking, and travel.

[0408] Furthermore, the server uses sentiment analysis tools to generate sentiment characteristics from the user's past posts. This process analyzes the images and comments posted by the user and classifies their emotions at the time into categories such as positive or negative. As a result, it becomes possible to understand what kind of emotions the user experiences in different situations.

[0409] The device primarily works in conjunction with the user's calendar application, identifying the user's free time through a schedule management system. This free time data is used as important basic information when making suggestions.

[0410] The proposed device integrates this analyzed data and uses a generative artificial intelligence model to select the optimal activity. This selection takes into account the user's interests, emotional characteristics, free time, and location. For example, if the device determines that the user has recently been feeling stressed, it can suggest watching a relaxing movie.

[0411] Selected activities are notified to the user in real time via the device's notification system. The user can review the suggested activities and choose whether or not to perform them. After performing the suggested activity, the user provides feedback on the activity to the system by entering it into the device. This feedback is provided to the server's generative model as new training material, leading to improved accuracy.

[0412] For example, if a user is unsure what to do on their day off, this system will suggest a movie for that day. After enjoying the movie, the user can provide feedback to the system, which will then generate even more personalized suggestions for the next time.

[0413] Example of a prompt:

[0414] "Analyze past posts to assess user interest in the movie category. Also, suggest relaxation activities that are recommended when users are experiencing strong negative emotions."

[0415] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0416] Step 1:

[0417] The device connects to the user's social networking service API using a communication device to retrieve posted data, images, and location information. The input is the user's public profile information, and the output is a dataset of these. This retrieved data is used in subsequent analysis steps.

[0418] Step 2:

[0419] The server uses an information processing device to input the data acquired in step 1 into a generative artificial intelligence model. The input data consists of user-submitted data and images, and the output is the user's interest characteristics. Specifically, the categories that the user is interested in are identified through text analysis and image recognition. For example, the analysis may extract interests such as "travel" and "cooking."

[0420] Step 3:

[0421] The server uses sentiment analysis tools to generate sentiment characteristics based on the data acquired in Step 1. The input is the user's posts and photos, and the output is sentiment characteristics. Using the sentiment analysis engine, it classifies emotions such as positive and negative from the text and images contained in the user's past posts and forms an sentiment profile.

[0422] Step 4:

[0423] The device works in conjunction with the user's calendar application to identify the user's free time through its schedule management function. The input is the user's schedule information, and the output is a list of free time slots. This allows the user to reserve time between appointments and notify the server of this.

[0424] Step 5:

[0425] The server's suggestion device integrates the interest characteristics generated in step 2, the emotional characteristics from step 3, and the free time data from step 4. The input is these analyzed datasets, and the output is the suggested activities. A generative AI model is used to select the most suitable activity based on this data. At this stage, for example, if the system determines that the user is tired, it will suggest relaxing activities.

[0426] Step 6:

[0427] The device's notification system will notify the user in real time of the activity suggested in step 5. The input is information about the suggested activity, and the output is a notification message to the user. The user can review the suggestion and decide on their next action.

[0428] Step 7:

[0429] After performing a suggested activity, the user enters feedback into the device. The input is the user's evaluation and impressions of the activity, and the output is feedback data sent to the server. The server updates its generative model based on this feedback to improve the accuracy of future suggestions.

[0430] (Application Example 2)

[0431] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0432] In modern society, users are often extremely busy, making it difficult to find suitable activities to effectively utilize their free time. Furthermore, personalized suggestions based on users' interests and emotions are rare, and the sheer volume of information can make selection difficult. This creates a challenge for users in leading fulfilling lives.

[0433] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0434] In this invention, the server includes an information processing means for acquiring the user's social media information via a communication means, a means for the information processing means to generate interest information based on the information, and an emotion analysis engine for analyzing the user's emotions and generating emotion data. This enables personalized activity suggestions based on the user's interests and emotions to be provided in real time.

[0435] "Information processing means" refers to a device that acquires a user's social media information through communication means and generates interest information based on that information.

[0436] "Communication means" refers to a system for exchanging information between information processing means and social media.

[0437] "Interest information" refers to data that represents a user's interests and preferences, and is generated based on information from social media.

[0438] An "emotion analysis engine" is a device that analyzes a user's emotions and generates emotional data based on that analysis.

[0439] "Emotional data" refers to data that represents a user's emotional state using numerical values ​​or categories, and is generated by an emotion analysis engine.

[0440] A "schedule management system" is a system for managing users' schedules and identifying available time slots.

[0441] An "activity suggestion device" is a device that suggests the most suitable activity to a user based on their interests, emotional data, and free time.

[0442] A "notification system" is a system for informing users about suggested activities.

[0443] "User geographical location information" refers to data about where the user is located on Earth.

[0444] A "generative model" is a model that has algorithms for making predictions and suggestions based on data, and is updated based on feedback from users.

[0445] "Opinions" refer to feedback and evaluations from users, which are used to update the generative model.

[0446] To realize this invention, the server first collects user data from social media via communication means using information processing means. This data includes posts, images, location information, etc. Next, the server's information processing means generates user interest information using a generative AI model based on the acquired data. Here, technologies such as natural language processing and image recognition are utilized, and specifically, software such as TextBlob and OpenCV may be used.

[0447] In parallel, the server uses an emotion analysis engine to analyze the user's emotions from past posts and images, and generates emotion data. This process reveals the user's emotional state in real time.

[0448] The device manages calendar data using a scheduling tool and identifies the user's free time. This information is sent to the server and used to suggest activities.

[0449] Next, the server's activity suggestion system integrates interest information, emotional data, free time, and geographical location information to select and suggest the most suitable activity for the user. In this process, for example, an activity that energizes the user is provided according to their current state.

[0450] The suggested activity is notified to the user in real time via the device's notification system. The user performs the suggested activity and sends feedback about their experience to the server. This feedback information is reflected in the generative model and used to improve the accuracy of future suggestions.

[0451] As a concrete example, suppose the user enjoys watching movies and is currently feeling down. In this case, the system can consider the emotional profile and suggest "watching a favorite comedy movie." An example of the suggestion is as follows:

[0452] "User's latest social media posts:

[0453] Cooking tags

[0454] Happy feelings

[0455] Current free time: 2 hours

[0456] Please propose cooking-based activities. Choose activities that can be done at home and use ingredients that are readily available.

[0457] In this way, users can make more fulfilling use of their time.

[0458] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0459] Step 1:

[0460] The server uses APIs from social media to retrieve user posts, images, and location information through information processing tools. It takes data from social media APIs as input and outputs raw data in digital format based on that data. This process involves accessing API endpoints and collecting necessary data using authentication tokens.

[0461] Step 2:

[0462] The server's information processing mechanism uses the data acquired in Step 1 to generate interest information using a generative AI model. The input is social media data, and the data is analyzed using natural language processing libraries (e.g., TextBlob) and image recognition tools (e.g., OpenCV) to output data indicating the user's interests. Specifically, it identifies a particular interest category through text analysis.

[0463] Step 3:

[0464] The server uses an emotion analysis engine to further analyze the acquired data and generate emotion data. The input is also social media data, and it identifies emotional states such as positive and negative through natural language processing and outputs the analysis results. Here, the content of the user's text and images is evaluated to form an emotion profile.

[0465] Step 4:

[0466] The device uses a schedule management tool to collect digital calendar data from calendar applications and other sources to identify the user's free time. The input is calendar data, and the analysis result outputs a list of free time slots. This operation includes searching for available slots using a calendar API.

[0467] Step 5:

[0468] The server's activity suggestion mechanism integrates interest information and sentiment data obtained in steps 2 and 3, and free time in step 4, to suggest the most suitable activity for the user. The inputs are interest information, sentiment data, and free time. Based on this data, it selects relevant activities and outputs suggestions. The resulting output is an activity list based on conditions pre-set in the prompt message.

[0469] Step 6:

[0470] The proposed activity is notified to the user in real time via the device's notification system. Activity information is entered into the notification system, and the notification result is output. Here, an instant message is sent to the user using the device's push notification service.

[0471] Step 7:

[0472] The user performs the activity notified by the server, inputs their opinion about the experience as feedback on their device, and sends it to the server. The feedback includes an evaluation of the activity, and data necessary for adjusting the generative model is output. This feedback information is used to optimize future suggestions.

[0473] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0474] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0475] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0476] [Third Embodiment]

[0477] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0478] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0479] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0480] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0481] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0482] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0483] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0484] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0485] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0486] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0487] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0488] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0489] The system for carrying out the present invention includes an information processing device, a communication device, a schedule management device, a suggestion device, and a notification device. By coordinating these, the system proposes the optimal activity to the user. The details are described below.

[0490] First, the device accesses the user's social media data via a communication device. With the user's permission, it uses social media APIs to retrieve past posts, images, and current location information. This enables real-time data collection.

[0491] Next, the information processing device on the server analyzes the acquired data and generates an interest profile that reflects the user's interests and preferences. Here, text analysis and image recognition technologies are used to extract relevant categories from the posted content. For example, "cafes," "travel," and "outdoors" are identified as the user's interests.

[0492] Next, the device retrieves the user's schedule information. It works in conjunction with the user's calendar app to identify periods of free time. At this time, it analyzes event information and reminders to calculate the exact time.

[0493] The server's suggestion system combines interest profiles and schedule information to propose the most suitable activities based on the user's current location. For example, if the user has two hours of free time, it might suggest visiting a popular nearby cafe. It can also prioritize places the user has never visited before, based on their past history.

[0494] The proposed content will be notified to the user in real time via the device's notification system. Using the smartphone's push notification function, the specific activity name, location, and summary will be provided to the user. This will result in a user-friendly interface.

[0495] After performing a suggested activity, users can input feedback on their device. This feedback includes information about satisfaction and the accuracy of the suggestion, and is used to improve the accuracy of subsequent suggestions. This feedback is also sent to the server and used to update the generative model. As a concrete example, a user might visit a suggested cafe and evaluate whether their experience met their expectations.

[0496] In this way, the present invention can effectively utilize users' free time and suggest activities that are best suited to their individual interests. This improves the quality of life for users and enables efficient use of their time.

[0497] The following describes the processing flow.

[0498] Step 1:

[0499] The device collects the user's past posts, images, and location information via social media APIs with the user's permission. This data includes the text content of posts, image metadata, and location information.

[0500] Step 2:

[0501] The device sends the collected social media data to the server. This transmission uses security protocols to protect the data.

[0502] Step 3:

[0503] The server uses a generative AI model to analyze the user's interests and preferences based on the data it receives. This extracts categories that indicate the user's interests (e.g., "cafes," "sports," "movies") and generates an interest profile.

[0504] Step 4:

[0505] The device connects with the user's calendar to retrieve schedule information. In this process, it analyzes the current and upcoming appointment times to identify free time.

[0506] Step 5:

[0507] The device obtains the user's current location information using GPS. This information is used to select activities to suggest during free time.

[0508] Step 6:

[0509] The server determines the most suitable activity for the user based on the generated interest profile, identified free time, and current location information. For example, if the user has two hours of free time, it might suggest visiting a nearby art gallery.

[0510] Step 7:

[0511] The device notifies the user of the suggested activity. This notification includes the activity's name, location, and detailed information. The information is displayed on the user's smartphone using push notifications or in-app notifications.

[0512] Step 8:

[0513] After the user completes the suggested activity, they enter feedback about their experience. This feedback includes satisfaction levels and an evaluation of the activity.

[0514] Step 9:

[0515] The device collects user feedback and sends that data to the server. This feedback is used to improve the quality of future suggestions.

[0516] Step 10:

[0517] The server updates its AI model based on the feedback it receives, improving the accuracy of future suggestions. This update enables more accurate activity suggestions based on the user's preferences and behavioral patterns.

[0518] (Example 1)

[0519] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0520] In modern society, there is a growing need for automated systems that offer suggestions for meaningfully utilizing limited free time based on an individual's interests and preferences. However, conventional systems have struggled to efficiently collect and analyze diverse user information and propose appropriate activities in real time. To address this challenge, it is necessary to provide a method that integrates user social media data, schedule information, and location information to provide highly accurate activity suggestions.

[0521] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0522] In this invention, the server includes means for generating an interest profile using text analysis and image recognition technology, means for suggesting activities based on the interest profile, free time, and location information, and means for updating the generated AI model using feedback. As a result, users can make meaningful use of their free time by being accurately suggested activities that match their individual interests.

[0523] A "terminal" is an information processing device that a user can directly operate and that has the function of receiving or transmitting external data via a communication device.

[0524] A "server" is a central information processing device that stores, processes, and manages various types of data over a network.

[0525] "Social media data" refers to digital data based on user activity on online platforms, including posts, comments, location information, and images.

[0526] An "interest profile" represents a user's preferences and interests, and is a collection of categorized information generated based on social media data.

[0527] A "calendar app" is software used to manage a user's schedule and events, allowing them to input and view schedule information.

[0528] "Push notifications" are information proactively sent from a server to a device, and are a means of communication that provides users with real-time information.

[0529] A "generative AI model" is an algorithm that utilizes artificial intelligence for data analysis and prediction, and is a model that is improved and updated through machine learning.

[0530] "Feedback" is evaluation information provided by users based on their experiences and impressions, and it is data that can be used to improve the system.

[0531] This invention provides an activity suggestion system that allows individuals to utilize their free time based on their interests. The system primarily consists of a terminal, server, communication device, schedule management device, suggestion device, and notification device. The detailed functions of each of these elements are described below.

[0532] A terminal is an information processing device operated by the user, such as a smartphone or tablet. The terminal accesses social media APIs via communication devices to obtain user-generated data and location information. This enables the terminal to collect data in real time.

[0533] The acquired data is sent to a server and analyzed by an information processing device. The server is equipped with software for text analysis and image recognition, specifically using natural language processing (NLP) and computer vision technologies. These technologies generate an interest profile that reflects the user's interests.

[0534] The schedule management device works in conjunction with the device's calendar app to analyze the user's schedule information. The device reads schedule data via an API to identify the user's free time and analyzes the importance of events.

[0535] The suggestion device resides on a server and proposes optimal activities based on the generated interest profile, free time, and current location information. This makes it possible to select the most suitable activity for the user. For example, if the user has two hours of free time, a visit to a nearby popular cafe might be suggested.

[0536] The suggested activities are notified to the user in real time via the device's notification system. These notifications are sent to the smartphone as push notifications, providing detailed activity information and resulting in a user-friendly interface.

[0537] For example, if a user frequently posts about cafes on social media, their interests profile will include "cafes," and they will be suggested to visit nearby cafes during their free time.

[0538] An example of a prompt for a generative AI model would be: "Based on the user's past social media posts, suggest an activity that would be suitable for them to perform. Consider the user's current location and calendar data to determine a place they should visit during their two-hour free time."

[0539] This invention enables the provision of activity suggestions based on the user's individual interests, thereby improving their quality of life and supporting efficient time management.

[0540] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0541] Step 1:

[0542] The device retrieves the user's social media data via a communication device. Specifically, the device sends a request to the social media API and, with the user's permission, downloads past posts, images, and location information. The input for this step is the API key and user ID, and the output is the set of retrieved social media data.

[0543] Step 2:

[0544] The server analyzes social media data sent from the terminal. The server applies text analysis and image recognition technologies to extract relevant categories from the user's posts and images. The input is social media data, and the output is an interest profile indicating the user's interests.

[0545] Step 3:

[0546] The device retrieves schedule information from the user's calendar app. The device reads the appointment data via an API and runs an algorithm to identify free time. The input is the calendar API key and user ID, and the output is a list of identified free time slots.

[0547] Step 4:

[0548] The server generates activities using a proposed device based on interest profiles and free time information. The server also considers location information and searches for relevant activities in the database. The inputs are interest profiles, free time, and location information, and the output is detailed information about the proposed activities.

[0549] Step 5:

[0550] The device notifies the user of activity suggestions received from the server. The device uses push notification functionality to send the activity name, location, and details to the user's smartphone. The input is the detailed data of the activity suggestion, and the output is the notification to the user.

[0551] Step 6:

[0552] The user performs the suggested activity and enters feedback about the experience into the device. The user records their thoughts and evaluations through a feedback form. The input is the user's feedback, and the output is an updated feedback dataset.

[0553] Step 7:

[0554] The server updates the generative AI model using user feedback. The server analyzes the feedback data and applies it to the learning algorithm to improve the accuracy of the next activity suggestion. The input is the updated feedback dataset, and the output is the improved generative AI model.

[0555] (Application Example 1)

[0556] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0557] In modern society, many users want to spend their free time efficiently based on their interests and passions. However, finding suitable activities is not easy and can be time-consuming and require effort. Furthermore, if activity suggestions do not match individual interests and schedules, users are less likely to take action, which can diminish the user's convenience.

[0558] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0559] In this invention, the server includes an information processing device that acquires the user's social media data via a communication device, a means for generating an interest profile, and a schedule management device that identifies free time. This makes it possible to suggest optimal activities in real time based on the user's individual interests, current location, and free time.

[0560] An "information processing device" is a device that acquires a user's social media data, location information, and schedule information, and generates an interest profile by analyzing this data.

[0561] A "communication device" is a device that connects a user's device to an information processing device, enabling real-time transmission and reception of data.

[0562] A "schedule management device" is a device that works in conjunction with the user's calendar app to understand the user's schedule and identify free time.

[0563] A "suggestion device" is a device that selects and suggests the most suitable activities to the user based on the interest profile and schedule information generated by an information processing device.

[0564] A "notification device" is a device that informs users of suggested activities, and it uses the push notification function of smartphones.

[0565] A "user interface device" is a device used to explain proposed content to the user via the robot's voice function.

[0566] To implement this invention, it is necessary to build a system in which three entities—a server, a terminal, and a user—work in cooperation. First, the server uses an information processing device to acquire the user's social media data via a communication device. This data is used to generate an interest profile that reflects the user's interests, using text analysis and image recognition technologies. For example, Amazon's AWS Lambda or Google Cloud Vision API can be used as the server.

[0567] Next, a schedule management device installed in the terminal checks the user's calendar app to confirm their schedule and identify free time. The suggestion device combines the interest profile and schedule information, and selects and suggests the most suitable activities while considering the user's current location. This suggestion is notified to the user in real time via a notification device, and the terminal uses a user interface device to explain the suggestion verbally. Smartphones and consumer robots can be used as terminals.

[0568] As a concrete example, a scenario could be envisioned where a user receives a suggestion from their smartphone or a consumer robot during their free time after work, such as "attending a music event being held at a nearby cafe." The user can provide satisfaction feedback on this suggestion, and the results are sent to a server and used to update the generated AI model.

[0569] Example of a prompt:

[0570] "The user is interested in cafes and music, is currently located in Tokyo, and has free time from 18:00 to 20:00. Based on these conditions, please suggest appropriate activities."

[0571] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0572] Step 1:

[0573] The server retrieves user social media data from the communication device via an information processing device. The input is the user's social media data obtained through the API, and the output is the raw data stored in the database. This process, with the user's permission, uses APIs such as Instagram and Twitter to collect past posts and location information.

[0574] Step 2:

[0575] The server analyzes acquired social media data and generates an interest profile that reflects the user's interests. The input is the raw data obtained in step 1, and the output is profile information indicating the user's interest categories. At this stage, text analysis technology is used to extract themes from the posts, and image recognition technology is used to identify visual interests.

[0576] Step 3:

[0577] The device uses a scheduling device to retrieve the user's calendar information and identify free time. The input is schedule data obtained from the user's calendar app, and the output is a list of available free time. In this process, appointments are read via a calendar API, and free time is calculated while avoiding overlaps.

[0578] Step 4:

[0579] The server's suggestion device combines the user's interest profile, free time, and current location information to select the most suitable activity. The input is the data and location information obtained in steps 2 and 3, and the output is a list of activities to suggest. Here, a geographical database is referenced to select events of interest near the user.

[0580] Step 5:

[0581] The terminal's notification device informs the user of the suggested activity and provides an audio explanation using the user interface device. The input is the activity information selected in step 4, and the output is the audio and text notification delivered to the user. A consumer robot provides detailed information via voice using the smartphone's push notification function.

[0582] Step 6:

[0583] After the user completes an activity, they provide feedback via their device. The input is feedback data provided by the user, and the output is feedback information stored on the server. This feedback evaluates satisfaction with the notified activity and the accuracy of the suggestions.

[0584] Step 7:

[0585] The server updates the AI ​​model based on user feedback to improve the accuracy of the suggestions. The input is the feedback data obtained in step 6, and the output is the improved suggestion algorithm. At this stage, machine learning techniques are used to tune the suggestion algorithm and incorporate the results into the next suggestion.

[0586] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0587] This invention is a system that proposes activities optimized for the user by combining social media data analysis, schedule management, activity suggestion, and an emotion engine. This system includes an information processing device, a communication device, a schedule management device, a suggestion device, a notification device, and an emotion engine.

[0588] First, the device uses a communication device to retrieve posted data, images, and location information from the user's social media API. The data obtained at this time is collected from the user's public profile.

[0589] Next, the server's information processing unit uses a generative AI model to generate a user interest profile based on the acquired data. This process involves text analysis and image recognition to extract user interest categories such as "movies," "hiking," and "cooking."

[0590] Furthermore, an emotion engine is used to analyze the user's past posts and photos to form an emotion profile. For example, if the content of a post indicates emotions such as "fun" or "satisfied," this will be reflected in the emotion profile.

[0591] The device syncs with the user's calendar to identify upcoming appointments and free time between appointments. This free time data is sent to the server via a scheduling device.

[0592] Subsequently, the server's suggestion device integrates the generated interest profile, schedule data, current location information, and emotion profile to select an activity. In this selection process, for example, if the emotion profile indicates that the user is "depressed," the server will suggest an activity that will cheer them up, such as "watch your favorite comedy movie."

[0593] Suggestions are notified to the user in real time via the device's notification system. User reactions and satisfaction with the suggestions are also monitored, and flexible suggestions are made in real time based on their emotions.

[0594] After performing a suggested activity, the user inputs the results into the terminal and provides feedback. This feedback is used in the server's generative model to enable more accurate suggestions. For example, after watching a suggested movie, the user inputs their thoughts and evaluation into the system.

[0595] Through the above operations, the present invention can support users in making better use of their time by suggesting activities that match their interests and emotional state.

[0596] The following describes the processing flow.

[0597] Step 1:

[0598] The device, with the user's permission, collects user-generated content, images, and location information via social media APIs. This data includes posts from the past few weeks.

[0599] Step 2:

[0600] The device transmits the collected data to the server via a secure communication method. Encryption is applied during this process to protect the confidentiality of the data.

[0601] Step 3:

[0602] The server analyzes the received data using text analysis and image recognition technologies to generate a user's interest profile. For example, it might extract categories such as "sports," "cafes," and "travel."

[0603] Step 4:

[0604] The server uses an emotion engine to analyze posted data and determine the user's emotions from their past posts. This creates an emotion profile, such as positive or negative.

[0605] Step 5:

[0606] The device uses a calendar app to retrieve the user's schedule information and identify available time slots. This data is obtained directly from the user's device.

[0607] Step 6:

[0608] The device uses GPS functionality to obtain the user's current location. This allows for geographical optimization of suggested activities.

[0609] Step 7:

[0610] The server combines interest profiles, emotional profiles, schedule information, and current location information, and uses a generative AI model to select the most suitable activity for the user. For example, if the user is feeling stressed, it might suggest a nearby relaxing cafe.

[0611] Step 8:

[0612] The device notifies the user of suggested activities via push notifications or in-app screens. This notification includes the activity name, location, and detailed information.

[0613] Step 9:

[0614] The user performs the suggested activity and enters feedback about the experience into the device. This feedback includes satisfaction levels and impressions.

[0615] Step 10:

[0616] The device sends user feedback to the server, updating the generative model. This feedback information is then incorporated into the next proposal, improving its accuracy.

[0617] (Example 2)

[0618] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0619] In modern society, despite individuals having access to vast amounts of data from diverse sources, there is a challenge in effectively utilizing that data based on individual behaviors and emotions. Therefore, there is a need for systems that appropriately analyze users' interests and emotional states and suggest activities tailored to their individual circumstances.

[0620] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0621] In this invention, the server includes means for acquiring social network data via a communication device, means for generating interest characteristics using a generative artificial intelligence model, means for forming emotional characteristics using emotion analysis means, means for identifying the user's free time through schedule management, means for suggesting appropriate activities based on this information, and means for collecting feedback and updating the generative model. This makes it possible to automatically and dynamically provide suggestions optimized for the user's individual interests and emotions.

[0622] An "information processing device" is an electronic device used for data collection, analysis, and information generation.

[0623] A "communication device" is a device used to send and receive information over a network.

[0624] A "generative artificial intelligence model" is an algorithm that learns patterns based on data and generates predictions and suggestions.

[0625] "Interest characteristics" refer to a collection of information that indicates the themes and topics that users are interested in.

[0626] "Emotional analysis means" refers to a method or technology for analyzing user data and determining their emotional state.

[0627] "Emotional characteristics" refer to a collection of information that indicates a user's emotional state.

[0628] A "schedule management device" is a device or function used to manage a user's schedule and identify available time slots.

[0629] "Activities" refer to actions or behaviors suggested to users, and include recreational and hobby activities.

[0630] "Feedback" refers to the user's comments and evaluations of the suggestions made by the system.

[0631] "Methods for updating generative models" refer to methods of modifying a model to improve its generative capabilities based on newly obtained data.

[0632] This system modernizes behavior based on individual interests and emotions by collecting and analyzing user data and suggesting optimized activities. The system primarily relies on the exchange of information and data between the server, the terminal, and the user.

[0633] The server is the core of this system, using an information processing device to collect publicly available user data via APIs of social networking services. The collected data is analyzed using a generative artificial intelligence model to generate user interest characteristics. Interest characteristics are data that reflects the user's hobbies and interests, such as movies, cooking, and travel.

[0634] Furthermore, the server uses sentiment analysis tools to generate sentiment characteristics from the user's past posts. This process analyzes the images and comments posted by the user and classifies their emotions at the time into categories such as positive or negative. As a result, it becomes possible to understand what kind of emotions the user experiences in different situations.

[0635] The device primarily works in conjunction with the user's calendar application, identifying the user's free time through a schedule management system. This free time data is used as important basic information when making suggestions.

[0636] The proposed device integrates this analyzed data and uses a generative artificial intelligence model to select the optimal activity. This selection takes into account the user's interests, emotional characteristics, free time, and location. For example, if the device determines that the user has recently been feeling stressed, it can suggest watching a relaxing movie.

[0637] Selected activities are notified to the user in real time via the device's notification system. The user can review the suggested activities and choose whether or not to perform them. After performing the suggested activity, the user provides feedback on the activity to the system by entering it into the device. This feedback is provided to the server's generative model as new training material, leading to improved accuracy.

[0638] For example, if a user is unsure what to do on their day off, this system will suggest a movie for that day. After enjoying the movie, the user can provide feedback to the system, which will then generate even more personalized suggestions for the next time.

[0639] Example of a prompt:

[0640] "Analyze past posts to assess user interest in the movie category. Also, suggest relaxation activities that are recommended when users are experiencing strong negative emotions."

[0641] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0642] Step 1:

[0643] The device connects to the user's social networking service API using a communication device to retrieve posted data, images, and location information. The input is the user's public profile information, and the output is a dataset of these. This retrieved data is used in subsequent analysis steps.

[0644] Step 2:

[0645] The server uses an information processing device to input the data acquired in step 1 into a generative artificial intelligence model. The input data consists of user-submitted data and images, and the output is the user's interest characteristics. Specifically, the categories that the user is interested in are identified through text analysis and image recognition. For example, the analysis may extract interests such as "travel" and "cooking."

[0646] Step 3:

[0647] The server uses sentiment analysis tools to generate sentiment characteristics based on the data acquired in Step 1. The input is the user's posts and photos, and the output is sentiment characteristics. Using the sentiment analysis engine, it classifies emotions such as positive and negative from the text and images contained in the user's past posts and forms an sentiment profile.

[0648] Step 4:

[0649] The device works in conjunction with the user's calendar application to identify the user's free time through its schedule management function. The input is the user's schedule information, and the output is a list of free time slots. This allows the user to reserve time between appointments and notify the server of this.

[0650] Step 5:

[0651] The server's suggestion device integrates the interest characteristics generated in step 2, the emotional characteristics from step 3, and the free time data from step 4. The input is these analyzed datasets, and the output is the suggested activities. A generative AI model is used to select the most suitable activity based on this data. At this stage, for example, if the system determines that the user is tired, it will suggest relaxing activities.

[0652] Step 6:

[0653] The device's notification system will notify the user in real time of the activity suggested in step 5. The input is information about the suggested activity, and the output is a notification message to the user. The user can review the suggestion and decide on their next action.

[0654] Step 7:

[0655] After performing a suggested activity, the user enters feedback into the device. The input is the user's evaluation and impressions of the activity, and the output is feedback data sent to the server. The server updates its generative model based on this feedback to improve the accuracy of future suggestions.

[0656] (Application Example 2)

[0657] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0658] In modern society, users are often extremely busy, making it difficult to find suitable activities to effectively utilize their free time. Furthermore, personalized suggestions based on users' interests and emotions are rare, and the sheer volume of information can make selection difficult. This creates a challenge for users in leading fulfilling lives.

[0659] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0660] In this invention, the server includes an information processing means for acquiring the user's social media information via a communication means, a means for the information processing means to generate interest information based on the information, and an emotion analysis engine for analyzing the user's emotions and generating emotion data. This enables personalized activity suggestions based on the user's interests and emotions to be provided in real time.

[0661] "Information processing means" refers to a device that acquires a user's social media information through communication means and generates interest information based on that information.

[0662] "Communication means" refers to a system for exchanging information between information processing means and social media.

[0663] "Interest information" refers to data that represents a user's interests and preferences, and is generated based on information from social media.

[0664] An "emotion analysis engine" is a device that analyzes a user's emotions and generates emotional data based on that analysis.

[0665] "Emotional data" refers to data that represents a user's emotional state using numerical values ​​or categories, and is generated by an emotion analysis engine.

[0666] A "schedule management system" is a system for managing users' schedules and identifying available time slots.

[0667] An "activity suggestion device" is a device that suggests the most suitable activity to a user based on their interests, emotional data, and free time.

[0668] A "notification system" is a system for informing users about suggested activities.

[0669] "User geographical location information" refers to data about where the user is located on Earth.

[0670] A "generative model" is a model that has algorithms for making predictions and suggestions based on data, and is updated based on feedback from users.

[0671] "Opinions" refer to feedback and evaluations from users, which are used to update the generative model.

[0672] To realize this invention, the server first collects user data from social media via communication means using information processing means. This data includes posts, images, location information, etc. Next, the server's information processing means generates user interest information using a generative AI model based on the acquired data. Here, technologies such as natural language processing and image recognition are utilized, and specifically, software such as TextBlob and OpenCV may be used.

[0673] In parallel, the server uses an emotion analysis engine to analyze the user's emotions from past posts and images, and generates emotion data. This process reveals the user's emotional state in real time.

[0674] The device manages calendar data using a scheduling tool and identifies the user's free time. This information is sent to the server and used to suggest activities.

[0675] Next, the server's activity suggestion system integrates interest information, emotional data, free time, and geographical location information to select and suggest the most suitable activity for the user. In this process, for example, an activity that energizes the user is provided according to their current state.

[0676] The suggested activity is notified to the user in real time via the device's notification system. The user performs the suggested activity and sends feedback about their experience to the server. This feedback information is reflected in the generative model and used to improve the accuracy of future suggestions.

[0677] As a concrete example, suppose the user enjoys watching movies and is currently feeling down. In this case, the system can consider the emotional profile and suggest "watching a favorite comedy movie." An example of the suggestion is as follows:

[0678] "User's latest social media posts:

[0679] Cooking tags

[0680] Happy feelings

[0681] Current free time: 2 hours

[0682] Please propose cooking-based activities. Choose activities that can be done at home and use ingredients that are readily available.

[0683] In this way, users can make more fulfilling use of their time.

[0684] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0685] Step 1:

[0686] The server uses APIs from social media to retrieve user posts, images, and location information through information processing tools. It takes data from social media APIs as input and outputs raw data in digital format based on that data. This process involves accessing API endpoints and collecting necessary data using authentication tokens.

[0687] Step 2:

[0688] The server's information processing mechanism uses the data acquired in Step 1 to generate interest information using a generative AI model. The input is social media data, and the data is analyzed using natural language processing libraries (e.g., TextBlob) and image recognition tools (e.g., OpenCV) to output data indicating the user's interests. Specifically, it identifies a particular interest category through text analysis.

[0689] Step 3:

[0690] The server uses an emotion analysis engine to further analyze the acquired data and generate emotion data. The input is also social media data, and it identifies emotional states such as positive and negative through natural language processing and outputs the analysis results. Here, the content of the user's text and images is evaluated to form an emotion profile.

[0691] Step 4:

[0692] The device uses a schedule management tool to collect digital calendar data from calendar applications and other sources to identify the user's free time. The input is calendar data, and the analysis result outputs a list of free time slots. This operation includes searching for available slots using a calendar API.

[0693] Step 5:

[0694] The server's activity suggestion mechanism integrates interest information and sentiment data obtained in steps 2 and 3, and free time in step 4, to suggest the most suitable activity for the user. The inputs are interest information, sentiment data, and free time. Based on this data, it selects relevant activities and outputs suggestions. The resulting output is an activity list based on conditions pre-set in the prompt message.

[0695] Step 6:

[0696] The proposed activity is notified to the user in real time via the device's notification system. Activity information is entered into the notification system, and the notification result is output. Here, an instant message is sent to the user using the device's push notification service.

[0697] Step 7:

[0698] The user performs the activity notified by the server, inputs their opinion about the experience as feedback on their device, and sends it to the server. The feedback includes an evaluation of the activity, and data necessary for adjusting the generative model is output. This feedback information is used to optimize future suggestions.

[0699] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0700] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0701] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0702] [Fourth Embodiment]

[0703] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0704] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0705] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0706] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0707] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0708] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0709] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0710] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0711] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0712] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0713] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0714] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0715] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0716] The system for carrying out the present invention includes an information processing device, a communication device, a schedule management device, a suggestion device, and a notification device. By coordinating these, the system proposes the optimal activity to the user. The details are described below.

[0717] First, the device accesses the user's social media data via a communication device. With the user's permission, it uses social media APIs to retrieve past posts, images, and current location information. This enables real-time data collection.

[0718] Next, the information processing device on the server analyzes the acquired data and generates an interest profile that reflects the user's interests and preferences. Here, text analysis and image recognition technologies are used to extract relevant categories from the posted content. For example, "cafes," "travel," and "outdoors" are identified as the user's interests.

[0719] Next, the device retrieves the user's schedule information. It works in conjunction with the user's calendar app to identify periods of free time. At this time, it analyzes event information and reminders to calculate the exact time.

[0720] The server's suggestion system combines interest profiles and schedule information to propose the most suitable activities based on the user's current location. For example, if the user has two hours of free time, it might suggest visiting a popular nearby cafe. It can also prioritize places the user has never visited before, based on their past history.

[0721] The proposed content will be notified to the user in real time via the device's notification system. Using the smartphone's push notification function, the specific activity name, location, and summary will be provided to the user. This will result in a user-friendly interface.

[0722] After performing a suggested activity, users can input feedback on their device. This feedback includes information about satisfaction and the accuracy of the suggestion, and is used to improve the accuracy of subsequent suggestions. This feedback is also sent to the server and used to update the generative model. As a concrete example, a user might visit a suggested cafe and evaluate whether their experience met their expectations.

[0723] In this way, the present invention can effectively utilize users' free time and suggest activities that are best suited to their individual interests. This improves the quality of life for users and enables efficient use of their time.

[0724] The following describes the processing flow.

[0725] Step 1:

[0726] The device collects the user's past posts, images, and location information via social media APIs with the user's permission. This data includes the text content of posts, image metadata, and location information.

[0727] Step 2:

[0728] The device sends the collected social media data to the server. This transmission uses security protocols to protect the data.

[0729] Step 3:

[0730] The server uses a generative AI model to analyze the user's interests and preferences based on the data it receives. This extracts categories that indicate the user's interests (e.g., "cafes," "sports," "movies") and generates an interest profile.

[0731] Step 4:

[0732] The device connects with the user's calendar to retrieve schedule information. In this process, it analyzes the current and upcoming appointment times to identify free time.

[0733] Step 5:

[0734] The device obtains the user's current location information using GPS. This information is used to select activities to suggest during free time.

[0735] Step 6:

[0736] The server determines the most suitable activity for the user based on the generated interest profile, identified free time, and current location information. For example, if the user has two hours of free time, it might suggest visiting a nearby art gallery.

[0737] Step 7:

[0738] The device notifies the user of the suggested activity. This notification includes the activity's name, location, and detailed information. The information is displayed on the user's smartphone using push notifications or in-app notifications.

[0739] Step 8:

[0740] After the user completes the suggested activity, they enter feedback about their experience. This feedback includes satisfaction levels and an evaluation of the activity.

[0741] Step 9:

[0742] The device collects user feedback and sends that data to the server. This feedback is used to improve the quality of future suggestions.

[0743] Step 10:

[0744] The server updates its AI model based on the feedback it receives, improving the accuracy of future suggestions. This update enables more accurate activity suggestions based on the user's preferences and behavioral patterns.

[0745] (Example 1)

[0746] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0747] In modern society, there is a growing need for automated systems that offer suggestions for meaningfully utilizing limited free time based on an individual's interests and preferences. However, conventional systems have struggled to efficiently collect and analyze diverse user information and propose appropriate activities in real time. To address this challenge, it is necessary to provide a method that integrates user social media data, schedule information, and location information to provide highly accurate activity suggestions.

[0748] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0749] In this invention, the server includes means for generating an interest profile using text analysis and image recognition technology, means for suggesting activities based on the interest profile, free time, and location information, and means for updating the generated AI model using feedback. As a result, users can make meaningful use of their free time by being accurately suggested activities that match their individual interests.

[0750] A "terminal" is an information processing device that a user can directly operate and that has the function of receiving or transmitting external data via a communication device.

[0751] A "server" is a central information processing device that stores, processes, and manages various types of data over a network.

[0752] "Social media data" refers to digital data based on user activity on online platforms, including posts, comments, location information, and images.

[0753] An "interest profile" represents a user's preferences and interests, and is a collection of categorized information generated based on social media data.

[0754] A "calendar app" is software used to manage a user's schedule and events, allowing them to input and view schedule information.

[0755] "Push notifications" are information proactively sent from a server to a device, and are a means of communication that provides users with real-time information.

[0756] A "generative AI model" is an algorithm that utilizes artificial intelligence for data analysis and prediction, and is a model that is improved and updated through machine learning.

[0757] "Feedback" is evaluation information provided by users based on their experiences and impressions, and it is data that can be used to improve the system.

[0758] This invention provides an activity suggestion system that allows individuals to utilize their free time based on their interests. The system primarily consists of a terminal, server, communication device, schedule management device, suggestion device, and notification device. The detailed functions of each of these elements are described below.

[0759] A terminal is an information processing device operated by the user, such as a smartphone or tablet. The terminal accesses social media APIs via communication devices to obtain user-generated data and location information. This enables the terminal to collect data in real time.

[0760] The acquired data is sent to a server and analyzed by an information processing device. The server is equipped with software for text analysis and image recognition, specifically using natural language processing (NLP) and computer vision technologies. These technologies generate an interest profile that reflects the user's interests.

[0761] The schedule management device works in conjunction with the device's calendar app to analyze the user's schedule information. The device reads schedule data via an API to identify the user's free time and analyzes the importance of events.

[0762] The suggestion device resides on a server and proposes optimal activities based on the generated interest profile, free time, and current location information. This makes it possible to select the most suitable activity for the user. For example, if the user has two hours of free time, a visit to a nearby popular cafe might be suggested.

[0763] The suggested activities are notified to the user in real time via the device's notification system. These notifications are sent to the smartphone as push notifications, providing detailed activity information and resulting in a user-friendly interface.

[0764] For example, if a user frequently posts about cafes on social media, their interests profile will include "cafes," and they will be suggested to visit nearby cafes during their free time.

[0765] An example of a prompt for a generative AI model would be: "Based on the user's past social media posts, suggest an activity that would be suitable for them to perform. Consider the user's current location and calendar data to determine a place they should visit during their two-hour free time."

[0766] This invention enables the provision of activity suggestions based on the user's individual interests, thereby improving their quality of life and supporting efficient time management.

[0767] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0768] Step 1:

[0769] The device retrieves the user's social media data via a communication device. Specifically, the device sends a request to the social media API and, with the user's permission, downloads past posts, images, and location information. The input for this step is the API key and user ID, and the output is the set of retrieved social media data.

[0770] Step 2:

[0771] The server analyzes social media data sent from the terminal. The server applies text analysis and image recognition technologies to extract relevant categories from the user's posts and images. The input is social media data, and the output is an interest profile indicating the user's interests.

[0772] Step 3:

[0773] The device retrieves schedule information from the user's calendar app. The device reads the appointment data via an API and runs an algorithm to identify free time. The input is the calendar API key and user ID, and the output is a list of identified free time slots.

[0774] Step 4:

[0775] The server generates activities using a proposed device based on interest profiles and free time information. The server also considers location information and searches for relevant activities in the database. The inputs are interest profiles, free time, and location information, and the output is detailed information about the proposed activities.

[0776] Step 5:

[0777] The device notifies the user of activity suggestions received from the server. The device uses push notification functionality to send the activity name, location, and details to the user's smartphone. The input is the detailed data of the activity suggestion, and the output is the notification to the user.

[0778] Step 6:

[0779] The user performs the suggested activity and enters feedback about the experience into the device. The user records their thoughts and evaluations through a feedback form. The input is the user's feedback, and the output is an updated feedback dataset.

[0780] Step 7:

[0781] The server updates the generative AI model using user feedback. The server analyzes the feedback data and applies it to the learning algorithm to improve the accuracy of the next activity suggestion. The input is the updated feedback dataset, and the output is the improved generative AI model.

[0782] (Application Example 1)

[0783] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0784] In modern society, many users want to spend their free time efficiently based on their interests and passions. However, finding suitable activities is not easy and can be time-consuming and require effort. Furthermore, if activity suggestions do not match individual interests and schedules, users are less likely to take action, which can diminish the user's convenience.

[0785] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0786] In this invention, the server includes an information processing device that acquires the user's social media data via a communication device, a means for generating an interest profile, and a schedule management device that identifies free time. This makes it possible to suggest optimal activities in real time based on the user's individual interests, current location, and free time.

[0787] An "information processing device" is a device that acquires a user's social media data, location information, and schedule information, and generates an interest profile by analyzing this data.

[0788] A "communication device" is a device that connects a user's device to an information processing device, enabling real-time transmission and reception of data.

[0789] A "schedule management device" is a device that works in conjunction with the user's calendar app to understand the user's schedule and identify free time.

[0790] A "suggestion device" is a device that selects and suggests the most suitable activities to the user based on the interest profile and schedule information generated by an information processing device.

[0791] A "notification device" is a device that informs users of suggested activities, and it uses the push notification function of smartphones.

[0792] A "user interface device" is a device used to explain proposed content to the user via the robot's voice function.

[0793] To implement this invention, it is necessary to build a system in which three entities—a server, a terminal, and a user—work in cooperation. First, the server uses an information processing device to acquire the user's social media data via a communication device. This data is used to generate an interest profile that reflects the user's interests, using text analysis and image recognition technologies. For example, Amazon's AWS Lambda or Google Cloud Vision API can be used as the server.

[0794] Next, a schedule management device installed in the terminal checks the user's calendar app to confirm their schedule and identify free time. The suggestion device combines the interest profile and schedule information, and selects and suggests the most suitable activities while considering the user's current location. This suggestion is notified to the user in real time via a notification device, and the terminal uses a user interface device to explain the suggestion verbally. Smartphones and consumer robots can be used as terminals.

[0795] As a concrete example, a scenario could be envisioned where a user receives a suggestion from their smartphone or a consumer robot during their free time after work, such as "attending a music event being held at a nearby cafe." The user can provide satisfaction feedback on this suggestion, and the results are sent to a server and used to update the generated AI model.

[0796] Example of a prompt:

[0797] "The user is interested in cafes and music, is currently located in Tokyo, and has free time from 18:00 to 20:00. Based on these conditions, please suggest appropriate activities."

[0798] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0799] Step 1:

[0800] The server retrieves user social media data from the communication device via an information processing device. The input is the user's social media data obtained through the API, and the output is the raw data stored in the database. This process, with the user's permission, uses APIs such as Instagram and Twitter to collect past posts and location information.

[0801] Step 2:

[0802] The server analyzes acquired social media data and generates an interest profile that reflects the user's interests. The input is the raw data obtained in step 1, and the output is profile information indicating the user's interest categories. At this stage, text analysis technology is used to extract themes from the posts, and image recognition technology is used to identify visual interests.

[0803] Step 3:

[0804] The device uses a scheduling device to retrieve the user's calendar information and identify free time. The input is schedule data obtained from the user's calendar app, and the output is a list of available free time. In this process, appointments are read via a calendar API, and free time is calculated while avoiding overlaps.

[0805] Step 4:

[0806] The server's suggestion device combines the user's interest profile, free time, and current location information to select the most suitable activity. The input is the data and location information obtained in steps 2 and 3, and the output is a list of activities to suggest. Here, a geographical database is referenced to select events of interest near the user.

[0807] Step 5:

[0808] The terminal's notification device informs the user of the suggested activity and provides an audio explanation using the user interface device. The input is the activity information selected in step 4, and the output is the audio and text notification delivered to the user. A consumer robot provides detailed information via voice using the smartphone's push notification function.

[0809] Step 6:

[0810] After the user completes an activity, they provide feedback via their device. The input is feedback data provided by the user, and the output is feedback information stored on the server. This feedback evaluates satisfaction with the notified activity and the accuracy of the suggestions.

[0811] Step 7:

[0812] The server updates the AI ​​model based on user feedback to improve the accuracy of the suggestions. The input is the feedback data obtained in step 6, and the output is the improved suggestion algorithm. At this stage, machine learning techniques are used to tune the suggestion algorithm and incorporate the results into the next suggestion.

[0813] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0814] This invention is a system that proposes activities optimized for the user by combining social media data analysis, schedule management, activity suggestion, and an emotion engine. This system includes an information processing device, a communication device, a schedule management device, a suggestion device, a notification device, and an emotion engine.

[0815] First, the device uses a communication device to retrieve posted data, images, and location information from the user's social media API. The data obtained at this time is collected from the user's public profile.

[0816] Next, the server's information processing unit uses a generative AI model to generate a user interest profile based on the acquired data. This process involves text analysis and image recognition to extract user interest categories such as "movies," "hiking," and "cooking."

[0817] Furthermore, an emotion engine is used to analyze the user's past posts and photos to form an emotion profile. For example, if the content of a post indicates emotions such as "fun" or "satisfied," this will be reflected in the emotion profile.

[0818] The device syncs with the user's calendar to identify upcoming appointments and free time between appointments. This free time data is sent to the server via a scheduling device.

[0819] Subsequently, the server's suggestion device integrates the generated interest profile, schedule data, current location information, and emotion profile to select an activity. In this selection process, for example, if the emotion profile indicates that the user is "depressed," the server will suggest an activity that will cheer them up, such as "watch your favorite comedy movie."

[0820] Suggestions are notified to the user in real time via the device's notification system. User reactions and satisfaction with the suggestions are also monitored, and flexible suggestions are made in real time based on their emotions.

[0821] After performing a suggested activity, the user inputs the results into the terminal and provides feedback. This feedback is used in the server's generative model to enable more accurate suggestions. For example, after watching a suggested movie, the user inputs their thoughts and evaluation into the system.

[0822] Through the above operations, the present invention can support users in making better use of their time by suggesting activities that match their interests and emotional state.

[0823] The following describes the processing flow.

[0824] Step 1:

[0825] The device, with the user's permission, collects user-generated content, images, and location information via social media APIs. This data includes posts from the past few weeks.

[0826] Step 2:

[0827] The device transmits the collected data to the server via a secure communication method. Encryption is applied during this process to protect the confidentiality of the data.

[0828] Step 3:

[0829] The server analyzes the received data using text analysis and image recognition technologies to generate a user's interest profile. For example, it might extract categories such as "sports," "cafes," and "travel."

[0830] Step 4:

[0831] The server uses an emotion engine to analyze posted data and determine the user's emotions from their past posts. This creates an emotion profile, such as positive or negative.

[0832] Step 5:

[0833] The device uses a calendar app to retrieve the user's schedule information and identify available time slots. This data is obtained directly from the user's device.

[0834] Step 6:

[0835] The device uses GPS functionality to obtain the user's current location. This allows for geographical optimization of suggested activities.

[0836] Step 7:

[0837] The server combines interest profiles, emotional profiles, schedule information, and current location information, and uses a generative AI model to select the most suitable activity for the user. For example, if the user is feeling stressed, it might suggest a nearby relaxing cafe.

[0838] Step 8:

[0839] The device notifies the user of suggested activities via push notifications or in-app screens. This notification includes the activity name, location, and detailed information.

[0840] Step 9:

[0841] The user performs the suggested activity and enters feedback about the experience into the device. This feedback includes satisfaction levels and impressions.

[0842] Step 10:

[0843] The device sends user feedback to the server, updating the generative model. This feedback information is then incorporated into the next proposal, improving its accuracy.

[0844] (Example 2)

[0845] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0846] In modern society, despite individuals having access to vast amounts of data from diverse sources, there is a challenge in effectively utilizing that data based on individual behaviors and emotions. Therefore, there is a need for systems that appropriately analyze users' interests and emotional states and suggest activities tailored to their individual circumstances.

[0847] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0848] In this invention, the server includes means for acquiring social network data via a communication device, means for generating interest characteristics using a generative artificial intelligence model, means for forming emotional characteristics using emotion analysis means, means for identifying the user's free time through schedule management, means for suggesting appropriate activities based on this information, and means for collecting feedback and updating the generative model. This makes it possible to automatically and dynamically provide suggestions optimized for the user's individual interests and emotions.

[0849] An "information processing device" is an electronic device used for data collection, analysis, and information generation.

[0850] A "communication device" is a device used to send and receive information over a network.

[0851] A "generative artificial intelligence model" is an algorithm that learns patterns based on data and generates predictions and suggestions.

[0852] "Interest characteristics" refer to a collection of information that indicates the themes and topics that users are interested in.

[0853] "Emotional analysis means" refers to a method or technology for analyzing user data and determining their emotional state.

[0854] "Emotional characteristics" refer to a collection of information that indicates a user's emotional state.

[0855] A "schedule management device" is a device or function used to manage a user's schedule and identify available time slots.

[0856] "Activities" refer to actions or behaviors suggested to users, and include recreational and hobby activities.

[0857] "Feedback" refers to the user's comments and evaluations of the suggestions made by the system.

[0858] "Methods for updating generative models" refer to methods of modifying a model to improve its generative capabilities based on newly obtained data.

[0859] This system modernizes behavior based on individual interests and emotions by collecting and analyzing user data and suggesting optimized activities. The system primarily relies on the exchange of information and data between the server, the terminal, and the user.

[0860] The server is the core of this system, using an information processing device to collect publicly available user data via APIs of social networking services. The collected data is analyzed using a generative artificial intelligence model to generate user interest characteristics. Interest characteristics are data that reflects the user's hobbies and interests, such as movies, cooking, and travel.

[0861] Furthermore, the server uses sentiment analysis tools to generate sentiment characteristics from the user's past posts. This process analyzes the images and comments posted by the user and classifies their emotions at the time into categories such as positive or negative. As a result, it becomes possible to understand what kind of emotions the user experiences in different situations.

[0862] The device primarily works in conjunction with the user's calendar application, identifying the user's free time through a schedule management system. This free time data is used as important basic information when making suggestions.

[0863] The proposed device integrates this analyzed data and uses a generative artificial intelligence model to select the optimal activity. This selection takes into account the user's interests, emotional characteristics, free time, and location. For example, if the device determines that the user has recently been feeling stressed, it can suggest watching a relaxing movie.

[0864] Selected activities are notified to the user in real time via the device's notification system. The user can review the suggested activities and choose whether or not to perform them. After performing the suggested activity, the user provides feedback on the activity to the system by entering it into the device. This feedback is provided to the server's generative model as new training material, leading to improved accuracy.

[0865] For example, if a user is unsure what to do on their day off, this system will suggest a movie for that day. After enjoying the movie, the user can provide feedback to the system, which will then generate even more personalized suggestions for the next time.

[0866] Example of a prompt:

[0867] "Analyze past posts to assess user interest in the movie category. Also, suggest relaxation activities that are recommended when users are experiencing strong negative emotions."

[0868] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0869] Step 1:

[0870] The device connects to the user's social networking service API using a communication device to retrieve posted data, images, and location information. The input is the user's public profile information, and the output is a dataset of these. This retrieved data is used in subsequent analysis steps.

[0871] Step 2:

[0872] The server uses an information processing device to input the data acquired in step 1 into a generative artificial intelligence model. The input data consists of user-submitted data and images, and the output is the user's interest characteristics. Specifically, the categories that the user is interested in are identified through text analysis and image recognition. For example, the analysis may extract interests such as "travel" and "cooking."

[0873] Step 3:

[0874] The server uses sentiment analysis tools to generate sentiment characteristics based on the data acquired in Step 1. The input is the user's posts and photos, and the output is sentiment characteristics. Using the sentiment analysis engine, it classifies emotions such as positive and negative from the text and images contained in the user's past posts and forms an sentiment profile.

[0875] Step 4:

[0876] The device works in conjunction with the user's calendar application to identify the user's free time through its schedule management function. The input is the user's schedule information, and the output is a list of free time slots. This allows the user to reserve time between appointments and notify the server of this.

[0877] Step 5:

[0878] The server's suggestion device integrates the interest characteristics generated in step 2, the emotional characteristics from step 3, and the free time data from step 4. The input is these analyzed datasets, and the output is the suggested activities. A generative AI model is used to select the most suitable activity based on this data. At this stage, for example, if the system determines that the user is tired, it will suggest relaxing activities.

[0879] Step 6:

[0880] The device's notification system will notify the user in real time of the activity suggested in step 5. The input is information about the suggested activity, and the output is a notification message to the user. The user can review the suggestion and decide on their next action.

[0881] Step 7:

[0882] After performing a suggested activity, the user enters feedback into the device. The input is the user's evaluation and impressions of the activity, and the output is feedback data sent to the server. The server updates its generative model based on this feedback to improve the accuracy of future suggestions.

[0883] (Application Example 2)

[0884] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0885] In modern society, users are often extremely busy, making it difficult to find suitable activities to effectively utilize their free time. Furthermore, personalized suggestions based on users' interests and emotions are rare, and the sheer volume of information can make selection difficult. This creates a challenge for users in leading fulfilling lives.

[0886] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0887] In this invention, the server includes an information processing means for acquiring the user's social media information via a communication means, a means for the information processing means to generate interest information based on the information, and an emotion analysis engine for analyzing the user's emotions and generating emotion data. This enables personalized activity suggestions based on the user's interests and emotions to be provided in real time.

[0888] "Information processing means" refers to a device that acquires a user's social media information through communication means and generates interest information based on that information.

[0889] "Communication means" refers to a system for exchanging information between information processing means and social media.

[0890] "Interest information" refers to data that represents a user's interests and preferences, and is generated based on information from social media.

[0891] An "emotion analysis engine" is a device that analyzes a user's emotions and generates emotional data based on that analysis.

[0892] "Emotional data" refers to data that represents a user's emotional state using numerical values ​​or categories, and is generated by an emotion analysis engine.

[0893] A "schedule management system" is a system for managing users' schedules and identifying available time slots.

[0894] An "activity suggestion device" is a device that suggests the most suitable activity to a user based on their interests, emotional data, and free time.

[0895] A "notification system" is a system for informing users about suggested activities.

[0896] "User geographical location information" refers to data about where the user is located on Earth.

[0897] A "generative model" is a model that has algorithms for making predictions and suggestions based on data, and is updated based on feedback from users.

[0898] "Opinions" refer to feedback and evaluations from users, which are used to update the generative model.

[0899] To realize this invention, the server first collects user data from social media via communication means using information processing means. This data includes posts, images, location information, etc. Next, the server's information processing means generates user interest information using a generative AI model based on the acquired data. Here, technologies such as natural language processing and image recognition are utilized, and specifically, software such as TextBlob and OpenCV may be used.

[0900] In parallel, the server uses an emotion analysis engine to analyze the user's emotions from past posts and images, and generates emotion data. This process reveals the user's emotional state in real time.

[0901] The device manages calendar data using a scheduling tool and identifies the user's free time. This information is sent to the server and used to suggest activities.

[0902] Next, the server's activity suggestion system integrates interest information, emotional data, free time, and geographical location information to select and suggest the most suitable activity for the user. In this process, for example, an activity that energizes the user is provided according to their current state.

[0903] The suggested activity is notified to the user in real time via the device's notification system. The user performs the suggested activity and sends feedback about their experience to the server. This feedback information is reflected in the generative model and used to improve the accuracy of future suggestions.

[0904] As a concrete example, suppose the user enjoys watching movies and is currently feeling down. In this case, the system can consider the emotional profile and suggest "watching a favorite comedy movie." An example of the suggestion is as follows:

[0905] "User's latest social media posts:

[0906] Cooking tags

[0907] Happy feelings

[0908] Current free time: 2 hours

[0909] Please propose cooking-based activities. Choose activities that can be done at home and use ingredients that are readily available.

[0910] In this way, users can make more fulfilling use of their time.

[0911] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0912] Step 1:

[0913] The server uses APIs from social media to retrieve user posts, images, and location information through information processing tools. It takes data from social media APIs as input and outputs raw data in digital format based on that data. This process involves accessing API endpoints and collecting necessary data using authentication tokens.

[0914] Step 2:

[0915] The server's information processing mechanism uses the data acquired in Step 1 to generate interest information using a generative AI model. The input is social media data, and the data is analyzed using natural language processing libraries (e.g., TextBlob) and image recognition tools (e.g., OpenCV) to output data indicating the user's interests. Specifically, it identifies a particular interest category through text analysis.

[0916] Step 3:

[0917] The server uses an emotion analysis engine to further analyze the acquired data and generate emotion data. The input is also social media data, and it identifies emotional states such as positive and negative through natural language processing and outputs the analysis results. Here, the content of the user's text and images is evaluated to form an emotion profile.

[0918] Step 4:

[0919] The device uses a schedule management tool to collect digital calendar data from calendar applications and other sources to identify the user's free time. The input is calendar data, and the analysis result outputs a list of free time slots. This operation includes searching for available slots using a calendar API.

[0920] Step 5:

[0921] The server's activity suggestion mechanism integrates interest information and sentiment data obtained in steps 2 and 3, and free time in step 4, to suggest the most suitable activity for the user. The inputs are interest information, sentiment data, and free time. Based on this data, it selects relevant activities and outputs suggestions. The resulting output is an activity list based on conditions pre-set in the prompt message.

[0922] Step 6:

[0923] The proposed activity is notified to the user in real time via the device's notification system. Activity information is entered into the notification system, and the notification result is output. Here, an instant message is sent to the user using the device's push notification service.

[0924] Step 7:

[0925] The user performs the activity notified by the server, inputs their opinion about the experience as feedback on their device, and sends it to the server. The feedback includes an evaluation of the activity, and data necessary for adjusting the generative model is output. This feedback information is used to optimize future suggestions.

[0926] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0927] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0928] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0929] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0930] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0931] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0932] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0933] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0934] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0935] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0936] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0937] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0938] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0939] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0940] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0941] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0942] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0943] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0944] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0945] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0946] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0947] The following is further disclosed regarding the embodiments described above.

[0948] (Claim 1)

[0949] An information processing device provides means for acquiring a user's social media data via a communication device,

[0950] The information processing device includes means for generating an interest profile based on the data,

[0951] The schedule management device provides a means for identifying the user's free time,

[0952] The proposed device includes means for suggesting activities based on the interest profile and free time,

[0953] The notification device includes means for notifying the user of the proposal,

[0954] A system that includes this.

[0955] (Claim 2)

[0956] The system according to claim 1, wherein the information processing device further includes means for acquiring the location information of a user and selecting an appropriate suggestion based on that location.

[0957] (Claim 3)

[0958] The system according to claim 1, wherein the proposed device further includes means for collecting feedback from users and updating the generative model.

[0959] "Example 1"

[0960] (Claim 1)

[0961] A means by which a terminal obtains the user's social media data via a communication device,

[0962] The server provides means for generating an interest profile based on the data using text analysis and image recognition technologies,

[0963] The device has a means of identifying the user's free time by linking with a calendar app,

[0964] The server provides means for suggesting activities based on the aforementioned interest profile, free time, and location information,

[0965] The terminal provides means for notifying the user of the aforementioned proposal via push notification,

[0966] A means for users to input feedback on the proposal,

[0967] The server has means for updating the generated AI model using the aforementioned feedback,

[0968] A system that includes this.

[0969] (Claim 2)

[0970] The system according to claim 1, further comprising means for a terminal to acquire the user's location information and for a server to select an appropriate suggestion based on that location.

[0971] (Claim 3)

[0972] The system according to claim 1, further comprising means for the proposed device to search a database for the user's past behavioral history and to preferentially suggest places the user has not yet visited.

[0973] "Application Example 1"

[0974] (Claim 1)

[0975] An information processing device provides means for acquiring a user's social media data via a communication device,

[0976] The information processing device includes means for generating an interest profile based on the data,

[0977] The schedule management device provides a means for identifying the user's free time,

[0978] The proposed device includes means for suggesting activities based on the interest profile and free time,

[0979] The notification device includes means for notifying the user of the proposal,

[0980] The user interface device includes a means for explaining the proposal verbally via a robot,

[0981] A system that includes this.

[0982] (Claim 2)

[0983] The system according to claim 1, wherein the information processing device further includes means for acquiring the location information of a user and selecting an appropriate suggestion based on that location.

[0984] (Claim 3)

[0985] The system according to claim 1, wherein the proposed device further includes means for collecting feedback from users and updating the generative model.

[0986] "Example 2 of combining an emotion engine"

[0987] (Claim 1)

[0988] The information processing device includes means for acquiring a user's social network data via a communication device,

[0989] The information processing device includes means for generating interest characteristics based on the data using a generative artificial intelligence model,

[0990] A means for analyzing the user's emotional state using emotion analysis tools and forming emotional characteristics,

[0991] The schedule management device provides a means for identifying the user's free time,

[0992] The proposed device includes means for suggesting activities based on the aforementioned interest characteristics, free time, current location information, and emotional characteristics,

[0993] The notification device includes means for notifying the user of the proposal,

[0994] A system that includes this.

[0995] (Claim 2)

[0996] The system according to claim 1, wherein the information processing device further includes means for acquiring spatial information of a user via a communication device and selecting an appropriate suggestion based on that space.

[0997] (Claim 3)

[0998] The system according to claim 1, wherein the proposed device further includes means for collecting feedback from users, updating generative models, and improving proposal accuracy.

[0999] "Application example 2 when combining with an emotional engine"

[1000] (Claim 1)

[1001] Information processing means include means for acquiring user social media information via communication means,

[1002] The information processing means includes means for generating interest information based on the information,

[1003] The emotion analysis engine provides a means for analyzing the user's emotions and generating emotion data,

[1004] The scheduling management method includes a means for identifying the user's free time,

[1005] The activity suggestion means includes means for suggesting activities based on the aforementioned interest information, emotional data, and free time,

[1006] The notification means includes a means for notifying the user of the aforementioned proposal,

[1007] A system that includes this.

[1008] (Claim 2)

[1009] The system according to claim 1, wherein the information processing means further includes means for acquiring the geographic location information of a user and selecting an appropriate suggestion based on that location.

[1010] (Claim 3)

[1011] The system according to claim 1, wherein the proposed means further includes means for collecting user feedback and updating the generation model. [Explanation of Symbols]

[1012] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. An information processing device provides means for acquiring a user's social media data via a communication device, The information processing device includes means for generating an interest profile based on the data, The schedule management device provides a means for identifying the user's free time, The proposed device includes means for suggesting activities based on the interest profile and free time, The notification device includes means for notifying the user of the proposal, A system that includes this.

2. The system according to claim 1, wherein the information processing device further includes means for acquiring the user's location information and selecting an appropriate suggestion based on that location.

3. The system according to claim 1, wherein the proposed device further includes means for collecting feedback from users and updating the generation model.

Images