system

Abstract

We provide the system. [Solution] A means of obtaining location information to obtain the user's current location, An interface for inputting user preferences and free time, Information processing means for suggesting the optimal activity based on the aforementioned location information and preferences, A means for presenting the proposal to the user, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Many users, including busy businesspersons and tourists, have difficulty finding meaningful activities suitable for themselves in a short time. To address such a situation, there is a need for means to quickly propose optimal activities according to the user's current location and individual preferences.

Means for Solving the Problems

[0005] This invention provides a system for effectively utilizing free time by including location information acquisition means for obtaining the user's current location, interface means for inputting the user's preferences and free time, information processing means for suggesting optimal activities based on location information and preferences, and display means for presenting the suggestions to the user. This system allows users to instantly discover activities based on their interests and make effective use of their time.

[0006] "Location information acquisition means" refers to technology or devices for identifying the user's current location and providing that information to the system.

[0007] An "interface means" is a function or technology that allows a user to input personal information such as preferences and free time, and to interact with the system.

[0008] "Information processing means" refers to technology or devices that perform processing to select and propose appropriate activities based on user input information and location information.

[0009] "Display means" refers to a device or technology for visually conveying information that the system proposes to the user.

[0010] "Information analysis means" refers to a technology or device that ranks potential activities using a user's past behavior history and feedback information.

[0011] "Learning methods" refer to technologies or functions that a system uses to incorporate user feedback and improve the accuracy of future activity suggestions. [Brief explanation of the drawing]

[0012] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Embodiments for Carrying Out the Invention

[0013] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0014] First, the language used in the following description will be explained.

[0015] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0016] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0017] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs, various parameters, and the like. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0018] In the following embodiments, the numbered communication I / F (Interface) is an interface including a communication processor, an antenna, and the like. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0019] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0020] [First Embodiment]

[0021] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0022] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0023] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0024] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0025] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0026] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0027] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0028] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0029] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0030] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0031] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0032] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0033] The system of this invention proposes optimal activities based on the user's current location, preferences, and available time, in order to effectively utilize the user's free time. The system mainly consists of three elements: a terminal, a server, and the user.

[0034] First, users register their preferences and interests using the device on which the application is installed. In addition to this registration information, users input their free time into the app, which provides the basic data necessary for the system to make suggestions.

[0035] The device uses its built-in GPS function to obtain the user's current location. This location information, along with the user's preferences and registered interests, is sent to the server. The server searches its database based on the received location information and preferences to collect information on activities and places of interest around the user's current location.

[0036] The server further analyzes the user's past behavior history and feedback information to determine the priority of suggested activities. Once the optimal activity is determined, that information is sent back to the device and displayed to the user.

[0037] As an example, let's consider a user who is in Tokyo. This user is interested in "cafe hopping" and has an hour of free time. The device sends its current location information to the server, and the server retrieves information about cafes around Tokyo. Taking into account past usage history and ratings from other users, the server generates a list of several cafes and returns it to the user. The user can then spend their time meaningfully by visiting a cafe of their choice from this list.

[0038] In this way, the system takes into account the user's current location and preferences, and proposes optimal and personalized activities, enabling smooth and efficient activity in a short amount of time.

[0039] The following describes the processing flow.

[0040] Step 1:

[0041] The user launches the application on their device and enters information about their free time and preferences. This allows the system to understand the user's search criteria.

[0042] Step 2:

[0043] The device uses GPS functionality to obtain the user's current location. This location information, along with the user's entered free time and preferences, is packaged and a request is generated to send to the server.

[0044] Step 3:

[0045] The server analyzes requests received from terminals and collects information about the user's surrounding activity based on their current location from databases and external sources.

[0046] Step 4:

[0047] The server filters the collected activity information to match the user's preferences and free time. Furthermore, it takes into account the user's past behavior history and feedback information to determine the priority of suggested activities.

[0048] Step 5:

[0049] The server ranks and lists the selected activity options, and adds detailed information (e.g., location, duration, reviews, etc.) to each included option.

[0050] Step 6:

[0051] The server sends the generated list of suggestions to the terminal. The terminal converts the suggestions into a viewable format and presents them to the user.

[0052] Step 7:

[0053] The user selects an activity from the presented list that interests them, views its details, and decides whether to proceed. After completing the selected activity, the user provides feedback to the system.

[0054] Step 8:

[0055] The server records the feedback it receives and uses it to perform a learning process to improve the accuracy of future activity suggestions.

[0056] (Example 1)

[0057] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0058] In modern society, effectively utilizing short periods of time amidst busy schedules is a crucial challenge. However, it is difficult for users to independently research and select the optimal activities to maximize their limited free time. In particular, there is a need for collecting activity options based on current location, utilizing past activity history, and improving the accuracy of suggestions, but there is a problem in that no efficient method exists to achieve this.

[0059] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0060] In this invention, the server includes data input means for inputting the user's preferences and available time; information analysis means for obtaining activity options near the current location from information storage means or external information sources and ranking the options based on the user's past behavior history and evaluation information; and learning means for improving the accuracy of information presentation by utilizing evaluation information collected from the user. This makes it possible to provide personalized and efficient activity suggestions so that users can make optimal use of their free time.

[0061] "Location information acquisition means" refers to technology that measures the user's current location and acquires that information in a format that the system can use.

[0062] A "data input method" is a technique for receiving input information such as user preferences and available time, and preparing the data necessary for the system to process it.

[0063] "Information processing means" refers to a technology that performs a series of processes to select and present the most suitable activity to the user based on acquired location information and user preferences.

[0064] "Information display means" refers to technology or devices that provide information determined by a system to users in an easily understandable format.

[0065] "Information analysis means" refers to technologies that analyze data related to the user's current location and past behavioral history, and then rank and provide information that is appropriate for the user.

[0066] A "learning method" is a technology or algorithm that utilizes evaluation information obtained from users to improve the accuracy of information suggestions in the future.

[0067] This invention consists of a system designed to effectively utilize users' free time. The system is primarily comprised of three elements: terminals, servers, and users.

[0068] First, a dedicated application is installed on the device. This application runs on mobile devices such as smartphones and tablets and provides a means of inputting data to register the user's preferences and interests. Users can input personal interests such as "cafe hopping" or "watching movies," and also register their free time via the device.

[0069] The device has a built-in GPS sensor and functions as a means of acquiring location information. It measures the user's current location in real time and sends the results to the server. This location information, along with the user's registered information (interests and free time), is sent to the server as a data package.

[0070] The server, as an information processing tool, analyzes received data and selects the most suitable activities and locations for the user. The server utilizes a generative AI model to rank suggestions from activity options obtained from databases and external information sources, based on the user's past behavior history and feedback. The AI ​​model analyzes this information and builds logic to provide the user with the most suitable options.

[0071] Ultimately, users receive suggestions through an information display system. For example, if a user sets up a café hopping activity, the server will list popular cafés around Shibuya, and this information will be displayed on the terminal screen. In this way, users can spend their free time efficiently.

[0072] For example, if a user enters "I'm currently in Shibuya, Tokyo. I have an hour to spare. I've recently become interested in cafe hopping. Can you recommend some places?", the server can return information about appropriate cafes. This system offers a new approach to maximizing the value of the user's time through location-based, personalized suggestions.

[0073] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0074] Step 1:

[0075] Users launch an application on their device and input their preferences, interests, and free time. Specifically, users select options such as "cafe hopping" or "watching sports" and set their available time. This input information is prepared as a dataset and used for subsequent processing.

[0076] Step 2:

[0077] The device uses its built-in GPS sensor to obtain the user's current location in real time. The location information is updated periodically, and the acquired geographic data is sent to the server. This process generates location coordinate data, which serves as input for the next step.

[0078] Step 3:

[0079] The server integrates location information received from the terminal with user registration information and searches the database. Using a generative AI model, it processes this input data to find relevant activities and locations. This process involves data discrimination, which extracts the most relevant options from the information stored in the database.

[0080] Step 4:

[0081] The server analyzes the user's past behavior history and feedback provided by other users. Using a generative AI model, it ranks activities based on this historical data. By analyzing the behavior history, the server outputs activity suggestions in the most optimal order for the user.

[0082] Step 5:

[0083] The server sends the ranked options to the terminal. A results data package is generated and becomes the output sent to the terminal.

[0084] Step 6:

[0085] The device presents information to the user based on the received data. A ranked list of options is provided on the display screen, from which the user selects their desired activity or location. At this point, the final suggestion to the user based on the prompt is complete, and the user's activity begins. In this step, the user's selections can be saved and used as a learning tool to improve the accuracy of future suggestions.

[0086] (Application Example 1)

[0087] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0088] The challenge lies in maximizing users' free time and suggesting optimal activities, stores, and events based on their preferences and real-time location information. Traditional technologies have struggled to fully utilize past behavioral history and feedback, making it difficult to provide users with truly meaningful information in a timely manner.

[0089] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0090] In this invention, the server includes location information acquisition means, an input device, and an information analysis device. This makes it possible to select activity options based on the user's current location and, in particular, perform real-time information processing that reflects purchase history and interests, thereby providing optimal choices and enriching the user experience.

[0091] "Location information acquisition means" refers to a technical method for obtaining the user's current location, and is a device that uses GPS or other location information technologies to obtain accurate location data.

[0092] An "input device" is an interface for users to input their preferences and free time into the system, and includes screens and sensors that can be operated on smartphones or other electronic devices.

[0093] An "information processing device" is a device that performs computational processing to generate optimal activities and suggestions based on location information and user preference data.

[0094] A "display device" is a device used to visually display information suggested to a user, and includes the screens of smartphones and tablets.

[0095] An "information analysis device" is a technology that analyzes activity options obtained from databases and external information sources, and ranks them while taking into account the user's behavioral history and feedback.

[0096] A "learning device" is a system that uses machine learning technology to improve the accuracy of its suggestions by utilizing feedback obtained from users.

[0097] A "real-time information processing device" is a technology that selects information about stores and events based on a user's purchase history and interests, and processes it immediately to notify them of discount information.

[0098] The system for realizing this invention provides usable activity and purchase information based on the user's location information, preferences, and past behavioral data. This system mainly consists of a terminal, a server, and the user.

[0099] The device uses GPS functionality to obtain the user's location information. It also has an interface for inputting the user's preferences and free time. This interface is implemented as a smartphone or tablet application. Through this application, users can input their interests and planned free time.

[0100] The servers operate on a cloud platform and perform data processing. Receiving location information and user preference data, the servers retrieve activity options from databases and external sources. The retrieved information is analyzed using machine learning models, and rankings are generated based on the user's past behavior and feedback. In particular, libraries such as Scikit-learn are utilized to improve the accuracy of suggestions by learning from feedback.

[0101] Furthermore, the server selects information on nearby stores and events in real time based on the user's purchase history and interests, and provides advantageous information, including discounts.

[0102] The display device visually presents the user with information provided and analyzed by the terminal. This allows the user to make appropriate activity choices and purchases based on their current location.

[0103] A concrete example is a user in a shopping mall in Tokyo who is guided by the system and receives real-time information about limited-time sales at stores they have previously visited.

[0104] An example of a prompt message might be: "Generate prompts that take into account the user's interests and past purchase data to provide the best possible shopping experience based on their current location."

[0105] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0106] Step 1:

[0107] The device obtains the user's current location using GPS functionality. This location information is sent to the server as basic data for subsequent information processing. The input is location data from GPS, and the output is the transmission of location information to the server.

[0108] Step 2:

[0109] Users input their preferences and free time through the terminal interface. This information forms the basis for the user's activity choices and is sent to the server. The input is the user's preferences and free time data, and the output is the transmission of this data to the server.

[0110] Step 3:

[0111] The server retrieves appropriate activity options from databases and external sources based on received location information and user preferences. This data is then analyzed to extract activities that match the user's surrounding environment. The input is location and preference data, and the output is a list of available activity options.

[0112] Step 4:

[0113] The server uses a machine learning model to rank activity options based on past behavior history and feedback. This process leverages the Scikit-learn library to calculate priority for each user. The input is past behavior history and feedback information, and the output is a list of ranked activity options.

[0114] Step 5:

[0115] The server utilizes a real-time information processing device to select store and event information based on the user's purchase history and interests, and generates special offers, including discount information. Inputs are purchase history and user interest data, while outputs are selected store / event information and discount information.

[0116] Step 6:

[0117] The terminal visually presents information received from the server to the user. This information is displayed in a concrete and easy-to-understand format to support the user's choices. The input is comprehensive information data from the server, and the output is the information displayed on the screen.

[0118] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0119] This invention relates to a system that recognizes a user's emotions and proposes the most suitable activity based on those emotions. This system can make suggestions considering the user's current location, preferences, free time, and emotional state.

[0120] The user launches an application on their device and inputs their preferences and available time. The device also uses built-in sensors, camera, and microphone to capture the user's facial expressions, voice tone, and, in some cases, heart rate, using emotion recognition techniques. This data forms the basis for determining the user's emotional state.

[0121] The device sends a request to the server, along with acquired sentiment data, including the user's current location, preferences, and free time. Based on the received data, the server searches for information on surrounding activities from its database and external sources, and generates optimal suggestions tailored to the user's sentiment. By also considering past sentiment history and feedback, more personalized suggestions can be made.

[0122] For example, consider a scenario where a user is feeling tired and wants to visit a cafe. The device detects the user's fatigue level from their facial expression and sends this emotional information to the server. The server prioritizes and ranks cafes with a relaxed atmosphere and suggests establishments that offer relaxing menus. The user can then rest at the suggested cafe and feel satisfied.

[0123] The system can also monitor the user's emotional state in real time and generate new suggestions as needed. In this way, the present invention makes the user's free time more productive and provides an experience optimized for their emotional state.

[0124] The following describes the processing flow.

[0125] Step 1:

[0126] The user launches an application on their device and enters their free time and preferred activities. This allows the system to collect basic information to understand the user's preferences.

[0127] Step 2:

[0128] The system uses the device's sensors (camera, microphone, heart rate sensor, etc.) to collect user emotional information. Facial recognition technology is used to analyze facial expressions and determine the user's emotional state.

[0129] Step 3:

[0130] The device sends a request to the server containing collected sentiment data, location information, preferences, and free time. This prepares the server to generate suggestions based on all the necessary data.

[0131] Step 4:

[0132] Based on the received location information and user input, the server searches for nearby activity options from its database and external sources.

[0133] Step 5:

[0134] The server analyzes the user's emotional state and takes feedback data and past emotional history into consideration to select the most appropriate activity. For example, it prioritizes suggesting relaxing spots to a tired user.

[0135] Step 6:

[0136] The server ranks and lists the selected activity plans and their details, and sends them to the terminal.

[0137] Step 7:

[0138] The device visually displays the received list of suggestions to the user in real time. The user can refer to this list and select activities that interest them.

[0139] Step 8:

[0140] The user performs a selected activity and periodically records changes in their emotional state on the device. This allows feedback to be automatically updated at the end of the activity and stored as learning data for the system.

[0141] Step 9:

[0142] The emotion engine uses the collected feedback to optimize the overall system performance in order to improve the accuracy of future suggestions.

[0143] (Example 2)

[0144] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0145] Conventional activity suggestion systems only provided suggestions based on the user's location and preferences, without considering the user's emotional state. Therefore, they were unable to provide optimal suggestions that reflected what the user was feeling, making it difficult to maximize user satisfaction.

[0146] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0147] In this invention, the server includes location information acquisition means for obtaining the user's current location, interface means for inputting the user's preferences and free time, and emotion recognition means for recognizing the user's facial expressions, tone of voice, and heart rate to determine their emotions. This makes it possible to suggest activities that take into account the user's current emotional state.

[0148] "Location information acquisition means" refers to devices or software that acquire geographical location data using technology to determine the user's current location.

[0149] "Interface means" refers to a mechanism that allows users to input information into a system, and includes screens and operating tools for inputting preferences and free time within an application.

[0150] "Emotion recognition means" refers to devices and technologies that determine a user's emotional state by analyzing their facial expressions, tone of voice, heart rate, etc.

[0151] "Information processing means" refers to devices and software used to perform calculations and data analysis based on acquired data in order to propose the most suitable activities for the user.

[0152] "Display means" refers to a display device or its function for visually presenting the proposed activity to the user.

[0153] "Information analysis means" refers to technology that analyzes information obtained from a database or external information source and performs rankings based on the user's past behavioral history, emotional history, and feedback.

[0154] "Learning methods" refer to machine learning algorithms and technologies used to improve the accuracy of information suggestions based on feedback obtained from users.

[0155] This system is an activity suggestion platform consisting of a server, terminals, and users. Users input their preferences and free time through an application on their terminals. The terminals are equipped with GPS modules and other location acquisition technologies, which allow the system to accurately determine the user's current location.

[0156] The device uses its built-in camera, microphone, and sensors to detect the user's facial expressions, voice tone, heart rate, and other data in real time, recognizing the user's emotions. This data is analyzed using an emotion recognition algorithm. The analyzed data is then sent to the server as information reflecting the user's current emotional state.

[0157] The server searches for relevant activity information from databases and external sources based on received location, preference, free time, and emotional state data. Generative AI models are used for information processing, which then suggests the most suitable activities to the user. This process also includes information analysis techniques that consider the user's past behavioral and emotional history, as well as feedback.

[0158] After a suggestion is generated, the server sends it to the terminal, which then visually presents the suggestion to the user via its display. This system has a learning function and can continuously improve the accuracy of its suggestions based on user feedback.

[0159] For example, if a user wants to visit a cafe because they feel "relaxed," the device provides this emotional state to the server. The server then prioritizes suggesting "quiet cafes." An example of a prompt to the generative AI model would be, "User's emotion is relaxed, and their activity is to visit a cafe."

[0160] In this way, the system aims to provide users with high-quality ways to spend their time by suggesting optimal activities that match their emotional state.

[0161] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0162] Step 1: User Input Step

[0163] The user launches an application on their device and enters their preferences and available time into the interface. This input might include phrases like "I want to go to a cafe" or "I have two hours free." The device temporarily stores this information and prepares it for use in the next step.

[0164] Step 2: Emotion Recognition Step

[0165] The device uses built-in sensors, a camera, and a microphone to collect the user's facial expressions, voice tone, and heart rate in real time, and analyzes them with an emotion recognition algorithm. The input data includes facial images, voice data, and heart rate, and based on this, it determines the emotional state, such as "fatigue" or "relaxation." The output result of the emotional state becomes data to be transmitted next.

[0166] Step 3: Data transmission step

[0167] The device sends acquired location information, user preferences, free time, and emotional state to the server. The input data consists of the emotional state and user selection information obtained in the previous step, and the output is a request composed of an aggregation of this data.

[0168] Step 4: Proposal Generation Step

[0169] Based on the received data, the server uses a generative AI model to generate activity suggestions best suited to the user. This includes location information, the user's emotional state, and past behavioral history, and the data processing outputs suggestions such as "a quiet cafe."

[0170] Step 5: Proposal Presentation Step

[0171] The server generates a suggestion and sends it to the terminal, which then displays it on the user's screen. The suggestion data is input and visually presented to the user. The user can then decide on their course of action based on this information.

[0172] Step 6: Feedback Collection and Analysis Step

[0173] After the user completes an action based on a suggestion, the device collects feedback from the user. This feedback becomes input data, and the server analyzes it using a learning algorithm to improve the accuracy of future suggestions. This improves the accuracy of future suggestions.

[0174] (Application Example 2)

[0175] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0176] Traditional systems that cannot suggest appropriate activities and content based on user emotions struggle to significantly improve the user experience. Therefore, there is a need for a system that can accurately assess each user's emotional state and provide optimal suggestions.

[0177] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0178] In this invention, the server includes emotion recognition means for determining the user's current emotional state, location information acquisition means for obtaining the user's current location, and interface means for inputting the user's preferences and free time. This makes it possible to suggest optimal activities and content based on the user's emotions.

[0179] "Emotion recognition means" refers to technology that analyzes a user's facial expressions, voice tone, etc., to determine the user's current emotional state.

[0180] "Location information acquisition means" refers to technology that identifies the user's current location and performs other data processing based on that information.

[0181] "Interface means" refers to the technology or device that users use to input their preferences and available time.

[0182] "Information processing means" refers to technologies that suggest optimal activities and content based on acquired location information, user preferences, and emotional state.

[0183] "Display means" refers to technologies that visually present proposed activities or content to users.

[0184] "Information analysis methods" refer to technologies that rank options obtained from databases and external information sources based on the user's past behavior history and feedback.

[0185] "Learning methods" refer to technologies that continuously improve the accuracy of information suggestions by utilizing feedback obtained from users.

[0186] This system is designed to suggest appropriate activities and content by focusing on recognizing the user's emotions. The hardware used includes devices such as smartphones and tablets, which have built-in cameras and microphones. Through these devices, the user's facial expressions and voice tone are analyzed by emotion recognition tools. This process utilizes image processing software such as OpenCV and machine learning models such as TENSORFLOW®.

[0187] Users can input their preferences and free time through the application interface. The user's current location is obtained through a location information acquisition system. This location information, preferences, and emotional state are all transmitted to an information processing system to suggest appropriate activities and content. This processing also references data and feedback from the user's past usage to create more personalized suggestions.

[0188] The suggestions are presented to the user through a display mechanism. For example, if the user is in the mood to relax, nearby relaxation spots or selected music will be suggested.

[0189] For example, if the system detects that the user is tired, it can suggest relaxing music and guide them to a pleasant cafe. An example of a prompt message when using a generative AI model to suggest music tailored to the user's situation might be, "Please list some music you recommend listening to when the user wants to relax." This allows users to enjoy activities suited to their state, ultimately improving their quality of life.

[0190] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0191] Step 1:

[0192] The device uses the user's camera and microphone to capture facial expressions and voice tone. The input consists of the acquired video and audio data, which is processed in real time using OpenCV or TensorFlow. An emotion recognition model analyzes this input data to determine the user's emotional state.

[0193] Step 2:

[0194] The device acquires the user's location information using GPS. This location information is sent to the server as data necessary for subsequent suggestion generation. The output is coordinate information indicating the user's current location.

[0195] Step 3:

[0196] Users input their preferences and available time through the terminal's interface. This provides input data representing the user's preferences and available time slots. The terminal then sends this data to the server for comparison with a database.

[0197] Step 4:

[0198] The server integrates the emotional state, location information, preferences, and free time obtained in steps 1 through 3 to process the information and suggest appropriate activities or content. Based on the input data, and taking into account past history and feedback data, it selects the optimal suggestion. As an output, it derives the suggestion that best suits the user.

[0199] Step 5:

[0200] The suggestions generated by the server are sent to the terminal. The terminal visually presents these suggestions to the user using a display device. The user can then select the options that interest them from the displayed choices.

[0201] Step 6:

[0202] User selections and feedback are sent back to the server and used as a learning tool to improve the accuracy of information suggestions. This allows the system to learn user preferences and make future suggestions more personalized.

[0203] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0204] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0205] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0206] [Second Embodiment]

[0207] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0208] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0209] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0210] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0211] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0212] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0213] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0214] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0215] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0216] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0217] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0218] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0219] The system of this invention proposes optimal activities based on the user's current location, preferences, and available time, in order to effectively utilize the user's free time. The system mainly consists of three elements: a terminal, a server, and the user.

[0220] First, users register their preferences and interests using the device on which the application is installed. In addition to this registration information, users input their free time into the app, which provides the basic data necessary for the system to make suggestions.

[0221] The device uses its built-in GPS function to obtain the user's current location. This location information, along with the user's preferences and registered interests, is sent to the server. The server searches its database based on the received location information and preferences to collect information on activities and places of interest around the user's current location.

[0222] The server further analyzes the user's past behavior history and feedback information to determine the priority of suggested activities. Once the optimal activity is determined, that information is sent back to the device and displayed to the user.

[0223] As an example, let's consider a user who is in Tokyo. This user is interested in "cafe hopping" and has an hour of free time. The device sends its current location information to the server, and the server retrieves information about cafes around Tokyo. Taking into account past usage history and ratings from other users, the server generates a list of several cafes and returns it to the user. The user can then spend their time meaningfully by visiting a cafe of their choice from this list.

[0224] In this way, the system takes into account the user's current location and preferences, and proposes optimal and personalized activities, enabling smooth and efficient activity in a short amount of time.

[0225] The following describes the processing flow.

[0226] Step 1:

[0227] The user launches the application on their device and enters information about their free time and preferences. This allows the system to understand the user's search criteria.

[0228] Step 2:

[0229] The device uses GPS functionality to obtain the user's current location. This location information, along with the user's entered free time and preferences, is packaged and a request is generated to send to the server.

[0230] Step 3:

[0231] The server analyzes requests received from terminals and collects information about the user's surrounding activity based on their current location from databases and external sources.

[0232] Step 4:

[0233] The server filters the collected activity information to match the user's preferences and free time. Furthermore, it takes into account the user's past behavior history and feedback information to determine the priority of suggested activities.

[0234] Step 5:

[0235] The server ranks and lists the selected activity options, and adds detailed information (e.g., location, duration, reviews, etc.) to each included option.

[0236] Step 6:

[0237] The server sends the generated list of suggestions to the terminal. The terminal converts the suggestions into a viewable format and presents them to the user.

[0238] Step 7:

[0239] The user selects an activity from the presented list that interests them, views its details, and decides whether to proceed. After completing the selected activity, the user provides feedback to the system.

[0240] Step 8:

[0241] The server records the feedback it receives and uses it to perform a learning process to improve the accuracy of future activity suggestions.

[0242] (Example 1)

[0243] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0244] In modern society, effectively utilizing short periods of time amidst busy schedules is a crucial challenge. However, it is difficult for users to independently research and select the optimal activities to maximize their limited free time. In particular, there is a need for collecting activity options based on current location, utilizing past activity history, and improving the accuracy of suggestions, but there is a problem in that no efficient method exists to achieve this.

[0245] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0246] In this invention, the server includes data input means for inputting the user's preferences and available time; information analysis means for obtaining activity options near the current location from information storage means or external information sources and ranking the options based on the user's past behavior history and evaluation information; and learning means for improving the accuracy of information presentation by utilizing evaluation information collected from the user. This makes it possible to provide personalized and efficient activity suggestions so that users can make optimal use of their free time.

[0247] "Location information acquisition means" refers to technology that measures the user's current location and acquires that information in a format that the system can use.

[0248] A "data input method" is a technique for receiving input information such as user preferences and available time, and preparing the data necessary for the system to process it.

[0249] "Information processing means" refers to a technology that performs a series of processes to select and present the most suitable activity to the user based on acquired location information and user preferences.

[0250] "Information display means" refers to technology or devices that provide information determined by a system to users in an easily understandable format.

[0251] "Information analysis means" refers to technologies that analyze data related to the user's current location and past behavioral history, and then rank and provide information that is appropriate for the user.

[0252] A "learning method" is a technology or algorithm that utilizes evaluation information obtained from users to improve the accuracy of information suggestions in the future.

[0253] This invention consists of a system designed to effectively utilize users' free time. The system is primarily comprised of three elements: terminals, servers, and users.

[0254] First, a dedicated application is installed on the device. This application runs on mobile devices such as smartphones and tablets and provides a means of inputting data to register the user's preferences and interests. Users can input personal interests such as "cafe hopping" or "watching movies," and also register their free time via the device.

[0255] The device has a built-in GPS sensor and functions as a means of acquiring location information. It measures the user's current location in real time and sends the results to the server. This location information, along with the user's registered information (interests and free time), is sent to the server as a data package.

[0256] The server, as an information processing tool, analyzes received data and selects the most suitable activities and locations for the user. The server utilizes a generative AI model to rank suggestions from activity options obtained from databases and external information sources, based on the user's past behavior history and feedback. The AI ​​model analyzes this information and builds logic to provide the user with the most suitable options.

[0257] Ultimately, users receive suggestions through an information display system. For example, if a user sets up a café hopping activity, the server will list popular cafés around Shibuya, and this information will be displayed on the terminal screen. In this way, users can spend their free time efficiently.

[0258] For example, if a user enters "I'm currently in Shibuya, Tokyo. I have an hour to spare. I've recently become interested in cafe hopping. Can you recommend some places?", the server can return information about appropriate cafes. This system offers a new approach to maximizing the value of the user's time through location-based, personalized suggestions.

[0259] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0260] Step 1:

[0261] Users launch an application on their device and input their preferences, interests, and free time. Specifically, users select options such as "cafe hopping" or "watching sports" and set their available time. This input information is prepared as a dataset and used for subsequent processing.

[0262] Step 2:

[0263] The device uses its built-in GPS sensor to obtain the user's current location in real time. The location information is updated periodically, and the acquired geographic data is sent to the server. This process generates location coordinate data, which serves as input for the next step.

[0264] Step 3:

[0265] The server integrates location information received from the terminal with user registration information and searches the database. Using a generative AI model, it processes this input data to find relevant activities and locations. This process involves data discrimination, which extracts the most relevant options from the information stored in the database.

[0266] Step 4:

[0267] The server analyzes the user's past behavior history and feedback provided by other users. Using a generative AI model, it ranks activities based on this historical data. By analyzing the behavior history, the server outputs activity suggestions in the most optimal order for the user.

[0268] Step 5:

[0269] The server sends the ranked options to the terminal. A results data package is generated and becomes the output sent to the terminal.

[0270] Step 6:

[0271] The device presents information to the user based on the received data. A ranked list of options is provided on the display screen, from which the user selects their desired activity or location. At this point, the final suggestion to the user based on the prompt is complete, and the user's activity begins. In this step, the user's selections can be saved and used as a learning tool to improve the accuracy of future suggestions.

[0272] (Application Example 1)

[0273] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0274] The challenge lies in maximizing users' free time and suggesting optimal activities, stores, and events based on their preferences and real-time location information. Traditional technologies have struggled to fully utilize past behavioral history and feedback, making it difficult to provide users with truly meaningful information in a timely manner.

[0275] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0276] In this invention, the server includes location information acquisition means, an input device, and an information analysis device. This makes it possible to select activity options based on the user's current location and, in particular, perform real-time information processing that reflects purchase history and interests, thereby providing optimal choices and enriching the user experience.

[0277] "Location information acquisition means" refers to a technical method for obtaining the user's current location, and is a device that uses GPS or other location information technologies to obtain accurate location data.

[0278] An "input device" is an interface for users to input their preferences and free time into the system, and includes screens and sensors that can be operated on smartphones or other electronic devices.

[0279] An "information processing device" is a device that performs computational processing to generate optimal activities and suggestions based on location information and user preference data.

[0280] A "display device" is a device used to visually display information suggested to a user, and includes the screens of smartphones and tablets.

[0281] An "information analysis device" is a technology that analyzes activity options obtained from databases and external information sources, and ranks them while taking into account the user's behavioral history and feedback.

[0282] The "learning device" is a system that uses machine learning technology to improve the accuracy of proposals by leveraging feedback obtained from users.

[0283] The "real-time information processing device" is a technology that performs immediate processing to select store and event information and notify discount information based on the user's purchase history and interests.

[0284] The system for realizing this invention provides available activity and purchase information based on the user's location information, preferences, and past behavior data. This system is mainly composed of a terminal, a server, and a user.

[0285] The terminal uses a GPS function to obtain the user's location information. It also has an interface for inputting the user's preferences and free time. This interface is implemented as an application on a smartphone or tablet. The user can input their interests and scheduled free time through this application.

[0286] The server is operated on a cloud platform and performs data processing. The server that receives location information and user preference data obtains activity options from a database or external information sources. The obtained information is analyzed using a machine learning model, and a ranking based on the user's past behavior and feedback is implemented. In particular, libraries such as Scikit-learn are utilized to learn from feedback and improve the accuracy of proposals.

[0287] In addition, the server selects nearby store information and event information in real time based on the user's purchase history and interests, and provides beneficial information including discount information.

[0288] The display device is provided by the terminal and visually presents the analyzed information to the user. As a result, the user can make appropriate activity selections and purchase activities based on their current location.

[0289] A concrete example is a user in a shopping mall in Tokyo who is guided by the system and receives real-time information about limited-time sales at stores they have previously visited.

[0290] An example of a prompt message might be: "Generate prompts that take into account the user's interests and past purchase data to provide the best possible shopping experience based on their current location."

[0291] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0292] Step 1:

[0293] The device obtains the user's current location using GPS functionality. This location information is sent to the server as basic data for subsequent information processing. The input is location data from GPS, and the output is the transmission of location information to the server.

[0294] Step 2:

[0295] Users input their preferences and free time through the terminal interface. This information forms the basis for the user's activity choices and is sent to the server. The input is the user's preferences and free time data, and the output is the transmission of this data to the server.

[0296] Step 3:

[0297] The server retrieves appropriate activity options from databases and external sources based on received location information and user preferences. This data is then analyzed to extract activities that match the user's surrounding environment. The input is location and preference data, and the output is a list of available activity options.

[0298] Step 4:

[0299] The server uses a machine learning model to rank activity options based on past behavior history and feedback. This process leverages the Scikit-learn library to calculate priority for each user. The input is past behavior history and feedback information, and the output is a list of ranked activity options.

[0300] Step 5:

[0301] The server utilizes a real-time information processing device to select store and event information based on the user's purchase history and interests, and generates special offers, including discount information. Inputs are purchase history and user interest data, while outputs are selected store / event information and discount information.

[0302] Step 6:

[0303] The terminal visually presents information received from the server to the user. This information is displayed in a concrete and easy-to-understand format to support the user's choices. The input is comprehensive information data from the server, and the output is the information displayed on the screen.

[0304] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0305] This invention relates to a system that recognizes a user's emotions and proposes the most suitable activity based on those emotions. This system can make suggestions considering the user's current location, preferences, free time, and emotional state.

[0306] The user launches an application on their device and inputs their preferences and available time. The device also uses built-in sensors, camera, and microphone to capture the user's facial expressions, voice tone, and, in some cases, heart rate, using emotion recognition techniques. This data forms the basis for determining the user's emotional state.

[0307] The terminal sends a request including the user's current location, preferences, and free time, along with the acquired emotion data, to the server. Based on the received data, the server searches for surrounding activity information from a database or external information sources and generates an optimal proposal according to the user's emotion. At that time, by also considering the past emotion history and feedback, a more personalized proposal becomes possible.

[0308] As an example, consider the case where a user wishes to visit a café while in a tired state. The terminal senses the fatigue from the user's expression and sends the emotion information to the server. The server ranks cafes with a relaxed atmosphere as a priority and proposes stores that offer relaxing menus. The user can take a break at the proposed café and obtain a sense of satisfaction.

[0309] The system can also monitor the user's emotional state in real time and generate new proposals as needed. In this way, the present invention makes the user's free time of high quality and provides an experience optimized for the emotional state.

[0310] The following describes the processing flow.

[0311] Step 1:

[0312] The user launches the application on the terminal and enters their free time and preferred activities. Thereby, the system collects basic information for understanding the user's wishes.

[0313] Step 2:

[0314] Using the device sensors (such as cameras, microphones, heart rate sensors, etc.) of the terminal, collect the user's emotion information. Analyze the facial expression using facial recognition technology to determine the user's emotional state.

[0315] Step 3:

[0316] The device sends a request to the server containing collected sentiment data, location information, preferences, and free time. This prepares the server to generate suggestions based on all the necessary data.

[0317] Step 4:

[0318] Based on the received location information and user input, the server searches for nearby activity options from its database and external sources.

[0319] Step 5:

[0320] The server analyzes the user's emotional state and takes feedback data and past emotional history into consideration to select the most appropriate activity. For example, it prioritizes suggesting relaxing spots to a tired user.

[0321] Step 6:

[0322] The server ranks and lists the selected activity plans and their details, and sends them to the terminal.

[0323] Step 7:

[0324] The device visually displays the received list of suggestions to the user in real time. The user can refer to this list and select activities that interest them.

[0325] Step 8:

[0326] The user performs a selected activity and periodically records changes in their emotional state on the device. This allows feedback to be automatically updated at the end of the activity and stored as learning data for the system.

[0327] Step 9:

[0328] The emotion engine uses the collected feedback to optimize the overall system performance in order to improve the accuracy of future suggestions.

[0329] (Example 2)

[0330] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0331] Conventional activity suggestion systems only provided suggestions based on the user's location and preferences, without considering the user's emotional state. Therefore, they were unable to provide optimal suggestions that reflected what the user was feeling, making it difficult to maximize user satisfaction.

[0332] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0333] In this invention, the server includes location information acquisition means for obtaining the user's current location, interface means for inputting the user's preferences and free time, and emotion recognition means for recognizing the user's facial expressions, tone of voice, and heart rate to determine their emotions. This makes it possible to suggest activities that take into account the user's current emotional state.

[0334] "Location information acquisition means" refers to devices or software that acquire geographical location data using technology to determine the user's current location.

[0335] "Interface means" refers to a mechanism that allows users to input information into a system, and includes screens and operating tools for inputting preferences and free time within an application.

[0336] "Emotion recognition means" refers to devices and technologies that determine a user's emotional state by analyzing their facial expressions, tone of voice, heart rate, etc.

[0337] "Information processing means" refers to devices and software used to perform calculations and data analysis based on acquired data in order to propose the most suitable activities for the user.

[0338] "Display means" refers to a display device or its function for visually presenting the proposed activity to the user.

[0339] "Information analysis means" refers to technology that analyzes information obtained from a database or external information source and performs rankings based on the user's past behavioral history, emotional history, and feedback.

[0340] "Learning methods" refer to machine learning algorithms and technologies used to improve the accuracy of information suggestions based on feedback obtained from users.

[0341] This system is an activity suggestion platform consisting of a server, terminals, and users. Users input their preferences and free time through an application on their terminals. The terminals are equipped with GPS modules and other location acquisition technologies, which allow the system to accurately determine the user's current location.

[0342] The device uses its built-in camera, microphone, and sensors to detect the user's facial expressions, voice tone, heart rate, and other data in real time, recognizing the user's emotions. This data is analyzed using an emotion recognition algorithm. The analyzed data is then sent to the server as information reflecting the user's current emotional state.

[0343] The server searches for relevant activity information from databases and external sources based on received location, preference, free time, and emotional state data. Generative AI models are used for information processing, which then suggests the most suitable activities to the user. This process also includes information analysis techniques that consider the user's past behavioral and emotional history, as well as feedback.

[0344] After a suggestion is generated, the server sends it to the terminal, which then visually presents the suggestion to the user via its display. This system has a learning function and can continuously improve the accuracy of its suggestions based on user feedback.

[0345] For example, if a user wants to visit a cafe because they feel "relaxed," the device provides this emotional state to the server. The server then prioritizes suggesting "quiet cafes." An example of a prompt to the generative AI model would be, "User's emotion is relaxed, and their activity is to visit a cafe."

[0346] In this way, the system aims to provide users with high-quality ways to spend their time by suggesting optimal activities that match their emotional state.

[0347] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0348] Step 1: User Input Step

[0349] The user launches an application on their device and enters their preferences and available time into the interface. This input might include phrases like "I want to go to a cafe" or "I have two hours free." The device temporarily stores this information and prepares it for use in the next step.

[0350] Step 2: Emotion Recognition Step

[0351] The device uses built-in sensors, a camera, and a microphone to collect the user's facial expressions, voice tone, and heart rate in real time, and analyzes them with an emotion recognition algorithm. The input data includes facial images, voice data, and heart rate, and based on this, it determines the emotional state, such as "fatigue" or "relaxation." The output result of the emotional state becomes data to be transmitted next.

[0352] Step 3: Data transmission step

[0353] The device sends acquired location information, user preferences, free time, and emotional state to the server. The input data consists of the emotional state and user selection information obtained in the previous step, and the output is a request composed of an aggregation of this data.

[0354] Step 4: Proposal Generation Step

[0355] Based on the received data, the server uses a generative AI model to generate activity suggestions best suited to the user. This includes location information, the user's emotional state, and past behavioral history, and the data processing outputs suggestions such as "a quiet cafe."

[0356] Step 5: Proposal Presentation Step

[0357] The server generates a suggestion and sends it to the terminal, which then displays it on the user's screen. The suggestion data is input and visually presented to the user. The user can then decide on their course of action based on this information.

[0358] Step 6: Feedback Collection and Analysis Step

[0359] After the user completes an action based on a suggestion, the device collects feedback from the user. This feedback becomes input data, and the server analyzes it using a learning algorithm to improve the accuracy of future suggestions. This improves the accuracy of future suggestions.

[0360] (Application Example 2)

[0361] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0362] Traditional systems that cannot suggest appropriate activities and content based on user emotions struggle to significantly improve the user experience. Therefore, there is a need for a system that can accurately assess each user's emotional state and provide optimal suggestions.

[0363] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0364] In this invention, the server includes emotion recognition means for determining the user's current emotional state, location information acquisition means for obtaining the user's current location, and interface means for inputting the user's preferences and free time. This makes it possible to suggest optimal activities and content based on the user's emotions.

[0365] "Emotion recognition means" refers to technology that analyzes a user's facial expressions, voice tone, etc., to determine the user's current emotional state.

[0366] "Location information acquisition means" refers to technology that identifies the user's current location and performs other data processing based on that information.

[0367] "Interface means" refers to the technology or device that users use to input their preferences and available time.

[0368] "Information processing means" refers to technologies that suggest optimal activities and content based on acquired location information, user preferences, and emotional state.

[0369] "Display means" refers to technologies that visually present proposed activities or content to users.

[0370] "Information analysis methods" refer to technologies that rank options obtained from databases and external information sources based on the user's past behavior history and feedback.

[0371] "Learning methods" refer to technologies that continuously improve the accuracy of information suggestions by utilizing feedback obtained from users.

[0372] This system is designed to suggest appropriate activities and content by focusing on user emotion recognition. The hardware used includes devices such as smartphones and tablets, which have built-in cameras and microphones. Through these devices, the user's facial expressions and voice tone are analyzed by emotion recognition tools. This process utilizes image processing software such as OpenCV and machine learning models such as TensorFlow.

[0373] Users can input their preferences and free time through the application interface. The user's current location is obtained through a location information acquisition system. This location information, preferences, and emotional state are all transmitted to an information processing system to suggest appropriate activities and content. This processing also references data and feedback from the user's past usage to create more personalized suggestions.

[0374] The suggestions are presented to the user through a display mechanism. For example, if the user is in the mood to relax, nearby relaxation spots or selected music will be suggested.

[0375] For example, if the system detects that the user is tired, it can suggest relaxing music and guide them to a pleasant cafe. An example of a prompt message when using a generative AI model to suggest music tailored to the user's situation might be, "Please list some music you recommend listening to when the user wants to relax." This allows users to enjoy activities suited to their state, ultimately improving their quality of life.

[0376] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0377] Step 1:

[0378] The device uses the user's camera and microphone to capture facial expressions and voice tone. The input consists of the acquired video and audio data, which is processed in real time using OpenCV or TensorFlow. An emotion recognition model analyzes this input data to determine the user's emotional state.

[0379] Step 2:

[0380] The device acquires the user's location information using GPS. This location information is sent to the server as data necessary for subsequent suggestion generation. The output is coordinate information indicating the user's current location.

[0381] Step 3:

[0382] Users input their preferences and available time through the terminal's interface. This provides input data representing the user's preferences and available time slots. The terminal then sends this data to the server for comparison with a database.

[0383] Step 4:

[0384] The server integrates the emotional state, location information, preferences, and free time obtained in steps 1 through 3 to process the information and suggest appropriate activities or content. Based on the input data, and taking into account past history and feedback data, it selects the optimal suggestion. As an output, it derives the suggestion that best suits the user.

[0385] Step 5:

[0386] The suggestions generated by the server are sent to the terminal. The terminal visually presents these suggestions to the user using a display device. The user can then select the options that interest them from the displayed choices.

[0387] Step 6:

[0388] User selections and feedback are sent back to the server and used as a learning tool to improve the accuracy of information suggestions. This allows the system to learn user preferences and make future suggestions more personalized.

[0389] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0390] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet Search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0391] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0392] [Third Embodiment]

[0393] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0394] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0395] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0396] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0397] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0398] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0399] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0400] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0401] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0402] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0403] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0404] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0405] The system of this invention proposes optimal activities based on the user's current location, preferences, and available time, in order to effectively utilize the user's free time. The system mainly consists of three elements: a terminal, a server, and the user.

[0406] First, users register their preferences and interests using the device on which the application is installed. In addition to this registration information, users input their free time into the app, which provides the basic data necessary for the system to make suggestions.

[0407] The device uses its built-in GPS function to obtain the user's current location. This location information, along with the user's preferences and registered interests, is sent to the server. The server searches its database based on the received location information and preferences to collect information on activities and places of interest around the user's current location.

[0408] The server further analyzes the user's past behavior history and feedback information to determine the priority of suggested activities. Once the optimal activity is determined, that information is sent back to the device and displayed to the user.

[0409] As an example, let's consider a user who is in Tokyo. This user is interested in "cafe hopping" and has an hour of free time. The device sends its current location information to the server, and the server retrieves information about cafes around Tokyo. Taking into account past usage history and ratings from other users, the server generates a list of several cafes and returns it to the user. The user can then spend their time meaningfully by visiting a cafe of their choice from this list.

[0410] In this way, the system takes into account the user's current location and preferences, and proposes optimal and personalized activities, enabling smooth and efficient activity in a short amount of time.

[0411] The following describes the processing flow.

[0412] Step 1:

[0413] The user launches the application on their device and enters information about their free time and preferences. This allows the system to understand the user's search criteria.

[0414] Step 2:

[0415] The device uses GPS functionality to obtain the user's current location. This location information, along with the user's entered free time and preferences, is packaged and a request is generated to send to the server.

[0416] Step 3:

[0417] The server analyzes requests received from terminals and collects information about the user's surrounding activity based on their current location from databases and external sources.

[0418] Step 4:

[0419] The server filters the collected activity information to match the user's preferences and free time. Furthermore, it takes into account the user's past behavior history and feedback information to determine the priority of suggested activities.

[0420] Step 5:

[0421] The server ranks and lists the selected activity options, and adds detailed information (e.g., location, duration, reviews, etc.) to each included option.

[0422] Step 6:

[0423] The server sends the generated list of suggestions to the terminal. The terminal converts the suggestions into a viewable format and presents them to the user.

[0424] Step 7:

[0425] The user selects an activity from the presented list that interests them, views its details, and decides whether to proceed. After completing the selected activity, the user provides feedback to the system.

[0426] Step 8:

[0427] The server records the feedback it receives and uses it to perform a learning process to improve the accuracy of future activity suggestions.

[0428] (Example 1)

[0429] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0430] In modern society, effectively utilizing short periods of time amidst busy schedules is a crucial challenge. However, it is difficult for users to independently research and select the optimal activities to maximize their limited free time. In particular, there is a need for collecting activity options based on current location, utilizing past activity history, and improving the accuracy of suggestions, but there is a problem in that no efficient method exists to achieve this.

[0431] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0432] In this invention, the server includes data input means for inputting the user's preferences and available time; information analysis means for obtaining activity options near the current location from information storage means or external information sources and ranking the options based on the user's past behavior history and evaluation information; and learning means for improving the accuracy of information presentation by utilizing evaluation information collected from the user. This makes it possible to provide personalized and efficient activity suggestions so that users can make optimal use of their free time.

[0433] "Location information acquisition means" refers to technology that measures the user's current location and acquires that information in a format that the system can use.

[0434] A "data input method" is a technique for receiving input information such as user preferences and available time, and preparing the data necessary for the system to process it.

[0435] "Information processing means" refers to a technology that performs a series of processes to select and present the most suitable activity to the user based on acquired location information and user preferences.

[0436] "Information display means" refers to technology or devices that provide information determined by a system to users in an easily understandable format.

[0437] "Information analysis means" refers to technologies that analyze data related to the user's current location and past behavioral history, and then rank and provide information that is appropriate for the user.

[0438] A "learning method" is a technology or algorithm that utilizes evaluation information obtained from users to improve the accuracy of information suggestions in the future.

[0439] This invention consists of a system designed to effectively utilize users' free time. The system is primarily comprised of three elements: terminals, servers, and users.

[0440] First, a dedicated application is installed on the device. This application runs on mobile devices such as smartphones and tablets and provides a means of inputting data to register the user's preferences and interests. Users can input personal interests such as "cafe hopping" or "watching movies," and also register their free time via the device.

[0441] The device has a built-in GPS sensor and functions as a means of acquiring location information. It measures the user's current location in real time and sends the results to the server. This location information, along with the user's registered information (interests and free time), is sent to the server as a data package.

[0442] The server, as an information processing tool, analyzes received data and selects the most suitable activities and locations for the user. The server utilizes a generative AI model to rank suggestions from activity options obtained from databases and external information sources, based on the user's past behavior history and feedback. The AI ​​model analyzes this information and builds logic to provide the user with the most suitable options.

[0443] Ultimately, users receive suggestions through an information display system. For example, if a user sets up a café hopping activity, the server will list popular cafés around Shibuya, and this information will be displayed on the terminal screen. In this way, users can spend their free time efficiently.

[0444] For example, if a user enters "I'm currently in Shibuya, Tokyo. I have an hour to spare. I've recently become interested in cafe hopping. Can you recommend some places?", the server can return information about appropriate cafes. This system offers a new approach to maximizing the value of the user's time through location-based, personalized suggestions.

[0445] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0446] Step 1:

[0447] Users launch an application on their device and input their preferences, interests, and free time. Specifically, users select options such as "cafe hopping" or "watching sports" and set their available time. This input information is prepared as a dataset and used for subsequent processing.

[0448] Step 2:

[0449] The device uses its built-in GPS sensor to obtain the user's current location in real time. The location information is updated periodically, and the acquired geographic data is sent to the server. This process generates location coordinate data, which serves as input for the next step.

[0450] Step 3:

[0451] The server integrates location information received from the terminal with user registration information and searches the database. Using a generative AI model, it processes this input data to find relevant activities and locations. This process involves data discrimination, which extracts the most relevant options from the information stored in the database.

[0452] Step 4:

[0453] The server analyzes the user's past behavior history and feedback provided by other users. Using a generative AI model, it ranks activities based on this historical data. By analyzing the behavior history, the server outputs activity suggestions in the most optimal order for the user.

[0454] Step 5:

[0455] The server sends the ranked options to the terminal. A results data package is generated and becomes the output sent to the terminal.

[0456] Step 6:

[0457] The device presents information to the user based on the received data. A ranked list of options is provided on the display screen, from which the user selects their desired activity or location. At this point, the final suggestion to the user based on the prompt is complete, and the user's activity begins. In this step, the user's selections can be saved and used as a learning tool to improve the accuracy of future suggestions.

[0458] (Application Example 1)

[0459] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0460] The challenge lies in maximizing users' free time and suggesting optimal activities, stores, and events based on their preferences and real-time location information. Traditional technologies have struggled to fully utilize past behavioral history and feedback, making it difficult to provide users with truly meaningful information in a timely manner.

[0461] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0462] In this invention, the server includes location information acquisition means, an input device, and an information analysis device. This makes it possible to select activity options based on the user's current location and, in particular, perform real-time information processing that reflects purchase history and interests, thereby providing optimal choices and enriching the user experience.

[0463] "Location information acquisition means" refers to a technical method for obtaining the user's current location, and is a device that uses GPS or other location information technologies to obtain accurate location data.

[0464] An "input device" is an interface for users to input their preferences and free time into the system, and includes screens and sensors that can be operated on smartphones or other electronic devices.

[0465] An "information processing device" is a device that performs computational processing to generate optimal activities and suggestions based on location information and user preference data.

[0466] A "display device" is a device used to visually display information suggested to a user, and includes the screens of smartphones and tablets.

[0467] An "information analysis device" is a technology that analyzes activity options obtained from databases and external information sources, and ranks them while taking into account the user's behavioral history and feedback.

[0468] A "learning device" is a system that uses machine learning technology to improve the accuracy of its suggestions by utilizing feedback obtained from users.

[0469] A "real-time information processing device" is a technology that selects information about stores and events based on a user's purchase history and interests, and processes it immediately to notify them of discount information.

[0470] The system for realizing this invention provides usable activity and purchase information based on the user's location information, preferences, and past behavioral data. This system mainly consists of a terminal, a server, and the user.

[0471] The device uses GPS functionality to obtain the user's location information. It also has an interface for inputting the user's preferences and free time. This interface is implemented as a smartphone or tablet application. Through this application, users can input their interests and planned free time.

[0472] The servers operate on a cloud platform and perform data processing. Receiving location information and user preference data, the servers retrieve activity options from databases and external sources. The retrieved information is analyzed using machine learning models, and rankings are generated based on the user's past behavior and feedback. In particular, libraries such as Scikit-learn are utilized to improve the accuracy of suggestions by learning from feedback.

[0473] Furthermore, the server selects information on nearby stores and events in real time based on the user's purchase history and interests, and provides advantageous information, including discounts.

[0474] The display device visually presents the user with information provided and analyzed by the terminal. This allows the user to make appropriate activity choices and purchases based on their current location.

[0475] A concrete example is a user in a shopping mall in Tokyo who is guided by the system and receives real-time information about limited-time sales at stores they have previously visited.

[0476] An example of a prompt message might be: "Generate prompts that take into account the user's interests and past purchase data to provide the best possible shopping experience based on their current location."

[0477] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0478] Step 1:

[0479] The device obtains the user's current location using GPS functionality. This location information is sent to the server as basic data for subsequent information processing. The input is location data from GPS, and the output is the transmission of location information to the server.

[0480] Step 2:

[0481] Users input their preferences and free time through the terminal interface. This information forms the basis for the user's activity choices and is sent to the server. The input is the user's preferences and free time data, and the output is the transmission of this data to the server.

[0482] Step 3:

[0483] The server retrieves appropriate activity options from databases and external sources based on received location information and user preferences. This data is then analyzed to extract activities that match the user's surrounding environment. The input is location and preference data, and the output is a list of available activity options.

[0484] Step 4:

[0485] The server uses a machine learning model to rank activity options based on past behavior history and feedback. This process leverages the Scikit-learn library to calculate priority for each user. The input is past behavior history and feedback information, and the output is a list of ranked activity options.

[0486] Step 5:

[0487] The server utilizes a real-time information processing device to select store and event information based on the user's purchase history and interests, and generates special offers, including discount information. Inputs are purchase history and user interest data, while outputs are selected store / event information and discount information.

[0488] Step 6:

[0489] The terminal visually presents information received from the server to the user. This information is displayed in a concrete and easy-to-understand format to support the user's choices. The input is comprehensive information data from the server, and the output is the information displayed on the screen.

[0490] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0491] This invention relates to a system that recognizes a user's emotions and proposes the most suitable activity based on those emotions. This system can make suggestions considering the user's current location, preferences, free time, and emotional state.

[0492] The user launches an application on their device and inputs their preferences and available time. The device also uses built-in sensors, camera, and microphone to capture the user's facial expressions, voice tone, and, in some cases, heart rate, using emotion recognition techniques. This data forms the basis for determining the user's emotional state.

[0493] The device sends a request to the server, along with acquired sentiment data, including the user's current location, preferences, and free time. Based on the received data, the server searches for information on surrounding activities from its database and external sources, and generates optimal suggestions tailored to the user's sentiment. By also considering past sentiment history and feedback, more personalized suggestions can be made.

[0494] For example, consider a scenario where a user is feeling tired and wants to visit a cafe. The device detects the user's fatigue level from their facial expression and sends this emotional information to the server. The server prioritizes and ranks cafes with a relaxed atmosphere and suggests establishments that offer relaxing menus. The user can then rest at the suggested cafe and feel satisfied.

[0495] The system can also monitor the user's emotional state in real time and generate new suggestions as needed. In this way, the present invention makes the user's free time more productive and provides an experience optimized for their emotional state.

[0496] The following describes the processing flow.

[0497] Step 1:

[0498] The user launches an application on their device and enters their free time and preferred activities. This allows the system to collect basic information to understand the user's preferences.

[0499] Step 2:

[0500] The system uses the device's sensors (camera, microphone, heart rate sensor, etc.) to collect user emotional information. Facial recognition technology is used to analyze facial expressions and determine the user's emotional state.

[0501] Step 3:

[0502] The device sends a request to the server containing collected sentiment data, location information, preferences, and free time. This prepares the server to generate suggestions based on all the necessary data.

[0503] Step 4:

[0504] Based on the received location information and user input, the server searches for nearby activity options from its database and external sources.

[0505] Step 5:

[0506] The server analyzes the user's emotional state and takes feedback data and past emotional history into consideration to select the most appropriate activity. For example, it prioritizes suggesting relaxing spots to a tired user.

[0507] Step 6:

[0508] The server ranks and lists the selected activity plans and their details, and sends them to the terminal.

[0509] Step 7:

[0510] The device visually displays the received list of suggestions to the user in real time. The user can refer to this list and select activities that interest them.

[0511] Step 8:

[0512] The user performs a selected activity and periodically records changes in their emotional state on the device. This allows feedback to be automatically updated at the end of the activity and stored as learning data for the system.

[0513] Step 9:

[0514] The emotion engine uses the collected feedback to optimize the overall system performance in order to improve the accuracy of future suggestions.

[0515] (Example 2)

[0516] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0517] Conventional activity suggestion systems only provided suggestions based on the user's location and preferences, without considering the user's emotional state. Therefore, they were unable to provide optimal suggestions that reflected what the user was feeling, making it difficult to maximize user satisfaction.

[0518] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0519] In this invention, the server includes location information acquisition means for obtaining the user's current location, interface means for inputting the user's preferences and free time, and emotion recognition means for recognizing the user's facial expressions, tone of voice, and heart rate to determine their emotions. This makes it possible to suggest activities that take into account the user's current emotional state.

[0520] "Location information acquisition means" refers to devices or software that acquire geographical location data using technology to determine the user's current location.

[0521] "Interface means" refers to a mechanism that allows users to input information into a system, and includes screens and operating tools for inputting preferences and free time within an application.

[0522] "Emotion recognition means" refers to devices and technologies that determine a user's emotional state by analyzing their facial expressions, tone of voice, heart rate, etc.

[0523] "Information processing means" refers to devices and software used to perform calculations and data analysis based on acquired data in order to propose the most suitable activities for the user.

[0524] "Display means" refers to a display device or its function for visually presenting the proposed activity to the user.

[0525] "Information analysis means" refers to technology that analyzes information obtained from a database or external information source and performs rankings based on the user's past behavioral history, emotional history, and feedback.

[0526] "Learning methods" refer to machine learning algorithms and technologies used to improve the accuracy of information suggestions based on feedback obtained from users.

[0527] This system is an activity suggestion platform consisting of a server, terminals, and users. Users input their preferences and free time through an application on their terminals. The terminals are equipped with GPS modules and other location acquisition technologies, which allow the system to accurately determine the user's current location.

[0528] The device uses its built-in camera, microphone, and sensors to detect the user's facial expressions, voice tone, heart rate, and other data in real time, recognizing the user's emotions. This data is analyzed using an emotion recognition algorithm. The analyzed data is then sent to the server as information reflecting the user's current emotional state.

[0529] The server searches for relevant activity information from databases and external sources based on received location, preference, free time, and emotional state data. Generative AI models are used for information processing, which then suggests the most suitable activities to the user. This process also includes information analysis techniques that consider the user's past behavioral and emotional history, as well as feedback.

[0530] After a suggestion is generated, the server sends it to the terminal, which then visually presents the suggestion to the user via its display. This system has a learning function and can continuously improve the accuracy of its suggestions based on user feedback.

[0531] For example, if a user wants to visit a cafe because they feel "relaxed," the device provides this emotional state to the server. The server then prioritizes suggesting "quiet cafes." An example of a prompt to the generative AI model would be, "User's emotion is relaxed, and their activity is to visit a cafe."

[0532] In this way, the system aims to provide users with high-quality ways to spend their time by suggesting optimal activities that match their emotional state.

[0533] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0534] Step 1: User Input Step

[0535] The user launches an application on their device and enters their preferences and available time into the interface. This input might include phrases like "I want to go to a cafe" or "I have two hours free." The device temporarily stores this information and prepares it for use in the next step.

[0536] Step 2: Emotion Recognition Step

[0537] The device uses built-in sensors, a camera, and a microphone to collect the user's facial expressions, voice tone, and heart rate in real time, and analyzes them with an emotion recognition algorithm. The input data includes facial images, voice data, and heart rate, and based on this, it determines the emotional state, such as "fatigue" or "relaxation." The output result of the emotional state becomes data to be transmitted next.

[0538] Step 3: Data transmission step

[0539] The device sends acquired location information, user preferences, free time, and emotional state to the server. The input data consists of the emotional state and user selection information obtained in the previous step, and the output is a request composed of an aggregation of this data.

[0540] Step 4: Proposal Generation Step

[0541] Based on the received data, the server uses a generative AI model to generate activity suggestions best suited to the user. This includes location information, the user's emotional state, and past behavioral history, and the data processing outputs suggestions such as "a quiet cafe."

[0542] Step 5: Proposal Presentation Step

[0543] The server generates a suggestion and sends it to the terminal, which then displays it on the user's screen. The suggestion data is input and visually presented to the user. The user can then decide on their course of action based on this information.

[0544] Step 6: Feedback Collection and Analysis Step

[0545] After the user completes an action based on a suggestion, the device collects feedback from the user. This feedback becomes input data, and the server analyzes it using a learning algorithm to improve the accuracy of future suggestions. This improves the accuracy of future suggestions.

[0546] (Application Example 2)

[0547] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0548] Traditional systems that cannot suggest appropriate activities and content based on user emotions struggle to significantly improve the user experience. Therefore, there is a need for a system that can accurately assess each user's emotional state and provide optimal suggestions.

[0549] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0550] In this invention, the server includes emotion recognition means for determining the user's current emotional state, location information acquisition means for obtaining the user's current location, and interface means for inputting the user's preferences and free time. This makes it possible to suggest optimal activities and content based on the user's emotions.

[0551] "Emotion recognition means" refers to technology that analyzes a user's facial expressions, voice tone, etc., to determine the user's current emotional state.

[0552] "Location information acquisition means" refers to technology that identifies the user's current location and performs other data processing based on that information.

[0553] "Interface means" refers to the technology or device that users use to input their preferences and available time.

[0554] "Information processing means" refers to technologies that suggest optimal activities and content based on acquired location information, user preferences, and emotional state.

[0555] "Display means" refers to technologies that visually present proposed activities or content to users.

[0556] "Information analysis methods" refer to technologies that rank options obtained from databases and external information sources based on the user's past behavior history and feedback.

[0557] "Learning methods" refer to technologies that continuously improve the accuracy of information suggestions by utilizing feedback obtained from users.

[0558] This system is designed to suggest appropriate activities and content by focusing on user emotion recognition. The hardware used includes devices such as smartphones and tablets, which have built-in cameras and microphones. Through these devices, the user's facial expressions and voice tone are analyzed by emotion recognition tools. This process utilizes image processing software such as OpenCV and machine learning models such as TensorFlow.

[0559] Users can input their preferences and free time through the application interface. The user's current location is obtained through a location information acquisition system. This location information, preferences, and emotional state are all transmitted to an information processing system to suggest appropriate activities and content. This processing also references data and feedback from the user's past usage to create more personalized suggestions.

[0560] The suggestions are presented to the user through a display mechanism. For example, if the user is in the mood to relax, nearby relaxation spots or selected music will be suggested.

[0561] For example, if the system detects that the user is tired, it can suggest relaxing music and guide them to a pleasant cafe. An example of a prompt message when using a generative AI model to suggest music tailored to the user's situation might be, "Please list some music you recommend listening to when the user wants to relax." This allows users to enjoy activities suited to their state, ultimately improving their quality of life.

[0562] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0563] Step 1:

[0564] The device uses the user's camera and microphone to capture facial expressions and voice tone. The input consists of the acquired video and audio data, which is processed in real time using OpenCV or TensorFlow. An emotion recognition model analyzes this input data to determine the user's emotional state.

[0565] Step 2:

[0566] The device acquires the user's location information using GPS. This location information is sent to the server as data necessary for subsequent suggestion generation. The output is coordinate information indicating the user's current location.

[0567] Step 3:

[0568] Users input their preferences and available time through the terminal's interface. This provides input data representing the user's preferences and available time slots. The terminal then sends this data to the server for comparison with a database.

[0569] Step 4:

[0570] The server integrates the emotional state, location information, preferences, and free time obtained in steps 1 through 3 to process the information and suggest appropriate activities or content. Based on the input data, and taking into account past history and feedback data, it selects the optimal suggestion. As an output, it derives the suggestion that best suits the user.

[0571] Step 5:

[0572] The suggestions generated by the server are sent to the terminal. The terminal visually presents these suggestions to the user using a display device. The user can then select the options that interest them from the displayed choices.

[0573] Step 6:

[0574] User selections and feedback are sent back to the server and used as a learning tool to improve the accuracy of information suggestions. This allows the system to learn user preferences and make future suggestions more personalized.

[0575] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0576] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0577] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0578] [Fourth Embodiment]

[0579] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0580] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0581] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0582] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0583] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0584] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0585] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0586] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0587] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0588] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0589] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0590] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0591] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0592] The system of this invention proposes optimal activities based on the user's current location, preferences, and available time, in order to effectively utilize the user's free time. The system mainly consists of three elements: a terminal, a server, and the user.

[0593] First, users register their preferences and interests using the device on which the application is installed. In addition to this registration information, users input their free time into the app, which provides the basic data necessary for the system to make suggestions.

[0594] The device uses its built-in GPS function to obtain the user's current location. This location information, along with the user's preferences and registered interests, is sent to the server. The server searches its database based on the received location information and preferences to collect information on activities and places of interest around the user's current location.

[0595] The server further analyzes the user's past behavior history and feedback information to determine the priority of suggested activities. Once the optimal activity is determined, that information is sent back to the device and displayed to the user.

[0596] As an example, let's consider a user who is in Tokyo. This user is interested in "cafe hopping" and has an hour of free time. The device sends its current location information to the server, and the server retrieves information about cafes around Tokyo. Taking into account past usage history and ratings from other users, the server generates a list of several cafes and returns it to the user. The user can then spend their time meaningfully by visiting a cafe of their choice from this list.

[0597] In this way, the system takes into account the user's current location and preferences, and proposes optimal and personalized activities, enabling smooth and efficient activity in a short amount of time.

[0598] The following describes the processing flow.

[0599] Step 1:

[0600] The user launches the application on their device and enters information about their free time and preferences. This allows the system to understand the user's search criteria.

[0601] Step 2:

[0602] The device uses GPS functionality to obtain the user's current location. This location information, along with the user's entered free time and preferences, is packaged and a request is generated to send to the server.

[0603] Step 3:

[0604] The server analyzes requests received from terminals and collects information about the user's surrounding activity based on their current location from databases and external sources.

[0605] Step 4:

[0606] The server filters the collected activity information to match the user's preferences and free time. Furthermore, it takes into account the user's past behavior history and feedback information to determine the priority of suggested activities.

[0607] Step 5:

[0608] The server ranks and lists the selected activity options, and adds detailed information (e.g., location, duration, reviews, etc.) to each included option.

[0609] Step 6:

[0610] The server sends the generated list of suggestions to the terminal. The terminal converts the suggestions into a viewable format and presents them to the user.

[0611] Step 7:

[0612] The user selects an activity from the presented list that interests them, views its details, and decides whether to proceed. After completing the selected activity, the user provides feedback to the system.

[0613] Step 8:

[0614] The server records the feedback it receives and uses it to perform a learning process to improve the accuracy of future activity suggestions.

[0615] (Example 1)

[0616] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0617] In modern society, effectively utilizing short periods of time amidst busy schedules is a crucial challenge. However, it is difficult for users to independently research and select the optimal activities to maximize their limited free time. In particular, there is a need for collecting activity options based on current location, utilizing past activity history, and improving the accuracy of suggestions, but there is a problem in that no efficient method exists to achieve this.

[0618] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0619] In this invention, the server includes data input means for inputting the user's preferences and available time; information analysis means for obtaining activity options near the current location from information storage means or external information sources and ranking the options based on the user's past behavior history and evaluation information; and learning means for improving the accuracy of information presentation by utilizing evaluation information collected from the user. This makes it possible to provide personalized and efficient activity suggestions so that users can make optimal use of their free time.

[0620] "Location information acquisition means" refers to technology that measures the user's current location and acquires that information in a format that the system can use.

[0621] A "data input method" is a technique for receiving input information such as user preferences and available time, and preparing the data necessary for the system to process it.

[0622] "Information processing means" refers to a technology that performs a series of processes to select and present the most suitable activity to the user based on acquired location information and user preferences.

[0623] "Information display means" refers to technology or devices that provide information determined by a system to users in an easily understandable format.

[0624] "Information analysis means" refers to technologies that analyze data related to the user's current location and past behavioral history, and then rank and provide information that is appropriate for the user.

[0625] A "learning method" is a technology or algorithm that utilizes evaluation information obtained from users to improve the accuracy of information suggestions in the future.

[0626] This invention consists of a system designed to effectively utilize users' free time. The system is primarily comprised of three elements: terminals, servers, and users.

[0627] First, a dedicated application is installed on the device. This application runs on mobile devices such as smartphones and tablets and provides a means of inputting data to register the user's preferences and interests. Users can input personal interests such as "cafe hopping" or "watching movies," and also register their free time via the device.

[0628] The device has a built-in GPS sensor and functions as a means of acquiring location information. It measures the user's current location in real time and sends the results to the server. This location information, along with the user's registered information (interests and free time), is sent to the server as a data package.

[0629] The server, as an information processing tool, analyzes received data and selects the most suitable activities and locations for the user. The server utilizes a generative AI model to rank suggestions from activity options obtained from databases and external information sources, based on the user's past behavior history and feedback. The AI ​​model analyzes this information and builds logic to provide the user with the most suitable options.

[0630] Ultimately, users receive suggestions through an information display system. For example, if a user sets up a café hopping activity, the server will list popular cafés around Shibuya, and this information will be displayed on the terminal screen. In this way, users can spend their free time efficiently.

[0631] For example, if a user enters "I'm currently in Shibuya, Tokyo. I have an hour to spare. I've recently become interested in cafe hopping. Can you recommend some places?", the server can return information about appropriate cafes. This system offers a new approach to maximizing the value of the user's time through location-based, personalized suggestions.

[0632] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0633] Step 1:

[0634] Users launch an application on their device and input their preferences, interests, and free time. Specifically, users select options such as "cafe hopping" or "watching sports" and set their available time. This input information is prepared as a dataset and used for subsequent processing.

[0635] Step 2:

[0636] The device uses its built-in GPS sensor to obtain the user's current location in real time. The location information is updated periodically, and the acquired geographic data is sent to the server. This process generates location coordinate data, which serves as input for the next step.

[0637] Step 3:

[0638] The server integrates location information received from the terminal with user registration information and searches the database. Using a generative AI model, it processes this input data to find relevant activities and locations. This process involves data discrimination, which extracts the most relevant options from the information stored in the database.

[0639] Step 4:

[0640] The server analyzes the user's past behavior history and feedback provided by other users. Using a generative AI model, it ranks activities based on this historical data. By analyzing the behavior history, the server outputs activity suggestions in the most optimal order for the user.

[0641] Step 5:

[0642] The server sends the ranked options to the terminal. A results data package is generated and becomes the output sent to the terminal.

[0643] Step 6:

[0644] The device presents information to the user based on the received data. A ranked list of options is provided on the display screen, from which the user selects their desired activity or location. At this point, the final suggestion to the user based on the prompt is complete, and the user's activity begins. In this step, the user's selections can be saved and used as a learning tool to improve the accuracy of future suggestions.

[0645] (Application Example 1)

[0646] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0647] The challenge lies in maximizing users' free time and suggesting optimal activities, stores, and events based on their preferences and real-time location information. Traditional technologies have struggled to fully utilize past behavioral history and feedback, making it difficult to provide users with truly meaningful information in a timely manner.

[0648] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0649] In this invention, the server includes location information acquisition means, an input device, and an information analysis device. This makes it possible to select activity options based on the user's current location and, in particular, perform real-time information processing that reflects purchase history and interests, thereby providing optimal choices and enriching the user experience.

[0650] "Location information acquisition means" refers to a technical method for obtaining the user's current location, and is a device that uses GPS or other location information technologies to obtain accurate location data.

[0651] An "input device" is an interface for users to input their preferences and free time into the system, and includes screens and sensors that can be operated on smartphones or other electronic devices.

[0652] An "information processing device" is a device that performs computational processing to generate optimal activities and suggestions based on location information and user preference data.

[0653] A "display device" is a device used to visually display information suggested to a user, and includes the screens of smartphones and tablets.

[0654] An "information analysis device" is a technology that analyzes activity options obtained from databases and external information sources, and ranks them while taking into account the user's behavioral history and feedback.

[0655] A "learning device" is a system that uses machine learning technology to improve the accuracy of its suggestions by utilizing feedback obtained from users.

[0656] A "real-time information processing device" is a technology that selects information about stores and events based on a user's purchase history and interests, and processes it immediately to notify them of discount information.

[0657] The system for realizing this invention provides usable activity and purchase information based on the user's location information, preferences, and past behavioral data. This system mainly consists of a terminal, a server, and the user.

[0658] The device uses GPS functionality to obtain the user's location information. It also has an interface for inputting the user's preferences and free time. This interface is implemented as a smartphone or tablet application. Through this application, users can input their interests and planned free time.

[0659] The servers operate on a cloud platform and perform data processing. Receiving location information and user preference data, the servers retrieve activity options from databases and external sources. The retrieved information is analyzed using machine learning models, and rankings are generated based on the user's past behavior and feedback. In particular, libraries such as Scikit-learn are utilized to improve the accuracy of suggestions by learning from feedback.

[0660] Furthermore, the server selects information on nearby stores and events in real time based on the user's purchase history and interests, and provides advantageous information, including discounts.

[0661] The display device visually presents the user with information provided and analyzed by the terminal. This allows the user to make appropriate activity choices and purchases based on their current location.

[0662] A concrete example is a user in a shopping mall in Tokyo who is guided by the system and receives real-time information about limited-time sales at stores they have previously visited.

[0663] An example of a prompt message might be: "Generate prompts that take into account the user's interests and past purchase data to provide the best possible shopping experience based on their current location."

[0664] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0665] Step 1:

[0666] The device obtains the user's current location using GPS functionality. This location information is sent to the server as basic data for subsequent information processing. The input is location data from GPS, and the output is the transmission of location information to the server.

[0667] Step 2:

[0668] Users input their preferences and free time through the terminal interface. This information forms the basis for the user's activity choices and is sent to the server. The input is the user's preferences and free time data, and the output is the transmission of this data to the server.

[0669] Step 3:

[0670] The server retrieves appropriate activity options from databases and external sources based on received location information and user preferences. This data is then analyzed to extract activities that match the user's surrounding environment. The input is location and preference data, and the output is a list of available activity options.

[0671] Step 4:

[0672] The server uses a machine learning model to rank activity options based on past behavior history and feedback. This process leverages the Scikit-learn library to calculate priority for each user. The input is past behavior history and feedback information, and the output is a list of ranked activity options.

[0673] Step 5:

[0674] The server utilizes a real-time information processing device to select store and event information based on the user's purchase history and interests, and generates special offers, including discount information. Inputs are purchase history and user interest data, while outputs are selected store / event information and discount information.

[0675] Step 6:

[0676] The terminal visually presents information received from the server to the user. This information is displayed in a concrete and easy-to-understand format to support the user's choices. The input is comprehensive information data from the server, and the output is the information displayed on the screen.

[0677] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0678] This invention relates to a system that recognizes a user's emotions and proposes the most suitable activity based on those emotions. This system can make suggestions considering the user's current location, preferences, free time, and emotional state.

[0679] The user launches an application on their device and inputs their preferences and available time. The device also uses built-in sensors, camera, and microphone to capture the user's facial expressions, voice tone, and, in some cases, heart rate, using emotion recognition techniques. This data forms the basis for determining the user's emotional state.

[0680] The device sends a request to the server, along with acquired sentiment data, including the user's current location, preferences, and free time. Based on the received data, the server searches for information on surrounding activities from its database and external sources, and generates optimal suggestions tailored to the user's sentiment. By also considering past sentiment history and feedback, more personalized suggestions can be made.

[0681] For example, consider a scenario where a user is feeling tired and wants to visit a cafe. The device detects the user's fatigue level from their facial expression and sends this emotional information to the server. The server prioritizes and ranks cafes with a relaxed atmosphere and suggests establishments that offer relaxing menus. The user can then rest at the suggested cafe and feel satisfied.

[0682] The system can also monitor the user's emotional state in real time and generate new suggestions as needed. In this way, the present invention makes the user's free time more productive and provides an experience optimized for their emotional state.

[0683] The following describes the processing flow.

[0684] Step 1:

[0685] The user launches an application on their device and enters their free time and preferred activities. This allows the system to collect basic information to understand the user's preferences.

[0686] Step 2:

[0687] The system uses the device's sensors (camera, microphone, heart rate sensor, etc.) to collect user emotional information. Facial recognition technology is used to analyze facial expressions and determine the user's emotional state.

[0688] Step 3:

[0689] The device sends a request to the server containing collected sentiment data, location information, preferences, and free time. This prepares the server to generate suggestions based on all the necessary data.

[0690] Step 4:

[0691] Based on the received location information and user input, the server searches for nearby activity options from its database and external sources.

[0692] Step 5:

[0693] The server analyzes the user's emotional state and takes feedback data and past emotional history into consideration to select the most appropriate activity. For example, it prioritizes suggesting relaxing spots to a tired user.

[0694] Step 6:

[0695] The server ranks and lists the selected activity plans and their details, and sends them to the terminal.

[0696] Step 7:

[0697] The device visually displays the received list of suggestions to the user in real time. The user can refer to this list and select activities that interest them.

[0698] Step 8:

[0699] The user performs a selected activity and periodically records changes in their emotional state on the device. This allows feedback to be automatically updated at the end of the activity and stored as learning data for the system.

[0700] Step 9:

[0701] The emotion engine uses the collected feedback to optimize the overall system performance in order to improve the accuracy of future suggestions.

[0702] (Example 2)

[0703] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0704] Conventional activity suggestion systems only provided suggestions based on the user's location and preferences, without considering the user's emotional state. Therefore, they were unable to provide optimal suggestions that reflected what the user was feeling, making it difficult to maximize user satisfaction.

[0705] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0706] In this invention, the server includes location information acquisition means for obtaining the user's current location, interface means for inputting the user's preferences and free time, and emotion recognition means for recognizing the user's facial expressions, tone of voice, and heart rate to determine their emotions. This makes it possible to suggest activities that take into account the user's current emotional state.

[0707] "Location information acquisition means" refers to devices or software that acquire geographical location data using technology to determine the user's current location.

[0708] "Interface means" refers to a mechanism that allows users to input information into a system, and includes screens and operating tools for inputting preferences and free time within an application.

[0709] "Emotion recognition means" refers to devices and technologies that determine a user's emotional state by analyzing their facial expressions, tone of voice, heart rate, etc.

[0710] "Information processing means" refers to devices and software used to perform calculations and data analysis based on acquired data in order to propose the most suitable activities for the user.

[0711] "Display means" refers to a display device or its function for visually presenting the proposed activity to the user.

[0712] "Information analysis means" refers to technology that analyzes information obtained from a database or external information source and performs rankings based on the user's past behavioral history, emotional history, and feedback.

[0713] "Learning methods" refer to machine learning algorithms and technologies used to improve the accuracy of information suggestions based on feedback obtained from users.

[0714] This system is an activity suggestion platform consisting of a server, terminals, and users. Users input their preferences and free time through an application on their terminals. The terminals are equipped with GPS modules and other location acquisition technologies, which allow the system to accurately determine the user's current location.

[0715] The device uses its built-in camera, microphone, and sensors to detect the user's facial expressions, voice tone, heart rate, and other data in real time, recognizing the user's emotions. This data is analyzed using an emotion recognition algorithm. The analyzed data is then sent to the server as information reflecting the user's current emotional state.

[0716] The server searches for relevant activity information from databases and external sources based on received location, preference, free time, and emotional state data. Generative AI models are used for information processing, which then suggests the most suitable activities to the user. This process also includes information analysis techniques that consider the user's past behavioral and emotional history, as well as feedback.

[0717] After a suggestion is generated, the server sends it to the terminal, which then visually presents the suggestion to the user via its display. This system has a learning function and can continuously improve the accuracy of its suggestions based on user feedback.

[0718] For example, if a user wants to visit a cafe because they feel "relaxed," the device provides this emotional state to the server. The server then prioritizes suggesting "quiet cafes." An example of a prompt to the generative AI model would be, "User's emotion is relaxed, and their activity is to visit a cafe."

[0719] In this way, the system aims to provide users with high-quality ways to spend their time by suggesting optimal activities that match their emotional state.

[0720] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0721] Step 1: User Input Step

[0722] The user launches an application on their device and enters their preferences and available time into the interface. This input might include phrases like "I want to go to a cafe" or "I have two hours free." The device temporarily stores this information and prepares it for use in the next step.

[0723] Step 2: Emotion Recognition Step

[0724] The device uses built-in sensors, a camera, and a microphone to collect the user's facial expressions, voice tone, and heart rate in real time, and analyzes them with an emotion recognition algorithm. The input data includes facial images, voice data, and heart rate, and based on this, it determines the emotional state, such as "fatigue" or "relaxation." The output result of the emotional state becomes data to be transmitted next.

[0725] Step 3: Data transmission step

[0726] The device sends acquired location information, user preferences, free time, and emotional state to the server. The input data consists of the emotional state and user selection information obtained in the previous step, and the output is a request composed of an aggregation of this data.

[0727] Step 4: Proposal Generation Step

[0728] Based on the received data, the server uses a generative AI model to generate activity suggestions best suited to the user. This includes location information, the user's emotional state, and past behavioral history, and the data processing outputs suggestions such as "a quiet cafe."

[0729] Step 5: Proposal Presentation Step

[0730] The server generates a suggestion and sends it to the terminal, which then displays it on the user's screen. The suggestion data is input and visually presented to the user. The user can then decide on their course of action based on this information.

[0731] Step 6: Feedback Collection and Analysis Step

[0732] After the user completes an action based on a suggestion, the device collects feedback from the user. This feedback becomes input data, and the server analyzes it using a learning algorithm to improve the accuracy of future suggestions. This improves the accuracy of future suggestions.

[0733] (Application Example 2)

[0734] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0735] Traditional systems that cannot suggest appropriate activities and content based on user emotions struggle to significantly improve the user experience. Therefore, there is a need for a system that can accurately assess each user's emotional state and provide optimal suggestions.

[0736] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0737] In this invention, the server includes emotion recognition means for determining the user's current emotional state, location information acquisition means for obtaining the user's current location, and interface means for inputting the user's preferences and free time. This makes it possible to suggest optimal activities and content based on the user's emotions.

[0738] "Emotion recognition means" refers to technology that analyzes a user's facial expressions, voice tone, etc., to determine the user's current emotional state.

[0739] "Location information acquisition means" refers to technology that identifies the user's current location and performs other data processing based on that information.

[0740] "Interface means" refers to the technology or device that users use to input their preferences and available time.

[0741] "Information processing means" refers to technologies that suggest optimal activities and content based on acquired location information, user preferences, and emotional state.

[0742] "Display means" refers to technologies that visually present proposed activities or content to users.

[0743] "Information analysis methods" refer to technologies that rank options obtained from databases and external information sources based on the user's past behavior history and feedback.

[0744] "Learning methods" refer to technologies that continuously improve the accuracy of information suggestions by utilizing feedback obtained from users.

[0745] This system is designed to suggest appropriate activities and content by focusing on user emotion recognition. The hardware used includes devices such as smartphones and tablets, which have built-in cameras and microphones. Through these devices, the user's facial expressions and voice tone are analyzed by emotion recognition tools. This process utilizes image processing software such as OpenCV and machine learning models such as TensorFlow.

[0746] Users can input their preferences and free time through the application interface. The user's current location is obtained through a location information acquisition system. This location information, preferences, and emotional state are all transmitted to an information processing system to suggest appropriate activities and content. This processing also references data and feedback from the user's past usage to create more personalized suggestions.

[0747] The suggestions are presented to the user through a display mechanism. For example, if the user is in the mood to relax, nearby relaxation spots or selected music will be suggested.

[0748] For example, if the system detects that the user is tired, it can suggest relaxing music and guide them to a pleasant cafe. An example of a prompt message when using a generative AI model to suggest music tailored to the user's situation might be, "Please list some music you recommend listening to when the user wants to relax." This allows users to enjoy activities suited to their state, ultimately improving their quality of life.

[0749] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0750] Step 1:

[0751] The device uses the user's camera and microphone to capture facial expressions and voice tone. The input consists of the acquired video and audio data, which is processed in real time using OpenCV or TensorFlow. An emotion recognition model analyzes this input data to determine the user's emotional state.

[0752] Step 2:

[0753] The device acquires the user's location information using GPS. This location information is sent to the server as data necessary for subsequent suggestion generation. The output is coordinate information indicating the user's current location.

[0754] Step 3:

[0755] Users input their preferences and available time through the terminal's interface. This provides input data representing the user's preferences and available time slots. The terminal then sends this data to the server for comparison with a database.

[0756] Step 4:

[0757] The server integrates the emotional state, location information, preferences, and free time obtained in steps 1 through 3 to process the information and suggest appropriate activities or content. Based on the input data, and taking into account past history and feedback data, it selects the optimal suggestion. As an output, it derives the suggestion that best suits the user.

[0758] Step 5:

[0759] The suggestions generated by the server are sent to the terminal. The terminal visually presents these suggestions to the user using a display device. The user can then select the options that interest them from the displayed choices.

[0760] Step 6:

[0761] User selections and feedback are sent back to the server and used as a learning tool to improve the accuracy of information suggestions. This allows the system to learn user preferences and make future suggestions more personalized.

[0762] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0763] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0764] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0765] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0766] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0767] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0768] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0769] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0770] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0771] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0772] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0773] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0774] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0775] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0776] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0777] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0778] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0779] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0780] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0781] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0782] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0783] The following is further disclosed regarding the embodiments described above.

[0784] (Claim 1)

[0785] A means of obtaining location information to obtain the user's current location,

[0786] An interface for inputting user preferences and free time,

[0787] Information processing means for suggesting the optimal activity based on the aforementioned location information and preferences,

[0788] A means for presenting the proposal to the user,

[0789] A system that includes this.

[0790] (Claim 2)

[0791] The system according to claim 1, comprising information analysis means for obtaining activity options around the user's current location from a database or external information source and ranking the candidates based on the user's past behavior history and feedback.

[0792] (Claim 3)

[0793] The system according to claim 1, comprising a learning means for improving the accuracy of information suggestions using feedback obtained from users.

[0794] "Example 1"

[0795] (Claim 1)

[0796] A means for obtaining location information to obtain the user's current location,

[0797] A data entry means for inputting user preferences and available time,

[0798] Information processing means for suggesting the optimal activity based on the aforementioned location information and preferences,

[0799] Information display means for showing the presentation to the user,

[0800] An information processing system that includes this.

[0801] (Claim 2)

[0802] The information processing system according to claim 1, which includes an information analysis means for obtaining activity options near the user's current location from an information storage means or an external information source, and for ranking the options based on the user's past behavioral history and evaluation information.

[0803] (Claim 3)

[0804] The information processing system according to claim 1, which includes a learning means for improving the accuracy of information presentation by utilizing evaluation information collected from users.

[0805] "Application Example 1"

[0806] (Claim 1)

[0807] A means of obtaining location information to obtain the user's current location,

[0808] An input device for entering the user's preferences and free time,

[0809] An information processing device for suggesting the optimal activity based on the aforementioned location information and preferences,

[0810] A display device for presenting the proposal to the user,

[0811] A system that includes this.

[0812] (Claim 2)

[0813] The system according to claim 1, comprising an information analysis device for acquiring activity options around the current location from a storage means or external information source, and for ranking candidates based on the user's past behavior history and feedback.

[0814] (Claim 3)

[0815] A learning device that improves the accuracy of information suggestions by utilizing feedback obtained from users,

[0816] A real-time information processing device that selects store and event information based on the user's purchase history and interests, and notifies them of discount information.

[0817] The system according to claim 1, including the following:

[0818] "Example 2 of combining an emotion engine"

[0819] (Claim 1)

[0820] A means for obtaining location information to obtain the user's current location,

[0821] An interface means for inputting user preferences and free time,

[0822] An emotion recognition system for recognizing the user's facial expressions, tone of voice, and heart rate to determine their emotions,

[0823] Information processing means for suggesting the optimal activity based on the aforementioned location information, preferences, and emotional state,

[0824] A means of displaying proposals to users,

[0825] A system that includes this.

[0826] (Claim 2)

[0827] The system according to claim 1, comprising information analysis means for obtaining activity options around the user's current location from a database or external information source, and for ranking candidates based on the user's past behavioral history, emotional history, and feedback.

[0828] (Claim 3)

[0829] The system according to claim 1, comprising a learning means for improving the accuracy of information suggestions using feedback obtained from users.

[0830] "Application example 2 when combining with an emotional engine"

[0831] (Claim 1)

[0832] An emotion recognition means for determining the user's current emotional state,

[0833] A means of obtaining location information to obtain the user's current location,

[0834] An interface for inputting user preferences and free time,

[0835] Information processing means for suggesting optimal activities and content based on the aforementioned location information, preferences, and emotional state,

[0836] A means for presenting the proposal to the user,

[0837] A system that includes this.

[0838] (Claim 2)

[0839] The system according to claim 1, comprising information analysis means for obtaining activity options or content around the user's current location from a database or external information source, and for ranking candidates based on the user's past behavior history and feedback.

[0840] (Claim 3)

[0841] The system according to claim 1, comprising a learning means for improving the accuracy of information suggestions using feedback obtained from users. [Explanation of Symbols]

[0842] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of obtaining location information to obtain the user's current location, An interface for inputting user preferences and free time, Information processing means for suggesting the optimal activity based on the aforementioned location information and preferences, A means for presenting the proposal to the user, A system that includes this.

2. The system according to claim 1, comprising information analysis means for obtaining activity options around the user's current location from a database or external information source and ranking the candidates based on the user's past behavior history and feedback.

3. The system according to claim 1, comprising a learning means for improving the accuracy of information suggestions using feedback obtained from users.

Images