System

Abstract

A system is provided.SOLUTION: The system includes a means for receiving information such as a user's interest, spare time, a place, and a mood, a means for acquiring weather information and event information, a means for analyzing a past selection history and feedback of the user and a similar user, a means for generating an optimal proposal for the user based on an analysis result, and a means for providing the generated proposal to the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In today's world, many people struggle to make effective use of their free time and spare time, plagued by information overload and a vast array of choices. As a result, time is often wasted, and reliance on the same activities and places can lead to a lack of freshness. To solve this problem, a method is needed to quickly and accurately provide optimal time-killing suggestions based on the user's current interests, time, location, mood, etc. [Means for solving the problem]

[0005] The present invention provides a system that acquires weather and event information based on information such as user input such as interests, free time, location, and mood, and then analyzes the user's and similar users' past selection history and feedback based on this information. This system includes a means for generating optimal suggestions based on the analysis results and providing them to the user. It also has a function for receiving questions from the user and providing related information, thereby saving the user the trouble of gathering additional information. It also includes a means for saving the user's selection history and feedback and improving the accuracy of suggestions for future visits. This allows users to make new discoveries and have interesting experiences without any effort, allowing them to make effective use of their time.

[0006] "User" refers to an individual or entity who uses the system and provides information such as interests, free time, location, and mood.

[0007] "Interests" refers to activities, themes, or areas of interest that a user has a particular interest in, and are used as the basis for the system to generate suggestions.

[0008] "Free time" refers to the free time a user can devote to a specific activity, and is the criterion used by the system to determine whether or not a suggestion is applicable.

[0009] "Location" refers to a user's current location or designated area, which the system uses to generate geographically relevant suggestions.

[0010] "Mood" is information that reflects the user's emotions and state at any given time, and is a factor that this system takes into consideration when generating suggestions that are appropriate for the user.

[0011] "Weather information" refers to weather data for the user's current location or a location specified by the user, and is information that the system incorporates as external conditions when generating suggestions.

[0012] "Event Information" refers to data about events taking place at the user's current location or a location specified by the user, and is used by the system as a factor in generating suggestions.

[0013] "Selection history" refers to a record of past choices and decisions made by the user or similar users, and is used as reference data when the system generates suggestions.

[0014] "Feedback" refers to the act or content of a user providing evaluation or opinion regarding the system's proposals or activities provided by the system.

[0015] "Analysis results" refers to information generated as a result of analysis based on the user's input information and acquired data, and serves as the basis for suggestions to the user.

[0016] "Means for generating suggestions" refers to the system's ability to analyze user input and related data to create activities and information that are optimal for the user.

[0017] A "question and answer function" is an interaction function that a user uses to request additional information, and refers to a system element that provides answers to questions.

[0018] "Means for storing selection history and feedback" refers to a system function that continuously stores the selections made by the user and the feedback provided in a database in order to improve the accuracy of suggestions in future visits. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0020] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0023] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] The present invention is a system for making effective use of users' idle time. This system generates optimal time-killing suggestions based on information such as the user's interests, free time, location, and mood, taking into account weather and event information. The specific system configuration and processing flow are described below.

[0041] System Configuration

[0042] This system consists of a user terminal, a server, and external information services (weather information API, event information API, etc.). User terminals are assumed to be devices such as smartphones and tablets. The server functions as a central management system, collecting, analyzing, and generating proposals for various types of data. The external information services are Internet resources that provide weather and event information.

[0043] Program processing

[0044] Enter and submit user information

[0045] 1. The user opens the application and selects the "Time-killing suggestions" feature.

[0046] 2. The device displays an interface for entering information such as the user's interests, availability, location, and mood, which may include selection forms or input forms.

[0047] 3. When the user enters the required information and presses the send button, the information is sent from the device to the server.

[0048] Data collection and analysis

[0049] 4. The server receives the information sent by the user and updates the user profile.

[0050] 5. The server calls the weather information API to obtain the latest weather data for the user's current location.

[0051] 6. The server calls the event information API and obtains information about events being held in the user's current location.

[0052] 7. The server retrieves the user's and similar users' past selection history and feedback from the database and passes it to the analysis engine.

[0053] Proposal Generation

[0054] 8. The server integrates weather data, event data, and user profiles and uses an analytics engine to generate optimal suggestions for users, using machine learning algorithms and a recommendation system based on similar user histories.

[0055] 9. The server formats the offer with detailed information (location, time, discounts, reviews, etc.) and sends it to the device.

[0056] Proposal presentation and user interaction

[0057] 10. The device displays the received suggestions to the user. The suggestions can be visualized in a list format, card format, or other format.

[0058] 11. Users review the suggestions and select the activities they are interested in. They can also use the Q&A feature if they need further clarification.

[0059] 12. The device sends the user's selections and questions to the server.

[0060] Data provision and storage

[0061] 13. The server obtains detailed information about the user's question, generates an answer, and sends it to the device.

[0062] 14. The device displays the answer to the user.

[0063] 15. The user finally performs the selected activity.

[0064] 16. The terminal sends the user's selections and execution status to the server, which stores them in a database.

[0065] 17. The server collects user feedback and uses it to improve the accuracy of suggestions in future.

[0066] Specific examples

[0067] Killing time at the station

[0068] 1. While waiting for a train at a station, the user opens the app and selects the "Time-killing suggestions" feature.

[0069] 2. The device will display a screen asking about your interests.

[0070] 3. The user enters "cafe hopping," "30 minutes," and "current location is the station," and submits.

[0071] 4. The server receives this information and updates the profile.

[0072] 5. The server calls the weather information API and confirms that the current weather is "sunny."

[0073] 6. The server calls the event information API and obtains event information around the station.

[0074] 7. The server integrates the data with past data and generates suggestions for new cafe coupons, art exhibitions near train stations, and the latest movie screenings.

[0075] 8. The server sends the suggestions to the device, which displays them to the user.

[0076] 9. The user selects a cafe coupon and views more details.

[0077] 10. The device sends the question to the server, which provides a detailed review.

[0078] 11. The user finally selects a cafe and decides to visit.

[0079] 12. Your device sends your selections to the server, which stores your history and feedback.

[0080] The above is an embodiment of the present invention, which allows users to easily discover new activities and have interesting experiences.

[0081] The processing flow will be explained below.

[0082] Step 1:

[0083] The user opens the application and selects the "Time-killing suggestions" feature. The device then displays an interface for entering information such as the user's interests, free time, location, and mood.

[0084] Step 2:

[0085] The user enters the necessary information and presses the send button. The device sends the information entered by the user to the server.

[0086] Step 3:

[0087] The server receives user information (interests, free time, location, mood, etc.) sent from the device and updates the user profile.

[0088] Step 4:

[0089] The server calls the weather information API to obtain the latest weather data for the user's current location, and stores the obtained weather information on the server.

[0090] Step 5:

[0091] The server calls the event information API to obtain information about events taking place in the user's current location, and stores the obtained event information on the server.

[0092] Step 6:

[0093] The server retrieves the user's and similar users' past selection history and feedback from the database and passes it to the analysis engine, which then begins the analysis process.

[0094] Step 7:

[0095] The server integrates weather data, event data, and user profiles, and runs the analysis through an analytics engine that uses machine learning algorithms and a recommendation system based on similar user history to generate optimal suggestions for users.

[0096] Step 8:

[0097] The server formats the generated suggestions by adding details (e.g., cafe location, hours, discounts, reviews, event location and time, etc.) and prepares the suggestions for delivery to the user.

[0098] Step 9:

[0099] The server sends the prepared suggestions to the device, which then displays them to the user. The suggestions are visualized in a list format, card format, or other format.

[0100] Step 10:

[0101] Users can review suggestions, select activities they are interested in, and use the Q&A feature to find out more information.

[0102] Step 11:

[0103] The device sends the user's selections and questions to the server, which retrieves relevant information from a database and generates an answer.

[0104] Step 12:

[0105] The server generates a response and sends it to the device, which displays it to the user, providing them with the necessary details.

[0106] Step 13:

[0107] The user then performs the activity they have finally selected (e.g., visiting a cafe, attending an event, watching a movie, etc.). The device then sends the user's selection and the execution status to the server.

[0108] Step 14:

[0109] The server stores the user's selection history and feedback in a database, which is used to improve the accuracy of suggestions in future.

[0110] Step 15:

[0111] After completing an activity, the user inputs and sends a simple feedback. The device sends the feedback to the server, which ends the user interaction.

[0112] Example 1

[0113] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0114] In modern society, it is a difficult task to effectively utilize users' free time and suggest activities appropriate for that time. Conventional systems often fail to provide suggestions that are in line with the user's interests and current situation, leaving users unable to make satisfying choices. The present invention aims to solve this problem and provide users with optimal time-killing suggestions.

[0115] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0116] In this invention, the server includes means for receiving information such as the user's interests, free time, location, and mood, means for acquiring weather information and event information, means for analyzing the user's and similar users' past selection history and feedback, means for integrating the weather data, event data, and user profile and generating suggestions using an analysis engine, means for formatting detailed information about the suggestions (location, time, discount details, reviews, etc.) and sending them to the terminal, means for receiving the user's questions and providing related information, means for sending the user's selections and questions to the server, means for saving the user's selection history and feedback to improve the accuracy of suggestions from next time onwards, and means for collecting user feedback and saving it in a database, thereby making it possible to suggest activities that are optimal for the user's free time.

[0117] "Means for receiving information such as user interests, free time, location, and mood" refers to an interface that allows users to input their interests, free time, current location, mood, etc. through the application, and the function for collecting that information.

[0118] "Means for obtaining weather information and event information" refers to API call functions and data collection functions that obtain current weather and nearby event information from external weather information services and event information services.

[0119] "Means for analyzing the past selection history and feedback of users and similar users" refers to a function for analyzing the past selection history and feedback stored in a database using a machine learning algorithm.

[0120] "Means for integrating weather data, event data, and user profiles and generating suggestions using an analytical engine" refers to a function that combines acquired weather data, event information, and user profiles and generates optimal suggestions using an analytical engine.

[0121] "Means for formatting detailed information about the proposal (location, time, discount details, reviews, etc.) and sending it to the terminal" refers to a function that adds detailed information about the proposal, such as location information, time, discount details, reviews, etc., to the generated proposal, formats it, and sends it to the user's terminal.

[0122] "Means for receiving user questions and providing relevant information" refers to a function that receives questions from users and provides appropriate answer information or additional detailed information.

[0123] The "means for transmitting user selections and questions to the server" is a function for transmitting information about a user's selection of an activity or question to the server.

[0124] "Means for saving user selection history and feedback to improve the accuracy of future suggestions" refers to a function that saves the user's selections and subsequent feedback in a database and uses that data when generating suggestions the next time.

[0125] "Means for collecting user feedback and saving it in a database" refers to a function that collects feedback on the activities experienced by users after they have entered it and saves it in a database.

[0126] The present invention is a system for making effective use of a user's free time, and generates optimal time-killing suggestions based on information such as the user's interests, free time, location, and mood, taking into account weather and event information. The specific system configuration and processing flow are described below.

[0127] System Configuration

[0128] This system consists of a user terminal, a server, and an external information service. The user terminal is assumed to be a device such as a smartphone or tablet. The server functions as a central management system, collecting, analyzing, and generating proposals for various data. The external information service is an Internet resource that provides weather and event information.

[0129] Hardware and software used

[0130] User devices are smartphones or tablets. The server is configured as a system including a database server, application server, and analysis engine. The weather information API uses the OpenWeatherMap API, and the event information API uses the Eventbrite API. The analysis engine implements a machine learning algorithm using Scikit-learn.

[0131] Program processing flow

[0132] 1. The user opens the application and selects the "Time-killing suggestions" feature.

[0133] 2. The device displays an interface for entering information such as the user's interests, availability, location, and mood, which may include selection forms or input forms.

[0134] 3. When the user enters the required information and presses the send button, the information is sent from the device to the server.

[0135] 4. The server receives the information sent by the user and updates the user profile.

[0136] 5. The server calls the weather information API to obtain the latest weather data for the user's current location.

[0137] 6. The server calls the event information API and obtains information about events being held in the user's current location.

[0138] 7. The server retrieves the user's and similar users' past selection history and feedback from the database and passes it to the analysis engine.

[0139] 8. The server integrates weather data, event data, and user profiles and uses an analytics engine to generate optimal suggestions for users, using machine learning algorithms and a recommendation system based on similar user histories.

[0140] 9. The server formats the offer with detailed information (location, time, discounts, reviews, etc.) and sends it to the device.

[0141] 10. The device displays the received suggestions to the user. The suggestions can be visualized in a list format, card format, or other format.

[0142] 11. Users review the suggestions and select the activities they are interested in. They can also use the Q&A feature if they need further clarification.

[0143] 12. The device sends the user's selections and questions to the server.

[0144] 13. The server obtains detailed information about the user's question, generates an answer, and sends it to the device.

[0145] 14. The device displays the answer to the user.

[0146] 15. The user finally performs the selected activity.

[0147] 16. The terminal sends the user's selections and execution status to the server, which stores them in a database.

[0148] 17. The server collects user feedback and uses it to improve the accuracy of suggestions in future.

[0149] Specific examples

[0150] Killing time at the station

[0151] 1. While waiting for a train at a station, the user opens the app and selects the "Time-killing suggestions" feature.

[0152] 2. The device will display a screen asking about your interests.

[0153] 3. The user enters "cafe hopping," "30 minutes," and "current location is the station" and submits.

[0154] 4. The server receives this information and updates the profile.

[0155] 5. The server calls the weather information API and confirms that the current weather is "sunny."

[0156] 6. The server calls the event information API and obtains event information around the station.

[0157] 7. The server integrates the data with past data and generates suggestions for new cafe coupons, art exhibitions near train stations, and the latest movie screenings.

[0158] 8. The server sends the suggestions to the device, which displays them to the user.

[0159] 9. The user selects the cafe coupon and views more details.

[0160] 10. The device sends the question to the server, which provides a detailed review.

[0161] 11. The user finally selects a cafe and decides to visit.

[0162] 12. The device sends the selection to the server, which stores the history and feedback.

[0163] Example prompts for generative AI models

[0164] "I'm currently waiting for a train at the station. I'd like to take advantage of the time I have to spend at a cafe. Do you have any recommendations? The conditions are that the weather is currently sunny and the journey will take about 30 minutes."

[0165] The above is an embodiment of the present invention, which allows users to easily discover new activities and have interesting experiences.

[0166] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0167] Step 1:

[0168] The user opens the application and selects the "Time-killing suggestions" function. The input is a user action, and the output is the launch of the application. The specific action is that the user taps the smartphone icon to launch the app.

[0169] Step 2:

[0170] The device displays an interface for entering information such as the user's interests, free time, location, and mood. The input is a signal to start the application, and the output is a screen for entering user information. Specifically, the application presents the user with options and a form for entering "interests," "free time," "location," and "mood."

[0171] Step 3:

[0172] The user enters the necessary information and presses the send button. The input is the information entered by the user (interests, free time, location, mood), and the output is sent from the device to the server. Specifically, the user enters "cafe hopping," "30 minutes," "current location," and "relax," and taps the "send" button.

[0173] Step 4:

[0174] The server receives the information submitted by the user and updates the user profile. The input is the information submitted by the user, and the output is the updated profile, which is saved in the database. Specifically, the server accesses the database and executes an SQL query to update the user profile.

[0175] Step 5:

[0176] The server calls the weather information API to obtain the latest weather data for the user's current location. The input is the user's current location information, and the output is the latest weather data. Specifically, the server sends a request to the OpenWeatherMap API and receives the current weather information (e.g., "Sunny, temperature 24 degrees").

[0177] Step 6:

[0178] The server calls the event information API to obtain information about events taking place in the user's current location. The input is the user's current location information, and the output is event information. Specifically, the server sends a request to the Eventbrite API and receives information about events in the area.

[0179] Step 7:

[0180] The server retrieves the past selection history and feedback of the user and similar users from the database and passes it to the analysis engine. The input is the user ID, and the output is the selection history and feedback data. Specifically, the server executes an SQL query to retrieve the past history and feedback from the database and passes it to the analysis engine.

[0181] Step 8:

[0182] The server integrates weather data, event data, and user profiles and uses an analytics engine to generate optimal suggestions for users. Weather data, event data, and user profiles are input, and generated suggestions are obtained as output. Specifically, the server runs a machine learning algorithm using Scikit-learn to generate optimal suggestions.

[0183] Step 9:

[0184] The server formats the proposal with detailed information (location, time, discount details, reviews, etc.) and sends it to the device. The generated proposal is the input, and the formatted proposal is sent to the device as the output. Specifically, the server formats the proposal content in JSON format and sends it to the device via an HTTP request.

[0185] Step 10:

[0186] The device displays the received suggestions to the user.,The input is the suggestion sent from the server, and the,output is the suggestion displayed on the screen.,Specifically, the device visualizes the suggestion,contents in list or card format and displays them to the,user.

[0187] Step 11:

[0188] The user reviews the suggestions and selects the activity they are interested in. If additional explanation is needed, they can use the question and answer function. The suggestions are displayed as input, and the selected activity and questions are sent to the server as output. The user taps the activity card to view more information.

[0189] Step 12:

[0190] The device sends the user's selections and questions to the server. The input is the user's selections and questions, and the output is the information sent to the server. Specifically, the device sends the selected activity ID and the question to the server.

[0191] Step 13:

[0192] The server obtains detailed information about the user's question, generates an answer, and sends it to the terminal. The user's question is the input, and the generated answer is sent to the terminal as the output. In concrete terms, the server obtains the necessary information from the database, generates the answer, and sends it to the terminal.

[0193] Step 14:

[0194] The terminal displays the answer to the user. The answer from the server is input, and the answer is displayed to the user as output. Specifically, the terminal displays the answer on the screen.

[0195] Step 15:

[0196] The user ultimately performs the selected activity. The input is the proposed activity, and the output is the execution of that activity. The concrete action is the user actually performing the selected activity.

[0197] Step 16:

[0198] The terminal sends the user's selection and execution status to the server. The user's selection and execution status are input, and are sent to the server as output. Specifically, the terminal sends information about the executed activity to the server.

[0199] Step 17:

[0200] The server collects user feedback and uses it to improve the accuracy of future proposals. The input is user feedback, and the output is saved in a database. Specifically, the server saves the feedback information in the database and uses it to generate future proposals.

[0201] (Application example 1)

[0202] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0203] In today's modern living environment, users often waste their time waiting or having free time. Particularly in physical stores, there are few ways to efficiently find available activities and promotions, and users cannot easily obtain information that interests them. This results in users not making effective use of their valuable time and a decrease in the quality of their experience.

[0204] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0205] In this invention, the server includes means for receiving information such as the user's interests, free time, location, and mood, means for acquiring weather information and event information, and means for analyzing the past selection history and feedback of the user and similar users, thereby enabling a user interface means for generating optimal suggestions for the user based on the analysis results, displaying the generated suggestions on the user terminal, providing detailed information and reviews, and providing detailed information and navigation for the activity selected by the user.

[0206] "User Terminal" means a smartphone, tablet or other mobile device through which a User inputs information and receives suggestions.

[0207] A "sensor" is a device used as a means of obtaining information such as a user's interests, free time, location, and mood.

[0208] "Weather information" is information that provides data related to the weather, and is obtained from an external information providing service.

[0209] "Event information" is information about events and activities related to a specific location and period, and is obtained from an external information providing service.

[0210] A "user profile" is a data structure that includes information such as a user's interests, availability, location, mood, past selections, and feedback.

[0211] "Analysis results" are information generated by the analysis engine based on weather information, event information, user profiles, etc., and include optimal suggestions for users.

[0212] A "user interface" is a screen or mechanism for displaying information and suggestions to users and receiving input and feedback from users.

[0213] "Navigation" is a function that shows and guides the user to the location of the activity selected by the user.

[0214] "Suggestions" are suggested time-killing activities or promotions that are generated based on the analysis results and provided to users.

[0215] "Details" means specific information related to the proposed activity, such as location, time, discounts, reviews, etc.

[0216] System Overview

[0217] This invention is designed as a system for effectively utilizing users' idle time, and is intended for use in brick-and-mortar stores in particular. The system consists of a user terminal, a server, and an external information service. The user terminal corresponds to a smartphone or tablet, and the server includes an analysis engine, a database, and a proposal generation system. The external information service includes a weather information API and an event information API. This allows the system to generate and execute time-killing proposals in real time that are optimized for the user's current situation and preferences.

[0218] Program structure and main processing

[0219] 1. Enter and submit user information

[0220] User device: The user launches the application and selects the "Time-killing suggestions" function. At this point, an interface is displayed for entering information such as the user's interests, free time, location, and mood. This interface consists of a selection form and an input form. The entered information is sent from the device to the server.

[0221] 2. Data collection and analysis

[0222] Server: The server receives the information sent by the user and updates the user profile. It then calls the weather information API to get the latest weather data for the user's current location, calls the event information API to get information about events taking place in the user's current location, and retrieves the user's and similar users' past selection history and feedback from the database and passes them to the analytics engine.

[0223] 3. Proposal Generation

[0224] Server: The server integrates weather data, event data, and user profiles and uses an analytics engine to generate optimal suggestions for users. The suggestions are generated using machine learning algorithms and a recommendation system based on similar user history. The generated suggestions are then sent to the user's device along with detailed information (location, time, discounts, reviews, etc.).

[0225] 4. Proposal Presentation and User Interaction

[0226] User device: The user device displays the received suggestions to the user. The suggestions are visualized in a list format, card format, etc. The user reviews the suggestions and selects the activities they are interested in. If further explanation is needed, the user can use the question and answer function. The device sends the user's selections and questions to the server.

[0227] 5. Data provision and storage

[0228] Server: The server obtains detailed information about the user's question, generates an answer, and sends it to the device. The device displays the answer to the user. The user then performs the selected activity, and the device sends subsequent feedback and selection history to the server. The server stores this information in a database and uses it to improve the accuracy of future suggestions.

[0229] Hardware and software used

[0230] Hardware: Servers, smartphones, tablets

[0231] Software: Python, SQLite, scikit-learn (machine learning library), requests (HTTP request library)

[0232] Specific examples

[0233] For example, a user sets their current location to a "major station," their interests to "cafes" and "art galleries," and their free time to 60 minutes. Based on this information, the server retrieves cloudy weather information from a weather information API and supplements it with information on art events taking place around the station from an event information API. Suggestions are generated for Cafe ABC (10% off coupon) and Art Gallery XYZ (highly rated reviews) in front of the station. The user selects Cafe ABC and views detailed information (coupon usage guide, navigation).

[0234] Prompt Sentence Examples

[0235] Prompt: "A user is interested in cafes and art galleries, is located at a major train station, and has 60 minutes of free time. Generate suggestions for the best ways to kill time for him. The current weather is cloudy, and there are art events happening around the station. Include location, time, discounts, and reviews in the suggestions."

[0236] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0237] Step 1:

[0238] Enter and submit user information

[0239] A user launches the application and selects the "Time-killing suggestions" function. The user's device displays an interface for entering information such as interests, free time, location, and mood.

[0240] Input: User interests, availability, location, and mood information

[0241] Output: Sending information from the device to the server

[0242] Step 2:

[0243] Update your user profile

[0244] The server receives the information submitted by the user and updates the user profile.

[0245] Input: User information sent from the device

[0246] Output: Updated user profile

[0247] Step 3:

[0248] Get weather information

[0249] The server calls the weather information API to obtain the latest weather data for the user's current location.

[0250] Input: User's current location

[0251] Output: Current weather data for your current location

[0252] Step 4:

[0253] Get event information

[0254] The server calls the event information API and obtains information about events being held in the user's current location.

[0255] Input: User's current location

[0256] Output: Event information for the current location

[0257] Step 5:

[0258] Get past selection history and feedback

[0259] The server retrieves the user's and similar users' past selection history and feedback from the database and passes it to the analysis engine.

[0260] Input: User profile

[0261] Output: Past selection history and feedback

[0262] Step 6:

[0263] Analysis and proposal generation

[0264] The server combines weather data, event data, and user profiles, and uses an analytics engine to generate optimal suggestions for users, using machine learning algorithms and recommendation systems.

[0265] Input: Weather data, event data, user profile

[0266] Output: Best time-killing suggestions

[0267] Step 7:

[0268] Submit a proposal

[0269] The server sends the generated proposal to the user's device, along with detailed information (location, time, discounts, reviews, etc.).

[0270] Input: Best time-killing suggestions

[0271] Output: Proposal sent to user's device

[0272] Step 8:

[0273] Presenting the proposal

[0274] The user's device displays the received suggestions to the user. The suggestions are visualized in a list or card format, and the user can review the suggestions and select the activities that interest them.

[0275] Input: Proposal

[0276] Output: Suggestions shown to the user

[0277] Step 9:

[0278] Ask questions and get more information

[0279] If the user requests further clarification, the device sends the question to the server, which obtains detailed information about the user's question, generates an answer, and sends it to the device.

[0280] Input: User question

[0281] Output: Answer to the question

[0282] Step 10:

[0283] Final Selection and Navigation

[0284] It provides detailed information and navigation for the activity that the user ultimately selects. The user device sends the selection to the server, which stores the history and feedback.

[0285] Input: Last selected activity

[0286] Output: Activity details and navigation, saving selection history

[0287] Usage examples and prompt statements

[0288] As a concrete example, suppose a user sets their current location to "major station," their interests to "cafes" and "art galleries," and their free time to 60 minutes.

[0289] Prompt: "A user is interested in cafes and art galleries, is located at a major train station, and has 60 minutes of free time. Generate suggestions for the best ways to kill time for him. The current weather is cloudy, and there are art events happening around the station. Include location, time, discounts, and reviews in the suggestions."

[0290] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0291] This invention is a system that helps users make effective use of their idle time. This system generates optimal time-killing suggestions based on information such as the user's interests, free time, location, and mood, taking into account weather and event information. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it is possible to realize more personalized suggestions.

[0292] System Configuration

[0293] This system consists of a user terminal, a server, an external information provision service (weather information API, event information API, etc.), and an emotion engine. User terminals are assumed to be devices such as smartphones and tablets. The server functions as a central management system, collecting and analyzing various data and generating proposals. The external information provision service is an Internet resource for providing weather and event information. The emotion engine is a system that recognizes emotions from the user's voice and facial expressions and reflects them in proposals.

[0294] Program processing

[0295] Enter and submit user information

[0296] 1. The user opens the application and selects the "Time-killing suggestions" feature.

[0297] 2. The device displays an interface for inputting the user's interests, availability, location, mood, and emotions, including selection and input forms.

[0298] 3. When the user enters the required information and presses the send button, the information is sent from the device to the server.

[0299] Data collection and analysis

[0300] 4. The server receives the information sent by the user and updates the user profile.

[0301] 5. The server calls the weather information API to obtain the latest weather data for the user's current location and stores the obtained weather information on the server.

[0302] 6. The server calls the event information API to obtain information about events taking place in the user's current location. The obtained event information is stored on the server.

[0303] 7. The server retrieves the user's and similar users' past selection history and feedback from the database and passes it to the analysis engine, which then begins the analysis process.

[0304] Acquiring and analyzing emotion data

[0305] 8. The emotion engine acquires the user's voice and facial expression data, recognizes their emotions, and sends the emotion information obtained by the emotion engine to the server.

[0306] 9. The server adds the emotion information received from the emotion engine to the user profile and passes it to the analysis engine.

[0307] Proposal Generation

[0308] 10. The server integrates weather data, event data, emotion data, and user profiles and analyzes them in an analytics engine, which uses machine learning algorithms and a recommendation system based on similar user history to generate optimal suggestions for the user.

[0309] 11. The server formats the offer with detailed information (e.g., cafe location, hours, discounts, reviews, etc.) and sends it to the device.

[0310] Proposal presentation and user interaction

[0311] 12. The device displays the received suggestions to the user. The suggestions can be visualized in a list format, card format, or other format.

[0312] 13. Users can review the suggestions, select the activities they are interested in, and use the Q&A feature to find out more information.

[0313] 14. The device sends the user's selections and questions to the server.

[0314] 15. The server obtains detailed information about the user's question, generates an answer, and sends it to the device.

[0315] 16. The device displays the answer to the user, providing them with the necessary details.

[0316] Data provision and storage

[0317] 17. The user finally performs the selected activity (e.g., visit a cafe, attend an event, watch a movie, etc.). The device sends the user's selection and execution status to the server.

[0318] 18. The server stores the user's selection history and feedback in a database, which will be used to improve the accuracy of future suggestions.

[0319] 19. After the user completes the activity, they input and submit a brief feedback. The device sends the feedback to the server, which ends the user interaction.

[0320] Specific examples

[0321] Killing time at the station

[0322] 1. While waiting for a train at a station, the user opens the app and selects the "Time-killing suggestions" feature.

[0323] 2. The device will display a screen asking for your interests, free time, location, mood, and emotions.

[0324] 3. The user inputs "cafe hopping," "30 minutes," "Shinjuku Station," "want to relax," and "emotional data via voice input (e.g., feeling stressed based on the tone of voice)" and submits.

[0325] 4. The server receives this information and updates the profile.

[0326] 5. The server calls the weather information API and confirms that the current weather is "sunny."

[0327] 6. The server calls the event information API and obtains event information around Shinjuku Station.

[0328] 7. The server integrates the data with past data and adds data from the emotion engine for analysis.

[0329] 8. The server generates suggestions for new cafes, art exhibits near train stations, and the latest movie screenings. The suggestions are focused on relaxing content that reflects the user’s emotional state.

[0330] 9. The server sends the proposal to the device, which displays it to the user.

[0331] 10. The user selects a cafe coupon and views more details.

[0332] 11. The device sends the question to the server, which provides a detailed review.

[0333] 12. The user finally selects a cafe and decides to visit.

[0334] 13. Your device sends your selections to the server, which stores your history and feedback.

[0335] The above is an embodiment of the present invention. This makes it possible to propose new activities that take emotions into consideration, allowing users to enjoy new, personalized experiences without any effort.

[0336] The processing flow will be explained below.

[0337] Step 1:

[0338] The user opens the application and selects the "Time-killing suggestions" function. The device then displays an interface for inputting information such as the user's interests, free time, location, mood, and emotions.

[0339] Step 2:

[0340] The user enters their interests (e.g., cafes, movies, art exhibitions), free time (e.g., 30 minutes), location (e.g., Shinjuku Station), mood (e.g., wanting to relax), and emotion (e.g., feeling stressed), and then presses the send button. The device then sends the entered information to the server.

[0341] Step 3:

[0342] The server receives user information (interests, free time, location, mood, emotions, etc.) sent from the device and updates the user profile.

[0343] Step 4:

[0344] The server calls the weather information API to obtain the latest weather data for the user's current location, and stores the obtained weather information on the server.

[0345] Step 5:

[0346] The server calls the event information API to obtain information about events taking place in the user's current location, and stores the obtained event information on the server.

[0347] Step 6:

[0348] The server retrieves the user's and similar users' past selection history and feedback from the database and passes it to the analysis engine, which then begins the analysis process.

[0349] Step 7:

[0350] The emotion engine acquires emotional data from the user's voice and facial expressions, recognizes that the user is feeling stressed, and sends this emotional information to the server.

[0351] Step 8:

[0352] The server adds the emotional information received from the emotion engine to the user profile and passes it to the analysis engine, which then performs an analysis based on the user's current emotional state.

[0353] Step 9:

[0354] The server integrates weather data, event data, user profiles, and sentiment data, and runs the analysis through an analytics engine that generates optimal recommendations for each user based on machine learning algorithms and similar user history.

[0355] Step 10:

[0356] The server formats the generated suggestions by adding details (e.g., cafe location, opening hours, discounts, reviews, event location and time, etc.) and prepares the suggestions for delivery to the user.

[0357] Step 11:

[0358] The server sends the prepared suggestions to the device, which then displays them to the user. The suggestions are visualized in a list format, card format, or other format.

[0359] Step 12:

[0360] Users can review the suggestions and select activities they are interested in. They can also use the Q&A feature to find out more information.

[0361] Step 13:

[0362] The device sends the user's selections and questions to the server, which retrieves relevant information from a database and generates an answer.

[0363] Step 14:

[0364] The server generates a response and sends it to the device, which displays it to the user, providing them with the necessary details.

[0365] Step 15:

[0366] The user then performs the activity they have finally selected (e.g., visiting a cafe, attending an event, watching a movie, etc.). The device then sends the user's selection and the execution status to the server.

[0367] Step 16:

[0368] The server stores the user's selection history and feedback in a database, which is used to improve the accuracy of suggestions in future.

[0369] Step 17:

[0370] After completing an activity, the user inputs and sends a simple feedback. The device sends the feedback to the server, which ends the user interaction.

[0371] Example 2

[0372] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0373] Previous time-killing suggestion systems were unable to take into account the user's subjective feelings and therefore had a low level of personalization. Furthermore, it was difficult to obtain and reflect weather and event information in real time, which led to a lack of reliability in terms of whether the system would actually be useful to the user. Furthermore, the system was unable to improve the accuracy of suggestions by utilizing past selection history and feedback, which led to issues such as low user satisfaction.

[0374] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for receiving information such as the user's interests, free time, location, mood, and emotions; means for acquiring weather information and event information; means for analyzing the past selection history and feedback of the user and similar users; means for generating optimal suggestions for the user based on the analysis results; means for providing the generated suggestions to the user; means for recognizing emotions from the user's voice and facial expressions and reflecting the emotion data in suggestions; means for allowing the user to select activities that interest the user; and means for providing detailed information about the activities selected by the user. This enables personalized suggestions that take into account the user's subjective emotions and reflect real-time information.

[0375] "User Information" refers to data such as a user's interests, free time, location, mood, and emotions.

[0376] "Weather Information" refers to data regarding weather conditions in the User's current location.

[0377] "Event Information" refers to data about an event taking place at a particular location and time.

[0378] "Selection history" refers to a record of a user's past choices and actions.

[0379] "Feedback" refers to information regarding opinions and ratings from users.

[0380] An "emotion engine" refers to a system that recognizes emotions from the user's voice and facial expressions.

[0381] A "profile" refers to a data set that compiles information such as a user's attributes, behavioral history, and emotions.

[0382] "Analysis" refers to the data processing and analysis work required to generate optimal proposals based on collected data.

[0383] "Suggestions" refers to options for activities or actions presented to users based on the analysis results.

[0384] This invention is a system that helps users make effective use of their idle time by generating optimal time-killing suggestions based on information such as the user's interests, free time, location, mood, and emotions, taking into account weather and event information. Furthermore, by using an emotion engine that recognizes emotions from the user's voice and facial expressions, more personalized suggestions become possible.

[0385] This system consists of a user terminal, a server, an external information provision service (weather information API, event information API, etc.), and an emotion engine. User terminals are assumed to be devices such as smartphones and tablets. The server functions as a central management system, collecting and analyzing various data and generating proposals. The external information provision service is an Internet resource for providing weather and event information. The emotion engine is a system that recognizes emotions from the user's voice and facial expressions and reflects them in proposals.

[0386] Hardware and software used

[0387] Hardware:

[0388] Smartphone

[0389] tablet

[0390] server

[0391] software:

[0392] Weather information API (e.g. OpenWeatherMap API)

[0393] Event information API (e.g. Eventbrite API)

[0394] Emotion recognition API (e.g., Microsoft® Azure® Cognitive Services)

[0395] Machine learning engine (e.g., TENSORFLOW (registered trademark))

[0396] Program processing

[0397] Enter and submit user information

[0398] A user opens the application on their smartphone or tablet and selects the "Time-killing suggestions" function. The device then displays an interface for inputting the user's interests, free time, location, mood, and emotions. This can include checkboxes, drop-down menus, text boxes, and other formats. Once the user enters the information and presses the submit button, the device sends the information to the server. The transmission is secured using the HTTPS protocol.

[0399] Data collection and analysis

[0400] The server receives the information sent by the user and updates the user profile. The server then calls a weather information API to obtain the latest weather data for the user's current location, for example, using the OpenWeatherMap API. The server then calls an event information API to obtain information about events taking place in the user's current location, for example, using the Eventbrite API. In addition, the server retrieves the user's and similar users' past selection history and feedback from a database and passes this information to the analytics engine.

[0401] Acquiring and analyzing emotion data

[0402] The emotion engine acquires the user's voice and facial expression data and recognizes their emotions. For example, it uses the emotion recognition API from Microsoft Azure Cognitive Services. The emotion information obtained by the emotion engine is sent to the server, which adds it to the user profile. This data is also passed to the analysis engine.

[0403] Proposal Generation

[0404] The server integrates weather data, event data, emotion data, and user profiles, and analyzes them using an analysis engine. The analysis engine uses machine learning algorithms (e.g., TensorFlow). Based on the analysis results, the server generates optimal suggestions for the user, adds detailed information, and sends them to the device.

[0405] Proposal presentation and user interaction

[0406] The device displays the received suggestions to the user. The suggestions are visualized in a list or card format. The user can review the suggestions and select activities that interest them. A question and answer function is also available for further information. The selections and questions are sent from the device to the server, which then provides further information.

[0407] Specific examples

[0408] Killing time at the station

[0409] While waiting for a train at a station, the user opens the app and selects the "Time-Killing Suggestions" feature. The device displays a screen asking for "Interests," "Free Time," "Current Location," "Mood," and "Emotions." The user inputs and submits "Cafe Hopping," "30 Minutes," "Shinjuku Station," "Want to Relax," and emotional data via voice input (voice tone indicates stress). The server receives this information and updates the profile. The server then calls the weather information API and confirms that the current weather is "sunny." The server then calls the event information API to obtain event information around Shinjuku Station. The server then combines this data with past data and adds data from the emotion engine for analysis. The server generates suggestions for new cafe coupons, art exhibitions near the station, and the latest movie screenings. The suggestions are focused on relaxing content tailored to the user's emotional state. The server sends the suggestions to the device, which then displays them to the user. The user selects a cafe coupon and browses for further details. The device then sends a question to the server, which provides a detailed review. The user finally selects a cafe and decides to visit. The device then sends the selection to the server, which stores the history and feedback. This will improve the accuracy of proposals from next time onwards.

[0410] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0411] Step 1:

[0412] User enters information

[0413] Users open the application on their smartphone or tablet and select the "Time-killing suggestions" function.

[0414] Input: Users input their interests, availability, location, mood, or emotions within the app, for example using checkboxes, drop-down menus, or text boxes.

[0415] Specific operation: The user inputs "cafe," "30 minutes," "Shinjuku Station," "relax," and "stress by tone of voice."

[0416] Output: The input data is stored in the user's device memory.

[0417] Step 2:

[0418] The device sends information

[0419] The terminal transmits the information entered by the user to the server.

[0420] Input: Data entered by the user.

[0421] What it does: The device uses the HTTPS protocol to communicate securely and send data to the server.

[0422] Output: The server receives the information sent by the user.

[0423] Step 3:

[0424] The server updates the user profile

[0425] The server records and updates the information received from the user as a user profile.

[0426] Input: User-entered data.

[0427] What happens: The server updates the profile using an SQL query against the database.

[0428] Output: Updated user profile data is saved to the database.

[0429] Step 4:

[0430] The server retrieves weather information

[0431] The server calls the weather information API and obtains the latest weather information for the current location (Shinjuku Station).

[0432] Input: User's current location.

[0433] Specific behavior: The server sends a request to the API endpoint to retrieve weather information.

[0434] Output: The latest weather information is saved on the server.

[0435] Step 5:

[0436] The server retrieves the event information

[0437] The server calls the event information API and obtains information about events being held at the current location (Shinjuku Station).

[0438] Input: User's current location.

[0439] Specific operation: The server sends a request to the API endpoint to obtain event information.

[0440] Output: The acquired event information is saved on the server.

[0441] Step 6:

[0442] The server retrieves the past selection history

[0443] The server retrieves the user's and similar users' past selection history and feedback from a database.

[0444] Input: User ID or similar user group.

[0445] What happens: The server executes an SQL query to get the required data.

[0446] Output: Past selection history and feedback are captured on the server.

[0447] Step 7:

[0448] The server passes the data to the analysis engine

[0449] The server passes the collected information to the analysis engine and begins analysis.

[0450] Input: User input, weather information, event information, past selections, feedback.

[0451] Specific operation: The server inputs the data into an analysis engine (e.g., TensorFlow) and performs the analysis.

[0452] Output: The analysis results are generated.

[0453] Step 8:

[0454] Emotion engine recognizes emotions

[0455] The emotion engine captures the user's voice data and recognizes emotions.

[0456] Input: User's voice data.

[0457] What it does: The emotion engine runs a speech recognition system to identify emotions (e.g., determine stress levels).

[0458] Output: Recognized emotion data is generated.

[0459] Step 9:

[0460] The emotion engine sends emotion information to the server

[0461] The emotion engine sends the recognized emotion information to the server.

[0462] Input: Recognized emotion data.

[0463] Specific operation: The emotion engine sends emotion data to the server using the HTTPS protocol.

[0464] Output: Emotion information is received on the server side.

[0465] Step 10:

[0466] The server adds emotional information to the profile

[0467] The server adds the emotional information to the user profile and passes it to the analysis engine.

[0468] Input: Recognized emotion data.

[0469] Specific operation: The server adds the emotion data to the database and updates the profile.

[0470] Output: Passes updated user profile data to the analytics engine.

[0471] Step 11:

[0472] The server integrates and analyzes the data

[0473] The server integrates weather data, event data, emotion data, and user profiles, and performs analysis using an analytical engine.

[0474] Input: All integrated data (weather information, event information, emotion data, user profile).

[0475] Specific operations: The server runs the analytical algorithm and performs data analysis.

[0476] Output: The learned suggestions are generated.

[0477] Step 12:

[0478] The server generates a proposal and sends it to the device

[0479] The server generates optimal suggestions for the user based on the analysis results, adds detailed information, and sends them to the device.

[0480] Input: Analysis results from the analysis engine.

[0481] What happens: The server formats the proposal and sends it to the device using the HTTPS protocol.

[0482] Output: The suggestion is sent to the device.

[0483] Step 13:

[0484] Your device will display suggestions

[0485] The device displays the received suggestions to the user.

[0486] Input: The proposal received from the server.

[0487] What it does: The device uses a user interface to display suggestions in list or card format.

[0488] Output: The suggestions are displayed on the screen.

[0489] Step 14:

[0490] User selects suggestion

[0491] Users select activities that interest them from the suggestions displayed.

[0492] Input: The displayed suggestion.

[0493] Specific Action: The user taps, clicks, or otherwise selects a suggestion.

[0494] Output: User selections are logged to the terminal.

[0495] Step 15:

[0496] The device sends the selection to the server

[0497] The device sends the user's selected suggestions to the server.

[0498] Input: User selection information.

[0499] Specific operation: The device sends the selection information to the server using the HTTPS protocol.

[0500] Output: The selection is sent to the server.

[0501] Step 16:

[0502] The server provides more information

[0503] The server retrieves detailed information (e.g., cafe reviews and discount information) based on the user's request and generates an answer.

[0504] Input: User selection information.

[0505] What happens: The server uses a database or external API to retrieve the details, format them, and send them.

[0506] Output: Detailed information is sent to the terminal.

[0507] Step 17:

[0508] The device displays detailed information

[0509] The terminal displays detailed information from the server to the user.

[0510] Input: The details received from the server.

[0511] Specific operation: The device displays detailed information on the interface.

[0512] Output: Detailed information is displayed to the user.

[0513] Step 18:

[0514] User performs an activity

[0515] The user performs a selected activity (e.g., visits a cafe).

[0516] Input: Proposal and details.

[0517] Concrete Action: The user physically performs an activity.

[0518] Output: Feedback to the user about the activity's execution status.

[0519] Step 19:

[0520] The device sends the execution status to the server

[0521] The device sends the user's selections and execution status to the server.

[0522] Input: Execution status data.

[0523] Specific operation: The device uses the HTTPS protocol to send execution status data to the server.

[0524] Output: Execution status data is sent to the server.

[0525] Step 20:

[0526] Server stores history and feedback

[0527] The server stores the user's selection history and feedback in a database, which improves the accuracy of suggestions in future.

[0528] Input: Execution status data and user feedback.

[0529] What happens: The server saves the data in a database and uses it for the next suggestion generation.

[0530] Output: Updated history and feedback data is saved in the database.

[0531] (Application example 2)

[0532] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0533] In modern society, there is a demand for systems that allow people to make effective use of their idle time. In particular, there is a need to improve the user experience by suggesting appropriate digital content that matches the user's emotions and mood. The purpose of this invention is to provide a personalized content suggestion system that meets these needs.

[0534] The specification process by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving information such as the user's interests, free time, location, mood, and emotions, means for acquiring weather information and trend information, means for analyzing the past selection history and feedback of the user and similar users, means for generating optimal digital content suggestions for the user based on the analysis results, means for providing the generated digital content suggestions to the user, means for recognizing emotions from the user's voice and facial expression data, means for personalizing the suggestions based on the emotion data, and means for acquiring and providing detailed information about the generated suggestions. This allows the user to spend their idle time effectively and fulfillingly.

[0535] "User interests" refers to information about themes and categories in which a user is interested and shows interest.

[0536] "Free time" is the extra time that a user can use freely.

[0537] "Location" refers to the specific location or area where the user is currently located.

[0538] A "mood" is a user's temporary emotional or mental state.

[0539] "Emotion" refers to the emotional state detected from the user's voice and facial expressions.

[0540] "Weather Information" means data about the climate and weather in a user's current location or a specified area.

[0541] "Trend information" is data about current trends and popular content.

[0542] "Past selection history of the user and similar users" refers to the history of content selected in the past by the user and similar users.

[0543] "Feedback" refers to evaluations and opinions regarding the services and content provided by users.

[0544] "Digital content" is information or entertainment provided in electronic form, such as videos, music, or articles.

[0545] "Digital content proposal" means proposing digital content that is considered to be optimal for the user.

[0546] A "server" is a central computer system that manages data, analyzes it, and generates proposals.

[0547] "Detailed information about the proposed content" is more specific information about the proposed content.

[0548] The present invention is a system that proposes personalized digital content to enrich users' idle time. This system includes a user terminal, a server, an external information service, and an emotion recognition engine. The user terminal is assumed to be a device such as a smartphone or tablet, and the server functions as a central management system. The external information service is an Internet resource that provides weather information and trend information. The emotion recognition engine is a system that recognizes emotions from the user's voice and facial expressions and reflects them in the proposals.

[0549] The server includes means for receiving information such as the user's interests, free time, location, mood, and emotions, means for acquiring weather information and trend information, means for analyzing the past selection history and feedback of the user and similar users, means for generating optimal digital content suggestions for the user based on the analysis results, means for providing the generated digital content suggestions to the user, means for recognizing emotions from the user's voice and facial expression data, means for personalizing the suggestions based on the emotion data, and means for acquiring and providing detailed information about the generated suggestions.

[0550] First, users open the application and select the "Personalized Recommendations" function. Then, the user's device displays an interface for inputting the user's interests, free time, location, mood, and emotions. This interface includes selection forms and input forms, where the user can enter and submit this information.

[0551] The server receives the information sent by the user and updates the user profile. It then uses a weather information API to obtain the latest weather data for the user's current location and a trend information API to obtain information on currently popular content. The server also analyzes past selection history and feedback, and passes this data to a machine learning algorithm to generate optimal content suggestions. TensorFlow and other similar algorithms are suitable for use.

[0552] In addition, an emotion recognition engine is used to recognize emotions from the user's voice and facial expression data, and the suggestions are further personalized based on that. An emotion recognition engine such as Microsoft Azure Emotion API is suitable for this. The generated suggestions, including detailed information, are sent to the user's device. The user can then review the suggestions, select and play content that interests them, and receive detailed information and responses to questions.

[0553] For example, consider a scenario where a user opens the app while waiting for a train at a station and selects the "Personalized Recommendations" feature. After the user enters and submits information such as "Movies, Music" or "30 minutes," the server receives the information and updates the profile. The app then checks the current weather using a weather information API, obtains content information from a trend information API, and adds emotional data from an emotion recognition engine for analysis. Finally, the app suggests relaxing short videos or music, allowing the user to select and watch them, effectively spending their idle time.

[0554] Examples of prompts are:

[0555] User: Movies, Music

[0556] Free time: 30 minutes

[0557] Current location: Shinjuku Station

[0558] Mood: Relax

[0559] Emotions: Feeling stressed

[0560] Prompt: Based on the user information above, recommend appropriate videos, music, and articles. Use weather and trending APIs and consider sentiment information from the sentiment engine.

[0561] This allows users to spend their idle time effectively and enjoy a wealth of personalized content.

[0562] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0563] Step 1:

[0564] A user opens the application and selects the "Personalized Recommendations" feature, which displays the application's main interface.

[0565] Step 2:

[0566] The device displays an interface for inputting the user's interests, availability, location, mood, and emotions, including selection and input forms.

[0567] Input: User interests, availability, location, mood, emotions

[0568] Output: The input form is displayed on the terminal.

[0569] Step 3:

[0570] The user enters the required information and presses the send button, which sends the entered data from the device to the server.

[0571] Input: Information entered by the user

[0572] Output: User information sent to the server

[0573] Step 4:

[0574] The server receives the information sent by the user and updates the user profile, ensuring the system reflects the user's most current information.

[0575] Input: User information sent to the server

[0576] Output: Updated user profile

[0577] Step 5:

[0578] The server calls the weather information API to obtain the latest weather data for the user's current location, which is then stored on the server for analysis.

[0579] Input: current user location

[0580] Output: Retrieved weather information

[0581] Step 6:

[0582] The server calls the trend information API to obtain information on currently popular content, which is then stored in the server for analysis.

[0583] Input: Request to get information on currently trending content

[0584] Output: Obtained trend information

[0585] Step 7:

[0586] The server retrieves the user's past selection history and feedback from the database and passes this information to the data analysis engine.

[0587] Input: User's past selections and feedback

[0588] Output: Passed user history information

[0589] Step 8:

[0590] The emotion engine recognizes emotions from the user's voice and facial expression data, and the recognized emotion information is sent to the server.

[0591] Input: User's voice and facial expression data

[0592] Output: Recognized emotion information

[0593] Step 9:

[0594] The server adds the emotion information received from the emotion engine to the user profile and passes it to the analysis engine.

[0595] Input: Emotion information from the emotion engine

[0596] Output: Updated user profile

[0597] Step 10:

[0598] The server integrates weather data, trend data, emotion data, and user profiles, and analyzes them using an analytics engine. Machine learning algorithms (such as TensorFlow) are used to analyze the data and generate optimal digital content recommendations for users.

[0599] Input: Weather data, Trend data, Emotion data, User profile

[0600] Output: Generated digital content proposal

[0601] Step 11:

[0602] The server obtains detailed information about the proposal (e.g., video thumbnail, music artist name, article headline), formats the generated proposal, and sends it to the terminal.

[0603] Input: Generated digital content proposal

[0604] Output: Submitted proposal with details

[0605] Step 12:

[0606] The device displays the received suggestions to the user, visualizing them in list or card format.

[0607] Input: Submitted proposal with detailed information

[0608] Output: Display of the visualized proposal

[0609] Step 13:

[0610] Users can review the suggestions, select and play content that interests them, and use the Q&A feature to find out more information.

[0611] Input: Visualized proposal

[0612] Output: User selection and request for more information

[0613] Step 14:

[0614] The device sends the user's selections and questions to the server, which obtains detailed information about the user's question, generates an answer, and sends it to the device.

[0615] Input: User choices and questions

[0616] Output: Detailed information provided

[0617] Step 15:

[0618] The device displays the answer to the user, allowing them to get the details they need.

[0619] Input: Details to be provided

[0620] Output: Detailed information displayed to the user

[0621] Step 16:

[0622] The user executes the content that they have finally selected. The device sends the user's selection and execution status to the server.

[0623] Input: Last selected content

[0624] Output: Selections and execution status sent to the server

[0625] Step 17:

[0626] The server stores the user's selection history and feedback in a database and uses this information to improve the accuracy of future suggestions.

[0627] Input: User selection history and feedback

[0628] Output: Saved selection history and feedback

[0629] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0630] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0631] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0632] [Second embodiment]

[0633] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0634] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0635] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0636] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0637] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0638] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0639] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0640] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0641] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0642] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0643] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0644] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0645] The present invention is a system for making effective use of users' idle time. This system generates optimal time-killing suggestions based on information such as the user's interests, free time, location, and mood, taking into account weather and event information. The specific system configuration and processing flow are described below.

[0646] System Configuration

[0647] This system consists of a user terminal, a server, and external information services (weather information API, event information API, etc.). User terminals are assumed to be devices such as smartphones and tablets. The server functions as a central management system, collecting, analyzing, and generating proposals for various types of data. The external information services are Internet resources that provide weather and event information.

[0648] Program processing

[0649] Enter and submit user information

[0650] 1. The user opens the application and selects the "Time-killing suggestions" feature.

[0651] 2. The device displays an interface for entering information such as the user's interests, availability, location, and mood, which may include selection forms or input forms.

[0652] 3. When the user enters the required information and presses the send button, the information is sent from the device to the server.

[0653] Data collection and analysis

[0654] 4. The server receives the information sent by the user and updates the user profile.

[0655] 5. The server calls the weather information API to obtain the latest weather data for the user's current location.

[0656] 6. The server calls the event information API and obtains information about events being held in the user's current location.

[0657] 7. The server retrieves the user's and similar users' past selection history and feedback from the database and passes it to the analysis engine.

[0658] Proposal Generation

[0659] 8. The server integrates weather data, event data, and user profiles and uses an analytics engine to generate optimal suggestions for users, using machine learning algorithms and a recommendation system based on similar user histories.

[0660] 9. The server formats the offer with detailed information (location, time, discounts, reviews, etc.) and sends it to the device.

[0661] Proposal presentation and user interaction

[0662] 10. The device displays the received suggestions to the user. The suggestions can be visualized in a list format, card format, or other format.

[0663] 11. Users review the suggestions and select the activities they are interested in. They can also use the Q&A feature if they need further clarification.

[0664] 12. The device sends the user's selections and questions to the server.

[0665] Data provision and storage

[0666] 13. The server obtains detailed information about the user's question, generates an answer, and sends it to the device.

[0667] 14. The device displays the answer to the user.

[0668] 15. The user finally performs the selected activity.

[0669] 16. The terminal sends the user's selections and execution status to the server, which stores them in a database.

[0670] 17. The server collects user feedback and uses it to improve the accuracy of suggestions in future.

[0671] Specific examples

[0672] Killing time at the station

[0673] 1. While waiting for a train at a station, the user opens the app and selects the "Time-killing suggestions" feature.

[0674] 2. The device will display a screen asking about your interests.

[0675] 3. The user enters "cafe hopping," "30 minutes," and "current location is the station," and submits.

[0676] 4. The server receives this information and updates the profile.

[0677] 5. The server calls the weather information API and confirms that the current weather is "sunny."

[0678] 6. The server calls the event information API and obtains event information around the station.

[0679] 7. The server integrates the data with past data and generates suggestions for new cafe coupons, art exhibitions near train stations, and the latest movie screenings.

[0680] 8. The server sends the suggestions to the device, which displays them to the user.

[0681] 9. The user selects a cafe coupon and views more details.

[0682] 10. The device sends the question to the server, which provides a detailed review.

[0683] 11. The user finally selects a cafe and decides to visit.

[0684] 12. Your device sends your selections to the server, which stores your history and feedback.

[0685] The above is an embodiment of the present invention, which allows users to easily discover new activities and have interesting experiences.

[0686] The processing flow will be explained below.

[0687] Step 1:

[0688] The user opens the application and selects the "Time-killing suggestions" feature. The device then displays an interface for entering information such as the user's interests, free time, location, and mood.

[0689] Step 2:

[0690] The user enters the necessary information and presses the send button. The device sends the information entered by the user to the server.

[0691] Step 3:

[0692] The server receives user information (interests, free time, location, mood, etc.) sent from the device and updates the user profile.

[0693] Step 4:

[0694] The server calls the weather information API to obtain the latest weather data for the user's current location, and stores the obtained weather information on the server.

[0695] Step 5:

[0696] The server calls the event information API to obtain information about events taking place in the user's current location, and stores the obtained event information on the server.

[0697] Step 6:

[0698] The server retrieves the user's and similar users' past selection history and feedback from the database and passes it to the analysis engine, which then begins the analysis process.

[0699] Step 7:

[0700] The server integrates weather data, event data, and user profiles, and runs the analysis through an analytics engine that uses machine learning algorithms and a recommendation system based on similar user history to generate optimal suggestions for users.

[0701] Step 8:

[0702] The server formats the generated suggestions by adding details (e.g., cafe location, hours, discounts, reviews, event location and time, etc.) and prepares the suggestions for delivery to the user.

[0703] Step 9:

[0704] The server sends the prepared suggestions to the device, which then displays them to the user. The suggestions are visualized in a list format, card format, or other format.

[0705] Step 10:

[0706] Users can review suggestions, select activities they are interested in, and use the Q&A feature to find out more information.

[0707] Step 11:

[0708] The device sends the user's selections and questions to the server, which retrieves relevant information from a database and generates an answer.

[0709] Step 12:

[0710] The server generates a response and sends it to the device, which displays it to the user, providing them with the necessary details.

[0711] Step 13:

[0712] The user then performs the activity they have finally selected (e.g., visiting a cafe, attending an event, watching a movie, etc.). The device then sends the user's selection and the execution status to the server.

[0713] Step 14:

[0714] The server stores the user's selection history and feedback in a database, which is used to improve the accuracy of suggestions in future.

[0715] Step 15:

[0716] After completing an activity, the user inputs and sends a simple feedback. The device sends the feedback to the server, which ends the user interaction.

[0717] Example 1

[0718] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0719] In modern society, it is a difficult task to effectively utilize users' free time and suggest activities appropriate for that time. Conventional systems often fail to provide suggestions that are in line with the user's interests and current situation, leaving users unable to make satisfying choices. The present invention aims to solve this problem and provide users with optimal time-killing suggestions.

[0720] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0721] In this invention, the server includes means for receiving information such as the user's interests, free time, location, and mood, means for acquiring weather information and event information, means for analyzing the user's and similar users' past selection history and feedback, means for integrating the weather data, event data, and user profile and generating suggestions using an analysis engine, means for formatting detailed information about the suggestions (location, time, discount details, reviews, etc.) and sending them to the terminal, means for receiving the user's questions and providing related information, means for sending the user's selections and questions to the server, means for saving the user's selection history and feedback to improve the accuracy of suggestions from next time onwards, and means for collecting user feedback and saving it in a database, thereby making it possible to suggest activities that are optimal for the user's free time.

[0722] "Means for receiving information such as user interests, free time, location, and mood" refers to an interface that allows users to input their interests, free time, current location, mood, etc. through the application, and the function for collecting that information.

[0723] "Means for obtaining weather information and event information" refers to API call functions and data collection functions that obtain current weather and nearby event information from external weather information services and event information services.

[0724] "Means for analyzing the past selection history and feedback of users and similar users" refers to a function for analyzing the past selection history and feedback stored in a database using a machine learning algorithm.

[0725] "Means for integrating weather data, event data, and user profiles and generating suggestions using an analytical engine" refers to a function that combines acquired weather data, event information, and user profiles and generates optimal suggestions using an analytical engine.

[0726] "Means for formatting detailed information about the proposal (location, time, discount details, reviews, etc.) and sending it to the terminal" refers to a function that adds detailed information about the proposal, such as location information, time, discount details, reviews, etc., to the generated proposal, formats it, and sends it to the user's terminal.

[0727] "Means for receiving user questions and providing relevant information" refers to a function that receives questions from users and provides appropriate answer information or additional detailed information.

[0728] The "means for transmitting user selections and questions to the server" is a function for transmitting information about a user's selection of an activity or question to the server.

[0729] "Means for saving user selection history and feedback to improve the accuracy of future suggestions" refers to a function that saves the user's selections and subsequent feedback in a database and uses that data when generating suggestions the next time.

[0730] "Means for collecting user feedback and saving it in a database" refers to a function that collects feedback on the activities experienced by users after they have entered it and saves it in a database.

[0731] The present invention is a system for making effective use of a user's free time, and generates optimal time-killing suggestions based on information such as the user's interests, free time, location, and mood, taking into account weather and event information. The specific system configuration and processing flow are described below.

[0732] System Configuration

[0733] This system consists of a user terminal, a server, and an external information service. The user terminal is assumed to be a device such as a smartphone or tablet. The server functions as a central management system, collecting, analyzing, and generating proposals for various data. The external information service is an Internet resource that provides weather and event information.

[0734] Hardware and software used

[0735] User devices are smartphones or tablets. The server is configured as a system including a database server, application server, and analysis engine. The weather information API uses the OpenWeatherMap API, and the event information API uses the Eventbrite API. The analysis engine implements a machine learning algorithm using Scikit-learn.

[0736] Program processing flow

[0737] 1. The user opens the application and selects the "Time-killing suggestions" feature.

[0738] 2. The device displays an interface for entering information such as the user's interests, availability, location, and mood, which may include selection forms or input forms.

[0739] 3. When the user enters the required information and presses the send button, the information is sent from the device to the server.

[0740] 4. The server receives the information sent by the user and updates the user profile.

[0741] 5. The server calls the weather information API to obtain the latest weather data for the user's current location.

[0742] 6. The server calls the event information API and obtains information about events being held in the user's current location.

[0743] 7. The server retrieves the user's and similar users' past selection history and feedback from the database and passes it to the analysis engine.

[0744] 8. The server integrates weather data, event data, and user profiles and uses an analytics engine to generate optimal suggestions for users, using machine learning algorithms and a recommendation system based on similar user histories.

[0745] 9. The server formats the offer with detailed information (location, time, discounts, reviews, etc.) and sends it to the device.

[0746] 10. The device displays the received suggestions to the user. The suggestions can be visualized in a list format, card format, or other format.

[0747] 11. Users review the suggestions and select the activities they are interested in. They can also use the Q&A feature if they need further clarification.

[0748] 12. The device sends the user's selections and questions to the server.

[0749] 13. The server obtains detailed information about the user's question, generates an answer, and sends it to the device.

[0750] 14. The device displays the answer to the user.

[0751] 15. The user finally performs the selected activity.

[0752] 16. The terminal sends the user's selections and execution status to the server, which stores them in a database.

[0753] 17. The server collects user feedback and uses it to improve the accuracy of suggestions in future.

[0754] Specific examples

[0755] Killing time at the station

[0756] 1. While waiting for a train at a station, the user opens the app and selects the "Time-killing suggestions" feature.

[0757] 2. The device will display a screen asking about your interests.

[0758] 3. The user enters "cafe hopping," "30 minutes," and "current location is the station" and submits.

[0759] 4. The server receives this information and updates the profile.

[0760] 5. The server calls the weather information API and confirms that the current weather is "sunny."

[0761] 6. The server calls the event information API and obtains event information around the station.

[0762] 7. The server integrates the data with past data and generates suggestions for new cafe coupons, art exhibitions near train stations, and the latest movie screenings.

[0763] 8. The server sends the suggestions to the device, which displays them to the user.

[0764] 9. The user selects the cafe coupon and views more details.

[0765] 10. The device sends the question to the server, which provides a detailed review.

[0766] 11. The user finally selects a cafe and decides to visit.

[0767] 12. The device sends the selection to the server, which stores the history and feedback.

[0768] Example prompts for generative AI models

[0769] "I'm currently waiting for a train at the station. I'd like to take advantage of the time I have to spend at a cafe. Do you have any recommendations? The conditions are that the weather is currently sunny and the journey will take about 30 minutes."

[0770] The above is an embodiment of the present invention, which allows users to easily discover new activities and have interesting experiences.

[0771] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0772] Step 1:

[0773] The user opens the application and selects the "Time-killing suggestions" function. The input is a user action, and the output is the launch of the application. The specific action is that the user taps the smartphone icon to launch the app.

[0774] Step 2:

[0775] The device displays an interface for entering information such as the user's interests, free time, location, and mood. The input is a signal to start the application, and the output is a screen for entering user information. Specifically, the application presents the user with options and a form for entering "interests," "free time," "location," and "mood."

[0776] Step 3:

[0777] The user enters the necessary information and presses the send button. The input is the information entered by the user (interests, free time, location, mood), and the output is sent from the device to the server. Specifically, the user enters "cafe hopping," "30 minutes," "current location," and "relax," and taps the "send" button.

[0778] Step 4:

[0779] The server receives the information submitted by the user and updates the user profile. The input is the information submitted by the user, and the output is the updated profile, which is saved in the database. Specifically, the server accesses the database and executes an SQL query to update the user profile.

[0780] Step 5:

[0781] The server calls the weather information API to obtain the latest weather data for the user's current location. The input is the user's current location information, and the output is the latest weather data. Specifically, the server sends a request to the OpenWeatherMap API and receives the current weather information (e.g., "Sunny, temperature 24 degrees").

[0782] Step 6:

[0783] The server calls the event information API to obtain information about events taking place in the user's current location. The input is the user's current location information, and the output is event information. Specifically, the server sends a request to the Eventbrite API and receives information about events in the area.

[0784] Step 7:

[0785] The server retrieves the past selection history and feedback of the user and similar users from the database and passes it to the analysis engine. The input is the user ID, and the output is the selection history and feedback data. Specifically, the server executes an SQL query to retrieve the past history and feedback from the database and passes it to the analysis engine.

[0786] Step 8:

[0787] The server integrates weather data, event data, and user profiles and uses an analytics engine to generate optimal suggestions for users. Weather data, event data, and user profiles are input, and generated suggestions are obtained as output. Specifically, the server runs a machine learning algorithm using Scikit-learn to generate optimal suggestions.

[0788] Step 9:

[0789] The server formats the proposal with detailed information (location, time, discount details, reviews, etc.) and sends it to the device. The generated proposal is the input, and the formatted proposal is sent to the device as the output. Specifically, the server formats the proposal content in JSON format and sends it to the device via an HTTP request.

[0790] Step 10:

[0791] The device displays the received suggestions to the user.,The input is the suggestion sent from the server, and the,output is the suggestion displayed on the screen.,Specifically, the device visualizes the suggestion,contents in list or card format and displays them to the,user.

[0792] Step 11:

[0793] The user reviews the suggestions and selects the activity they are interested in. If additional explanation is needed, they can use the question and answer function. The suggestions are displayed as input, and the selected activity and questions are sent to the server as output. The user taps the activity card to view more information.

[0794] Step 12:

[0795] The device sends the user's selections and questions to the server. The input is the user's selections and questions, and the output is the information sent to the server. Specifically, the device sends the selected activity ID and the question to the server.

[0796] Step 13:

[0797] The server obtains detailed information about the user's question, generates an answer, and sends it to the terminal. The user's question is the input, and the generated answer is sent to the terminal as the output. In concrete terms, the server obtains the necessary information from the database, generates the answer, and sends it to the terminal.

[0798] Step 14:

[0799] The terminal displays the answer to the user. The answer from the server is input, and the answer is displayed to the user as output. Specifically, the terminal displays the answer on the screen.

[0800] Step 15:

[0801] The user ultimately performs the selected activity. The input is the proposed activity, and the output is the execution of that activity. The concrete action is the user actually performing the selected activity.

[0802] Step 16:

[0803] The terminal sends the user's selection and execution status to the server. The user's selection and execution status are input, and are sent to the server as output. Specifically, the terminal sends information about the executed activity to the server.

[0804] Step 17:

[0805] The server collects user feedback and uses it to improve the accuracy of future proposals. The input is user feedback, and the output is saved in a database. Specifically, the server saves the feedback information in the database and uses it to generate future proposals.

[0806] (Application example 1)

[0807] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0808] In today's modern living environment, users often waste their time waiting or having free time. Particularly in physical stores, there are few ways to efficiently find available activities and promotions, and users cannot easily obtain information that interests them. This results in users not making effective use of their valuable time and a decrease in the quality of their experience.

[0809] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0810] In this invention, the server includes means for receiving information such as the user's interests, free time, location, and mood, means for acquiring weather information and event information, and means for analyzing the past selection history and feedback of the user and similar users, thereby enabling a user interface means for generating optimal suggestions for the user based on the analysis results, displaying the generated suggestions on the user terminal, providing detailed information and reviews, and providing detailed information and navigation for the activity selected by the user.

[0811] "User Terminal" means a smartphone, tablet or other mobile device through which a User inputs information and receives suggestions.

[0812] A "sensor" is a device used as a means of obtaining information such as a user's interests, free time, location, and mood.

[0813] "Weather information" is information that provides data related to the weather, and is obtained from an external information providing service.

[0814] "Event information" is information about events and activities related to a specific location and period, and is obtained from an external information providing service.

[0815] A "user profile" is a data structure that includes information such as a user's interests, availability, location, mood, past selections, and feedback.

[0816] "Analysis results" are information generated by the analysis engine based on weather information, event information, user profiles, etc., and include optimal suggestions for users.

[0817] A "user interface" is a screen or mechanism for displaying information and suggestions to users and receiving input and feedback from users.

[0818] "Navigation" is a function that shows and guides the user to the location of the activity selected by the user.

[0819] "Suggestions" are suggested time-killing activities or promotions that are generated based on the analysis results and provided to users.

[0820] "Details" means specific information related to the proposed activity, such as location, time, discounts, reviews, etc.

[0821] System Overview

[0822] This invention is designed as a system for effectively utilizing users' idle time, and is intended for use in brick-and-mortar stores in particular. The system consists of a user terminal, a server, and an external information service. The user terminal corresponds to a smartphone or tablet, and the server includes an analysis engine, a database, and a proposal generation system. The external information service includes a weather information API and an event information API. This allows the system to generate and execute time-killing proposals in real time that are optimized for the user's current situation and preferences.

[0823] Program structure and main processing

[0824] 1. Enter and submit user information

[0825] User device: The user launches the application and selects the "Time-killing suggestions" function. At this point, an interface is displayed for entering information such as the user's interests, free time, location, and mood. This interface consists of a selection form and an input form. The entered information is sent from the device to the server.

[0826] 2. Data collection and analysis

[0827] Server: The server receives the information sent by the user and updates the user profile. It then calls the weather information API to get the latest weather data for the user's current location, calls the event information API to get information about events taking place in the user's current location, and retrieves the user's and similar users' past selection history and feedback from the database and passes them to the analytics engine.

[0828] 3. Proposal Generation

[0829] Server: The server integrates weather data, event data, and user profiles and uses an analytics engine to generate optimal suggestions for users. The suggestions are generated using machine learning algorithms and a recommendation system based on similar user history. The generated suggestions are then sent to the user's device along with detailed information (location, time, discounts, reviews, etc.).

[0830] 4. Proposal Presentation and User Interaction

[0831] User device: The user device displays the received suggestions to the user. The suggestions are visualized in a list format, card format, etc. The user reviews the suggestions and selects the activities they are interested in. If further explanation is needed, the user can use the question and answer function. The device sends the user's selections and questions to the server.

[0832] 5. Data provision and storage

[0833] Server: The server obtains detailed information about the user's question, generates an answer, and sends it to the device. The device displays the answer to the user. The user then performs the selected activity, and the device sends subsequent feedback and selection history to the server. The server stores this information in a database and uses it to improve the accuracy of future suggestions.

[0834] Hardware and software used

[0835] Hardware: Servers, smartphones, tablets

[0836] Software: Python, SQLite, scikit-learn (machine learning library), requests (HTTP request library)

[0837] Specific examples

[0838] For example, a user sets their current location to a "major station," their interests to "cafes" and "art galleries," and their free time to 60 minutes. Based on this information, the server retrieves cloudy weather information from a weather information API and supplements it with information on art events taking place around the station from an event information API. Suggestions are generated for Cafe ABC (10% off coupon) and Art Gallery XYZ (highly rated reviews) in front of the station. The user selects Cafe ABC and views detailed information (coupon usage guide, navigation).

[0839] Prompt Sentence Examples

[0840] Prompt: "A user is interested in cafes and art galleries, is located at a major train station, and has 60 minutes of free time. Generate suggestions for the best ways to kill time for him. The current weather is cloudy, and there are art events happening around the station. Include location, time, discounts, and reviews in the suggestions."

[0841] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0842] Step 1:

[0843] Enter and submit user information

[0844] A user launches the application and selects the "Time-killing suggestions" function. The user's device displays an interface for entering information such as interests, free time, location, and mood.

[0845] Input: User interests, availability, location, and mood information

[0846] Output: Sending information from the device to the server

[0847] Step 2:

[0848] Update your user profile

[0849] The server receives the information submitted by the user and updates the user profile.

[0850] Input: User information sent from the device

[0851] Output: Updated user profile

[0852] Step 3:

[0853] Get weather information

[0854] The server calls the weather information API to obtain the latest weather data for the user's current location.

[0855] Input: User's current location

[0856] Output: Current weather data for your current location

[0857] Step 4:

[0858] Get event information

[0859] The server calls the event information API and obtains information about events being held in the user's current location.

[0860] Input: User's current location

[0861] Output: Event information for the current location

[0862] Step 5:

[0863] Get past selection history and feedback

[0864] The server retrieves the user's and similar users' past selection history and feedback from the database and passes it to the analysis engine.

[0865] Input: User profile

[0866] Output: Past selection history and feedback

[0867] Step 6:

[0868] Analysis and proposal generation

[0869] The server combines weather data, event data, and user profiles, and uses an analytics engine to generate optimal suggestions for users, using machine learning algorithms and recommendation systems.

[0870] Input: Weather data, event data, user profile

[0871] Output: Best time-killing suggestions

[0872] Step 7:

[0873] Submit a proposal

[0874] The server sends the generated proposal to the user's device, along with detailed information (location, time, discounts, reviews, etc.).

[0875] Input: Best time-killing suggestions

[0876] Output: Proposal sent to user's device

[0877] Step 8:

[0878] Presenting the proposal

[0879] The user's device displays the received suggestions to the user. The suggestions are visualized in a list or card format, and the user can review the suggestions and select the activities that interest them.

[0880] Input: Proposal

[0881] Output: Suggestions shown to the user

[0882] Step 9:

[0883] Ask questions and get more information

[0884] If the user requests further clarification, the device sends the question to the server, which obtains detailed information about the user's question, generates an answer, and sends it to the device.

[0885] Input: User question

[0886] Output: Answer to the question

[0887] Step 10:

[0888] Final Selection and Navigation

[0889] It provides detailed information and navigation for the activity that the user ultimately selects. The user device sends the selection to the server, which stores the history and feedback.

[0890] Input: Last selected activity

[0891] Output: Activity details and navigation, saving selection history

[0892] Usage examples and prompt statements

[0893] As a concrete example, suppose a user sets their current location to "major station," their interests to "cafes" and "art galleries," and their free time to 60 minutes.

[0894] Prompt: "A user is interested in cafes and art galleries, is located at a major train station, and has 60 minutes of free time. Generate suggestions for the best ways to kill time for him. The current weather is cloudy, and there are art events happening around the station. Include location, time, discounts, and reviews in the suggestions."

[0895] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0896] This invention is a system that helps users make effective use of their idle time. This system generates optimal time-killing suggestions based on information such as the user's interests, free time, location, and mood, taking into account weather and event information. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it is possible to realize more personalized suggestions.

[0897] System Configuration

[0898] This system consists of a user terminal, a server, an external information provision service (weather information API, event information API, etc.), and an emotion engine. User terminals are assumed to be devices such as smartphones and tablets. The server functions as a central management system, collecting and analyzing various data and generating proposals. The external information provision service is an Internet resource for providing weather and event information. The emotion engine is a system that recognizes emotions from the user's voice and facial expressions and reflects them in proposals.

[0899] Program processing

[0900] Enter and submit user information

[0901] 1. The user opens the application and selects the "Time-killing suggestions" feature.

[0902] 2. The device displays an interface for inputting the user's interests, availability, location, mood, and emotions, including selection and input forms.

[0903] 3. When the user enters the required information and presses the send button, the information is sent from the device to the server.

[0904] Data collection and analysis

[0905] 4. The server receives the information sent by the user and updates the user profile.

[0906] 5. The server calls the weather information API to obtain the latest weather data for the user's current location and stores the obtained weather information on the server.

[0907] 6. The server calls the event information API to obtain information about events taking place in the user's current location. The obtained event information is stored on the server.

[0908] 7. The server retrieves the user's and similar users' past selection history and feedback from the database and passes it to the analysis engine, which then begins the analysis process.

[0909] Acquiring and analyzing emotion data

[0910] 8. The emotion engine acquires the user's voice and facial expression data, recognizes their emotions, and sends the emotion information obtained by the emotion engine to the server.

[0911] 9. The server adds the emotion information received from the emotion engine to the user profile and passes it to the analysis engine.

[0912] Proposal Generation

[0913] 10. The server integrates weather data, event data, emotion data, and user profiles and analyzes them in an analytics engine, which uses machine learning algorithms and a recommendation system based on similar user history to generate optimal suggestions for the user.

[0914] 11. The server formats the offer with detailed information (e.g., cafe location, hours, discounts, reviews, etc.) and sends it to the device.

[0915] Proposal presentation and user interaction

[0916] 12. The device displays the received suggestions to the user. The suggestions can be visualized in a list format, card format, or other format.

[0917] 13. Users can review the suggestions, select the activities they are interested in, and use the Q&A feature to find out more information.

[0918] 14. The device sends the user's selections and questions to the server.

[0919] 15. The server obtains detailed information about the user's question, generates an answer, and sends it to the device.

[0920] 16. The device displays the answer to the user, providing them with the necessary details.

[0921] Data provision and storage

[0922] 17. The user finally performs the selected activity (e.g., visit a cafe, attend an event, watch a movie, etc.). The device sends the user's selection and execution status to the server.

[0923] 18. The server stores the user's selection history and feedback in a database, which will be used to improve the accuracy of future suggestions.

[0924] 19. After the user completes the activity, they input and submit a brief feedback. The device sends the feedback to the server, which ends the user interaction.

[0925] Specific examples

[0926] Killing time at the station

[0927] 1. While waiting for a train at a station, the user opens the app and selects the "Time-killing suggestions" feature.

[0928] 2. The device will display a screen asking for your interests, free time, location, mood, and emotions.

[0929] 3. The user inputs "cafe hopping," "30 minutes," "Shinjuku Station," "want to relax," and "emotional data via voice input (e.g., feeling stressed based on the tone of voice)" and submits.

[0930] 4. The server receives this information and updates the profile.

[0931] 5. The server calls the weather information API and confirms that the current weather is "sunny."

[0932] 6. The server calls the event information API and obtains event information around Shinjuku Station.

[0933] 7. The server integrates the data with past data and adds data from the emotion engine for analysis.

[0934] 8. The server generates suggestions for new cafes, art exhibits near train stations, and the latest movie screenings. The suggestions are focused on relaxing content that reflects the user’s emotional state.

[0935] 9. The server sends the proposal to the device, which displays it to the user.

[0936] 10. The user selects a cafe coupon and views more details.

[0937] 11. The device sends the question to the server, which provides a detailed review.

[0938] 12. The user finally selects a cafe and decides to visit.

[0939] 13. Your device sends your selections to the server, which stores your history and feedback.

[0940] The above is an embodiment of the present invention. This makes it possible to propose new activities that take emotions into consideration, allowing users to enjoy new, personalized experiences without any effort.

[0941] The processing flow will be explained below.

[0942] Step 1:

[0943] The user opens the application and selects the "Time-killing suggestions" function. The device then displays an interface for inputting information such as the user's interests, free time, location, mood, and emotions.

[0944] Step 2:

[0945] The user enters their interests (e.g., cafes, movies, art exhibitions), free time (e.g., 30 minutes), location (e.g., Shinjuku Station), mood (e.g., wanting to relax), and emotion (e.g., feeling stressed), and then presses the send button. The device then sends the entered information to the server.

[0946] Step 3:

[0947] The server receives user information (interests, free time, location, mood, emotions, etc.) sent from the device and updates the user profile.

[0948] Step 4:

[0949] The server calls the weather information API to obtain the latest weather data for the user's current location, and stores the obtained weather information on the server.

[0950] Step 5:

[0951] The server calls the event information API to obtain information about events taking place in the user's current location, and stores the obtained event information on the server.

[0952] Step 6:

[0953] The server retrieves the user's and similar users' past selection history and feedback from the database and passes it to the analysis engine, which then begins the analysis process.

[0954] Step 7:

[0955] The emotion engine acquires emotional data from the user's voice and facial expressions, recognizes that the user is feeling stressed, and sends this emotional information to the server.

[0956] Step 8:

[0957] The server adds the emotional information received from the emotion engine to the user profile and passes it to the analysis engine, which then performs an analysis based on the user's current emotional state.

[0958] Step 9:

[0959] The server integrates weather data, event data, user profiles, and sentiment data, and runs the analysis through an analytics engine that generates optimal recommendations for each user based on machine learning algorithms and similar user history.

[0960] Step 10:

[0961] The server formats the generated suggestions by adding details (e.g., cafe location, opening hours, discounts, reviews, event location and time, etc.) and prepares the suggestions for delivery to the user.

[0962] Step 11:

[0963] The server sends the prepared suggestions to the device, which then displays them to the user. The suggestions are visualized in a list format, card format, or other format.

[0964] Step 12:

[0965] Users can review the suggestions and select activities they are interested in. They can also use the Q&A feature to find out more information.

[0966] Step 13:

[0967] The device sends the user's selections and questions to the server, which retrieves relevant information from a database and generates an answer.

[0968] Step 14:

[0969] The server generates a response and sends it to the device, which displays it to the user, providing them with the necessary details.

[0970] Step 15:

[0971] The user then performs the activity they have finally selected (e.g., visiting a cafe, attending an event, watching a movie, etc.). The device then sends the user's selection and the execution status to the server.

[0972] Step 16:

[0973] The server stores the user's selection history and feedback in a database, which is used to improve the accuracy of suggestions in future.

[0974] Step 17:

[0975] After completing an activity, the user inputs and sends a simple feedback. The device sends the feedback to the server, which ends the user interaction.

[0976] Example 2

[0977] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0978] Previous time-killing suggestion systems were unable to take into account the user's subjective feelings and therefore had a low level of personalization. Furthermore, it was difficult to obtain and reflect weather and event information in real time, which led to a lack of reliability in terms of whether the system would actually be useful to the user. Furthermore, the system was unable to improve the accuracy of suggestions by utilizing past selection history and feedback, which led to issues such as low user satisfaction.

[0979] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for receiving information such as the user's interests, free time, location, mood, and emotions; means for acquiring weather information and event information; means for analyzing the past selection history and feedback of the user and similar users; means for generating optimal suggestions for the user based on the analysis results; means for providing the generated suggestions to the user; means for recognizing emotions from the user's voice and facial expressions and reflecting the emotion data in suggestions; means for allowing the user to select activities that interest the user; and means for providing detailed information about the activities selected by the user. This enables personalized suggestions that take into account the user's subjective emotions and reflect real-time information.

[0980] "User Information" refers to data such as a user's interests, free time, location, mood, and emotions.

[0981] "Weather Information" refers to data regarding weather conditions in the User's current location.

[0982] "Event Information" refers to data about an event taking place at a particular location and time.

[0983] "Selection history" refers to a record of a user's past choices and actions.

[0984] "Feedback" refers to information regarding opinions and ratings from users.

[0985] An "emotion engine" refers to a system that recognizes emotions from the user's voice and facial expressions.

[0986] A "profile" refers to a data set that compiles information such as a user's attributes, behavioral history, and emotions.

[0987] "Analysis" refers to the data processing and analysis work required to generate optimal proposals based on collected data.

[0988] "Suggestions" refers to options for activities or actions presented to users based on the analysis results.

[0989] This invention is a system that helps users make effective use of their idle time by generating optimal time-killing suggestions based on information such as the user's interests, free time, location, mood, and emotions, taking into account weather and event information. Furthermore, by using an emotion engine that recognizes emotions from the user's voice and facial expressions, more personalized suggestions become possible.

[0990] This system consists of a user terminal, a server, an external information provision service (weather information API, event information API, etc.), and an emotion engine. User terminals are assumed to be devices such as smartphones and tablets. The server functions as a central management system, collecting and analyzing various data and generating proposals. The external information provision service is an Internet resource for providing weather and event information. The emotion engine is a system that recognizes emotions from the user's voice and facial expressions and reflects them in proposals.

[0991] Hardware and software used

[0992] Hardware:

[0993] Smartphone

[0994] tablet

[0995] server

[0996] software:

[0997] Weather information API (e.g. OpenWeatherMap API)

[0998] Event information API (e.g. Eventbrite API)

[0999] Emotion recognition API (e.g. Microsoft Azure Cognitive Services)

[1000] Machine learning engines (e.g. TensorFlow)

[1001] Program processing

[1002] Enter and submit user information

[1003] A user opens the application on their smartphone or tablet and selects the "Time-killing suggestions" function. The device then displays an interface for inputting the user's interests, free time, location, mood, and emotions. This can include checkboxes, drop-down menus, text boxes, and other formats. Once the user enters the information and presses the submit button, the device sends the information to the server. The transmission is secured using the HTTPS protocol.

[1004] Data collection and analysis

[1005] The server receives the information sent by the user and updates the user profile. The server then calls a weather information API to obtain the latest weather data for the user's current location, for example, using the OpenWeatherMap API. The server then calls an event information API to obtain information about events taking place in the user's current location, for example, using the Eventbrite API. In addition, the server retrieves the user's and similar users' past selection history and feedback from a database and passes this information to the analytics engine.

[1006] Acquiring and analyzing emotion data

[1007] The emotion engine acquires the user's voice and facial expression data and recognizes their emotions. For example, it uses the emotion recognition API from Microsoft Azure Cognitive Services. The emotion information obtained by the emotion engine is sent to the server, which adds it to the user profile. This data is also passed to the analysis engine.

[1008] Proposal Generation

[1009] The server integrates weather data, event data, emotion data, and user profiles, and analyzes them using an analysis engine. The analysis engine uses machine learning algorithms (e.g., TensorFlow). Based on the analysis results, the server generates optimal suggestions for the user, adds detailed information, and sends them to the device.

[1010] Proposal presentation and user interaction

[1011] The device displays the received suggestions to the user. The suggestions are visualized in a list or card format. The user can review the suggestions and select activities that interest them. A question and answer function is also available for further information. The selections and questions are sent from the device to the server, which then provides further information.

[1012] Specific examples

[1013] Killing time at the station

[1014] While waiting for a train at a station, the user opens the app and selects the "Time-Killing Suggestions" feature. The device displays a screen asking for "Interests," "Free Time," "Current Location," "Mood," and "Emotions." The user inputs and submits "Cafe Hopping," "30 Minutes," "Shinjuku Station," "Want to Relax," and emotional data via voice input (voice tone indicates stress). The server receives this information and updates the profile. The server then calls the weather information API and confirms that the current weather is "sunny." The server then calls the event information API to obtain event information around Shinjuku Station. The server then combines this data with past data and adds data from the emotion engine for analysis. The server generates suggestions for new cafe coupons, art exhibitions near the station, and the latest movie screenings. The suggestions are focused on relaxing content tailored to the user's emotional state. The server sends the suggestions to the device, which then displays them to the user. The user selects a cafe coupon and browses for further details. The device then sends a question to the server, which provides a detailed review. The user finally selects a cafe and decides to visit. The device then sends the selection to the server, which stores the history and feedback. This will improve the accuracy of proposals from next time onwards.

[1015] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1016] Step 1:

[1017] User enters information

[1018] Users open the application on their smartphone or tablet and select the "Time-killing suggestions" function.

[1019] Input: Users input their interests, availability, location, mood, or emotions within the app, for example using checkboxes, drop-down menus, or text boxes.

[1020] Specific operation: The user inputs "cafe," "30 minutes," "Shinjuku Station," "relax," and "stress by tone of voice."

[1021] Output: The input data is stored in the user's device memory.

[1022] Step 2:

[1023] The device sends information

[1024] The terminal transmits the information entered by the user to the server.

[1025] Input: Data entered by the user.

[1026] What it does: The device uses the HTTPS protocol to communicate securely and send data to the server.

[1027] Output: The server receives the information sent by the user.

[1028] Step 3:

[1029] The server updates the user profile

[1030] The server records and updates the information received from the user as a user profile.

[1031] Input: User-entered data.

[1032] What happens: The server updates the profile using an SQL query against the database.

[1033] Output: Updated user profile data is saved to the database.

[1034] Step 4:

[1035] The server retrieves weather information

[1036] The server calls the weather information API and obtains the latest weather information for the current location (Shinjuku Station).

[1037] Input: User's current location.

[1038] Specific behavior: The server sends a request to the API endpoint to retrieve weather information.

[1039] Output: The latest weather information is saved on the server.

[1040] Step 5:

[1041] The server retrieves the event information

[1042] The server calls the event information API and obtains information about events being held at the current location (Shinjuku Station).

[1043] Input: User's current location.

[1044] Specific operation: The server sends a request to the API endpoint to obtain event information.

[1045] Output: The acquired event information is saved on the server.

[1046] Step 6:

[1047] The server retrieves the past selection history

[1048] The server retrieves the user's and similar users' past selection history and feedback from a database.

[1049] Input: User ID or similar user group.

[1050] What happens: The server executes an SQL query to get the required data.

[1051] Output: Past selection history and feedback are captured on the server.

[1052] Step 7:

[1053] The server passes the data to the analysis engine

[1054] The server passes the collected information to the analysis engine and begins analysis.

[1055] Input: User input, weather information, event information, past selections, feedback.

[1056] Specific operation: The server inputs the data into an analysis engine (e.g., TensorFlow) and performs the analysis.

[1057] Output: The analysis results are generated.

[1058] Step 8:

[1059] Emotion engine recognizes emotions

[1060] The emotion engine captures the user's voice data and recognizes emotions.

[1061] Input: User's voice data.

[1062] What it does: The emotion engine runs a speech recognition system to identify emotions (e.g., determine stress levels).

[1063] Output: Recognized emotion data is generated.

[1064] Step 9:

[1065] The emotion engine sends emotion information to the server

[1066] The emotion engine sends the recognized emotion information to the server.

[1067] Input: Recognized emotion data.

[1068] Specific operation: The emotion engine sends emotion data to the server using the HTTPS protocol.

[1069] Output: Emotion information is received on the server side.

[1070] Step 10:

[1071] The server adds emotional information to the profile

[1072] The server adds the emotional information to the user profile and passes it to the analysis engine.

[1073] Input: Recognized emotion data.

[1074] Specific operation: The server adds the emotion data to the database and updates the profile.

[1075] Output: Passes updated user profile data to the analytics engine.

[1076] Step 11:

[1077] The server integrates and analyzes the data

[1078] The server integrates weather data, event data, emotion data, and user profiles, and performs analysis using an analytical engine.

[1079] Input: All integrated data (weather information, event information, emotion data, user profile).

[1080] Specific operations: The server runs the analytical algorithm and performs data analysis.

[1081] Output: The learned suggestions are generated.

[1082] Step 12:

[1083] The server generates a proposal and sends it to the device

[1084] The server generates optimal suggestions for the user based on the analysis results, adds detailed information, and sends them to the device.

[1085] Input: Analysis results from the analysis engine.

[1086] What happens: The server formats the proposal and sends it to the device using the HTTPS protocol.

[1087] Output: The suggestion is sent to the device.

[1088] Step 13:

[1089] Your device will display suggestions

[1090] The device displays the received suggestions to the user.

[1091] Input: The proposal received from the server.

[1092] What it does: The device uses a user interface to display suggestions in list or card format.

[1093] Output: The suggestions are displayed on the screen.

[1094] Step 14:

[1095] User selects suggestion

[1096] Users select activities that interest them from the suggestions displayed.

[1097] Input: The displayed suggestion.

[1098] Specific Action: The user taps, clicks, or otherwise selects a suggestion.

[1099] Output: User selections are logged to the terminal.

[1100] Step 15:

[1101] The device sends the selection to the server

[1102] The device sends the user's selected suggestions to the server.

[1103] Input: User selection information.

[1104] Specific operation: The device sends the selection information to the server using the HTTPS protocol.

[1105] Output: The selection is sent to the server.

[1106] Step 16:

[1107] The server provides more information

[1108] The server retrieves detailed information (e.g., cafe reviews and discount information) based on the user's request and generates an answer.

[1109] Input: User selection information.

[1110] What happens: The server uses a database or external API to retrieve the details, format them, and send them.

[1111] Output: Detailed information is sent to the terminal.

[1112] Step 17:

[1113] The device displays detailed information

[1114] The terminal displays detailed information from the server to the user.

[1115] Input: The details received from the server.

[1116] Specific operation: The device displays detailed information on the interface.

[1117] Output: Detailed information is displayed to the user.

[1118] Step 18:

[1119] User performs an activity

[1120] The user performs a selected activity (e.g., visits a cafe).

[1121] Input: Proposal and details.

[1122] Concrete Action: The user physically performs an activity.

[1123] Output: Feedback to the user about the activity's execution status.

[1124] Step 19:

[1125] The device sends the execution status to the server

[1126] The device sends the user's selections and execution status to the server.

[1127] Input: Execution status data.

[1128] Specific operation: The device uses the HTTPS protocol to send execution status data to the server.

[1129] Output: Execution status data is sent to the server.

[1130] Step 20:

[1131] Server stores history and feedback

[1132] The server stores the user's selection history and feedback in a database, which improves the accuracy of suggestions in future.

[1133] Input: Execution status data and user feedback.

[1134] What happens: The server saves the data in a database and uses it for the next suggestion generation.

[1135] Output: Updated history and feedback data is saved in the database.

[1136] (Application example 2)

[1137] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1138] In modern society, there is a demand for systems that allow people to make effective use of their idle time. In particular, there is a need to improve the user experience by suggesting appropriate digital content that matches the user's emotions and mood. The purpose of this invention is to provide a personalized content suggestion system that meets these needs.

[1139] The specification process by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving information such as the user's interests, free time, location, mood, and emotions, means for acquiring weather information and trend information, means for analyzing the past selection history and feedback of the user and similar users, means for generating optimal digital content suggestions for the user based on the analysis results, means for providing the generated digital content suggestions to the user, means for recognizing emotions from the user's voice and facial expression data, means for personalizing the suggestions based on the emotion data, and means for acquiring and providing detailed information about the generated suggestions. This allows the user to spend their idle time effectively and fulfillingly.

[1140] "User interests" refers to information about themes and categories in which a user is interested and shows interest.

[1141] "Free time" is the extra time that a user can use freely.

[1142] "Location" refers to the specific location or area where the user is currently located.

[1143] A "mood" is a user's temporary emotional or mental state.

[1144] "Emotion" refers to the emotional state detected from the user's voice and facial expressions.

[1145] "Weather Information" means data about the climate and weather in a user's current location or a specified area.

[1146] "Trend information" is data about current trends and popular content.

[1147] "Past selection history of the user and similar users" refers to the history of content selected in the past by the user and similar users.

[1148] "Feedback" refers to evaluations and opinions regarding the services and content provided by users.

[1149] "Digital content" is information or entertainment provided in electronic form, such as videos, music, or articles.

[1150] "Digital content proposal" means proposing digital content that is considered to be optimal for the user.

[1151] A "server" is a central computer system that manages data, analyzes it, and generates proposals.

[1152] "Detailed information about the proposed content" is more specific information about the proposed content.

[1153] The present invention is a system that proposes personalized digital content to enrich users' idle time. This system includes a user terminal, a server, an external information service, and an emotion recognition engine. The user terminal is assumed to be a device such as a smartphone or tablet, and the server functions as a central management system. The external information service is an Internet resource that provides weather information and trend information. The emotion recognition engine is a system that recognizes emotions from the user's voice and facial expressions and reflects them in the proposals.

[1154] The server includes means for receiving information such as the user's interests, free time, location, mood, and emotions, means for acquiring weather information and trend information, means for analyzing the past selection history and feedback of the user and similar users, means for generating optimal digital content suggestions for the user based on the analysis results, means for providing the generated digital content suggestions to the user, means for recognizing emotions from the user's voice and facial expression data, means for personalizing the suggestions based on the emotion data, and means for acquiring and providing detailed information about the generated suggestions.

[1155] First, users open the application and select the "Personalized Recommendations" function. Then, the user's device displays an interface for inputting the user's interests, free time, location, mood, and emotions. This interface includes selection forms and input forms, where the user can enter and submit this information.

[1156] The server receives the information sent by the user and updates the user profile. It then uses a weather information API to obtain the latest weather data for the user's current location and a trend information API to obtain information on currently popular content. The server also analyzes past selection history and feedback, and passes this data to a machine learning algorithm to generate optimal content suggestions. TensorFlow and other similar algorithms are suitable for use.

[1157] In addition, an emotion recognition engine is used to recognize emotions from the user's voice and facial expression data, and the suggestions are further personalized based on that. An emotion recognition engine such as Microsoft Azure Emotion API is suitable for this. The generated suggestions, including detailed information, are sent to the user's device. The user can then review the suggestions, select and play content that interests them, and receive detailed information and responses to questions.

[1158] For example, consider a scenario where a user opens the app while waiting for a train at a station and selects the "Personalized Recommendations" feature. After the user enters and submits information such as "Movies, Music" or "30 minutes," the server receives the information and updates the profile. The app then checks the current weather using a weather information API, obtains content information from a trend information API, and adds emotional data from an emotion recognition engine for analysis. Finally, the app suggests relaxing short videos or music, allowing the user to select and watch them, effectively spending their idle time.

[1159] Examples of prompts are:

[1160] User: Movies, Music

[1161] Free time: 30 minutes

[1162] Current location: Shinjuku Station

[1163] Mood: Relax

[1164] Emotions: Feeling stressed

[1165] Prompt: Based on the user information above, recommend appropriate videos, music, and articles. Use weather and trending APIs and consider sentiment information from the sentiment engine.

[1166] This allows users to spend their idle time effectively and enjoy a wealth of personalized content.

[1167] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1168] Step 1:

[1169] A user opens the application and selects the "Personalized Recommendations" feature, which displays the application's main interface.

[1170] Step 2:

[1171] The device displays an interface for inputting the user's interests, availability, location, mood, and emotions, including selection and input forms.

[1172] Input: User interests, availability, location, mood, emotions

[1173] Output: The input form is displayed on the terminal.

[1174] Step 3:

[1175] The user enters the required information and presses the send button, which sends the entered data from the device to the server.

[1176] Input: Information entered by the user

[1177] Output: User information sent to the server

[1178] Step 4:

[1179] The server receives the information sent by the user and updates the user profile, ensuring the system reflects the user's most current information.

[1180] Input: User information sent to the server

[1181] Output: Updated user profile

[1182] Step 5:

[1183] The server calls the weather information API to obtain the latest weather data for the user's current location, which is then stored on the server for analysis.

[1184] Input: current user location

[1185] Output: Retrieved weather information

[1186] Step 6:

[1187] The server calls the trend information API to obtain information on currently popular content, which is then stored in the server for analysis.

[1188] Input: Request to get information on currently trending content

[1189] Output: Obtained trend information

[1190] Step 7:

[1191] The server retrieves the user's past selection history and feedback from the database and passes this information to the data analysis engine.

[1192] Input: User's past selections and feedback

[1193] Output: Passed user history information

[1194] Step 8:

[1195] The emotion engine recognizes emotions from the user's voice and facial expression data, and the recognized emotion information is sent to the server.

[1196] Input: User's voice and facial expression data

[1197] Output: Recognized emotion information

[1198] Step 9:

[1199] The server adds the emotion information received from the emotion engine to the user profile and passes it to the analysis engine.

[1200] Input: Emotion information from the emotion engine

[1201] Output: Updated user profile

[1202] Step 10:

[1203] The server integrates weather data, trend data, emotion data, and user profiles, and analyzes them using an analytics engine. Machine learning algorithms (such as TensorFlow) are used to analyze the data and generate optimal digital content recommendations for users.

[1204] Input: Weather data, Trend data, Emotion data, User profile

[1205] Output: Generated digital content proposal

[1206] Step 11:

[1207] The server obtains detailed information about the proposal (e.g., video thumbnail, music artist name, article headline), formats the generated proposal, and sends it to the terminal.

[1208] Input: Generated digital content proposal

[1209] Output: Submitted proposal with details

[1210] Step 12:

[1211] The device displays the received suggestions to the user, visualizing them in list or card format.

[1212] Input: Submitted proposal with detailed information

[1213] Output: Display of the visualized proposal

[1214] Step 13:

[1215] Users can review the suggestions, select and play content that interests them, and use the Q&A feature to find out more information.

[1216] Input: Visualized proposal

[1217] Output: User selection and request for more information

[1218] Step 14:

[1219] The device sends the user's selections and questions to the server, which obtains detailed information about the user's question, generates an answer, and sends it to the device.

[1220] Input: User choices and questions

[1221] Output: Detailed information provided

[1222] Step 15:

[1223] The device displays the answer to the user, allowing them to get the details they need.

[1224] Input: Details to be provided

[1225] Output: Detailed information displayed to the user

[1226] Step 16:

[1227] The user executes the content that they have finally selected. The device sends the user's selection and execution status to the server.

[1228] Input: Last selected content

[1229] Output: Selections and execution status sent to the server

[1230] Step 17:

[1231] The server stores the user's selection history and feedback in a database and uses this information to improve the accuracy of future suggestions.

[1232] Input: User selection history and feedback

[1233] Output: Saved selection history and feedback

[1234] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1235] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1236] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1237] [Third embodiment]

[1238] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1239] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1240] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1241] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1242] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1243] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1244] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1245] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1246] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1247] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1248] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1249] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1250] The present invention is a system for making effective use of users' idle time. This system generates optimal time-killing suggestions based on information such as the user's interests, free time, location, and mood, taking into account weather and event information. The specific system configuration and processing flow are described below.

[1251] System Configuration

[1252] This system consists of a user terminal, a server, and external information services (weather information API, event information API, etc.). User terminals are assumed to be devices such as smartphones and tablets. The server functions as a central management system, collecting, analyzing, and generating proposals for various types of data. The external information services are Internet resources that provide weather and event information.

[1253] Program processing

[1254] Enter and submit user information

[1255] 1. The user opens the application and selects the "Time-killing suggestions" feature.

[1256] 2. The device displays an interface for entering information such as the user's interests, availability, location, and mood, which may include selection forms or input forms.

[1257] 3. When the user enters the required information and presses the send button, the information is sent from the device to the server.

[1258] Data collection and analysis

[1259] 4. The server receives the information sent by the user and updates the user profile.

[1260] 5. The server calls the weather information API to obtain the latest weather data for the user's current location.

[1261] 6. The server calls the event information API and obtains information about events being held in the user's current location.

[1262] 7. The server retrieves the user's and similar users' past selection history and feedback from the database and passes it to the analysis engine.

[1263] Proposal Generation

[1264] 8. The server integrates weather data, event data, and user profiles and uses an analytics engine to generate optimal suggestions for users, using machine learning algorithms and a recommendation system based on similar user histories.

[1265] 9. The server formats the offer with detailed information (location, time, discounts, reviews, etc.) and sends it to the device.

[1266] Proposal presentation and user interaction

[1267] 10. The device displays the received suggestions to the user. The suggestions can be visualized in a list format, card format, or other format.

[1268] 11. Users review the suggestions and select the activities they are interested in. They can also use the Q&A feature if they need further clarification.

[1269] 12. The device sends the user's selections and questions to the server.

[1270] Data provision and storage

[1271] 13. The server obtains detailed information about the user's question, generates an answer, and sends it to the device.

[1272] 14. The device displays the answer to the user.

[1273] 15. The user finally performs the selected activity.

[1274] 16. The terminal sends the user's selections and execution status to the server, which stores them in a database.

[1275] 17. The server collects user feedback and uses it to improve the accuracy of suggestions in future.

[1276] Specific examples

[1277] Killing time at the station

[1278] 1. While waiting for a train at a station, the user opens the app and selects the "Time-killing suggestions" feature.

[1279] 2. The device will display a screen asking about your interests.

[1280] 3. The user enters "cafe hopping," "30 minutes," and "current location is the station," and submits.

[1281] 4. The server receives this information and updates the profile.

[1282] 5. The server calls the weather information API and confirms that the current weather is "sunny."

[1283] 6. The server calls the event information API and obtains event information around the station.

[1284] 7. The server integrates the data with past data and generates suggestions for new cafe coupons, art exhibitions near train stations, and the latest movie screenings.

[1285] 8. The server sends the suggestions to the device, which displays them to the user.

[1286] 9. The user selects a cafe coupon and views more details.

[1287] 10. The device sends the question to the server, which provides a detailed review.

[1288] 11. The user finally selects a cafe and decides to visit.

[1289] 12. Your device sends your selections to the server, which stores your history and feedback.

[1290] The above is an embodiment of the present invention, which allows users to easily discover new activities and have interesting experiences.

[1291] The processing flow will be explained below.

[1292] Step 1:

[1293] The user opens the application and selects the "Time-killing suggestions" feature. The device then displays an interface for entering information such as the user's interests, free time, location, and mood.

[1294] Step 2:

[1295] The user enters the necessary information and presses the send button. The device sends the information entered by the user to the server.

[1296] Step 3:

[1297] The server receives user information (interests, free time, location, mood, etc.) sent from the device and updates the user profile.

[1298] Step 4:

[1299] The server calls the weather information API to obtain the latest weather data for the user's current location, and stores the obtained weather information on the server.

[1300] Step 5:

[1301] The server calls the event information API to obtain information about events taking place in the user's current location, and stores the obtained event information on the server.

[1302] Step 6:

[1303] The server retrieves the user's and similar users' past selection history and feedback from the database and passes it to the analysis engine, which then begins the analysis process.

[1304] Step 7:

[1305] The server integrates weather data, event data, and user profiles, and runs the analysis through an analytics engine that uses machine learning algorithms and a recommendation system based on similar user history to generate optimal suggestions for users.

[1306] Step 8:

[1307] The server formats the generated suggestions by adding details (e.g., cafe location, hours, discounts, reviews, event location and time, etc.) and prepares the suggestions for delivery to the user.

[1308] Step 9:

[1309] The server sends the prepared suggestions to the device, which then displays them to the user. The suggestions are visualized in a list format, card format, or other format.

[1310] Step 10:

[1311] Users can review suggestions, select activities they are interested in, and use the Q&A feature to find out more information.

[1312] Step 11:

[1313] The device sends the user's selections and questions to the server, which retrieves relevant information from a database and generates an answer.

[1314] Step 12:

[1315] The server generates a response and sends it to the device, which displays it to the user, providing them with the necessary details.

[1316] Step 13:

[1317] The user then performs the activity they have finally selected (e.g., visiting a cafe, attending an event, watching a movie, etc.). The device then sends the user's selection and the execution status to the server.

[1318] Step 14:

[1319] The server stores the user's selection history and feedback in a database, which is used to improve the accuracy of suggestions in future.

[1320] Step 15:

[1321] After completing an activity, the user inputs and sends a simple feedback. The device sends the feedback to the server, which ends the user interaction.

[1322] Example 1

[1323] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1324] In modern society, it is a difficult task to effectively utilize users' free time and suggest activities appropriate for that time. Conventional systems often fail to provide suggestions that are in line with the user's interests and current situation, leaving users unable to make satisfying choices. The present invention aims to solve this problem and provide users with optimal time-killing suggestions.

[1325] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1326] In this invention, the server includes means for receiving information such as the user's interests, free time, location, and mood, means for acquiring weather information and event information, means for analyzing the user's and similar users' past selection history and feedback, means for integrating the weather data, event data, and user profile and generating suggestions using an analysis engine, means for formatting detailed information about the suggestions (location, time, discount details, reviews, etc.) and sending them to the terminal, means for receiving the user's questions and providing related information, means for sending the user's selections and questions to the server, means for saving the user's selection history and feedback to improve the accuracy of suggestions from next time onwards, and means for collecting user feedback and saving it in a database, thereby making it possible to suggest activities that are optimal for the user's free time.

[1327] "Means for receiving information such as user interests, free time, location, and mood" refers to an interface that allows users to input their interests, free time, current location, mood, etc. through the application, and the function for collecting that information.

[1328] "Means for obtaining weather information and event information" refers to API call functions and data collection functions that obtain current weather and nearby event information from external weather information services and event information services.

[1329] "Means for analyzing the past selection history and feedback of users and similar users" refers to a function for analyzing the past selection history and feedback stored in a database using a machine learning algorithm.

[1330] "Means for integrating weather data, event data, and user profiles and generating suggestions using an analytical engine" refers to a function that combines acquired weather data, event information, and user profiles and generates optimal suggestions using an analytical engine.

[1331] "Means for formatting detailed information about the proposal (location, time, discount details, reviews, etc.) and sending it to the terminal" refers to a function that adds detailed information about the proposal, such as location information, time, discount details, reviews, etc., to the generated proposal, formats it, and sends it to the user's terminal.

[1332] "Means for receiving user questions and providing relevant information" refers to a function that receives questions from users and provides appropriate answer information or additional detailed information.

[1333] The "means for transmitting user selections and questions to the server" is a function for transmitting information about a user's selection of an activity or question to the server.

[1334] "Means for saving user selection history and feedback to improve the accuracy of future suggestions" refers to a function that saves the user's selections and subsequent feedback in a database and uses that data when generating suggestions the next time.

[1335] "Means for collecting user feedback and saving it in a database" refers to a function that collects feedback on the activities experienced by users after they have entered it and saves it in a database.

[1336] The present invention is a system for making effective use of a user's free time, and generates optimal time-killing suggestions based on information such as the user's interests, free time, location, and mood, taking into account weather and event information. The specific system configuration and processing flow are described below.

[1337] System Configuration

[1338] This system consists of a user terminal, a server, and an external information service. The user terminal is assumed to be a device such as a smartphone or tablet. The server functions as a central management system, collecting, analyzing, and generating proposals for various data. The external information service is an Internet resource that provides weather and event information.

[1339] Hardware and software used

[1340] User devices are smartphones or tablets. The server is configured as a system including a database server, application server, and analysis engine. The weather information API uses the OpenWeatherMap API, and the event information API uses the Eventbrite API. The analysis engine implements a machine learning algorithm using Scikit-learn.

[1341] Program processing flow

[1342] 1. The user opens the application and selects the "Time-killing suggestions" feature.

[1343] 2. The device displays an interface for entering information such as the user's interests, availability, location, and mood, which may include selection forms or input forms.

[1344] 3. When the user enters the required information and presses the send button, the information is sent from the device to the server.

[1345] 4. The server receives the information sent by the user and updates the user profile.

[1346] 5. The server calls the weather information API to obtain the latest weather data for the user's current location.

[1347] 6. The server calls the event information API and obtains information about events being held in the user's current location.

[1348] 7. The server retrieves the user's and similar users' past selection history and feedback from the database and passes it to the analysis engine.

[1349] 8. The server integrates weather data, event data, and user profiles and uses an analytics engine to generate optimal suggestions for users, using machine learning algorithms and a recommendation system based on similar user histories.

[1350] 9. The server formats the offer with detailed information (location, time, discounts, reviews, etc.) and sends it to the device.

[1351] 10. The device displays the received suggestions to the user. The suggestions can be visualized in a list format, card format, or other format.

[1352] 11. Users review the suggestions and select the activities they are interested in. They can also use the Q&A feature if they need further clarification.

[1353] 12. The device sends the user's selections and questions to the server.

[1354] 13. The server obtains detailed information about the user's question, generates an answer, and sends it to the device.

[1355] 14. The device displays the answer to the user.

[1356] 15. The user finally performs the selected activity.

[1357] 16. The terminal sends the user's selections and execution status to the server, which stores them in a database.

[1358] 17. The server collects user feedback and uses it to improve the accuracy of suggestions in future.

[1359] Specific examples

[1360] Killing time at the station

[1361] 1. While waiting for a train at a station, the user opens the app and selects the "Time-killing suggestions" feature.

[1362] 2. The device will display a screen asking about your interests.

[1363] 3. The user enters "cafe hopping," "30 minutes," and "current location is the station" and submits.

[1364] 4. The server receives this information and updates the profile.

[1365] 5. The server calls the weather information API and confirms that the current weather is "sunny."

[1366] 6. The server calls the event information API and obtains event information around the station.

[1367] 7. The server integrates the data with past data and generates suggestions for new cafe coupons, art exhibitions near train stations, and the latest movie screenings.

[1368] 8. The server sends the suggestions to the device, which displays them to the user.

[1369] 9. The user selects the cafe coupon and views more details.

[1370] 10. The device sends the question to the server, which provides a detailed review.

[1371] 11. The user finally selects a cafe and decides to visit.

[1372] 12. The device sends the selection to the server, which stores the history and feedback.

[1373] Example prompts for generative AI models

[1374] "I'm currently waiting for a train at the station. I'd like to take advantage of the time I have to spend at a cafe. Do you have any recommendations? The conditions are that the weather is currently sunny and the journey will take about 30 minutes."

[1375] The above is an embodiment of the present invention, which allows users to easily discover new activities and have interesting experiences.

[1376] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1377] Step 1:

[1378] The user opens the application and selects the "Time-killing suggestions" function. The input is a user action, and the output is the launch of the application. The specific action is that the user taps the smartphone icon to launch the app.

[1379] Step 2:

[1380] The device displays an interface for entering information such as the user's interests, free time, location, and mood. The input is a signal to start the application, and the output is a screen for entering user information. Specifically, the application presents the user with options and a form for entering "interests," "free time," "location," and "mood."

[1381] Step 3:

[1382] The user enters the necessary information and presses the send button. The input is the information entered by the user (interests, free time, location, mood), and the output is sent from the device to the server. Specifically, the user enters "cafe hopping," "30 minutes," "current location," and "relax," and taps the "send" button.

[1383] Step 4:

[1384] The server receives the information submitted by the user and updates the user profile. The input is the information submitted by the user, and the output is the updated profile, which is saved in the database. Specifically, the server accesses the database and executes an SQL query to update the user profile.

[1385] Step 5:

[1386] The server calls the weather information API to obtain the latest weather data for the user's current location. The input is the user's current location information, and the output is the latest weather data. Specifically, the server sends a request to the OpenWeatherMap API and receives the current weather information (e.g., "Sunny, temperature 24 degrees").

[1387] Step 6:

[1388] The server calls the event information API to obtain information about events taking place in the user's current location. The input is the user's current location information, and the output is event information. Specifically, the server sends a request to the Eventbrite API and receives information about events in the area.

[1389] Step 7:

[1390] The server retrieves the past selection history and feedback of the user and similar users from the database and passes it to the analysis engine. The input is the user ID, and the output is the selection history and feedback data. Specifically, the server executes an SQL query to retrieve the past history and feedback from the database and passes it to the analysis engine.

[1391] Step 8:

[1392] The server integrates weather data, event data, and user profiles and uses an analytics engine to generate optimal suggestions for users. Weather data, event data, and user profiles are input, and generated suggestions are obtained as output. Specifically, the server runs a machine learning algorithm using Scikit-learn to generate optimal suggestions.

[1393] Step 9:

[1394] The server formats the proposal with detailed information (location, time, discount details, reviews, etc.) and sends it to the device. The generated proposal is the input, and the formatted proposal is sent to the device as the output. Specifically, the server formats the proposal content in JSON format and sends it to the device via an HTTP request.

[1395] Step 10:

[1396] The device displays the received suggestions to the user.,The input is the suggestion sent from the server, and the,output is the suggestion displayed on the screen.,Specifically, the device visualizes the suggestion,contents in list or card format and displays them to the,user.

[1397] Step 11:

[1398] The user reviews the suggestions and selects the activity they are interested in. If additional explanation is needed, they can use the question and answer function. The suggestions are displayed as input, and the selected activity and questions are sent to the server as output. The user taps the activity card to view more information.

[1399] Step 12:

[1400] The device sends the user's selections and questions to the server. The input is the user's selections and questions, and the output is the information sent to the server. Specifically, the device sends the selected activity ID and the question to the server.

[1401] Step 13:

[1402] The server obtains detailed information about the user's question, generates an answer, and sends it to the terminal. The user's question is the input, and the generated answer is sent to the terminal as the output. In concrete terms, the server obtains the necessary information from the database, generates the answer, and sends it to the terminal.

[1403] Step 14:

[1404] The terminal displays the answer to the user. The answer from the server is input, and the answer is displayed to the user as output. Specifically, the terminal displays the answer on the screen.

[1405] Step 15:

[1406] The user ultimately performs the selected activity. The input is the proposed activity, and the output is the execution of that activity. The concrete action is the user actually performing the selected activity.

[1407] Step 16:

[1408] The terminal sends the user's selection and execution status to the server. The user's selection and execution status are input, and are sent to the server as output. Specifically, the terminal sends information about the executed activity to the server.

[1409] Step 17:

[1410] The server collects user feedback and uses it to improve the accuracy of future proposals. The input is user feedback, and the output is saved in a database. Specifically, the server saves the feedback information in the database and uses it to generate future proposals.

[1411] (Application example 1)

[1412] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1413] In today's modern living environment, users often waste their time waiting or having free time. Particularly in physical stores, there are few ways to efficiently find available activities and promotions, and users cannot easily obtain information that interests them. This results in users not making effective use of their valuable time and a decrease in the quality of their experience.

[1414] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1415] In this invention, the server includes means for receiving information such as the user's interests, free time, location, and mood, means for acquiring weather information and event information, and means for analyzing the past selection history and feedback of the user and similar users, thereby enabling a user interface means for generating optimal suggestions for the user based on the analysis results, displaying the generated suggestions on the user terminal, providing detailed information and reviews, and providing detailed information and navigation for the activity selected by the user.

[1416] "User Terminal" means a smartphone, tablet or other mobile device through which a User inputs information and receives suggestions.

[1417] A "sensor" is a device used as a means of obtaining information such as a user's interests, free time, location, and mood.

[1418] "Weather information" is information that provides data related to the weather, and is obtained from an external information providing service.

[1419] "Event information" is information about events and activities related to a specific location and period, and is obtained from an external information providing service.

[1420] A "user profile" is a data structure that includes information such as a user's interests, availability, location, mood, past selections, and feedback.

[1421] "Analysis results" are information generated by the analysis engine based on weather information, event information, user profiles, etc., and include optimal suggestions for users.

[1422] A "user interface" is a screen or mechanism for displaying information and suggestions to users and receiving input and feedback from users.

[1423] "Navigation" is a function that shows and guides the user to the location of the activity selected by the user.

[1424] "Suggestions" are suggested time-killing activities or promotions that are generated based on the analysis results and provided to users.

[1425] "Details" means specific information related to the proposed activity, such as location, time, discounts, reviews, etc.

[1426] System Overview

[1427] This invention is designed as a system for effectively utilizing users' idle time, and is intended for use in brick-and-mortar stores in particular. The system consists of a user terminal, a server, and an external information service. The user terminal corresponds to a smartphone or tablet, and the server includes an analysis engine, a database, and a proposal generation system. The external information service includes a weather information API and an event information API. This allows the system to generate and execute time-killing proposals in real time that are optimized for the user's current situation and preferences.

[1428] Program structure and main processing

[1429] 1. Enter and submit user information

[1430] User device: The user launches the application and selects the "Time-killing suggestions" function. At this point, an interface is displayed for entering information such as the user's interests, free time, location, and mood. This interface consists of a selection form and an input form. The entered information is sent from the device to the server.

[1431] 2. Data collection and analysis

[1432] Server: The server receives the information sent by the user and updates the user profile. It then calls the weather information API to get the latest weather data for the user's current location, calls the event information API to get information about events taking place in the user's current location, and retrieves the user's and similar users' past selection history and feedback from the database and passes them to the analytics engine.

[1433] 3. Proposal Generation

[1434] Server: The server integrates weather data, event data, and user profiles and uses an analytics engine to generate optimal suggestions for users. The suggestions are generated using machine learning algorithms and a recommendation system based on similar user history. The generated suggestions are then sent to the user's device along with detailed information (location, time, discounts, reviews, etc.).

[1435] 4. Proposal Presentation and User Interaction

[1436] User device: The user device displays the received suggestions to the user. The suggestions are visualized in a list format, card format, etc. The user reviews the suggestions and selects the activities they are interested in. If further explanation is needed, the user can use the question and answer function. The device sends the user's selections and questions to the server.

[1437] 5. Data provision and storage

[1438] Server: The server obtains detailed information about the user's question, generates an answer, and sends it to the device. The device displays the answer to the user. The user then performs the selected activity, and the device sends subsequent feedback and selection history to the server. The server stores this information in a database and uses it to improve the accuracy of future suggestions.

[1439] Hardware and software used

[1440] Hardware: Servers, smartphones, tablets

[1441] Software: Python, SQLite, scikit-learn (machine learning library), requests (HTTP request library)

[1442] Specific examples

[1443] For example, a user sets their current location to a "major station," their interests to "cafes" and "art galleries," and their free time to 60 minutes. Based on this information, the server retrieves cloudy weather information from a weather information API and supplements it with information on art events taking place around the station from an event information API. Suggestions are generated for Cafe ABC (10% off coupon) and Art Gallery XYZ (highly rated reviews) in front of the station. The user selects Cafe ABC and views detailed information (coupon usage guide, navigation).

[1444] Prompt Sentence Examples

[1445] Prompt: "A user is interested in cafes and art galleries, is located at a major train station, and has 60 minutes of free time. Generate suggestions for the best ways to kill time for him. The current weather is cloudy, and there are art events happening around the station. Include location, time, discounts, and reviews in the suggestions."

[1446] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1447] Step 1:

[1448] Enter and submit user information

[1449] A user launches the application and selects the "Time-killing suggestions" function. The user's device displays an interface for entering information such as interests, free time, location, and mood.

[1450] Input: User interests, availability, location, and mood information

[1451] Output: Sending information from the device to the server

[1452] Step 2:

[1453] Update your user profile

[1454] The server receives the information submitted by the user and updates the user profile.

[1455] Input: User information sent from the device

[1456] Output: Updated user profile

[1457] Step 3:

[1458] Get weather information

[1459] The server calls the weather information API to obtain the latest weather data for the user's current location.

[1460] Input: User's current location

[1461] Output: Current weather data for your current location

[1462] Step 4:

[1463] Get event information

[1464] The server calls the event information API and obtains information about events being held in the user's current location.

[1465] Input: User's current location

[1466] Output: Event information for the current location

[1467] Step 5:

[1468] Get past selection history and feedback

[1469] The server retrieves the user's and similar users' past selection history and feedback from the database and passes it to the analysis engine.

[1470] Input: User profile

[1471] Output: Past selection history and feedback

[1472] Step 6:

[1473] Analysis and proposal generation

[1474] The server combines weather data, event data, and user profiles, and uses an analytics engine to generate optimal suggestions for users, using machine learning algorithms and recommendation systems.

[1475] Input: Weather data, event data, user profile

[1476] Output: Best time-killing suggestions

[1477] Step 7:

[1478] Submit a proposal

[1479] The server sends the generated proposal to the user's device, along with detailed information (location, time, discounts, reviews, etc.).

[1480] Input: Best time-killing suggestions

[1481] Output: Proposal sent to user's device

[1482] Step 8:

[1483] Presenting the proposal

[1484] The user's device displays the received suggestions to the user. The suggestions are visualized in a list or card format, and the user can review the suggestions and select the activities that interest them.

[1485] Input: Proposal

[1486] Output: Suggestions shown to the user

[1487] Step 9:

[1488] Ask questions and get more information

[1489] If the user requests further clarification, the device sends the question to the server, which obtains detailed information about the user's question, generates an answer, and sends it to the device.

[1490] Input: User question

[1491] Output: Answer to the question

[1492] Step 10:

[1493] Final Selection and Navigation

[1494] It provides detailed information and navigation for the activity that the user ultimately selects. The user device sends the selection to the server, which stores the history and feedback.

[1495] Input: Last selected activity

[1496] Output: Activity details and navigation, saving selection history

[1497] Usage examples and prompt statements

[1498] As a concrete example, suppose a user sets their current location to "major station," their interests to "cafes" and "art galleries," and their free time to 60 minutes.

[1499] Prompt: "A user is interested in cafes and art galleries, is located at a major train station, and has 60 minutes of free time. Generate suggestions for the best ways to kill time for him. The current weather is cloudy, and there are art events happening around the station. Include location, time, discounts, and reviews in the suggestions."

[1500] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1501] This invention is a system that helps users make effective use of their idle time. This system generates optimal time-killing suggestions based on information such as the user's interests, free time, location, and mood, taking into account weather and event information. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it is possible to realize more personalized suggestions.

[1502] System Configuration

[1503] This system consists of a user terminal, a server, an external information provision service (weather information API, event information API, etc.), and an emotion engine. User terminals are assumed to be devices such as smartphones and tablets. The server functions as a central management system, collecting and analyzing various data and generating proposals. The external information provision service is an Internet resource for providing weather and event information. The emotion engine is a system that recognizes emotions from the user's voice and facial expressions and reflects them in proposals.

[1504] Program processing

[1505] Enter and submit user information

[1506] 1. The user opens the application and selects the "Time-killing suggestions" feature.

[1507] 2. The device displays an interface for inputting the user's interests, availability, location, mood, and emotions, including selection and input forms.

[1508] 3. When the user enters the required information and presses the send button, the information is sent from the device to the server.

[1509] Data collection and analysis

[1510] 4. The server receives the information sent by the user and updates the user profile.

[1511] 5. The server calls the weather information API to obtain the latest weather data for the user's current location and stores the obtained weather information on the server.

[1512] 6. The server calls the event information API to obtain information about events taking place in the user's current location. The obtained event information is stored on the server.

[1513] 7. The server retrieves the user's and similar users' past selection history and feedback from the database and passes it to the analysis engine, which then begins the analysis process.

[1514] Acquiring and analyzing emotion data

[1515] 8. The emotion engine acquires the user's voice and facial expression data, recognizes their emotions, and sends the emotion information obtained by the emotion engine to the server.

[1516] 9. The server adds the emotion information received from the emotion engine to the user profile and passes it to the analysis engine.

[1517] Proposal Generation

[1518] 10. The server integrates weather data, event data, emotion data, and user profiles and analyzes them in an analytics engine, which uses machine learning algorithms and a recommendation system based on similar user history to generate optimal suggestions for the user.

[1519] 11. The server formats the offer with detailed information (e.g., cafe location, hours, discounts, reviews, etc.) and sends it to the device.

[1520] Proposal presentation and user interaction

[1521] 12. The device displays the received suggestions to the user. The suggestions can be visualized in a list format, card format, or other format.

[1522] 13. Users can review the suggestions, select the activities they are interested in, and use the Q&A feature to find out more information.

[1523] 14. The device sends the user's selections and questions to the server.

[1524] 15. The server obtains detailed information about the user's question, generates an answer, and sends it to the device.

[1525] 16. The device displays the answer to the user, providing them with the necessary details.

[1526] Data provision and storage

[1527] 17. The user finally performs the selected activity (e.g., visit a cafe, attend an event, watch a movie, etc.). The device sends the user's selection and execution status to the server.

[1528] 18. The server stores the user's selection history and feedback in a database, which will be used to improve the accuracy of future suggestions.

[1529] 19. After the user completes the activity, they input and submit a brief feedback. The device sends the feedback to the server, which ends the user interaction.

[1530] Specific examples

[1531] Killing time at the station

[1532] 1. While waiting for a train at a station, the user opens the app and selects the "Time-killing suggestions" feature.

[1533] 2. The device will display a screen asking for your interests, free time, location, mood, and emotions.

[1534] 3. The user inputs "cafe hopping," "30 minutes," "Shinjuku Station," "want to relax," and "emotional data via voice input (e.g., feeling stressed based on the tone of voice)" and submits.

[1535] 4. The server receives this information and updates the profile.

[1536] 5. The server calls the weather information API and confirms that the current weather is "sunny."

[1537] 6. The server calls the event information API and obtains event information around Shinjuku Station.

[1538] 7. The server integrates the data with past data and adds data from the emotion engine for analysis.

[1539] 8. The server generates suggestions for new cafes, art exhibits near train stations, and the latest movie screenings. The suggestions are focused on relaxing content that reflects the user’s emotional state.

[1540] 9. The server sends the proposal to the device, which displays it to the user.

[1541] 10. The user selects a cafe coupon and views more details.

[1542] 11. The device sends the question to the server, which provides a detailed review.

[1543] 12. The user finally selects a cafe and decides to visit.

[1544] 13. Your device sends your selections to the server, which stores your history and feedback.

[1545] The above is an embodiment of the present invention. This makes it possible to propose new activities that take emotions into consideration, allowing users to enjoy new, personalized experiences without any effort.

[1546] The processing flow will be explained below.

[1547] Step 1:

[1548] The user opens the application and selects the "Time-killing suggestions" function. The device then displays an interface for inputting information such as the user's interests, free time, location, mood, and emotions.

[1549] Step 2:

[1550] The user enters their interests (e.g., cafes, movies, art exhibitions), free time (e.g., 30 minutes), location (e.g., Shinjuku Station), mood (e.g., wanting to relax), and emotion (e.g., feeling stressed), and then presses the send button. The device then sends the entered information to the server.

[1551] Step 3:

[1552] The server receives user information (interests, free time, location, mood, emotions, etc.) sent from the device and updates the user profile.

[1553] Step 4:

[1554] The server calls the weather information API to obtain the latest weather data for the user's current location, and stores the obtained weather information on the server.

[1555] Step 5:

[1556] The server calls the event information API to obtain information about events taking place in the user's current location, and stores the obtained event information on the server.

[1557] Step 6:

[1558] The server retrieves the user's and similar users' past selection history and feedback from the database and passes it to the analysis engine, which then begins the analysis process.

[1559] Step 7:

[1560] The emotion engine acquires emotional data from the user's voice and facial expressions, recognizes that the user is feeling stressed, and sends this emotional information to the server.

[1561] Step 8:

[1562] The server adds the emotional information received from the emotion engine to the user profile and passes it to the analysis engine, which then performs an analysis based on the user's current emotional state.

[1563] Step 9:

[1564] The server integrates weather data, event data, user profiles, and sentiment data, and runs the analysis through an analytics engine that generates optimal recommendations for each user based on machine learning algorithms and similar user history.

[1565] Step 10:

[1566] The server formats the generated suggestions by adding details (e.g., cafe location, opening hours, discounts, reviews, event location and time, etc.) and prepares the suggestions for delivery to the user.

[1567] Step 11:

[1568] The server sends the prepared suggestions to the device, which then displays them to the user. The suggestions are visualized in a list format, card format, or other format.

[1569] Step 12:

[1570] Users can review the suggestions and select activities they are interested in. They can also use the Q&A feature to find out more information.

[1571] Step 13:

[1572] The device sends the user's selections and questions to the server, which retrieves relevant information from a database and generates an answer.

[1573] Step 14:

[1574] The server generates a response and sends it to the device, which displays it to the user, providing them with the necessary details.

[1575] Step 15:

[1576] The user then performs the activity they have finally selected (e.g., visiting a cafe, attending an event, watching a movie, etc.). The device then sends the user's selection and the execution status to the server.

[1577] Step 16:

[1578] The server stores the user's selection history and feedback in a database, which is used to improve the accuracy of suggestions in future.

[1579] Step 17:

[1580] After completing an activity, the user inputs and sends a simple feedback. The device sends the feedback to the server, which ends the user interaction.

[1581] Example 2

[1582] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1583] Previous time-killing suggestion systems were unable to take into account the user's subjective feelings and therefore had a low level of personalization. Furthermore, it was difficult to obtain and reflect weather and event information in real time, which led to a lack of reliability in terms of whether the system would actually be useful to the user. Furthermore, the system was unable to improve the accuracy of suggestions by utilizing past selection history and feedback, which led to issues such as low user satisfaction.

[1584] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for receiving information such as the user's interests, free time, location, mood, and emotions; means for acquiring weather information and event information; means for analyzing the past selection history and feedback of the user and similar users; means for generating optimal suggestions for the user based on the analysis results; means for providing the generated suggestions to the user; means for recognizing emotions from the user's voice and facial expressions and reflecting the emotion data in suggestions; means for allowing the user to select activities that interest the user; and means for providing detailed information about the activities selected by the user. This enables personalized suggestions that take into account the user's subjective emotions and reflect real-time information.

[1585] "User Information" refers to data such as a user's interests, free time, location, mood, and emotions.

[1586] "Weather Information" refers to data regarding weather conditions in the User's current location.

[1587] "Event Information" refers to data about an event taking place at a particular location and time.

[1588] "Selection history" refers to a record of a user's past choices and actions.

[1589] "Feedback" refers to information regarding opinions and ratings from users.

[1590] An "emotion engine" refers to a system that recognizes emotions from the user's voice and facial expressions.

[1591] A "profile" refers to a data set that compiles information such as a user's attributes, behavioral history, and emotions.

[1592] "Analysis" refers to the data processing and analysis work required to generate optimal proposals based on collected data.

[1593] "Suggestions" refers to options for activities or actions presented to users based on the analysis results.

[1594] This invention is a system that helps users make effective use of their idle time by generating optimal time-killing suggestions based on information such as the user's interests, free time, location, mood, and emotions, taking into account weather and event information. Furthermore, by using an emotion engine that recognizes emotions from the user's voice and facial expressions, more personalized suggestions become possible.

[1595] This system consists of a user terminal, a server, an external information provision service (weather information API, event information API, etc.), and an emotion engine. User terminals are assumed to be devices such as smartphones and tablets. The server functions as a central management system, collecting and analyzing various data and generating proposals. The external information provision service is an Internet resource for providing weather and event information. The emotion engine is a system that recognizes emotions from the user's voice and facial expressions and reflects them in proposals.

[1596] Hardware and software used

[1597] Hardware:

[1598] Smartphone

[1599] tablet

[1600] server

[1601] software:

[1602] Weather information API (e.g. OpenWeatherMap API)

[1603] Event information API (e.g. Eventbrite API)

[1604] Emotion recognition API (e.g. Microsoft Azure Cognitive Services)

[1605] Machine learning engines (e.g. TensorFlow)

[1606] Program processing

[1607] Enter and submit user information

[1608] A user opens the application on their smartphone or tablet and selects the "Time-killing suggestions" function. The device then displays an interface for inputting the user's interests, free time, location, mood, and emotions. This can include checkboxes, drop-down menus, text boxes, and other formats. Once the user enters the information and presses the submit button, the device sends the information to the server. The transmission is secured using the HTTPS protocol.

[1609] Data collection and analysis

[1610] The server receives the information sent by the user and updates the user profile. The server then calls a weather information API to obtain the latest weather data for the user's current location, for example, using the OpenWeatherMap API. The server then calls an event information API to obtain information about events taking place in the user's current location, for example, using the Eventbrite API. In addition, the server retrieves the user's and similar users' past selection history and feedback from a database and passes this information to the analytics engine.

[1611] Acquiring and analyzing emotion data

[1612] The emotion engine acquires the user's voice and facial expression data and recognizes their emotions. For example, it uses the emotion recognition API from Microsoft Azure Cognitive Services. The emotion information obtained by the emotion engine is sent to the server, which adds it to the user profile. This data is also passed to the analysis engine.

[1613] Proposal Generation

[1614] The server integrates weather data, event data, emotion data, and user profiles, and analyzes them using an analysis engine. The analysis engine uses machine learning algorithms (e.g., TensorFlow). Based on the analysis results, the server generates optimal suggestions for the user, adds detailed information, and sends them to the device.

[1615] Proposal presentation and user interaction

[1616] The device displays the received suggestions to the user. The suggestions are visualized in a list or card format. The user can review the suggestions and select activities that interest them. A question and answer function is also available for further information. The selections and questions are sent from the device to the server, which then provides further information.

[1617] Specific examples

[1618] Killing time at the station

[1619] While waiting for a train at a station, the user opens the app and selects the "Time-Killing Suggestions" feature. The device displays a screen asking for "Interests," "Free Time," "Current Location," "Mood," and "Emotions." The user inputs and submits "Cafe Hopping," "30 Minutes," "Shinjuku Station," "Want to Relax," and emotional data via voice input (voice tone indicates stress). The server receives this information and updates the profile. The server then calls the weather information API and confirms that the current weather is "sunny." The server then calls the event information API to obtain event information around Shinjuku Station. The server then combines this data with past data and adds data from the emotion engine for analysis. The server generates suggestions for new cafe coupons, art exhibitions near the station, and the latest movie screenings. The suggestions are focused on relaxing content tailored to the user's emotional state. The server sends the suggestions to the device, which then displays them to the user. The user selects a cafe coupon and browses for further details. The device then sends a question to the server, which provides a detailed review. The user finally selects a cafe and decides to visit. The device then sends the selection to the server, which stores the history and feedback. This will improve the accuracy of proposals from next time onwards.

[1620] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1621] Step 1:

[1622] User enters information

[1623] Users open the application on their smartphone or tablet and select the "Time-killing suggestions" function.

[1624] Input: Users input their interests, availability, location, mood, or emotions within the app, for example using checkboxes, drop-down menus, or text boxes.

[1625] Specific operation: The user inputs "cafe," "30 minutes," "Shinjuku Station," "relax," and "stress by tone of voice."

[1626] Output: The input data is stored in the user's device memory.

[1627] Step 2:

[1628] The device sends information

[1629] The terminal transmits the information entered by the user to the server.

[1630] Input: Data entered by the user.

[1631] What it does: The device uses the HTTPS protocol to communicate securely and send data to the server.

[1632] Output: The server receives the information sent by the user.

[1633] Step 3:

[1634] The server updates the user profile

[1635] The server records and updates the information received from the user as a user profile.

[1636] Input: User-entered data.

[1637] What happens: The server updates the profile using an SQL query against the database.

[1638] Output: Updated user profile data is saved to the database.

[1639] Step 4:

[1640] The server retrieves weather information

[1641] The server calls the weather information API and obtains the latest weather information for the current location (Shinjuku Station).

[1642] Input: User's current location.

[1643] Specific behavior: The server sends a request to the API endpoint to retrieve weather information.

[1644] Output: The latest weather information is saved on the server.

[1645] Step 5:

[1646] The server retrieves the event information

[1647] The server calls the event information API and obtains information about events being held at the current location (Shinjuku Station).

[1648] Input: User's current location.

[1649] Specific operation: The server sends a request to the API endpoint to obtain event information.

[1650] Output: The acquired event information is saved on the server.

[1651] Step 6:

[1652] The server retrieves the past selection history

[1653] The server retrieves the user's and similar users' past selection history and feedback from a database.

[1654] Input: User ID or similar user group.

[1655] What happens: The server executes an SQL query to get the required data.

[1656] Output: Past selection history and feedback are captured on the server.

[1657] Step 7:

[1658] The server passes the data to the analysis engine

[1659] The server passes the collected information to the analysis engine and begins analysis.

[1660] Input: User input, weather information, event information, past selections, feedback.

[1661] Specific operation: The server inputs the data into an analysis engine (e.g., TensorFlow) and performs the analysis.

[1662] Output: The analysis results are generated.

[1663] Step 8:

[1664] Emotion engine recognizes emotions

[1665] The emotion engine captures the user's voice data and recognizes emotions.

[1666] Input: User's voice data.

[1667] What it does: The emotion engine runs a speech recognition system to identify emotions (e.g., determine stress levels).

[1668] Output: Recognized emotion data is generated.

[1669] Step 9:

[1670] The emotion engine sends emotion information to the server

[1671] The emotion engine sends the recognized emotion information to the server.

[1672] Input: Recognized emotion data.

[1673] Specific operation: The emotion engine sends emotion data to the server using the HTTPS protocol.

[1674] Output: Emotion information is received on the server side.

[1675] Step 10:

[1676] The server adds emotional information to the profile

[1677] The server adds the emotional information to the user profile and passes it to the analysis engine.

[1678] Input: Recognized emotion data.

[1679] Specific operation: The server adds the emotion data to the database and updates the profile.

[1680] Output: Passes updated user profile data to the analytics engine.

[1681] Step 11:

[1682] The server integrates and analyzes the data

[1683] The server integrates weather data, event data, emotion data, and user profiles, and performs analysis using an analytical engine.

[1684] Input: All integrated data (weather information, event information, emotion data, user profile).

[1685] Specific operations: The server runs the analytical algorithm and performs data analysis.

[1686] Output: The learned suggestions are generated.

[1687] Step 12:

[1688] The server generates a proposal and sends it to the device

[1689] The server generates optimal suggestions for the user based on the analysis results, adds detailed information, and sends them to the device.

[1690] Input: Analysis results from the analysis engine.

[1691] What happens: The server formats the proposal and sends it to the device using the HTTPS protocol.

[1692] Output: The suggestion is sent to the device.

[1693] Step 13:

[1694] Your device will display suggestions

[1695] The device displays the received suggestions to the user.

[1696] Input: The proposal received from the server.

[1697] What it does: The device uses a user interface to display suggestions in list or card format.

[1698] Output: The suggestions are displayed on the screen.

[1699] Step 14:

[1700] User selects suggestion

[1701] Users select activities that interest them from the suggestions displayed.

[1702] Input: The displayed suggestion.

[1703] Specific Action: The user taps, clicks, or otherwise selects a suggestion.

[1704] Output: User selections are logged to the terminal.

[1705] Step 15:

[1706] The device sends the selection to the server

[1707] The device sends the user's selected suggestions to the server.

[1708] Input: User selection information.

[1709] Specific operation: The device sends the selection information to the server using the HTTPS protocol.

[1710] Output: The selection is sent to the server.

[1711] Step 16:

[1712] The server provides more information

[1713] The server retrieves detailed information (e.g., cafe reviews and discount information) based on the user's request and generates an answer.

[1714] Input: User selection information.

[1715] What happens: The server uses a database or external API to retrieve the details, format them, and send them.

[1716] Output: Detailed information is sent to the terminal.

[1717] Step 17:

[1718] The device displays detailed information

[1719] The terminal displays detailed information from the server to the user.

[1720] Input: The details received from the server.

[1721] Specific operation: The device displays detailed information on the interface.

[1722] Output: Detailed information is displayed to the user.

[1723] Step 18:

[1724] User performs an activity

[1725] The user performs a selected activity (e.g., visits a cafe).

[1726] Input: Proposal and details.

[1727] Concrete Action: The user physically performs an activity.

[1728] Output: Feedback to the user about the activity's execution status.

[1729] Step 19:

[1730] The device sends the execution status to the server

[1731] The device sends the user's selections and execution status to the server.

[1732] Input: Execution status data.

[1733] Specific operation: The device uses the HTTPS protocol to send execution status data to the server.

[1734] Output: Execution status data is sent to the server.

[1735] Step 20:

[1736] Server stores history and feedback

[1737] The server stores the user's selection history and feedback in a database, which improves the accuracy of suggestions in future.

[1738] Input: Execution status data and user feedback.

[1739] What happens: The server saves the data in a database and uses it for the next suggestion generation.

[1740] Output: Updated history and feedback data is saved in the database.

[1741] (Application example 2)

[1742] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1743] In modern society, there is a demand for systems that allow people to make effective use of their idle time. In particular, there is a need to improve the user experience by suggesting appropriate digital content that matches the user's emotions and mood. The purpose of this invention is to provide a personalized content suggestion system that meets these needs.

[1744] The specification process by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving information such as the user's interests, free time, location, mood, and emotions, means for acquiring weather information and trend information, means for analyzing the past selection history and feedback of the user and similar users, means for generating optimal digital content suggestions for the user based on the analysis results, means for providing the generated digital content suggestions to the user, means for recognizing emotions from the user's voice and facial expression data, means for personalizing the suggestions based on the emotion data, and means for acquiring and providing detailed information about the generated suggestions. This allows the user to spend their idle time effectively and fulfillingly.

[1745] "User interests" refers to information about themes and categories in which a user is interested and shows interest.

[1746] "Free time" is the extra time that a user can use freely.

[1747] "Location" refers to the specific location or area where the user is currently located.

[1748] A "mood" is a user's temporary emotional or mental state.

[1749] "Emotion" refers to the emotional state detected from the user's voice and facial expressions.

[1750] "Weather Information" means data about the climate and weather in a user's current location or a specified area.

[1751] "Trend information" is data about current trends and popular content.

[1752] "Past selection history of the user and similar users" refers to the history of content selected in the past by the user and similar users.

[1753] "Feedback" refers to evaluations and opinions regarding the services and content provided by users.

[1754] "Digital content" is information or entertainment provided in electronic form, such as videos, music, or articles.

[1755] "Digital content proposal" means proposing digital content that is considered to be optimal for the user.

[1756] A "server" is a central computer system that manages data, analyzes it, and generates proposals.

[1757] "Detailed information about the proposed content" is more specific information about the proposed content.

[1758] The present invention is a system that proposes personalized digital content to enrich users' idle time. This system includes a user terminal, a server, an external information service, and an emotion recognition engine. The user terminal is assumed to be a device such as a smartphone or tablet, and the server functions as a central management system. The external information service is an Internet resource that provides weather information and trend information. The emotion recognition engine is a system that recognizes emotions from the user's voice and facial expressions and reflects them in the proposals.

[1759] The server includes means for receiving information such as the user's interests, free time, location, mood, and emotions, means for acquiring weather information and trend information, means for analyzing the past selection history and feedback of the user and similar users, means for generating optimal digital content suggestions for the user based on the analysis results, means for providing the generated digital content suggestions to the user, means for recognizing emotions from the user's voice and facial expression data, means for personalizing the suggestions based on the emotion data, and means for acquiring and providing detailed information about the generated suggestions.

[1760] First, users open the application and select the "Personalized Recommendations" function. Then, the user's device displays an interface for inputting the user's interests, free time, location, mood, and emotions. This interface includes selection forms and input forms, where the user can enter and submit this information.

[1761] The server receives the information sent by the user and updates the user profile. It then uses a weather information API to obtain the latest weather data for the user's current location and a trend information API to obtain information on currently popular content. The server also analyzes past selection history and feedback, and passes this data to a machine learning algorithm to generate optimal content suggestions. TensorFlow and other similar algorithms are suitable for use.

[1762] In addition, an emotion recognition engine is used to recognize emotions from the user's voice and facial expression data, and the suggestions are further personalized based on that. An emotion recognition engine such as Microsoft Azure Emotion API is suitable for this. The generated suggestions, including detailed information, are sent to the user's device. The user can then review the suggestions, select and play content that interests them, and receive detailed information and responses to questions.

[1763] For example, consider a scenario where a user opens the app while waiting for a train at a station and selects the "Personalized Recommendations" feature. After the user enters and submits information such as "Movies, Music" or "30 minutes," the server receives the information and updates the profile. The app then checks the current weather using a weather information API, obtains content information from a trend information API, and adds emotional data from an emotion recognition engine for analysis. Finally, the app suggests relaxing short videos or music, allowing the user to select and watch them, effectively spending their idle time.

[1764] Examples of prompts are:

[1765] User: Movies, Music

[1766] Free time: 30 minutes

[1767] Current location: Shinjuku Station

[1768] Mood: Relax

[1769] Emotions: Feeling stressed

[1770] Prompt: Based on the user information above, recommend appropriate videos, music, and articles. Use weather and trending APIs and consider sentiment information from the sentiment engine.

[1771] This allows users to spend their idle time effectively and enjoy a wealth of personalized content.

[1772] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1773] Step 1:

[1774] A user opens the application and selects the "Personalized Recommendations" feature, which displays the application's main interface.

[1775] Step 2:

[1776] The device displays an interface for inputting the user's interests, availability, location, mood, and emotions, including selection and input forms.

[1777] Input: User interests, availability, location, mood, emotions

[1778] Output: The input form is displayed on the terminal.

[1779] Step 3:

[1780] The user enters the required information and presses the send button, which sends the entered data from the device to the server.

[1781] Input: Information entered by the user

[1782] Output: User information sent to the server

[1783] Step 4:

[1784] The server receives the information sent by the user and updates the user profile, ensuring the system reflects the user's most current information.

[1785] Input: User information sent to the server

[1786] Output: Updated user profile

[1787] Step 5:

[1788] The server calls the weather information API to obtain the latest weather data for the user's current location, which is then stored on the server for analysis.

[1789] Input: current user location

[1790] Output: Retrieved weather information

[1791] Step 6:

[1792] The server calls the trend information API to obtain information on currently popular content, which is then stored in the server for analysis.

[1793] Input: Request to get information on currently trending content

[1794] Output: Obtained trend information

[1795] Step 7:

[1796] The server retrieves the user's past selection history and feedback from the database and passes this information to the data analysis engine.

[1797] Input: User's past selections and feedback

[1798] Output: Passed user history information

[1799] Step 8:

[1800] The emotion engine recognizes emotions from the user's voice and facial expression data, and the recognized emotion information is sent to the server.

[1801] Input: User's voice and facial expression data

[1802] Output: Recognized emotion information

[1803] Step 9:

[1804] The server adds the emotion information received from the emotion engine to the user profile and passes it to the analysis engine.

[1805] Input: Emotion information from the emotion engine

[1806] Output: Updated user profile

[1807] Step 10:

[1808] The server integrates weather data, trend data, emotion data, and user profiles, and analyzes them using an analytics engine. Machine learning algorithms (such as TensorFlow) are used to analyze the data and generate optimal digital content recommendations for users.

[1809] Input: Weather data, Trend data, Emotion data, User profile

[1810] Output: Generated digital content proposal

[1811] Step 11:

[1812] The server obtains detailed information about the proposal (e.g., video thumbnail, music artist name, article headline), formats the generated proposal, and sends it to the terminal.

[1813] Input: Generated digital content proposal

[1814] Output: Submitted proposal with details

[1815] Step 12:

[1816] The device displays the received suggestions to the user, visualizing them in list or card format.

[1817] Input: Submitted proposal with detailed information

[1818] Output: Display of the visualized proposal

[1819] Step 13:

[1820] Users can review the suggestions, select and play content that interests them, and use the Q&A feature to find out more information.

[1821] Input: Visualized proposal

[1822] Output: User selection and request for more information

[1823] Step 14:

[1824] The device sends the user's selections and questions to the server, which obtains detailed information about the user's question, generates an answer, and sends it to the device.

[1825] Input: User choices and questions

[1826] Output: Detailed information provided

[1827] Step 15:

[1828] The device displays the answer to the user, allowing them to get the details they need.

[1829] Input: Details to be provided

[1830] Output: Detailed information displayed to the user

[1831] Step 16:

[1832] The user executes the content that they have finally selected. The device sends the user's selection and execution status to the server.

[1833] Input: Last selected content

[1834] Output: Selections and execution status sent to the server

[1835] Step 17:

[1836] The server stores the user's selection history and feedback in a database and uses this information to improve the accuracy of future suggestions.

[1837] Input: User selection history and feedback

[1838] Output: Saved selection history and feedback

[1839] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1840] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1841] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1842] [Fourth embodiment]

[1843] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1844] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1845] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1846] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1847] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1848] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1849] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1850] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1851] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1852] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1853] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1854] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1855] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1856] The present invention is a system for making effective use of users' idle time. This system generates optimal time-killing suggestions based on information such as the user's interests, free time, location, and mood, taking into account weather and event information. The specific system configuration and processing flow are described below.

[1857] System Configuration

[1858] This system consists of a user terminal, a server, and external information services (weather information API, event information API, etc.). User terminals are assumed to be devices such as smartphones and tablets. The server functions as a central management system, collecting, analyzing, and generating proposals for various types of data. The external information services are Internet resources that provide weather and event information.

[1859] Program processing

[1860] Enter and submit user information

[1861] 1. The user opens the application and selects the "Time-killing suggestions" feature.

[1862] 2. The device displays an interface for entering information such as the user's interests, availability, location, and mood, which may include selection forms or input forms.

[1863] 3. When the user enters the required information and presses the send button, the information is sent from the device to the server.

[1864] Data collection and analysis

[1865] 4. The server receives the information sent by the user and updates the user profile.

[1866] 5. The server calls the weather information API to obtain the latest weather data for the user's current location.

[1867] 6. The server calls the event information API and obtains information about events being held in the user's current location.

[1868] 7. The server retrieves the user's and similar users' past selection history and feedback from the database and passes it to the analysis engine.

[1869] Proposal Generation

[1870] 8. The server integrates weather data, event data, and user profiles and uses an analytics engine to generate optimal suggestions for users, using machine learning algorithms and a recommendation system based on similar user histories.

[1871] 9. The server formats the offer with detailed information (location, time, discounts, reviews, etc.) and sends it to the device.

[1872] Proposal presentation and user interaction

[1873] 10. The device displays the received suggestions to the user. The suggestions can be visualized in a list format, card format, or other format.

[1874] 11. Users review the suggestions and select the activities they are interested in. They can also use the Q&A feature if they need further clarification.

[1875] 12. The device sends the user's selections and questions to the server.

[1876] Data provision and storage

[1877] 13. The server obtains detailed information about the user's question, generates an answer, and sends it to the device.

[1878] 14. The device displays the answer to the user.

[1879] 15. The user finally performs the selected activity.

[1880] 16. The terminal sends the user's selections and execution status to the server, which stores them in a database.

[1881] 17. The server collects user feedback and uses it to improve the accuracy of suggestions in future.

[1882] Specific examples

[1883] Killing time at the station

[1884] 1. While waiting for a train at a station, the user opens the app and selects the "Time-killing suggestions" feature.

[1885] 2. The device will display a screen asking about your interests.

[1886] 3. The user enters "cafe hopping," "30 minutes," and "current location is the station," and submits.

[1887] 4. The server receives this information and updates the profile.

[1888] 5. The server calls the weather information API and confirms that the current weather is "sunny."

[1889] 6. The server calls the event information API and obtains event information around the station.

[1890] 7. The server integrates the data with past data and generates suggestions for new cafe coupons, art exhibitions near train stations, and the latest movie screenings.

[1891] 8. The server sends the suggestions to the device, which displays them to the user.

[1892] 9. The user selects a cafe coupon and views more details.

[1893] 10. The device sends the question to the server, which provides a detailed review.

[1894] 11. The user finally selects a cafe and decides to visit.

[1895] 12. Your device sends your selections to the server, which stores your history and feedback.

[1896] The above is an embodiment of the present invention, which allows users to easily discover new activities and have interesting experiences.

[1897] The processing flow will be explained below.

[1898] Step 1:

[1899] The user opens the application and selects the "Time-killing suggestions" feature. The device then displays an interface for entering information such as the user's interests, free time, location, and mood.

[1900] Step 2:

[1901] The user enters the necessary information and presses the send button. The device sends the information entered by the user to the server.

[1902] Step 3:

[1903] The server receives user information (interests, free time, location, mood, etc.) sent from the device and updates the user profile.

[1904] Step 4:

[1905] The server calls the weather information API to obtain the latest weather data for the user's current location, and stores the obtained weather information on the server.

[1906] Step 5:

[1907] The server calls the event information API to obtain information about events taking place in the user's current location, and stores the obtained event information on the server.

[1908] Step 6:

[1909] The server retrieves the user's and similar users' past selection history and feedback from the database and passes it to the analysis engine, which then begins the analysis process.

[1910] Step 7:

[1911] The server integrates weather data, event data, and user profiles, and runs the analysis through an analytics engine that uses machine learning algorithms and a recommendation system based on similar user history to generate optimal suggestions for users.

[1912] Step 8:

[1913] The server formats the generated suggestions by adding details (e.g., cafe location, hours, discounts, reviews, event location and time, etc.) and prepares the suggestions for delivery to the user.

[1914] Step 9:

[1915] The server sends the prepared suggestions to the device, which then displays them to the user. The suggestions are visualized in a list format, card format, or other format.

[1916] Step 10:

[1917] Users can review suggestions, select activities they are interested in, and use the Q&A feature to find out more information.

[1918] Step 11:

[1919] The device sends the user's selections and questions to the server, which retrieves relevant information from a database and generates an answer.

[1920] Step 12:

[1921] The server generates a response and sends it to the device, which displays it to the user, providing them with the necessary details.

[1922] Step 13:

[1923] The user then performs the activity they have finally selected (e.g., visiting a cafe, attending an event, watching a movie, etc.). The device then sends the user's selection and the execution status to the server.

[1924] Step 14:

[1925] The server stores the user's selection history and feedback in a database, which is used to improve the accuracy of suggestions in future.

[1926] Step 15:

[1927] After completing an activity, the user inputs and sends a simple feedback. The device sends the feedback to the server, which ends the user interaction.

[1928] Example 1

[1929] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1930] In modern society, it is a difficult task to effectively utilize users' free time and suggest activities appropriate for that time. Conventional systems often fail to provide suggestions that are in line with the user's interests and current situation, leaving users unable to make satisfying choices. The present invention aims to solve this problem and provide users with optimal time-killing suggestions.

[1931] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1932] In this invention, the server includes means for receiving information such as the user's interests, free time, location, and mood, means for acquiring weather information and event information, means for analyzing the user's and similar users' past selection history and feedback, means for integrating the weather data, event data, and user profile and generating suggestions using an analysis engine, means for formatting detailed information about the suggestions (location, time, discount details, reviews, etc.) and sending them to the terminal, means for receiving the user's questions and providing related information, means for sending the user's selections and questions to the server, means for saving the user's selection history and feedback to improve the accuracy of suggestions from next time onwards, and means for collecting user feedback and saving it in a database, thereby making it possible to suggest activities that are optimal for the user's free time.

[1933] "Means for receiving information such as user interests, free time, location, and mood" refers to an interface that allows users to input their interests, free time, current location, mood, etc. through the application, and the function for collecting that information.

[1934] "Means for obtaining weather information and event information" refers to API call functions and data collection functions that obtain current weather and nearby event information from external weather information services and event information services.

[1935] "Means for analyzing the past selection history and feedback of users and similar users" refers to a function for analyzing the past selection history and feedback stored in a database using a machine learning algorithm.

[1936] "Means for integrating weather data, event data, and user profiles and generating suggestions using an analytical engine" refers to a function that combines acquired weather data, event information, and user profiles and generates optimal suggestions using an analytical engine.

[1937] "Means for formatting detailed information about the proposal (location, time, discount details, reviews, etc.) and sending it to the terminal" refers to a function that adds detailed information about the proposal, such as location information, time, discount details, reviews, etc., to the generated proposal, formats it, and sends it to the user's terminal.

[1938] "Means for receiving user questions and providing relevant information" refers to a function that receives questions from users and provides appropriate answer information or additional detailed information.

[1939] The "means for transmitting user selections and questions to the server" is a function for transmitting information about a user's selection of an activity or question to the server.

[1940] "Means for saving user selection history and feedback to improve the accuracy of future suggestions" refers to a function that saves the user's selections and subsequent feedback in a database and uses that data when generating suggestions the next time.

[1941] "Means for collecting user feedback and saving it in a database" refers to a function that collects feedback on the activities experienced by users after they have entered it and saves it in a database.

[1942] The present invention is a system for making effective use of a user's free time, and generates optimal time-killing suggestions based on information such as the user's interests, free time, location, and mood, taking into account weather and event information. The specific system configuration and processing flow are described below.

[1943] System Configuration

[1944] This system consists of a user terminal, a server, and an external information service. The user terminal is assumed to be a device such as a smartphone or tablet. The server functions as a central management system, collecting, analyzing, and generating proposals for various data. The external information service is an Internet resource that provides weather and event information.

[1945] Hardware and software used

[1946] User devices are smartphones or tablets. The server is configured as a system including a database server, application server, and analysis engine. The weather information API uses the OpenWeatherMap API, and the event information API uses the Eventbrite API. The analysis engine implements a machine learning algorithm using Scikit-learn.

[1947] Program processing flow

[1948] 1. The user opens the application and selects the "Time-killing suggestions" feature.

[1949] 2. The device displays an interface for entering information such as the user's interests, availability, location, and mood, which may include selection forms or input forms.

[1950] 3. When the user enters the required information and presses the send button, the information is sent from the device to the server.

[1951] 4. The server receives the information sent by the user and updates the user profile.

[1952] 5. The server calls the weather information API to obtain the latest weather data for the user's current location.

[1953] 6. The server calls the event information API and obtains information about events being held in the user's current location.

[1954] 7. The server retrieves the user's and similar users' past selection history and feedback from the database and passes it to the analysis engine.

[1955] 8. The server integrates weather data, event data, and user profiles and uses an analytics engine to generate optimal suggestions for users, using machine learning algorithms and a recommendation system based on similar user histories.

[1956] 9. The server formats the offer with detailed information (location, time, discounts, reviews, etc.) and sends it to the device.

[1957] 10. The device displays the received suggestions to the user. The suggestions can be visualized in a list format, card format, or other format.

[1958] 11. Users review the suggestions and select the activities they are interested in. They can also use the Q&A feature if they need further clarification.

[1959] 12. The device sends the user's selections and questions to the server.

[1960] 13. The server obtains detailed information about the user's question, generates an answer, and sends it to the device.

[1961] 14. The device displays the answer to the user.

[1962] 15. The user finally performs the selected activity.

[1963] 16. The terminal sends the user's selections and execution status to the server, which stores them in a database.

[1964] 17. The server collects user feedback and uses it to improve the accuracy of suggestions in future.

[1965] Specific examples

[1966] Killing time at the station

[1967] 1. While waiting for a train at a station, the user opens the app and selects the "Time-killing suggestions" feature.

[1968] 2. The device will display a screen asking about your interests.

[1969] 3. The user enters "cafe hopping," "30 minutes," and "current location is the station" and submits.

[1970] 4. The server receives this information and updates the profile.

[1971] 5. The server calls the weather information API and confirms that the current weather is "sunny."

[1972] 6. The server calls the event information API and obtains event information around the station.

[1973] 7. The server integrates the data with past data and generates suggestions for new cafe coupons, art exhibitions near train stations, and the latest movie screenings.

[1974] 8. The server sends the suggestions to the device, which displays them to the user.

[1975] 9. The user selects the cafe coupon and views more details.

[1976] 10. The device sends the question to the server, which provides a detailed review.

[1977] 11. The user finally selects a cafe and decides to visit.

[1978] 12. The device sends the selection to the server, which stores the history and feedback.

[1979] Example prompts for generative AI models

[1980] "I'm currently waiting for a train at the station. I'd like to take advantage of the time I have to spend at a cafe. Do you have any recommendations? The conditions are that the weather is currently sunny and the journey will take about 30 minutes."

[1981] The above is an embodiment of the present invention, which allows users to easily discover new activities and have interesting experiences.

[1982] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1983] Step 1:

[1984] The user opens the application and selects the "Time-killing suggestions" function. The input is a user action, and the output is the launch of the application. The specific action is that the user taps the smartphone icon to launch the app.

[1985] Step 2:

[1986] The device displays an interface for entering information such as the user's interests, free time, location, and mood. The input is a signal to start the application, and the output is a screen for entering user information. Specifically, the application presents the user with options and a form for entering "interests," "free time," "location," and "mood."

[1987] Step 3:

[1988] The user enters the necessary information and presses the send button. The input is the information entered by the user (interests, free time, location, mood), and the output is sent from the device to the server. Specifically, the user enters "cafe hopping," "30 minutes," "current location," and "relax," and taps the "send" button.

[1989] Step 4:

[1990] The server receives the information submitted by the user and updates the user profile. The input is the information submitted by the user, and the output is the updated profile, which is saved in the database. Specifically, the server accesses the database and executes an SQL query to update the user profile.

[1991] Step 5:

[1992] The server calls the weather information API to obtain the latest weather data for the user's current location. The input is the user's current location information, and the output is the latest weather data. Specifically, the server sends a request to the OpenWeatherMap API and receives the current weather information (e.g., "Sunny, temperature 24 degrees").

[1993] Step 6:

[1994] The server calls the event information API to obtain information about events taking place in the user's current location. The input is the user's current location information, and the output is event information. Specifically, the server sends a request to the Eventbrite API and receives information about events in the area.

[1995] Step 7:

[1996] The server retrieves the past selection history and feedback of the user and similar users from the database and passes it to the analysis engine. The input is the user ID, and the output is the selection history and feedback data. Specifically, the server executes an SQL query to retrieve the past history and feedback from the database and passes it to the analysis engine.

[1997] Step 8:

[1998] The server integrates weather data, event data, and user profiles and uses an analytics engine to generate optimal suggestions for users. Weather data, event data, and user profiles are input, and generated suggestions are obtained as output. Specifically, the server runs a machine learning algorithm using Scikit-learn to generate optimal suggestions.

[1999] Step 9:

[2000] The server formats the proposal with detailed information (location, time, discount details, reviews, etc.) and sends it to the device. The generated proposal is the input, and the formatted proposal is sent to the device as the output. Specifically, the server formats the proposal content in JSON format and sends it to the device via an HTTP request.

[2001] Step 10:

[2002] The device displays the received suggestions to the user.,The input is the suggestion sent from the server, and the,output is the suggestion displayed on the screen.,Specifically, the device visualizes the suggestion,contents in list or card format and displays them to the,user.

[2003] Step 11:

[2004] The user reviews the suggestions and selects the activity they are interested in. If additional explanation is needed, they can use the question and answer function. The suggestions are displayed as input, and the selected activity and questions are sent to the server as output. The user taps the activity card to view more information.

[2005] Step 12:

[2006] The device sends the user's selections and questions to the server. The input is the user's selections and questions, and the output is the information sent to the server. Specifically, the device sends the selected activity ID and the question to the server.

[2007] Step 13:

[2008] The server obtains detailed information about the user's question, generates an answer, and sends it to the terminal. The user's question is the input, and the generated answer is sent to the terminal as the output. In concrete terms, the server obtains the necessary information from the database, generates the answer, and sends it to the terminal.

[2009] Step 14:

[2010] The terminal displays the answer to the user. The answer from the server is input, and the answer is displayed to the user as output. Specifically, the terminal displays the answer on the screen.

[2011] Step 15:

[2012] The user ultimately performs the selected activity. The input is the proposed activity, and the output is the execution of that activity. The concrete action is the user actually performing the selected activity.

[2013] Step 16:

[2014] The terminal sends the user's selection and execution status to the server. The user's selection and execution status are input, and are sent to the server as output. Specifically, the terminal sends information about the executed activity to the server.

[2015] Step 17:

[2016] The server collects user feedback and uses it to improve the accuracy of future proposals. The input is user feedback, and the output is saved in a database. Specifically, the server saves the feedback information in the database and uses it to generate future proposals.

[2017] (Application example 1)

[2018] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2019] In today's modern living environment, users often waste their time waiting or having free time. Particularly in physical stores, there are few ways to efficiently find available activities and promotions, and users cannot easily obtain information that interests them. This results in users not making effective use of their valuable time and a decrease in the quality of their experience.

[2020] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2021] In this invention, the server includes means for receiving information such as the user's interests, free time, location, and mood, means for acquiring weather information and event information, and means for analyzing the past selection history and feedback of the user and similar users, thereby enabling a user interface means for generating optimal suggestions for the user based on the analysis results, displaying the generated suggestions on the user terminal, providing detailed information and reviews, and providing detailed information and navigation for the activity selected by the user.

[2022] "User Terminal" means a smartphone, tablet or other mobile device through which a User inputs information and receives suggestions.

[2023] A "sensor" is a device used as a means of obtaining information such as a user's interests, free time, location, and mood.

[2024] "Weather information" is information that provides data related to the weather, and is obtained from an external information providing service.

[2025] "Event information" is information about events and activities related to a specific location and period, and is obtained from an external information providing service.

[2026] A "user profile" is a data structure that includes information such as a user's interests, availability, location, mood, past selections, and feedback.

[2027] "Analysis results" are information generated by the analysis engine based on weather information, event information, user profiles, etc., and include optimal suggestions for users.

[2028] A "user interface" is a screen or mechanism for displaying information and suggestions to users and receiving input and feedback from users.

[2029] "Navigation" is a function that shows and guides the user to the location of the activity selected by the user.

[2030] "Suggestions" are suggested time-killing activities or promotions that are generated based on the analysis results and provided to users.

[2031] "Details" means specific information related to the proposed activity, such as location, time, discounts, reviews, etc.

[2032] System Overview

[2033] This invention is designed as a system for effectively utilizing users' idle time, and is intended for use in brick-and-mortar stores in particular. The system consists of a user terminal, a server, and an external information service. The user terminal corresponds to a smartphone or tablet, and the server includes an analysis engine, a database, and a proposal generation system. The external information service includes a weather information API and an event information API. This allows the system to generate and execute time-killing proposals in real time that are optimized for the user's current situation and preferences.

[2034] Program structure and main processing

[2035] 1. Enter and submit user information

[2036] User device: The user launches the application and selects the "Time-killing suggestions" function. At this point, an interface is displayed for entering information such as the user's interests, free time, location, and mood. This interface consists of a selection form and an input form. The entered information is sent from the device to the server.

[2037] 2. Data collection and analysis

[2038] Server: The server receives the information sent by the user and updates the user profile. It then calls the weather information API to get the latest weather data for the user's current location, calls the event information API to get information about events taking place in the user's current location, and retrieves the user's and similar users' past selection history and feedback from the database and passes them to the analytics engine.

[2039] 3. Proposal Generation

[2040] Server: The server integrates weather data, event data, and user profiles and uses an analytics engine to generate optimal suggestions for users. The suggestions are generated using machine learning algorithms and a recommendation system based on similar user history. The generated suggestions are then sent to the user's device along with detailed information (location, time, discounts, reviews, etc.).

[2041] 4. Proposal Presentation and User Interaction

[2042] User device: The user device displays the received suggestions to the user. The suggestions are visualized in a list format, card format, etc. The user reviews the suggestions and selects the activities they are interested in. If further explanation is needed, the user can use the question and answer function. The device sends the user's selections and questions to the server.

[2043] 5. Data provision and storage

[2044] Server: The server obtains detailed information about the user's question, generates an answer, and sends it to the device. The device displays the answer to the user. The user then performs the selected activity, and the device sends subsequent feedback and selection history to the server. The server stores this information in a database and uses it to improve the accuracy of future suggestions.

[2045] Hardware and software used

[2046] Hardware: Servers, smartphones, tablets

[2047] Software: Python, SQLite, scikit-learn (machine learning library), requests (HTTP request library)

[2048] Specific examples

[2049] For example, a user sets their current location to a "major station," their interests to "cafes" and "art galleries," and their free time to 60 minutes. Based on this information, the server retrieves cloudy weather information from a weather information API and supplements it with information on art events taking place around the station from an event information API. Suggestions are generated for Cafe ABC (10% off coupon) and Art Gallery XYZ (highly rated reviews) in front of the station. The user selects Cafe ABC and views detailed information (coupon usage guide, navigation).

[2050] Prompt Sentence Examples

[2051] Prompt: "A user is interested in cafes and art galleries, is located at a major train station, and has 60 minutes of free time. Generate suggestions for the best ways to kill time for him. The current weather is cloudy, and there are art events happening around the station. Include location, time, discounts, and reviews in the suggestions."

[2052] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2053] Step 1:

[2054] Enter and submit user information

[2055] A user launches the application and selects the "Time-killing suggestions" function. The user's device displays an interface for entering information such as interests, free time, location, and mood.

[2056] Input: User interests, availability, location, and mood information

[2057] Output: Sending information from the device to the server

[2058] Step 2:

[2059] Update your user profile

[2060] The server receives the information submitted by the user and updates the user profile.

[2061] Input: User information sent from the device

[2062] Output: Updated user profile

[2063] Step 3:

[2064] Get weather information

[2065] The server calls the weather information API to obtain the latest weather data for the user's current location.

[2066] Input: User's current location

[2067] Output: Current weather data for your current location

[2068] Step 4:

[2069] Get event information

[2070] The server calls the event information API and obtains information about events being held in the user's current location.

[2071] Input: User's current location

[2072] Output: Event information for the current location

[2073] Step 5:

[2074] Get past selection history and feedback

[2075] The server retrieves the user's and similar users' past selection history and feedback from the database and passes it to the analysis engine.

[2076] Input: User profile

[2077] Output: Past selection history and feedback

[2078] Step 6:

[2079] Analysis and proposal generation

[2080] The server combines weather data, event data, and user profiles, and uses an analytics engine to generate optimal suggestions for users, using machine learning algorithms and recommendation systems.

[2081] Input: Weather data, event data, user profile

[2082] Output: Best time-killing suggestions

[2083] Step 7:

[2084] Submit a proposal

[2085] The server sends the generated proposal to the user's device, along with detailed information (location, time, discounts, reviews, etc.).

[2086] Input: Best time-killing suggestions

[2087] Output: Proposal sent to user's device

[2088] Step 8:

[2089] Presenting the proposal

[2090] The user's device displays the received suggestions to the user. The suggestions are visualized in a list or card format, and the user can review the suggestions and select the activities that interest them.

[2091] Input: Proposal

[2092] Output: Suggestions shown to the user

[2093] Step 9:

[2094] Ask questions and get more information

[2095] If the user requests further clarification, the device sends the question to the server, which obtains detailed information about the user's question, generates an answer, and sends it to the device.

[2096] Input: User question

[2097] Output: Answer to the question

[2098] Step 10:

[2099] Final Selection and Navigation

[2100] It provides detailed information and navigation for the activity that the user ultimately selects. The user device sends the selection to the server, which stores the history and feedback.

[2101] Input: Last selected activity

[2102] Output: Activity details and navigation, saving selection history

[2103] Usage examples and prompt statements

[2104] As a concrete example, suppose a user sets their current location to "major station," their interests to "cafes" and "art galleries," and their free time to 60 minutes.

[2105] Prompt: "A user is interested in cafes and art galleries, is located at a major train station, and has 60 minutes of free time. Generate suggestions for the best ways to kill time for him. The current weather is cloudy, and there are art events happening around the station. Include location, time, discounts, and reviews in the suggestions."

[2106] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2107] This invention is a system that helps users make effective use of their idle time. This system generates optimal time-killing suggestions based on information such as the user's interests, free time, location, and mood, taking into account weather and event information. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it is possible to realize more personalized suggestions.

[2108] System Configuration

[2109] This system consists of a user terminal, a server, an external information provision service (weather information API, event information API, etc.), and an emotion engine. User terminals are assumed to be devices such as smartphones and tablets. The server functions as a central management system, collecting and analyzing various data and generating proposals. The external information provision service is an Internet resource for providing weather and event information. The emotion engine is a system that recognizes emotions from the user's voice and facial expressions and reflects them in proposals.

[2110] Program processing

[2111] Enter and submit user information

[2112] 1. The user opens the application and selects the "Time-killing suggestions" feature.

[2113] 2. The device displays an interface for inputting the user's interests, availability, location, mood, and emotions, including selection and input forms.

[2114] 3. When the user enters the required information and presses the send button, the information is sent from the device to the server.

[2115] Data collection and analysis

[2116] 4. The server receives the information sent by the user and updates the user profile.

[2117] 5. The server calls the weather information API to obtain the latest weather data for the user's current location and stores the obtained weather information on the server.

[2118] 6. The server calls the event information API to obtain information about events taking place in the user's current location. The obtained event information is stored on the server.

[2119] 7. The server retrieves the user's and similar users' past selection history and feedback from the database and passes it to the analysis engine, which then begins the analysis process.

[2120] Acquiring and analyzing emotion data

[2121] 8. The emotion engine acquires the user's voice and facial expression data, recognizes their emotions, and sends the emotion information obtained by the emotion engine to the server.

[2122] 9. The server adds the emotion information received from the emotion engine to the user profile and passes it to the analysis engine.

[2123] Proposal Generation

[2124] 10. The server integrates weather data, event data, emotion data, and user profiles and analyzes them in an analytics engine, which uses machine learning algorithms and a recommendation system based on similar user history to generate optimal suggestions for the user.

[2125] 11. The server formats the offer with detailed information (e.g., cafe location, hours, discounts, reviews, etc.) and sends it to the device.

[2126] Proposal presentation and user interaction

[2127] 12. The device displays the received suggestions to the user. The suggestions can be visualized in a list format, card format, or other format.

[2128] 13. Users can review the suggestions, select the activities they are interested in, and use the Q&A feature to find out more information.

[2129] 14. The device sends the user's selections and questions to the server.

[2130] 15. The server obtains detailed information about the user's question, generates an answer, and sends it to the device.

[2131] 16. The device displays the answer to the user, providing them with the necessary details.

[2132] Data provision and storage

[2133] 17. The user finally performs the selected activity (e.g., visit a cafe, attend an event, watch a movie, etc.). The device sends the user's selection and execution status to the server.

[2134] 18. The server stores the user's selection history and feedback in a database, which will be used to improve the accuracy of future suggestions.

[2135] 19. After the user completes the activity, they input and submit a brief feedback. The device sends the feedback to the server, which ends the user interaction.

[2136] Specific examples

[2137] Killing time at the station

[2138] 1. While waiting for a train at a station, the user opens the app and selects the "Time-killing suggestions" feature.

[2139] 2. The device will display a screen asking for your interests, free time, location, mood, and emotions.

[2140] 3. The user inputs "cafe hopping," "30 minutes," "Shinjuku Station," "want to relax," and "emotional data via voice input (e.g., feeling stressed based on the tone of voice)" and submits.

[2141] 4. The server receives this information and updates the profile.

[2142] 5. The server calls the weather information API and confirms that the current weather is "sunny."

[2143] 6. The server calls the event information API and obtains event information around Shinjuku Station.

[2144] 7. The server integrates the data with past data and adds data from the emotion engine for analysis.

[2145] 8. The server generates suggestions for new cafes, art exhibits near train stations, and the latest movie screenings. The suggestions are focused on relaxing content that reflects the user’s emotional state.

[2146] 9. The server sends the proposal to the device, which displays it to the user.

[2147] 10. The user selects a cafe coupon and views more details.

[2148] 11. The device sends the question to the server, which provides a detailed review.

[2149] 12. The user finally selects a cafe and decides to visit.

[2150] 13. Your device sends your selections to the server, which stores your history and feedback.

[2151] The above is an embodiment of the present invention. This makes it possible to propose new activities that take emotions into consideration, allowing users to enjoy new, personalized experiences without any effort.

[2152] The processing flow will be explained below.

[2153] Step 1:

[2154] The user opens the application and selects the "Time-killing suggestions" function. The device then displays an interface for inputting information such as the user's interests, free time, location, mood, and emotions.

[2155] Step 2:

[2156] The user enters their interests (e.g., cafes, movies, art exhibitions), free time (e.g., 30 minutes), location (e.g., Shinjuku Station), mood (e.g., wanting to relax), and emotion (e.g., feeling stressed), and then presses the send button. The device then sends the entered information to the server.

[2157] Step 3:

[2158] The server receives user information (interests, free time, location, mood, emotions, etc.) sent from the device and updates the user profile.

[2159] Step 4:

[2160] The server calls the weather information API to obtain the latest weather data for the user's current location, and stores the obtained weather information on the server.

[2161] Step 5:

[2162] The server calls the event information API to obtain information about events taking place in the user's current location, and stores the obtained event information on the server.

[2163] Step 6:

[2164] The server retrieves the user's and similar users' past selection history and feedback from the database and passes it to the analysis engine, which then begins the analysis process.

[2165] Step 7:

[2166] The emotion engine acquires emotional data from the user's voice and facial expressions, recognizes that the user is feeling stressed, and sends this emotional information to the server.

[2167] Step 8:

[2168] The server adds the emotional information received from the emotion engine to the user profile and passes it to the analysis engine, which then performs an analysis based on the user's current emotional state.

[2169] Step 9:

[2170] The server integrates weather data, event data, user profiles, and sentiment data, and runs the analysis through an analytics engine that generates optimal recommendations for each user based on machine learning algorithms and similar user history.

[2171] Step 10:

[2172] The server formats the generated suggestions by adding details (e.g., cafe location, opening hours, discounts, reviews, event location and time, etc.) and prepares the suggestions for delivery to the user.

[2173] Step 11:

[2174] The server sends the prepared suggestions to the device, which then displays them to the user. The suggestions are visualized in a list format, card format, or other format.

[2175] Step 12:

[2176] Users can review the suggestions and select activities they are interested in. They can also use the Q&A feature to find out more information.

[2177] Step 13:

[2178] The device sends the user's selections and questions to the server, which retrieves relevant information from a database and generates an answer.

[2179] Step 14:

[2180] The server generates a response and sends it to the device, which displays it to the user, providing them with the necessary details.

[2181] Step 15:

[2182] The user then performs the activity they have finally selected (e.g., visiting a cafe, atten...

Claims

1. A means of receiving information about users' interests, availability, location, mood, etc. A means of obtaining weather and event information; A means of analyzing the user's and similar users' past selection history and feedback; A means for generating optimal recommendations for users based on the analysis results; The system includes a means for providing the generated suggestions to a user.

2. 10. The system of claim 1, further comprising means for receiving a user question and providing related information.

3. The system of claim 1 further comprising means for storing a user's selection history and feedback to improve the accuracy of subsequent suggestions.

Images