System

Abstract

A system is provided.SOLUTION: A system comprising: means for collecting information from a device of a user; means for transmitting the collected information; means for storing the transmitted information; means for analyzing the stored information; means for generating an offer associated with the user; means for notifying the user of the generated offer; means for collecting a rating by the user; means for calculating a reward based on the user's rating; and means for distributing the reward to a provider.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In today's information-saturated society, it is difficult to provide users with the information they need at the right time. Conventional methods have limitations in effectively providing users with relevant information, and lack mechanisms to properly remind them of regular activities and important tasks. As a result, users are prone to missing information and missing opportunities, which can cause stress. There is a need to solve these issues, provide users with useful information, and realize stress-free lives. [Means for solving the problem]

[0005] This invention is a system that collects information from a user's device, analyzes that information, and generates and notifies relevant suggestions. This system analyzes the user's behavioral patterns, location information, and time information to provide suggestions at optimal times. It also includes a mechanism for distributing rewards to providers based on suggestions and reminders that the user rates highly. Specifically, the system includes a means for collecting information from the user's device, a means for transmitting the collected information, a means for analyzing stored information, a means for generating relevant suggestions for the user, a means for notifying the user of the generated suggestions, a means for collecting user ratings, and a means for calculating and providing rewards. This configuration provides highly relevant information to the user at the appropriate time and reminds the user of regular activities, preventing opportunity loss and reducing stress. Furthermore, the provision of mini-apps allows for efficient addition of necessary functions, improving the flexibility and convenience of the overall system.

[0006] "User device" is a general term for electronic devices used by users to collect information and send notifications, such as smart glasses and smartphones.

[0007] "Means of collecting information" refers to the function of obtaining visual information, location information, text information, etc. from the user's device and storing it as data.

[0008] "Means for transmitting information" refers to the mechanism for transmitting collected data to a server via a network.

[0009] "Means for storing information" refers to the function of safely storing transmitted data in a database, etc.

[0010] "Means for analyzing information" refers to the functions of algorithms or software that process stored data and identify user interests and behavioral patterns.

[0011] "Means for generating suggestions" refers to the ability to create optimal ads and reminders for users based on analyzed data.

[0012] "Means for notifying suggestions" refers to the mechanism by which generated ads or reminders are displayed on the user's device.

[0013] "Means to collect likes" refers to the functionality that collects feedback that users give on suggestions and reminders.

[0014] "Means for calculating rewards" refers to the algorithms or functions that calculate rewards for providers based on positive ratings from users.

[0015] "Means for distributing rewards to providers" refers to a system for appropriately distributing the calculated rewards to the providers, such as developers.

[0016] "Means for recording behavior" refers to a function that records a user's repeated behavior as data.

[0017] "Means for generating reminders" refers to the ability to create reminders to notify users based on recorded behavioral data.

[0018] "Means of notifying at the appropriate time" refers to a function that analyzes the user's location and time information and sends suggestions and reminders at the optimal time. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0020] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0023] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] The present invention relates to a system that collects information from a user's device, analyzes that information, and generates and notifies relevant suggestions to the user, with the aim of providing useful information to the user at the optimal time, making daily life more efficient and comfortable.

[0041] System configuration

[0042] First, the overall configuration of the system will be described. The system of the present invention is composed of the following main components.

[0043] User's devices: Electronic devices for collecting information and sending notifications, such as smart glasses and smartphones.

[0044] Server: A central processing unit that stores data, analyzes it, and generates suggestions.

[0045] Communications network: The infrastructure that connects user devices and servers to exchange information.

[0046] Program processing

[0047] Information Collection and Storage

[0048] Terminal

[0049] 1. Smart glasses capture visual information from the user's everyday life, such as the scenery they see, search queries, and voice commands.

[0050] 2. Your smartphone records your location, the apps you use, and your text messages.

[0051] 3. Once this information is collected, it is periodically sent to a server.

[0052] Data storage and analysis

[0053] server

[0054] 1. The received information is stored in a database. The data is organized by category, such as "visual information," "location information," and "text information."

[0055] 2. Preprocess the stored data and filter out unnecessary information.

[0056] 3. Analyze text data using natural language processing (NLP) techniques to identify user interests and behavioral patterns.

[0057] Proposal generation and notification

[0058] server

[0059] 1. Generate highly relevant ads and reminders based on user interests and behavioral patterns. For example, if a user searches for "stylish cafes," coupon information for popular nearby cafes will be generated.

[0060] 2. Analyze the user's location and time information and calculate a schedule for notifying them at the optimal time.

[0061] Terminal

[0062] 1. Receive notification information sent from the server.

[0063] 2. The notification is displayed at a time that is appropriate for the user, for example, via a smartglasses display or a smartphone push notification.

[0064] Likes and reward distribution

[0065] User

[0066] 1. Check the ads and reminders you receive and select the ones that interest you.

[0067] 2. If you find these useful, give them a "like" in the system interface.

[0068] server

[0069] 1. Aggregate user likes. Track which ads and reminders are liked and how many.

[0070] 2. Calculate and appropriately distribute rewards to the app developers.

[0071] Specific examples

[0072] For example, if a user searches for "stylish cafes" through smart glasses, the glasses capture this information and send it to a server. The server analyzes this information and generates advertisements for nearby cafes. These advertisements are then pushed to the user's smartphone at the optimal time while they are walking around town. If the user finds the advertisement useful, they can rate it highly. Based on this high rating, rewards are distributed to the cafes and app developers that provided the advertisements.

[0073] The above is a specific embodiment of the system of the present invention.

[0074] The processing flow will be explained below.

[0075] Step 1: Gather information

[0076] Terminal

[0077] 1. Smart glasses capture the user's visual information, specifically images of the scenery and objects the user is looking at, as well as search queries and voice commands.

[0078] 2. Your smartphone records your location in real time, and also collects your smartphone usage history and text messages.

[0079] 3. The collected data is periodically compressed and sent over the network to a server.

[0080] Step 2: Save your data

[0081] server

[0082] 1. The received information is stored in a database and classified into categories such as "location information," "visual information," and "text information."

[0083] 2. Preprocessing the information stored in the database to filter out noise and unnecessary information. To protect privacy, personally identifiable information is excluded.

[0084] Step 3: Analyze the data

[0085] server

[0086] 1. Analyze text information using natural language processing (NLP) technology. For example, if a user searches for "stylish cafe," analyze the meaning.

[0087] 2. Identify user interests and patterns by comparing them with past user behavior data. For example, find a pattern such as "I go to a cafe every Wednesday."

[0088] Step 4: Generate proposals

[0089] server

[0090] 1. Based on the analysis results, generate relevant ads and reminders for users. For example, generate coupon ads for "trendy cafes."

[0091] 2. Calculate the optimal timing for notifications based on the user's location and time of day. For example, if the user is near a cafe, send a coupon at that time.

[0092] Step 5: Proposal Notification

[0093] Terminal

[0094] 1. Receive advertisements and reminders sent from the server.

[0095] 2. The received information is notified to the user. This can be done using the smartglasses display or a push notification on a smartphone. For example, a notification could say, "There's a coupon you can use at a nearby cafe."

[0096] Step 6: Like and feedback

[0097] User

[0098] 1. Check the ad or reminder you received and view the details if you are interested.

[0099] 2. If you find an ad or reminder useful, you can give it a "like." You can do this through the smartglasses or smartphone interface.

[0100] Step 7: Counting likes and calculating rewards

[0101] server

[0102] 1. Collect data on user ratings. Analyze which ads and reminders received high ratings.

[0103] 2. Calculate rewards for advertising and reminder providers based on the likes. Determine the reward amount according to the number of likes.

[0104] Step 8: Reward Distribution

[0105] server

[0106] 1. Distribute the calculated rewards to the app developers and advertising providers.

[0107] 2. Record the distribution results and use them for the next reward calculation.

[0108] The above are the specific processing steps of the program of the present invention. This flow makes it possible to provide useful information to users at the optimal timing and to give incentives to developers based on high ratings.

[0109] Example 1

[0110] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0111] Conventional information provision systems face challenges in efficiently and effectively analyzing information collected from users' devices and generating personalized suggestions for users. They also lack a mechanism for properly reflecting users' high ratings and distributing rewards based on those ratings to providers. Furthermore, they do not provide notifications at optimal times that take into account the user's location and time information. This results in a poor user experience and reduces the effectiveness of information provision.

[0112] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0113] In this invention, the server includes means for collecting data from the user's electronic device, means for transmitting the collected data, means for storing the transmitted data, means for preprocessing the stored data, means for analyzing the preprocessed data using natural language processing technology, means for generating suggestions relevant to the user based on the analyzed data, means for notifying the user's electronic device of the generated suggestions, means for collecting user ratings, means for calculating rewards based on the user ratings, and means for distributing rewards to information providers. This makes it possible to generate personalized suggestions based on the user's interests and behavioral patterns and notify them at an appropriate time. Furthermore, by reflecting user rating data and distributing rewards fairly and effectively, it is possible to increase the motivation of information providers.

[0114] "User's electronic devices" refers to various types of electronic devices that users use on a daily basis, including smartphones, smart glasses, and tablet devices.

[0115] "Means of collecting data" refers to a combination of hardware and software that obtains information about a user's daily life and behavior through electronic devices.

[0116] "Means for transmitting data" refers to the protocols and functionality for transmitting collected data from the user's electronic device to the server.

[0117] "Means for storing data" refers to a system for temporarily or permanently storing transmitted data in a database on a server or on a storage medium.

[0118] "Preprocessing means" refers to processes and techniques that perform filtering and noise reduction to convert the stored raw data into a format that is easier to analyze.

[0119] "Means of analysis using natural language processing techniques" refers to algorithms and models (e.g., BERT and GPT) that use collected and pre-processed text data to identify user interests and behavioral patterns.

[0120] "Means for generating suggestions" refers to algorithms and programs for automatically generating relevant suggestions for goods and services to the User based on the analyzed data.

[0121] "Means for notifying" refers to mechanisms and protocols for displaying generated suggestions on a user's electronic device in a timely manner.

[0122] "Means for collecting ratings" refers to an interface for users to input feedback and ratings on received suggestions, and a means for transmitting such data to the server.

[0123] "Means for calculating rewards" refers to the algorithms and systems for calculating rewards to advertisers and content creators based on collected user rating data.

[0124] "Means for distribution to providers" refers to the economic and technical mechanisms for appropriately distributing the calculated rewards to advertising providers and content creators.

[0125] The present invention relates to a system that collects data from a user's electronic devices, analyzes the data, and generates and notifies the user of relevant suggestions, thereby providing useful information to the user at the optimal time, making daily life more efficient and comfortable.

[0126] System configuration

[0127] The system of the present invention is comprised of the following major components:

[0128] User electronic devices: smart glasses, smartphones, etc.

[0129] Server: Central processing unit that stores data, preprocesses, analyzes, and generates proposals

[0130] Communications network: the infrastructure that connects users' electronic devices to servers and exchanges information

[0131] Explanation of program processing

[0132] Information gathering

[0133] Terminal

[0134] When a user wears smart glasses, they are equipped with a built-in camera that captures visual information from the user's daily life. Specifically, it collects the user's surroundings, search queries, voice commands, etc. The smartphone also uses its GPS function to record the user's location in real time. In addition, the smartphone also collects information on the usage of installed apps and the sending and receiving of text messages.

[0135] Sending data

[0136] Collected visual information, location information, voice commands and app usage data is periodically transmitted to a server using encrypted protocols such as HTTPS.

[0137] Data storage and preprocessing

[0138] server

[0139] The server stores the received information in a database. The data is classified into categories such as "visual information," "location information," and "text information." The stored raw data is then preprocessed to convert it into a format that is easier to analyze. Specifically, unnecessary image frames are removed from the visual information, and noisy audio data is filtered out.

[0140] Data analysis

[0141] The stored and pre-processed data is then analyzed using natural language processing (NLP) techniques, such as generative AI models like BERT and GPT, to identify user interests and behavioral patterns from the text data.

[0142] Proposal generation and notification

[0143] server

[0144] The server automatically generates relevant suggestions for the user based on the analysis results. For example, if a user searches for "stylish cafes," coupon information for nearby cafes will be generated. In addition, the server analyzes the user's location and time information to calculate a schedule for sending notifications at the optimal time.

[0145] Terminal

[0146] Smartphones or smart glasses receive notification information sent from the server and display it at a time that suits the user. For example, a user walking down the street might receive a push notification on their smartphone screen saying, "There's a popular cafe nearby."

[0147] Likes and reward distribution

[0148] User

[0149] Users can check the suggestions and give them a "like" if they find them useful. This rating is done via a button on the interface of their smartphone or smart glasses.

[0150] server

[0151] The server tracks the ratings of each ad and reminder based on the aggregated rating data, and calculates and fairly distributes rewards to ad providers and content creators based on the rating data.

[0152] Specific examples

[0153] For example, if a user searches for "stylish cafes" through smart glasses, the glasses capture visual information along with the search query and send it to a server. The server analyzes this information and generates advertisements for nearby cafes. It then sends a push notification to the user's smartphone at the optimal time while the user is walking in the area. If the user checks the notification and finds it useful, they can rate it highly, and rewards are distributed to the cafes and app developers that provided the advertisements based on the results.

[0154] Prompt Sentence Examples

[0155] If a user searches for "stylish cafes" using smart glasses, what process will be performed and what notification will be sent to the user as a result?

[0156] Please explain in detail how you will collect user rating data and distribute rewards.

[0157] The above is a specific embodiment of the system of the present invention.

[0158] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0159] Step 1:

[0160] Information gathering

[0161] The devices collect visual information, location information, voice commands, and app usage data from users' daily lives. Specifically, smart glasses use a built-in camera to capture visual information and a microphone to collect voice commands from users. Smartphones use GPS to record location information in real time, and also collect information on app usage and text message sending and receiving.

[0162] Input: Visual information from the user's daily life, location, voice commands, and app usage

[0163] Output: Collected information data

[0164] Step 2:

[0165] Sending data

[0166] The device sends the collected information data to the server. At this time, the data is sent securely using an encryption protocol such as HTTPS. For example, smart glasses and smartphones use batch processing to send collected data to the server at regular intervals.

[0167] Input: Collected information data

[0168] Output: Information data sent to the server

[0169] Step 3:

[0170] Data storage

[0171] The server receives the transmitted information data and stores it in a database. The data is stored in categories such as "visual information," "location information," and "text information."

[0172] Input: Transmitted information data

[0173] Output: Information stored in the database

[0174] Step 4:

[0175] Data Preprocessing

[0176] The server preprocesses the stored information data, specifically removing unnecessary image frames from visual information and noise from audio data, as well as filtering text data to standardize its format.

[0177] Input: Information stored in a database

[0178] Output: Preprocessed data

[0179] Step 5:

[0180] Natural Language Processing (NLP) Analysis

[0181] The server analyzes the preprocessed text data using natural language processing (NLP) technology, using generative AI models such as BERT and GPT, to identify interests and behavioral patterns from users' search queries and text messages.

[0182] Input: Preprocessed text data

[0183] Output: Analyzed user interests and behavior patterns

[0184] Step 6:

[0185] Proposal Generation

[0186] The server generates relevant suggestions for the user based on the analysis results. For example, if a user searches for "stylish cafes," it will generate coupon information for nearby cafes.

[0187] Input: Analyzed user interests and behavioral patterns

[0188] Output: Generated proposal information

[0189] Step 7:

[0190] Scheduling Notifications

[0191] The server calculates the optimal timing to notify the user of the suggested information based on the user's location and time information.

[0192] Input: Generated suggestion information, user location information, time information

[0193] Output: Notification Schedule

[0194] Step 8:

[0195] notification

[0196] The device receives the suggested information sent from the server and notifies the user, who is then provided with the information in the form of a push notification or display on their smartphone or smart glasses.

[0197] Input: Notification schedule, generated proposal information

[0198] Output: Notification displayed to the user

[0199] Step 9:

[0200] Collecting likes

[0201] The user checks the proposed information that has been notified to them, and if they find it useful, they give it a "like" through the system interface.

[0202] Input: Notified proposal information

[0203] Output: User likes data

[0204] Step 10:

[0205] Aggregation of evaluation data and reward calculation

[0206] The server collects user rating data, aggregates which suggestions and ads received high ratings, and then calculates rewards for advertising providers and content creators based on this data.

[0207] Input: User's high rating data

[0208] Output: Calculated reward amount

[0209] Step 11:

[0210] Reward distribution

[0211] The server distributes the calculated rewards to providers, providing them with economic incentives based on high ratings from users.

[0212] Input: Calculated reward amount

[0213] Output: Reward distribution to providers

[0214] In this way, each step works together to create a system that provides users with optimal information and distributes rewards based on their evaluation.

[0215] (Application example 1)

[0216] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0217] Conventional information provision systems have difficulty providing timely suggestions based on the user's interests and concerns, and these suggestions are not reflected in the user's visual perception in real time. As a result, the efficiency of providing optimal information to users is reduced, and user satisfaction cannot be increased.

[0218] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0219] In this invention, the server includes means for collecting information from the user's device, means for transmitting the collected information, means for storing the transmitted information, means for analyzing the stored information, means for generating suggestions related to the user, means for notifying the user of the generated suggestions, means for capturing the user's visual information and location information, means for analyzing the captured visual information, means for analyzing the user's location information, means for displaying the suggestions in the user's field of vision in real time, means for collecting user likes, means for calculating rewards based on the user likes, and means for distributing the rewards to providers. This makes it possible to utilize the user's visual information and location information in real time to provide optimal information based on the user's interests and concerns.

[0220] "User device" means an electronic device that collects and receives information, such as smart glasses or a smartphone.

[0221] "Information collection means" refers to devices or software that have the functionality to capture a user's visual information, location information, apps used, and other related data.

[0222] "Means for transmitting information" refers to a communication device or protocol for transmitting the collected information to a server over a network.

[0223] "Means for storing information" refers to a database or storage device for securely storing collected and transmitted data.

[0224] "Means for analyzing information" refers to algorithms and programs that use statistical analysis and natural language processing of stored data to identify user behavior patterns and interests.

[0225] The "means for generating suggestions" is a system that automatically creates advertisements and notifications relevant to users based on the analysis results.

[0226] "Means for notifying the user of the proposal" refers to a display or push notification function for displaying the generated proposal on the user's device.

[0227] A "means for capturing visual information" is an electronic device equipped with a camera or sensor to collect visual data about what the user sees.

[0228] A "means for capturing location information" is an electronic device equipped with a GPS function for determining the user's current location.

[0229] "Means for analyzing visual information" refers to image analysis technology for analyzing captured visual information and identifying user interests and behavior.

[0230] The "means for analyzing location information" is a geographic information system for analyzing the captured location information and identifying user behavior patterns.

[0231] "Means for displaying in real time within the visual field" refers to a function that displays suggestions in real time on smart glasses or a head-mounted display worn by the user.

[0232] The "means for collecting user likes" is an interface for users to record their likes on the suggestions provided and transmit them to the server.

[0233] The "means for calculating rewards based on high ratings" is a program that analyzes the collected rating data and calculates rewards to advertising providers and app developers based on the results.

[0234] The "means for distributing rewards to providers" is a system for transferring the calculated rewards to the appropriate providers.

[0235] The present invention relates to a system that utilizes a user's visual and location information to provide relevant suggestions in real time. The system collects information from the user's device (mainly smart glasses or smartphone), analyzes the information, and notifies the user at an appropriate time. The following describes an embodiment of the present invention.

[0236] System configuration

[0237] The system of the present invention comprises the following main components:

[0238] 1. User devices: Electronic devices such as smart glasses and smartphones that collect and notify information.

[0239] 2. Server: A central processing unit that stores, analyzes, and generates recommendations based on received information.

[0240] 3. Communication network: The infrastructure that connects user devices and servers and allows information to be exchanged.

[0241] Program Overview

[0242] Your device captures visual and location information, allowing it to understand your interests and behavioral patterns in real time and generate relevant suggestions.

[0243] Information collection and transmission

[0244] 1. Smart glasses capture the user's visual information through a camera and collect location information using GPS.

[0245] 2. Your smartphone records additional location information and information about the apps you use, which are periodically sent to a server.

[0246] Data storage and analysis

[0247] 1. The server stores the received information in a database, organizing the data into categories such as visual information, location information, and app usage information.

[0248] 2. Preprocessing is performed to filter out unnecessary data. Next, natural language processing (NLP) techniques are used to analyze the text data and identify user interests and behavioral patterns. Specifically, Google Cloud Natural Language API and Amazon Comprehend are used.

[0249] Proposal generation and notification

[0250] 1. The server generates advertisements and notifications relevant to the user based on the analysis results. For example, if the user shows interest in cafes, it generates information about nearby cafes.

[0251] 2. The system analyzes the user's location and time information and calculates the optimal timing for notifications. The generated suggestions are sent in real time to the smartglasses display or smartphone.

[0252] Specific examples

[0253] For example, while a user is walking around Shibuya, the smart glasses capture visual information such as "cafe" and send it along with the user's location information to a server. The server analyzes this information and generates advertisements for nearby cafes, which are then displayed on the smart glasses in real time.

[0254] Prompt Sentence Examples

[0255] An example of a prompt sentence to input to the generative AI model is as follows:

[0256] "A user is visiting the tourist destination of Shibuya. Their interest is in cafes. Please suggest popular cafes to the user in the area."

[0257] The above is an embodiment of the present invention. The present invention is capable of providing optimal information to users in real time through a series of processes including information collection, analysis, proposal generation, and notification.

[0258] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0259] Step 1:

[0260] The device captures visual and location information. Specifically, it uses the smart glasses' camera to collect the user's visual data and the GPS module to collect location data. These data are input.

[0261] Step 2:

[0262] The device transmits the captured data to a server. The collected visual and location information is sent to the server via the communication network (e.g., Wi-Fi or 4G / 5G) used by the smart glasses or smartphone.

[0263] Step 3:

[0264] The server stores the received information in a database, which includes categorizing and tagging it and organizing it into categories such as visual information, location information, app usage information, etc. The input is the captured data, and the output is the organized database.

[0265] Step 4:

[0266] The server preprocesses the stored data, filtering out unnecessary data and noise and cleansing the data. The input is the stored raw data and the output is the cleansed data.

[0267] Step 5:

[0268] The server analyzes the preprocessed data and uses natural language processing techniques (e.g., Google Cloud Natural Language API) to identify user interests and behavioral patterns. The input is the preprocessed data, and the output is the analysis results about user interests and patterns.

[0269] Step 6:

[0270] The server generates recommendations based on the analysis results. It uses a generative AI model (e.g., OpenAI GPT-3) to create relevant ads and notifications for the user. The input is the analysis results, and the output is the recommendations provided to the user.

[0271] Step 7:

[0272] The server analyzes the user's location and time information and calculates a schedule for sending suggestions at the optimal timing. The input is location and time information, and the output is a notification schedule.

[0273] Step 8:

[0274] The device receives the suggestion notification from the server and displays it in the user's field of vision in real time. The input is the suggestion notification sent from the server, and the output is the suggestion displayed on the smart glasses display.

[0275] Step 9:

[0276] Users can rate the provided suggestions by entering a positive rating for how helpful the suggestions were via the interface of smart glasses or a smartphone.

[0277] Step 10:

[0278] The server collects and analyzes user likes. It then aggregates which ads and notifications received how many likes they received. The input is the likes data from users, and the output is the aggregated results of the likes.

[0279] Step 11:

[0280] The server calculates rewards based on the tally results and distributes them to ad providers and app developers. Specifically, it calculates rewards based on evaluation data analysis and transfers them to a designated account. The input is the tally results of high ratings, and the output is the completion of reward distribution.

[0281] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0282] The present invention relates to a system that combines a system that collects information from a user's device, analyzes that information, and generates and notifies relevant suggestions to the user with an emotion engine that recognizes the user's emotions. This system recognizes the user's state in real time and makes it possible to provide appropriate suggestions and reminders according to that state.

[0283] System configuration

[0284] The system consists of the following main components:

[0285] User devices: Electronic devices used to collect information and provide notifications, such as smart glasses and smartphones.

[0286] Server: A central processing unit that stores data, analyzes it, and generates suggestions.

[0287] Emotion engine: Software for recognizing user emotions and analyzing and storing that information.

[0288] Communication network: The infrastructure that connects users' devices with the server and emotion engine.

[0289] Program processing

[0290] Information Collection and Storage

[0291] Terminal

[0292] 1. Smart glasses capture the user's visual information and collect emotional data such as voice commands and facial expressions.

[0293] 2. Your smartphone records your location, the app history you use, text messages, and voice input data.

[0294] 3. The collected information is periodically compressed and then sent to the server and emotion engine via the network.

[0295] Data storage and analysis

[0296] server

[0297] 1. The received information is stored in a database and classified into categories such as "visual information," "location information," "text information," and "emotional information."

[0298] 2. Use natural language processing (NLP) techniques to analyze text data and identify user interests and behavioral patterns.

[0299] Emotion Engine

[0300] 1. Analyze the user's tone of voice, facial expressions, and choice of words to recognize their emotional state in real time.

[0301] 2. The recognized emotion data is sent to the server for further analysis.

[0302] Proposal generation and notification

[0303] server

[0304] 1. Generate optimal suggestions and reminders based on user interest information, behavioral patterns, and emotional information.

[0305] 2. Calculate a schedule for optimal notifications, taking into account the user's location, time of day, and emotional state.

[0306] Terminal

[0307] 1. Receive notification information sent from the server and emotion engine.

[0308] 2. Notify the user of the received information, for example, via the smart glasses display or a smartphone push notification.

[0309] Likes and Feedback

[0310] User

[0311] 1. Check the ads and reminders you've been notified of and browse what interests you.

[0312] 2. If you find the ad or reminder useful, give it a thumbs up.

[0313] server

[0314] 1. Collect user rating data and analyze which suggestions and reminders received the highest ratings.

[0315] 2. Calculate rewards to providers based on their likes.

[0316] Use of emotion engine

[0317] Emotion Engine

[0318] 1. Analyze users' real-time emotional data and customize suggestions based on that information.

[0319] 2. Adjust the tone and wording of notifications depending on the user's emotional state. For example, if the user is feeling stressed, prioritize suggestions for relaxing activities.

[0320] Specific examples

[0321] For example, when a user searches for "stylish cafes" through smart glasses, the glasses capture this information and send it to the server along with related information. At the same time, the emotion engine analyzes the user's facial expressions and tone of voice to determine if the user is in a happy mood. The server analyzes this information and generates coupon advertisements for nearby cafes, suggesting cafes that best fit the user's current emotional state. While the user is walking around town, a notification appears on their smartphone saying, "There's a coupon for a nearby cafe." If the user finds this advertisement useful and rates it highly, the information is sent to the server, and rewards are distributed to the cafes and app developers that provided the advertisements based on the high ratings.

[0322] The above is a specific embodiment of the system of the present invention. The addition of an emotion engine enables more personalized suggestions tailored to the user's situation, further improving the user experience.

[0323] The processing flow will be explained below.

[0324] Step 1: Gather information

[0325] Terminal

[0326] 1. Smart glasses capture the user's visual information, specifically, images of the scenery, web pages, and search queries the user is viewing.

[0327] 2. Smart glasses collect the user's voice commands and facial expression information, which is used to infer the user's emotional state.

[0328] 3. Your smartphone records your location, app usage history, text messages, and voice data.

[0329] 4. The collected data is temporarily stored on the device, periodically compressed, and sent to the server and emotion engine via the network.

[0330] Step 2: Save your data

[0331] server

[0332] 1. Receives information sent from the device and stores it in a database. The data is classified into categories such as "visual information," "location information," "text information," and "emotional information."

[0333] 2. Preprocessing the stored data to filter out unnecessary information and noise, for example, excluding personally identifiable information to protect privacy.

[0334] Step 3: Analyze the data

[0335] server

[0336] 1. Analyze the stored text information using natural language processing (NLP) technology to identify user interests and behavioral patterns. For example, analyze the keyword "stylish cafe."

[0337] 2. Compare the data with the user's past behavioral data to extract patterns of interest and behavior. For example, find a pattern such as "going to a cafe every Wednesday."

[0338] Emotion Engine

[0339] 1. Recognize the user's emotional state by analyzing their tone of voice, facial expressions, and choice of words in real time.

[0340] 2. The recognized emotion data is sent to the server for further analysis.

[0341] Step 4: Generate proposals

[0342] server

[0343] 1. Based on the analysis results, generate relevant ads and reminders for users. For example, generate coupon ads for "trendy cafes."

[0344] 2. Calculate the optimal timing for notifications based on the user's location, time, and emotional state. For example, if the user is near a cafe, send a coupon at that time.

[0345] Step 5: Proposal Notification

[0346] Terminal

[0347] 1. Receives advertisements and reminders sent from the server, as well as emotion information from the emotion engine.

[0348] 2. Display the received information at a time that suits the user, for example, using the display on smart glasses or push notifications on a smartphone.

[0349] Step 6: Like and feedback

[0350] User

[0351] 1. Check the ad or reminder you received and view the details if you are interested.

[0352] 2. If you find the ad or reminder useful, give it a "like" through the UI, for example, through the smartglasses or smartphone interface.

[0353] Step 7: Counting likes and calculating rewards

[0354] server

[0355] 1. Collect data on user ratings. Analyze which ads and reminders received high ratings.

[0356] 2. Based on the likes, the app developer and advertisers will be rewarded. The reward amount will be determined according to the number of likes.

[0357] Step 8: Reward Distribution

[0358] server

[0359] 1. Distribute the calculated rewards to the providers. Record the reward distribution log and reflect it in the next reward calculation.

[0360] Specific examples

[0361] For example, when a user searches for "stylish cafes" through smart glasses, the glasses capture this information and send it to a server. At the same time, the emotion engine analyzes the user's happy facial expressions and tone of voice to recognize that the user is in a positive emotional state. The server uses this information to generate coupon ads for nearby cafes. Next, while the user is walking around town, a notification appears on the user's smartphone saying, "There's a coupon available for a nearby cafe." If the user finds the ad useful and rates it highly, the information is sent to the server, and rewards are distributed to the cafe or app developer that provided the ad based on the high rating.

[0362] The above is a specific embodiment of the system of the present invention. This process realizes a system that provides useful information to users at an appropriate time and distributes rewards to developers and providers based on their reactions.

[0363] Example 2

[0364] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0365] In recent years, with the increasing use of digital devices and the spread of wearable devices, there has been a growing demand for systems that can effectively collect and analyze individual behavioral and emotional data. However, existing systems do not adequately perform real-time analysis based on user emotions or generate personalized suggestions based on individual behavioral patterns. Furthermore, they lack mechanisms for efficiently reflecting user ratings and improving the quality of suggestions. As a result, the user experience does not improve and the effectiveness of the system is limited.

[0366] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0367] In this invention, the server includes: means for collecting information from the user's electronic device; means for storing the transmitted information; means for classifying the stored information into "visual information," "location information," "text information," "emotional information," etc.; means for analyzing the text information using natural language processing technology; means for recognizing the user's emotional state in real time using emotion analysis; means for scheduling notifications taking into account the user's current location, time of day, and emotional state; and means for collecting user ratings, calculating rewards based on the ratings, and distributing them to providers. This allows users to receive more personalized suggestions based on their emotions and behavior at the time, improving the user experience. Furthermore, utilizing real-time emotion analysis enables efficient generation and notification of suggestions.

[0368] "User's electronic device" refers to a portable electronic device such as a smartphone or smart glasses, which collects the user's visual information, location information, text information, and emotional information.

[0369] "Means for collecting information" refers to functions that use the user's electronic devices to collect visual information, voice commands, location information, text messages, voice input data, etc.

[0370] The "means for transmitting information" is a function that compresses the information collected from the terminal and transmits it to the server and emotion analysis system via a communication network.

[0371] "Means for storing information" refers to a function that stores the transmitted information in a database so that it can be analyzed or referenced later.

[0372] "Means for classifying information" is a function that organizes saved information into categories such as "visual information," "location information," "text information," and "emotional information."

[0373] "Means for analyzing text information using natural language processing technology" refers to technology that analyzes collected text messages and app usage history to identify users' interests and behavioral patterns.

[0374] "Means for recognizing a user's emotional state in real time using emotion analysis means" refers to a function that analyzes voice tone and facial expressions to identify a user's emotional state in real time.

[0375] "Means for scheduling notifications" is a feature that calculates the optimal time to notify based on the user's current location, time of day, and emotional state, and generates appropriate suggestions and reminders.

[0376] "Means for collecting likes" is a function that collects the interest and ratings that users show in suggestions and reminders and stores them as feedback.

[0377] The "means for calculating and distributing rewards to providers" is a function that analyzes which suggestions and reminders have been highly rated based on the high ratings, calculates rewards accordingly, and distributes them to providers.

[0378] The present invention is a system that collects and analyzes information from a user's electronic device to generate and notify relevant suggestions to the user. The system also incorporates an emotion analysis engine that recognizes the user's emotional state in real time and adaptively changes the content of the suggestions based on that information.

[0379] System configuration

[0380] The system consists of the following main components:

[0381] User electronic devices: Portable electronic devices such as smart glasses and smartphones collect users' visual information, voice commands, emotional data such as facial expressions, location information, text messages, and app usage history.

[0382] Server: A central processing unit that stores data, performs classification and analysis, and generates recommendations for users. Natural language processing technology is used here to analyze text information.

[0383] Emotion Analysis Engine: A software component that analyzes voice tone, facial expressions, and language to recognize the user's emotional state in real time.

[0384] Communication network: The infrastructure that connects users' electronic devices with the server and emotion analysis engine.

[0385] Information collection and analysis

[0386] Terminal

[0387] 1. When a user puts on the smart glasses and starts an activity, the smart glasses capture visual information and collect emotional data such as voice commands and facial expressions.

[0388] 2. Your smartphone regularly collects location information and records your app usage history, text messages, and voice input data.

[0389] 3. The data collected on the device is compressed and sent via a communication network to a server and emotion analysis engine.

[0390] server

[0391] 1. The server stores the information sent from the device in a database. The stored data is categorized into "visual information," "location information," "text information," "emotional information," etc.

[0392] 2. Using natural language processing (NLP) technology, the stored text data is analyzed to identify user interests and behavioral patterns.

[0393] 3. Real-time emotional data sent from the emotion analysis engine is also stored on the server and used for analysis.

[0394] Sentiment Analysis Engine

[0395] 1. The emotion analysis engine analyzes voice tone and facial expressions to recognize the user's emotional state in real time.

[0396] 2. The recognized emotion information is sent to the server for further analysis and suggestion generation.

[0397] Proposal generation and notification

[0398] server

[0399] 1. Generate optimal suggestions and reminders based on user interest information, behavioral patterns, and emotional information.

[0400] 2. Calculate the notification schedule and make suggestions at the right time, taking into account the user's location, time of day, and emotional state.

[0401] Terminal

[0402] 1. Receive notification information sent from the server and sentiment analysis engine.

[0403] 2. Users can check the received suggestions through the smart glasses display or push notifications on their smartphones.

[0404] Likes and Feedback

[0405] User

[0406] 1. Users can review the suggestions they receive and rate them highly if they find them useful.

[0407] 2. Likes are sent to the server via the device.

[0408] server

[0409] 1. Aggregate the received positive feedback data and analyze which proposals received positive feedback.

[0410] 2. Execute a process to calculate and distribute rewards to the providers of the suggestions based on the high ratings.

[0411] Specific examples

[0412] When a user searches for "stylish cafes" through the smart glasses, the glasses capture this information and send it along with related information to a server. At the same time, an emotion analysis engine analyzes the user's facial expressions and tone of voice to determine if the user is in a happy mood. The server analyzes this information and generates coupon advertisements for nearby cafes, and notifies the user on their smartphone while they are walking around town that "there is a coupon available for a nearby cafe."

[0413] Prompt Sentence Examples

[0414] While searching for "stylish cafes," the system analyzes the user's facial expressions and tone of voice, recognizes that the user is in a good mood, and sends recommendations and coupons for nearby cafes to the user's smartphone. Furthermore, if the user gives a high rating, the system also explains how the rewards will be distributed to the advertiser.

[0415] The above is a specific embodiment of the system of the present invention, which allows users to receive personalized suggestions based on their emotions and behavior at the time, and further improves the quality of the suggestions by efficiently incorporating high-rated feedback.

[0416] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0417] Step 1: Gather information

[0418] Terminal

[0419] 1. The user puts on the smart glasses and begins their daily activities.

[0420] Input: User visual information, voice commands, facial expression data

[0421] How it works: The smart glasses' camera captures visual information from the surroundings, the microphone collects voice commands and emotional data (such as tone and intonation), and facial recognition technology is used to collect facial expression data.

[0422] Output: Visual information, audio data, emotion data

[0423] 2. Your smartphone records your location, app usage history, text messages, and voice input data.

[0424] Input: GPS information, app usage history, text messages, voice input

[0425] How it works: It uses the smartphone's GPS to obtain location information, and records app usage history, text messages, and voice input in the background.

[0426] Output: Location information, app usage history, text messages, voice data

[0427] Step 2: Submit your information

[0428] Terminal

[0429] 1. The collected information is periodically compressed and sent to a server and sentiment analysis engine via a communication network.

[0430] Input: Collected visual information, audio data, emotional data, location information, app usage history, text messages

[0431] Specific operation: A data compression algorithm is applied to reduce the size of the transmitted data, and the data is transferred to the server and sentiment analysis engine using a secure communication protocol.

[0432] Output: Compressed data packets

[0433] Step 3: Store and classify data

[0434] server

[0435] 1. Save the submitted information in a database.

[0436] Input: Compressed data packet

[0437] Specific operation: The data packet is extracted and stored in a database. Each data item is classified into "visual information," "location information," "text information," and "emotion information."

[0438] Output: Categorized data entries

[0439] Step 4: Data analysis

[0440] server

[0441] 1. Analyze text data using natural language processing (NLP) techniques to identify user interests and behavioral patterns.

[0442] Input: Text information stored on the server

[0443] What it does: It applies NLP algorithms to analyze text data and extract user interests and behavioral patterns, thereby identifying what users are interested in.

[0444] Output: User interest information, behavioral patterns

[0445] Step 5: Analyze the sentiment data

[0446] Sentiment Analysis Engine

[0447] 1. Analyzes voice tone and facial expressions to recognize the user's emotional state in real time.

[0448] Input: Voice data and facial expression data sent from the server

[0449] How it works: It applies voice analysis and facial recognition algorithms to identify the user's emotional state in real time. For example, if the voice tone is high, it is judged to be "happy," and if it is low, it is judged to be "depressed."

[0450] Output: Real-time emotional state data

[0451] Step 6: Generate proposals

[0452] server

[0453] 1. Generate optimal suggestions and reminders based on user interest information, behavioral patterns, and emotional information.

[0454] Inputs: Interest information, behavioral patterns, real-time emotional state data

[0455] How it works: Using a generative AI model, it generates the most suitable suggestions and reminders for the user based on this data. For example, if the user is in a happy mood and interested in "cafes," it generates coupons for nearby cafes.

[0456] Output: Generated suggestions and reminders

[0457] Step 7: Scheduling and sending notifications

[0458] server

[0459] 1. Schedule notifications at optimal times, taking into account the user's location, time of day, and emotional state.

[0460] Input: current location, time, emotion

[0461] Specific behavior: Apply a scheduling algorithm to calculate the optimal timing for notifications. For example, if the user is walking to a nearby cafe, a coupon will be instantly notified.

[0462] Output: Scheduled notification information

[0463] Step 8: View notifications

[0464] Terminal

[0465] 1. Present notification information sent from the server and sentiment analysis engine to the user.

[0466] Input: Scheduled notification information

[0467] Specific operation: A notification is displayed to the user via the smart glasses display or a push notification on the smartphone.

[0468] Output: Suggestions and reminders presented to the user

[0469] Step 9: Collect likes

[0470] User

[0471] 1. Users review and like suggestions and reminders.

[0472] Input: Notified suggestions and reminders

[0473] Specific operation: Operate the "Like" button using smart glasses or a smartphone.

[0474] Output: Liked feedback

[0475] Step 10: Analyze Likes and Calculate Rewards

[0476] server

[0477] 1. Aggregate the received positive feedback data and analyze which proposals received positive feedback.

[0478] Input: Liked feedback

[0479] Specific operation: Stores the high-rating data in a database and uses statistical analysis to identify the high-rating suggestions.

[0480] Output: Highly rated proposal data

[0481] 2. Calculate rewards based on the highly rated data and distribute them to providers.

[0482] Input: Highly rated proposal data

[0483] Specific operation: Apply the reward calculation algorithm to calculate the reward for the proposal provider and perform distribution processing.

[0484] Output: Distributed reward data

[0485] (Application example 2)

[0486] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0487] Conventional food delivery services do not take into account the user's current emotional state when making suggestions, resulting in an insufficient user experience and the inability to recommend the most suitable dishes or restaurants for the user. Furthermore, the suggestions given to users are uniform and not personalized, resulting in low user satisfaction.

[0488] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0489] In this invention, the server includes means for collecting information from the user's device, means for transmitting the collected information, means for storing the transmitted information, means for analyzing the stored information, means for analyzing text data using natural language processing technology, an emotion engine means for analyzing voice tone, facial expressions, and wording to recognize an emotional state, means for generating suggestions relevant to the user based on the analysis results, means for notifying the user of the generated suggestions, means for collecting user ratings, means for calculating rewards based on the user ratings, and means for distributing rewards to providers. This enables optimal food delivery suggestions based on the user's emotional state in the real world, improving the user experience.

[0490] "User device" refers to an electronic device used by a user to collect and transmit information, such as smart glasses or a smartphone.

[0491] A "means for transmitting collected information" is a process or device for transmitting information from a user's terminal to a server.

[0492] The "means for storing transmitted information" refers to a system or process within the server for storing the received information in a storage device such as a database.

[0493] "Means for analyzing stored information" means the process of analyzing collected information to identify user behavior patterns and interests.

[0494] "Natural language processing technology" is a technology for analyzing text data and understanding and generating human language.

[0495] The "emotion engine that recognizes emotional states by analyzing voice tone, facial expressions, and language" is a software component that identifies emotions from the user's voice and facial expressions and evaluates their state in real time.

[0496] The "means for generating relevant suggestions for users based on analysis results" refers to a system for automatically creating optimal suggestions based on users' emotional and behavioral data.

[0497] "Means for notifying users of generated suggestions" refers to the process or device for delivering generated suggestions to users, such as a smartglasses display or a smartphone push notification.

[0498] The "means for collecting high ratings from users" is a system that allows users to rate proposals and collect the rating data.

[0499] The "means for calculating remuneration based on user's high ratings" is a process for calculating appropriate remuneration for the provider based on the collected high rating data.

[0500] "Means for distributing rewards to providers" refers to a system or process for distributing calculated rewards to advertisers and service providers.

[0501] "A means for suggesting optimal food delivery options based on real-world emotional states" is a system that automatically suggests the most suitable dishes and restaurants to users based on their real-time emotional data.

[0502] The present invention relates to a system for providing optimal food delivery suggestions based on a user's emotional state. The system includes a user terminal, a server, and an emotion engine.

[0503] System configuration

[0504] The system consists of the following main components:

[0505] User devices: Electronic devices such as smart glasses and smartphones that collect information and send notifications.

[0506] Server: A central processing unit that stores data, analyzes it, and generates suggestions.

[0507] Emotion engine: Software that analyzes voice tone, facial expressions, and language to recognize emotional states.

[0508] Hardware and software used

[0509] Smart glasses: capturing visual information and voice commands.

[0510] Smartphones: Collecting location, text messages, and voice input data.

[0511] Server: Python, Django, MySQL for data storage and analysis.

[0512] Emotion engine: TensorFlow and PyTorch for voice tone and facial expression analysis.

[0513] Communication network: Wi-Fi, 4G / 5G communication infrastructure.

[0514] What the program does

[0515] Information gathering

[0516] The user's devices collect information: smart glasses capture the user's visual information and voice commands, and smartphones record location information, text messages, and voice input data. This information is periodically compressed and sent over the network to a server.

[0517] Data storage and analysis

[0518] The server stores the received information in a database and categorizes it into categories such as "visual information," "location information," "text information," and "emotion information." Natural language processing (NLP) is used to analyze the text data and identify the user's interests and behavioral patterns. Meanwhile, the emotion engine analyzes voice tone, facial expressions, and vocabulary to recognize the user's emotional state in real time. The recognized emotion data is also sent to the server for further analysis.

[0519] Proposal generation and notification

[0520] The server generates optimal food delivery suggestions based on the user's interest information, behavioral patterns, and emotional information. The suggestions provide restaurants and dishes that best suit the user's current emotional state. The suggestions are delivered to the user via the smartglasses display or push notifications on their smartphone.

[0521] For example, if a user is feeling stressed, the system will send a notification to their smartphone suggesting a relaxing dish or restaurant. If the user rates the suggestion favorably, the data will be sent to the server, and appropriate rewards will be distributed to advertisers and food delivery service providers.

[0522] Example prompt sentence:

[0523] Capture visual information and voice commands from the user's smart glasses, and collect location information and text messages from their smartphone, then send them to a server. Design a system that uses this data to analyze emotions in real time with an emotion engine and suggest food delivery options that fit the user's current emotional state. This system should also display personalized restaurant and food recommendation notifications on the smartphone, and if the user gives a high rating, send that data to the provider to calculate a reward.

[0524] The present invention enables optimal food delivery suggestions tailored to the user's real-world emotional state, improving the user experience. This specific example details how a system can be implemented to provide personalized suggestions that take the user's emotional state into account.

[0525] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0526] Step 1:

[0527] Information gathering

[0528] The devices collect information. Smart glasses capture the user's visual information and voice commands, while the smartphone records location information, text messages, and voice input data. This input data (visual information, voice commands, location information, text messages, and voice input) is obtained from the user's device. This data includes facial recognition, voice analysis, and location tracking. The obtained data is temporarily stored on the device.

[0529] Step 2:

[0530] Data transmission

[0531] The device sends the collected information to the server. The collected data is pre-compressed and sent to the server over the network. The compressed information packets are input and transferred to the server via the network infrastructure. The compressed data is sent as output.

[0532] Step 3:

[0533] Data storage

[0534] The server stores the received information in a database. The database uses MySQL and stores the information by categorizing it into categories such as "visual information," "location information," "text information," and "emotional information." This data includes video frames, GPS data, text messages, and audio files. The input is compressed data packets, and the output is classified database entries.

[0535] Step 4:

[0536] Data analysis

[0537] The server analyzes the stored data. In particular, it uses natural language processing (NLP) to analyze text data and identify user interests and behavioral patterns. It uses TensorFlow and PyTorch to analyze voice tone and facial expressions to recognize emotional states in real time. The input is visual information, audio data, and text data read from the database, and the output is the analyzed user's behavioral patterns and emotional state.

[0538] Step 5:

[0539] Proposal generation

[0540] The server generates relevant suggestions for the user based on the analysis results. It combines the user's interest information, behavioral patterns, and emotional information to create optimal food delivery suggestions. The input is the analyzed behavioral patterns and emotional state data, and the output is specific restaurant and food suggestions.

[0541] Step 6:

[0542] notification

[0543] The device notifies the user of the generated suggestions. The suggestions are displayed to the user on the smartglasses display or via a smartphone push notification. The input is the generated suggestion data, and the output is a notification displayed on the user's device.

[0544] Step 7:

[0545] Evaluation collection

[0546] The terminal collects likes from users. The user rates the notified proposal and sends the rating data to the server. The input is the user's rating data, and the output is the rating information sent to the server.

[0547] Step 8:

[0548] Reward Calculation and Distribution

[0549] The server calculates rewards based on users' likes and distributes them to providers. Based on the collected likes data, appropriate rewards are calculated for advertisers and food delivery service providers. The input is the users' likes data, and the output is the calculated reward amount and its distribution information.

[0550] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0551] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0552] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0553] [Second embodiment]

[0554] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0555] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0556] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0557] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0558] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0559] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0560] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0561] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0562] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0563] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0564] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0565] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0566] The present invention relates to a system that collects information from a user's device, analyzes that information, and generates and notifies relevant suggestions to the user, with the aim of providing useful information to the user at the optimal time, making daily life more efficient and comfortable.

[0567] System configuration

[0568] First, the overall configuration of the system will be described. The system of the present invention is composed of the following main components.

[0569] User's devices: Electronic devices for collecting information and sending notifications, such as smart glasses and smartphones.

[0570] Server: A central processing unit that stores data, analyzes it, and generates suggestions.

[0571] Communications network: The infrastructure that connects user devices and servers to exchange information.

[0572] Program processing

[0573] Information Collection and Storage

[0574] Terminal

[0575] 1. Smart glasses capture visual information from the user's everyday life, such as the scenery they see, their search queries, and their voice commands.

[0576] 2. Your smartphone records your location, the apps you use, and your text messages.

[0577] 3. Once this information is collected, it is periodically sent to a server.

[0578] Data storage and analysis

[0579] server

[0580] 1. The received information is stored in a database. The data is organized by category, such as "visual information," "location information," and "text information."

[0581] 2. Preprocess the stored data and filter out unnecessary information.

[0582] 3. Analyze text data using natural language processing (NLP) techniques to identify user interests and behavioral patterns.

[0583] Proposal generation and notification

[0584] server

[0585] 1. Generate highly relevant ads and reminders based on user interests and behavioral patterns. For example, if a user searches for "stylish cafes," coupon information for popular nearby cafes will be generated.

[0586] 2. Analyze the user's location and time information and calculate a schedule for notifying them at the optimal time.

[0587] Terminal

[0588] 1. Receive notification information sent from the server.

[0589] 2. The notification is displayed at a time that is appropriate for the user, for example, via a smartglasses display or a smartphone push notification.

[0590] Likes and reward distribution

[0591] User

[0592] 1. Check the ads and reminders you receive and select the ones that interest you.

[0593] 2. If you find these useful, give them a "like" in the system interface.

[0594] server

[0595] 1. Aggregate user likes. Track which ads and reminders are liked and how many.

[0596] 2. Calculate and appropriately distribute rewards to the app developers.

[0597] Specific examples

[0598] For example, if a user searches for "stylish cafes" through smart glasses, the glasses capture this information and send it to a server. The server analyzes this information and generates advertisements for nearby cafes. These advertisements are then pushed to the user's smartphone at the optimal time while they are walking around town. If the user finds the advertisement useful, they can rate it highly. Based on this high rating, rewards are distributed to the cafes and app developers that provided the advertisements.

[0599] The above is a specific embodiment of the system of the present invention.

[0600] The processing flow will be explained below.

[0601] Step 1: Gather information

[0602] Terminal

[0603] 1. Smart glasses capture the user's visual information, specifically images of the scenery and objects the user is looking at, as well as search queries and voice commands.

[0604] 2. Your smartphone records your location in real time, and also collects your smartphone usage history and text messages.

[0605] 3. The collected data is periodically compressed and sent over the network to a server.

[0606] Step 2: Save your data

[0607] server

[0608] 1. The received information is stored in a database and classified into categories such as "location information," "visual information," and "text information."

[0609] 2. Preprocessing the information stored in the database to filter out noise and unnecessary information. To protect privacy, personally identifiable information is excluded.

[0610] Step 3: Analyze the data

[0611] server

[0612] 1. Analyze text information using natural language processing (NLP) technology. For example, if a user searches for "stylish cafe," analyze the meaning.

[0613] 2. Identify user interests and patterns by comparing them with past user behavior data. For example, find a pattern such as "I go to a cafe every Wednesday."

[0614] Step 4: Generate proposals

[0615] server

[0616] 1. Based on the analysis results, generate relevant ads and reminders for users. For example, generate coupon ads for "trendy cafes."

[0617] 2. Calculate the optimal timing for notifications based on the user's location and time of day. For example, if the user is near a cafe, send a coupon at that time.

[0618] Step 5: Proposal Notification

[0619] Terminal

[0620] 1. Receive advertisements and reminders sent from the server.

[0621] 2. The received information is notified to the user. This can be done using the smartglasses display or a push notification on a smartphone. For example, a notification could say, "There's a coupon you can use at a nearby cafe."

[0622] Step 6: Like and feedback

[0623] User

[0624] 1. Check the ad or reminder you received and view the details if you are interested.

[0625] 2. If you find an ad or reminder useful, you can give it a "like." You can do this through the smartglasses or smartphone interface.

[0626] Step 7: Counting likes and calculating rewards

[0627] server

[0628] 1. Collect data on user ratings. Analyze which ads and reminders received high ratings.

[0629] 2. Calculate rewards for advertising and reminder providers based on the likes. Determine the reward amount according to the number of likes.

[0630] Step 8: Reward Distribution

[0631] server

[0632] 1. Distribute the calculated rewards to the app developers and advertising providers.

[0633] 2. Record the distribution results and use them for the next reward calculation.

[0634] The above are the specific processing steps of the program of the present invention. This flow makes it possible to provide useful information to users at the optimal timing and to give incentives to developers based on high ratings.

[0635] Example 1

[0636] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0637] Conventional information provision systems face challenges in efficiently and effectively analyzing information collected from users' devices and generating personalized suggestions for users. They also lack a mechanism for properly reflecting users' high ratings and distributing rewards based on those ratings to providers. Furthermore, they do not provide notifications at optimal times that take into account the user's location and time information. This results in a poor user experience and reduces the effectiveness of information provision.

[0638] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0639] In this invention, the server includes means for collecting data from the user's electronic device, means for transmitting the collected data, means for storing the transmitted data, means for preprocessing the stored data, means for analyzing the preprocessed data using natural language processing technology, means for generating suggestions relevant to the user based on the analyzed data, means for notifying the user's electronic device of the generated suggestions, means for collecting user ratings, means for calculating rewards based on the user ratings, and means for distributing rewards to information providers. This makes it possible to generate personalized suggestions based on the user's interests and behavioral patterns and notify them at an appropriate time. Furthermore, by reflecting user rating data and distributing rewards fairly and effectively, it is possible to increase the motivation of information providers.

[0640] "User's electronic devices" refers to various types of electronic devices that users use on a daily basis, including smartphones, smart glasses, and tablet devices.

[0641] "Means of collecting data" refers to a combination of hardware and software that obtains information about a user's daily life and behavior through electronic devices.

[0642] "Means for transmitting data" refers to the protocols and functionality for transmitting collected data from the user's electronic device to the server.

[0643] "Means for storing data" refers to a system for temporarily or permanently storing transmitted data in a database on a server or on a storage medium.

[0644] "Preprocessing means" refers to processes and techniques that perform filtering and noise reduction to convert the stored raw data into a format that is easier to analyze.

[0645] "Means of analysis using natural language processing techniques" refers to algorithms and models (e.g., BERT and GPT) that use collected and pre-processed text data to identify user interests and behavioral patterns.

[0646] "Means for generating suggestions" refers to algorithms and programs for automatically generating relevant suggestions for goods and services to the User based on the analyzed data.

[0647] "Means for notifying" refers to mechanisms and protocols for displaying generated suggestions on a user's electronic device in a timely manner.

[0648] "Means for collecting ratings" refers to an interface for users to input feedback and ratings on received suggestions, and a means for transmitting such data to the server.

[0649] "Means for calculating rewards" refers to the algorithms and systems for calculating rewards to advertisers and content creators based on collected user rating data.

[0650] "Means for distribution to providers" refers to the economic and technical mechanisms for appropriately distributing the calculated rewards to advertising providers and content creators.

[0651] The present invention relates to a system that collects data from a user's electronic devices, analyzes the data, and generates and notifies the user of relevant suggestions, thereby providing useful information to the user at the optimal time, making daily life more efficient and comfortable.

[0652] System configuration

[0653] The system of the present invention is comprised of the following major components:

[0654] User electronic devices: smart glasses, smartphones, etc.

[0655] Server: Central processing unit that stores data, preprocesses, analyzes, and generates proposals

[0656] Communications network: the infrastructure that connects users' electronic devices to servers and exchanges information

[0657] Explanation of program processing

[0658] Information gathering

[0659] Terminal

[0660] When a user wears smart glasses, they are equipped with a built-in camera that captures visual information from the user's daily life. Specifically, it collects the user's surroundings, search queries, voice commands, etc. The smartphone also uses its GPS function to record the user's location in real time. In addition, the smartphone also collects information on the usage of installed apps and the sending and receiving of text messages.

[0661] Sending data

[0662] Collected visual information, location information, voice commands and app usage data is periodically transmitted to a server using encrypted protocols such as HTTPS.

[0663] Data storage and preprocessing

[0664] server

[0665] The server stores the received information in a database. The data is classified into categories such as "visual information," "location information," and "text information." The stored raw data is then preprocessed to convert it into a format that is easier to analyze. Specifically, unnecessary image frames are removed from the visual information, and noisy audio data is filtered out.

[0666] Data analysis

[0667] The stored and pre-processed data is then analyzed using natural language processing (NLP) techniques, such as generative AI models like BERT and GPT, to identify user interests and behavioral patterns from the text data.

[0668] Proposal generation and notification

[0669] server

[0670] The server automatically generates relevant suggestions for the user based on the analysis results. For example, if a user searches for "stylish cafes," coupon information for nearby cafes will be generated. In addition, the server analyzes the user's location and time information to calculate a schedule for sending notifications at the optimal time.

[0671] Terminal

[0672] Smartphones or smart glasses receive notification information sent from the server and display it at a time that suits the user. For example, a user walking down the street might receive a push notification on their smartphone screen saying, "There's a popular cafe nearby."

[0673] Likes and reward distribution

[0674] User

[0675] Users can check the suggestions and give them a "like" if they find them useful. This rating is done via a button on the interface of their smartphone or smart glasses.

[0676] server

[0677] The server tracks the ratings of each ad and reminder based on the aggregated rating data, and calculates and fairly distributes rewards to ad providers and content creators based on the rating data.

[0678] Specific examples

[0679] For example, if a user searches for "stylish cafes" through smart glasses, the glasses capture visual information along with the search query and send it to a server. The server analyzes this information and generates advertisements for nearby cafes. It then sends a push notification to the user's smartphone at the optimal time while the user is walking in the area. If the user checks the notification and finds it useful, they can rate it highly, and rewards are distributed to the cafes and app developers that provided the advertisements based on the results.

[0680] Prompt Sentence Examples

[0681] If a user searches for "stylish cafes" using smart glasses, what process will be performed and what notification will be sent to the user as a result?

[0682] Please explain in detail how you will collect user rating data and distribute rewards.

[0683] The above is a specific embodiment of the system of the present invention.

[0684] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0685] Step 1:

[0686] Information gathering

[0687] The devices collect visual information, location information, voice commands, and app usage data from users' daily lives. Specifically, smart glasses use a built-in camera to capture visual information and a microphone to collect voice commands from users. Smartphones use GPS to record location information in real time, and also collect information on app usage and text message sending and receiving.

[0688] Input: Visual information from the user's daily life, location, voice commands, and app usage

[0689] Output: Collected information data

[0690] Step 2:

[0691] Sending data

[0692] The device sends the collected information data to the server. At this time, the data is sent securely using an encryption protocol such as HTTPS. For example, smart glasses and smartphones use batch processing to send collected data to the server at regular intervals.

[0693] Input: Collected information data

[0694] Output: Information data sent to the server

[0695] Step 3:

[0696] Data storage

[0697] The server receives the transmitted information data and stores it in a database. The data is stored in categories such as "visual information," "location information," and "text information."

[0698] Input: Transmitted information data

[0699] Output: Information stored in the database

[0700] Step 4:

[0701] Data Preprocessing

[0702] The server preprocesses the stored information data, specifically removing unnecessary image frames from visual information and noise from audio data, as well as filtering text data to standardize its format.

[0703] Input: Information stored in a database

[0704] Output: Preprocessed data

[0705] Step 5:

[0706] Natural Language Processing (NLP) Analysis

[0707] The server analyzes the preprocessed text data using natural language processing (NLP) technology, using generative AI models such as BERT and GPT, to identify interests and behavioral patterns from users' search queries and text messages.

[0708] Input: Preprocessed text data

[0709] Output: Analyzed user interests and behavior patterns

[0710] Step 6:

[0711] Proposal Generation

[0712] The server generates relevant suggestions for the user based on the analysis results. For example, if a user searches for "stylish cafes," it will generate coupon information for nearby cafes.

[0713] Input: Analyzed user interests and behavioral patterns

[0714] Output: Generated proposal information

[0715] Step 7:

[0716] Scheduling Notifications

[0717] The server calculates the optimal timing to notify the user of the suggested information based on the user's location and time information.

[0718] Input: Generated suggestion information, user location information, time information

[0719] Output: Notification Schedule

[0720] Step 8:

[0721] notification

[0722] The device receives the suggested information sent from the server and notifies the user, who is then provided with the information in the form of a push notification or display on their smartphone or smart glasses.

[0723] Input: Notification schedule, generated proposal information

[0724] Output: Notification displayed to the user

[0725] Step 9:

[0726] Collecting likes

[0727] The user checks the proposed information that has been notified to them, and if they find it useful, they give it a "like" through the system interface.

[0728] Input: Notified proposal information

[0729] Output: User likes data

[0730] Step 10:

[0731] Aggregation of evaluation data and reward calculation

[0732] The server collects user rating data, aggregates which suggestions and ads received high ratings, and then calculates rewards for advertising providers and content creators based on this data.

[0733] Input: User's high rating data

[0734] Output: Calculated reward amount

[0735] Step 11:

[0736] Reward distribution

[0737] The server distributes the calculated rewards to providers, providing them with economic incentives based on high ratings from users.

[0738] Input: Calculated reward amount

[0739] Output: Reward distribution to providers

[0740] In this way, each step works together to create a system that provides users with optimal information and distributes rewards based on their evaluation.

[0741] (Application example 1)

[0742] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0743] Conventional information provision systems have difficulty providing timely suggestions based on the user's interests and concerns, and these suggestions are not reflected in the user's visual perception in real time. As a result, the efficiency of providing optimal information to users is reduced, and user satisfaction cannot be increased.

[0744] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0745] In this invention, the server includes means for collecting information from the user's device, means for transmitting the collected information, means for storing the transmitted information, means for analyzing the stored information, means for generating suggestions related to the user, means for notifying the user of the generated suggestions, means for capturing the user's visual information and location information, means for analyzing the captured visual information, means for analyzing the user's location information, means for displaying the suggestions in the user's field of vision in real time, means for collecting user likes, means for calculating rewards based on the user likes, and means for distributing the rewards to providers. This makes it possible to utilize the user's visual information and location information in real time to provide optimal information based on the user's interests and concerns.

[0746] "User device" means an electronic device that collects and receives information, such as smart glasses or a smartphone.

[0747] "Information collection means" refers to devices or software that have the functionality to capture a user's visual information, location information, apps used, and other related data.

[0748] "Means for transmitting information" refers to a communication device or protocol for transmitting the collected information to a server over a network.

[0749] "Means for storing information" refers to a database or storage device for securely storing collected and transmitted data.

[0750] "Means for analyzing information" refers to algorithms and programs that use statistical analysis and natural language processing of stored data to identify user behavior patterns and interests.

[0751] The "means for generating suggestions" is a system that automatically creates advertisements and notifications relevant to users based on the analysis results.

[0752] "Means for notifying the user of the proposal" refers to a display or push notification function for displaying the generated proposal on the user's device.

[0753] A "means for capturing visual information" is an electronic device equipped with a camera or sensor to collect visual data about what the user sees.

[0754] A "means for capturing location information" is an electronic device equipped with a GPS function for determining the user's current location.

[0755] "Means for analyzing visual information" refers to image analysis technology for analyzing captured visual information and identifying user interests and behavior.

[0756] The "means for analyzing location information" is a geographic information system for analyzing the captured location information and identifying user behavior patterns.

[0757] "Means for displaying in real time within the visual field" refers to a function that displays suggestions in real time on smart glasses or a head-mounted display worn by the user.

[0758] The "means for collecting user likes" is an interface for users to record their likes on the suggestions provided and transmit them to the server.

[0759] The "means for calculating rewards based on high ratings" is a program that analyzes the collected rating data and calculates rewards to advertising providers and app developers based on the results.

[0760] The "means for distributing rewards to providers" is a system for transferring the calculated rewards to the appropriate providers.

[0761] The present invention relates to a system that utilizes a user's visual and location information to provide relevant suggestions in real time. The system collects information from the user's device (mainly smart glasses or smartphone), analyzes the information, and notifies the user at an appropriate time. The following describes an embodiment of the present invention.

[0762] System configuration

[0763] The system of the present invention comprises the following main components:

[0764] 1. User devices: Electronic devices such as smart glasses and smartphones that collect and notify information.

[0765] 2. Server: A central processing unit that stores, analyzes, and generates recommendations based on received information.

[0766] 3. Communication network: The infrastructure that connects user devices and servers and allows information to be exchanged.

[0767] Program Overview

[0768] Your device captures visual and location information, allowing it to understand your interests and behavioral patterns in real time and generate relevant suggestions.

[0769] Information collection and transmission

[0770] 1. Smart glasses capture the user's visual information through a camera and collect location information using GPS.

[0771] 2. Your smartphone records additional location information and information about the apps you use, which are periodically sent to a server.

[0772] Data storage and analysis

[0773] 1. The server stores the received information in a database, organizing the data into categories such as visual information, location information, and app usage information.

[0774] 2. Preprocessing is performed to filter out unnecessary data. Next, natural language processing (NLP) techniques are used to analyze the text data and identify user interests and behavioral patterns. Specifically, Google Cloud Natural Language API and Amazon Comprehend are used.

[0775] Proposal generation and notification

[0776] 1. The server generates advertisements and notifications relevant to the user based on the analysis results. For example, if the user shows interest in cafes, it generates information about nearby cafes.

[0777] 2. The system analyzes the user's location and time information and calculates the optimal timing for notifications. The generated suggestions are sent in real time to the smartglasses display or smartphone.

[0778] Specific examples

[0779] For example, while a user is walking around Shibuya, the smart glasses capture visual information such as "cafe" and send it along with the user's location information to a server. The server analyzes this information and generates advertisements for nearby cafes, which are then displayed on the smart glasses in real time.

[0780] Prompt Sentence Examples

[0781] An example of a prompt sentence to input to the generative AI model is as follows:

[0782] "A user is visiting the tourist destination of Shibuya. Their interest is in cafes. Please suggest popular cafes to the user in the area."

[0783] The above is an embodiment of the present invention. The present invention is capable of providing optimal information to users in real time through a series of processes including information collection, analysis, proposal generation, and notification.

[0784] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0785] Step 1:

[0786] The device captures visual and location information. Specifically, it uses the smart glasses' camera to collect the user's visual data and the GPS module to collect location data. These data are input.

[0787] Step 2:

[0788] The device transmits the captured data to a server. The collected visual and location information is sent to the server via the communication network (e.g., Wi-Fi or 4G / 5G) used by the smart glasses or smartphone.

[0789] Step 3:

[0790] The server stores the received information in a database, which includes categorizing and tagging it and organizing it into categories such as visual information, location information, app usage information, etc. The input is the captured data, and the output is the organized database.

[0791] Step 4:

[0792] The server preprocesses the stored data, filtering out unnecessary data and noise and cleansing the data. The input is the stored raw data and the output is the cleansed data.

[0793] Step 5:

[0794] The server analyzes the preprocessed data and uses natural language processing techniques (e.g., Google Cloud Natural Language API) to identify user interests and behavioral patterns. The input is the preprocessed data, and the output is the analysis results about user interests and patterns.

[0795] Step 6:

[0796] The server generates recommendations based on the analysis results. It uses a generative AI model (e.g., OpenAI GPT-3) to create relevant ads and notifications for the user. The input is the analysis results, and the output is the recommendations provided to the user.

[0797] Step 7:

[0798] The server analyzes the user's location and time information and calculates a schedule for sending suggestions at the optimal timing. The input is location and time information, and the output is a notification schedule.

[0799] Step 8:

[0800] The device receives the suggestion notification from the server and displays it in the user's field of vision in real time. The input is the suggestion notification sent from the server, and the output is the suggestion displayed on the smart glasses display.

[0801] Step 9:

[0802] Users can rate the provided suggestions by entering a positive rating for how helpful the suggestions were via the interface of smart glasses or a smartphone.

[0803] Step 10:

[0804] The server collects and analyzes user likes. It then aggregates which ads and notifications received how many likes they received. The input is the likes data from users, and the output is the aggregated results of the likes.

[0805] Step 11:

[0806] The server calculates rewards based on the tally results and distributes them to ad providers and app developers. Specifically, it calculates rewards based on evaluation data analysis and transfers them to a designated account. The input is the tally results of high ratings, and the output is the completion of reward distribution.

[0807] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0808] The present invention relates to a system that combines a system that collects information from a user's device, analyzes that information, and generates and notifies relevant suggestions to the user with an emotion engine that recognizes the user's emotions. This system recognizes the user's state in real time and makes it possible to provide appropriate suggestions and reminders according to that state.

[0809] System configuration

[0810] The system consists of the following main components:

[0811] User devices: Electronic devices used to collect information and provide notifications, such as smart glasses and smartphones.

[0812] Server: A central processing unit that stores data, analyzes it, and generates suggestions.

[0813] Emotion engine: Software for recognizing user emotions and analyzing and storing that information.

[0814] Communication network: The infrastructure that connects users' devices with the server and emotion engine.

[0815] Program processing

[0816] Information Collection and Storage

[0817] Terminal

[0818] 1. Smart glasses capture the user's visual information and collect emotional data such as voice commands and facial expressions.

[0819] 2. Your smartphone records your location, the app history you use, text messages, and voice input data.

[0820] 3. The collected information is periodically compressed and then sent to the server and emotion engine via the network.

[0821] Data storage and analysis

[0822] server

[0823] 1. The received information is stored in a database and classified into categories such as "visual information," "location information," "text information," and "emotional information."

[0824] 2. Use natural language processing (NLP) techniques to analyze text data and identify user interests and behavioral patterns.

[0825] Emotion Engine

[0826] 1. Analyze the user's tone of voice, facial expressions, and choice of words to recognize their emotional state in real time.

[0827] 2. The recognized emotion data is sent to the server for further analysis.

[0828] Proposal generation and notification

[0829] server

[0830] 1. Generate optimal suggestions and reminders based on user interest information, behavioral patterns, and emotional information.

[0831] 2. Calculate a schedule for optimal notifications, taking into account the user's location, time of day, and emotional state.

[0832] Terminal

[0833] 1. Receive notification information sent from the server and emotion engine.

[0834] 2. Notify the user of the received information, for example, via the smart glasses display or a smartphone push notification.

[0835] Likes and Feedback

[0836] User

[0837] 1. Check the ads and reminders you've been notified of and browse what interests you.

[0838] 2. If you find the ad or reminder useful, give it a thumbs up.

[0839] server

[0840] 1. Collect user rating data and analyze which suggestions and reminders received the highest ratings.

[0841] 2. Calculate rewards to providers based on their likes.

[0842] Use of emotion engine

[0843] Emotion Engine

[0844] 1. Analyze users' real-time emotional data and customize suggestions based on that information.

[0845] 2. Adjust the tone and wording of notifications depending on the user's emotional state. For example, if the user is feeling stressed, prioritize suggestions for relaxing activities.

[0846] Specific examples

[0847] For example, when a user searches for "stylish cafes" through smart glasses, the glasses capture this information and send it to the server along with related information. At the same time, the emotion engine analyzes the user's facial expressions and tone of voice to determine if the user is in a happy mood. The server analyzes this information and generates coupon advertisements for nearby cafes, suggesting cafes that best fit the user's current emotional state. While the user is walking around town, a notification appears on their smartphone saying, "There's a coupon for a nearby cafe." If the user finds this advertisement useful and rates it highly, the information is sent to the server, and rewards are distributed to the cafes and app developers that provided the advertisements based on the high ratings.

[0848] The above is a specific embodiment of the system of the present invention. The addition of an emotion engine enables more personalized suggestions tailored to the user's situation, further improving the user experience.

[0849] The processing flow will be explained below.

[0850] Step 1: Gather information

[0851] Terminal

[0852] 1. Smart glasses capture the user's visual information, specifically, images of the scenery, web pages, and search queries the user is viewing.

[0853] 2. Smart glasses collect the user's voice commands and facial expression information, which is used to infer the user's emotional state.

[0854] 3. Your smartphone records your location, app usage history, text messages, and voice data.

[0855] 4. The collected data is temporarily stored on the device, periodically compressed, and sent to the server and emotion engine via the network.

[0856] Step 2: Save your data

[0857] server

[0858] 1. Receives information sent from the device and stores it in a database. The data is classified into categories such as "visual information," "location information," "text information," and "emotional information."

[0859] 2. Preprocessing the stored data to filter out unnecessary information and noise, for example, excluding personally identifiable information to protect privacy.

[0860] Step 3: Analyze the data

[0861] server

[0862] 1. Analyze the stored text information using natural language processing (NLP) technology to identify user interests and behavioral patterns. For example, analyze the keyword "stylish cafe."

[0863] 2. Compare the data with the user's past behavioral data to extract patterns of interest and behavior. For example, find a pattern such as "going to a cafe every Wednesday."

[0864] Emotion Engine

[0865] 1. Recognize the user's emotional state by analyzing their tone of voice, facial expressions, and choice of words in real time.

[0866] 2. The recognized emotion data is sent to the server for further analysis.

[0867] Step 4: Generate proposals

[0868] server

[0869] 1. Based on the analysis results, generate relevant ads and reminders for users. For example, generate coupon ads for "trendy cafes."

[0870] 2. Calculate the optimal timing for notifications based on the user's location, time, and emotional state. For example, if the user is near a cafe, send a coupon at that time.

[0871] Step 5: Proposal Notification

[0872] Terminal

[0873] 1. Receives advertisements and reminders sent from the server, as well as emotion information from the emotion engine.

[0874] 2. Display the received information at a time that suits the user, for example, using the display on smart glasses or push notifications on a smartphone.

[0875] Step 6: Like and feedback

[0876] User

[0877] 1. Check the ad or reminder you received and view the details if you are interested.

[0878] 2. If you find the ad or reminder useful, give it a "like" through the UI, for example, through the smartglasses or smartphone interface.

[0879] Step 7: Counting likes and calculating rewards

[0880] server

[0881] 1. Collect data on user ratings. Analyze which ads and reminders received high ratings.

[0882] 2. Based on the likes, the app developer and advertisers will be rewarded. The reward amount will be determined according to the number of likes.

[0883] Step 8: Reward Distribution

[0884] server

[0885] 1. Distribute the calculated rewards to the providers. Record the reward distribution log and reflect it in the next reward calculation.

[0886] Specific examples

[0887] For example, when a user searches for "stylish cafes" through smart glasses, the glasses capture this information and send it to a server. At the same time, the emotion engine analyzes the user's happy facial expressions and tone of voice to recognize that the user is in a positive emotional state. The server uses this information to generate coupon ads for nearby cafes. Next, while the user is walking around town, a notification appears on the user's smartphone saying, "There's a coupon available for a nearby cafe." If the user finds the ad useful and rates it highly, the information is sent to the server, and rewards are distributed to the cafe or app developer that provided the ad based on the high rating.

[0888] The above is a specific embodiment of the system of the present invention. This process realizes a system that provides useful information to users at an appropriate time and distributes rewards to developers and providers based on their reactions.

[0889] Example 2

[0890] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0891] In recent years, with the increasing use of digital devices and the spread of wearable devices, there has been a growing demand for systems that can effectively collect and analyze individual behavioral and emotional data. However, existing systems do not adequately perform real-time analysis based on user emotions or generate personalized suggestions based on individual behavioral patterns. Furthermore, they lack mechanisms for efficiently reflecting user ratings and improving the quality of suggestions. As a result, the user experience does not improve and the effectiveness of the system is limited.

[0892] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0893] In this invention, the server includes: means for collecting information from the user's electronic device; means for storing the transmitted information; means for classifying the stored information into "visual information," "location information," "text information," "emotional information," etc.; means for analyzing the text information using natural language processing technology; means for recognizing the user's emotional state in real time using emotion analysis; means for scheduling notifications taking into account the user's current location, time of day, and emotional state; and means for collecting user ratings, calculating rewards based on the ratings, and distributing them to providers. This allows users to receive more personalized suggestions based on their emotions and behavior at the time, improving the user experience. Furthermore, utilizing real-time emotion analysis enables efficient generation and notification of suggestions.

[0894] "User's electronic device" refers to a portable electronic device such as a smartphone or smart glasses, which collects the user's visual information, location information, text information, and emotional information.

[0895] "Means for collecting information" refers to functions that use the user's electronic devices to collect visual information, voice commands, location information, text messages, voice input data, etc.

[0896] The "means for transmitting information" is a function that compresses the information collected from the terminal and transmits it to the server and emotion analysis system via a communication network.

[0897] "Means for storing information" refers to a function that stores the transmitted information in a database so that it can be analyzed or referenced later.

[0898] "Means for classifying information" is a function that organizes saved information into categories such as "visual information," "location information," "text information," and "emotional information."

[0899] "Means for analyzing text information using natural language processing technology" refers to technology that analyzes collected text messages and app usage history to identify users' interests and behavioral patterns.

[0900] "Means for recognizing a user's emotional state in real time using emotion analysis means" refers to a function that analyzes voice tone and facial expressions to identify a user's emotional state in real time.

[0901] "Means for scheduling notifications" is a feature that calculates the optimal time to notify based on the user's current location, time of day, and emotional state, and generates appropriate suggestions and reminders.

[0902] "Means for collecting likes" is a function that collects the interest and ratings that users show in suggestions and reminders and stores them as feedback.

[0903] The "means for calculating and distributing rewards to providers" is a function that analyzes which suggestions and reminders have been highly rated based on the high ratings, calculates rewards accordingly, and distributes them to providers.

[0904] The present invention is a system that collects and analyzes information from a user's electronic device to generate and notify relevant suggestions to the user. The system also incorporates an emotion analysis engine that recognizes the user's emotional state in real time and adaptively changes the content of the suggestions based on that information.

[0905] System configuration

[0906] The system consists of the following main components:

[0907] User electronic devices: Portable electronic devices such as smart glasses and smartphones collect users' visual information, voice commands, emotional data such as facial expressions, location information, text messages, and app usage history.

[0908] Server: A central processing unit that stores data, performs classification and analysis, and generates recommendations for users. Natural language processing technology is used here to analyze text information.

[0909] Emotion Analysis Engine: A software component that analyzes voice tone, facial expressions, and language to recognize the user's emotional state in real time.

[0910] Communication network: The infrastructure that connects users' electronic devices with the server and emotion analysis engine.

[0911] Information collection and analysis

[0912] Terminal

[0913] 1. When a user puts on the smart glasses and starts an activity, the smart glasses capture visual information and collect emotional data such as voice commands and facial expressions.

[0914] 2. Your smartphone regularly collects location information and records your app usage history, text messages, and voice input data.

[0915] 3. The data collected on the device is compressed and sent via a communication network to a server and emotion analysis engine.

[0916] server

[0917] 1. The server stores the information sent from the device in a database. The stored data is categorized into "visual information," "location information," "text information," "emotional information," etc.

[0918] 2. Using natural language processing (NLP) technology, the stored text data is analyzed to identify user interests and behavioral patterns.

[0919] 3. Real-time emotional data sent from the emotion analysis engine is also stored on the server and used for analysis.

[0920] Sentiment Analysis Engine

[0921] 1. The emotion analysis engine analyzes voice tone and facial expressions to recognize the user's emotional state in real time.

[0922] 2. The recognized emotion information is sent to the server for further analysis and suggestion generation.

[0923] Proposal generation and notification

[0924] server

[0925] 1. Generate optimal suggestions and reminders based on user interest information, behavioral patterns, and emotional information.

[0926] 2. Calculate the notification schedule and make suggestions at the right time, taking into account the user's location, time of day, and emotional state.

[0927] Terminal

[0928] 1. Receive notification information sent from the server and sentiment analysis engine.

[0929] 2. Users can check the received suggestions through the smart glasses display or push notifications on their smartphones.

[0930] Likes and Feedback

[0931] User

[0932] 1. Users can review the suggestions they receive and rate them highly if they find them useful.

[0933] 2. Likes are sent to the server via the device.

[0934] server

[0935] 1. Aggregate the received positive feedback data and analyze which proposals received positive feedback.

[0936] 2. Execute a process to calculate and distribute rewards to the providers of the suggestions based on the high ratings.

[0937] Specific examples

[0938] When a user searches for "stylish cafes" through the smart glasses, the glasses capture this information and send it along with related information to a server. At the same time, an emotion analysis engine analyzes the user's facial expressions and tone of voice to determine if the user is in a happy mood. The server analyzes this information and generates coupon advertisements for nearby cafes, and notifies the user on their smartphone while they are walking around town that "there is a coupon available for a nearby cafe."

[0939] Prompt Sentence Examples

[0940] While searching for "stylish cafes," the system analyzes the user's facial expressions and tone of voice, recognizes that the user is in a good mood, and sends recommendations and coupons for nearby cafes to the user's smartphone. Furthermore, if the user gives a high rating, the system also explains how the rewards will be distributed to the advertiser.

[0941] The above is a specific embodiment of the system of the present invention, which allows users to receive personalized suggestions based on their emotions and behavior at the time, and further improves the quality of the suggestions by efficiently incorporating high-rated feedback.

[0942] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0943] Step 1: Gather information

[0944] Terminal

[0945] 1. The user puts on the smart glasses and begins their daily activities.

[0946] Input: User visual information, voice commands, facial expression data

[0947] How it works: The smart glasses' camera captures visual information from the surroundings, the microphone collects voice commands and emotional data (such as tone and intonation), and facial recognition technology is used to collect facial expression data.

[0948] Output: Visual information, audio data, emotion data

[0949] 2. Your smartphone records your location, app usage history, text messages, and voice input data.

[0950] Input: GPS information, app usage history, text messages, voice input

[0951] How it works: It uses the smartphone's GPS to obtain location information, and records app usage history, text messages, and voice input in the background.

[0952] Output: Location information, app usage history, text messages, voice data

[0953] Step 2: Submit your information

[0954] Terminal

[0955] 1. The collected information is periodically compressed and sent to a server and sentiment analysis engine via a communication network.

[0956] Input: Collected visual information, audio data, emotional data, location information, app usage history, text messages

[0957] Specific operation: A data compression algorithm is applied to reduce the size of the transmitted data, and the data is transferred to the server and sentiment analysis engine using a secure communication protocol.

[0958] Output: Compressed data packets

[0959] Step 3: Store and classify data

[0960] server

[0961] 1. Save the submitted information in a database.

[0962] Input: Compressed data packet

[0963] Specific operation: The data packet is extracted and stored in a database. Each data item is classified into "visual information," "location information," "text information," and "emotion information."

[0964] Output: Categorized data entries

[0965] Step 4: Data analysis

[0966] server

[0967] 1. Analyze text data using natural language processing (NLP) techniques to identify user interests and behavioral patterns.

[0968] Input: Text information stored on the server

[0969] What it does: It applies NLP algorithms to analyze text data and extract user interests and behavioral patterns, thereby identifying what users are interested in.

[0970] Output: User interest information, behavioral patterns

[0971] Step 5: Analyze the sentiment data

[0972] Sentiment Analysis Engine

[0973] 1. Analyzes voice tone and facial expressions to recognize the user's emotional state in real time.

[0974] Input: Voice data and facial expression data sent from the server

[0975] How it works: It applies voice analysis and facial recognition algorithms to identify the user's emotional state in real time. For example, if the voice tone is high, it is judged to be "happy," and if it is low, it is judged to be "depressed."

[0976] Output: Real-time emotional state data

[0977] Step 6: Generate proposals

[0978] server

[0979] 1. Generate optimal suggestions and reminders based on user interest information, behavioral patterns, and emotional information.

[0980] Inputs: Interest information, behavioral patterns, real-time emotional state data

[0981] How it works: Using a generative AI model, it generates the most suitable suggestions and reminders for the user based on this data. For example, if the user is in a happy mood and interested in "cafes," it generates coupons for nearby cafes.

[0982] Output: Generated suggestions and reminders

[0983] Step 7: Scheduling and sending notifications

[0984] server

[0985] 1. Schedule notifications at optimal times, taking into account the user's location, time of day, and emotional state.

[0986] Input: current location, time, emotion

[0987] Specific behavior: Apply a scheduling algorithm to calculate the optimal timing for notifications. For example, if the user is walking to a nearby cafe, a coupon will be instantly notified.

[0988] Output: Scheduled notification information

[0989] Step 8: View notifications

[0990] Terminal

[0991] 1. Present notification information sent from the server and sentiment analysis engine to the user.

[0992] Input: Scheduled notification information

[0993] Specific operation: A notification is displayed to the user via the smart glasses display or a push notification on the smartphone.

[0994] Output: Suggestions and reminders presented to the user

[0995] Step 9: Collect likes

[0996] User

[0997] 1. Users review and like suggestions and reminders.

[0998] Input: Notified suggestions and reminders

[0999] Specific operation: Operate the "Like" button using smart glasses or a smartphone.

[1000] Output: Liked feedback

[1001] Step 10: Analyze Likes and Calculate Rewards

[1002] server

[1003] 1. Aggregate the received positive feedback data and analyze which proposals received positive feedback.

[1004] Input: Liked feedback

[1005] Specific operation: Stores the high-rating data in a database and uses statistical analysis to identify the high-rating suggestions.

[1006] Output: Highly rated proposal data

[1007] 2. Calculate rewards based on the highly rated data and distribute them to providers.

[1008] Input: Highly rated proposal data

[1009] Specific operation: Apply the reward calculation algorithm to calculate the reward for the proposal provider and perform distribution processing.

[1010] Output: Distributed reward data

[1011] (Application example 2)

[1012] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1013] Conventional food delivery services do not take into account the user's current emotional state when making suggestions, resulting in an insufficient user experience and the inability to recommend the most suitable dishes or restaurants for the user. Furthermore, the suggestions given to users are uniform and not personalized, resulting in low user satisfaction.

[1014] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1015] In this invention, the server includes means for collecting information from the user's device, means for transmitting the collected information, means for storing the transmitted information, means for analyzing the stored information, means for analyzing text data using natural language processing technology, an emotion engine means for analyzing voice tone, facial expressions, and wording to recognize an emotional state, means for generating suggestions relevant to the user based on the analysis results, means for notifying the user of the generated suggestions, means for collecting user ratings, means for calculating rewards based on the user ratings, and means for distributing rewards to providers. This enables optimal food delivery suggestions based on the user's emotional state in the real world, improving the user experience.

[1016] "User device" refers to an electronic device used by a user to collect and transmit information, such as smart glasses or a smartphone.

[1017] A "means for transmitting collected information" is a process or device for transmitting information from a user's terminal to a server.

[1018] The "means for storing transmitted information" refers to a system or process within the server for storing the received information in a storage device such as a database.

[1019] "Means for analyzing stored information" means the process of analyzing collected information to identify user behavior patterns and interests.

[1020] "Natural language processing technology" is a technology for analyzing text data and understanding and generating human language.

[1021] The "emotion engine that recognizes emotional states by analyzing voice tone, facial expressions, and language" is a software component that identifies emotions from the user's voice and facial expressions and evaluates their state in real time.

[1022] The "means for generating relevant suggestions for users based on analysis results" refers to a system for automatically creating optimal suggestions based on users' emotional and behavioral data.

[1023] "Means for notifying users of generated suggestions" refers to the process or device for delivering generated suggestions to users, such as a smartglasses display or a smartphone push notification.

[1024] The "means for collecting high ratings from users" is a system that allows users to rate proposals and collect the rating data.

[1025] The "means for calculating remuneration based on user's high ratings" is a process for calculating appropriate remuneration for the provider based on the collected high rating data.

[1026] "Means for distributing rewards to providers" refers to a system or process for distributing calculated rewards to advertisers and service providers.

[1027] "A means for suggesting optimal food delivery options based on real-world emotional states" is a system that automatically suggests the most suitable dishes and restaurants to users based on their real-time emotional data.

[1028] The present invention relates to a system for providing optimal food delivery suggestions based on a user's emotional state. The system includes a user terminal, a server, and an emotion engine.

[1029] System configuration

[1030] The system consists of the following main components:

[1031] User devices: Electronic devices such as smart glasses and smartphones that collect information and send notifications.

[1032] Server: A central processing unit that stores data, analyzes it, and generates suggestions.

[1033] Emotion engine: Software that analyzes voice tone, facial expressions, and language to recognize emotional states.

[1034] Hardware and software used

[1035] Smart glasses: capturing visual information and voice commands.

[1036] Smartphones: Collecting location, text messages, and voice input data.

[1037] Server: Python, Django, MySQL for data storage and analysis.

[1038] Emotion engine: TensorFlow and PyTorch for voice tone and facial expression analysis.

[1039] Communication network: Wi-Fi, 4G / 5G communication infrastructure.

[1040] What the program does

[1041] Information gathering

[1042] The user's devices collect information: smart glasses capture the user's visual information and voice commands, and smartphones record location information, text messages, and voice input data. This information is periodically compressed and sent over the network to a server.

[1043] Data storage and analysis

[1044] The server stores the received information in a database and categorizes it into categories such as "visual information," "location information," "text information," and "emotion information." Natural language processing (NLP) is used to analyze the text data and identify the user's interests and behavioral patterns. Meanwhile, the emotion engine analyzes voice tone, facial expressions, and vocabulary to recognize the user's emotional state in real time. The recognized emotion data is also sent to the server for further analysis.

[1045] Proposal generation and notification

[1046] The server generates optimal food delivery suggestions based on the user's interest information, behavioral patterns, and emotional information. The suggestions provide restaurants and dishes that best suit the user's current emotional state. The suggestions are delivered to the user via the smartglasses display or push notifications on their smartphone.

[1047] For example, if a user is feeling stressed, the system will send a notification to their smartphone suggesting a relaxing dish or restaurant. If the user rates the suggestion favorably, the data will be sent to the server, and appropriate rewards will be distributed to advertisers and food delivery service providers.

[1048] Example prompt sentence:

[1049] Capture visual information and voice commands from the user's smart glasses, and collect location information and text messages from their smartphone, then send them to a server. Design a system that uses this data to analyze emotions in real time with an emotion engine and suggest food delivery options that fit the user's current emotional state. This system should also display personalized restaurant and food recommendation notifications on the smartphone, and if the user gives a high rating, send that data to the provider to calculate a reward.

[1050] The present invention enables optimal food delivery suggestions tailored to the user's real-world emotional state, improving the user experience. This specific example details how a system can be implemented to provide personalized suggestions that take the user's emotional state into account.

[1051] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1052] Step 1:

[1053] Information gathering

[1054] The devices collect information. Smart glasses capture the user's visual information and voice commands, while the smartphone records location information, text messages, and voice input data. This input data (visual information, voice commands, location information, text messages, and voice input) is obtained from the user's device. This data includes facial recognition, voice analysis, and location tracking. The obtained data is temporarily stored on the device.

[1055] Step 2:

[1056] Data transmission

[1057] The device sends the collected information to the server. The collected data is pre-compressed and sent to the server over the network. The compressed information packets are the input and are transferred to the server via the network infrastructure. The compressed data is sent as the output.

[1058] Step 3:

[1059] Data storage

[1060] The server stores the received information in a database. The database uses MySQL and stores the information by categorizing it into categories such as "visual information," "location information," "text information," and "emotional information." This data includes video frames, GPS data, text messages, and audio files. The input is compressed data packets, and the output is classified database entries.

[1061] Step 4:

[1062] Data analysis

[1063] The server analyzes the stored data. In particular, it uses natural language processing (NLP) to analyze text data and identify user interests and behavioral patterns. It uses TensorFlow and PyTorch to analyze voice tone and facial expressions to recognize emotional states in real time. The input is visual information, audio data, and text data read from the database, and the output is the analyzed user's behavioral patterns and emotional state.

[1064] Step 5:

[1065] Proposal generation

[1066] The server generates relevant suggestions for the user based on the analysis results. It combines the user's interest information, behavioral patterns, and emotional information to create optimal food delivery suggestions. The input is the analyzed behavioral patterns and emotional state data, and the output is specific restaurant and food suggestions.

[1067] Step 6:

[1068] notification

[1069] The device notifies the user of the generated suggestions. The suggestions are displayed to the user on the smartglasses display or via a smartphone push notification. The input is the generated suggestion data, and the output is a notification displayed on the user's device.

[1070] Step 7:

[1071] Evaluation collection

[1072] The terminal collects likes from users. The user rates the notified proposal and sends the rating data to the server. The input is the user's rating data, and the output is the rating information sent to the server.

[1073] Step 8:

[1074] Reward Calculation and Distribution

[1075] The server calculates rewards based on users' likes and distributes them to providers. Based on the collected likes data, appropriate rewards are calculated for advertisers and food delivery service providers. The input is the users' likes data, and the output is the calculated reward amount and its distribution information.

[1076] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1077] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1078] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1079] [Third embodiment]

[1080] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1081] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1082] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1083] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1084] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1085] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1086] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1087] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1088] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1089] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1090] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1091] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1092] The present invention relates to a system that collects information from a user's device, analyzes that information, and generates and notifies relevant suggestions to the user, with the aim of providing useful information to the user at the optimal time, making daily life more efficient and comfortable.

[1093] System configuration

[1094] First, the overall configuration of the system will be described. The system of the present invention is composed of the following main components.

[1095] User's devices: Electronic devices for collecting information and sending notifications, such as smart glasses and smartphones.

[1096] Server: A central processing unit that stores data, analyzes it, and generates suggestions.

[1097] Communications network: The infrastructure that connects user devices and servers to exchange information.

[1098] Program processing

[1099] Information Collection and Storage

[1100] Terminal

[1101] 1. Smart glasses capture visual information from the user's everyday life, such as the scenery they see, their search queries, and their voice commands.

[1102] 2. Your smartphone records your location, the apps you use, and your text messages.

[1103] 3. Once this information is collected, it is periodically sent to a server.

[1104] Data storage and analysis

[1105] server

[1106] 1. The received information is stored in a database. The data is organized by category, such as "visual information," "location information," and "text information."

[1107] 2. Preprocess the stored data and filter out unnecessary information.

[1108] 3. Analyze text data using natural language processing (NLP) techniques to identify user interests and behavioral patterns.

[1109] Proposal generation and notification

[1110] server

[1111] 1. Generate highly relevant ads and reminders based on user interests and behavioral patterns. For example, if a user searches for "stylish cafes," coupon information for popular nearby cafes will be generated.

[1112] 2. Analyze the user's location and time information and calculate a schedule for notifying them at the optimal time.

[1113] Terminal

[1114] 1. Receive notification information sent from the server.

[1115] 2. The notification is displayed at a time that is appropriate for the user, for example, via a smartglasses display or a smartphone push notification.

[1116] Likes and reward distribution

[1117] User

[1118] 1. Check the ads and reminders you receive and select the ones that interest you.

[1119] 2. If you find these useful, give them a "like" in the system interface.

[1120] server

[1121] 1. Aggregate user likes. Track which ads and reminders are liked and how many.

[1122] 2. Calculate and appropriately distribute rewards to the app developers.

[1123] Specific examples

[1124] For example, if a user searches for "stylish cafes" through smart glasses, the glasses capture this information and send it to a server. The server analyzes this information and generates advertisements for nearby cafes. These advertisements are then pushed to the user's smartphone at the optimal time while they are walking around town. If the user finds the advertisement useful, they can rate it highly. Based on this high rating, rewards are distributed to the cafes and app developers that provided the advertisements.

[1125] The above is a specific embodiment of the system of the present invention.

[1126] The processing flow will be explained below.

[1127] Step 1: Gather information

[1128] Terminal

[1129] 1. Smart glasses capture the user's visual information, specifically images of the scenery and objects the user is looking at, as well as search queries and voice commands.

[1130] 2. Your smartphone records your location in real time, and also collects your smartphone usage history and text messages.

[1131] 3. The collected data is periodically compressed and sent over the network to a server.

[1132] Step 2: Save your data

[1133] server

[1134] 1. The received information is stored in a database and classified into categories such as "location information," "visual information," and "text information."

[1135] 2. Preprocessing the information stored in the database to filter out noise and unnecessary information. To protect privacy, personally identifiable information is excluded.

[1136] Step 3: Analyze the data

[1137] server

[1138] 1. Analyze text information using natural language processing (NLP) technology. For example, if a user searches for "stylish cafe," analyze the meaning.

[1139] 2. Identify user interests and patterns by comparing them with past user behavior data. For example, find a pattern such as "I go to a cafe every Wednesday."

[1140] Step 4: Generate proposals

[1141] server

[1142] 1. Based on the analysis results, generate relevant ads and reminders for users. For example, generate coupon ads for "trendy cafes."

[1143] 2. Calculate the optimal timing for notifications based on the user's location and time of day. For example, if the user is near a cafe, send a coupon at that time.

[1144] Step 5: Proposal Notification

[1145] Terminal

[1146] 1. Receive advertisements and reminders sent from the server.

[1147] 2. The received information is notified to the user. This can be done using the smartglasses display or a push notification on a smartphone. For example, a notification could say, "There's a coupon you can use at a nearby cafe."

[1148] Step 6: Like and feedback

[1149] User

[1150] 1. Check the ad or reminder you received and view the details if you are interested.

[1151] 2. If you find an ad or reminder useful, you can give it a "like." You can do this through the smartglasses or smartphone interface.

[1152] Step 7: Counting likes and calculating rewards

[1153] server

[1154] 1. Collect data on user ratings. Analyze which ads and reminders received high ratings.

[1155] 2. Calculate rewards for advertising and reminder providers based on the likes. Determine the reward amount according to the number of likes.

[1156] Step 8: Reward Distribution

[1157] server

[1158] 1. Distribute the calculated rewards to the app developers and advertising providers.

[1159] 2. Record the distribution results and use them for the next reward calculation.

[1160] The above are the specific processing steps of the program of the present invention. This flow makes it possible to provide useful information to users at the optimal timing and to give incentives to developers based on high ratings.

[1161] Example 1

[1162] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1163] Conventional information provision systems face challenges in efficiently and effectively analyzing information collected from users' devices and generating personalized suggestions for users. They also lack a mechanism for properly reflecting users' high ratings and distributing rewards based on those ratings to providers. Furthermore, they do not provide notifications at optimal times that take into account the user's location and time information. This results in a poor user experience and reduces the effectiveness of information provision.

[1164] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1165] In this invention, the server includes means for collecting data from the user's electronic device, means for transmitting the collected data, means for storing the transmitted data, means for preprocessing the stored data, means for analyzing the preprocessed data using natural language processing technology, means for generating suggestions relevant to the user based on the analyzed data, means for notifying the user's electronic device of the generated suggestions, means for collecting user ratings, means for calculating rewards based on the user ratings, and means for distributing rewards to information providers. This makes it possible to generate personalized suggestions based on the user's interests and behavioral patterns and notify them at an appropriate time. Furthermore, by reflecting user rating data and distributing rewards fairly and effectively, it is possible to increase the motivation of information providers.

[1166] "User's electronic devices" refers to various types of electronic devices that users use on a daily basis, including smartphones, smart glasses, and tablet devices.

[1167] "Means of collecting data" refers to a combination of hardware and software that obtains information about a user's daily life and behavior through electronic devices.

[1168] "Means for transmitting data" refers to the protocols and functionality for transmitting collected data from the user's electronic device to the server.

[1169] "Means for storing data" refers to a system for temporarily or permanently storing transmitted data in a database on a server or on a storage medium.

[1170] "Preprocessing means" refers to processes and techniques that perform filtering and noise reduction to convert the stored raw data into a format that is easier to analyze.

[1171] "Means of analysis using natural language processing techniques" refers to algorithms and models (e.g., BERT and GPT) that use collected and pre-processed text data to identify user interests and behavioral patterns.

[1172] "Means for generating suggestions" refers to algorithms and programs for automatically generating relevant suggestions for goods and services to the User based on the analyzed data.

[1173] "Means for notifying" refers to mechanisms and protocols for displaying generated suggestions on a user's electronic device in a timely manner.

[1174] "Means for collecting ratings" refers to an interface for users to input feedback and ratings on received suggestions, and a means for transmitting such data to the server.

[1175] "Means for calculating rewards" refers to the algorithms and systems for calculating rewards to advertisers and content creators based on collected user rating data.

[1176] "Means for distribution to providers" refers to the economic and technical mechanisms for appropriately distributing the calculated rewards to advertising providers and content creators.

[1177] The present invention relates to a system that collects data from a user's electronic devices, analyzes the data, and generates and notifies the user of relevant suggestions, thereby providing useful information to the user at the optimal time, making daily life more efficient and comfortable.

[1178] System configuration

[1179] The system of the present invention is comprised of the following major components:

[1180] User electronic devices: smart glasses, smartphones, etc.

[1181] Server: Central processing unit that stores data, preprocesses, analyzes, and generates proposals

[1182] Communications network: the infrastructure that connects users' electronic devices to servers and exchanges information

[1183] Explanation of program processing

[1184] Information gathering

[1185] Terminal

[1186] When a user wears smart glasses, they are equipped with a built-in camera that captures visual information from the user's daily life. Specifically, it collects the user's surroundings, search queries, voice commands, etc. The smartphone also uses its GPS function to record the user's location in real time. In addition, the smartphone also collects information on the usage of installed apps and the sending and receiving of text messages.

[1187] Sending data

[1188] Collected visual information, location information, voice commands and app usage data is periodically transmitted to a server using encrypted protocols such as HTTPS.

[1189] Data storage and preprocessing

[1190] server

[1191] The server stores the received information in a database. The data is classified into categories such as "visual information," "location information," and "text information." The stored raw data is then preprocessed to convert it into a format that is easier to analyze. Specifically, unnecessary image frames are removed from the visual information, and noisy audio data is filtered out.

[1192] Data analysis

[1193] The stored and pre-processed data is then analyzed using natural language processing (NLP) techniques, such as generative AI models like BERT and GPT, to identify user interests and behavioral patterns from the text data.

[1194] Proposal generation and notification

[1195] server

[1196] The server automatically generates relevant suggestions for the user based on the analysis results. For example, if a user searches for "stylish cafes," coupon information for nearby cafes will be generated. In addition, the server analyzes the user's location and time information to calculate a schedule for sending notifications at the optimal time.

[1197] Terminal

[1198] Smartphones or smart glasses receive notification information sent from the server and display it at a time that suits the user. For example, a user walking down the street might receive a push notification on their smartphone screen saying, "There's a popular cafe nearby."

[1199] Likes and reward distribution

[1200] User

[1201] Users can check the suggestions and give them a "like" if they find them useful. This rating is done via a button on the interface of their smartphone or smart glasses.

[1202] server

[1203] The server tracks the ratings of each ad and reminder based on the aggregated rating data, and calculates and fairly distributes rewards to ad providers and content creators based on the rating data.

[1204] Specific examples

[1205] For example, if a user searches for "stylish cafes" through smart glasses, the glasses capture visual information along with the search query and send it to a server. The server analyzes this information and generates advertisements for nearby cafes. It then sends a push notification to the user's smartphone at the optimal time while the user is walking in the area. If the user checks the notification and finds it useful, they can rate it highly, and rewards are distributed to the cafes and app developers that provided the advertisements based on the results.

[1206] Prompt Sentence Examples

[1207] If a user searches for "stylish cafes" using smart glasses, what process will be performed and what notification will be sent to the user as a result?

[1208] Please explain in detail how you will collect user rating data and distribute rewards.

[1209] The above is a specific embodiment of the system of the present invention.

[1210] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1211] Step 1:

[1212] Information gathering

[1213] The devices collect visual information, location information, voice commands, and app usage data from users' daily lives. Specifically, smart glasses use a built-in camera to capture visual information and a microphone to collect voice commands from users. Smartphones use GPS to record location information in real time, and also collect information on app usage and text message sending and receiving.

[1214] Input: Visual information from the user's daily life, location, voice commands, and app usage

[1215] Output: Collected information data

[1216] Step 2:

[1217] Sending data

[1218] The device sends the collected information data to the server. At this time, the data is sent securely using an encryption protocol such as HTTPS. For example, smart glasses and smartphones use batch processing to send collected data to the server at regular intervals.

[1219] Input: Collected information data

[1220] Output: Information data sent to the server

[1221] Step 3:

[1222] Data storage

[1223] The server receives the transmitted information data and stores it in a database. The data is stored in categories such as "visual information," "location information," and "text information."

[1224] Input: Transmitted information data

[1225] Output: Information stored in the database

[1226] Step 4:

[1227] Data Preprocessing

[1228] The server preprocesses the stored information data, specifically removing unnecessary image frames from visual information and noise from audio data, as well as filtering text data to standardize its format.

[1229] Input: Information stored in a database

[1230] Output: Preprocessed data

[1231] Step 5:

[1232] Natural Language Processing (NLP) Analysis

[1233] The server analyzes the preprocessed text data using natural language processing (NLP) technology, using generative AI models such as BERT and GPT, to identify interests and behavioral patterns from users' search queries and text messages.

[1234] Input: Preprocessed text data

[1235] Output: Analyzed user interests and behavior patterns

[1236] Step 6:

[1237] Proposal Generation

[1238] The server generates relevant suggestions for the user based on the analysis results. For example, if a user searches for "stylish cafes," it will generate coupon information for nearby cafes.

[1239] Input: Analyzed user interests and behavioral patterns

[1240] Output: Generated proposal information

[1241] Step 7:

[1242] Scheduling Notifications

[1243] The server calculates the optimal timing to notify the user of the suggested information based on the user's location and time information.

[1244] Input: Generated suggestion information, user location information, time information

[1245] Output: Notification Schedule

[1246] Step 8:

[1247] notification

[1248] The device receives the suggested information sent from the server and notifies the user, who is then provided with the information in the form of a push notification or display on their smartphone or smart glasses.

[1249] Input: Notification schedule, generated proposal information

[1250] Output: Notification displayed to the user

[1251] Step 9:

[1252] Collecting likes

[1253] The user checks the proposed information that has been notified to them, and if they find it useful, they give it a "like" through the system interface.

[1254] Input: Notified proposal information

[1255] Output: User likes data

[1256] Step 10:

[1257] Aggregation of evaluation data and reward calculation

[1258] The server collects user rating data, aggregates which suggestions and ads received high ratings, and then calculates rewards for advertising providers and content creators based on this data.

[1259] Input: User's high rating data

[1260] Output: Calculated reward amount

[1261] Step 11:

[1262] Reward distribution

[1263] The server distributes the calculated rewards to providers, providing them with economic incentives based on high ratings from users.

[1264] Input: Calculated reward amount

[1265] Output: Reward distribution to providers

[1266] In this way, each step works together to create a system that provides users with optimal information and distributes rewards based on their evaluation.

[1267] (Application example 1)

[1268] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1269] Conventional information provision systems have difficulty providing timely suggestions based on the user's interests and concerns, and these suggestions are not reflected in the user's visual perception in real time. As a result, the efficiency of providing optimal information to users is reduced, and user satisfaction cannot be increased.

[1270] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1271] In this invention, the server includes means for collecting information from the user's device, means for transmitting the collected information, means for storing the transmitted information, means for analyzing the stored information, means for generating suggestions related to the user, means for notifying the user of the generated suggestions, means for capturing the user's visual information and location information, means for analyzing the captured visual information, means for analyzing the user's location information, means for displaying the suggestions in the user's field of vision in real time, means for collecting user likes, means for calculating rewards based on the user likes, and means for distributing the rewards to providers. This makes it possible to utilize the user's visual information and location information in real time to provide optimal information based on the user's interests and concerns.

[1272] "User device" means an electronic device that collects and receives information, such as smart glasses or a smartphone.

[1273] "Information collection means" refers to devices or software that have the functionality to capture a user's visual information, location information, apps used, and other related data.

[1274] "Means for transmitting information" refers to a communication device or protocol for transmitting the collected information to a server over a network.

[1275] "Means for storing information" refers to a database or storage device for securely storing collected and transmitted data.

[1276] "Means for analyzing information" refers to algorithms and programs that use statistical analysis and natural language processing of stored data to identify user behavior patterns and interests.

[1277] The "means for generating suggestions" is a system that automatically creates advertisements and notifications relevant to users based on the analysis results.

[1278] "Means for notifying the user of the proposal" refers to a display or push notification function for displaying the generated proposal on the user's device.

[1279] A "means for capturing visual information" is an electronic device equipped with a camera or sensor to collect visual data about what the user sees.

[1280] A "means for capturing location information" is an electronic device equipped with a GPS function for determining the user's current location.

[1281] "Means for analyzing visual information" refers to image analysis technology for analyzing captured visual information and identifying user interests and behavior.

[1282] The "means for analyzing location information" is a geographic information system for analyzing the captured location information and identifying user behavior patterns.

[1283] "Means for displaying in real time within the visual field" refers to a function that displays suggestions in real time on smart glasses or a head-mounted display worn by the user.

[1284] The "means for collecting user likes" is an interface for users to record their likes on the suggestions provided and transmit them to the server.

[1285] The "means for calculating rewards based on high ratings" is a program that analyzes the collected rating data and calculates rewards to advertising providers and app developers based on the results.

[1286] The "means for distributing rewards to providers" is a system for transferring the calculated rewards to the appropriate providers.

[1287] The present invention relates to a system that utilizes a user's visual and location information to provide relevant suggestions in real time. The system collects information from the user's device (mainly smart glasses or smartphone), analyzes the information, and notifies the user at an appropriate time. The following describes an embodiment of the present invention.

[1288] System configuration

[1289] The system of the present invention comprises the following main components:

[1290] 1. User devices: Electronic devices such as smart glasses and smartphones that collect and notify information.

[1291] 2. Server: A central processing unit that stores, analyzes, and generates recommendations based on received information.

[1292] 3. Communication network: The infrastructure that connects user devices and servers and allows information to be exchanged.

[1293] Program Overview

[1294] Your device captures visual and location information, allowing it to understand your interests and behavioral patterns in real time and generate relevant suggestions.

[1295] Information collection and transmission

[1296] 1. Smart glasses capture the user's visual information through a camera and collect location information using GPS.

[1297] 2. Your smartphone records additional location information and information about the apps you use, which are periodically sent to a server.

[1298] Data storage and analysis

[1299] 1. The server stores the received information in a database, organizing the data into categories such as visual information, location information, and app usage information.

[1300] 2. Preprocessing is performed to filter out unnecessary data. Next, natural language processing (NLP) techniques are used to analyze the text data and identify user interests and behavioral patterns. Specifically, Google Cloud Natural Language API and Amazon Comprehend are used.

[1301] Proposal generation and notification

[1302] 1. The server generates advertisements and notifications relevant to the user based on the analysis results. For example, if the user shows interest in cafes, it generates information about nearby cafes.

[1303] 2. The system analyzes the user's location and time information and calculates the optimal timing for notifications. The generated suggestions are sent in real time to the smartglasses display or smartphone.

[1304] Specific examples

[1305] For example, while a user is walking around Shibuya, the smart glasses capture visual information such as "cafe" and send it along with the user's location information to a server. The server analyzes this information and generates advertisements for nearby cafes, which are then displayed on the smart glasses in real time.

[1306] Prompt Sentence Examples

[1307] An example of a prompt sentence to input to the generative AI model is as follows:

[1308] "A user is visiting the tourist destination of Shibuya. Their interest is in cafes. Please suggest popular cafes to the user in the area."

[1309] The above is an embodiment of the present invention. The present invention is capable of providing optimal information to users in real time through a series of processes including information collection, analysis, proposal generation, and notification.

[1310] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1311] Step 1:

[1312] The device captures visual and location information. Specifically, it uses the smart glasses' camera to collect the user's visual data and the GPS module to collect location data. These data are input.

[1313] Step 2:

[1314] The device transmits the captured data to a server. The collected visual and location information is sent to the server via the communication network (e.g., Wi-Fi or 4G / 5G) used by the smart glasses or smartphone.

[1315] Step 3:

[1316] The server stores the received information in a database, which includes categorizing and tagging it and organizing it into categories such as visual information, location information, app usage information, etc. The input is the captured data, and the output is the organized database.

[1317] Step 4:

[1318] The server preprocesses the stored data, filtering out unnecessary data and noise and cleansing the data. The input is the stored raw data and the output is the cleansed data.

[1319] Step 5:

[1320] The server analyzes the preprocessed data and uses natural language processing techniques (e.g., Google Cloud Natural Language API) to identify user interests and behavioral patterns. The input is the preprocessed data, and the output is the analysis results about user interests and patterns.

[1321] Step 6:

[1322] The server generates recommendations based on the analysis results. It uses a generative AI model (e.g., OpenAI GPT-3) to create relevant ads and notifications for the user. The input is the analysis results, and the output is the recommendations provided to the user.

[1323] Step 7:

[1324] The server analyzes the user's location and time information and calculates a schedule for sending suggestions at the optimal timing. The input is location and time information, and the output is a notification schedule.

[1325] Step 8:

[1326] The device receives the suggestion notification from the server and displays it in the user's field of vision in real time. The input is the suggestion notification sent from the server, and the output is the suggestion displayed on the smart glasses display.

[1327] Step 9:

[1328] Users can rate the provided suggestions by entering a positive rating for how helpful the suggestions were via the interface of smart glasses or a smartphone.

[1329] Step 10:

[1330] The server collects and analyzes user likes. It then aggregates which ads and notifications received how many likes they received. The input is the likes data from users, and the output is the aggregated results of the likes.

[1331] Step 11:

[1332] The server calculates rewards based on the tally results and distributes them to ad providers and app developers. Specifically, it calculates rewards based on evaluation data analysis and transfers them to a designated account. The input is the tally results of high ratings, and the output is the completion of reward distribution.

[1333] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1334] The present invention relates to a system that combines a system that collects information from a user's device, analyzes that information, and generates and notifies relevant suggestions to the user with an emotion engine that recognizes the user's emotions. This system recognizes the user's state in real time and makes it possible to provide appropriate suggestions and reminders according to that state.

[1335] System configuration

[1336] The system consists of the following main components:

[1337] User devices: Electronic devices used to collect information and provide notifications, such as smart glasses and smartphones.

[1338] Server: A central processing unit that stores data, analyzes it, and generates suggestions.

[1339] Emotion engine: Software for recognizing user emotions and analyzing and storing that information.

[1340] Communication network: The infrastructure that connects users' devices with the server and emotion engine.

[1341] Program processing

[1342] Information Collection and Storage

[1343] Terminal

[1344] 1. Smart glasses capture the user's visual information and collect emotional data such as voice commands and facial expressions.

[1345] 2. Your smartphone records your location, the app history you use, text messages, and voice input data.

[1346] 3. The collected information is periodically compressed and then sent to the server and emotion engine via the network.

[1347] Data storage and analysis

[1348] server

[1349] 1. The received information is stored in a database and classified into categories such as "visual information," "location information," "text information," and "emotional information."

[1350] 2. Use natural language processing (NLP) techniques to analyze text data and identify user interests and behavioral patterns.

[1351] Emotion Engine

[1352] 1. Analyze the user's tone of voice, facial expressions, and choice of words to recognize their emotional state in real time.

[1353] 2. The recognized emotion data is sent to the server for further analysis.

[1354] Proposal generation and notification

[1355] server

[1356] 1. Generate optimal suggestions and reminders based on user interest information, behavioral patterns, and emotional information.

[1357] 2. Calculate a schedule for optimal notifications, taking into account the user's location, time of day, and emotional state.

[1358] Terminal

[1359] 1. Receive notification information sent from the server and emotion engine.

[1360] 2. Notify the user of the received information, for example, via the smart glasses display or a smartphone push notification.

[1361] Likes and Feedback

[1362] User

[1363] 1. Check the ads and reminders you've been notified of and browse what interests you.

[1364] 2. If you find the ad or reminder useful, give it a thumbs up.

[1365] server

[1366] 1. Collect user rating data and analyze which suggestions and reminders received the highest ratings.

[1367] 2. Calculate rewards to providers based on their likes.

[1368] Use of emotion engine

[1369] Emotion Engine

[1370] 1. Analyze users' real-time emotional data and customize suggestions based on that information.

[1371] 2. Adjust the tone and wording of notifications depending on the user's emotional state. For example, if the user is feeling stressed, prioritize suggestions for relaxing activities.

[1372] Specific examples

[1373] For example, when a user searches for "stylish cafes" through smart glasses, the glasses capture this information and send it to the server along with related information. At the same time, the emotion engine analyzes the user's facial expressions and tone of voice to determine if the user is in a happy mood. The server analyzes this information and generates coupon advertisements for nearby cafes, suggesting cafes that best fit the user's current emotional state. While the user is walking around town, a notification appears on their smartphone saying, "There's a coupon for a nearby cafe." If the user finds this advertisement useful and rates it highly, the information is sent to the server, and rewards are distributed to the cafes and app developers that provided the advertisements based on the high ratings.

[1374] The above is a specific embodiment of the system of the present invention. The addition of an emotion engine enables more personalized suggestions tailored to the user's situation, further improving the user experience.

[1375] The processing flow will be explained below.

[1376] Step 1: Gather information

[1377] Terminal

[1378] 1. Smart glasses capture the user's visual information, specifically, images of the scenery, web pages, and search queries the user is viewing.

[1379] 2. Smart glasses collect the user's voice commands and facial expression information, which is used to infer the user's emotional state.

[1380] 3. Your smartphone records your location, app usage history, text messages, and voice data.

[1381] 4. The collected data is temporarily stored on the device, periodically compressed, and sent to the server and emotion engine via the network.

[1382] Step 2: Save your data

[1383] server

[1384] 1. Receives information sent from the device and stores it in a database. The data is classified into categories such as "visual information," "location information," "text information," and "emotional information."

[1385] 2. Preprocessing the stored data to filter out unnecessary information and noise, for example, excluding personally identifiable information to protect privacy.

[1386] Step 3: Analyze the data

[1387] server

[1388] 1. Analyze the stored text information using natural language processing (NLP) technology to identify user interests and behavioral patterns. For example, analyze the keyword "stylish cafe."

[1389] 2. Compare the data with the user's past behavioral data to extract patterns of interest and behavior. For example, find a pattern such as "going to a cafe every Wednesday."

[1390] Emotion Engine

[1391] 1. Recognize the user's emotional state by analyzing their tone of voice, facial expressions, and choice of words in real time.

[1392] 2. The recognized emotion data is sent to the server for further analysis.

[1393] Step 4: Generate proposals

[1394] server

[1395] 1. Based on the analysis results, generate relevant ads and reminders for users. For example, generate coupon ads for "trendy cafes."

[1396] 2. Calculate the optimal timing for notifications based on the user's location, time, and emotional state. For example, if the user is near a cafe, send a coupon at that time.

[1397] Step 5: Proposal Notification

[1398] Terminal

[1399] 1. Receives advertisements and reminders sent from the server, as well as emotion information from the emotion engine.

[1400] 2. Display the received information at a time that suits the user, for example, using the display on smart glasses or push notifications on a smartphone.

[1401] Step 6: Like and feedback

[1402] User

[1403] 1. Check the ad or reminder you received and view the details if you are interested.

[1404] 2. If you find the ad or reminder useful, give it a "like" through the UI, for example, through the smartglasses or smartphone interface.

[1405] Step 7: Counting likes and calculating rewards

[1406] server

[1407] 1. Collect data on user ratings. Analyze which ads and reminders received high ratings.

[1408] 2. Based on the likes, the app developer and advertisers will be rewarded. The reward amount will be determined according to the number of likes.

[1409] Step 8: Reward Distribution

[1410] server

[1411] 1. Distribute the calculated rewards to the providers. Record the reward distribution log and reflect it in the next reward calculation.

[1412] Specific examples

[1413] For example, when a user searches for "stylish cafes" through smart glasses, the glasses capture this information and send it to a server. At the same time, the emotion engine analyzes the user's happy facial expressions and tone of voice to recognize that the user is in a positive emotional state. The server uses this information to generate coupon ads for nearby cafes. Next, while the user is walking around town, a notification appears on the user's smartphone saying, "There's a coupon available for a nearby cafe." If the user finds the ad useful and rates it highly, the information is sent to the server, and rewards are distributed to the cafe or app developer that provided the ad based on the high rating.

[1414] The above is a specific embodiment of the system of the present invention. This process realizes a system that provides useful information to users at an appropriate time and distributes rewards to developers and providers based on their reactions.

[1415] Example 2

[1416] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1417] In recent years, with the increasing use of digital devices and the spread of wearable devices, there has been a growing demand for systems that can effectively collect and analyze individual behavioral and emotional data. However, existing systems do not adequately perform real-time analysis based on user emotions or generate personalized suggestions based on individual behavioral patterns. Furthermore, they lack mechanisms for efficiently reflecting user ratings and improving the quality of suggestions. As a result, the user experience does not improve and the effectiveness of the system is limited.

[1418] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1419] In this invention, the server includes: means for collecting information from the user's electronic device; means for storing the transmitted information; means for classifying the stored information into "visual information," "location information," "text information," "emotional information," etc.; means for analyzing the text information using natural language processing technology; means for recognizing the user's emotional state in real time using emotion analysis; means for scheduling notifications taking into account the user's current location, time of day, and emotional state; and means for collecting user ratings, calculating rewards based on the ratings, and distributing them to providers. This allows users to receive more personalized suggestions based on their emotions and behavior at the time, improving the user experience. Furthermore, utilizing real-time emotion analysis enables efficient generation and notification of suggestions.

[1420] "User's electronic device" refers to a portable electronic device such as a smartphone or smart glasses, which collects the user's visual information, location information, text information, and emotional information.

[1421] "Means for collecting information" refers to functions that use the user's electronic devices to collect visual information, voice commands, location information, text messages, voice input data, etc.

[1422] The "means for transmitting information" is a function that compresses the information collected from the terminal and transmits it to the server and emotion analysis system via a communication network.

[1423] "Means for storing information" refers to a function that stores the transmitted information in a database so that it can be analyzed or referenced later.

[1424] "Means for classifying information" is a function that organizes saved information into categories such as "visual information," "location information," "text information," and "emotional information."

[1425] "Means for analyzing text information using natural language processing technology" refers to technology that analyzes collected text messages and app usage history to identify users' interests and behavioral patterns.

[1426] "Means for recognizing a user's emotional state in real time using emotion analysis means" refers to a function that analyzes voice tone and facial expressions to identify a user's emotional state in real time.

[1427] "Means for scheduling notifications" is a feature that calculates the optimal time to notify based on the user's current location, time of day, and emotional state, and generates appropriate suggestions and reminders.

[1428] "Means for collecting likes" is a function that collects the interest and ratings that users show in suggestions and reminders and stores them as feedback.

[1429] The "means for calculating and distributing rewards to providers" is a function that analyzes which suggestions and reminders have been highly rated based on the high ratings, calculates rewards accordingly, and distributes them to providers.

[1430] The present invention is a system that collects and analyzes information from a user's electronic device to generate and notify relevant suggestions to the user. The system also incorporates an emotion analysis engine that recognizes the user's emotional state in real time and adaptively changes the content of the suggestions based on that information.

[1431] System configuration

[1432] The system consists of the following main components:

[1433] User electronic devices: Portable electronic devices such as smart glasses and smartphones collect users' visual information, voice commands, emotional data such as facial expressions, location information, text messages, and app usage history.

[1434] Server: A central processing unit that stores data, performs classification and analysis, and generates recommendations for users. Natural language processing technology is used here to analyze text information.

[1435] Emotion Analysis Engine: A software component that analyzes voice tone, facial expressions, and language to recognize the user's emotional state in real time.

[1436] Communication network: The infrastructure that connects users' electronic devices with the server and emotion analysis engine.

[1437] Information collection and analysis

[1438] Terminal

[1439] 1. When a user puts on the smart glasses and starts an activity, the smart glasses capture visual information and collect emotional data such as voice commands and facial expressions.

[1440] 2. Your smartphone regularly collects location information and records your app usage history, text messages, and voice input data.

[1441] 3. The data collected on the device is compressed and sent via a communication network to a server and emotion analysis engine.

[1442] server

[1443] 1. The server stores the information sent from the device in a database. The stored data is categorized into "visual information," "location information," "text information," "emotional information," etc.

[1444] 2. Using natural language processing (NLP) technology, the stored text data is analyzed to identify user interests and behavioral patterns.

[1445] 3. Real-time emotional data sent from the emotion analysis engine is also stored on the server and used for analysis.

[1446] Sentiment Analysis Engine

[1447] 1. The emotion analysis engine analyzes voice tone and facial expressions to recognize the user's emotional state in real time.

[1448] 2. The recognized emotion information is sent to the server for further analysis and suggestion generation.

[1449] Proposal generation and notification

[1450] server

[1451] 1. Generate optimal suggestions and reminders based on user interest information, behavioral patterns, and emotional information.

[1452] 2. Calculate the notification schedule and make suggestions at the right time, taking into account the user's location, time of day, and emotional state.

[1453] Terminal

[1454] 1. Receive notification information sent from the server and sentiment analysis engine.

[1455] 2. Users can check the received suggestions through the smart glasses display or push notifications on their smartphones.

[1456] Likes and Feedback

[1457] User

[1458] 1. Users can review the suggestions they receive and rate them highly if they find them useful.

[1459] 2. Likes are sent to the server via the device.

[1460] server

[1461] 1. Aggregate the received positive feedback data and analyze which proposals received positive feedback.

[1462] 2. Execute a process to calculate and distribute rewards to the providers of the suggestions based on the high ratings.

[1463] Specific examples

[1464] When a user searches for "stylish cafes" through the smart glasses, the glasses capture this information and send it along with related information to a server. At the same time, an emotion analysis engine analyzes the user's facial expressions and tone of voice to determine if the user is in a happy mood. The server analyzes this information and generates coupon advertisements for nearby cafes, and notifies the user on their smartphone while they are walking around town that "there is a coupon available for a nearby cafe."

[1465] Prompt Sentence Examples

[1466] While searching for "stylish cafes," the system analyzes the user's facial expressions and tone of voice, recognizes that the user is in a good mood, and sends recommendations and coupons for nearby cafes to the user's smartphone. Furthermore, if the user gives a high rating, the system also explains how the rewards will be distributed to the advertiser.

[1467] The above is a specific embodiment of the system of the present invention, which allows users to receive personalized suggestions based on their emotions and behavior at the time, and further improves the quality of the suggestions by efficiently incorporating high-rated feedback.

[1468] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1469] Step 1: Gather information

[1470] Terminal

[1471] 1. The user puts on the smart glasses and begins their daily activities.

[1472] Input: User visual information, voice commands, facial expression data

[1473] How it works: The smart glasses' camera captures visual information from the surroundings, the microphone collects voice commands and emotional data (such as tone and intonation), and facial recognition technology is used to collect facial expression data.

[1474] Output: Visual information, audio data, emotion data

[1475] 2. Your smartphone records your location, app usage history, text messages, and voice input data.

[1476] Input: GPS information, app usage history, text messages, voice input

[1477] How it works: It uses the smartphone's GPS to obtain location information, and records app usage history, text messages, and voice input in the background.

[1478] Output: Location information, app usage history, text messages, voice data

[1479] Step 2: Submit your information

[1480] Terminal

[1481] 1. The collected information is periodically compressed and sent to a server and sentiment analysis engine via a communication network.

[1482] Input: Collected visual information, audio data, emotional data, location information, app usage history, text messages

[1483] Specific operation: A data compression algorithm is applied to reduce the size of the transmitted data, and the data is transferred to the server and sentiment analysis engine using a secure communication protocol.

[1484] Output: Compressed data packets

[1485] Step 3: Store and classify data

[1486] server

[1487] 1. Save the submitted information in a database.

[1488] Input: Compressed data packet

[1489] Specific operation: The data packet is extracted and stored in a database. Each data item is classified into "visual information," "location information," "text information," and "emotion information."

[1490] Output: Categorized data entries

[1491] Step 4: Data analysis

[1492] server

[1493] 1. Analyze text data using natural language processing (NLP) techniques to identify user interests and behavioral patterns.

[1494] Input: Text information stored on the server

[1495] What it does: It applies NLP algorithms to analyze text data and extract user interests and behavioral patterns, thereby identifying what users are interested in.

[1496] Output: User interest information, behavioral patterns

[1497] Step 5: Analyze the sentiment data

[1498] Sentiment Analysis Engine

[1499] 1. Analyzes voice tone and facial expressions to recognize the user's emotional state in real time.

[1500] Input: Voice data and facial expression data sent from the server

[1501] How it works: It applies voice analysis and facial recognition algorithms to identify the user's emotional state in real time. For example, if the voice tone is high, it is judged to be "happy," and if it is low, it is judged to be "depressed."

[1502] Output: Real-time emotional state data

[1503] Step 6: Generate proposals

[1504] server

[1505] 1. Generate optimal suggestions and reminders based on user interest information, behavioral patterns, and emotional information.

[1506] Inputs: Interest information, behavioral patterns, real-time emotional state data

[1507] How it works: Using a generative AI model, it generates the most suitable suggestions and reminders for the user based on this data. For example, if the user is in a happy mood and interested in "cafes," it generates coupons for nearby cafes.

[1508] Output: Generated suggestions and reminders

[1509] Step 7: Scheduling and sending notifications

[1510] server

[1511] 1. Schedule notifications at optimal times, taking into account the user's location, time of day, and emotional state.

[1512] Input: current location, time, emotion

[1513] Specific behavior: Apply a scheduling algorithm to calculate the optimal timing for notifications. For example, if the user is walking to a nearby cafe, a coupon will be instantly notified.

[1514] Output: Scheduled notification information

[1515] Step 8: View notifications

[1516] Terminal

[1517] 1. Present notification information sent from the server and sentiment analysis engine to the user.

[1518] Input: Scheduled notification information

[1519] Specific operation: A notification is displayed to the user via the smart glasses display or a push notification on the smartphone.

[1520] Output: Suggestions and reminders presented to the user

[1521] Step 9: Collect likes

[1522] User

[1523] 1. Users review and like suggestions and reminders.

[1524] Input: Notified suggestions and reminders

[1525] Specific operation: Operate the "Like" button using smart glasses or a smartphone.

[1526] Output: Liked feedback

[1527] Step 10: Analyze Likes and Calculate Rewards

[1528] server

[1529] 1. Aggregate the received positive feedback data and analyze which proposals received positive feedback.

[1530] Input: Liked feedback

[1531] Specific operation: Stores the high-rating data in a database and uses statistical analysis to identify the high-rating suggestions.

[1532] Output: Highly rated proposal data

[1533] 2. Calculate rewards based on the highly rated data and distribute them to providers.

[1534] Input: Highly rated proposal data

[1535] Specific operation: Apply the reward calculation algorithm to calculate the reward for the proposal provider and perform distribution processing.

[1536] Output: Distributed reward data

[1537] (Application example 2)

[1538] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1539] Conventional food delivery services do not take into account the user's current emotional state when making suggestions, resulting in an insufficient user experience and the inability to recommend the most suitable dishes or restaurants for the user. Furthermore, the suggestions given to users are uniform and not personalized, resulting in low user satisfaction.

[1540] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1541] In this invention, the server includes means for collecting information from the user's device, means for transmitting the collected information, means for storing the transmitted information, means for analyzing the stored information, means for analyzing text data using natural language processing technology, an emotion engine means for analyzing voice tone, facial expressions, and wording to recognize an emotional state, means for generating suggestions relevant to the user based on the analysis results, means for notifying the user of the generated suggestions, means for collecting user ratings, means for calculating rewards based on the user ratings, and means for distributing rewards to providers. This enables optimal food delivery suggestions based on the user's emotional state in the real world, improving the user experience.

[1542] "User device" refers to an electronic device used by a user to collect and transmit information, such as smart glasses or a smartphone.

[1543] A "means for transmitting collected information" is a process or device for transmitting information from a user's terminal to a server.

[1544] The "means for storing transmitted information" refers to a system or process within the server for storing the received information in a storage device such as a database.

[1545] "Means for analyzing stored information" means the process of analyzing collected information to identify user behavior patterns and interests.

[1546] "Natural language processing technology" is a technology for analyzing text data and understanding and generating human language.

[1547] The "emotion engine that recognizes emotional states by analyzing voice tone, facial expressions, and language" is a software component that identifies emotions from the user's voice and facial expressions and evaluates their state in real time.

[1548] The "means for generating relevant suggestions for users based on analysis results" refers to a system for automatically creating optimal suggestions based on users' emotional and behavioral data.

[1549] "Means for notifying users of generated suggestions" refers to the process or device for delivering generated suggestions to users, such as a smartglasses display or a smartphone push notification.

[1550] The "means for collecting high ratings from users" is a system that allows users to rate proposals and collect the rating data.

[1551] The "means for calculating remuneration based on user's high ratings" is a process for calculating appropriate remuneration for the provider based on the collected high rating data.

[1552] "Means for distributing rewards to providers" refers to a system or process for distributing calculated rewards to advertisers and service providers.

[1553] "A means for suggesting optimal food delivery options based on real-world emotional states" is a system that automatically suggests the most suitable dishes and restaurants to users based on their real-time emotional data.

[1554] The present invention relates to a system for providing optimal food delivery suggestions based on a user's emotional state. The system includes a user terminal, a server, and an emotion engine.

[1555] System configuration

[1556] The system consists of the following main components:

[1557] User devices: Electronic devices such as smart glasses and smartphones that collect information and send notifications.

[1558] Server: A central processing unit that stores data, analyzes it, and generates suggestions.

[1559] Emotion engine: Software that analyzes voice tone, facial expressions, and language to recognize emotional states.

[1560] Hardware and software used

[1561] Smart glasses: capturing visual information and voice commands.

[1562] Smartphones: Collecting location, text messages, and voice input data.

[1563] Server: Python, Django, MySQL for data storage and analysis.

[1564] Emotion engine: TensorFlow and PyTorch for voice tone and facial expression analysis.

[1565] Communication network: Wi-Fi, 4G / 5G communication infrastructure.

[1566] What the program does

[1567] Information gathering

[1568] The user's devices collect information: smart glasses capture the user's visual information and voice commands, and smartphones record location information, text messages, and voice input data. This information is periodically compressed and sent over the network to a server.

[1569] Data storage and analysis

[1570] The server stores the received information in a database and categorizes it into categories such as "visual information," "location information," "text information," and "emotion information." Natural language processing (NLP) is used to analyze the text data and identify the user's interests and behavioral patterns. Meanwhile, the emotion engine analyzes voice tone, facial expressions, and vocabulary to recognize the user's emotional state in real time. The recognized emotion data is also sent to the server for further analysis.

[1571] Proposal generation and notification

[1572] The server generates optimal food delivery suggestions based on the user's interest information, behavioral patterns, and emotional information. The suggestions provide restaurants and dishes that best suit the user's current emotional state. The suggestions are delivered to the user via the smartglasses display or push notifications on their smartphone.

[1573] For example, if a user is feeling stressed, the system will send a notification to their smartphone suggesting a relaxing dish or restaurant. If the user rates the suggestion favorably, the data will be sent to the server, and appropriate rewards will be distributed to advertisers and food delivery service providers.

[1574] Example prompt sentence:

[1575] Capture visual information and voice commands from the user's smart glasses, and collect location information and text messages from their smartphone, then send them to a server. Design a system that uses this data to analyze emotions in real time with an emotion engine and suggest food delivery options that fit the user's current emotional state. This system should also display personalized restaurant and food recommendation notifications on the smartphone, and if the user gives a high rating, send that data to the provider to calculate a reward.

[1576] The present invention enables optimal food delivery suggestions tailored to the user's real-world emotional state, improving the user experience. This specific example details how a system can be implemented to provide personalized suggestions that take the user's emotional state into account.

[1577] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1578] Step 1:

[1579] Information gathering

[1580] The devices collect information. Smart glasses capture the user's visual information and voice commands, while the smartphone records location information, text messages, and voice input data. This input data (visual information, voice commands, location information, text messages, and voice input) is obtained from the user's device. This data includes facial recognition, voice analysis, and location tracking. The obtained data is temporarily stored on the device.

[1581] Step 2:

[1582] Data transmission

[1583] The device sends the collected information to the server. The collected data is pre-compressed and sent to the server over the network. The compressed information packets are input and transferred to the server via the network infrastructure. The compressed data is sent as output.

[1584] Step 3:

[1585] Data storage

[1586] The server stores the received information in a database. The database uses MySQL and stores the information by categorizing it into categories such as "visual information," "location information," "text information," and "emotional information." This data includes video frames, GPS data, text messages, and audio files. The input is compressed data packets, and the output is classified database entries.

[1587] Step 4:

[1588] Data analysis

[1589] The server analyzes the stored data. In particular, it uses natural language processing (NLP) to analyze text data and identify user interests and behavioral patterns. It uses TensorFlow and PyTorch to analyze voice tone and facial expressions to recognize emotional states in real time. The input is visual information, audio data, and text data read from the database, and the output is the analyzed user's behavioral patterns and emotional state.

[1590] Step 5:

[1591] Proposal generation

[1592] The server generates relevant suggestions for the user based on the analysis results. It combines the user's interest information, behavioral patterns, and emotional information to create optimal food delivery suggestions. The input is the analyzed behavioral patterns and emotional state data, and the output is specific restaurant and food suggestions.

[1593] Step 6:

[1594] notification

[1595] The device notifies the user of the generated suggestions. The suggestions are displayed to the user on the smartglasses display or via a smartphone push notification. The input is the generated suggestion data, and the output is a notification displayed on the user's device.

[1596] Step 7:

[1597] Evaluation collection

[1598] The terminal collects likes from users. The user rates the notified proposal and sends the rating data to the server. The input is the user's rating data, and the output is the rating information sent to the server.

[1599] Step 8:

[1600] Reward Calculation and Distribution

[1601] The server calculates rewards based on users' likes and distributes them to providers. Based on the collected likes data, appropriate rewards are calculated for advertisers and food delivery service providers. The input is the users' likes data, and the output is the calculated reward amount and its distribution information.

[1602] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1603] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1604] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1605] [Fourth embodiment]

[1606] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1607] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1608] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1609] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1610] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1611] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1612] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1613] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1614] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1615] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1616] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1617] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1618] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1619] The present invention relates to a system that collects information from a user's device, analyzes that information, and generates and notifies relevant suggestions to the user, with the aim of providing useful information to the user at the optimal time, making daily life more efficient and comfortable.

[1620] System configuration

[1621] First, the overall configuration of the system will be described. The system of the present invention is composed of the following main components.

[1622] User's devices: Electronic devices for collecting information and sending notifications, such as smart glasses and smartphones.

[1623] Server: A central processing unit that stores data, analyzes it, and generates suggestions.

[1624] Communications network: The infrastructure that connects user devices and servers to exchange information.

[1625] Program processing

[1626] Information Collection and Storage

[1627] Terminal

[1628] 1. Smart glasses capture visual information from the user's everyday life, such as the scenery they see, their search queries, and their voice commands.

[1629] 2. Your smartphone records your location, the apps you use, and your text messages.

[1630] 3. Once this information is collected, it is periodically sent to a server.

[1631] Data storage and analysis

[1632] server

[1633] 1. The received information is stored in a database. The data is organized by category, such as "visual information," "location information," and "text information."

[1634] 2. Preprocess the stored data and filter out unnecessary information.

[1635] 3. Analyze text data using natural language processing (NLP) techniques to identify user interests and behavioral patterns.

[1636] Proposal generation and notification

[1637] server

[1638] 1. Generate highly relevant ads and reminders based on user interests and behavioral patterns. For example, if a user searches for "stylish cafes," coupon information for popular nearby cafes will be generated.

[1639] 2. Analyze the user's location and time information and calculate a schedule for notifying them at the optimal time.

[1640] Terminal

[1641] 1. Receive notification information sent from the server.

[1642] 2. The notification is displayed at a time that is appropriate for the user, for example, via a smartglasses display or a smartphone push notification.

[1643] Likes and reward distribution

[1644] User

[1645] 1. Check the ads and reminders you receive and select the ones that interest you.

[1646] 2. If you find these useful, give them a "like" in the system interface.

[1647] server

[1648] 1. Aggregate user likes. Track which ads and reminders are liked and how many.

[1649] 2. Calculate and appropriately distribute rewards to the app developers.

[1650] Specific examples

[1651] For example, if a user searches for "stylish cafes" through smart glasses, the glasses capture this information and send it to a server. The server analyzes this information and generates advertisements for nearby cafes. These advertisements are then pushed to the user's smartphone at the optimal time while they are walking around town. If the user finds the advertisement useful, they can rate it highly. Based on this high rating, rewards are distributed to the cafes and app developers that provided the advertisements.

[1652] The above is a specific embodiment of the system of the present invention.

[1653] The processing flow will be explained below.

[1654] Step 1: Gather information

[1655] Terminal

[1656] 1. Smart glasses capture the user's visual information, specifically images of the scenery and objects the user is looking at, as well as search queries and voice commands.

[1657] 2. Your smartphone records your location in real time, and also collects your smartphone usage history and text messages.

[1658] 3. The collected data is periodically compressed and sent over the network to a server.

[1659] Step 2: Save your data

[1660] server

[1661] 1. The received information is stored in a database and classified into categories such as "location information," "visual information," and "text information."

[1662] 2. Preprocessing the information stored in the database to filter out noise and unnecessary information. To protect privacy, personally identifiable information is excluded.

[1663] Step 3: Analyze the data

[1664] server

[1665] 1. Analyze text information using natural language processing (NLP) technology. For example, if a user searches for "stylish cafe," analyze the meaning.

[1666] 2. Identify user interests and patterns of behavior by comparing them with past user behavior data. For example, find a pattern such as "I go to a cafe every Wednesday."

[1667] Step 4: Generate proposals

[1668] server

[1669] 1. Based on the analysis results, generate relevant ads and reminders for users. For example, generate coupon ads for "trendy cafes."

[1670] 2. Calculate the optimal timing for notifications based on the user's location and time of day. For example, if the user is near a cafe, send a coupon at that time.

[1671] Step 5: Proposal Notification

[1672] Terminal

[1673] 1. Receive advertisements and reminders sent from the server.

[1674] 2. The received information is notified to the user. This can be done using the smartglasses display or a push notification on a smartphone. For example, a notification could say, "There's a coupon you can use at a nearby cafe."

[1675] Step 6: Like and feedback

[1676] User

[1677] 1. Check the ad or reminder you received and view the details if you are interested.

[1678] 2. If you find an ad or reminder useful, you can give it a "like." You can do this through the smartglasses or smartphone interface.

[1679] Step 7: Counting likes and calculating rewards

[1680] server

[1681] 1. Collect data on user ratings. Analyze which ads and reminders received high ratings.

[1682] 2. Calculate rewards for advertising and reminder providers based on the likes. Determine the reward amount according to the number of likes.

[1683] Step 8: Reward Distribution

[1684] server

[1685] 1. Distribute the calculated rewards to the app developers and advertising providers.

[1686] 2. Record the distribution results and use them for the next reward calculation.

[1687] The above are the specific processing steps of the program of the present invention. This flow makes it possible to provide useful information to users at the optimal timing and to give incentives to developers based on high ratings.

[1688] Example 1

[1689] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1690] Conventional information provision systems face challenges in efficiently and effectively analyzing information collected from users' devices and generating personalized suggestions for users. They also lack a mechanism for properly reflecting users' high ratings and distributing rewards based on those ratings to providers. Furthermore, they do not provide notifications at optimal times that take into account the user's location and time information. This results in a poor user experience and reduces the effectiveness of information provision.

[1691] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1692] In this invention, the server includes means for collecting data from the user's electronic device, means for transmitting the collected data, means for storing the transmitted data, means for preprocessing the stored data, means for analyzing the preprocessed data using natural language processing technology, means for generating suggestions relevant to the user based on the analyzed data, means for notifying the user's electronic device of the generated suggestions, means for collecting user ratings, means for calculating rewards based on the user ratings, and means for distributing rewards to information providers. This makes it possible to generate personalized suggestions based on the user's interests and behavioral patterns and notify them at an appropriate time. Furthermore, by reflecting user rating data and distributing rewards fairly and effectively, it is possible to increase the motivation of information providers.

[1693] "User's electronic devices" refers to various types of electronic devices that users use on a daily basis, including smartphones, smart glasses, and tablet devices.

[1694] "Means of collecting data" refers to a combination of hardware and software that obtains information about a user's daily life and behavior through electronic devices.

[1695] "Means for transmitting data" refers to the protocols and functionality for transmitting collected data from the user's electronic device to the server.

[1696] "Means for storing data" refers to a system for temporarily or permanently storing transmitted data in a database on a server or on a storage medium.

[1697] "Preprocessing means" refers to processes and techniques that perform filtering and noise reduction to convert the stored raw data into a format that is easier to analyze.

[1698] "Means of analysis using natural language processing techniques" refers to algorithms and models (e.g., BERT and GPT) that use collected and pre-processed text data to identify user interests and behavioral patterns.

[1699] "Means for generating suggestions" refers to algorithms and programs for automatically generating relevant suggestions for goods and services to the User based on the analyzed data.

[1700] "Means for notifying" refers to mechanisms and protocols for displaying generated suggestions on a user's electronic device in a timely manner.

[1701] "Means for collecting ratings" refers to an interface for users to input feedback and ratings on received suggestions, and a means for transmitting such data to the server.

[1702] "Means for calculating rewards" refers to the algorithms and systems for calculating rewards to advertisers and content creators based on collected user rating data.

[1703] "Means for distribution to providers" refers to the economic and technical mechanisms for appropriately distributing the calculated rewards to advertising providers and content creators.

[1704] The present invention relates to a system that collects data from a user's electronic devices, analyzes the data, and generates and notifies the user of relevant suggestions, thereby providing useful information to the user at the optimal time, making daily life more efficient and comfortable.

[1705] System configuration

[1706] The system of the present invention is comprised of the following major components:

[1707] User electronic devices: smart glasses, smartphones, etc.

[1708] Server: Central processing unit for data storage, preprocessing, analysis, and proposal generation

[1709] Communications network: the infrastructure that connects users' electronic devices to servers and exchanges information

[1710] Explanation of program processing

[1711] Information gathering

[1712] Terminal

[1713] When a user wears smart glasses, they are equipped with a built-in camera that captures visual information from the user's daily life. Specifically, it collects the user's surroundings, search queries, voice commands, etc. The smartphone also uses its GPS function to record the user's location in real time. In addition, the smartphone also collects information on the usage of installed apps and the sending and receiving of text messages.

[1714] Sending data

[1715] Collected visual information, location information, voice commands and app usage data is periodically transmitted to a server using encrypted protocols such as HTTPS.

[1716] Data storage and preprocessing

[1717] server

[1718] The server stores the received information in a database. The data is classified into categories such as "visual information," "location information," and "text information." The stored raw data is then preprocessed to convert it into a format that is easier to analyze. Specifically, unnecessary image frames are removed from the visual information, and noisy audio data is filtered out.

[1719] Data analysis

[1720] The stored and pre-processed data is then analyzed using natural language processing (NLP) techniques, such as generative AI models like BERT and GPT, to identify user interests and behavioral patterns from the text data.

[1721] Proposal generation and notification

[1722] server

[1723] The server automatically generates relevant suggestions for the user based on the analysis results. For example, if a user searches for "stylish cafes," coupon information for nearby cafes will be generated. In addition, the server analyzes the user's location and time information to calculate a schedule for sending notifications at the optimal time.

[1724] Terminal

[1725] Smartphones or smart glasses receive notification information sent from the server and display it at a time that suits the user. For example, a user walking down the street might receive a push notification on their smartphone screen saying, "There's a popular cafe nearby."

[1726] Likes and reward distribution

[1727] User

[1728] Users can check the suggestions and give them a "like" if they find them useful. This rating is done via a button on the interface of their smartphone or smart glasses.

[1729] server

[1730] The server tracks the ratings of each ad and reminder based on the aggregated rating data, and calculates and fairly distributes rewards to ad providers and content creators based on the rating data.

[1731] Specific examples

[1732] For example, if a user searches for "stylish cafes" through smart glasses, the glasses capture visual information along with the search query and send it to a server. The server analyzes this information and generates advertisements for nearby cafes. It then sends a push notification to the user's smartphone at the optimal time while the user is walking in the area. If the user checks the notification and finds it useful, they can rate it highly, and rewards are distributed to the cafes and app developers that provided the advertisements based on the results.

[1733] Prompt Sentence Examples

[1734] If a user searches for "stylish cafes" using smart glasses, what process will be performed and what notification will be sent to the user as a result?

[1735] Please explain in detail how you will collect user rating data and distribute rewards.

[1736] The above is a specific embodiment of the system of the present invention.

[1737] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1738] Step 1:

[1739] Information gathering

[1740] The devices collect visual information, location information, voice commands, and app usage data from users' daily lives. Specifically, smart glasses use a built-in camera to capture visual information and a microphone to collect voice commands from users. Smartphones use GPS to record location information in real time, and also collect information on app usage and text message sending and receiving.

[1741] Input: Visual information from the user's daily life, location, voice commands, and app usage

[1742] Output: Collected information data

[1743] Step 2:

[1744] Sending data

[1745] The device sends the collected information data to the server. At this time, the data is sent securely using an encryption protocol such as HTTPS. For example, smart glasses and smartphones use batch processing to send collected data to the server at regular intervals.

[1746] Input: Collected information data

[1747] Output: Information data sent to the server

[1748] Step 3:

[1749] Data storage

[1750] The server receives the transmitted information data and stores it in a database. The data is stored in categories such as "visual information," "location information," and "text information."

[1751] Input: Transmitted information data

[1752] Output: Information stored in the database

[1753] Step 4:

[1754] Data Preprocessing

[1755] The server preprocesses the stored information data, specifically removing unnecessary image frames from visual information and noise from audio data, as well as filtering text data to standardize its format.

[1756] Input: Information stored in a database

[1757] Output: Preprocessed data

[1758] Step 5:

[1759] Natural Language Processing (NLP) Analysis

[1760] The server analyzes the preprocessed text data using natural language processing (NLP) technology, using generative AI models such as BERT and GPT, to identify interests and behavioral patterns from users' search queries and text messages.

[1761] Input: Preprocessed text data

[1762] Output: Analyzed user interests and behavior patterns

[1763] Step 6:

[1764] Proposal Generation

[1765] The server generates relevant suggestions for the user based on the analysis results. For example, if a user searches for "stylish cafes," it will generate coupon information for nearby cafes.

[1766] Input: Analyzed user interests and behavioral patterns

[1767] Output: Generated proposal information

[1768] Step 7:

[1769] Scheduling Notifications

[1770] The server calculates the optimal timing to notify the user of the suggested information based on the user's location and time information.

[1771] Input: Generated suggestion information, user location information, time information

[1772] Output: Notification Schedule

[1773] Step 8:

[1774] notification

[1775] The device receives the suggested information sent from the server and notifies the user, who is then provided with the information in the form of a push notification or display on their smartphone or smart glasses.

[1776] Input: Notification schedule, generated proposal information

[1777] Output: Notification displayed to the user

[1778] Step 9:

[1779] Collecting likes

[1780] The user checks the proposed information that has been notified to them, and if they find it useful, they give it a "like" through the system interface.

[1781] Input: Notified proposal information

[1782] Output: User likes data

[1783] Step 10:

[1784] Aggregation of evaluation data and reward calculation

[1785] The server collects user rating data, aggregates which suggestions and ads received high ratings, and then calculates rewards for advertising providers and content creators based on this data.

[1786] Input: User's high rating data

[1787] Output: Calculated reward amount

[1788] Step 11:

[1789] Reward distribution

[1790] The server distributes the calculated rewards to providers, providing them with economic incentives based on high ratings from users.

[1791] Input: Calculated reward amount

[1792] Output: Reward distribution to providers

[1793] In this way, each step works together to create a system that provides users with optimal information and distributes rewards based on their evaluation.

[1794] (Application example 1)

[1795] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1796] Conventional information provision systems have difficulty providing timely suggestions based on the user's interests and concerns, and these suggestions are not reflected in the user's visual perception in real time. As a result, the efficiency of providing optimal information to users is reduced, and user satisfaction cannot be increased.

[1797] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1798] In this invention, the server includes means for collecting information from the user's device, means for transmitting the collected information, means for storing the transmitted information, means for analyzing the stored information, means for generating suggestions related to the user, means for notifying the user of the generated suggestions, means for capturing the user's visual information and location information, means for analyzing the captured visual information, means for analyzing the user's location information, means for displaying the suggestions in the user's field of vision in real time, means for collecting user likes, means for calculating rewards based on the user likes, and means for distributing the rewards to providers. This makes it possible to utilize the user's visual information and location information in real time to provide optimal information based on the user's interests and concerns.

[1799] "User device" means an electronic device that collects and receives information, such as smart glasses or a smartphone.

[1800] "Information collection means" refers to devices or software that have the functionality to capture a user's visual information, location information, apps used, and other related data.

[1801] "Means for transmitting information" refers to a communication device or protocol for transmitting the collected information to a server over a network.

[1802] "Means for storing information" refers to a database or storage device for securely storing collected and transmitted data.

[1803] "Means for analyzing information" refers to algorithms and programs that use statistical analysis and natural language processing of stored data to identify user behavior patterns and interests.

[1804] The "means for generating suggestions" is a system that automatically creates advertisements and notifications relevant to users based on the analysis results.

[1805] "Means for notifying the user of the proposal" refers to a display or push notification function for displaying the generated proposal on the user's device.

[1806] A "means for capturing visual information" is an electronic device equipped with a camera or sensor to collect visual data about what the user sees.

[1807] A "means for capturing location information" is an electronic device equipped with a GPS function for determining the user's current location.

[1808] "Means for analyzing visual information" refers to image analysis technology for analyzing captured visual information and identifying user interests and behavior.

[1809] The "means for analyzing location information" is a geographic information system for analyzing the captured location information and identifying user behavior patterns.

[1810] "Means for displaying in real time within the visual field" refers to a function that displays suggestions in real time on smart glasses or a head-mounted display worn by the user.

[1811] The "means for collecting user likes" is an interface for users to record their likes on the suggestions provided and transmit them to the server.

[1812] The "means for calculating rewards based on high ratings" is a program that analyzes the collected rating data and calculates rewards to advertising providers and app developers based on the results.

[1813] The "means for distributing rewards to providers" is a system for transferring the calculated rewards to the appropriate providers.

[1814] The present invention relates to a system that utilizes a user's visual and location information to provide relevant suggestions in real time. The system collects information from the user's device (mainly smart glasses or smartphone), analyzes the information, and notifies the user at an appropriate time. The following describes an embodiment of the present invention.

[1815] System configuration

[1816] The system of the present invention comprises the following main components:

[1817] 1. User devices: Electronic devices such as smart glasses and smartphones that collect and notify information.

[1818] 2. Server: A central processing unit that stores, analyzes, and generates recommendations based on received information.

[1819] 3. Communication network: The infrastructure that connects user devices and servers and allows information to be exchanged.

[1820] Program Overview

[1821] Your device captures visual and location information, allowing it to understand your interests and behavioral patterns in real time and generate relevant suggestions.

[1822] Information collection and transmission

[1823] 1. Smart glasses capture the user's visual information through a camera and collect location information using GPS.

[1824] 2. Your smartphone records additional location information and information about the apps you use, which are periodically sent to a server.

[1825] Data storage and analysis

[1826] 1. The server stores the received information in a database, organizing the data into categories such as visual information, location information, and app usage information.

[1827] 2. Preprocessing is performed to filter out unnecessary data. Next, natural language processing (NLP) techniques are used to analyze the text data and identify user interests and behavioral patterns. Specifically, Google Cloud Natural Language API and Amazon Comprehend are used.

[1828] Proposal generation and notification

[1829] 1. The server generates advertisements and notifications relevant to the user based on the analysis results. For example, if the user shows interest in cafes, it generates information about nearby cafes.

[1830] 2. The system analyzes the user's location and time information and calculates the optimal timing for notifications. The generated suggestions are sent in real time to the smartglasses display or smartphone.

[1831] Specific examples

[1832] For example, while a user is walking around Shibuya, the smart glasses capture visual information such as "cafe" and send it along with the user's location information to a server. The server analyzes this information and generates advertisements for nearby cafes, which are then displayed on the smart glasses in real time.

[1833] Prompt Sentence Examples

[1834] An example of a prompt sentence to input to the generative AI model is as follows:

[1835] "A user is visiting the tourist destination of Shibuya. Their interest is in cafes. Please suggest popular cafes to the user in the area."

[1836] The above is an embodiment of the present invention. The present invention is capable of providing optimal information to users in real time through a series of processes including information collection, analysis, proposal generation, and notification.

[1837] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1838] Step 1:

[1839] The device captures visual and location information. Specifically, it uses the smart glasses' camera to collect the user's visual data and the GPS module to collect location data. These data are input.

[1840] Step 2:

[1841] The device transmits the captured data to a server. The collected visual and location information is sent to the server via the communication network (e.g., Wi-Fi or 4G / 5G) used by the smart glasses or smartphone.

[1842] Step 3:

[1843] The server stores the received information in a database, which includes categorizing and tagging it and organizing it into categories such as visual information, location information, app usage information, etc. The input is the captured data, and the output is the organized database.

[1844] Step 4:

[1845] The server preprocesses the stored data, filtering out unnecessary data and noise and cleansing the data. The input is the stored raw data and the output is the cleansed data.

[1846] Step 5:

[1847] The server analyzes the preprocessed data and uses natural language processing techniques (e.g., Google Cloud Natural Language API) to identify user interests and behavioral patterns. The input is the preprocessed data, and the output is the analysis results about user interests and patterns.

[1848] Step 6:

[1849] The server generates recommendations based on the analysis results. It uses a generative AI model (e.g., OpenAI GPT-3) to create relevant ads and notifications for the user. The input is the analysis results, and the output is the recommendations provided to the user.

[1850] Step 7:

[1851] The server analyzes the user's location and time information and calculates a schedule for sending suggestions at the optimal timing. The input is location and time information, and the output is a notification schedule.

[1852] Step 8:

[1853] The device receives the suggestion notification from the server and displays it in the user's field of vision in real time. The input is the suggestion notification sent from the server, and the output is the suggestion displayed on the smart glasses display.

[1854] Step 9:

[1855] Users can rate the provided suggestions by entering a positive rating for how helpful the suggestions were via the interface of smart glasses or a smartphone.

[1856] Step 10:

[1857] The server collects and analyzes user likes. It then aggregates which ads and notifications received how many likes they received. The input is the likes data from users, and the output is the aggregated results of the likes.

[1858] Step 11:

[1859] The server calculates rewards based on the tally results and distributes them to ad providers and app developers. Specifically, it calculates rewards based on evaluation data analysis and transfers them to a designated account. The input is the tally results of high ratings, and the output is the completion of reward distribution.

[1860] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1861] The present invention relates to a system that combines a system that collects information from a user's device, analyzes that information, and generates and notifies relevant suggestions to the user with an emotion engine that recognizes the user's emotions. This system recognizes the user's state in real time and makes it possible to provide appropriate suggestions and reminders according to that state.

[1862] System configuration

[1863] The system consists of the following main components:

[1864] User devices: Electronic devices used to collect information and provide notifications, such as smart glasses and smartphones.

[1865] Server: A central processing unit that stores data, analyzes it, and generates suggestions.

[1866] Emotion engine: Software for recognizing user emotions and analyzing and storing that information.

[1867] Communication network: The infrastructure that connects users' devices with the server and emotion engine.

[1868] Program processing

[1869] Information Collection and Storage

[1870] Terminal

[1871] 1. Smart glasses capture the user's visual information and collect emotional data such as voice commands and facial expressions.

[1872] 2. Your smartphone records your location, the app history you use, text messages, and voice input data.

[1873] 3. The collected information is periodically compressed and then sent to the server and emotion engine via the network.

[1874] Data storage and analysis

[1875] server

[1876] 1. The received information is stored in a database and classified into categories such as "visual information," "location information," "text information," and "emotional information."

[1877] 2. Use natural language processing (NLP) techniques to analyze text data and identify user interests and behavioral patterns.

[1878] Emotion Engine

[1879] 1. Analyze the user's tone of voice, facial expressions, and choice of words to recognize their emotional state in real time.

[1880] 2. The recognized emotion data is sent to the server for further analysis.

[1881] Proposal generation and notification

[1882] server

[1883] 1. Generate optimal suggestions and reminders based on user interest information, behavioral patterns, and emotional information.

[1884] 2. Calculate a schedule for optimal notifications, taking into account the user's location, time of day, and emotional state.

[1885] Terminal

[1886] 1. Receive notification information sent from the server and emotion engine.

[1887] 2. Notify the user of the received information, for example, via the smart glasses display or a smartphone push notification.

[1888] Likes and Feedback

[1889] User

[1890] 1. Check the ads and reminders you've been notified of and browse what interests you.

[1891] 2. If you find the ad or reminder useful, give it a thumbs up.

[1892] server

[1893] 1. Collect user rating data and analyze which suggestions and reminders received the highest ratings.

[1894] 2. Calculate rewards to providers based on their likes.

[1895] Use of emotion engine

[1896] Emotion Engine

[1897] 1. Analyze users' real-time emotional data and customize suggestions based on that information.

[1898] 2. Adjust the tone and wording of notifications depending on the user's emotional state. For example, if the user is feeling stressed, prioritize suggestions for relaxing activities.

[1899] Specific examples

[1900] For example, when a user searches for "stylish cafes" through smart glasses, the glasses capture this information and send it to the server along with related information. At the same time, the emotion engine analyzes the user's facial expressions and tone of voice to determine if the user is in a happy mood. The server analyzes this information and generates coupon advertisements for nearby cafes, suggesting cafes that best fit the user's current emotional state. While the user is walking around town, a notification appears on their smartphone saying, "There's a coupon for a nearby cafe." If the user finds this advertisement useful and rates it highly, the information is sent to the server, and rewards are distributed to the cafes and app developers that provided the advertisements based on the high ratings.

[1901] The above is a specific embodiment of the system of the present invention. The addition of an emotion engine enables more personalized suggestions tailored to the user's situation, further improving the user experience.

[1902] The processing flow will be explained below.

[1903] Step 1: Gather information

[1904] Terminal

[1905] 1. Smart glasses capture the user's visual information, specifically, images of the scenery, web pages, and search queries the user is viewing.

[1906] 2. Smart glasses collect the user's voice commands and facial expression information, which is used to infer the user's emotional state.

[1907] 3. Your smartphone records your location, app usage history, text messages, and voice data.

[1908] 4. The collected data is temporarily stored on the device, periodically compressed, and sent to the server and emotion engine via the network.

[1909] Step 2: Save your data

[1910] server

[1911] 1. Receives information sent from the device and stores it in a database. The data is classified into categories such as "visual information," "location information," "text information," and "emotional information."

[1912] 2. Preprocessing the stored data to filter out unnecessary information and noise, for example, excluding personally identifiable information to protect privacy.

[1913] Step 3: Analyze the data

[1914] server

[1915] 1. Analyze the stored text information using natural language processing (NLP) technology to identify user interests and behavioral patterns. For example, analyze the keyword "stylish cafe."

[1916] 2. Compare the data with the user's past behavioral data to extract patterns of interest and behavior. For example, find a pattern such as "going to a cafe every Wednesday."

[1917] Emotion Engine

[1918] 1. Recognize the user's emotional state by analyzing their tone of voice, facial expressions, and choice of words in real time.

[1919] 2. The recognized emotion data is sent to the server for further analysis.

[1920] Step 4: Generate proposals

[1921] server

[1922] 1. Based on the analysis results, generate relevant ads and reminders for users. For example, generate coupon ads for "trendy cafes."

[1923] 2. Calculate the optimal timing for notifications based on the user's location, time, and emotional state. For example, if the user is near a cafe, send a coupon at that time.

[1924] Step 5: Proposal Notification

[1925] Terminal

[1926] 1. Receives advertisements and reminders sent from the server, as well as emotion information from the emotion engine.

[1927] 2. Display the received information at a time that suits the user, for example, using the display on smart glasses or push notifications on a smartphone.

[1928] Step 6: Like and feedback

[1929] User

[1930] 1. Check the ad or reminder you received and view the details if you are interested.

[1931] 2. If you find the ad or reminder useful, give it a "like" through the UI, for example, through the smartglasses or smartphone interface.

[1932] Step 7: Counting likes and calculating rewards

[1933] server

[1934] 1. Collect data on user ratings. Analyze which ads and reminders received high ratings.

[1935] 2. Based on the likes, the app developer and advertisers will be rewarded. The reward amount will be determined according to the number of likes.

[1936] Step 8: Reward Distribution

[1937] server

[1938] 1. Distribute the calculated rewards to the providers. Record the reward distribution log and reflect it in the next reward calculation.

[1939] Specific examples

[1940] For example, when a user searches for "stylish cafes" through smart glasses, the glasses capture this information and send it to a server. At the same time, the emotion engine analyzes the user's happy facial expressions and tone of voice to recognize that the user is in a positive emotional state. The server uses this information to generate coupon ads for nearby cafes. Next, while the user is walking around town, a notification appears on the user's smartphone saying, "There's a coupon available for a nearby cafe." If the user finds the ad useful and rates it highly, the information is sent to the server, and rewards are distributed to the cafe or app developer that provided the ad based on the high rating.

[1941] The above is a specific embodiment of the system of the present invention. This process realizes a system that provides useful information to users at an appropriate time and distributes rewards to developers and providers based on their reactions.

[1942] Example 2

[1943] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1944] In recent years, with the increasing use of digital devices and the spread of wearable devices, there has been a growing demand for systems that can effectively collect and analyze individual behavioral and emotional data. However, existing systems do not adequately perform real-time analysis based on user emotions or generate personalized suggestions based on individual behavioral patterns. Furthermore, they lack mechanisms for efficiently reflecting user ratings and improving the quality of suggestions. As a result, the user experience does not improve and the effectiveness of the system is limited.

[1945] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1946] In this invention, the server includes: means for collecting information from the user's electronic device; means for storing the transmitted information; means for classifying the stored information into "visual information," "location information," "text information," "emotional information," etc.; means for analyzing the text information using natural language processing technology; means for recognizing the user's emotional state in real time using emotion analysis; means for scheduling notifications taking into account the user's current location, time of day, and emotional state; and means for collecting user ratings, calculating rewards based on the ratings, and distributing them to providers. This allows users to receive more personalized suggestions based on their emotions and behavior at the time, improving the user experience. Furthermore, utilizing real-time emotion analysis enables efficient generation and notification of suggestions.

[1947] "User's electronic device" refers to a portable electronic device such as a smartphone or smart glasses, which collects the user's visual information, location information, text information, and emotional information.

[1948] "Means for collecting information" refers to functions that use the user's electronic devices to collect visual information, voice commands, location information, text messages, voice input data, etc.

[1949] The "means for transmitting information" is a function that compresses the information collected from the terminal and transmits it to the server and emotion analysis system via a communication network.

[1950] "Means for storing information" refers to a function that stores the transmitted information in a database so that it can be analyzed or referenced later.

[1951] "Means for classifying information" is a function that organizes saved information into categories such as "visual information," "location information," "text information," and "emotional information."

[1952] "Means for analyzing text information using natural language processing technology" refers to technology that analyzes collected text messages and app usage history to identify users' interests and behavioral patterns.

[1953] "Means for recognizing a user's emotional state in real time using emotion analysis means" refers to a function that analyzes voice tone and facial expressions to identify a user's emotional state in real time.

[1954] "Means for scheduling notifications" is a feature that calculates the optimal time to notify based on the user's current location, time of day, and emotional state, and generates appropriate suggestions and reminders.

[1955] "Means for collecting likes" is a function that collects the interest and ratings that users show in suggestions and reminders and stores them as feedback.

[1956] The "means for calculating and distributing rewards to providers" is a function that analyzes which suggestions and reminders have been highly rated based on the high ratings, calculates rewards accordingly, and distributes them to providers.

[1957] The present invention is a system that collects and analyzes information from a user's electronic device to generate and notify relevant suggestions to the user. The system also incorporates an emotion analysis engine that recognizes the user's emotional state in real time and adaptively changes the content of the suggestions based on that information.

[1958] System configuration

[1959] The system consists of the following main components:

[1960] User electronic devices: Portable electronic devices such as smart glasses and smartphones collect users' visual information, voice commands, emotional data such as facial expressions, location information, text messages, and app usage history.

[1961] Server: A central processing unit that stores data, performs classification and analysis, and generates recommendations for users. Natural language processing technology is used here to analyze text information.

[1962] Emotion Analysis Engine: A software component that analyzes voice tone, facial expressions, and language to recognize the user's emotional state in real time.

[1963] Communication network: The infrastructure that connects users' electronic devices with the server and emotion analysis engine.

[1964] Information collection and analysis

[1965] Terminal

[1966] 1. When a user puts on the smart glasses and starts an activity, the smart glasses capture visual information and collect emotional data such as voice commands and facial expressions.

[1967] 2. Your smartphone regularly collects location information and records your app usage history, text messages, and voice input data.

[1968] 3. The data collected on the device is compressed and sent via a communication network to a server and emotion analysis engine.

[1969] server

[1970] 1. The server stores the information sent from the device in a database. The stored data is categorized into "visual information," "location information," "text information," "emotional information," etc.

[1971] 2. Using natural language processing (NLP) technology, the stored text data is analyzed to identify user interests and behavioral patterns.

[1972] 3. Real-time emotional data sent from the emotion analysis engine is also stored on the server and used for analysis.

[1973] Sentiment Analysis Engine

[1974] 1. The emotion analysis engine analyzes voice tone and facial expressions to recognize the user's emotional state in real time.

[1975] 2. The recognized emotion information is sent to the server for further analysis and suggestion generation.

[1976] Proposal generation and notification

[1977] server

[1978] 1. Generate optimal suggestions and reminders based on user interest information, behavioral patterns, and emotional information.

[1979] 2. Calculate the notification schedule and make suggestions at the right time, taking into account the user's location, time of day, and emotional state.

[1980] Terminal

[1981] 1. Receive notification information sent from the server and sentiment analysis engine.

[1982] 2. Users can check the received suggestions through the smart glasses display or push notifications on their smartphones.

[1983] Likes and Feedback

[1984] User

[1985] 1. Users can review the suggestions they receive and rate them highly if they find them useful.

[1986] 2. Likes are sent to the server via the device.

[1987] server

[1988] 1. Aggregate the received positive feedback data and analyze which proposals received positive feedback.

[1989] 2. Execute a process to calculate and distribute rewards to the providers of the suggestions based on the high ratings.

[1990] Specific examples

[1991] When a user searches for "stylish cafes" through the smart glasses, the glasses capture this information and send it along with related information to a server. At the same time, an emotion analysis engine analyzes the user's facial expressions and tone of voice to determine if the user is in a happy mood. The server analyzes this information and generates coupon advertisements for nearby cafes, and notifies the user on their smartphone while they are walking around town that "there is a coupon available for a nearby cafe."

[1992] Prompt Sentence Examples

[1993] While searching for "stylish cafes," the system analyzes the user's facial expressions and tone of voice, recognizes that the user is in a good mood, and sends recommendations and coupons for nearby cafes to the user's smartphone. Furthermore, if the user gives a high rating, the system also explains how the rewards will be distributed to the advertiser.

[1994] The above is a specific embodiment of the system of the present invention, which allows users to receive personalized suggestions based on their emotions and behavior at the time, and further improves the quality of the suggestions by efficiently incorporating high-rated feedback.

[1995] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1996] Step 1: Gather information

[1997] Terminal

[1998] 1. The user puts on the smart glasses and begins their daily activities.

[1999] Input: User visual information, voice commands, facial expression data

[2000] How it works: The smart glasses' camera captures visual information from the surroundings, the microphone collects voice commands and emotional data (such as tone and intonation), and facial recognition technology is used to collect facial expression data.

[2001] Output: Visual information, audio data, emotion data

[2002] 2. Your smartphone records your location, app usage history, text messages, and voice input data.

[2003] Input: GPS information, app usage history, text messages, voice input

[2004] How it works: It uses the smartphone's GPS to obtain location information, and records app usage history, text messages, and voice input in the background.

[2005] Output: Location information, app usage history, text messages, voice data

[2006] Step 2: Submit your information

[2007] Terminal

[2008] 1. The collected information is periodically compressed and sent to a server and sentiment analysis engine via a communication network.

[2009] Input: Collected visual information, audio data, emotional data, location information, app usage history, text messages

[2010] Specific operation: A data compression algorithm is applied to reduce the size of the transmitted data, and the data is transferred to the server and sentiment analysis engine using a secure communication protocol.

[2011] Output: Compressed data packets

[2012] Step 3: Store and classify data

[2013] server

[2014] 1. Save the submitted information in a database.

[2015] Input: Compressed data packet

[2016] Specific operation: The data packet is extracted and stored in a database. Each data item is classified into "visual information," "location information," "text information," and "emotion information."

[2017] Output: Categorized data entries

[2018] Step 4: Data analysis

[2019] server

[2020] 1. Analyze text data using natural language processing (NLP) techniques to identify user interests and behavioral patterns.

[2021] Input: Text information stored on the server

[2022] What it does: It applies NLP algorithms to analyze text data and extract user interests and behavioral patterns, thereby identifying what users are interested in.

[2023] Output: User interest information, behavioral patterns

[2024] Step 5: Analyze the sentiment data

[2025] Sentiment Analysis Engine

[2026] 1. Analyzes voice tone and facial expressions to recognize the user's emotional state in real time.

[2027] Input: Voice data and facial expression data sent from the server

[2028] How it works: It applies voice analysis and facial recognition algorithms to identify the user's emotional state in real time. For example, if the voice tone is high, it is judged to be "happy," and if it is low, it is judged to be "depressed."

[2029] Output: Real-time emotional state data

[2030] Step 6: Generate proposals

[2031] server

[2032] 1. Generate optimal suggestions and reminders based on user interest information, behavioral patterns, and emotional information.

[2033] Inputs: Interest information, behavioral patterns, real-time emotional state data

[2034] How it works: Using a generative AI model, it generates the most suitable suggestions and reminders for the user based on this data. For example, if the user is in a happy mood and interested in "cafes," it generates coupons for nearby cafes.

[2035] Output: Generated suggestions and reminders

[2036] Step 7: Scheduling and sending notifications

[2037] server

[2038] 1. Schedule notifications at optimal times, taking into account the user's location, time of day, and emotional state.

[2039] Input: current location, time, emotion

[2040] Specific behavior: Apply a scheduling algorithm to calculate the optimal timing for notifications. For example, if the user is walking to a nearby cafe, a coupon will be instantly notified.

[2041] Output: Scheduled notification information

[2042] Step 8: View notifications

[2043] Terminal

[2044] 1. Present notification information sent from the server and sentiment analysis engine to the user.

[2045] Input: Scheduled notification information

[2046] Specific operation: A notification is displayed to the user via the smart glasses display or a push notification on the smartphone.

[2047] Output: Suggestions and reminders presented to the user

[2048] Step 9: Collect likes

[2049] User

[2050] 1. Users review and like suggestions and reminders.

[2051] Input: Notified suggestions and reminders

[2052] Specific operation: Operate the "Like" button using smart glasses or a smartphone.

[2053] Output: Liked feedback

[2054] Step 10: Analyze Likes and Calculate Rewards

[2055] server

[2056] 1. Aggregate the received positive feedback data and analyze which proposals received positive feedback.

[2057] Input: Liked feedback

[2058] Specific operation: Stores the high-rating data in a database and uses statistical analysis to identify the high-rating suggestions.

[2059] Output: Highly rated proposal data

[2060] 2. Calculate rewards based on the highly rated data and distribute them to providers.

[2061] Input: Highly rated proposal data

[2062] Specific operation: Apply the reward calculation algorithm to calculate the reward for the proposal provider and perform distribution processing.

[2063] Output: Distributed reward data

[2064] (Application example 2)

[2065] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2066] Conventional food delivery services do not take into account the user's current emotional state when making suggestions, resulting in an insufficient user experience and the inability to recommend the most suitable dishes or restaurants for the user. Furthermore, the suggestions given to users are uniform and not personalized, resulting in low user satisfaction.

[2067] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2068] In this invention, the server includes means for collecting information from the user's device, means for transmitting the collected information, means for storing the transmitted information, means for analyzing the stored information, means for analyzing text data using natural language processing technology, an emotion engine means for analyzing voice tone, facial expressions, and wording to recognize an emotional state, means for generating suggestions relevant to the user based on the analysis results, means for notifying the user of the generated suggestions, means for collecting user ratings, means for calculating rewards based on the user ratings, and means for distributing rewards to providers. This enables optimal food delivery suggestions based on the user's emotional state in the real world, improving the user experience.

[2069] "User device" refers to an electronic device used by a user to collect and transmit information, such as smart glasses or a smartphone.

[2070] A "means for transmitting collected information" is a process or device for transmitting information from a user's terminal to a server.

[2071] The "means for storing transmitted information" refers to a system or process within the server for storing the received information in a storage device such as a database.

[2072] "Means for analyzing stored information" means the process of analyzing collected information to identify user behavior patterns and interests.

[2073] "Natural language processing technology" is a technology for analyzing text data and understanding and generating human language.

[2074] The "emotion engine that recognizes emotional states by analyzing voice tone, facial expressions, and language" is a software component that identifies emotions from the user's voice and facial expressions and evaluates their state in real time.

[2075] The "means for generating relevant suggestions for users based on analysis results" refers to a system for automatically creating optimal suggestions based on users' emotional and behavioral data.

[2076] "Means for notifying users of generated suggestions" refers to the process or device for delivering generated suggestions to users, such as a smartglasses display or a smartphone push notification.

[2077] The "means for collecting high ratings from users" is a system that allows users to rate proposals and collect the rating data.

[2078] The "means for calculating remuneration based on user's high ratings" is a process for calculating appropriate remuneration for the provider based on the collected high rating data.

[2079] "Means for distributing rewards to providers" refers to a system or process for distributing calculated rewards to advertisers and service providers.

[2080] "A means for suggesting optimal food delivery options based on real-world emotional states" is a system that automatically suggests the most suitable dishes and restaurants to users based on their real-time emotional data.

[2081] The present invention relates to a system for providing optimal food delivery suggestions based on a user's emotional state. The system includes a user terminal, a server, and an emotion engine.

[2082] System configuration

[2083] The system consists of the following main components:

[2084] User devices: Electronic devices such as smart glasses and smartphones that collect information and send notifications.

[2085] Server: A central processing unit that stores data, analyzes it, and generates suggestions.

[2086] Emotion engine: Software that analyzes voice tone, facial expressions, and language to recognize emotional states.

[2087] Hardware and software used

[2088] Smart glasses: capturing visual information and voice commands.

[2089] Smartphones: Collecting location, text messages, and voice input data.

[2090] Server: Python, Django, MySQL for data storage and analysis.

[2091] Emotion engine: TensorFlow and PyTorch for voice tone and facial expression analysis.

[2092] Communication network: Wi-Fi, 4G / 5G communication infrastructure.

[2093] What the program does

[2094] Information gathering

[2095] The user's devices collect information: smart glasses capture the user's visual information and voice commands, and smartphones record location information, text messages, and voice input data. This information is periodically compressed and sent over the network to a server.

[2096] Data storage and analysis

[2097] The server stores the received information in a database and categorizes it into categories such as "visual information," "location information," "text information," and "emotion information." Natural language processing (NLP) is used to analyze the text data and identify the user's interests and behavioral patterns. Meanwhile, the emotion engine analyzes voice tone, facial expressions, and vocabulary to recognize the user's emotional state in real time. The recognized emotion data is also sent to the server for further analysis.

[2098] Proposal generation and notification

[2099] The server generates optimal food delivery suggestions based on the user's interest information, behavioral patterns, and emotional information. The suggestions provide restaurants and dishes that best suit the user's current emotional state. The suggestions are delivered to the user via the smartglasses display or push notifications on their smartphone.

[2100] For example, if a user is feeling stressed, the system will send a notification to their smartphone suggesting a relaxing dish or restaurant. If the user rates the suggestion favorably, the data will be sent to the server, and appropriate rewards will be distributed to advertisers and food delivery service providers.

[2101] Example prompt sentence:

[2102] Capture visual information and voice commands from the user's smart glasses, and collect location information and text messages from their smartphone, then send them to a server. Design a system that uses this data to analyze emotions in real time with an emotion engine and suggest food delivery options that fit the user's current emotional state. This system should also display personalized restaurant and food recommendation notifications on the smartphone, and if the user gives a high rating, send that data to the provider to calculate a reward.

[2103] The present invention enables optimal food delivery suggestions tailored to the user's real-world emotional state, improving the user experience. This specific example details how a system can be implemented to provide personalized suggestions that take the user's emotional state into account.

[2104] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2105] Step 1:

[2106] Information gathering

[2107] The devices collect information. Smart glasses capture the user's visual information and voice commands, while the smartphone records location information, text messages, and voice input data. This input data (visual information, voice commands, location information, text messages, and voice input) is obtained from the user's device. This data includes facial recognition, voice analysis, and location tracking. The obtained data is temporarily stored on the device.

[2108] Step 2:

[2109] Data transmission

[2110] The device sends the collected information to the server. The collected data is pre-compressed and sent to the server over the network. The compressed information packets are the input and are transferred to the server via the network infrastructure. The compressed data is sent as the output.

[2111] Step 3:

[2112] Data storage

[2113] The server stores the received information in a database. The database uses MySQL and stores the information by categorizing it into categories such as "visual information," "location information," "text information," and "emotional information." This data includes video frames, GPS data, text messages, and audio files. The input is compressed data packets, and the output is classified database entries.

[2114] Step 4:

[2115] Data analysis

[2116] The server analyzes the stored data. In particular, it uses natural language processing (NLP) to analyze text data and identify user interests and behavioral patterns. It uses TensorFlow and PyTorch to analyze voice tone and facial expressions to recognize emotional states in real time. The input is visual information, audio data, and text data read from the database, and the output is the analyzed user's behavioral patterns and emotional state.

[2117] Step 5:

[2118] Proposal generation

[2119] The server generates relevant suggestions for the user based on the analysis results. It combines the user's interest information, behavioral patterns, and emotional information to create optimal food delivery suggestions. The input is the analyzed behavioral patterns and emotional state data, and the output is specific restaurant and food suggestions.

[2120] Step 6:

[2121] notification

[2122] The device notifies the user of the generated suggestions. The suggestions are displayed to the user on the smartglasses display or via a smartphone push notification. The input is the generated suggestion data, and the output is a notification displayed on the user's device.

[2123] Step 7:

[2124] Evaluation collection

[2125] The terminal collects likes from users. The user rates the notified proposal and sends the rating data to the server. The input is the user's rating data, and the output is the rating information sent to the server.

[2126] Step 8:

[2127] Reward Calculation and Distribution

[2128] The server calculates rewards based on users' likes and distributes them to providers. Based on the collected likes data, appropriate rewards are calculated for advertisers and food delivery service providers. The input is the users' likes data, and the output is the calculated reward amount and its distribution information.

[2129] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user inpu...

Claims

1. How we collect information from your device; a means for transmitting the collected information; a means for storing the transmitted information; means for analyzing the stored information; means for generating relevant suggestions for a user; a means for notifying the user of the generated suggestions; A means of collecting positive user ratings, A means for calculating rewards based on user likes; a means for distributing rewards to providers; A system including:

2. a means of recording the user's periodic actions; means for generating reminders based on the recorded behavior; a means for notifying the user of the generated reminder; The system of claim 1 further comprising:

3. By analyzing the user's location and time information, The system of claim 1 further comprising means for providing timely suggestions and reminders.

Images