System

Abstract

A system is provided.SOLUTION: A system including means for collecting behavior data of a user, means for transmitting the collected behavior data to a cloud server, means for analyzing the behavior data transmitted to the cloud server, means for generating an option matching an interest or a behavior of the user based on an analysis result, means for notifying the user of the generated option, means for collecting and analyzing feedback from the user, and means for updating the analysis means and the option generation means based on the feedback.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In modern society, many people are faced with important life choices and find it difficult to make the best choice based on their interests and past behavior. This problem leads many to regret or feel anxious about their choices. Furthermore, even those around them and their families are unable to provide appropriate advice because they do not fully understand their individual detailed behaviors and interests. Given this background, there is a need for a system that can suggest optimal options based on one's past behaviors and interests and constantly assist in self-analysis. [Means for solving the problem]

[0005] The present invention provides a system that collects, transmits, and analyzes user behavioral data and generates options that match the user's interests and behavior. Specifically, the system includes means for collecting user behavioral data, means for transmitting the collected behavioral data to a cloud server, means for analyzing the behavioral data transmitted to the cloud server, means for generating options that match the user's interests and behavior based on the analysis results, means for notifying the user of the generated options, means for collecting and analyzing feedback from the user, and means for updating the analysis means and the option generation means based on the feedback. This allows the user to always receive optimal options based on their behavior and interests, thereby reducing anxiety and regret about important life choices.

[0006] "Behavioral data" refers to data that includes a user's daily activities, movement history, interests, hobbies, photos, text, voice memos, and other related information.

[0007] A "cloud server" is a centralized server system that stores, analyzes, and manages user behavior data via the Internet.

[0008] "Analysis" is the process of identifying and extracting users' behavioral patterns, interests, and hobbies based on collected behavioral data, and generating predictive models.

[0009] "Option generation" is the process of creating recommended information on occupations, further education, hobbies, and other topics that match the user's interests and behavior based on the analysis results.

[0010] "Notification" refers to an interface or means for informing the user of the information obtained through option generation, such as push notifications on a smartphone or in-app notifications.

[0011] "Feedback" refers to the user's action of providing a reaction or opinion, such as "interested" or "not interested," in response to the options notified to them.

[0012] "Updating" is the process of adjusting and improving the system's analysis and choice generation methods based on feedback obtained from users. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0014] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0017] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0018] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0019] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0021] [First embodiment]

[0022] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0023] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0026] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0029] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0033] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0034] This invention relates to an AI life partner system that collects behavioral data from users' daily lives, transmits it to a cloud server, and analyzes it to provide users with optimal choices. This system helps users make confident choices based on their own interests and behaviors. The specific program processing is described below.

[0035] System configuration

[0036] This system consists of the user's smartphone (device) and a cloud server (server). The device collects the user's behavioral data and sends it to the server. The server analyzes the received data and generates and notifies the user of the optimal options based on the results.

[0037] Program processing

[0038] 1. Data collection

[0039] User

[0040] Users can manually record important events and moving moments in their daily lives using smartphone apps. For example, they can take photos while traveling or record their impressions as voice memos.

[0041] Users can also record detailed events and emotions by entering text as needed.

[0042] Terminal

[0043] The device automatically collects the user's GPS information and activity data from social media, and records their behavioral patterns, such as the places they visited and the events they attended.

[0044] The device also stores manually entered text, photos, and voice memos in its database.

[0045] 2. Data transmission and storage

[0046] Terminal

[0047] The device periodically sends the collected data to a cloud server. For example, the device uploads the day's data to the server at 2:00 AM every day.

[0048] Attempts to send data as soon as an internet connection is established.

[0049] server

[0050] The server organizes and stores the data received from the device for each user. For example, it stores User A's travel data and voice memos together.

[0051] The server performs two-way data communication to verify data integrity.

[0052] Regularly back up your data to protect it from loss or corruption.

[0053] 3. Data Analysis

[0054] server

[0055] The server analyzes the stored data set to extract user behavior patterns, hobbies, and interests. For example, it identifies User B's interests based on the places he frequently visited and the events he attended over the past year.

[0056] Run machine learning algorithms to generate predictive models based on the data, then compare them with historical data to improve the accuracy of the analysis.

[0057] 4. Choice generation and notification

[0058] server

[0059] Based on the analysis results, options (occupations, schools, hobbies, etc.) that match the user's interests and behavior are generated. For example, based on User C's interest in chemistry, a list of appropriate schools and employment options is created.

[0060] Sends the formatted selection to the terminal.

[0061] Terminal

[0062] The device notifies the user of the suggested options and displays more information, for example, by using a push notification to let them know that the options have arrived.

[0063] The user checks the options notified and views the detailed information.

[0064] 5. User Feedback

[0065] User

[0066] The user provides feedback on the suggested options, for example by providing responses such as "interested" or "not interested."

[0067] Any further details or additional information can also be provided.

[0068] Terminal

[0069] The terminal collects user feedback and sends it to the server, and provides an interface for feedback input.

[0070] server

[0071] The server analyzes the received feedback and updates the proposal algorithm. The feedback data is analyzed and reflected in the next proposal.

[0072] This ensures that users always receive the best options based on their interests and behavior, giving them confidence and peace of mind when making important life decisions.

[0073] The processing flow will be explained below.

[0074] Step 1: Collect data

[0075] Users can launch the app on their smartphones and manually record important events or moving moments, for example, by taking photos while traveling or recording voice memos about how they felt that day.

[0076] The device automatically collects the user's GPS information and activity data from social networking services, and records their behavioral patterns, such as the places they visit and the events they attend.

[0077] The device stores manually entered text, photos, and voice memos in a local database.

[0078] Step 2: Data transmission and storage

[0079] The device periodically (e.g., at 2:00 AM every day) sends the collected data to the cloud server. When an Internet connection is established, the device automatically attempts to send the data.

[0080] The server organizes and stores the data received from the device for each user. For example, user A's travel data and voice memos are stored together in one folder.

[0081] The server communicates data in two directions to ensure data integrity, and regularly backs up data to protect it from loss or corruption.

[0082] Step 3: Analyze the data

[0083] The server runs machine learning algorithms to analyze the stored datasets, extracting user behavioral patterns, hobbies, and interests from the collected behavioral data.

[0084] The server compares the results with past data, generates predictive models, and keeps the analysis results up to date. For example, it identifies User B's interests based on the places he visited and events he attended over the past year.

[0085] Step 4: Generate and notify options

[0086] Based on the analysis results, the server generates options (occupations, schools, hobbies, etc.) that match the user's interests and behavior. For example, based on User C's interest in chemistry, it creates a list of suitable schools and employment options.

[0087] The server formats the generated options and transmits them to the terminal.

[0088] The device will send a push notification to the user to let them know that the suggested options have arrived, and will provide an interface for the user to view more information.

[0089] Step 5: User feedback

[0090] The user provides feedback on the suggested options, for example by responding to the options as "interested" or "not interested."

[0091] The device collects feedback from users and sends it to a cloud server, and also provides an interface for feedback input.

[0092] Step 6: Incorporating feedback and updating the system

[0093] The server analyzes the received feedback and updates the proposed algorithm and predictive model. For example, based on User D's feedback, the server adjusts the algorithm to improve the accuracy of future choice generation.

[0094] The server reflects the updated analysis results in the next data analysis and option generation, thereby continuously improving the accuracy of suggestions to users.

[0095] Example 1

[0096] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0097] In recent years, there has been an increasing demand for systems that utilize user behavioral and interest data to make more personalized and effective recommendations. However, conventional systems have faced many technical challenges in efficiently collecting diverse data from users' daily lives and analyzing it using machine learning algorithms. In particular, it has been difficult to verify the completeness of the data, ensure the consistency of the collected data, and appropriately incorporate user feedback. This has made it difficult to provide highly accurate options that match users' behavior and interests.

[0098] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0099] In this invention, the server includes: means for collecting user behavioral data; means for transmitting the collected behavioral data to a cloud server; means for analyzing the behavioral data transmitted to the cloud server; means for generating options that match the user's interests and behavior based on the analysis results; means for notifying the user of the generated options; means for collecting and analyzing feedback from the user; means for updating the analysis means and the option generation means based on the feedback; a mobile information terminal equipped with an application for collecting behavioral data in the user's daily life; means for storing the collected behavioral data on the cloud and verifying its integrity; means for analyzing the user's interests by executing a machine learning algorithm based on the stored data; means for creating suggestions using a generative AI model based on the results; and means for transmitting the generated suggestions to the terminal and notifying the user. This enables the generation and notification of highly accurate options that match the user's behavior and interests while ensuring the integrity of the collected data.

[0100] "Behavioral data" refers to information about a user's daily activities, such as their location, the applications they use, their social media activity, the places they visit, and the events they attend.

[0101] A "cloud server" refers to a server system that stores, analyzes, and processes data in a data center located in a remote location via the Internet.

[0102] "Machine learning algorithms" refer to computer programs and mathematical models that automatically learn patterns and knowledge from data.

[0103] A "generative AI model" refers to a model that uses artificial intelligence technology to create suggestions or products tailored to specific purposes.

[0104] "Mobile information terminal" refers to an electronic device that can be carried by a user, such as a smartphone or tablet.

[0105] "Application" refers to software that runs on a mobile information terminal and provides specific functions.

[0106] "Feedback" refers to the user's thoughts, evaluations, additional information, etc., entered in response to suggestions received from the system.

[0107] "Push notification" refers to a mechanism for sending information from a server to a user's device in real time.

[0108] "Data integrity" refers to the state of ensuring that data is accurate and free of omissions and errors.

[0109] This invention relates to an AI life partner system that collects behavioral data from users' daily lives, transmits it to a cloud server, and analyzes it to provide optimal options to users. This system proposes advanced options based on the user's interests and behavior, helping the user make choices with confidence.

[0110] System configuration

[0111] This system consists of a user's mobile information device (e.g., a smartphone) and a cloud server. The mobile information device collects the user's behavioral data and sends it to the cloud server. The cloud server analyzes the received data and, based on the results, generates and notifies the user of the optimal options.

[0112] Hardware and software used

[0113] Hardware:

[0114] Mobile devices: Users' smartphones, tablets, etc.

[0115] Cloud server: A remote data center that stores, analyzes, and processes data over the internet.

[0116] software:

[0117] Database Management System (DBMS): A data storage system such as SQLite or MySQL.

[0118] Machine learning libraries: Algorithms such as TensorFlow, scikit-learn, etc.

[0119] Communication protocol: Uses HTTP to send and receive data through cloud API.

[0120] Push notification service: Firebase Cloud Messaging (FCM), etc.

[0121] Program processing

[0122] Each process of the program of this system will be explained in detail below.

[0123] Data collection

[0124] Users can manually record important events and moving moments in their daily lives using a smartphone app. For example, they can save photos taken while traveling or their thoughts as voice memos. They can also record detailed events and emotions by entering text as needed.

[0125] The device automatically acquires the user's GPS information and activity data from social networking services (SNS), records their behavioral patterns, such as the places they visit and the events they attend, and also stores manually entered text, photos, and voice memos in a database.

[0126] Data transmission and storage

[0127] The device periodically sends collected data to the cloud server every day. For example, it uploads the day's data to the server at 2:00 a.m. Data is sent using the HTTP protocol, and if the transmission fails, it retries.

[0128] The server organizes and stores data received from the device for each user. For example, it stores User A's travel data and voice memos in cloud storage. It also compares the data with the original data to ensure data integrity. It also regularly backs up the data.

[0129] Data analysis

[0130] The server analyzes the stored data using machine learning algorithms. For example, it analyzes data on places that User B frequently visited and events that he or she attended over the past year to identify the user's interests. The analysis uses cloud computing resources and machine learning libraries such as TensorFlow and scikit-learn.

[0131] Choice generation and notification

[0132] The server generates options that match the user based on the analysis results. For example, it lists future schools and careers based on User C's interest in science. It uses a generative AI model to create prompts and configure the options. The generated options are then sent to the device in JSON format and notified.

[0133] The device notifies the user of the choices sent by the server. Push notifications are used to let the user know that new suggestions have arrived. If the user responds to the notification, more information can be displayed in the app.

[0134] User Feedback

[0135] The user inputs feedback on the proposed options, including impressions such as "interested" or "not interested," as well as new hopes and requests.

[0136] The device collects the feedback input by the user and transmits it to the cloud server, and provides an interface for feedback input.

[0137] The server analyzes the received feedback and reflects it in the next proposal, thereby updating the analysis means and option generation means, so that the next proposal will be more useful to the user.

[0138] Specific examples

[0139] For example, if a user records photos and voice memos taken during a trip in a smartphone app, the following process will occur.

[0140] Example prompt sentence:

[0141] Photos and voice memos taken during a user's trip are uploaded to a cloud server. The system analyzes the user's interests based on travel data from the past year and suggests new travel destinations and events to participate in.

[0142] Based on this prompt, the server collects user behavior data and sends, analyzes, and generates options to suggest suitable travel destinations and events to the user. In this way, the system can continually provide the user with the best options.

[0143] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0144] Step 1: Collect data

[0145] Users can manually record important events and moving moments in their daily lives using a smartphone app. For example, users can record photos taken while traveling and their feelings as voice memos. They can also record detailed events and emotions by entering text as needed.

[0146] The device automatically acquires the user's GPS information and social media activity data. For example, it collects data on the places the user has visited and the events they have attended. This data is stored in a database within the device. Inputs include photos, voice memos, text, GPS information, and social media data, and by storing these in the internal database, a database of behavioral patterns is generated.

[0147] Step 2: Data transmission and storage

[0148] The device periodically sends collected data to the cloud server every day. For example, the day's data is uploaded to the server at 2:00 AM every day. If the internet connection is unstable, the device attempts to send the data the moment a connection is established. This is the process by which data is collected and sent from the device to the server. The input is saved user behavior data (photos, voice memos, text, GPS information, SNS data), which is sent to the cloud server via the HTTP protocol.

[0149] The server organizes the data received from the device for each user and stores it in cloud storage. For example, it stores User A's travel data and voice memos using a specific folder structure. To ensure data integrity, the server compares the received data with the original data. In addition, it regularly backs up the data to prevent data loss.

[0150] Step 3: Analyze the data

[0151] The server analyzes the stored data using machine learning algorithms. For example, it can identify user interests based on the places User B frequently visited and events he or she participated in over the past year. The input is the user's behavioral data stored in the cloud. This is input to a machine learning algorithm (TensorFlow or scikit-learn) and analyzed to obtain results on the user's interests and behavioral patterns. This analysis process uses high-performance computing resources on the cloud.

[0152] Step 4: Generate and notify options

[0153] The server generates options that match the user based on the analysis results. For example, it lists suitable educational and career options based on User C's interest in science. The input is the analysis results, and a generative AI model (e.g., GPT-4) is used to create a new prompt. This process generates appropriate option suggestions for the user.

[0154] The device notifies the user of the choices sent by the server. A push notification is used to let the user know that a new suggestion has arrived. When the user interacts with the notification, more information can be displayed within the app, for example, details on science studies or career opportunities. The input to the prompt is the generated suggestion, and the output is a notification to the user and a display of the more information.

[0155] Step 5: User feedback

[0156] The user inputs feedback on the proposed options, including their impressions such as "interested" or "not interested," as well as new hopes and requests. The input is text data of the feedback, which is sent to the next processing step.

[0157] The device collects the feedback entered by the user and transmits it back to the cloud server, which then receives the feedback data and performs analysis.

[0158] The server analyzes the received feedback and reflects it in the next proposal. The input is the user's feedback data, which is analyzed to update the proposed algorithm. The feedback data is also used as training data for the next round. The output is an improved proposed algorithm and the next proposal.

[0159] (Application example 1)

[0160] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0161] While conventional behavioral data collection systems can collect and analyze user data, they have a problem in that they cannot use the results to suggest optimal products to users in real time. In particular, to suggest products that are in line with a user's interests and behavior, rapid and highly accurate data analysis is required, but no system with this function currently exists.

[0162] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0163] In this invention, the server includes means for collecting user behavioral data, means for transmitting the collected behavioral data to a cloud server, and means for analyzing the behavioral data transmitted to the cloud server, thereby enabling the server to utilize a generative AI model to suggest optimal products in accordance with prompt sentences based on the user's preferences.

[0164] "Behavioral data" refers to information that indicates a user's movement history, activities, and interests in their daily lives.

[0165] "Cloud server" refers to a remote server for storing, analyzing, and managing data via the Internet.

[0166] "Analysis means" refers to a system or device for analyzing collected data and extracting user behavior patterns and preferences.

[0167] "Option generation means" refers to a function for generating optimal suggestions and recommendations for users based on analyzed data.

[0168] "Notification means" refers to a function or mechanism for notifying the user of generated options or suggestions.

[0169] "Feedback" refers to data entered by a user in response to or comment on the suggestions or options provided.

[0170] "Smart devices" refers to portable electronic devices with advanced functions such as smartphones, smart glasses, and head-mounted displays.

[0171] A "generative AI model" refers to a model that uses artificial intelligence to analyze data and generate suggestions and recommendations that are appropriate for the user.

[0172] A "prompt" refers to an instruction or input used by a generative AI model to make a suggestion.

[0173] This invention relates to an AI life partner system that collects user behavioral data, transmits it to a cloud server, and analyzes it to provide users with optimal choices. This system proposes products that best suit the user's interests and preferences based on the user's behavioral data in their daily lives.

[0174] System configuration

[0175] This system mainly consists of the following hardware and software:

[0176] Hardware: Smart devices such as smartphones, smart glasses, and head-mounted displays

[0177] Software: Python, machine learning libraries (such as Scikit-learn), internet connection

[0178] The terminal (smart device) collects user behavior data and sends it to a cloud server. The cloud server analyzes the received data and generates and notifies the user of the best options based on the results. This allows users to receive product suggestions that match their interests in real time.

[0179] Program processing

[0180] Data collection

[0181] User

[0182] Users use smart devices to record their activities as they go about their daily lives. The system automatically collects location and activity data from GPS and social media. Users can also record photos and voice memos of their travels.

[0183] Terminal

[0184] The device automatically collects this data and stores it in a database, including GPS information, places visited, events attended, and manually entered text, photos, and voice notes.

[0185] Data transmission and analysis

[0186] Terminal

[0187] The device periodically sends collected data to the cloud server, and attempts to send the data when an Internet connection is established.

[0188] server

[0189] The server organizes and stores the data received from the device for each user, and performs two-way data communication to verify the integrity of the data. It then analyzes the data using machine learning algorithms (e.g., KMeans clustering) to extract user behavioral patterns and preferences. Specifically, it analyzes the places the user visited, the events they attended, and the content they posted to identify the user's interests.

[0190] Product suggestions and notifications

[0191] server

[0192] Based on the analysis results, a generative AI model is used to generate optimal options (product suggestions) for the user, such as fashion items, books, travel destinations, etc.

[0193] Terminal

[0194] The device notifies the user of the generated options and displays detailed information. For example, the device can use the push notification function on a smartphone or smart glasses to notify the user of suggested product information in real time.

[0195] Collecting and analyzing feedback

[0196] User

[0197] The user provides feedback on the proposed options, for example by providing responses such as "interested" or "not interested."

[0198] Terminal

[0199] The terminal collects user feedback and sends it to the server.

[0200] server

[0201] The server analyzes the received feedback and updates its proposal algorithm, which improves the accuracy of the next proposal.

[0202] Specific examples

[0203] For example, if the user's behavioral data is recorded as "visiting a fashion shop in Shibuya and looking for a new jacket," the system can analyze that data to identify the user's preferences and suggest the most suitable jacket. In this case, the system uses a generative AI model to generate prompt sentences.

[0204] Example prompt sentence:

[0205] User visited: Shibuya, Tokyo

[0206] Activities: Fashion shop visits

[0207] Data collected: GPS data, locations visited, activities, text input ("Looking for a new jacket")

[0208] Analysis-based suggestions: Recommending the perfect jacket for the user

[0209] In this way, users can receive optimal product suggestions in real time based on their interests and behavior.

[0210] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0211] Step 1:

[0212] The device collects data on the user's daily life activities. Specifically, it stores GPS data, SNS activity data, and text, photos, and voice memos manually entered by the user in a database. The input is the user's behavioral information data, and the output is the behavioral data stored in the device's database.

[0213] Step 2:

[0214] The device sends the collected behavioral data to a cloud server. The data is uploaded the moment an internet connection is established or at a specific time (for example, every night at 2:00 AM). The input is the behavioral data stored on the device, and the output is the data sent to the cloud server.

[0215] Step 3:

[0216] The server organizes and stores the received data for each user. The server performs two-way data communication and checks the integrity of the data. The input is the data received from the terminal, and the output is the organized and stored data.

[0217] Step 4:

[0218] The server analyzes the stored dataset to extract user behavioral patterns, hobbies, and interests. Specifically, it runs machine learning algorithms (e.g., KMeans clustering) and generates predictive models based on the data. The input is an organized dataset, and the output is an analysis of the user's interests and preferences.

[0219] Step 5:

[0220] The server uses a generative AI model based on the analysis results to generate optimal options (e.g., product suggestions) for the user. It uses prompt sentences to list products that match the user's interests. The input is the analysis results and associated prompt sentences, and the output is the generated product options.

[0221] Step 6:

[0222] The terminal notifies the user of the generated options. Using the push notification function, suggested product information is communicated to the user in real time. The input is the option data from the server, and the output is a notification to the user.

[0223] Step 7:

[0224] The user provides feedback on the proposed options, for example, by inputting a response such as "interested" or "not interested." The input is the user's feedback, and the output is the feedback data sent to the terminal.

[0225] Step 8:

[0226] The terminal collects user feedback and sends it to the cloud server. The input is the user feedback data, and the output is the data sent to the cloud server.

[0227] Step 9:

[0228] The server analyzes the received feedback and updates the proposal algorithm, improving the accuracy of the next proposal. The input is the feedback data, and the output is an updated predictive model.

[0229] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0230] This invention relates to an AI life partner system that collects behavioral and emotional data from users' daily lives, transmits it to a cloud server for analysis, and then provides optimal options to users. This system helps users make confident choices based on their interests and behavior, and by combining it with an emotion recognition engine, it realizes more personalized suggestions.

[0231] System configuration

[0232] This system consists of the user's smartphone (device), a cloud-based server (server), and an emotion recognition engine. The device collects the user's behavioral and emotional data and sends it to the server. The server analyzes the received data and generates and notifies the user of the optimal options based on the results. The emotion recognition engine also analyzes emotions from the user's input data and adds them to the behavioral analysis to generate more accurate options.

[0233] Program processing

[0234] 1. Data collection

[0235] User

[0236] Users can manually record important events and moving moments in their daily lives using smartphone apps, for example, by taking photos while traveling and recording voice memos about their feelings that day.

[0237] If desired, the user can enter text to record detailed events and emotions.

[0238] Terminal

[0239] The device automatically collects the user's GPS information and activity data from social media, and records their behavioral patterns, such as the places they visited and the events they attended.

[0240] It also stores manually entered text, photos, and voice notes in a local database and sends this data to an emotion recognition engine.

[0241] 2. Data transmission and storage

[0242] Terminal

[0243] The device periodically (e.g., at 2:00 AM every day) sends the collected data to the cloud server. When an Internet connection is established, the device automatically attempts to send the data.

[0244] server

[0245] The server organizes and stores the data received from the device for each user. For example, user A's travel data and voice memos are stored together in one folder.

[0246] Two-way data communication is conducted to verify data integrity, and data is backed up regularly to protect it from loss or corruption.

[0247] 3. Emotion Data Analysis

[0248] Emotion Recognition Engine

[0249] The emotion recognition engine extracts emotions from user-entered text, photos, and voice memos. For example, it analyzes voice to identify emotions such as "happiness" or "sadness," and identifies emotions from facial expressions and text in photos.

[0250] server

[0251] The server sorts the emotion data received from the emotion recognition engine by purpose and integrates it with behavioral data for analysis. For example, it comprehensively analyzes the places visited and events attended by User B over the past year, as well as the emotions they felt at the time.

[0252] 4. Choice generation and notification

[0253] server

[0254] Based on the analysis results, options (occupations, schools, hobbies, etc.) that match the user's interests, behavior, and emotions are generated. For example, a list of optimal schools and employment options is created based on User C's past behavior and emotional data.

[0255] The generated options are formatted and sent to the terminal.

[0256] Terminal

[0257] The system sends a push notification to the user to let them know when the suggested options arrive, and provides an interface for the user to view more information.

[0258] 5. User Feedback

[0259] User

[0260] Enter feedback on the suggested options, for example, providing responses such as "interested" or "not interested."

[0261] Any further details or additional information can also be provided.

[0262] Terminal

[0263] Collects feedback from users and sends it to the cloud server. Provides an interface for feedback input.

[0264] 6. Reflecting feedback and updating the system

[0265] server

[0266] Analyze the received feedback and update the proposed algorithm and predictive model. For example, adjust the algorithm based on User D's feedback to improve the accuracy of future option generation.

[0267] The updated analysis results are reflected in the next data analysis and option generation, thereby continuously improving the accuracy of suggestions to users.

[0268] As a result, users can receive more personalized options based on detailed behavioral data, including emotional data. This system provides powerful support for users to make confident choices based on their interests, behaviors, and emotions.

[0269] The processing flow will be explained below.

[0270] Step 1: Collect data

[0271] Users can launch the app on their smartphones and manually record important events or moving moments, for example, by taking photos while traveling and recording voice memos about their feelings that day.

[0272] Users can optionally enter text to record detailed events and emotions.

[0273] The device automatically collects the user's GPS information and activity data from social media, and records their behavioral patterns, such as the places they visited and the events they attended.

[0274] The device stores manually entered text, photos, and voice notes in a local database and sends this data to an emotion recognition engine.

[0275] Step 2: Data transmission and storage

[0276] The device periodically (e.g., at 2:00 AM every day) sends the collected data to the cloud server. When an Internet connection is established, the device automatically attempts to send the data.

[0277] The server organizes and stores the data received from the device for each user. For example, user A's travel data and voice memos are stored together in one folder.

[0278] The server communicates with the data in two directions to verify data integrity, and regularly backs up the data to protect it from loss or corruption.

[0279] Step 3: Analyze the emotion data

[0280] The emotion recognition engine extracts emotions from user-entered text, photos, and voice memos. For example, it analyzes voice to identify emotions such as "happiness" or "sadness," and identifies emotions from facial expressions and text in photos.

[0281] The server sorts the emotion data received from the emotion recognition engine by purpose and integrates it with behavioral data for analysis. For example, it comprehensively analyzes the places visited and events attended by User B over the past year, as well as the emotions they felt at the time.

[0282] Step 4: Analyze behavioral data and generate options

[0283] The server runs machine learning algorithms to analyze the stored behavioral datasets, extracting user behavior patterns, hobbies, and interests from the collected behavioral data.

[0284] By integrating the emotional data, the server can obtain more personalized analysis results. For example, by incorporating the emotional data of User C at a specific event, the server can refine future event suggestions.

[0285] Based on the analysis results, options (occupations, schools, hobbies, etc.) that match the user's interests, behavior, and emotions are generated. For example, a list of optimal schools and employment options is created based on User D's past behavior and emotional data.

[0286] Step 5: Communicate your options

[0287] The server formats the generated options and transmits them to the terminal.

[0288] The device will send a push notification to the user to let them know that a suggested option has arrived, and will also provide an interface for the user to view more information.

[0289] Step 6: User feedback

[0290] The user provides feedback on the suggested options, for example by providing responses such as "interested" or "not interested."

[0291] The user can also provide further details or additional information.

[0292] The device collects feedback from users and sends it to the cloud server, and provides an interface for feedback input.

[0293] Step 7: Incorporating feedback and updating the system

[0294] The server analyzes the received feedback and updates the proposal algorithm and predictive model. For example, based on user E's feedback, the server adjusts the algorithm to improve the accuracy of future choice generation.

[0295] The server reflects the updated analysis results in the next data analysis and option generation, thereby continuously improving the accuracy of suggestions to users.

[0296] As a specific example, by suggesting other suitable universities based on emotion data to the university that User F has given feedback that he or she is "interested in," the user's interest may increase. This can reduce anxiety and regret regarding important life decisions.

[0297] Example 2

[0298] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0299] Many modern users generate large amounts of behavioral and emotional data in their daily lives, but there are only a limited number of systems that effectively utilize this data to provide personalized options. Furthermore, existing systems have difficulty fully analyzing users' emotions and feedback and making highly accurate suggestions based on the user's interests and behavior. Given this background, there is a need for a system that effectively utilizes users' behavioral and emotional data to provide the best options for the user.

[0300] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0301] In this invention, the server includes means for collecting user behavioral data and emotional data, means for transmitting the collected data to a cloud server, means for analyzing the received data, means for analyzing emotions using an emotion recognition engine, means for generating optimal options based on the analysis results, means for notifying the user of the generated options, means for collecting and analyzing user feedback, and means for updating the system based on the feedback, thereby making it possible to provide more personalized options based on the user's behavior and emotions.

[0302] A "user" is an individual or entity that utilizes the system to provide behavioral and emotional data.

[0303] "Behavioral data" is data related to the user's behavior, such as location information and events attended in the user's daily life.

[0304] "Emotion data" is data that indicates the emotions such as joy and sadness felt by the user, and is information extracted from voice memos, photographs, and text.

[0305] A "cloud server" is a remote computer server used to store, manage, and analyze data over the Internet.

[0306] An "emotion recognition engine" is software or hardware for analyzing a user's emotions from data such as voice, photos, and text.

[0307] "Choices" are multiple suggestions for a particular action or decision that are generated based on the user's interests, behavior, and emotions.

[0308] "Feedback" refers to the reactions and opinions that users provide to the system regarding the proposed options.

[0309] "Analysis means" refers to the process or function that analyzes the collected behavioral and emotional data and extracts meaningful information based on the user's interests and behavior.

[0310] "Update means" refers to means for improving the analysis algorithm and the proposal generation process based on user feedback.

[0311] This invention relates to an AI life partner system that collects behavioral and emotional data from users' daily lives, transmits it to a cloud server for analysis, and then provides optimal options to users. This system assists users in making choices based on their interests and behavior, and by combining it with an emotion recognition engine, it realizes more personalized suggestions.

[0312] System configuration

[0313] This system consists of the user's smartphone (device), a cloud-based server (server), and an emotion recognition engine. The device collects the user's behavioral and emotional data and sends it to the server. The server analyzes the received data and generates and notifies the user of the optimal options based on the results. The emotion recognition engine also analyzes emotions from the user's input data and adds them to the behavioral analysis to generate more accurate options.

[0314] Hardware and software used

[0315] 1. Device: A smartphone is used, which allows for the acquisition of location data such as GPS information, automatic collection of SNS activity data, and collection of manually entered humorous text, photos, and voice memos.

[0316] 2. Cloud Server: Use a cloud platform (e.g., Amazon Web Services, Google Cloud Platform, etc.) to store, organize, and analyze data.

[0317] 3. Emotion Recognition Engine: Use software tools to analyze emotions from voice, photos, and text, such as voice recognition software (e.g., Google Speech-to-Text API), image recognition software (e.g., Google Vision API), and natural language processing tools (e.g., TensorFlow, PyTorch, etc.).

[0318] Specific examples

[0319] As a specific example, consider the case where User A takes photos using a smartphone while traveling and records the emotions he or she felt that day as a voice memo. In this case, the device acquires this data along with GPS information and SNS activity history, and sends them to a cloud server. The emotion recognition engine analyzes emotions such as "I was moved" from the voice memo, and the server integrates this data to suggest places that User A might be interested in based on his or her travel experiences.

[0320] Example prompts to input to the generative AI model

[0321] "Please explain the processing details of the AI ​​Life Partner system's program, which analyzes behavioral and emotional data from daily life on a cloud server and suggests optimal options for the user. As a concrete example, please include a scenario in which data recorded during a trip is analyzed to suggest the best tourist spots."

[0322] A system configured in this way can effectively utilize behavioral and emotional data from the user's daily life to provide personalized options.

[0323] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0324] Step 1: Collect data

[0325] User

[0326] Users can manually record important events and moving moments in their daily lives using a dedicated smartphone app. For example, they can take photos while traveling and save the day's emotions as voice memos. This allows detailed data such as time, place, and event to be entered into the device.

[0327] Terminal

[0328] The device automatically acquires the user's GPS information and social media activity data and records their behavioral patterns. For example, data on places visited and events attended is automatically collected. It also stores text, photos, and voice memos manually entered by the user in a local database, and sends this data to an emotion recognition engine. The inputs are the user's location information, photos, text, and voice data, and the device prepares these data for transmission to a cloud server in bulk.

[0329] Step 2: Data transmission and storage

[0330] Terminal

[0331] The device periodically (e.g., at 2:00 AM every day) transmits the collected data to the cloud server. Once an Internet connection is established, the device automatically attempts to transmit data. The input data includes all collected behavioral and emotional data.

[0332] server

[0333] The server organizes and stores data received from the device for each user. For example, user A's travel data and voice memos are stored together in one folder. The input data includes all data sent from the device, and the output data is stored in an organized folder structure. In addition, two-way data communication is performed to verify data integrity, and data is regularly backed up to protect against loss or damage.

[0334] Step 3: Analyze the emotion data

[0335] Emotion Recognition Engine

[0336] The emotion recognition engine analyzes text, photos, and voice memos entered by the user to extract emotions. For example, it extracts emotions such as "happiness" or "sadness" from voice memos and identifies emotions from facial expressions and text included in photos. Input data includes various data provided by the user, and the analyzed emotional data is obtained as output.

[0337] server

[0338] The server sorts the emotion data received from the emotion recognition engine by purpose, and integrates it with behavioral data for analysis. For example, it analyzes the places visited and events attended by user B over the past year, along with the emotion data they felt at the time. The input data includes the emotion data and behavioral data sent from the emotion recognition engine, and the output is the result of the integrated analysis.

[0339] Step 4: Generate and notify options

[0340] server

[0341] Based on the analysis results, options that match the user's interests, behavior, and emotions are generated. For example, a list of optimal educational and employment options is created based on User C's past behavior and emotional data. The input data includes the results of the integrated analysis, and the generated options are formatted as the output.

[0342] Terminal

[0343] The device sends a push notification to the user informing them of the new options. The user can review the proposed options through an interface that allows them to view detailed information. Input data includes the generated options from the server, and the output is a notification to the user.

[0344] Step 5: User feedback

[0345] User

[0346] The user provides feedback on the proposed options, such as "interested" or "not interested," and can also provide additional information. The input data includes the user's feedback and additional information.

[0347] Terminal

[0348] The terminal collects feedback from the user and transmits it to the cloud server. An interface for inputting feedback is provided, and the input data includes the user's feedback, and the output data is transmitted to the cloud server.

[0349] Step 6: Incorporating feedback and updating the system

[0350] server

[0351] The server analyzes the received feedback and updates the proposal algorithm and prediction model. For example, based on user D's feedback, the algorithm is adjusted to improve the accuracy of future option generation. The input data includes the user's feedback, and the output is an updated analysis algorithm and prediction model. The updated analysis results are then reflected in the next data analysis and option generation. This allows the accuracy of suggestions to the user to be continuously improved.

[0352] (Application example 2)

[0353] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0354] Many users today face problems such as indecision and long selection times when choosing products in physical stores. In addition, there is a lack of product recommendations based on personal preferences and emotions, making it difficult to have an efficient and satisfying shopping experience.

[0355] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for collecting behavioral data and emotional data of users connected to a cloud server, a means for generating options that match the user's interests and behavior based on the analysis results, and a means connected to the smart glasses and providing information to the user in real time. This makes it possible to make personalized product recommendations to the user, providing an efficient and satisfying shopping experience.

[0356] "User behavior data" is information about how a user behaves and what they do in a particular situation or environment.

[0357] "Emotion data" is data that expresses the user's emotional state and is extracted from text, audio, images, and the like.

[0358] A "cloud server" is a collection of remote servers accessible via the Internet that provide data processing and storage capabilities.

[0359] "Analysis means" refers to the method or device used to process and analyze collected data and obtain the results.

[0360] The "option generation means" refers to a method or device for generating options to be presented to the user based on the analysis results.

[0361] "Notification means" refers to a method or device for notifying the user of the generated options or information.

[0362] The "feedback collection means" refers to a method or device for collecting feedback such as reactions and evaluations from users.

[0363] "Smart glasses" are eyeglass-like devices that have built-in electronic devices such as a display and camera, and are capable of displaying information and collecting data.

[0364] "Real-time" refers to a time unit where processing and response are nearly instantaneous, with minimal delay.

[0365] This invention is a system that provides users with personalized options by collecting behavioral and emotional data from their daily lives and analyzing them on a cloud server. Here, we describe a specific embodiment that uses smart glasses to enhance the shopping experience in brick-and-mortar stores.

[0366] System configuration

[0367] The system consists of the following major components:

[0368] 1. Smart Glasses

[0369] 2. Cloud Server

[0370] 3. Emotion Recognition Engine

[0371] 4. Shopping recommendation engine

[0372] Collecting user behavioral and emotional data

[0373] Smart glasses are equipped with a camera that can capture images of the user's face and surrounding environment, a microphone that can record audio, and a display that can display information. When a user shops in a physical store, the smart glasses collect data in the following ways:

[0374] A camera is used to periodically capture images of the user's face.

[0375] Use a microphone to record the user's speech and reactions.

[0376] Data transmission and storage

[0377] The smart glasses send the collected behavioral and emotional data to a cloud server, where the data is processed through the following steps:

[0378] The data is periodically uploaded to a cloud server via the Internet.

[0379] The data is organized and stored on a cloud server, and data integrity is confirmed through two-way communication.

[0380] Emotional Data Analysis

[0381] The emotion recognition engine analyzes the user's emotions from the data sent to the cloud server. Specifically, it includes the following processes:

[0382] Facial expressions are analyzed from facial images to identify emotions such as joy, anger, sadness, and happiness.

[0383] Emotions are extracted by analyzing the tone and content of the voice from the audio data.

[0384] Choice generation and notification

[0385] The shopping recommendation engine comprehensively analyzes emotional and behavioral data to recommend the most suitable products and services to users. This process is as follows:

[0386] Based on the results of the emotion recognition engine, the shopping recommendation engine generates the optimal product list.

[0387] Product information and recommendations are displayed in real time on the smart glasses display.

[0388] Feedback and System Updates

[0389] Users can provide feedback through the smart glasses, for example, by rating the recommended products using voice commands or a touch interface. This feedback information is sent to a cloud server and used to improve the accuracy of the analysis and choice generation algorithms.

[0390] Natural language explanation of program processing

[0391] The server includes a means for collecting user behavioral data and emotional data connected to the cloud server, a means for generating options that match the user's interests and behavior based on the analysis results, and a means for connecting to the smart glasses and providing information to the user in real time, thereby enabling personalized product recommendations to be made to the user, providing an efficient and satisfying shopping experience.

[0392] Specific examples

[0393] For example, when a user is browsing products in a physical store, the camera in the smart glasses captures the user's facial expressions, and the emotion recognition engine analyzes the images to determine their interests and satisfaction. The analysis results are sent to a cloud server, and the shopping recommendation engine selects products that match the user's preferences and displays them on the smart glasses' display. The user can provide feedback on the displayed information, and the results will be reflected in the next recommendation algorithm.

[0394] Prompt Sentence Examples

[0395] In developing this application, the prompts input to the generative AI model are as follows:

[0396] We are developing an application for smart glasses that uses a user's facial image data to analyze emotions in real time and, based on the analysis results, recommends optimal products from a shopping recommendation engine. The captured image data is sent to an emotion recognition API on the cloud, and the received emotion data is used to generate personalized product recommendations. The application also includes a function that allows users to provide feedback in real time.

[0397] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0398] Step 1: Collect data

[0399] Subject: Terminal

[0400] The smart glasses use a built-in camera and microphone to collect the user's facial images and voice data in real time. Facial images are captured periodically, and voice data is recorded of the user's speech and reactions. These data are temporarily stored in the device's local memory. The input data are facial images and voice data, and the output is facial image files and voice files stored in the local memory.

[0401] Step 2: Data transmission and storage

[0402] Subject: Terminal / Server

[0403] The collected facial image and voice data is sent to a cloud server via the Internet. The data is periodically batch processed and securely stored on the cloud server. During transmission, a checksum is generated to verify the integrity of the data, and two-way communication is performed. The input data is facial image and voice data from the device, and the output is a data file stored on the cloud server.

[0404] Step 3: Analyze the emotion data

[0405] Subject: Server

[0406] The cloud server sends the collected facial images and voice data to an emotion recognition engine. The emotion recognition engine analyzes facial expressions from images and extracts emotions from voice. This process uses a generative AI model. For example, image analysis identifies emotions such as "surprise" or "anger," while voice analysis extracts emotions from the tone and content of voice. The input data are the facial images and voice data stored on the cloud server, and the output is emotion data from the emotion recognition engine.

[0407] Step 4: Generate options

[0408] Subject: Server

[0409] Based on emotional and behavioral data, the shopping recommendation engine recommends the most suitable products and services to the user. Based on the analysis results, a product list that matches the user's interests is generated. The generated recommendation list is displayed in real time on the smart glasses display. The input data is emotional data from the emotion recognition engine, and the output is a personalized product recommendation list.

[0410] Step 5: Notify users

[0411] Subject: Terminal

[0412] The smart glasses display displays recommended product information sent from the server in real time. For example, information such as "Recommended Product: Product X - Price: 1000 yen" is displayed in the user's field of view. The input data is the recommended product list sent from the server, and the output is the information displayed on the display screen of the smart glasses.

[0413] Step 6: Gather feedback

[0414] Subject: Terminal / User

[0415] The user provides feedback on the recommended products through the smart glasses using voice commands or a touchpad. For example, the user inputs a voice command such as "I like this product." The device then sends this feedback data to the cloud server. The input data is the user's feedback comment, and the output is the feedback data sent to the cloud server.

[0416] Step 7: Analyze feedback and update the system

[0417] Subject: Server

[0418] The cloud server analyzes the collected feedback data and updates the algorithms of the analysis means and option generation means. For example, it may increase the frequency of recommending a particular product or modify the algorithm to recommend a different product. This feedback analysis improves the accuracy of the next recommendation. The input data is the feedback data from the user, and the output is the updated algorithm.

[0419] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0420] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0421] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0422] [Second embodiment]

[0423] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0424] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0425] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0426] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0427] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0428] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0429] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0430] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0431] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0432] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0433] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0434] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0435] This invention relates to an AI life partner system that collects behavioral data from users' daily lives, transmits it to a cloud server, and analyzes it to provide users with optimal choices. This system helps users make confident choices based on their own interests and behaviors. The specific program processing is described below.

[0436] System configuration

[0437] This system consists of the user's smartphone (device) and a cloud server (server). The device collects the user's behavioral data and sends it to the server. The server analyzes the received data and generates and notifies the user of the optimal options based on the results.

[0438] Program processing

[0439] 1. Data collection

[0440] User

[0441] Users can manually record important events and moving moments in their daily lives using smartphone apps. For example, they can take photos while traveling or record their impressions as voice memos.

[0442] Users can also record detailed events and emotions by entering text as needed.

[0443] Terminal

[0444] The device automatically collects the user's GPS information and activity data from social media, and records their behavioral patterns, such as the places they visited and the events they attended.

[0445] The device also stores manually entered text, photos, and voice memos in its database.

[0446] 2. Data transmission and storage

[0447] Terminal

[0448] The device periodically sends the collected data to a cloud server. For example, the device uploads the day's data to the server at 2:00 AM every day.

[0449] Attempts to send data as soon as an internet connection is established.

[0450] server

[0451] The server organizes and stores the data received from the device for each user. For example, it stores User A's travel data and voice memos together.

[0452] The server performs two-way data communication to verify data integrity.

[0453] Regularly back up your data to protect it from loss or corruption.

[0454] 3. Data Analysis

[0455] server

[0456] The server analyzes the stored data set to extract user behavior patterns, hobbies, and interests. For example, it identifies User B's interests based on the places he frequently visited and the events he attended over the past year.

[0457] Run machine learning algorithms to generate predictive models based on the data, then compare them with historical data to improve the accuracy of the analysis.

[0458] 4. Choice generation and notification

[0459] server

[0460] Based on the analysis results, options (occupations, schools, hobbies, etc.) that match the user's interests and behavior are generated. For example, based on User C's interest in chemistry, a list of appropriate schools and employment options is created.

[0461] Sends the formatted selection to the terminal.

[0462] Terminal

[0463] The device notifies the user of the suggested options and displays more information, for example, by using a push notification to let them know that the options have arrived.

[0464] The user checks the options notified and views the detailed information.

[0465] 5. User Feedback

[0466] User

[0467] The user provides feedback on the suggested options, for example by providing responses such as "interested" or "not interested."

[0468] Any further details or additional information can also be provided.

[0469] Terminal

[0470] The terminal collects user feedback and sends it to the server, and provides an interface for feedback input.

[0471] server

[0472] The server analyzes the received feedback and updates the proposal algorithm. The feedback data is analyzed and reflected in the next proposal.

[0473] This ensures that users always receive the best options based on their interests and behavior, giving them confidence and peace of mind when making important life decisions.

[0474] The processing flow will be explained below.

[0475] Step 1: Collect data

[0476] Users can launch the app on their smartphones and manually record important events or moving moments, for example, by taking photos while traveling or recording voice memos about how they felt that day.

[0477] The device automatically collects the user's GPS information and activity data from social networking services, and records their behavioral patterns, such as the places they visit and the events they attend.

[0478] The device stores manually entered text, photos, and voice memos in a local database.

[0479] Step 2: Data transmission and storage

[0480] The device periodically (e.g., at 2:00 AM every day) sends the collected data to the cloud server. When an Internet connection is established, the device automatically attempts to send the data.

[0481] The server organizes and stores the data received from the device for each user. For example, user A's travel data and voice memos are stored together in one folder.

[0482] The server communicates data in two directions to ensure data integrity, and regularly backs up data to protect it from loss or corruption.

[0483] Step 3: Analyze the data

[0484] The server runs machine learning algorithms to analyze the stored datasets, extracting user behavioral patterns, hobbies, and interests from the collected behavioral data.

[0485] The server compares the results with past data, generates predictive models, and keeps the analysis results up to date. For example, it identifies User B's interests based on the places he visited and events he attended over the past year.

[0486] Step 4: Generate and notify options

[0487] Based on the analysis results, the server generates options (occupations, schools, hobbies, etc.) that match the user's interests and behavior. For example, based on User C's interest in chemistry, it creates a list of suitable schools and employment options.

[0488] The server formats the generated options and transmits them to the terminal.

[0489] The device will send a push notification to the user to let them know that the suggested options have arrived, and will provide an interface for the user to view more information.

[0490] Step 5: User feedback

[0491] The user provides feedback on the suggested options, for example by responding to the options as "interested" or "not interested."

[0492] The device collects feedback from users and sends it to a cloud server, and also provides an interface for feedback input.

[0493] Step 6: Incorporating feedback and updating the system

[0494] The server analyzes the received feedback and updates the proposed algorithm and predictive model. For example, based on User D's feedback, the server adjusts the algorithm to improve the accuracy of future choice generation.

[0495] The server reflects the updated analysis results in the next data analysis and option generation, thereby continuously improving the accuracy of suggestions to users.

[0496] Example 1

[0497] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0498] In recent years, there has been an increasing demand for systems that utilize user behavioral and interest data to make more personalized and effective recommendations. However, conventional systems have faced many technical challenges in efficiently collecting diverse data from users' daily lives and analyzing it using machine learning algorithms. In particular, it has been difficult to verify the completeness of the data, ensure the consistency of the collected data, and appropriately incorporate user feedback. This has made it difficult to provide highly accurate options that match users' behavior and interests.

[0499] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0500] In this invention, the server includes: means for collecting user behavioral data; means for transmitting the collected behavioral data to a cloud server; means for analyzing the behavioral data transmitted to the cloud server; means for generating options that match the user's interests and behavior based on the analysis results; means for notifying the user of the generated options; means for collecting and analyzing feedback from the user; means for updating the analysis means and the option generation means based on the feedback; a mobile information terminal equipped with an application for collecting behavioral data in the user's daily life; means for storing the collected behavioral data on the cloud and verifying its integrity; means for analyzing the user's interests by executing a machine learning algorithm based on the stored data; means for creating suggestions using a generative AI model based on the results; and means for transmitting the generated suggestions to the terminal and notifying the user. This enables the generation and notification of highly accurate options that match the user's behavior and interests while ensuring the integrity of the collected data.

[0501] "Behavioral data" refers to information about a user's daily activities, such as their location, the applications they use, their social media activity, the places they visit, and the events they attend.

[0502] A "cloud server" refers to a server system that stores, analyzes, and processes data in a data center located in a remote location via the Internet.

[0503] "Machine learning algorithms" refer to computer programs and mathematical models that automatically learn patterns and knowledge from data.

[0504] A "generative AI model" refers to a model that uses artificial intelligence technology to create suggestions or products tailored to specific purposes.

[0505] "Mobile information terminal" refers to an electronic device that can be carried by a user, such as a smartphone or tablet.

[0506] "Application" refers to software that runs on a mobile information terminal and provides specific functions.

[0507] "Feedback" refers to the user's thoughts, evaluations, additional information, etc., entered in response to suggestions received from the system.

[0508] "Push notification" refers to a mechanism for sending information from a server to a user's device in real time.

[0509] "Data integrity" refers to the state of ensuring that data is accurate and free of omissions and errors.

[0510] This invention relates to an AI life partner system that collects behavioral data from users' daily lives, transmits it to a cloud server, and analyzes it to provide optimal options to users. This system proposes advanced options based on the user's interests and behavior, helping the user make choices with confidence.

[0511] System configuration

[0512] This system consists of a user's mobile information device (e.g., a smartphone) and a cloud server. The mobile information device collects the user's behavioral data and sends it to the cloud server. The cloud server analyzes the received data and, based on the results, generates and notifies the user of the optimal options.

[0513] Hardware and software used

[0514] Hardware:

[0515] Mobile devices: Users' smartphones, tablets, etc.

[0516] Cloud server: A remote data center that stores, analyzes, and processes data over the internet.

[0517] software:

[0518] Database Management System (DBMS): A data storage system such as SQLite or MySQL.

[0519] Machine learning libraries: Algorithms such as TensorFlow, scikit-learn, etc.

[0520] Communication protocol: Uses HTTP to send and receive data through cloud API.

[0521] Push notification service: Firebase Cloud Messaging (FCM), etc.

[0522] Program processing

[0523] Each process of the program of this system will be explained in detail below.

[0524] Data collection

[0525] Users can manually record important events and moving moments in their daily lives using a smartphone app. For example, they can save photos taken while traveling or their thoughts as voice memos. They can also record detailed events and emotions by entering text as needed.

[0526] The device automatically acquires the user's GPS information and activity data from social networking services (SNS), records their behavioral patterns, such as the places they visit and the events they attend, and also stores manually entered text, photos, and voice memos in a database.

[0527] Data transmission and storage

[0528] The device periodically sends collected data to the cloud server every day. For example, it uploads the day's data to the server at 2:00 a.m. Data is sent using the HTTP protocol, and if the transmission fails, it retries.

[0529] The server organizes and stores data received from the device for each user. For example, it stores User A's travel data and voice memos in cloud storage. It also compares the data with the original data to ensure data integrity. It also regularly backs up the data.

[0530] Data analysis

[0531] The server analyzes the stored data using machine learning algorithms. For example, it analyzes data on places that User B frequently visited and events that he or she attended over the past year to identify the user's interests. The analysis uses cloud computing resources and machine learning libraries such as TensorFlow and scikit-learn.

[0532] Choice generation and notification

[0533] The server generates options that match the user based on the analysis results. For example, it lists future schools and careers based on User C's interest in science. It uses a generative AI model to create prompts and configure the options. The generated options are then sent to the device in JSON format and notified.

[0534] The device notifies the user of the choices sent by the server. Push notifications are used to let the user know that new suggestions have arrived. If the user responds to the notification, more information can be displayed in the app.

[0535] User Feedback

[0536] The user inputs feedback on the proposed options, including impressions such as "interested" or "not interested," as well as new hopes and requests.

[0537] The device collects the feedback input by the user and transmits it to the cloud server, and provides an interface for feedback input.

[0538] The server analyzes the received feedback and reflects it in the next proposal, thereby updating the analysis means and option generation means, so that the next proposal will be more useful to the user.

[0539] Specific examples

[0540] For example, if a user records photos and voice memos taken during a trip in a smartphone app, the following process will occur.

[0541] Example prompt sentence:

[0542] Photos and voice memos taken during a user's trip are uploaded to a cloud server. The system analyzes the user's interests based on travel data from the past year and suggests new travel destinations and events to participate in.

[0543] Based on this prompt, the server collects user behavior data and sends, analyzes, and generates options to suggest suitable travel destinations and events to the user. In this way, the system can continually provide the user with the best options.

[0544] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0545] Step 1: Collect data

[0546] Users can manually record important events and moving moments in their daily lives using a smartphone app. For example, users can record photos taken while traveling and their feelings as voice memos. They can also record detailed events and emotions by entering text as needed.

[0547] The device automatically acquires the user's GPS information and social media activity data. For example, it collects data on the places the user has visited and the events they have attended. This data is stored in a database within the device. Inputs include photos, voice memos, text, GPS information, and social media data, and by storing these in the internal database, a database of behavioral patterns is generated.

[0548] Step 2: Data transmission and storage

[0549] The device periodically sends collected data to the cloud server every day. For example, the day's data is uploaded to the server at 2:00 AM every day. If the internet connection is unstable, the device attempts to send the data the moment a connection is established. This is the process by which data is collected and sent from the device to the server. The input is saved user behavior data (photos, voice memos, text, GPS information, SNS data), which is sent to the cloud server via the HTTP protocol.

[0550] The server organizes the data received from the device for each user and stores it in cloud storage. For example, it stores User A's travel data and voice memos using a specific folder structure. To ensure data integrity, the server compares the received data with the original data. In addition, it regularly backs up the data to prevent data loss.

[0551] Step 3: Analyze the data

[0552] The server analyzes the stored data using machine learning algorithms. For example, it can identify user interests based on the places User B frequently visited and events he or she participated in over the past year. The input is the user's behavioral data stored in the cloud. This is input to a machine learning algorithm (TensorFlow or scikit-learn) and analyzed to obtain results on the user's interests and behavioral patterns. This analysis process uses high-performance computing resources on the cloud.

[0553] Step 4: Generate and notify options

[0554] The server generates options that match the user based on the analysis results. For example, it lists suitable educational and career options based on User C's interest in science. The input is the analysis results, and a generative AI model (e.g., GPT-4) is used to create a new prompt. This process generates appropriate option suggestions for the user.

[0555] The device notifies the user of the choices sent by the server. A push notification is used to let the user know that a new suggestion has arrived. When the user interacts with the notification, more information can be displayed within the app, for example, details on science studies or career opportunities. The input to the prompt is the generated suggestion, and the output is a notification to the user and a display of the more information.

[0556] Step 5: User feedback

[0557] The user inputs feedback on the proposed options, including their impressions such as "interested" or "not interested," as well as new hopes and requests. The input is text data of the feedback, which is sent to the next processing step.

[0558] The device collects the feedback entered by the user and transmits it back to the cloud server, which then receives the feedback data and performs analysis.

[0559] The server analyzes the received feedback and reflects it in the next proposal. The input is the user's feedback data, which is analyzed to update the proposed algorithm. The feedback data is also used as training data for the next round. The output is an improved proposed algorithm and the next proposal.

[0560] (Application example 1)

[0561] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0562] While conventional behavioral data collection systems can collect and analyze user data, they have a problem in that they cannot use the results to suggest optimal products to users in real time. In particular, to suggest products that are in line with a user's interests and behavior, rapid and highly accurate data analysis is required, but no system with this function currently exists.

[0563] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0564] In this invention, the server includes means for collecting user behavioral data, means for transmitting the collected behavioral data to a cloud server, and means for analyzing the behavioral data transmitted to the cloud server, thereby enabling the server to utilize a generative AI model to suggest optimal products in accordance with prompt sentences based on the user's preferences.

[0565] "Behavioral data" refers to information that indicates a user's movement history, activities, and interests in their daily lives.

[0566] "Cloud server" refers to a remote server for storing, analyzing, and managing data via the Internet.

[0567] "Analysis means" refers to a system or device for analyzing collected data and extracting user behavior patterns and preferences.

[0568] "Option generation means" refers to a function for generating optimal suggestions and recommendations for users based on analyzed data.

[0569] "Notification means" refers to a function or mechanism for notifying the user of generated options or suggestions.

[0570] "Feedback" refers to data entered by a user in response to or comment on the suggestions or options provided.

[0571] "Smart devices" refers to portable electronic devices with advanced functions such as smartphones, smart glasses, and head-mounted displays.

[0572] A "generative AI model" refers to a model that uses artificial intelligence to analyze data and generate suggestions and recommendations that are appropriate for the user.

[0573] A "prompt" refers to an instruction or input used by a generative AI model to make a suggestion.

[0574] This invention relates to an AI life partner system that collects user behavioral data, transmits it to a cloud server, and analyzes it to provide users with optimal choices. This system proposes products that best suit the user's interests and preferences based on the user's behavioral data in their daily lives.

[0575] System configuration

[0576] This system mainly consists of the following hardware and software:

[0577] Hardware: Smart devices such as smartphones, smart glasses, and head-mounted displays

[0578] Software: Python, machine learning libraries (such as Scikit-learn), internet connection

[0579] The terminal (smart device) collects user behavior data and sends it to a cloud server. The cloud server analyzes the received data and generates and notifies the user of the best options based on the results. This allows users to receive product suggestions that match their interests in real time.

[0580] Program processing

[0581] Data collection

[0582] User

[0583] Users use smart devices to record their activities as they go about their daily lives. The system automatically collects location and activity data from GPS and social media. Users can also record photos and voice memos of their travels.

[0584] Terminal

[0585] The device automatically collects this data and stores it in a database, including GPS information, places visited, events attended, and manually entered text, photos, and voice notes.

[0586] Data transmission and analysis

[0587] Terminal

[0588] The device periodically sends collected data to the cloud server, and attempts to send the data when an Internet connection is established.

[0589] server

[0590] The server organizes and stores the data received from the device for each user, and performs two-way data communication to verify the integrity of the data. It then analyzes the data using machine learning algorithms (e.g., KMeans clustering) to extract user behavioral patterns and preferences. Specifically, it analyzes the places the user visited, the events they attended, and the content they posted to identify the user's interests.

[0591] Product suggestions and notifications

[0592] server

[0593] Based on the analysis results, a generative AI model is used to generate optimal options (product suggestions) for the user, such as fashion items, books, travel destinations, etc.

[0594] Terminal

[0595] The device notifies the user of the generated options and displays detailed information. For example, the device can use the push notification function on a smartphone or smart glasses to notify the user of suggested product information in real time.

[0596] Collecting and analyzing feedback

[0597] User

[0598] The user provides feedback on the proposed options, for example by providing responses such as "interested" or "not interested."

[0599] Terminal

[0600] The terminal collects user feedback and sends it to the server.

[0601] server

[0602] The server analyzes the received feedback and updates its proposal algorithm, which improves the accuracy of the next proposal.

[0603] Specific examples

[0604] For example, if the user's behavioral data is recorded as "visiting a fashion shop in Shibuya and looking for a new jacket," the system can analyze that data to identify the user's preferences and suggest the most suitable jacket. In this case, the system uses a generative AI model to generate prompt sentences.

[0605] Example prompt sentence:

[0606] User visited: Shibuya, Tokyo

[0607] Activities: Fashion shop visits

[0608] Data collected: GPS data, locations visited, activities, text input ("Looking for a new jacket")

[0609] Analysis-based suggestions: Recommending the perfect jacket for the user

[0610] In this way, users can receive optimal product suggestions in real time based on their interests and behavior.

[0611] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0612] Step 1:

[0613] The device collects data on the user's daily life activities. Specifically, it stores GPS data, SNS activity data, and text, photos, and voice memos manually entered by the user in a database. The input is the user's behavioral information data, and the output is the behavioral data stored in the device's database.

[0614] Step 2:

[0615] The device sends the collected behavioral data to a cloud server. The data is uploaded the moment an internet connection is established or at a specific time (for example, every night at 2:00 AM). The input is the behavioral data stored on the device, and the output is the data sent to the cloud server.

[0616] Step 3:

[0617] The server organizes and stores the received data for each user. The server performs two-way data communication and checks the integrity of the data. The input is the data received from the terminal, and the output is the organized and stored data.

[0618] Step 4:

[0619] The server analyzes the stored dataset to extract user behavioral patterns, hobbies, and interests. Specifically, it runs machine learning algorithms (e.g., KMeans clustering) and generates predictive models based on the data. The input is an organized dataset, and the output is an analysis of the user's interests and preferences.

[0620] Step 5:

[0621] The server uses a generative AI model based on the analysis results to generate optimal options (e.g., product suggestions) for the user. It uses prompt sentences to list products that match the user's interests. The input is the analysis results and associated prompt sentences, and the output is the generated product options.

[0622] Step 6:

[0623] The terminal notifies the user of the generated options. Using the push notification function, suggested product information is communicated to the user in real time. The input is the option data from the server, and the output is a notification to the user.

[0624] Step 7:

[0625] The user provides feedback on the proposed options, for example, by inputting a response such as "interested" or "not interested." The input is the user's feedback, and the output is the feedback data sent to the terminal.

[0626] Step 8:

[0627] The terminal collects user feedback and sends it to the cloud server. The input is the user feedback data, and the output is the data sent to the cloud server.

[0628] Step 9:

[0629] The server analyzes the received feedback and updates the proposal algorithm, improving the accuracy of the next proposal. The input is the feedback data, and the output is an updated predictive model.

[0630] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0631] This invention relates to an AI life partner system that collects behavioral and emotional data from users' daily lives, transmits it to a cloud server for analysis, and then provides optimal options to users. This system helps users make confident choices based on their interests and behavior, and by combining it with an emotion recognition engine, it realizes more personalized suggestions.

[0632] System configuration

[0633] This system consists of the user's smartphone (device), a cloud-based server (server), and an emotion recognition engine. The device collects the user's behavioral and emotional data and sends it to the server. The server analyzes the received data and generates and notifies the user of the optimal options based on the results. The emotion recognition engine also analyzes emotions from the user's input data and adds them to the behavioral analysis to generate more accurate options.

[0634] Program processing

[0635] 1. Data collection

[0636] User

[0637] Users can manually record important events and moving moments in their daily lives using smartphone apps, for example, by taking photos while traveling and recording voice memos about their feelings that day.

[0638] If desired, the user can enter text to record detailed events and emotions.

[0639] Terminal

[0640] The device automatically collects the user's GPS information and activity data from social media, and records their behavioral patterns, such as the places they visited and the events they attended.

[0641] It also stores manually entered text, photos, and voice notes in a local database and sends this data to an emotion recognition engine.

[0642] 2. Data transmission and storage

[0643] Terminal

[0644] The device periodically (e.g., at 2:00 AM every day) sends the collected data to the cloud server. When an Internet connection is established, the device automatically attempts to send the data.

[0645] server

[0646] The server organizes and stores the data received from the device for each user. For example, user A's travel data and voice memos are stored together in one folder.

[0647] Two-way data communication is conducted to verify data integrity, and data is backed up regularly to protect it from loss or corruption.

[0648] 3. Emotion Data Analysis

[0649] Emotion Recognition Engine

[0650] The emotion recognition engine extracts emotions from user-entered text, photos, and voice memos. For example, it analyzes voice to identify emotions such as "happiness" or "sadness," and identifies emotions from facial expressions and text in photos.

[0651] server

[0652] The server sorts the emotion data received from the emotion recognition engine by purpose and integrates it with behavioral data for analysis. For example, it comprehensively analyzes the places visited and events attended by User B over the past year, as well as the emotions they felt at the time.

[0653] 4. Choice generation and notification

[0654] server

[0655] Based on the analysis results, options (occupations, schools, hobbies, etc.) that match the user's interests, behavior, and emotions are generated. For example, a list of optimal schools and employment options is created based on User C's past behavior and emotional data.

[0656] The generated options are formatted and sent to the terminal.

[0657] Terminal

[0658] The system sends a push notification to the user to let them know when the suggested options arrive, and provides an interface for the user to view more information.

[0659] 5. User Feedback

[0660] User

[0661] Enter feedback on the suggested options, for example, providing responses such as "interested" or "not interested."

[0662] Any further details or additional information can also be provided.

[0663] Terminal

[0664] Collects feedback from users and sends it to the cloud server. Provides an interface for feedback input.

[0665] 6. Reflecting feedback and updating the system

[0666] server

[0667] Analyze the received feedback and update the proposed algorithm and predictive model. For example, adjust the algorithm based on User D's feedback to improve the accuracy of future option generation.

[0668] The updated analysis results are reflected in the next data analysis and option generation, thereby continuously improving the accuracy of suggestions to users.

[0669] As a result, users can receive more personalized options based on detailed behavioral data, including emotional data. This system provides powerful support for users to make confident choices based on their interests, behaviors, and emotions.

[0670] The processing flow will be explained below.

[0671] Step 1: Collect data

[0672] Users can launch the app on their smartphones and manually record important events or moving moments, for example, by taking photos while traveling and recording voice memos about their feelings that day.

[0673] Users can optionally enter text to record detailed events and emotions.

[0674] The device automatically collects the user's GPS information and activity data from social media, and records their behavioral patterns, such as the places they visited and the events they attended.

[0675] The device stores manually entered text, photos, and voice notes in a local database and sends this data to an emotion recognition engine.

[0676] Step 2: Data transmission and storage

[0677] The device periodically (e.g., at 2:00 AM every day) sends the collected data to the cloud server. When an Internet connection is established, the device automatically attempts to send the data.

[0678] The server organizes and stores the data received from the device for each user. For example, user A's travel data and voice memos are stored together in one folder.

[0679] The server communicates with the data in two directions to verify data integrity, and regularly backs up the data to protect it from loss or corruption.

[0680] Step 3: Analyze the emotion data

[0681] The emotion recognition engine extracts emotions from user-entered text, photos, and voice memos. For example, it analyzes voice to identify emotions such as "happiness" or "sadness," and identifies emotions from facial expressions and text in photos.

[0682] The server sorts the emotion data received from the emotion recognition engine by purpose and integrates it with behavioral data for analysis. For example, it comprehensively analyzes the places visited and events attended by User B over the past year, as well as the emotions they felt at the time.

[0683] Step 4: Analyze behavioral data and generate options

[0684] The server runs machine learning algorithms to analyze the stored behavioral datasets, extracting user behavior patterns, hobbies, and interests from the collected behavioral data.

[0685] By integrating the emotional data, the server can obtain more personalized analysis results. For example, by incorporating the emotional data of User C at a specific event, the server can refine future event suggestions.

[0686] Based on the analysis results, options (occupations, schools, hobbies, etc.) that match the user's interests, behavior, and emotions are generated. For example, a list of optimal schools and employment options is created based on User D's past behavior and emotional data.

[0687] Step 5: Communicate your options

[0688] The server formats the generated options and transmits them to the terminal.

[0689] The device will send a push notification to the user to let them know that a suggested option has arrived, and will also provide an interface for the user to view more information.

[0690] Step 6: User feedback

[0691] The user provides feedback on the suggested options, for example by providing responses such as "interested" or "not interested."

[0692] The user can also provide further details or additional information.

[0693] The device collects feedback from users and sends it to the cloud server, and provides an interface for feedback input.

[0694] Step 7: Incorporating feedback and updating the system

[0695] The server analyzes the received feedback and updates the proposal algorithm and predictive model. For example, based on user E's feedback, the server adjusts the algorithm to improve the accuracy of future choice generation.

[0696] The server reflects the updated analysis results in the next data analysis and option generation, thereby continuously improving the accuracy of suggestions to users.

[0697] As a specific example, by suggesting other suitable universities based on emotion data to the university that User F has given feedback that he or she is "interested in," the user's interest may increase. This can reduce anxiety and regret regarding important life decisions.

[0698] Example 2

[0699] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0700] Many modern users generate large amounts of behavioral and emotional data in their daily lives, but there are only a limited number of systems that effectively utilize this data to provide personalized options. Furthermore, existing systems have difficulty fully analyzing users' emotions and feedback and making highly accurate suggestions based on the user's interests and behavior. Given this background, there is a need for a system that effectively utilizes users' behavioral and emotional data to provide the best options for the user.

[0701] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0702] In this invention, the server includes means for collecting user behavioral data and emotional data, means for transmitting the collected data to a cloud server, means for analyzing the received data, means for analyzing emotions using an emotion recognition engine, means for generating optimal options based on the analysis results, means for notifying the user of the generated options, means for collecting and analyzing user feedback, and means for updating the system based on the feedback, thereby making it possible to provide more personalized options based on the user's behavior and emotions.

[0703] A "user" is an individual or entity that utilizes the system to provide behavioral and emotional data.

[0704] "Behavioral data" is data related to the user's behavior, such as location information and events attended in the user's daily life.

[0705] "Emotion data" is data that indicates the emotions such as joy and sadness felt by the user, and is information extracted from voice memos, photographs, and text.

[0706] A "cloud server" is a remote computer server used to store, manage, and analyze data over the Internet.

[0707] An "emotion recognition engine" is software or hardware for analyzing a user's emotions from data such as voice, photos, and text.

[0708] "Choices" are multiple suggestions for a particular action or decision that are generated based on the user's interests, behavior, and emotions.

[0709] "Feedback" refers to the reactions and opinions that users provide to the system regarding the proposed options.

[0710] "Analysis means" refers to the process or function that analyzes the collected behavioral and emotional data and extracts meaningful information based on the user's interests and behavior.

[0711] "Update means" refers to means for improving the analysis algorithm and the proposal generation process based on user feedback.

[0712] This invention relates to an AI life partner system that collects behavioral and emotional data from users' daily lives, transmits it to a cloud server for analysis, and then provides optimal options to users. This system assists users in making choices based on their interests and behavior, and by combining it with an emotion recognition engine, it realizes more personalized suggestions.

[0713] System configuration

[0714] This system consists of the user's smartphone (device), a cloud-based server (server), and an emotion recognition engine. The device collects the user's behavioral and emotional data and sends it to the server. The server analyzes the received data and generates and notifies the user of the optimal options based on the results. The emotion recognition engine also analyzes emotions from the user's input data and adds them to the behavioral analysis to generate more accurate options.

[0715] Hardware and software used

[0716] 1. Device: A smartphone is used, which allows for the acquisition of location data such as GPS information, automatic collection of SNS activity data, and collection of manually entered humorous text, photos, and voice memos.

[0717] 2. Cloud Server: Use a cloud platform (e.g., Amazon Web Services, Google Cloud Platform, etc.) to store, organize, and analyze data.

[0718] 3. Emotion Recognition Engine: Use software tools to analyze emotions from voice, photos, and text, such as voice recognition software (e.g., Google Speech-to-Text API), image recognition software (e.g., Google Vision API), and natural language processing tools (e.g., TensorFlow, PyTorch, etc.).

[0719] Specific examples

[0720] As a specific example, consider the case where User A takes photos using a smartphone while traveling and records the emotions he or she felt that day as a voice memo. In this case, the device acquires this data along with GPS information and SNS activity history, and sends them to a cloud server. The emotion recognition engine analyzes emotions such as "I was moved" from the voice memo, and the server integrates this data to suggest places that User A might be interested in based on his or her travel experiences.

[0721] Example prompts to input to the generative AI model

[0722] "Please explain the processing details of the AI ​​Life Partner system's program, which analyzes behavioral and emotional data from daily life on a cloud server and suggests optimal options for the user. As a concrete example, please include a scenario in which data recorded during a trip is analyzed to suggest the best tourist spots."

[0723] A system configured in this way can effectively utilize behavioral and emotional data from the user's daily life to provide personalized options.

[0724] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0725] Step 1: Collect data

[0726] User

[0727] Users can manually record important events and moving moments in their daily lives using a dedicated smartphone app. For example, they can take photos while traveling and save the day's emotions as voice memos. This allows detailed data such as time, place, and event to be entered into the device.

[0728] Terminal

[0729] The device automatically acquires the user's GPS information and social media activity data and records their behavioral patterns. For example, data on places visited and events attended is automatically collected. It also stores text, photos, and voice memos manually entered by the user in a local database, and sends this data to an emotion recognition engine. The inputs are the user's location information, photos, text, and voice data, and the device prepares these data for transmission to a cloud server in bulk.

[0730] Step 2: Data transmission and storage

[0731] Terminal

[0732] The device periodically (e.g., at 2:00 AM every day) transmits the collected data to the cloud server. Once an Internet connection is established, the device automatically attempts to transmit data. The input data includes all collected behavioral and emotional data.

[0733] server

[0734] The server organizes and stores data received from the device for each user. For example, user A's travel data and voice memos are stored together in one folder. The input data includes all data sent from the device, and the output data is stored in an organized folder structure. In addition, two-way data communication is performed to verify data integrity, and data is regularly backed up to protect against loss or damage.

[0735] Step 3: Analyze the emotion data

[0736] Emotion Recognition Engine

[0737] The emotion recognition engine analyzes text, photos, and voice memos entered by the user to extract emotions. For example, it extracts emotions such as "happiness" or "sadness" from voice memos and identifies emotions from facial expressions and text included in photos. Input data includes various data provided by the user, and the analyzed emotional data is obtained as output.

[0738] server

[0739] The server sorts the emotion data received from the emotion recognition engine by purpose, and integrates it with behavioral data for analysis. For example, it analyzes the places visited and events attended by user B over the past year, along with the emotion data they felt at the time. The input data includes the emotion data and behavioral data sent from the emotion recognition engine, and the output is the result of the integrated analysis.

[0740] Step 4: Generate and notify options

[0741] server

[0742] Based on the analysis results, options that match the user's interests, behavior, and emotions are generated. For example, a list of optimal educational and employment options is created based on User C's past behavior and emotional data. The input data includes the results of the integrated analysis, and the generated options are formatted as the output.

[0743] Terminal

[0744] The device sends a push notification to the user informing them of the new options. The user can review the proposed options through an interface that allows them to view detailed information. Input data includes the generated options from the server, and the output is a notification to the user.

[0745] Step 5: User feedback

[0746] User

[0747] The user provides feedback on the proposed options, such as "interested" or "not interested," and can also provide additional information. The input data includes the user's feedback and additional information.

[0748] Terminal

[0749] The terminal collects feedback from the user and transmits it to the cloud server. An interface for inputting feedback is provided, and the input data includes the user's feedback, and the output data is transmitted to the cloud server.

[0750] Step 6: Incorporating feedback and updating the system

[0751] server

[0752] The server analyzes the received feedback and updates the proposal algorithm and prediction model. For example, based on user D's feedback, the algorithm is adjusted to improve the accuracy of future option generation. The input data includes the user's feedback, and the output is an updated analysis algorithm and prediction model. The updated analysis results are then reflected in the next data analysis and option generation. This allows the accuracy of suggestions to the user to be continuously improved.

[0753] (Application example 2)

[0754] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0755] Many users today face problems such as indecision and long selection times when choosing products in physical stores. In addition, there is a lack of product recommendations based on personal preferences and emotions, making it difficult to have an efficient and satisfying shopping experience.

[0756] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for collecting behavioral data and emotional data of users connected to a cloud server, a means for generating options that match the user's interests and behavior based on the analysis results, and a means connected to the smart glasses and providing information to the user in real time. This makes it possible to make personalized product recommendations to the user, providing an efficient and satisfying shopping experience.

[0757] "User behavior data" is information about how a user behaves and what they do in a particular situation or environment.

[0758] "Emotion data" is data that expresses the user's emotional state and is extracted from text, audio, images, and the like.

[0759] A "cloud server" is a collection of remote servers accessible via the Internet that provide data processing and storage capabilities.

[0760] "Analysis means" refers to the method or device used to process and analyze collected data and obtain the results.

[0761] The "option generation means" refers to a method or device for generating options to be presented to the user based on the analysis results.

[0762] "Notification means" refers to a method or device for notifying the user of the generated options or information.

[0763] The "feedback collection means" refers to a method or device for collecting feedback such as reactions and evaluations from users.

[0764] "Smart glasses" are eyeglass-like devices that have built-in electronic devices such as a display and camera, and are capable of displaying information and collecting data.

[0765] "Real-time" refers to a time unit where processing and response are nearly instantaneous, with minimal delay.

[0766] This invention is a system that provides users with personalized options by collecting behavioral and emotional data from their daily lives and analyzing them on a cloud server. Here, we describe a specific embodiment that uses smart glasses to enhance the shopping experience in brick-and-mortar stores.

[0767] System configuration

[0768] The system consists of the following major components:

[0769] 1. Smart Glasses

[0770] 2. Cloud Server

[0771] 3. Emotion Recognition Engine

[0772] 4. Shopping recommendation engine

[0773] Collecting user behavioral and emotional data

[0774] Smart glasses are equipped with a camera that can capture images of the user's face and surrounding environment, a microphone that can record audio, and a display that can display information. When a user shops in a physical store, the smart glasses collect data in the following ways:

[0775] A camera is used to periodically capture images of the user's face.

[0776] Use a microphone to record the user's speech and reactions.

[0777] Data transmission and storage

[0778] The smart glasses send the collected behavioral and emotional data to a cloud server, where the data is processed through the following steps:

[0779] The data is periodically uploaded to a cloud server via the Internet.

[0780] The data is organized and stored on a cloud server, and data integrity is confirmed through two-way communication.

[0781] Emotional Data Analysis

[0782] The emotion recognition engine analyzes the user's emotions from the data sent to the cloud server. Specifically, it includes the following processes:

[0783] Facial expressions are analyzed from facial images to identify emotions such as joy, anger, sadness, and happiness.

[0784] Emotions are extracted by analyzing the tone and content of the voice from the audio data.

[0785] Choice generation and notification

[0786] The shopping recommendation engine comprehensively analyzes emotional and behavioral data to recommend the most suitable products and services to users. This process is as follows:

[0787] Based on the results of the emotion recognition engine, the shopping recommendation engine generates the optimal product list.

[0788] Product information and recommendations are displayed in real time on the smart glasses display.

[0789] Feedback and System Updates

[0790] Users can provide feedback through the smart glasses, for example, by rating the recommended products using voice commands or a touch interface. This feedback information is sent to a cloud server and used to improve the accuracy of the analysis and choice generation algorithms.

[0791] Natural language explanation of program processing

[0792] The server includes a means for collecting user behavioral data and emotional data connected to the cloud server, a means for generating options that match the user's interests and behavior based on the analysis results, and a means for connecting to the smart glasses and providing information to the user in real time, thereby enabling personalized product recommendations to be made to the user, providing an efficient and satisfying shopping experience.

[0793] Specific examples

[0794] For example, when a user is browsing products in a physical store, the camera in the smart glasses captures the user's facial expressions, and the emotion recognition engine analyzes the images to determine their interests and satisfaction. The analysis results are sent to a cloud server, and the shopping recommendation engine selects products that match the user's preferences and displays them on the smart glasses' display. The user can provide feedback on the displayed information, and the results will be reflected in the next recommendation algorithm.

[0795] Prompt Sentence Examples

[0796] In developing this application, the prompts input to the generative AI model are as follows:

[0797] We are developing an application for smart glasses that uses a user's facial image data to analyze emotions in real time and, based on the analysis results, recommends optimal products from a shopping recommendation engine. The captured image data is sent to an emotion recognition API on the cloud, and the received emotion data is used to generate personalized product recommendations. The application also includes a function that allows users to provide feedback in real time.

[0798] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0799] Step 1: Collect data

[0800] Subject: Terminal

[0801] The smart glasses use a built-in camera and microphone to collect the user's facial images and voice data in real time. Facial images are captured periodically, and voice data is recorded of the user's speech and reactions. These data are temporarily stored in the device's local memory. The input data are facial images and voice data, and the output is facial image files and voice files stored in the local memory.

[0802] Step 2: Data transmission and storage

[0803] Subject: Terminal / Server

[0804] The collected facial image and voice data is sent to a cloud server via the Internet. The data is periodically batch processed and securely stored on the cloud server. During transmission, a checksum is generated to verify the integrity of the data, and two-way communication is performed. The input data is facial image and voice data from the device, and the output is a data file stored on the cloud server.

[0805] Step 3: Analyze the emotion data

[0806] Subject: Server

[0807] The cloud server sends the collected facial images and voice data to an emotion recognition engine. The emotion recognition engine analyzes facial expressions from images and extracts emotions from voice. This process uses a generative AI model. For example, image analysis identifies emotions such as "surprise" or "anger," while voice analysis extracts emotions from the tone and content of voice. The input data are the facial images and voice data stored on the cloud server, and the output is emotion data from the emotion recognition engine.

[0808] Step 4: Generate options

[0809] Subject: Server

[0810] Based on emotional and behavioral data, the shopping recommendation engine recommends the most suitable products and services to the user. Based on the analysis results, a product list that matches the user's interests is generated. The generated recommendation list is displayed in real time on the smart glasses display. The input data is emotional data from the emotion recognition engine, and the output is a personalized product recommendation list.

[0811] Step 5: Notify users

[0812] Subject: Terminal

[0813] The smart glasses display displays recommended product information sent from the server in real time. For example, information such as "Recommended Product: Product X - Price: 1000 yen" is displayed in the user's field of view. The input data is the recommended product list sent from the server, and the output is the information displayed on the display screen of the smart glasses.

[0814] Step 6: Gather feedback

[0815] Subject: Terminal / User

[0816] The user provides feedback on the recommended products through the smart glasses using voice commands or a touchpad. For example, the user inputs a voice command such as "I like this product." The device then sends this feedback data to the cloud server. The input data is the user's feedback comment, and the output is the feedback data sent to the cloud server.

[0817] Step 7: Analyze feedback and update the system

[0818] Subject: Server

[0819] The cloud server analyzes the collected feedback data and updates the algorithms of the analysis means and option generation means. For example, it may increase the frequency of recommending a particular product or modify the algorithm to recommend a different product. This feedback analysis improves the accuracy of the next recommendation. The input data is the feedback data from the user, and the output is the updated algorithm.

[0820] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0821] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0822] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0823] [Third embodiment]

[0824] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0825] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0826] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0827] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0828] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0829] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0830] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0831] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0832] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0833] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0834] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0835] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0836] This invention relates to an AI life partner system that collects behavioral data from users' daily lives, transmits it to a cloud server, and analyzes it to provide users with optimal choices. This system helps users make confident choices based on their own interests and behaviors. The specific program processing is described below.

[0837] System configuration

[0838] This system consists of the user's smartphone (device) and a cloud server (server). The device collects the user's behavioral data and sends it to the server. The server analyzes the received data and generates and notifies the user of the optimal options based on the results.

[0839] Program processing

[0840] 1. Data collection

[0841] User

[0842] Users can manually record important events and moving moments in their daily lives using smartphone apps. For example, they can take photos while traveling or record their impressions as voice memos.

[0843] Users can also record detailed events and emotions by entering text as needed.

[0844] Terminal

[0845] The device automatically collects the user's GPS information and activity data from social media, and records their behavioral patterns, such as the places they visited and the events they attended.

[0846] The device also stores manually entered text, photos, and voice memos in its database.

[0847] 2. Data transmission and storage

[0848] Terminal

[0849] The device periodically sends the collected data to a cloud server. For example, the device uploads the day's data to the server at 2:00 AM every day.

[0850] Attempts to send data as soon as an internet connection is established.

[0851] server

[0852] The server organizes and stores the data received from the device for each user. For example, it stores User A's travel data and voice memos together.

[0853] The server performs two-way data communication to verify data integrity.

[0854] Regularly back up your data to protect it from loss or corruption.

[0855] 3. Data Analysis

[0856] server

[0857] The server analyzes the stored data set to extract user behavior patterns, hobbies, and interests. For example, it identifies User B's interests based on the places he frequently visited and the events he attended over the past year.

[0858] Run machine learning algorithms to generate predictive models based on the data, then compare them with historical data to improve the accuracy of the analysis.

[0859] 4. Choice generation and notification

[0860] server

[0861] Based on the analysis results, options (occupations, schools, hobbies, etc.) that match the user's interests and behavior are generated. For example, based on User C's interest in chemistry, a list of appropriate schools and employment options is created.

[0862] Sends the formatted selection to the terminal.

[0863] Terminal

[0864] The device notifies the user of the suggested options and displays more information, for example, by using a push notification to let them know that the options have arrived.

[0865] The user checks the options notified and views the detailed information.

[0866] 5. User Feedback

[0867] User

[0868] The user provides feedback on the suggested options, for example by providing responses such as "interested" or "not interested."

[0869] Any further details or additional information can also be provided.

[0870] Terminal

[0871] The terminal collects user feedback and sends it to the server, and provides an interface for feedback input.

[0872] server

[0873] The server analyzes the received feedback and updates the proposal algorithm. The feedback data is analyzed and reflected in the next proposal.

[0874] This ensures that users always receive the best options based on their interests and behavior, giving them confidence and peace of mind when making important life decisions.

[0875] The processing flow will be explained below.

[0876] Step 1: Collect data

[0877] Users can launch the app on their smartphones and manually record important events or moving moments, for example, by taking photos while traveling or recording voice memos about how they felt that day.

[0878] The device automatically collects the user's GPS information and activity data from social networking services, and records their behavioral patterns, such as the places they visit and the events they attend.

[0879] The device stores manually entered text, photos, and voice memos in a local database.

[0880] Step 2: Data transmission and storage

[0881] The device periodically (e.g., at 2:00 AM every day) sends the collected data to the cloud server. When an Internet connection is established, the device automatically attempts to send the data.

[0882] The server organizes and stores the data received from the device for each user. For example, user A's travel data and voice memos are stored together in one folder.

[0883] The server communicates data in two directions to ensure data integrity, and regularly backs up data to protect it from loss or corruption.

[0884] Step 3: Analyze the data

[0885] The server runs machine learning algorithms to analyze the stored datasets, extracting user behavioral patterns, hobbies, and interests from the collected behavioral data.

[0886] The server compares the results with past data, generates predictive models, and keeps the analysis results up to date. For example, it identifies User B's interests based on the places he visited and events he attended over the past year.

[0887] Step 4: Generate and notify options

[0888] Based on the analysis results, the server generates options (occupations, schools, hobbies, etc.) that match the user's interests and behavior. For example, based on User C's interest in chemistry, it creates a list of suitable schools and employment options.

[0889] The server formats the generated options and transmits them to the terminal.

[0890] The device will send a push notification to the user to let them know that the suggested options have arrived, and will provide an interface for the user to view more information.

[0891] Step 5: User feedback

[0892] The user provides feedback on the suggested options, for example by responding to the options as "interested" or "not interested."

[0893] The device collects feedback from users and sends it to a cloud server, and also provides an interface for feedback input.

[0894] Step 6: Incorporating feedback and updating the system

[0895] The server analyzes the received feedback and updates the proposed algorithm and predictive model. For example, based on User D's feedback, the server adjusts the algorithm to improve the accuracy of future choice generation.

[0896] The server reflects the updated analysis results in the next data analysis and option generation, thereby continuously improving the accuracy of suggestions to users.

[0897] Example 1

[0898] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0899] In recent years, there has been an increasing demand for systems that utilize user behavioral and interest data to make more personalized and effective recommendations. However, conventional systems have faced many technical challenges in efficiently collecting diverse data from users' daily lives and analyzing it using machine learning algorithms. In particular, it has been difficult to verify the completeness of the data, ensure the consistency of the collected data, and appropriately incorporate user feedback. This has made it difficult to provide highly accurate options that match users' behavior and interests.

[0900] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0901] In this invention, the server includes: means for collecting user behavioral data; means for transmitting the collected behavioral data to a cloud server; means for analyzing the behavioral data transmitted to the cloud server; means for generating options that match the user's interests and behavior based on the analysis results; means for notifying the user of the generated options; means for collecting and analyzing feedback from the user; means for updating the analysis means and the option generation means based on the feedback; a mobile information terminal equipped with an application for collecting behavioral data in the user's daily life; means for storing the collected behavioral data on the cloud and verifying its integrity; means for analyzing the user's interests by executing a machine learning algorithm based on the stored data; means for creating suggestions using a generative AI model based on the results; and means for transmitting the generated suggestions to the terminal and notifying the user. This enables the generation and notification of highly accurate options that match the user's behavior and interests while ensuring the integrity of the collected data.

[0902] "Behavioral data" refers to information about a user's daily activities, such as their location, the applications they use, their social media activity, the places they visit, and the events they attend.

[0903] A "cloud server" refers to a server system that stores, analyzes, and processes data in a data center located in a remote location via the Internet.

[0904] "Machine learning algorithms" refer to computer programs and mathematical models that automatically learn patterns and knowledge from data.

[0905] A "generative AI model" refers to a model that uses artificial intelligence technology to create suggestions or products tailored to specific purposes.

[0906] "Mobile information terminal" refers to an electronic device that can be carried by a user, such as a smartphone or tablet.

[0907] "Application" refers to software that runs on a mobile information terminal and provides specific functions.

[0908] "Feedback" refers to the user's thoughts, evaluations, additional information, etc., entered in response to suggestions received from the system.

[0909] "Push notification" refers to a mechanism for sending information from a server to a user's device in real time.

[0910] "Data integrity" refers to the state of ensuring that data is accurate and free of omissions and errors.

[0911] This invention relates to an AI life partner system that collects behavioral data from users' daily lives, transmits it to a cloud server, and analyzes it to provide optimal options to users. This system proposes advanced options based on the user's interests and behavior, helping the user make choices with confidence.

[0912] System configuration

[0913] This system consists of a user's mobile information device (e.g., a smartphone) and a cloud server. The mobile information device collects the user's behavioral data and sends it to the cloud server. The cloud server analyzes the received data and, based on the results, generates and notifies the user of the optimal options.

[0914] Hardware and software used

[0915] Hardware:

[0916] Mobile devices: Users' smartphones, tablets, etc.

[0917] Cloud server: A remote data center that stores, analyzes, and processes data over the internet.

[0918] software:

[0919] Database Management System (DBMS): A data storage system such as SQLite or MySQL.

[0920] Machine learning libraries: Algorithms such as TensorFlow, scikit-learn, etc.

[0921] Communication protocol: Uses HTTP to send and receive data through cloud API.

[0922] Push notification service: Firebase Cloud Messaging (FCM), etc.

[0923] Program processing

[0924] Each process of the program of this system will be explained in detail below.

[0925] Data collection

[0926] Users can manually record important events and moving moments in their daily lives using a smartphone app. For example, they can save photos taken while traveling or their thoughts as voice memos. They can also record detailed events and emotions by entering text as needed.

[0927] The device automatically acquires the user's GPS information and activity data from social networking services (SNS), records their behavioral patterns, such as the places they visit and the events they attend, and also stores manually entered text, photos, and voice memos in a database.

[0928] Data transmission and storage

[0929] The device periodically sends collected data to the cloud server every day. For example, it uploads the day's data to the server at 2:00 a.m. Data is sent using the HTTP protocol, and if the transmission fails, it retries.

[0930] The server organizes and stores data received from the device for each user. For example, it stores User A's travel data and voice memos in cloud storage. It also compares the data with the original data to ensure data integrity. It also regularly backs up the data.

[0931] Data analysis

[0932] The server analyzes the stored data using machine learning algorithms. For example, it analyzes data on places that User B frequently visited and events that he or she attended over the past year to identify the user's interests. The analysis uses cloud computing resources and machine learning libraries such as TensorFlow and scikit-learn.

[0933] Choice generation and notification

[0934] The server generates options that match the user based on the analysis results. For example, it lists future schools and careers based on User C's interest in science. It uses a generative AI model to create prompts and configure the options. The generated options are then sent to the device in JSON format and notified.

[0935] The device notifies the user of the choices sent by the server. Push notifications are used to let the user know that new suggestions have arrived. If the user responds to the notification, more information can be displayed in the app.

[0936] User Feedback

[0937] The user inputs feedback on the proposed options, including impressions such as "interested" or "not interested," as well as new hopes and requests.

[0938] The device collects the feedback input by the user and transmits it to the cloud server, and provides an interface for feedback input.

[0939] The server analyzes the received feedback and reflects it in the next proposal, thereby updating the analysis means and option generation means, so that the next proposal will be more useful to the user.

[0940] Specific examples

[0941] For example, if a user records photos and voice memos taken during a trip in a smartphone app, the following process will occur.

[0942] Example prompt sentence:

[0943] Photos and voice memos taken during a user's trip are uploaded to a cloud server. The system analyzes the user's interests based on travel data from the past year and suggests new travel destinations and events to participate in.

[0944] Based on this prompt, the server collects user behavior data and sends, analyzes, and generates options to suggest suitable travel destinations and events to the user. In this way, the system can continually provide the user with the best options.

[0945] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0946] Step 1: Collect data

[0947] Users can manually record important events and moving moments in their daily lives using a smartphone app. For example, users can record photos taken while traveling and their feelings as voice memos. They can also record detailed events and emotions by entering text as needed.

[0948] The device automatically acquires the user's GPS information and social media activity data. For example, it collects data on the places the user has visited and the events they have attended. This data is stored in a database within the device. Inputs include photos, voice memos, text, GPS information, and social media data, and by storing these in the internal database, a database of behavioral patterns is generated.

[0949] Step 2: Data transmission and storage

[0950] The device periodically sends collected data to the cloud server every day. For example, the day's data is uploaded to the server at 2:00 AM every day. If the internet connection is unstable, the device attempts to send the data the moment a connection is established. This is the process by which data is collected and sent from the device to the server. The input is saved user behavior data (photos, voice memos, text, GPS information, SNS data), which is sent to the cloud server via the HTTP protocol.

[0951] The server organizes the data received from the device for each user and stores it in cloud storage. For example, it stores User A's travel data and voice memos using a specific folder structure. To ensure data integrity, the server compares the received data with the original data. In addition, it regularly backs up the data to prevent data loss.

[0952] Step 3: Analyze the data

[0953] The server analyzes the stored data using machine learning algorithms. For example, it can identify user interests based on the places User B frequently visited and events he or she participated in over the past year. The input is the user's behavioral data stored in the cloud. This is input to a machine learning algorithm (TensorFlow or scikit-learn) and analyzed to obtain results on the user's interests and behavioral patterns. This analysis process uses high-performance computing resources on the cloud.

[0954] Step 4: Generate and notify options

[0955] The server generates options that match the user based on the analysis results. For example, it lists suitable educational and career options based on User C's interest in science. The input is the analysis results, and a generative AI model (e.g., GPT-4) is used to create a new prompt. This process generates appropriate option suggestions for the user.

[0956] The device notifies the user of the choices sent by the server. A push notification is used to let the user know that a new suggestion has arrived. When the user interacts with the notification, more information can be displayed within the app, for example, details on science studies or career opportunities. The input to the prompt is the generated suggestion, and the output is a notification to the user and a display of the more information.

[0957] Step 5: User feedback

[0958] The user inputs feedback on the proposed options, including their impressions such as "interested" or "not interested," as well as new hopes and requests. The input is text data of the feedback, which is sent to the next processing step.

[0959] The device collects the feedback entered by the user and transmits it back to the cloud server, which then receives the feedback data and performs analysis.

[0960] The server analyzes the received feedback and reflects it in the next proposal. The input is the user's feedback data, which is analyzed to update the proposed algorithm. The feedback data is also used as training data for the next round. The output is an improved proposed algorithm and the next proposal.

[0961] (Application example 1)

[0962] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0963] While conventional behavioral data collection systems can collect and analyze user data, they have a problem in that they cannot use the results to suggest optimal products to users in real time. In particular, to suggest products that are in line with a user's interests and behavior, rapid and highly accurate data analysis is required, but no system with this function currently exists.

[0964] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0965] In this invention, the server includes means for collecting user behavioral data, means for transmitting the collected behavioral data to a cloud server, and means for analyzing the behavioral data transmitted to the cloud server, thereby enabling the server to utilize a generative AI model to suggest optimal products in accordance with prompt sentences based on the user's preferences.

[0966] "Behavioral data" refers to information that indicates a user's movement history, activities, and interests in their daily lives.

[0967] "Cloud server" refers to a remote server for storing, analyzing, and managing data via the Internet.

[0968] "Analysis means" refers to a system or device for analyzing collected data and extracting user behavior patterns and preferences.

[0969] "Option generation means" refers to a function for generating optimal suggestions and recommendations for users based on analyzed data.

[0970] "Notification means" refers to a function or mechanism for notifying the user of generated options or suggestions.

[0971] "Feedback" refers to data entered by a user in response to or comment on the suggestions or options provided.

[0972] "Smart devices" refers to portable electronic devices with advanced functions such as smartphones, smart glasses, and head-mounted displays.

[0973] A "generative AI model" refers to a model that uses artificial intelligence to analyze data and generate suggestions and recommendations that are appropriate for the user.

[0974] A "prompt" refers to an instruction or input used by a generative AI model to make a suggestion.

[0975] This invention relates to an AI life partner system that collects user behavioral data, transmits it to a cloud server, and analyzes it to provide users with optimal choices. This system proposes products that best suit the user's interests and preferences based on the user's behavioral data in their daily lives.

[0976] System configuration

[0977] This system mainly consists of the following hardware and software:

[0978] Hardware: Smart devices such as smartphones, smart glasses, and head-mounted displays

[0979] Software: Python, machine learning libraries (such as Scikit-learn), internet connection

[0980] The terminal (smart device) collects user behavior data and sends it to a cloud server. The cloud server analyzes the received data and generates and notifies the user of the best options based on the results. This allows users to receive product suggestions that match their interests in real time.

[0981] Program processing

[0982] Data collection

[0983] User

[0984] Users use smart devices to record their activities as they go about their daily lives. The system automatically collects location and activity data from GPS and social media. Users can also record photos and voice memos of their travels.

[0985] Terminal

[0986] The device automatically collects this data and stores it in a database, including GPS information, places visited, events attended, and manually entered text, photos, and voice notes.

[0987] Data transmission and analysis

[0988] Terminal

[0989] The device periodically sends collected data to the cloud server, and attempts to send the data when an Internet connection is established.

[0990] server

[0991] The server organizes and stores the data received from the device for each user, and performs two-way data communication to verify the integrity of the data. It then analyzes the data using machine learning algorithms (e.g., KMeans clustering) to extract user behavioral patterns and preferences. Specifically, it analyzes the places the user visited, the events they attended, and the content they posted to identify the user's interests.

[0992] Product suggestions and notifications

[0993] server

[0994] Based on the analysis results, a generative AI model is used to generate optimal options (product suggestions) for the user, such as fashion items, books, travel destinations, etc.

[0995] Terminal

[0996] The device notifies the user of the generated options and displays detailed information. For example, the device can use the push notification function on a smartphone or smart glasses to notify the user of suggested product information in real time.

[0997] Collecting and analyzing feedback

[0998] User

[0999] The user provides feedback on the proposed options, for example by providing responses such as "interested" or "not interested."

[1000] Terminal

[1001] The terminal collects user feedback and sends it to the server.

[1002] server

[1003] The server analyzes the received feedback and updates its proposal algorithm, which improves the accuracy of the next proposal.

[1004] Specific examples

[1005] For example, if the user's behavioral data is recorded as "visiting a fashion shop in Shibuya and looking for a new jacket," the system can analyze that data to identify the user's preferences and suggest the most suitable jacket. In this case, the system uses a generative AI model to generate prompt sentences.

[1006] Example prompt sentence:

[1007] User visited: Shibuya, Tokyo

[1008] Activities: Fashion shop visits

[1009] Data collected: GPS data, locations visited, activities, text input ("Looking for a new jacket")

[1010] Analysis-based suggestions: Recommending the perfect jacket for the user

[1011] In this way, users can receive optimal product suggestions in real time based on their interests and behavior.

[1012] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1013] Step 1:

[1014] The device collects data on the user's daily life activities. Specifically, it stores GPS data, SNS activity data, and text, photos, and voice memos manually entered by the user in a database. The input is the user's behavioral information data, and the output is the behavioral data stored in the device's database.

[1015] Step 2:

[1016] The device sends the collected behavioral data to a cloud server. The data is uploaded the moment an internet connection is established or at a specific time (for example, every night at 2:00 AM). The input is the behavioral data stored on the device, and the output is the data sent to the cloud server.

[1017] Step 3:

[1018] The server organizes and stores the received data for each user. The server performs two-way data communication and checks the integrity of the data. The input is the data received from the terminal, and the output is the organized and stored data.

[1019] Step 4:

[1020] The server analyzes the stored dataset to extract user behavioral patterns, hobbies, and interests. Specifically, it runs machine learning algorithms (e.g., KMeans clustering) and generates predictive models based on the data. The input is an organized dataset, and the output is an analysis of the user's interests and preferences.

[1021] Step 5:

[1022] The server uses a generative AI model based on the analysis results to generate optimal options (e.g., product suggestions) for the user. It uses prompt sentences to list products that match the user's interests. The input is the analysis results and associated prompt sentences, and the output is the generated product options.

[1023] Step 6:

[1024] The terminal notifies the user of the generated options. Using the push notification function, suggested product information is communicated to the user in real time. The input is the option data from the server, and the output is a notification to the user.

[1025] Step 7:

[1026] The user provides feedback on the proposed options, for example, by inputting a response such as "interested" or "not interested." The input is the user's feedback, and the output is the feedback data sent to the terminal.

[1027] Step 8:

[1028] The terminal collects user feedback and sends it to the cloud server. The input is the user feedback data, and the output is the data sent to the cloud server.

[1029] Step 9:

[1030] The server analyzes the received feedback and updates the proposal algorithm, improving the accuracy of the next proposal. The input is the feedback data, and the output is an updated predictive model.

[1031] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1032] This invention relates to an AI life partner system that collects behavioral and emotional data from users' daily lives, transmits it to a cloud server for analysis, and then provides optimal options to users. This system helps users make confident choices based on their interests and behavior, and by combining it with an emotion recognition engine, it realizes more personalized suggestions.

[1033] System configuration

[1034] This system consists of the user's smartphone (device), a cloud-based server (server), and an emotion recognition engine. The device collects the user's behavioral and emotional data and sends it to the server. The server analyzes the received data and generates and notifies the user of the optimal options based on the results. The emotion recognition engine also analyzes emotions from the user's input data and adds them to the behavioral analysis to generate more accurate options.

[1035] Program processing

[1036] 1. Data collection

[1037] User

[1038] Users can manually record important events and moving moments in their daily lives using smartphone apps, for example, by taking photos while traveling and recording voice memos about their feelings that day.

[1039] If desired, the user can enter text to record detailed events and emotions.

[1040] Terminal

[1041] The device automatically collects the user's GPS information and activity data from social media, and records their behavioral patterns, such as the places they visited and the events they attended.

[1042] It also stores manually entered text, photos, and voice notes in a local database and sends this data to an emotion recognition engine.

[1043] 2. Data transmission and storage

[1044] Terminal

[1045] The device periodically (e.g., at 2:00 AM every day) sends the collected data to the cloud server. When an Internet connection is established, the device automatically attempts to send the data.

[1046] server

[1047] The server organizes and stores the data received from the device for each user. For example, user A's travel data and voice memos are stored together in one folder.

[1048] Two-way data communication is conducted to verify data integrity, and data is backed up regularly to protect it from loss or corruption.

[1049] 3. Emotion Data Analysis

[1050] Emotion Recognition Engine

[1051] The emotion recognition engine extracts emotions from user-entered text, photos, and voice memos. For example, it analyzes voice to identify emotions such as "happiness" or "sadness," and identifies emotions from facial expressions and text in photos.

[1052] server

[1053] The server sorts the emotion data received from the emotion recognition engine by purpose and integrates it with behavioral data for analysis. For example, it comprehensively analyzes the places visited and events attended by User B over the past year, as well as the emotions they felt at the time.

[1054] 4. Choice generation and notification

[1055] server

[1056] Based on the analysis results, options (occupations, schools, hobbies, etc.) that match the user's interests, behavior, and emotions are generated. For example, a list of optimal schools and employment options is created based on User C's past behavior and emotional data.

[1057] The generated options are formatted and sent to the terminal.

[1058] Terminal

[1059] The system sends a push notification to the user to let them know when the suggested options arrive, and provides an interface for the user to view more information.

[1060] 5. User Feedback

[1061] User

[1062] Enter feedback on the suggested options, for example, providing responses such as "interested" or "not interested."

[1063] Any further details or additional information can also be provided.

[1064] Terminal

[1065] Collects feedback from users and sends it to the cloud server. Provides an interface for feedback input.

[1066] 6. Reflecting feedback and updating the system

[1067] server

[1068] Analyze the received feedback and update the proposed algorithm and predictive model. For example, adjust the algorithm based on User D's feedback to improve the accuracy of future option generation.

[1069] The updated analysis results are reflected in the next data analysis and option generation, thereby continuously improving the accuracy of suggestions to users.

[1070] As a result, users can receive more personalized options based on detailed behavioral data, including emotional data. This system provides powerful support for users to make confident choices based on their interests, behaviors, and emotions.

[1071] The processing flow will be explained below.

[1072] Step 1: Collect data

[1073] Users can launch the app on their smartphones and manually record important events or moving moments, for example, by taking photos while traveling and recording voice memos about their feelings that day.

[1074] Users can optionally enter text to record detailed events and emotions.

[1075] The device automatically collects the user's GPS information and activity data from social media, and records their behavioral patterns, such as the places they visited and the events they attended.

[1076] The device stores manually entered text, photos, and voice notes in a local database and sends this data to an emotion recognition engine.

[1077] Step 2: Data transmission and storage

[1078] The device periodically (e.g., at 2:00 AM every day) sends the collected data to the cloud server. When an Internet connection is established, the device automatically attempts to send the data.

[1079] The server organizes and stores the data received from the device for each user. For example, user A's travel data and voice memos are stored together in one folder.

[1080] The server communicates with the data in two directions to verify data integrity, and regularly backs up the data to protect it from loss or corruption.

[1081] Step 3: Analyze the emotion data

[1082] The emotion recognition engine extracts emotions from user-entered text, photos, and voice memos. For example, it analyzes voice to identify emotions such as "happiness" or "sadness," and identifies emotions from facial expressions and text in photos.

[1083] The server sorts the emotion data received from the emotion recognition engine by purpose and integrates it with behavioral data for analysis. For example, it comprehensively analyzes the places visited and events attended by User B over the past year, as well as the emotions they felt at the time.

[1084] Step 4: Analyze behavioral data and generate options

[1085] The server runs machine learning algorithms to analyze the stored behavioral datasets, extracting user behavior patterns, hobbies, and interests from the collected behavioral data.

[1086] By integrating the emotional data, the server can obtain more personalized analysis results. For example, by incorporating the emotional data of User C at a specific event, the server can refine future event suggestions.

[1087] Based on the analysis results, options (occupations, schools, hobbies, etc.) that match the user's interests, behavior, and emotions are generated. For example, a list of optimal schools and employment options is created based on User D's past behavior and emotional data.

[1088] Step 5: Communicate your options

[1089] The server formats the generated options and transmits them to the terminal.

[1090] The device will send a push notification to the user to let them know that a suggested option has arrived, and will also provide an interface for the user to view more information.

[1091] Step 6: User feedback

[1092] The user provides feedback on the suggested options, for example by providing responses such as "interested" or "not interested."

[1093] The user can also provide further details or additional information.

[1094] The device collects feedback from users and sends it to the cloud server, and provides an interface for feedback input.

[1095] Step 7: Incorporating feedback and updating the system

[1096] The server analyzes the received feedback and updates the proposal algorithm and predictive model. For example, based on user E's feedback, the server adjusts the algorithm to improve the accuracy of future choice generation.

[1097] The server reflects the updated analysis results in the next data analysis and option generation, thereby continuously improving the accuracy of suggestions to users.

[1098] As a specific example, by suggesting other suitable universities based on emotion data to the university that User F has given feedback that he or she is "interested in," the user's interest may increase. This can reduce anxiety and regret regarding important life decisions.

[1099] Example 2

[1100] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1101] Many modern users generate large amounts of behavioral and emotional data in their daily lives, but there are only a limited number of systems that effectively utilize this data to provide personalized options. Furthermore, existing systems have difficulty fully analyzing users' emotions and feedback and making highly accurate suggestions based on the user's interests and behavior. Given this background, there is a need for a system that effectively utilizes users' behavioral and emotional data to provide the best options for the user.

[1102] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1103] In this invention, the server includes means for collecting user behavioral data and emotional data, means for transmitting the collected data to a cloud server, means for analyzing the received data, means for analyzing emotions using an emotion recognition engine, means for generating optimal options based on the analysis results, means for notifying the user of the generated options, means for collecting and analyzing user feedback, and means for updating the system based on the feedback, thereby making it possible to provide more personalized options based on the user's behavior and emotions.

[1104] A "user" is an individual or entity that utilizes the system to provide behavioral and emotional data.

[1105] "Behavioral data" is data related to the user's behavior, such as location information and events attended in the user's daily life.

[1106] "Emotion data" is data that indicates the emotions such as joy and sadness felt by the user, and is information extracted from voice memos, photographs, and text.

[1107] A "cloud server" is a remote computer server used to store, manage, and analyze data over the Internet.

[1108] An "emotion recognition engine" is software or hardware for analyzing a user's emotions from data such as voice, photos, and text.

[1109] "Choices" are multiple suggestions for a particular action or decision that are generated based on the user's interests, behavior, and emotions.

[1110] "Feedback" refers to the reactions and opinions that users provide to the system regarding the proposed options.

[1111] "Analysis means" refers to the process or function that analyzes the collected behavioral and emotional data and extracts meaningful information based on the user's interests and behavior.

[1112] "Update means" refers to means for improving the analysis algorithm and the proposal generation process based on user feedback.

[1113] This invention relates to an AI life partner system that collects behavioral and emotional data from users' daily lives, transmits it to a cloud server for analysis, and then provides optimal options to users. This system assists users in making choices based on their interests and behavior, and by combining it with an emotion recognition engine, it realizes more personalized suggestions.

[1114] System configuration

[1115] This system consists of the user's smartphone (device), a cloud-based server (server), and an emotion recognition engine. The device collects the user's behavioral and emotional data and sends it to the server. The server analyzes the received data and generates and notifies the user of the optimal options based on the results. The emotion recognition engine also analyzes emotions from the user's input data and adds them to the behavioral analysis to generate more accurate options.

[1116] Hardware and software used

[1117] 1. Device: A smartphone is used, which allows for the acquisition of location data such as GPS information, automatic collection of SNS activity data, and collection of manually entered humorous text, photos, and voice memos.

[1118] 2. Cloud Server: Use a cloud platform (e.g., Amazon Web Services, Google Cloud Platform, etc.) to store, organize, and analyze data.

[1119] 3. Emotion Recognition Engine: Use software tools to analyze emotions from voice, photos, and text, such as voice recognition software (e.g., Google Speech-to-Text API), image recognition software (e.g., Google Vision API), and natural language processing tools (e.g., TensorFlow, PyTorch, etc.).

[1120] Specific examples

[1121] As a specific example, consider the case where User A takes photos using a smartphone while traveling and records the emotions he or she felt that day as a voice memo. In this case, the device acquires this data along with GPS information and SNS activity history, and sends them to a cloud server. The emotion recognition engine analyzes emotions such as "I was moved" from the voice memo, and the server integrates this data to suggest places that User A might be interested in based on his or her travel experiences.

[1122] Example prompts to input to the generative AI model

[1123] "Please explain the processing details of the AI ​​Life Partner system's program, which analyzes behavioral and emotional data from daily life on a cloud server and suggests optimal options for the user. As a concrete example, please include a scenario in which data recorded during a trip is analyzed to suggest the best tourist spots."

[1124] A system configured in this way can effectively utilize behavioral and emotional data from the user's daily life to provide personalized options.

[1125] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1126] Step 1: Collect data

[1127] User

[1128] Users can manually record important events and moving moments in their daily lives using a dedicated smartphone app. For example, they can take photos while traveling and save the day's emotions as voice memos. This allows detailed data such as time, place, and event to be entered into the device.

[1129] Terminal

[1130] The device automatically acquires the user's GPS information and social media activity data and records their behavioral patterns. For example, data on places visited and events attended is automatically collected. It also stores text, photos, and voice memos manually entered by the user in a local database, and sends this data to an emotion recognition engine. The inputs are the user's location information, photos, text, and voice data, and the device prepares these data for transmission to a cloud server in bulk.

[1131] Step 2: Data transmission and storage

[1132] Terminal

[1133] The device periodically (e.g., at 2:00 AM every day) transmits the collected data to the cloud server. Once an Internet connection is established, the device automatically attempts to transmit data. The input data includes all collected behavioral and emotional data.

[1134] server

[1135] The server organizes and stores data received from the device for each user. For example, user A's travel data and voice memos are stored together in one folder. The input data includes all data sent from the device, and the output data is stored in an organized folder structure. In addition, two-way data communication is performed to verify data integrity, and data is regularly backed up to protect against loss or damage.

[1136] Step 3: Analyze the emotion data

[1137] Emotion Recognition Engine

[1138] The emotion recognition engine analyzes text, photos, and voice memos entered by the user to extract emotions. For example, it extracts emotions such as "happiness" or "sadness" from voice memos and identifies emotions from facial expressions and text included in photos. Input data includes various data provided by the user, and the analyzed emotional data is obtained as output.

[1139] server

[1140] The server sorts the emotion data received from the emotion recognition engine by purpose, and integrates it with behavioral data for analysis. For example, it analyzes the places visited and events attended by user B over the past year, along with the emotion data they felt at the time. The input data includes the emotion data and behavioral data sent from the emotion recognition engine, and the output is the result of the integrated analysis.

[1141] Step 4: Generate and notify options

[1142] server

[1143] Based on the analysis results, options that match the user's interests, behavior, and emotions are generated. For example, a list of optimal educational and employment options is created based on User C's past behavior and emotional data. The input data includes the results of the integrated analysis, and the generated options are formatted as the output.

[1144] Terminal

[1145] The device sends a push notification to the user informing them of the new options. The user can review the proposed options through an interface that allows them to view detailed information. Input data includes the generated options from the server, and the output is a notification to the user.

[1146] Step 5: User feedback

[1147] User

[1148] The user provides feedback on the proposed options, such as "interested" or "not interested," and can also provide additional information. The input data includes the user's feedback and additional information.

[1149] Terminal

[1150] The terminal collects feedback from the user and transmits it to the cloud server. An interface for inputting feedback is provided, and the input data includes the user's feedback, and the output data is transmitted to the cloud server.

[1151] Step 6: Incorporating feedback and updating the system

[1152] server

[1153] The server analyzes the received feedback and updates the proposal algorithm and prediction model. For example, based on user D's feedback, the algorithm is adjusted to improve the accuracy of future option generation. The input data includes the user's feedback, and the output is an updated analysis algorithm and prediction model. The updated analysis results are then reflected in the next data analysis and option generation. This allows the accuracy of suggestions to the user to be continuously improved.

[1154] (Application example 2)

[1155] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1156] Many users today face problems such as indecision and long selection times when choosing products in physical stores. In addition, there is a lack of product recommendations based on personal preferences and emotions, making it difficult to have an efficient and satisfying shopping experience.

[1157] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for collecting behavioral data and emotional data of users connected to a cloud server, a means for generating options that match the user's interests and behavior based on the analysis results, and a means connected to the smart glasses and providing information to the user in real time. This makes it possible to make personalized product recommendations to the user, providing an efficient and satisfying shopping experience.

[1158] "User behavior data" is information about how a user behaves and what they do in a particular situation or environment.

[1159] "Emotion data" is data that expresses the user's emotional state and is extracted from text, audio, images, and the like.

[1160] A "cloud server" is a collection of remote servers accessible via the Internet that provide data processing and storage capabilities.

[1161] "Analysis means" refers to the method or device used to process and analyze collected data and obtain the results.

[1162] The "option generation means" refers to a method or device for generating options to be presented to the user based on the analysis results.

[1163] "Notification means" refers to a method or device for notifying the user of the generated options or information.

[1164] The "feedback collection means" refers to a method or device for collecting feedback such as reactions and evaluations from users.

[1165] "Smart glasses" are eyeglass-like devices that have built-in electronic devices such as a display and camera, and are capable of displaying information and collecting data.

[1166] "Real-time" refers to a time unit where processing and response are nearly instantaneous, with minimal delay.

[1167] This invention is a system that provides users with personalized options by collecting behavioral and emotional data from their daily lives and analyzing them on a cloud server. Here, we describe a specific embodiment that uses smart glasses to enhance the shopping experience in brick-and-mortar stores.

[1168] System configuration

[1169] The system consists of the following major components:

[1170] 1. Smart Glasses

[1171] 2. Cloud Server

[1172] 3. Emotion Recognition Engine

[1173] 4. Shopping recommendation engine

[1174] Collecting user behavioral and emotional data

[1175] Smart glasses are equipped with a camera that can capture images of the user's face and surrounding environment, a microphone that can record audio, and a display that can display information. When a user shops in a physical store, the smart glasses collect data in the following ways:

[1176] A camera is used to periodically capture images of the user's face.

[1177] Use a microphone to record the user's speech and reactions.

[1178] Data transmission and storage

[1179] The smart glasses send the collected behavioral and emotional data to a cloud server, where the data is processed through the following steps:

[1180] The data is periodically uploaded to a cloud server via the Internet.

[1181] The data is organized and stored on a cloud server, and data integrity is confirmed through two-way communication.

[1182] Emotional Data Analysis

[1183] The emotion recognition engine analyzes the user's emotions from the data sent to the cloud server. Specifically, it includes the following processes:

[1184] Facial expressions are analyzed from facial images to identify emotions such as joy, anger, sadness, and happiness.

[1185] Emotions are extracted by analyzing the tone and content of the voice from the audio data.

[1186] Choice generation and notification

[1187] The shopping recommendation engine comprehensively analyzes emotional and behavioral data to recommend the most suitable products and services to users. This process is as follows:

[1188] Based on the results of the emotion recognition engine, the shopping recommendation engine generates the optimal product list.

[1189] Product information and recommendations are displayed in real time on the smart glasses display.

[1190] Feedback and System Updates

[1191] Users can provide feedback through the smart glasses, for example, by rating the recommended products using voice commands or a touch interface. This feedback information is sent to a cloud server and used to improve the accuracy of the analysis and choice generation algorithms.

[1192] Natural language explanation of program processing

[1193] The server includes a means for collecting user behavioral data and emotional data connected to the cloud server, a means for generating options that match the user's interests and behavior based on the analysis results, and a means for connecting to the smart glasses and providing information to the user in real time, thereby enabling personalized product recommendations to be made to the user, providing an efficient and satisfying shopping experience.

[1194] Specific examples

[1195] For example, when a user is browsing products in a physical store, the camera in the smart glasses captures the user's facial expressions, and the emotion recognition engine analyzes the images to determine their interests and satisfaction. The analysis results are sent to a cloud server, and the shopping recommendation engine selects products that match the user's preferences and displays them on the smart glasses' display. The user can provide feedback on the displayed information, and the results will be reflected in the next recommendation algorithm.

[1196] Prompt Sentence Examples

[1197] In developing this application, the prompts input to the generative AI model are as follows:

[1198] We are developing an application for smart glasses that uses a user's facial image data to analyze emotions in real time and, based on the analysis results, recommends optimal products from a shopping recommendation engine. The captured image data is sent to an emotion recognition API on the cloud, and the received emotion data is used to generate personalized product recommendations. The application also includes a function that allows users to provide feedback in real time.

[1199] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1200] Step 1: Collect data

[1201] Subject: Terminal

[1202] The smart glasses use a built-in camera and microphone to collect the user's facial images and voice data in real time. Facial images are captured periodically, and voice data is recorded of the user's speech and reactions. These data are temporarily stored in the device's local memory. The input data are facial images and voice data, and the output is facial image files and voice files stored in the local memory.

[1203] Step 2: Data transmission and storage

[1204] Subject: Terminal / Server

[1205] The collected facial image and voice data is sent to a cloud server via the Internet. The data is periodically batch processed and securely stored on the cloud server. During transmission, a checksum is generated to verify the integrity of the data, and two-way communication is performed. The input data is facial image and voice data from the device, and the output is a data file stored on the cloud server.

[1206] Step 3: Analyze the emotion data

[1207] Subject: Server

[1208] The cloud server sends the collected facial images and voice data to an emotion recognition engine. The emotion recognition engine analyzes facial expressions from images and extracts emotions from voice. This process uses a generative AI model. For example, image analysis identifies emotions such as "surprise" or "anger," while voice analysis extracts emotions from the tone and content of voice. The input data are the facial images and voice data stored on the cloud server, and the output is emotion data from the emotion recognition engine.

[1209] Step 4: Generate options

[1210] Subject: Server

[1211] Based on emotional and behavioral data, the shopping recommendation engine recommends the most suitable products and services to the user. Based on the analysis results, a product list that matches the user's interests is generated. The generated recommendation list is displayed in real time on the smart glasses display. The input data is emotional data from the emotion recognition engine, and the output is a personalized product recommendation list.

[1212] Step 5: Notify users

[1213] Subject: Terminal

[1214] The smart glasses display displays recommended product information sent from the server in real time. For example, information such as "Recommended Product: Product X - Price: 1000 yen" is displayed in the user's field of view. The input data is the recommended product list sent from the server, and the output is the information displayed on the display screen of the smart glasses.

[1215] Step 6: Gather feedback

[1216] Subject: Terminal / User

[1217] The user provides feedback on the recommended products through the smart glasses using voice commands or a touchpad. For example, the user inputs a voice command such as "I like this product." The device then sends this feedback data to the cloud server. The input data is the user's feedback comment, and the output is the feedback data sent to the cloud server.

[1218] Step 7: Analyze feedback and update the system

[1219] Subject: Server

[1220] The cloud server analyzes the collected feedback data and updates the algorithms of the analysis means and option generation means. For example, it may increase the frequency of recommending a particular product or modify the algorithm to recommend a different product. This feedback analysis improves the accuracy of the next recommendation. The input data is the feedback data from the user, and the output is the updated algorithm.

[1221] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1222] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1223] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1224] [Fourth embodiment]

[1225] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1226] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1227] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1228] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1229] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1230] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1231] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1232] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1233] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1234] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1235] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1236] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1237] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1238] This invention relates to an AI life partner system that collects behavioral data from users' daily lives, transmits it to a cloud server, and analyzes it to provide users with optimal choices. This system helps users make confident choices based on their own interests and behaviors. The specific program processing is described below.

[1239] System configuration

[1240] This system consists of the user's smartphone (device) and a cloud server (server). The device collects the user's behavioral data and sends it to the server. The server analyzes the received data and generates and notifies the user of the optimal options based on the results.

[1241] Program processing

[1242] 1. Data collection

[1243] User

[1244] Users can manually record important events and moving moments in their daily lives using smartphone apps. For example, they can take photos while traveling or record their impressions as voice memos.

[1245] Users can also record detailed events and emotions by entering text as needed.

[1246] Terminal

[1247] The device automatically collects the user's GPS information and activity data from social media, and records their behavioral patterns, such as the places they visited and the events they attended.

[1248] The device also stores manually entered text, photos, and voice memos in its database.

[1249] 2. Data transmission and storage

[1250] Terminal

[1251] The device periodically sends the collected data to a cloud server. For example, the device uploads the day's data to the server at 2:00 AM every day.

[1252] Attempts to send data as soon as an internet connection is established.

[1253] server

[1254] The server organizes and stores the data received from the device for each user. For example, it stores User A's travel data and voice memos together.

[1255] The server performs two-way data communication to verify data integrity.

[1256] Regularly back up your data to protect it from loss or corruption.

[1257] 3. Data Analysis

[1258] server

[1259] The server analyzes the stored data set to extract user behavior patterns, hobbies, and interests. For example, it identifies User B's interests based on the places he frequently visited and the events he attended over the past year.

[1260] Run machine learning algorithms to generate predictive models based on the data, then compare them with historical data to improve the accuracy of the analysis.

[1261] 4. Choice generation and notification

[1262] server

[1263] Based on the analysis results, options (occupations, schools, hobbies, etc.) that match the user's interests and behavior are generated. For example, based on User C's interest in chemistry, a list of appropriate schools and employment options is created.

[1264] Sends the formatted selection to the terminal.

[1265] Terminal

[1266] The device notifies the user of the suggested options and displays more information, for example, by using a push notification to let them know that the options have arrived.

[1267] The user checks the options notified and views the detailed information.

[1268] 5. User Feedback

[1269] User

[1270] The user provides feedback on the suggested options, for example by providing responses such as "interested" or "not interested."

[1271] Any further details or additional information can also be provided.

[1272] Terminal

[1273] The terminal collects user feedback and sends it to the server, and provides an interface for feedback input.

[1274] server

[1275] The server analyzes the received feedback and updates the proposal algorithm. The feedback data is analyzed and reflected in the next proposal.

[1276] This ensures that users always receive the best options based on their interests and behavior, giving them confidence and peace of mind when making important life decisions.

[1277] The processing flow will be explained below.

[1278] Step 1: Collect data

[1279] Users can launch the app on their smartphones and manually record important events or moving moments, for example, by taking photos while traveling or recording voice memos about how they felt that day.

[1280] The device automatically collects the user's GPS information and activity data from social networking services, and records their behavioral patterns, such as the places they visit and the events they attend.

[1281] The device stores manually entered text, photos, and voice memos in a local database.

[1282] Step 2: Data transmission and storage

[1283] The device periodically (e.g., at 2:00 AM every day) sends the collected data to the cloud server. When an Internet connection is established, the device automatically attempts to send the data.

[1284] The server organizes and stores the data received from the device for each user. For example, user A's travel data and voice memos are stored together in one folder.

[1285] The server communicates data in two directions to ensure data integrity, and regularly backs up data to protect it from loss or corruption.

[1286] Step 3: Analyze the data

[1287] The server runs machine learning algorithms to analyze the stored datasets, extracting user behavioral patterns, hobbies, and interests from the collected behavioral data.

[1288] The server compares the results with past data, generates predictive models, and keeps the analysis results up to date. For example, it identifies User B's interests based on the places he visited and events he attended over the past year.

[1289] Step 4: Generate and notify options

[1290] Based on the analysis results, the server generates options (occupations, schools, hobbies, etc.) that match the user's interests and behavior. For example, based on User C's interest in chemistry, it creates a list of suitable schools and employment options.

[1291] The server formats the generated options and transmits them to the terminal.

[1292] The device will send a push notification to the user to let them know that the suggested options have arrived, and will provide an interface for the user to view more information.

[1293] Step 5: User feedback

[1294] The user provides feedback on the suggested options, for example by responding to the options as "interested" or "not interested."

[1295] The device collects feedback from users and sends it to a cloud server, and also provides an interface for feedback input.

[1296] Step 6: Incorporating feedback and updating the system

[1297] The server analyzes the received feedback and updates the proposed algorithm and predictive model. For example, based on User D's feedback, the server adjusts the algorithm to improve the accuracy of future choice generation.

[1298] The server reflects the updated analysis results in the next data analysis and option generation, thereby continuously improving the accuracy of suggestions to users.

[1299] Example 1

[1300] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1301] In recent years, there has been an increasing demand for systems that utilize user behavioral and interest data to make more personalized and effective recommendations. However, conventional systems have faced many technical challenges in efficiently collecting diverse data from users' daily lives and analyzing it using machine learning algorithms. In particular, it has been difficult to verify the completeness of the data, ensure the consistency of the collected data, and appropriately incorporate user feedback. This has made it difficult to provide highly accurate options that match users' behavior and interests.

[1302] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1303] In this invention, the server includes: means for collecting user behavioral data; means for transmitting the collected behavioral data to a cloud server; means for analyzing the behavioral data transmitted to the cloud server; means for generating options that match the user's interests and behavior based on the analysis results; means for notifying the user of the generated options; means for collecting and analyzing feedback from the user; means for updating the analysis means and the option generation means based on the feedback; a mobile information terminal equipped with an application for collecting behavioral data in the user's daily life; means for storing the collected behavioral data on the cloud and verifying its integrity; means for analyzing the user's interests by executing a machine learning algorithm based on the stored data; means for creating suggestions using a generative AI model based on the results; and means for transmitting the generated suggestions to the terminal and notifying the user. This enables the generation and notification of highly accurate options that match the user's behavior and interests while ensuring the integrity of the collected data.

[1304] "Behavioral data" refers to information about a user's daily activities, such as their location, the applications they use, their social media activity, the places they visit, and the events they attend.

[1305] A "cloud server" refers to a server system that stores, analyzes, and processes data in a data center located in a remote location via the Internet.

[1306] "Machine learning algorithms" refer to computer programs and mathematical models that automatically learn patterns and knowledge from data.

[1307] A "generative AI model" refers to a model that uses artificial intelligence technology to create suggestions or products tailored to specific purposes.

[1308] "Mobile information terminal" refers to an electronic device that can be carried by a user, such as a smartphone or tablet.

[1309] "Application" refers to software that runs on a mobile information terminal and provides specific functions.

[1310] "Feedback" refers to the user's thoughts, evaluations, additional information, etc., entered in response to suggestions received from the system.

[1311] "Push notification" refers to a mechanism for sending information from a server to a user's device in real time.

[1312] "Data integrity" refers to the state of ensuring that data is accurate and free of omissions and errors.

[1313] This invention relates to an AI life partner system that collects behavioral data from users' daily lives, transmits it to a cloud server, and analyzes it to provide optimal options to users. This system proposes advanced options based on the user's interests and behavior, helping the user make choices with confidence.

[1314] System configuration

[1315] This system consists of a user's mobile information device (e.g., a smartphone) and a cloud server. The mobile information device collects the user's behavioral data and sends it to the cloud server. The cloud server analyzes the received data and, based on the results, generates and notifies the user of the optimal options.

[1316] Hardware and software used

[1317] Hardware:

[1318] Mobile devices: Users' smartphones, tablets, etc.

[1319] Cloud server: A remote data center that stores, analyzes, and processes data over the internet.

[1320] software:

[1321] Database Management System (DBMS): A data storage system such as SQLite or MySQL.

[1322] Machine learning libraries: Algorithms such as TensorFlow, scikit-learn, etc.

[1323] Communication protocol: Uses HTTP to send and receive data through cloud API.

[1324] Push notification service: Firebase Cloud Messaging (FCM), etc.

[1325] Program processing

[1326] Each process of the program of this system will be explained in detail below.

[1327] Data collection

[1328] Users can manually record important events and moving moments in their daily lives using a smartphone app. For example, they can save photos taken while traveling or their thoughts as voice memos. They can also record detailed events and emotions by entering text as needed.

[1329] The device automatically acquires the user's GPS information and activity data from social networking services (SNS), records their behavioral patterns, such as the places they visit and the events they attend, and also stores manually entered text, photos, and voice memos in a database.

[1330] Data transmission and storage

[1331] The device periodically sends collected data to the cloud server every day. For example, it uploads the day's data to the server at 2:00 a.m. Data is sent using the HTTP protocol, and if the transmission fails, it retries.

[1332] The server organizes and stores data received from the device for each user. For example, it stores User A's travel data and voice memos in cloud storage. It also compares the data with the original data to ensure data integrity. It also regularly backs up the data.

[1333] Data analysis

[1334] The server analyzes the stored data using machine learning algorithms. For example, it analyzes data on places that User B frequently visited and events that he or she attended over the past year to identify the user's interests. The analysis uses cloud computing resources and machine learning libraries such as TensorFlow and scikit-learn.

[1335] Choice generation and notification

[1336] The server generates options that match the user based on the analysis results. For example, it lists future schools and careers based on User C's interest in science. It uses a generative AI model to create prompts and configure the options. The generated options are then sent to the device in JSON format and notified.

[1337] The device notifies the user of the choices sent by the server. Push notifications are used to let the user know that new suggestions have arrived. If the user responds to the notification, more information can be displayed in the app.

[1338] User Feedback

[1339] The user inputs feedback on the proposed options, including impressions such as "interested" or "not interested," as well as new hopes and requests.

[1340] The device collects the feedback input by the user and transmits it to the cloud server, and provides an interface for feedback input.

[1341] The server analyzes the received feedback and reflects it in the next proposal, thereby updating the analysis means and option generation means, so that the next proposal will be more useful to the user.

[1342] Specific examples

[1343] For example, if a user records photos and voice memos taken during a trip in a smartphone app, the following process will occur.

[1344] Example prompt sentence:

[1345] Photos and voice memos taken during a user's trip are uploaded to a cloud server. The system analyzes the user's interests based on travel data from the past year and suggests new travel destinations and events to participate in.

[1346] Based on this prompt, the server collects user behavior data and sends, analyzes, and generates options to suggest suitable travel destinations and events to the user. In this way, the system can continually provide the user with the best options.

[1347] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1348] Step 1: Collect data

[1349] Users can manually record important events and moving moments in their daily lives using a smartphone app. For example, users can record photos taken while traveling and their feelings as voice memos. They can also record detailed events and emotions by entering text as needed.

[1350] The device automatically acquires the user's GPS information and social media activity data. For example, it collects data on the places the user has visited and the events they have attended. This data is stored in a database within the device. Inputs include photos, voice memos, text, GPS information, and social media data, and by storing these in the internal database, a database of behavioral patterns is generated.

[1351] Step 2: Data transmission and storage

[1352] The device periodically sends collected data to the cloud server every day. For example, the day's data is uploaded to the server at 2:00 AM every day. If the internet connection is unstable, the device attempts to send the data the moment a connection is established. This is the process by which data is collected and sent from the device to the server. The input is saved user behavior data (photos, voice memos, text, GPS information, SNS data), which is sent to the cloud server via the HTTP protocol.

[1353] The server organizes the data received from the device for each user and stores it in cloud storage. For example, it stores User A's travel data and voice memos using a specific folder structure. To ensure data integrity, the server compares the received data with the original data. In addition, it regularly backs up the data to prevent data loss.

[1354] Step 3: Analyze the data

[1355] The server analyzes the stored data using machine learning algorithms. For example, it can identify user interests based on the places User B frequently visited and events he or she participated in over the past year. The input is the user's behavioral data stored in the cloud. This is input to a machine learning algorithm (TensorFlow or scikit-learn) and analyzed to obtain results on the user's interests and behavioral patterns. This analysis process uses high-performance computing resources on the cloud.

[1356] Step 4: Generate and notify options

[1357] The server generates options that match the user based on the analysis results. For example, it lists suitable educational and career options based on User C's interest in science. The input is the analysis results, and a generative AI model (e.g., GPT-4) is used to create a new prompt. This process generates appropriate option suggestions for the user.

[1358] The device notifies the user of the choices sent by the server. A push notification is used to let the user know that a new suggestion has arrived. When the user interacts with the notification, more information can be displayed within the app, for example, details on science studies or career opportunities. The input to the prompt is the generated suggestion, and the output is a notification to the user and a display of the more information.

[1359] Step 5: User feedback

[1360] The user inputs feedback on the proposed options, including their impressions such as "interested" or "not interested," as well as new hopes and requests. The input is text data of the feedback, which is sent to the next processing step.

[1361] The device collects the feedback entered by the user and transmits it back to the cloud server, which then receives the feedback data and performs analysis.

[1362] The server analyzes the received feedback and reflects it in the next proposal. The input is the user's feedback data, which is analyzed to update the proposed algorithm. The feedback data is also used as training data for the next round. The output is an improved proposed algorithm and the next proposal.

[1363] (Application example 1)

[1364] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1365] While conventional behavioral data collection systems can collect and analyze user data, they have a problem in that they cannot use the results to suggest optimal products to users in real time. In particular, to suggest products that are in line with a user's interests and behavior, rapid and highly accurate data analysis is required, but no system with this function currently exists.

[1366] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1367] In this invention, the server includes means for collecting user behavioral data, means for transmitting the collected behavioral data to a cloud server, and means for analyzing the behavioral data transmitted to the cloud server, thereby enabling the server to utilize a generative AI model to suggest optimal products in accordance with prompt sentences based on the user's preferences.

[1368] "Behavioral data" refers to information that indicates a user's movement history, activities, and interests in their daily lives.

[1369] "Cloud server" refers to a remote server for storing, analyzing, and managing data via the Internet.

[1370] "Analysis means" refers to a system or device for analyzing collected data and extracting user behavior patterns and preferences.

[1371] "Option generation means" refers to a function for generating optimal suggestions and recommendations for users based on analyzed data.

[1372] "Notification means" refers to a function or mechanism for notifying the user of generated options or suggestions.

[1373] "Feedback" refers to data entered by a user in response to or comment on the suggestions or options provided.

[1374] "Smart devices" refers to portable electronic devices with advanced functions such as smartphones, smart glasses, and head-mounted displays.

[1375] A "generative AI model" refers to a model that uses artificial intelligence to analyze data and generate suggestions and recommendations that are appropriate for the user.

[1376] A "prompt" refers to an instruction or input used by a generative AI model to make a suggestion.

[1377] This invention relates to an AI life partner system that collects user behavioral data, transmits it to a cloud server, and analyzes it to provide users with optimal choices. This system proposes products that best suit the user's interests and preferences based on the user's behavioral data in their daily lives.

[1378] System configuration

[1379] This system mainly consists of the following hardware and software:

[1380] Hardware: Smart devices such as smartphones, smart glasses, and head-mounted displays

[1381] Software: Python, machine learning libraries (such as Scikit-learn), internet connection

[1382] The terminal (smart device) collects user behavior data and sends it to a cloud server. The cloud server analyzes the received data and generates and notifies the user of the best options based on the results. This allows users to receive product suggestions that match their interests in real time.

[1383] Program processing

[1384] Data collection

[1385] User

[1386] Users use smart devices to record their activities as they go about their daily lives. The system automatically collects location and activity data from GPS and social media. Users can also record photos and voice memos of their travels.

[1387] Terminal

[1388] The device automatically collects this data and stores it in a database, including GPS information, places visited, events attended, and manually entered text, photos, and voice notes.

[1389] Data transmission and analysis

[1390] Terminal

[1391] The device periodically sends collected data to the cloud server, and attempts to send the data when an Internet connection is established.

[1392] server

[1393] The server organizes and stores the data received from the device for each user, and performs two-way data communication to verify the integrity of the data. It then analyzes the data using machine learning algorithms (e.g., KMeans clustering) to extract user behavioral patterns and preferences. Specifically, it analyzes the places the user visited, the events they attended, and the content they posted to identify the user's interests.

[1394] Product suggestions and notifications

[1395] server

[1396] Based on the analysis results, a generative AI model is used to generate optimal options (product suggestions) for the user, such as fashion items, books, travel destinations, etc.

[1397] Terminal

[1398] The device notifies the user of the generated options and displays detailed information. For example, the device can use the push notification function on a smartphone or smart glasses to notify the user of suggested product information in real time.

[1399] Collecting and analyzing feedback

[1400] User

[1401] The user provides feedback on the proposed options, for example by providing responses such as "interested" or "not interested."

[1402] Terminal

[1403] The terminal collects user feedback and sends it to the server.

[1404] server

[1405] The server analyzes the received feedback and updates its proposal algorithm, which improves the accuracy of the next proposal.

[1406] Specific examples

[1407] For example, if the user's behavioral data is recorded as "visiting a fashion shop in Shibuya and looking for a new jacket," the system can analyze that data to identify the user's preferences and suggest the most suitable jacket. In this case, the system uses a generative AI model to generate prompt sentences.

[1408] Example prompt sentence:

[1409] User visited: Shibuya, Tokyo

[1410] Activities: Fashion shop visits

[1411] Data collected: GPS data, locations visited, activities, text input ("Looking for a new jacket")

[1412] Analysis-based suggestions: Recommending the perfect jacket for the user

[1413] In this way, users can receive optimal product suggestions in real time based on their interests and behavior.

[1414] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1415] Step 1:

[1416] The device collects data on the user's daily life activities. Specifically, it stores GPS data, SNS activity data, and text, photos, and voice memos manually entered by the user in a database. The input is the user's behavioral information data, and the output is the behavioral data stored in the device's database.

[1417] Step 2:

[1418] The device sends the collected behavioral data to a cloud server. The data is uploaded the moment an internet connection is established or at a specific time (for example, every night at 2:00 AM). The input is the behavioral data stored on the device, and the output is the data sent to the cloud server.

[1419] Step 3:

[1420] The server organizes and stores the received data for each user. The server performs two-way data communication and checks the integrity of the data. The input is the data received from the terminal, and the output is the organized and stored data.

[1421] Step 4:

[1422] The server analyzes the stored dataset to extract user behavioral patterns, hobbies, and interests. Specifically, it runs machine learning algorithms (e.g., KMeans clustering) and generates predictive models based on the data. The input is an organized dataset, and the output is an analysis of the user's interests and preferences.

[1423] Step 5:

[1424] The server uses a generative AI model based on the analysis results to generate optimal options (e.g., product suggestions) for the user. It uses prompt sentences to list products that match the user's interests. The input is the analysis results and associated prompt sentences, and the output is the generated product options.

[1425] Step 6:

[1426] The terminal notifies the user of the generated options. Using the push notification function, suggested product information is communicated to the user in real time. The input is the option data from the server, and the output is a notification to the user.

[1427] Step 7:

[1428] The user provides feedback on the proposed options, for example, by inputting a response such as "interested" or "not interested." The input is the user's feedback, and the output is the feedback data sent to the terminal.

[1429] Step 8:

[1430] The terminal collects user feedback and sends it to the cloud server. The input is the user feedback data, and the output is the data sent to the cloud server.

[1431] Step 9:

[1432] The server analyzes the received feedback and updates the proposal algorithm, improving the accuracy of the next proposal. The input is the feedback data, and the output is an updated predictive model.

[1433] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1434] This invention relates to an AI life partner system that collects behavioral and emotional data from users' daily lives, transmits it to a cloud server for analysis, and then provides optimal options to users. This system helps users make confident choices based on their interests and behavior, and by combining it with an emotion recognition engine, it realizes more personalized suggestions.

[1435] System configuration

[1436] This system consists of the user's smartphone (device), a cloud-based server (server), and an emotion recognition engine. The device collects the user's behavioral and emotional data and sends it to the server. The server analyzes the received data and generates and notifies the user of the optimal options based on the results. The emotion recognition engine also analyzes emotions from the user's input data and adds them to the behavioral analysis to generate more accurate options.

[1437] Program processing

[1438] 1. Data collection

[1439] User

[1440] Users can manually record important events and moving moments in their daily lives using smartphone apps, for example, by taking photos while traveling and recording voice memos about their feelings that day.

[1441] If desired, the user can enter text to record detailed events and emotions.

[1442] Terminal

[1443] The device automatically collects the user's GPS information and activity data from social media, and records their behavioral patterns, such as the places they visited and the events they attended.

[1444] It also stores manually entered text, photos, and voice notes in a local database and sends this data to an emotion recognition engine.

[1445] 2. Data transmission and storage

[1446] Terminal

[1447] The device periodically (e.g., at 2:00 AM every day) sends the collected data to the cloud server. When an Internet connection is established, the device automatically attempts to send the data.

[1448] server

[1449] The server organizes and stores the data received from the device for each user. For example, user A's travel data and voice memos are stored together in one folder.

[1450] Two-way data communication is conducted to verify data integrity, and data is backed up regularly to protect it from loss or corruption.

[1451] 3. Emotion Data Analysis

[1452] Emotion Recognition Engine

[1453] The emotion recognition engine extracts emotions from user-entered text, photos, and voice memos. For example, it analyzes voice to identify emotions such as "happiness" or "sadness," and identifies emotions from facial expressions and text in photos.

[1454] server

[1455] The server sorts the emotion data received from the emotion recognition engine by purpose and integrates it with behavioral data for analysis. For example, it comprehensively analyzes the places visited and events attended by User B over the past year, as well as the emotions they felt at the time.

[1456] 4. Choice generation and notification

[1457] server

[1458] Based on the analysis results, options (occupations, schools, hobbies, etc.) that match the user's interests, behavior, and emotions are generated. For example, a list of optimal schools and employment options is created based on User C's past behavior and emotional data.

[1459] The generated options are formatted and sent to the terminal.

[1460] Terminal

[1461] The system sends a push notification to the user to let them know when the suggested options arrive, and provides an interface for the user to view more information.

[1462] 5. User Feedback

[1463] User

[1464] Enter feedback on the suggested options, for example, providing responses such as "interested" or "not interested."

[1465] Any further details or additional information can also be provided.

[1466] Terminal

[1467] Collects feedback from users and sends it to the cloud server. Provides an interface for feedback input.

[1468] 6. Reflecting feedback and updating the system

[1469] server

[1470] Analyze the received feedback and update the proposed algorithm and predictive model. For example, adjust the algorithm based on User D's feedback to improve the accuracy of future option generation.

[1471] The updated analysis results are reflected in the next data analysis and option generation, thereby continuously improving the accuracy of suggestions to users.

[1472] As a result, users can receive more personalized options based on detailed behavioral data, including emotional data. This system provides powerful support for users to make confident choices based on their interests, behaviors, and emotions.

[1473] The processing flow will be explained below.

[1474] Step 1: Collect data

[1475] Users can launch the app on their smartphones and manually record important events or moving moments, for example, by taking photos while traveling and recording voice memos about their feelings that day.

[1476] Users can optionally enter text to record detailed events and emotions.

[1477] The device automatically collects the user's GPS information and activity data from social media, and records their behavioral patterns, such as the places they visited and the events they attended.

[1478] The device stores manually entered text, photos, and voice notes in a local database and sends this data to an emotion recognition engine.

[1479] Step 2: Data transmission and storage

[1480] The device periodically (e.g., at 2:00 AM every day) sends the collected data to the cloud server. When an Internet connection is established, the device automatically attempts to send the data.

[1481] The server organizes and stores the data received from the device for each user. For example, user A's travel data and voice memos are stored together in one folder.

[1482] The server communicates with the data in two directions to verify data integrity, and regularly backs up the data to protect it from loss or corruption.

[1483] Step 3: Analyze the emotion data

[1484] The emotion recognition engine extracts emotions from user-entered text, photos, and voice memos. For example, it analyzes voice to identify emotions such as "happiness" or "sadness," and identifies emotions from facial expressions and text in photos.

[1485] The server sorts the emotion data received from the emotion recognition engine by purpose and integrates it with behavioral data for analysis. For example, it comprehensively analyzes the places visited and events attended by User B over the past year, as well as the emotions they felt at the time.

[1486] Step 4: Analyze behavioral data and generate options

[1487] The server runs machine learning algorithms to analyze the stored behavioral datasets, extracting user behavior patterns, hobbies, and interests from the collected behavioral data.

[1488] By integrating the emotional data, the server can obtain more personalized analysis results. For example, by incorporating the emotional data of User C at a specific event, the server can refine future event suggestions.

[1489] Based on the analysis results, options (occupations, schools, hobbies, etc.) that match the user's interests, behavior, and emotions are generated. For example, a list of optimal schools and employment options is created based on User D's past behavior and emotional data.

[1490] Step 5: Communicate your options

[1491] The server formats the generated options and transmits them to the terminal.

[1492] The device will send a push notification to the user to let them know that a suggested option has arrived, and will also provide an interface for the user to view more information.

[1493] Step 6: User feedback

[1494] The user provides feedback on the suggested options, for example by providing responses such as "interested" or "not interested."

[1495] The user can also provide further details or additional information.

[1496] The device collects feedback from users and sends it to the cloud server, and provides an interface for feedback input.

[1497] Step 7: Incorporating feedback and updating the system

[1498] The server analyzes the received feedback and updates the proposal algorithm and predictive model. For example, based on user E's feedback, the server adjusts the algorithm to improve the accuracy of future choice generation.

[1499] The server reflects the updated analysis results in the next data analysis and option generation, thereby continuously improving the accuracy of suggestions to users.

[1500] As a specific example, by suggesting other suitable universities based on emotion data to the university that User F has given feedback that he or she is "interested in," the user's interest may increase. This can reduce anxiety and regret regarding important life decisions.

[1501] Example 2

[1502] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1503] Many modern users generate large amounts of behavioral and emotional data in their daily lives, but there are only a limited number of systems that effectively utilize this data to provide personalized options. Furthermore, existing systems have difficulty fully analyzing users' emotions and feedback and making highly accurate suggestions based on the user's interests and behavior. Given this background, there is a need for a system that effectively utilizes users' behavioral and emotional data to provide the best options for the user.

[1504] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1505] In this invention, the server includes means for collecting user behavioral data and emotional data, means for transmitting the collected data to a cloud server, means for analyzing the received data, means for analyzing emotions using an emotion recognition engine, means for generating optimal options based on the analysis results, means for notifying the user of the generated options, means for collecting and analyzing user feedback, and means for updating the system based on the feedback, thereby making it possible to provide more personalized options based on the user's behavior and emotions.

[1506] A "user" is an individual or entity that utilizes the system to provide behavioral and emotional data.

[1507] "Behavioral data" is data related to the user's behavior, such as location information and events attended in the user's daily life.

[1508] "Emotion data" is data that indicates the emotions such as joy and sadness felt by the user, and is information extracted from voice memos, photographs, and text.

[1509] A "cloud server" is a remote computer server used to store, manage, and analyze data over the Internet.

[1510] An "emotion recognition engine" is software or hardware for analyzing a user's emotions from data such as voice, photos, and text.

[1511] "Choices" are multiple suggestions for a particular action or decision that are generated based on the user's interests, behavior, and emotions.

[1512] "Feedback" refers to the reactions and opinions that users provide to the system regarding the proposed options.

[1513] "Analysis means" refers to the process or function that analyzes the collected behavioral and emotional data and extracts meaningful information based on the user's interests and behavior.

[1514] "Update means" refers to means for improving the analysis algorithm and the proposal generation process based on user feedback.

[1515] This invention relates to an AI life partner system that collects behavioral and emotional data from users' daily lives, transmits it to a cloud server for analysis, and then provides optimal options to users. This system assists users in making choices based on their interests and behavior, and by combining it with an emotion recognition engine, it realizes more personalized suggestions.

[1516] System configuration

[1517] This system consists of the user's smartphone (device), a cloud-based server (server), and an emotion recognition engine. The device collects the user's behavioral and emotional data and sends it to the server. The server analyzes the received data and generates and notifies the user of the optimal options based on the results. The emotion recognition engine also analyzes emotions from the user's input data and adds them to the behavioral analysis to generate more accurate options.

[1518] Hardware and software used

[1519] 1. Device: A smartphone is used, which allows for the acquisition of location data such as GPS information, automatic collection of SNS activity data, and collection of manually entered humorous text, photos, and voice memos.

[1520] 2. Cloud Server: Use a cloud platform (e.g., Amazon Web Services, Google Cloud Platform, etc.) to store, organize, and analyze data.

[1521] 3. Emotion Recognition Engine: Use software tools to analyze emotions from voice, photos, and text, such as voice recognition software (e.g., Google Speech-to-Text API), image recognition software (e.g., Google Vision API), and natural language processing tools (e.g., TensorFlow, PyTorch, etc.).

[1522] Specific examples

[1523] As a specific example, consider the case where User A takes photos using a smartphone while traveling and records the emotions he or she felt that day as a voice memo. In this case, the device acquires this data along with GPS information and SNS activity history, and sends them to a cloud server. The emotion recognition engine analyzes emotions such as "I was moved" from the voice memo, and the server integrates this data to suggest places that User A might be interested in based on his or her travel experiences.

[1524] Example prompts to input to the generative AI model

[1525] "Please explain the processing details of the AI ​​Life Partner system's program, which analyzes behavioral and emotional data from daily life on a cloud server and suggests optimal options for the user. As a concrete example, please include a scenario in which data recorded during a trip is analyzed to suggest the best tourist spots."

[1526] A system configured in this way can effectively utilize behavioral and emotional data from the user's daily life to provide personalized options.

[1527] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1528] Step 1: Collect data

[1529] User

[1530] Users can manually record important events and moving moments in their daily lives using a dedicated smartphone app. For example, they can take photos while traveling and save the day's emotions as voice memos. This allows detailed data such as time, place, and event to be entered into the device.

[1531] Terminal

[1532] The device automatically acquires the user's GPS information and social media activity data and records their behavioral patterns. For example, data on places visited and events attended is automatically collected. It also stores text, photos, and voice memos manually entered by the user in a local database, and sends this data to an emotion recognition engine. The inputs are the user's location information, photos, text, and voice data, and the device prepares these data for transmission to a cloud server in bulk.

[1533] Step 2: Data transmission and storage

[1534] Terminal

[1535] The device periodically (e.g., at 2:00 AM every day) transmits the collected data to the cloud server. Once an Internet connection is established, the device automatically attempts to transmit data. The input data includes all collected behavioral and emotional data.

[1536] server

[1537] The server organizes and stores data received from the device for each user. For example, user A's travel data and voice memos are stored together in one folder. The input data includes all data sent from the device, and the output data is stored in an organized folder structure. In addition, two-way data communication is performed to verify data integrity, and data is regularly backed up to protect against loss or damage.

[1538] Step 3: Analyze the emotion data

[1539] Emotion Recognition Engine

[1540] The emotion recognition engine analyzes text, photos, and voice memos entered by the user to extract emotions. For example, it extracts emotions such as "happiness" or "sadness" from voice memos and identifies emotions from facial expressions and text included in photos. Input data includes various data provided by the user, and the analyzed emotional data is obtained as output.

[1541] server

[1542] The server sorts the emotion data received from the emotion recognition engine by purpose, and integrates it with behavioral data for analysis. For example, it analyzes the places visited and events attended by user B over the past year, along with the emotion data they felt at the time. The input data includes the emotion data and behavioral data sent from the emotion recognition engine, and the output is the result of the integrated analysis.

[1543] Step 4: Generate and notify options

[1544] server

[1545] Based on the analysis results, options that match the user's interests, behavior, and emotions are generated. For example, a list of optimal educational and employment options is created based on User C's past behavior and emotional data. The input data includes the results of the integrated analysis, and the generated options are formatted as the output.

[1546] Terminal

[1547] The device sends a push notification to the user informing them of the new options. The user can review the proposed options through an interface that allows them to view detailed information. Input data includes the generated options from the server, and the output is a notification to the user.

[1548] Step 5: User feedback

[1549] User

[1550] The user provides feedback on the proposed options, such as "interested" or "not interested," and can also provide additional information. The input data includes the user's feedback and additional information.

[1551] Terminal

[1552] The terminal collects feedback from the user and transmits it to the cloud server. An interface for inputting feedback is provided, and the input data includes the user's feedback, and the output data is transmitted to the cloud server.

[1553] Step 6: Incorporating feedback and updating the system

[1554] server

[1555] The server analyzes the received feedback and updates the proposal algorithm and prediction model. For example, based on user D's feedback, the algorithm is adjusted to improve the accuracy of future option generation. The input data includes the user's feedback, and the output is an updated analysis algorithm and prediction model. The updated analysis results are then reflected in the next data analysis and option generation. This allows the accuracy of suggestions to the user to be continuously improved.

[1556] (Application example 2)

[1557] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1558] Many users today face problems such as indecision and long selection times when choosing products in physical stores. In addition, there is a lack of product recommendations based on personal preferences and emotions, making it difficult to have an efficient and satisfying shopping experience.

[1559] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for collecting behavioral data and emotional data of users connected to a cloud server, a means for generating options that match the user's interests and behavior based on the analysis results, and a means connected to the smart glasses and providing information to the user in real time. This makes it possible to make personalized product recommendations to the user, providing an efficient and satisfying shopping experience.

[1560] "User behavior data" is information about how a user behaves and what they do in a particular situation or environment.

[1561] "Emotion data" is data that expresses the user's emotional state and is extracted from text, audio, images, and the like.

[1562] A "cloud server" is a collection of remote servers accessible via the Internet that provide data processing and storage capabilities.

[1563] "Analysis means" refers to the method or device used to process and analyze collected data and obtain the results.

[1564] The "option generation means" refers to a method or device for generating options to be presented to the user based on the analysis results.

[1565] "Notification means" refers to a method or device for notifying the user of the generated options or information.

[1566] The "feedback collection means" refers to a method or device for collecting feedback such as reactions and evaluations from users.

[1567] "Smart glasses" are eyeglass-like devices that have built-in electronic devices such as a display and camera, and are capable of displaying information and collecting data.

[1568] "Real-time" refers to a time unit where processing and response are nearly instantaneous, with minimal delay.

[1569] This invention is a system that provides users with personalized options by collecting behavioral and emotional data from their daily lives and analyzing them on a cloud server. Here, we describe a specific embodiment that uses smart glasses to enhance the shopping experience in brick-and-mortar stores.

[1570] System configuration

[1571] The system consists of the following major components:

[1572] 1. Smart Glasses

[1573] 2. Cloud Server

[1574] 3. Emotion Recognition Engine

[1575] 4. Shopping recommendation engine

[1576] Collecting user behavioral and emotional data

[1577] Smart glasses are equipped with a camera that can capture images of the user's face and surrounding environment, a microphone that can record audio, and a display that can display information. When a user shops in a physical store, the smart glasses collect data in the following ways:

[1578] A camera is used to periodically capture images of the user's face.

[1579] Use a microphone to record the user's speech and reactions.

[1580] Data transmission and storage

[1581] The smart glasses send the collected behavioral and emotional data to a cloud server, where the data is processed through the following steps:

[1582] The data is periodically uploaded to a cloud server via the Internet.

[1583] The data is organized and stored on a cloud server, and data integrity is confirmed through two-way communication.

[1584] Emotional Data Analysis

[1585] The emotion recognition engine analyzes the user's emotions from the data sent to the cloud server. Specifically, it includes the following processes:

[1586] Facial expressions are analyzed from facial images to identify emotions such as joy, anger, sadness, and happiness.

[1587] Emotions are extracted by analyzing the tone and content of the voice from the audio data.

[1588] Choice generation and notification

[1589] The shopping recommendation engine comprehensively analyzes emotional and behavioral data to recommend the most suitable products and services to users. This process is as follows:

[1590] Based on the results of the emotion recognition engine, the shopping recommendation engine generates the optimal product list.

[1591] Product information and recommendations are displayed in real time on the smart glasses display.

[1592] Feedback and System Updates

[1593] Users can provide feedback through the smart glasses, for example, by rating the recommended products using voice commands or a touch interface. This feedback information is sent to a cloud server and used to improve the accuracy of the analysis and choice generation algorithms.

[1594] Natural language explanation of program processing

[1595] The server includes a means for collecting user behavioral data and emotional data connected to the cloud server, a means for generating options that match the user's interests and behavior based on the analysis results, and a means for connecting to the smart glasses and providing information to the user in real time, thereby enabling personalized product recommendations to be made to the user, providing an efficient and satisfying shopping experience.

[1596] Specific examples

[1597] For example, when a user is browsing products in a physical store, the camera in the smart glasses captures the user's facial expressions, and the emotion recognition engine analyzes the images to determine their interests and satisfaction. The analysis results are sent to a cloud server, and the shopping recommendation engine selects products that match the user's preferences and displays them on the smart glasses' display. The user can provide feedback on the displayed information, and the results will be reflected in the next recommendation algorithm.

[1598] Prompt Sentence Examples

[1599] In developing this application, the prompts input to the generative AI model are as follows:

[1600] We are developing an application for smart glasses that uses a user's facial image data to analyze emotions in real time and, based on the analysis results, recommends optimal products from a shopping recommendation engine. The captured image data is sent to an emotion recognition API on the cloud, and the received emotion data is used to generate personalized product recommendations. The application also includes a function that allows users to provide feedback in real time.

[1601] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1602] Step 1: Collect data

[1603] Subject: Terminal

[1604] The smart glasses use a built-in camera and microphone to collect the user's facial images and voice data in real time. Facial images are captured periodically, and voice data is recorded of the user's speech and reactions. These data are temporarily stored in the device's local memory. The input data are facial images and voice data, and the output is facial image files and voice files stored in the local memory.

[1605] Step 2: Data transmission and storage

[1606] Subject: Terminal / Server

[1607] The collected facial image and voice data is sent to a cloud server via the Internet. The data is periodically batch processed and securely stored on the cloud server. During transmission, a checksum is generated to verify the integrity of the data, and two-way communication is performed. The input data is facial image and voice data from the device, and the output is a data file stored on the cloud server.

[1608] Step 3: Analyze the emotion data

[1609] Subject: Server

[1610] The cloud server sends the collected facial images and voice data to an emotion recognition engine. The emotion recognition engine analyzes facial expressions from images and extracts emotions from voice. This process uses a generative AI model. For example, image analysis identifies emotions such as "surprise" or "anger," while voice analysis extracts emotions from the tone and content of voice. The input data are the facial images and voice data stored on the cloud server, and the output is emotion data from the emotion recognition engine.

[1611] Step 4: Generate options

[1612] Subject: Server

[1613] Based on emotional and behavioral data, the shopping recommendation engine recommends the most suitable products and services to the user. Based on the analysis results, a product list that matches the user's interests is generated. The generated recommendation list is displayed in real time on the smart glasses display. The input data is emotional data from the emotion recognition engine, and the output is a personalized product recommendation list.

[1614] Step 5: Notify users

[1615] Subject: Terminal

[1616] The smart glasses display displays recommended product information sent from the server in real time. For example, information such as "Recommended Product: Product X - Price: 1000 yen" is displayed in the user's field of view. The input data is the recommended product list sent from the server, and the output is the information displayed on the display screen of the smart glasses.

[1617] Step 6: Gather feedback

[1618] Subject: Terminal / User

[1619] The user provides feedback on the recommended products through the smart glasses using voice commands or a touchpad. For example, the user inputs a voice command such as "I like this product." The device then sends this feedback data to the cloud server. The input data is the user's feedback comment, and the output is the feedback data sent to the cloud server.

[1620] Step 7: Analyze feedback and update the system

[1621] Subject: Server

[1622] The cloud server analyzes the collected feedback data and updates the algorithms of the analysis means and option generation means. For example, it may increase the frequency of recommending a particular product or modify the algorithm to recommend a different product. This feedback analysis improves the accuracy of the next recommendation. The input data is the feedback data from the user, and the output is the updated algorithm.

[1623] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1624] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1625] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1626] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1627] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1628] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1629] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1630] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1631] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1632] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1633] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1634] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1635] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1636] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1637] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1638] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1639] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1640] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1641] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1642] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1643] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1644] The following is further disclosed regarding the above embodiment.

[1645] (Claim 1)

[1646] a means for collecting user behavior data;

[1647] means for transmitting the collected behavioral data to a cloud server;

[1648] means for analyzing the behavioral data transmitted to the cloud server;

[1649] A means for generating options that match the user's interests and behavior based on the analysis results;

[1650] means for notifying the user of the generated options;

[1651] a means for collecting and analyzing user feedback;

[1652] means for updating the analysis means and the choice generation means based on the feedback;

[1653] A system including:

[1654] (Claim 2)

[1655] 10. The system of claim 1, wherein the system automatically acquires location data and activity data from social networking services of the user.

[1656] (Claim 3)

[1657] 10. The system of claim 1, wherein the system collects text, photos, and voice notes manually entered by a user.

[1658] "Example 1"

[1659] (Claim 1)

[1660] a means for collecting user behavior data;

[1661] means for transmitting the collected behavioral data to a cloud server;

[1662] means for analyzing the behavioral data transmitted to the cloud server;

[1663] A means for generating options that match the user's interests and behavior based on the analysis results;

[1664] means for notifying the user of the generated options;

[1665] a means for collecting and analyzing user feedback;

[1666] means for updating the analysis means and the choice generation means based on the feedback;

[1667] a mobile information terminal equipped with an application that collects behavioral data in a user's daily life;

[1668] A means of storing collected behavioral data on the cloud and verifying its integrity;

[1669] means for running machine learning algorithms based on the stored data to analyze user interests;

[1670] A means to construct proposals based on the results using generative AI models;

[1671] means for transmitting and notifying the generated proposal to the terminal;

[1672] A system including:

[1673] (Claim 2)

[1674] 10. The system of claim 1, wherein the system automatically acquires location data and activity data from social networking services of the user.

[1675] (Claim 3)

[1676] 10. The system of claim 1, wherein the system collects text, photos, and voice notes manually entered by a user.

[1677] "Application Example 1"

[1678] (Claim 1)

[1679] a means for collecting user behavior data;

[1680] means for transmitting the collected behavioral data to a cloud server;

[1681] means for analyzing the behavioral data transmitted to the cloud server;

[1682] A means for generating options that match the user's interests and behavior based on the analysis results;

[1683] means for notifying the user of the generated options;

[1684] a means for collecting and analyzing user feedback;

[1685] means for updating the analysis means and the choice generation means based on the feedback;

[1686] A means of suggesting optimal products to users on smart devices,

[1687] A system including:

[1688] (Claim 2)

[1689] 10. The system of claim 1, wherein the system automatically acquires location data and activity data from social networking services of the user.

[1690] (Claim 3)

[1691] 10. The system of claim 1, wherein the system collects text, photos, and voice notes manually entered by a user.

[1692] (Claim 4)

[1693] The system of claim 1 utilizes a generative AI model based on the user's preferences to suggest products in response to prompt statements.

[1694] "Example 2: Combining Emotion Engines"

[1695] (Claim 1)

[1696] a means for collecting user behavior data;

[1697] means for transmitting the collected behavioral data and emotion data to a cloud server;

[1698] means for analyzing the behavioral data and emotion data transmitted to the cloud server;

[1699] means for analyzing a user's emotions using an emotion recognition engine;

[1700] A means for generating options that match the user's interests, behavior, and emotions based on the analysis results;

[1701] means for notifying the user of the generated options;

[1702] a means for collecting and analyzing user feedback;

[1703] means for updating the analysis means and the choice generation means based on the feedback;

[1704] A system including:

[1705] (Claim 2)

[1706] 10. The system of claim 1, wherein the system automatically acquires location data and activity data from social networking services of the user.

[1707] (Claim 3)

[1708] 10. The system of claim 1, wherein the system collects text, photos, and voice notes manually entered by a user.

[1709] "Application example 2 when combining emotion engines"

[1710] (Claim 1)

[1711] a means for collecting user behavior data;

[1712] means for transmitting the collected behavioral data and emotion data to a cloud server;

[1713] means for analyzing the behavioral data and emotion data transmitted to the cloud server;

[1714] A means for generating options that match the user's interests and behavior based on the analysis results;

[1715] means for notifying the user of the generated options;

[1716] a means for collecting and analyzing user feedback;

[1717] means for updating the analysis means and the choice generation means based on the feedback;

[1718] means connected to the smart glasses for providing information to a user in real time;

[1719] A system including:

[1720] (Claim 2)

[1721] 10. The system of claim 1, wherein the system automatically acquires location data and activity data from social networking services of the user.

[1722] (Claim 3)

[1723] 10. The system of claim 1, wherein the system collects text, photos, and voice notes manually entered by a user. [Explanation of symbols]

[1724] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means for collecting user behavior data; means for transmitting the collected behavioral data to a cloud server; means for analyzing the behavioral data transmitted to the cloud server; A means for generating options that match the user's interests and behavior based on the analysis results; means for notifying the user of the generated options; a means for collecting and analyzing user feedback; means for updating the analysis means and the choice generation means based on the feedback; A system including:

2. The system according to claim 1 , wherein the system automatically acquires location data and activity data of the user from a social networking service.

3. The system of claim 1 , wherein the system collects text, photos, and voice notes manually entered by the user.

Images