system

Abstract

We provide the system. [Solution] Means for collecting user information, Means of obtaining relevant information from multiple sources, A means of organizing and summarizing acquired information using a generative model, A means of generating individual proposals based on summarized information, A means of notifying the user of the generated suggestions, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In modern society, users are required to efficiently select useful information and services according to their life stages and interests from a vast amount of information. However, many users spend time on information selection and are troubled by excessive information and difficulty in appropriate information selection. Therefore, a system capable of quickly providing appropriate service and product information according to user needs is necessary.

Means for Solving the Problems

[0005] This invention provides means for collecting user information and obtaining data from multiple sources. It also organizes and summarizes the acquired information using a generative model and generates individual suggestions based on the results. These suggestions are notified to the user, and feedback is collected as needed to improve accuracy by adjusting the generative model. Furthermore, by filtering trend information and rankings from partner companies, it achieves information provision optimized for the user's profile.

[0006] "User information" refers to attribute data about individual users, including information such as age, occupation, hobbies, interests, and purchase history.

[0007] "Information sources" refer to sources from which relevant information can be obtained from a specific industry or market, including partner companies and online ranking sites.

[0008] A "generative model" is an algorithm that applies artificial intelligence technology to organize and summarize acquired information.

[0009] "Suggestions" refer to personalized service and product information generated based on the user's profile information and related data.

[0010] "Notifications" refer to communication methods used to quickly deliver generated suggestions to users, including methods such as in-app notifications and push notifications.

[0011] "Feedback" refers to the opinions and evaluations that users provide after receiving a suggestion regarding its usefulness and appropriateness.

[0012] "Trend information" refers to the latest information on products, themes, and other topics that are in high demand during a specific period.

[0013] "Ranking" refers to information that ranks services, products, or data based on specific criteria.

[0014] "Filtering" refers to the act of selecting and extracting information that meets specific criteria from a vast amount of data. [Brief explanation of the drawing]

[0015] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14]It is a sequence diagram showing the processing flow of a data processing system in Application Example 2 when a sentiment engine is combined.

Embodiments for Carrying Out the Invention

[0016] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a processor with a reference number (hereinafter simply referred to as "processor") may be one arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be one type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), etc.

[0019] In the following embodiments, a RAM (Random Access Memory) with a reference number is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0020] In the following embodiments, a storage with a reference number is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0021] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0023] [First Embodiment]

[0024] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0025] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0028] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0031] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0035] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0036] This invention is implemented as a system that allows users to efficiently receive information and services. This system provides users with optimized suggestions through the coordinated operation of the server, terminal, and user.

[0037] The server first collects user information. This includes data such as profile information and past behavioral history entered by the user through their device. The server then retrieves relevant information from various partner sources, extracting data that is particularly relevant to the user's profile. This includes data from ranking sites and companies that provide trend information.

[0038] Furthermore, the server utilizes a generative model to automatically organize and summarize the acquired information. This generative model is based on machine learning algorithms, learning patterns based on user interests and behaviors, and selecting the most relevant information. The summarized information is then structured as individual suggestions for each user.

[0039] The generated suggestions are notified to the user via their device. Notification methods include smartphone push notifications and in-app notifications, allowing users to quickly review the suggestions. After receiving the notification, users can view detailed information and take further action.

[0040] For example, if a user is interested in buying a home, the server can summarize suitable housing options and financial support plans based on the user's profile and notify them via their device. Based on this notification, the user can consider the suggested options and make the best choice.

[0041] This system collects user feedback, records and analyzes it on the server, and uses it to refine the generative model. The entire system is designed to be progressively improved so that future suggestions become more accurate. This process ensures that users always receive suggestions optimized for them.

[0042] The following describes the processing flow.

[0043] Step 1:

[0044] The device displays an interface where users can enter or update their profile information when they log into the application. Here, users can enter information such as their age, occupation, interests, and past purchase history.

[0045] Step 2:

[0046] The server receives user information sent from the terminal and securely stores it in an internal database. This database forms the basis for building user profiles.

[0047] Step 3:

[0048] The server retrieves data via APIs from partner information sources. This data includes information from ranking sites and companies that provide trend information.

[0049] Step 4:

[0050] The server filters the retrieved data based on the user's profile, selecting the most relevant information. In this process, it compares information from different data sources, prioritizing the most up-to-date and reliable data.

[0051] Step 5:

[0052] The server uses a generative AI model to organize and summarize the selected information. This generative model analyzes patterns in the information and generates summaries tailored to the user's interests.

[0053] Step 6:

[0054] The server then constructs individual suggestions based on the summarized information, which are stored as personalized content for each user.

[0055] Step 7:

[0056] The server notifies the device with personalized suggestions. These notifications are delivered via push notifications or in-app messaging, ensuring users can see them immediately.

[0057] Step 8:

[0058] Users can receive notifications on their devices and check their contents. By viewing the details, they can obtain more information about the suggested services and products.

[0059] Step 9:

[0060] When a user provides feedback, the device sends that information to the server. The server analyzes this information and uses it to improve the generated AI model, thereby increasing the accuracy of future suggestions.

[0061] (Example 1)

[0062] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0063] In recent years, in our information-saturated society, it has become difficult for users to efficiently acquire information and services that are relevant to them. To solve this problem, it is necessary to automatically generate personalized suggestions tailored to the user's preferences and behavioral patterns and deliver them at the appropriate time.

[0064] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0065] In this invention, the server includes means for collecting user information as product data, means for acquiring profile-related information from multiple information sources, and means for organizing and summarizing the acquired information using a generation AI model. This makes it possible to provide personalized information and services that meet the user's needs.

[0066] "Methods for accumulating user information as product data" refers to methods for collecting information that forms the basis for personalized recommendations by accumulating profile information and data on areas of interest entered by customers.

[0067] "Means of obtaining profile-related information from multiple sources" refers to methods of collecting information related to customer preferences and behavior using external databases or APIs.

[0068] "Methods for organizing and summarizing acquired information using a generative AI model" refers to a process of analyzing collected data using artificial intelligence technology to efficiently extract and summarize information that is beneficial to the customer.

[0069] "Means of constructing personalized service proposals" refers to a method of automatically generating recommendations for the most suitable products and services based on the client's profile information.

[0070] "Means of providing information to customers via terminals" refers to the process of notifying customers of collected and generated suggestions through terminal devices.

[0071] "A means of collecting customer evaluation data and adjusting the parameters of the generative model based on it" refers to a method of optimizing the generative AI model based on feedback provided by users to improve the accuracy of suggestions.

[0072] "A means of acquiring trend information and ranking data from external organizations and selecting information based on customer attribute information" refers to the process of obtaining the latest market trends and popularity rankings from partner companies and institutions and filtering them according to customer needs.

[0073] This invention provides a system that enables users to efficiently receive personalized information and services. The system operates through the coordinated roles of server, terminal, and user.

[0074] The server collects user information, including profile information and behavioral history entered by the user via their device. A relational database management system (RDBMS) is used for data management to ensure data consistency and security. The server also retrieves relevant information from other data sources via external APIs. This process involves collecting various types of information via RESTful APIs and parsing the data in JSON format. The collected information is then passed to a generative AI model. This AI model is a machine learning model based on the Transformer architecture, which learns from the user's past behavioral patterns and automatically summarizes relevant information.

[0075] The device is responsible for notifying the user of personalized suggestions received from the server. The application uses the smartphone's push notification function to deliver suggestions to the user in real time. By opening the notification, the user can view details of the suggested services or products and choose further action as needed.

[0076] Users input information into the system according to their circumstances and interests, and provide feedback on the suggestions and services they receive. This feedback is also analyzed on the server and used to improve the accuracy of future suggestions.

[0077] As a concrete example, when a user is considering purchasing a home, the server provides the user with the most suitable housing options and financial plans. An example of a prompt in this case would be the text, "I am currently considering purchasing a home. I would like to know about housing options and financial support plans that are right for me." Based on this prompt, the server collects the necessary data, summarizes the information using a generative AI model, and provides the user with optimized suggestions.

[0078] This system allows users to quickly receive personalized information and suggestions that always match their needs.

[0079] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0080] Step 1:

[0081] Users enter profile information and areas of interest through their devices. This includes age, product categories of interest, and past browsing history. This input data is sent to the server as foundational data to generate personalized suggestions for the user.

[0082] Step 2:

[0083] The server stores information submitted by users in a product database. The data is managed using an RDBMS to ensure security and data consistency. Input data is stored as a user profile and used for subsequent processing.

[0084] Step 3:

[0085] The server retrieves relevant information from multiple external sources. Specifically, it obtains data from trend information and ranking services via RESTful APIs and parses it in JSON format. This retrieved data is then processed to match each user's profile.

[0086] Step 4:

[0087] The server inputs the collected information into a generating AI model. This AI model uses a machine learning algorithm based on the Transformer architecture to analyze user behavior patterns and organize and summarize the information based on the results. Through this process, the input data is transformed into content that is useful to the user.

[0088] Step 5:

[0089] Based on the summary information output from the generative AI model, the server generates personalized recommendations. These include product lists and service options tailored to each user's interests. The generated recommendations are then prepared to be communicated to the user in the next step.

[0090] Step 6:

[0091] The device notifies the user of the suggestion information received from the server. Notifications are sent via smartphone push notifications or in-app notifications, allowing users to quickly review the suggestions.

[0092] Step 7:

[0093] Users can review the suggested information through their device and view detailed information. If necessary, they can take specific actions such as purchasing or inquiring about the suggested products or services.

[0094] Step 8:

[0095] Users submit feedback on the information and suggestions provided. The server collects this feedback and uses it as data to adjust the parameters of the generated AI model. This process improves the accuracy of future suggestions, allowing users to receive more relevant information.

[0096] (Application Example 1)

[0097] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0098] In urban life, there is a challenge in receiving new information and services quickly and individually. To solve this problem, a system is needed that efficiently delivers personalized suggestions based on the user's interests.

[0099] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0100] In this invention, the server includes means for collecting user information, means for obtaining relevant information from multiple information sources, and means for obtaining urban area information and providing local event information based on the user's interests. This makes it possible to provide personalized and rapid information and services optimized for the user.

[0101] "User information" refers to data that the system uses to optimize suggestions, including user profiles and behavioral data.

[0102] "Information sources" refer to external and internal data providers that partner organizations and systems refer to in order to collect data.

[0103] A "generative model" is a computer program that uses machine learning algorithms to organize and summarize information acquired according to the user's interests and behavior.

[0104] "Personalized suggestions" refer to customized information and service suggestions generated based on the user's profile and behavior.

[0105] A "smart city" is a general term for a city that collects and analyzes urban information and aims to provide services optimized for residents and visitors.

[0106] "Local event information" refers to data about events and activities held within a specific city or region.

[0107] This invention is implemented in smart cities as a system for users to receive optimized information and services. The server acquires data from devices such as smartphones and tablets to collect user information and stores that information on a cloud server. As for hardware, cloud services such as AWS® and Microsoft® Azure® can be used, and databases such as Amazon RDS and MongoDB can be used.

[0108] The server retrieves relevant information from multiple sources, including data feeds from public institutions and trend information provided by private companies. This retrieved information is then organized and summarized on the server using generative AI models such as TENSORFLOW® and PyTorch.

[0109] The AI ​​model analyzes user behavior patterns and interests, extracting the most relevant information and structuring it into personalized recommendations. These recommendations are notified in real time on the user's smart device, allowing the user to view detailed information and take further action as needed.

[0110] For example, if a user is interested in visiting art museums, the server analyzes the user's browsing history, organizes information on nearby art events, and provides it as the most suitable suggestion. Furthermore, by collecting feedback on this suggestion, the accuracy of future suggestions can be improved.

[0111] A concrete example of a prompt message would be: "Suggest information about art events currently taking place or scheduled in the area to users who are interested in art."

[0112] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0113] Step 1:

[0114] The device collects user profile and location information and sends it to the server. Input includes the user's basic data and current location, while output is the collected data sent to the server in JSON format.

[0115] Step 2:

[0116] The server stores received user information in a cloud database and simultaneously retrieves local event and general trend information from multiple publicly available sources. Inputs are user information and data from the sources, while output is a diverse list of retrieved information. Amazon RDS is commonly used for the database.

[0117] Step 3:

[0118] The server analyzes the collected information using a generating AI model and generates optimal suggestions based on the user's interests and behavior. In this step, user information and various event information stored in the database are handled as input, and the output is the generated, customized suggestions. TensorFlow is used for the AI ​​model.

[0119] Step 4:

[0120] The generated suggestions are sent from the server to the device via push notifications. The device receives this notification data as input and displays the suggestions visually to the user as output. The user interface is intuitive and is often developed using React Native.

[0121] Step 5:

[0122] The user reviews the suggested event information, selects those of interest, and learns more details. The input is the displayed list of suggestions, and the output is the details page for the selected event. Feedback based on this selection is also sent to the server.

[0123] Step 6:

[0124] The server records user feedback in a database and uses it to adjust the generating AI model to improve the accuracy of future suggestions. Inputs include user preferences and behavioral history, while output are updated model parameters.

[0125] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0126] The present invention is implemented as a system that recognizes a user's emotions and provides personalized suggestions based on those emotions. This system consists of a server, a terminal, and an emotion engine that analyzes the user's emotions.

[0127] The server collects profile information provided by the user through their device and builds a user profile based on this information. This profile includes personal attribute information and behavioral history. The server also obtains ranking and trend information from multiple partner sources via APIs. This data is filtered based on the user profile to select the most relevant information.

[0128] The emotion engine uses the camera and microphone to recognize the user's emotions in real time during user-device interaction. This engine can identify the user's current emotional state through voice tone and facial expression analysis. For example, it can identify various emotions such as happiness, sadness, or interest.

[0129] The server incorporates emotional data obtained from the emotion engine into a generative model and dynamically adjusts the suggested content according to the user's emotional state. This adjustment makes it possible to provide information that is most appropriate to the user's needs and emotions at the right time.

[0130] Individual suggestions are notified to the user via their device. Since notifications are made in real time, users can receive the most appropriate suggestions based on their current emotional state. For example, if a user is feeling stressed, they may receive suggestions for relaxation products or services. Furthermore, users who receive suggestions can view detailed information and take relevant actions immediately.

[0131] This system further collects user feedback and uses it to refine the emotion engine and generative models. The server analyzes the feedback and improves the overall accuracy of the system to better meet user needs. Through this process, users can always receive information optimized for their own emotions and needs.

[0132] The following describes the processing flow.

[0133] Step 1:

[0134] The device displays an interface for users to enter or update their profile information when they log into the application. Here, users can enter or update information such as age, occupation, interests, and past purchase history.

[0135] Step 2:

[0136] The server receives profile information sent from the user via their device and stores it in a database. This builds the user's profile.

[0137] Step 3:

[0138] The server retrieves data via APIs from partner information sources. This data includes information from ranking sites and companies that provide trend information.

[0139] Step 4:

[0140] The server filters the retrieved data based on the user's profile, selecting the most relevant information. This process is crucial for prioritizing the most up-to-date and reliable information.

[0141] Step 5:

[0142] The device uses its built-in camera and microphone to analyze voice tone and facial expressions during user interaction, and its emotion engine recognizes the user's emotions in real time.

[0143] Step 6:

[0144] The emotion engine sends recognized emotion data to the server, and based on this, the generative AI model dynamically adjusts the suggested content. This adjustment provides information optimized for the user's emotions.

[0145] Step 7:

[0146] The server generates personalized suggestions based on sentiment data and filtered information, and notifies the device of these suggestions. Notifications are sent via push notifications or in-app messages.

[0147] Step 8:

[0148] Users can receive notifications on their devices and, by reviewing the details, take appropriate action regarding the suggested products and services.

[0149] Step 9:

[0150] If a user provides feedback after using a suggestion, the device sends that feedback to the server. The server uses this information to further refine the sentiment engine and generative model, improving the system's accuracy.

[0151] (Example 2)

[0152] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0153] Traditional systems had the challenge of not being able to appropriately provide personalized suggestions that responded to the user's emotional state in real time. Furthermore, the feedback loop for improving the accuracy and relevance of suggestions to the user was not functioning adequately, resulting in a limited user experience.

[0154] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0155] In this invention, the server includes means for collecting information including user attribute information and behavioral history, storing it in a database, and building a profile; means for analyzing voice and video data to recognize the user's emotional state in real time; and means for obtaining relevant information from external sources and filtering it based on the profile. This makes it possible to provide appropriate suggestions in real time that meet the individual needs of the user.

[0156] "User attribute information" refers to identifiable data about a user, including basic personal characteristics such as age, gender, and hobbies.

[0157] "Behavioral history" refers to records of specific actions and habits a user has taken in the past, and is data that shows what choices and actions they have taken over time.

[0158] A "database" is a part of a computer system for systematically storing and managing information, and it has a structure that enables efficient data retrieval and manipulation.

[0159] A "profile" is a data set formed by integrating a user's attribute information and behavioral history, and it includes information that reflects the user's characteristics and interests.

[0160] "Emotional state" refers to the user's mental and emotional responses and conditions, and is measured through real-time audio and video data.

[0161] "Real-time recognition" refers to a process that instantly processes data acquired from users and constantly updates the state changes and information at that moment.

[0162] "External information sources" refer to data providers or platforms that exist outside the system and are sources of information such as rankings and trends.

[0163] A "generative model" refers to algorithms and technologies that use artificial intelligence to create new information and suggestions based on input data.

[0164] "Filtering" refers to the process of selecting data based on conditions or specific criteria, excluding unnecessary information, and extracting useful information.

[0165] This invention is a system that provides personalized suggestions based on the user's emotional state. It consists of a server, a terminal, and an emotion recognition engine.

[0166] The server stores attribute information and past behavioral history provided by users in a database and uses it to build user profiles. It also retrieves trend information and rankings from external sources and filters this information based on the user profile. Relational database management systems are commonly used for the databases, and SQL queries are utilized for filtering.

[0167] The device uses a camera and microphone to record user interaction and provides audio and video data to an emotion engine. This data is analyzed in real time by the emotion engine to recognize the user's emotional state. For example, it is common to use the OpenCV library for computer vision and a speech recognition API for speech analysis.

[0168] The emotion engine recognizes the user's emotional state, which is then sent to a server and input into a generative AI model. This generative model generates optimal suggestions based on the user's current emotional state and relevant filtered information. The generative AI model employs advanced natural language processing to create text-based prompts. For example, a prompt might ask, "What kind of music should be suggested if the user wants to relax in the afternoon?"

[0169] As a concrete example, if a user indicates a desire to relax via the camera and microphone on their device after returning home from work, the server inputs data analyzed by the emotion engine into a generative model, which then suggests music and relaxation techniques best suited for relaxation. In this way, the system can provide real-time suggestions tailored to the individual needs of each user.

[0170] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0171] Step 1:

[0172] The server collects attribute information and behavioral history entered by users through their devices. This data includes age, hobbies, and past usage history, and is stored in a database. The server uses SQL to organize the data and build user profiles. These profiles form the basis of the information.

[0173] Step 2:

[0174] The device uses its camera and microphone to collect audio and video data obtained during user interaction. The input data is sent to the emotion engine in real time. The emotion engine uses a computer vision library for video and a speech analysis API for audio to analyze the user's emotional state. As a result, a specific emotional state, such as happy or sad, is output.

[0175] Step 3:

[0176] The server receives emotional state data output from the emotion engine and combines it with profile data and trend information obtained from external sources. This data is then fed into a filtering algorithm to extract the most relevant information for the user. The output is then the relevant information.

[0177] Step 4:

[0178] The server inputs filtered relevant information and emotional state data into a generating AI model. The server generates prompt sentences, and based on these prompts, generates suggestions. As a result, personalized suggestions based on the user's emotions and profile are output. For example, it might generate suggestions for "music to recommend for relaxation."

[0179] Step 5:

[0180] The device notifies the user in real time of generated suggestions received from the server. Push notification technology is used for notifications, allowing users to receive suggestions immediately. This process involves the user reviewing the notified suggestions and, if necessary, viewing detailed information or taking action directly.

[0181] Step 6:

[0182] Users provide feedback on the suggestions offered. This feedback is collected via the device and sent to the server. The server analyzes this feedback and uses it to further improve the emotion engine and generative AI model. As a result, the overall accuracy of the system and the user experience improve.

[0183] (Application Example 2)

[0184] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0185] Modern users have diverse needs based on varying emotions and circumstances each day, making it difficult to provide them with timely and appropriate information and products. Many services are uniform, lacking personalized suggestions tailored to individual emotions and situations, thus increasing user satisfaction. Furthermore, providing emotionally-based suggestions, which offer services optimized for each individual user, has been difficult to achieve with conventional technologies.

[0186] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0187] In this invention, the server includes means for collecting user information, means for obtaining relevant information from multiple information sources, and means for analyzing emotional data. This makes it possible to suggest individually optimized products and services based on the user's emotions and to assist the user in purchasing products that suit their situation.

[0188] "User information" refers to data collected by the system about users to understand their preferences and attributes.

[0189] "Information source" refers to an external data provider that the system uses to obtain relevant information.

[0190] A "generative model" is a technical mechanism that uses collected data and information to organize it and present it to users in a meaningful way.

[0191] "Emotional data" refers to information used to analyze a user's current emotional state, and includes characteristics derived from facial expressions, voice, and other sources.

[0192] "Means of recommending the purchase of goods or services" refers to a function that presents suitable goods or services based on the user's emotions and circumstances.

[0193] "Means of assisting the purchase process" refers to a system designed to help users purchase proposed goods or services quickly and smoothly.

[0194] "Feedback" refers to the reactions and opinions received from users, which are used to improve the system.

[0195] "Partner providers" refers to external businesses or organizations that cooperate in providing information or products.

[0196] "Trend information and rankings" refer to information about current trends and popular topics, and are highly likely to attract users' interest.

[0197] "Filtering" is the process of selecting only the necessary information from collected data based on specific criteria.

[0198] The system implementing this invention runs via the user's smartphone, smart glasses, or other device. The server collects profile information provided by the user and builds a detailed user profile based on it. This profile includes personal attributes and past behavioral history. The server also utilizes APIs to obtain trend information and rankings from multiple partner sources, enabling the collection of relevant information.

[0199] The emotion recognition engine on the device uses the camera and microphone to recognize the user's emotions in real time. This engine analyzes the user's emotions from their voice tone and facial expressions, identifying states such as joy, sadness, and interest. This allows the device to determine the user's current emotional state.

[0200] Based on the user's emotions, the server uses a generative AI model to generate optimal suggestions. For example, if a user is feeling stressed, the system can recommend purchasing relaxation products or services. This suggestion also includes a function to assist with the purchase process so that the purchase can be made immediately.

[0201] As a concrete example, when the system determines that a user is feeling a little down, it suggests a discount coupon for a nearby spa and allows for immediate purchase. This process involves dynamic suggestion generation using a generative AI model.

[0202] An example of a prompt to input into the generation AI model is, "Generate specific suggestions for products and services that can help a user relax when they are tired."

[0203] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0204] Step 1:

[0205] The device sends user profile information to the server. This information includes user attribute data and behavioral history, and is used by the server to create a user profile. The input is the user's profile information, and the output is the user profile data within the server.

[0206] Step 2:

[0207] The server retrieves trend information and rankings from partner sources via API. This information is filtered based on the user profile to show only the most relevant data. The input is trend data from external sources, and the output is filtered information associated with the user profile.

[0208] Step 3:

[0209] The device's emotion recognition engine uses the camera and microphone to collect and analyze user emotion data in real time. The input is camera video and audio data, and the output is emotion data indicating the user's current emotional state.

[0210] Step 4:

[0211] The server uses a generative AI model to generate personalized suggestions based on the user's emotional data. This process takes the user profile and filtered information as input and outputs suggestions optimized for the user's emotions.

[0212] Step 5:

[0213] The terminal notifies the user of suggestions sent from the server. These notifications include information about products and services suitable for the user. The input is suggestion data from the server, and the output is a notification message to the user.

[0214] Step 6:

[0215] The user reviews the notified products and services and proceeds with the purchase if they decide to buy them. The server assists with the purchase process based on the results of the generated AI model. The input is the user's purchase intent data, and the output is confirmation data of the completed purchase.

[0216] Step 7:

[0217] The server collects user feedback and uses it to refine the generative model. This feedback is used to improve the accuracy of suggestions and sentiment recognition. The input is user feedback data, and the output is the refined generative model.

[0218] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0219] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0220] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0221] [Second Embodiment]

[0222] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0223] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0224] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0225] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0226] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0227] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0228] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0229] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0230] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0231] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0232] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0233] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0234] This invention is implemented as a system that allows users to efficiently receive information and services. This system provides users with optimized suggestions through the coordinated operation of the server, terminal, and user.

[0235] The server first collects user information. This includes data such as profile information and past behavioral history entered by the user through their device. The server then retrieves relevant information from various partner sources, extracting data that is particularly relevant to the user's profile. This includes data from ranking sites and companies that provide trend information.

[0236] Furthermore, the server utilizes a generative model to automatically organize and summarize the acquired information. This generative model is based on machine learning algorithms, learning patterns based on user interests and behaviors, and selecting the most relevant information. The summarized information is then structured as individual suggestions for each user.

[0237] The generated suggestions are notified to the user via their device. Notification methods include smartphone push notifications and in-app notifications, allowing users to quickly review the suggestions. After receiving the notification, users can view detailed information and take further action.

[0238] For example, if a user is interested in buying a home, the server can summarize suitable housing options and financial support plans based on the user's profile and notify them via their device. Based on this notification, the user can consider the suggested options and make the best choice.

[0239] This system collects user feedback, records and analyzes it on the server, and uses it to refine the generative model. The entire system is designed to be progressively improved so that future suggestions become more accurate. This process ensures that users always receive suggestions optimized for them.

[0240] The following describes the processing flow.

[0241] Step 1:

[0242] The device displays an interface where users can enter or update their profile information when they log into the application. Here, users can enter information such as their age, occupation, interests, and past purchase history.

[0243] Step 2:

[0244] The server receives user information sent from the terminal and securely stores it in an internal database. This database forms the basis for building user profiles.

[0245] Step 3:

[0246] The server retrieves data via APIs from partner information sources. This data includes information from ranking sites and companies that provide trend information.

[0247] Step 4:

[0248] The server filters the retrieved data based on the user's profile, selecting the most relevant information. In this process, it compares information from different data sources, prioritizing the most up-to-date and reliable data.

[0249] Step 5:

[0250] The server uses a generative AI model to organize and summarize the selected information. This generative model analyzes patterns in the information and generates summaries tailored to the user's interests.

[0251] Step 6:

[0252] The server then constructs individual suggestions based on the summarized information, which are stored as personalized content for each user.

[0253] Step 7:

[0254] The server notifies the device with personalized suggestions. These notifications are delivered via push notifications or in-app messaging, ensuring users can see them immediately.

[0255] Step 8:

[0256] Users can receive notifications on their devices and check their contents. By viewing the details, they can obtain more information about the suggested services and products.

[0257] Step 9:

[0258] When a user provides feedback, the device sends that information to the server. The server analyzes this information and uses it to improve the generated AI model, thereby increasing the accuracy of future suggestions.

[0259] (Example 1)

[0260] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0261] In recent years, in our information-saturated society, it has become difficult for users to efficiently acquire information and services that are relevant to them. To solve this problem, it is necessary to automatically generate personalized suggestions tailored to the user's preferences and behavioral patterns and deliver them at the appropriate time.

[0262] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0263] In this invention, the server includes means for collecting user information as product data, means for acquiring profile-related information from multiple information sources, and means for organizing and summarizing the acquired information using a generation AI model. This makes it possible to provide personalized information and services that meet the user's needs.

[0264] "Methods for accumulating user information as product data" refers to methods for collecting information that forms the basis for personalized recommendations by accumulating profile information and data on areas of interest entered by customers.

[0265] "Means of obtaining profile-related information from multiple sources" refers to methods of collecting information related to customer preferences and behavior using external databases or APIs.

[0266] "Methods for organizing and summarizing acquired information using a generative AI model" refers to a process of analyzing collected data using artificial intelligence technology to efficiently extract and summarize information that is beneficial to the customer.

[0267] "Means of constructing personalized service proposals" refers to a method of automatically generating recommendations for the most suitable products and services based on the client's profile information.

[0268] "Means of providing information to customers via terminals" refers to the process of notifying customers of collected and generated suggestions through terminal devices.

[0269] "A means of collecting customer evaluation data and adjusting the parameters of the generative model based on it" refers to a method of optimizing the generative AI model based on feedback provided by users to improve the accuracy of suggestions.

[0270] "A means of acquiring trend information and ranking data from external organizations and selecting information based on customer attribute information" refers to the process of obtaining the latest market trends and popularity rankings from partner companies and institutions and filtering them according to customer needs.

[0271] This invention provides a system that enables users to efficiently receive personalized information and services. The system operates through the coordinated roles of server, terminal, and user.

[0272] The server collects user information, including profile information and behavioral history entered by the user via their device. A relational database management system (RDBMS) is used for data management to ensure data consistency and security. The server also retrieves relevant information from other data sources via external APIs. This process involves collecting various types of information via RESTful APIs and parsing the data in JSON format. The collected information is then passed to a generative AI model. This AI model is a machine learning model based on the Transformer architecture, which learns from the user's past behavioral patterns and automatically summarizes relevant information.

[0273] The device is responsible for notifying the user of personalized suggestions received from the server. The application uses the smartphone's push notification function to deliver suggestions to the user in real time. By opening the notification, the user can view details of the suggested services or products and choose further action as needed.

[0274] Users input information into the system according to their circumstances and interests, and provide feedback on the suggestions and services they receive. This feedback is also analyzed on the server and used to improve the accuracy of future suggestions.

[0275] As a concrete example, when a user is considering purchasing a home, the server provides the user with the most suitable housing options and financial plans. An example of a prompt in this case would be the text, "I am currently considering purchasing a home. I would like to know about housing options and financial support plans that are right for me." Based on this prompt, the server collects the necessary data, summarizes the information using a generative AI model, and provides the user with optimized suggestions.

[0276] This system allows users to quickly receive personalized information and suggestions that always match their needs.

[0277] The flow of the specific process in Example 1 will be described using Fig. 11.

[0278] Step 1:

[0279] The user inputs profile information and areas of interest through the terminal. This includes age, product categories of interest, past browsing history, etc. This input data is sent to the server as basic data for generating personalized proposals for the user.

[0280] Step 2:

[0281] The server saves the information sent from the user in the product database. At this time, the data is managed using an RDBMS, ensuring security and data consistency. The input data is saved as a user profile and used in subsequent processing.

[0282] Step 3:

[0283] The server obtains relevant information from multiple external information sources. Specifically, data is obtained from trend information and ranking services via a RESTful API and parsed in JSON format. The obtained data is processed as information that matches the profile of each user.

[0284] Step 4:

[0285] The server inputs the collected information into the generated AI model. This AI model uses a machine learning algorithm based on the Transformer architecture to analyze the user's behavior patterns and organize and summarize information based on the results. Through this process, the input data is converted into content useful for the user.

[0286] Step 5:

[0287] Based on the summary information output from the generative AI model, the server generates personalized recommendations. These include product lists and service options tailored to each user's interests. The generated recommendations are then prepared to be communicated to the user in the next step.

[0288] Step 6:

[0289] The device notifies the user of the suggestion information received from the server. Notifications are sent via smartphone push notifications or in-app notifications, allowing users to quickly review the suggestions.

[0290] Step 7:

[0291] Users can review the suggested information through their device and view detailed information. If necessary, they can take specific actions such as purchasing or inquiring about the suggested products or services.

[0292] Step 8:

[0293] Users submit feedback on the information and suggestions provided. The server collects this feedback and uses it as data to adjust the parameters of the generated AI model. This process improves the accuracy of future suggestions, allowing users to receive more relevant information.

[0294] (Application Example 1)

[0295] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0296] In urban life, there is a challenge in receiving new information and services quickly and individually. To solve this problem, a system is needed that efficiently delivers personalized suggestions based on the user's interests.

[0297] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0298] In this invention, the server includes means for collecting user information, means for obtaining relevant information from multiple information sources, and means for obtaining urban area information and providing local event information based on the user's interests. This makes it possible to provide personalized and rapid information and services optimized for the user.

[0299] "User information" refers to data that the system uses to optimize suggestions, including user profiles and behavioral data.

[0300] "Information sources" refer to external and internal data providers that partner organizations and systems refer to in order to collect data.

[0301] A "generative model" is a computer program that uses machine learning algorithms to organize and summarize information acquired according to the user's interests and behavior.

[0302] "Personalized suggestions" refer to customized information and service suggestions generated based on the user's profile and behavior.

[0303] A "smart city" is a general term for a city that collects and analyzes urban information and aims to provide services optimized for residents and visitors.

[0304] "Local event information" refers to data about events and activities held within a specific city or region.

[0305] This invention is implemented as a system for users to receive optimized information and services in a smart city. The server acquires data from terminals such as smartphones and tablets to collect user information, and saves this information to a cloud server. As hardware, cloud services can use AWS or Microsoft Azure, and databases can utilize Amazon RDS or MongoDB.

[0306] The server acquires relevant information from multiple information sources. This includes data feeds from public institutions and trend information provided by general enterprises. The acquired information is sorted and summarized on the server using generative AI models such as TensorFlow or PyTorch.

[0307] The AI model analyzes the user's behavior patterns and interests, extracts the most relevant information, and constructs it as individual proposals. These proposals are notified in real time to the smart devices owned by the user, and the user can confirm detailed information through the notification and take further actions if necessary.

[0308] For example, if a certain user is interested in visiting art museums, the server analyzes the user's behavior history, sorts out information on art events held in the vicinity, and provides it as an optimal proposal. Also, by collecting feedback on this proposal, the accuracy can be improved for the next proposal.

[0309] Specific examples of the prompt sentence include "Please propose information on art events currently being held or planned in the region for users interested in art."

[0310] The flow of specific processing in Application Example 1 will be described using FIG. 12.

[0311] Step 1:

[0312] The device collects user profile and location information and sends it to the server. Input includes the user's basic data and current location, while output is the collected data sent to the server in JSON format.

[0313] Step 2:

[0314] The server stores received user information in a cloud database and simultaneously retrieves local event and general trend information from multiple publicly available sources. Inputs are user information and data from the sources, while output is a diverse list of retrieved information. Amazon RDS is commonly used for the database.

[0315] Step 3:

[0316] The server analyzes the collected information using a generating AI model and generates optimal suggestions based on the user's interests and behavior. In this step, user information and various event information stored in the database are handled as input, and the output is the generated, customized suggestions. TensorFlow is used for the AI ​​model.

[0317] Step 4:

[0318] The generated suggestions are sent from the server to the device via push notifications. The device receives this notification data as input and displays the suggestions visually to the user as output. The user interface is intuitive and is often developed using React Native.

[0319] Step 5:

[0320] The user reviews the suggested event information, selects those of interest, and learns more details. The input is the displayed list of suggestions, and the output is the details page for the selected event. Feedback based on this selection is also sent to the server.

[0321] Step 6:

[0322] The server records user feedback in a database and uses it to adjust the generating AI model to improve the accuracy of future suggestions. Inputs include user preferences and behavioral history, while output are updated model parameters.

[0323] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0324] The present invention is implemented as a system that recognizes a user's emotions and provides personalized suggestions based on those emotions. This system consists of a server, a terminal, and an emotion engine that analyzes the user's emotions.

[0325] The server collects profile information provided by the user through their device and builds a user profile based on this information. This profile includes personal attribute information and behavioral history. The server also obtains ranking and trend information from multiple partner sources via APIs. This data is filtered based on the user profile to select the most relevant information.

[0326] The emotion engine uses the camera and microphone to recognize the user's emotions in real time during user-device interaction. This engine can identify the user's current emotional state through voice tone and facial expression analysis. For example, it can identify various emotions such as happiness, sadness, or interest.

[0327] The server incorporates emotional data obtained from the emotion engine into a generative model and dynamically adjusts the suggested content according to the user's emotional state. This adjustment makes it possible to provide information that is most appropriate to the user's needs and emotions at the right time.

[0328] Individual suggestions are notified to the user via their device. Since notifications are made in real time, users can receive the most appropriate suggestions based on their current emotional state. For example, if a user is feeling stressed, they may receive suggestions for relaxation products or services. Furthermore, users who receive suggestions can view detailed information and take relevant actions immediately.

[0329] This system further collects user feedback and uses it to refine the emotion engine and generative models. The server analyzes the feedback and improves the overall accuracy of the system to better meet user needs. Through this process, users can always receive information optimized for their own emotions and needs.

[0330] The following describes the processing flow.

[0331] Step 1:

[0332] The device displays an interface for users to enter or update their profile information when they log into the application. Here, users can enter or update information such as age, occupation, interests, and past purchase history.

[0333] Step 2:

[0334] The server receives profile information sent from the user via their device and stores it in a database. This builds the user's profile.

[0335] Step 3:

[0336] The server retrieves data via APIs from partner information sources. This data includes information from ranking sites and companies that provide trend information.

[0337] Step 4:

[0338] The server filters the retrieved data based on the user's profile, selecting the most relevant information. This process is crucial for prioritizing the most up-to-date and reliable information.

[0339] Step 5:

[0340] The device uses its built-in camera and microphone to analyze voice tone and facial expressions during user interaction, and its emotion engine recognizes the user's emotions in real time.

[0341] Step 6:

[0342] The emotion engine sends recognized emotion data to the server, and based on this, the generative AI model dynamically adjusts the suggested content. This adjustment provides information optimized for the user's emotions.

[0343] Step 7:

[0344] The server generates personalized suggestions based on sentiment data and filtered information, and notifies the device of these suggestions. Notifications are sent via push notifications or in-app messages.

[0345] Step 8:

[0346] Users can receive notifications on their devices and, by reviewing the details, take appropriate action regarding the suggested products and services.

[0347] Step 9:

[0348] If a user provides feedback after using a suggestion, the device sends that feedback to the server. The server uses this information to further refine the sentiment engine and generative model, improving the system's accuracy.

[0349] (Example 2)

[0350] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0351] Traditional systems had the challenge of not being able to appropriately provide personalized suggestions that responded to the user's emotional state in real time. Furthermore, the feedback loop for improving the accuracy and relevance of suggestions to the user was not functioning adequately, resulting in a limited user experience.

[0352] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0353] In this invention, the server includes means for collecting information including user attribute information and behavioral history, storing it in a database, and building a profile; means for analyzing voice and video data to recognize the user's emotional state in real time; and means for obtaining relevant information from external sources and filtering it based on the profile. This makes it possible to provide appropriate suggestions in real time that meet the individual needs of the user.

[0354] "User attribute information" refers to identifiable data about a user, including basic personal characteristics such as age, gender, and hobbies.

[0355] "Behavioral history" refers to records of specific actions and habits a user has taken in the past, and is data that shows what choices and actions they have taken over time.

[0356] A "database" is a part of a computer system for systematically storing and managing information, and it has a structure that enables efficient data retrieval and manipulation.

[0357] A "profile" is a data set formed by integrating a user's attribute information and behavioral history, and it includes information that reflects the user's characteristics and interests.

[0358] "Emotional state" refers to the user's mental and emotional responses and conditions, and is measured through real-time audio and video data.

[0359] "Real-time recognition" refers to a process that instantly processes data acquired from users and constantly updates the state changes and information at that moment.

[0360] "External information sources" refer to data providers or platforms that exist outside the system and are sources of information such as rankings and trends.

[0361] A "generative model" refers to algorithms and technologies that use artificial intelligence to create new information and suggestions based on input data.

[0362] "Filtering" refers to the process of selecting data based on conditions or specific criteria, excluding unnecessary information, and extracting useful information.

[0363] This invention is a system that provides personalized suggestions based on the user's emotional state. It consists of a server, a terminal, and an emotion recognition engine.

[0364] The server stores attribute information and past behavioral history provided by users in a database and uses it to build user profiles. It also retrieves trend information and rankings from external sources and filters this information based on the user profile. Relational database management systems are commonly used for the databases, and SQL queries are utilized for filtering.

[0365] The device uses a camera and microphone to record user interaction and provides audio and video data to an emotion engine. This data is analyzed in real time by the emotion engine to recognize the user's emotional state. For example, it is common to use the OpenCV library for computer vision and a speech recognition API for speech analysis.

[0366] The emotion engine recognizes the user's emotional state, which is then sent to a server and input into a generative AI model. This generative model generates optimal suggestions based on the user's current emotional state and relevant filtered information. The generative AI model employs advanced natural language processing to create text-based prompts. For example, a prompt might ask, "What kind of music should be suggested if the user wants to relax in the afternoon?"

[0367] As a concrete example, if a user indicates a desire to relax via the camera and microphone on their device after returning home from work, the server inputs data analyzed by the emotion engine into a generative model, which then suggests music and relaxation techniques best suited for relaxation. In this way, the system can provide real-time suggestions tailored to the individual needs of each user.

[0368] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0369] Step 1:

[0370] The server collects attribute information and behavioral history entered by users through their devices. This data includes age, hobbies, and past usage history, and is stored in a database. The server uses SQL to organize the data and build user profiles. These profiles form the basis of the information.

[0371] Step 2:

[0372] The device uses its camera and microphone to collect audio and video data obtained during user interaction. The input data is sent to the emotion engine in real time. The emotion engine uses a computer vision library for video and a speech analysis API for audio to analyze the user's emotional state. As a result, a specific emotional state, such as happy or sad, is output.

[0373] Step 3:

[0374] The server receives emotional state data output from the emotion engine and combines it with profile data and trend information obtained from external sources. This data is then fed into a filtering algorithm to extract the most relevant information for the user. The output is then the relevant information.

[0375] Step 4:

[0376] The server inputs filtered relevant information and emotional state data into a generating AI model. The server generates prompt sentences, and based on these prompts, generates suggestions. As a result, personalized suggestions based on the user's emotions and profile are output. For example, it might generate suggestions for "music to recommend for relaxation."

[0377] Step 5:

[0378] The device notifies the user in real time of generated suggestions received from the server. Push notification technology is used for notifications, allowing users to receive suggestions immediately. This process involves the user reviewing the notified suggestions and, if necessary, viewing detailed information or taking action directly.

[0379] Step 6:

[0380] Users provide feedback on the suggestions offered. This feedback is collected via the device and sent to the server. The server analyzes this feedback and uses it to further improve the emotion engine and generative AI model. As a result, the overall accuracy of the system and the user experience improve.

[0381] (Application Example 2)

[0382] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0383] Modern users have diverse needs based on varying emotions and circumstances each day, making it difficult to provide them with timely and appropriate information and products. Many services are uniform, lacking personalized suggestions tailored to individual emotions and situations, thus increasing user satisfaction. Furthermore, providing emotionally-based suggestions, which offer services optimized for each individual user, has been difficult to achieve with conventional technologies.

[0384] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0385] In this invention, the server includes means for collecting user information, means for obtaining relevant information from multiple information sources, and means for analyzing emotional data. This makes it possible to suggest individually optimized products and services based on the user's emotions and to assist the user in purchasing products that suit their situation.

[0386] "User information" refers to data collected by the system about users to understand their preferences and attributes.

[0387] "Information source" refers to an external data provider that the system uses to obtain relevant information.

[0388] A "generative model" is a technical mechanism that uses collected data and information to organize it and present it to users in a meaningful way.

[0389] "Emotional data" refers to information used to analyze a user's current emotional state, and includes characteristics derived from facial expressions, voice, and other sources.

[0390] "Means of recommending the purchase of goods or services" refers to a function that presents suitable goods or services based on the user's emotions and circumstances.

[0391] "Means of assisting the purchase process" refers to a system designed to help users purchase proposed goods or services quickly and smoothly.

[0392] "Feedback" refers to the reactions and opinions received from users, which are used to improve the system.

[0393] "Partner providers" refers to external businesses or organizations that cooperate in providing information or products.

[0394] "Trend information and rankings" refer to information about current trends and popular topics, and are highly likely to attract users' interest.

[0395] "Filtering" is the process of selecting only the necessary information from collected data based on specific criteria.

[0396] The system implementing this invention runs via the user's smartphone, smart glasses, or other device. The server collects profile information provided by the user and builds a detailed user profile based on it. This profile includes personal attributes and past behavioral history. The server also utilizes APIs to obtain trend information and rankings from multiple partner sources, enabling the collection of relevant information.

[0397] The emotion recognition engine on the device uses the camera and microphone to recognize the user's emotions in real time. This engine analyzes the user's emotions from their voice tone and facial expressions, identifying states such as joy, sadness, and interest. This allows the device to determine the user's current emotional state.

[0398] Based on the user's emotions, the server uses a generative AI model to generate optimal suggestions. For example, if a user is feeling stressed, the system can recommend purchasing relaxation products or services. This suggestion also includes a function to assist with the purchase process so that the purchase can be made immediately.

[0399] As a concrete example, when the system determines that a user is feeling a little down, it suggests a discount coupon for a nearby spa and allows for immediate purchase. This process involves dynamic suggestion generation using a generative AI model.

[0400] An example of a prompt to input into the generation AI model is, "Generate specific suggestions for products and services that can help a user relax when they are tired."

[0401] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0402] Step 1:

[0403] The device sends user profile information to the server. This information includes user attribute data and behavioral history, and is used by the server to create a user profile. The input is the user's profile information, and the output is the user profile data within the server.

[0404] Step 2:

[0405] The server retrieves trend information and rankings from partner sources via API. This information is filtered based on the user profile to show only the most relevant data. The input is trend data from external sources, and the output is filtered information associated with the user profile.

[0406] Step 3:

[0407] The device's emotion recognition engine uses the camera and microphone to collect and analyze user emotion data in real time. The input is camera video and audio data, and the output is emotion data indicating the user's current emotional state.

[0408] Step 4:

[0409] The server uses a generative AI model to generate personalized suggestions based on the user's emotional data. This process takes the user profile and filtered information as input and outputs suggestions optimized for the user's emotions.

[0410] Step 5:

[0411] The terminal notifies the user of suggestions sent from the server. These notifications include information about products and services suitable for the user. The input is suggestion data from the server, and the output is a notification message to the user.

[0412] Step 6:

[0413] The user reviews the notified products and services and proceeds with the purchase if they decide to buy them. The server assists with the purchase process based on the results of the generated AI model. The input is the user's purchase intent data, and the output is confirmation data of the completed purchase.

[0414] Step 7:

[0415] The server collects user feedback and uses it to refine the generative model. This feedback is used to improve the accuracy of suggestions and sentiment recognition. The input is user feedback data, and the output is the refined generative model.

[0416] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0417] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0418] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0419] [Third Embodiment]

[0420] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0421] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0422] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0423] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0424] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0425] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0426] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0427] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0428] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0429] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0430] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0431] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0432] This invention is implemented as a system that allows users to efficiently receive information and services. This system provides users with optimized suggestions through the coordinated operation of the server, terminal, and user.

[0433] The server first collects user information. This includes data such as profile information and past behavioral history entered by the user through their device. The server then retrieves relevant information from various partner sources, extracting data that is particularly relevant to the user's profile. This includes data from ranking sites and companies that provide trend information.

[0434] Furthermore, the server utilizes a generative model to automatically organize and summarize the acquired information. This generative model is based on machine learning algorithms, learning patterns based on user interests and behaviors, and selecting the most relevant information. The summarized information is then structured as individual suggestions for each user.

[0435] The generated suggestions are notified to the user via their device. Notification methods include smartphone push notifications and in-app notifications, allowing users to quickly review the suggestions. After receiving the notification, users can view detailed information and take further action.

[0436] For example, if a user is interested in buying a home, the server can summarize suitable housing options and financial support plans based on the user's profile and notify them via their device. Based on this notification, the user can consider the suggested options and make the best choice.

[0437] This system collects user feedback, records and analyzes it on the server, and uses it to refine the generative model. The entire system is designed to be progressively improved so that future suggestions become more accurate. This process ensures that users always receive suggestions optimized for them.

[0438] The following describes the processing flow.

[0439] Step 1:

[0440] The device displays an interface where users can enter or update their profile information when they log into the application. Here, users can enter information such as their age, occupation, interests, and past purchase history.

[0441] Step 2:

[0442] The server receives user information sent from the terminal and securely stores it in an internal database. This database forms the basis for building user profiles.

[0443] Step 3:

[0444] The server retrieves data via APIs from partner information sources. This data includes information from ranking sites and companies that provide trend information.

[0445] Step 4:

[0446] The server filters the retrieved data based on the user's profile, selecting the most relevant information. In this process, it compares information from different data sources, prioritizing the most up-to-date and reliable data.

[0447] Step 5:

[0448] The server uses a generative AI model to organize and summarize the selected information. This generative model analyzes patterns in the information and generates summaries tailored to the user's interests.

[0449] Step 6:

[0450] The server then constructs individual suggestions based on the summarized information, which are stored as personalized content for each user.

[0451] Step 7:

[0452] The server notifies the device with personalized suggestions. These notifications are delivered via push notifications or in-app messaging, ensuring users can see them immediately.

[0453] Step 8:

[0454] Users can receive notifications on their devices and check their contents. By viewing the details, they can obtain more information about the suggested services and products.

[0455] Step 9:

[0456] When a user provides feedback, the device sends that information to the server. The server analyzes this information and uses it to improve the generated AI model, thereby increasing the accuracy of future suggestions.

[0457] (Example 1)

[0458] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0459] In recent years, in our information-saturated society, it has become difficult for users to efficiently acquire information and services that are relevant to them. To solve this problem, it is necessary to automatically generate personalized suggestions tailored to the user's preferences and behavioral patterns and deliver them at the appropriate time.

[0460] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0461] In this invention, the server includes means for collecting user information as product data, means for acquiring profile-related information from multiple information sources, and means for organizing and summarizing the acquired information using a generation AI model. This makes it possible to provide personalized information and services that meet the user's needs.

[0462] "Methods for accumulating user information as product data" refers to methods for collecting information that forms the basis for personalized recommendations by accumulating profile information and data on areas of interest entered by customers.

[0463] "Means of obtaining profile-related information from multiple sources" refers to methods of collecting information related to customer preferences and behavior using external databases or APIs.

[0464] "Methods for organizing and summarizing acquired information using a generative AI model" refers to a process of analyzing collected data using artificial intelligence technology to efficiently extract and summarize information that is beneficial to the customer.

[0465] "Means of constructing personalized service proposals" refers to a method of automatically generating recommendations for the most suitable products and services based on the client's profile information.

[0466] "Means of providing information to customers via terminals" refers to the process of notifying customers of collected and generated suggestions through terminal devices.

[0467] "A means of collecting customer evaluation data and adjusting the parameters of the generative model based on it" refers to a method of optimizing the generative AI model based on feedback provided by users to improve the accuracy of suggestions.

[0468] "A means of acquiring trend information and ranking data from external organizations and selecting information based on customer attribute information" refers to the process of obtaining the latest market trends and popularity rankings from partner companies and institutions and filtering them according to customer needs.

[0469] This invention provides a system that enables users to efficiently receive personalized information and services. The system operates through the coordinated roles of server, terminal, and user.

[0470] The server collects user information, including profile information and behavioral history entered by the user via their device. A relational database management system (RDBMS) is used for data management to ensure data consistency and security. The server also retrieves relevant information from other data sources via external APIs. This process involves collecting various types of information via RESTful APIs and parsing the data in JSON format. The collected information is then passed to a generative AI model. This AI model is a machine learning model based on the Transformer architecture, which learns from the user's past behavioral patterns and automatically summarizes relevant information.

[0471] The device is responsible for notifying the user of personalized suggestions received from the server. The application uses the smartphone's push notification function to deliver suggestions to the user in real time. By opening the notification, the user can view details of the suggested services or products and choose further action as needed.

[0472] Users input information into the system according to their circumstances and interests, and provide feedback on the suggestions and services they receive. This feedback is also analyzed on the server and used to improve the accuracy of future suggestions.

[0473] As a concrete example, when a user is considering purchasing a home, the server provides the user with the most suitable housing options and financial plans. An example of a prompt in this case would be the text, "I am currently considering purchasing a home. I would like to know about housing options and financial support plans that are right for me." Based on this prompt, the server collects the necessary data, summarizes the information using a generative AI model, and provides the user with optimized suggestions.

[0474] This system allows users to quickly receive personalized information and suggestions that always match their needs.

[0475] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0476] Step 1:

[0477] Users enter profile information and areas of interest through their devices. This includes age, product categories of interest, and past browsing history. This input data is sent to the server as foundational data to generate personalized suggestions for the user.

[0478] Step 2:

[0479] The server stores information submitted by users in a product database. The data is managed using an RDBMS to ensure security and data consistency. Input data is stored as a user profile and used for subsequent processing.

[0480] Step 3:

[0481] The server retrieves relevant information from multiple external sources. Specifically, it obtains data from trend information and ranking services via RESTful APIs and parses it in JSON format. This retrieved data is then processed to match each user's profile.

[0482] Step 4:

[0483] The server inputs the collected information into a generating AI model. This AI model uses a machine learning algorithm based on the Transformer architecture to analyze user behavior patterns and organize and summarize the information based on the results. Through this process, the input data is transformed into content that is useful to the user.

[0484] Step 5:

[0485] Based on the summary information output from the generative AI model, the server generates personalized recommendations. These include product lists and service options tailored to each user's interests. The generated recommendations are then prepared to be communicated to the user in the next step.

[0486] Step 6:

[0487] The device notifies the user of the suggestion information received from the server. Notifications are sent via smartphone push notifications or in-app notifications, allowing users to quickly review the suggestions.

[0488] Step 7:

[0489] Users can review the suggested information through their device and view detailed information. If necessary, they can take specific actions such as purchasing or inquiring about the suggested products or services.

[0490] Step 8:

[0491] Users submit feedback on the information and suggestions provided. The server collects this feedback and uses it as data to adjust the parameters of the generated AI model. This process improves the accuracy of future suggestions, allowing users to receive more relevant information.

[0492] (Application Example 1)

[0493] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0494] In urban life, there is a challenge in receiving new information and services quickly and individually. To solve this problem, a system is needed that efficiently delivers personalized suggestions based on the user's interests.

[0495] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0496] In this invention, the server includes means for collecting user information, means for obtaining relevant information from multiple information sources, and means for obtaining urban area information and providing local event information based on the user's interests. This makes it possible to provide personalized and rapid information and services optimized for the user.

[0497] "User information" refers to data that the system uses to optimize suggestions, including user profiles and behavioral data.

[0498] "Information sources" refer to external and internal data providers that partner organizations and systems refer to in order to collect data.

[0499] A "generative model" is a computer program that uses machine learning algorithms to organize and summarize information acquired according to the user's interests and behavior.

[0500] "Personalized suggestions" refer to customized information and service suggestions generated based on the user's profile and behavior.

[0501] A "smart city" is a general term for a city that collects and analyzes urban information and aims to provide services optimized for residents and visitors.

[0502] "Local event information" refers to data about events and activities held within a specific city or region.

[0503] This invention is implemented in smart cities as a system for users to receive optimized information and services. The server acquires data from devices such as smartphones and tablets to collect user information and stores that information on a cloud server. In terms of hardware, AWS or Microsoft Azure can be used for cloud services, and Amazon RDS or MongoDB can be used for databases.

[0504] The server retrieves relevant information from multiple sources, including data feeds from public institutions and trend information provided by private companies. This retrieved information is then organized and summarized on the server using generative AI models such as TensorFlow and PyTorch.

[0505] The AI ​​model analyzes user behavior patterns and interests, extracting the most relevant information and structuring it into personalized recommendations. These recommendations are notified in real time on the user's smart device, allowing the user to view detailed information and take further action as needed.

[0506] For example, if a user is interested in visiting art museums, the server analyzes the user's browsing history, organizes information on nearby art events, and provides it as the most suitable suggestion. Furthermore, by collecting feedback on this suggestion, the accuracy of future suggestions can be improved.

[0507] A concrete example of a prompt message would be: "Suggest information about art events currently taking place or scheduled in the area to users who are interested in art."

[0508] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0509] Step 1:

[0510] The device collects user profile and location information and sends it to the server. Input includes the user's basic data and current location, while output is the collected data sent to the server in JSON format.

[0511] Step 2:

[0512] The server stores received user information in a cloud database and simultaneously retrieves local event and general trend information from multiple publicly available sources. Inputs are user information and data from the sources, while output is a diverse list of retrieved information. Amazon RDS is commonly used for the database.

[0513] Step 3:

[0514] The server analyzes the collected information using a generating AI model and generates optimal suggestions based on the user's interests and behavior. In this step, user information and various event information stored in the database are handled as input, and the output is the generated, customized suggestions. TensorFlow is used for the AI ​​model.

[0515] Step 4:

[0516] The generated suggestions are sent from the server to the device via push notifications. The device receives this notification data as input and displays the suggestions visually to the user as output. The user interface is intuitive and is often developed using React Native.

[0517] Step 5:

[0518] The user reviews the suggested event information, selects those of interest, and learns more details. The input is the displayed list of suggestions, and the output is the details page for the selected event. Feedback based on this selection is also sent to the server.

[0519] Step 6:

[0520] The server records user feedback in a database and uses it to adjust the generating AI model to improve the accuracy of future suggestions. Inputs include user preferences and behavioral history, while output are updated model parameters.

[0521] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0522] The present invention is implemented as a system that recognizes a user's emotions and provides personalized suggestions based on those emotions. This system consists of a server, a terminal, and an emotion engine that analyzes the user's emotions.

[0523] The server collects profile information provided by the user through their device and builds a user profile based on this information. This profile includes personal attribute information and behavioral history. The server also obtains ranking and trend information from multiple partner sources via APIs. This data is filtered based on the user profile to select the most relevant information.

[0524] The emotion engine uses the camera and microphone to recognize the user's emotions in real time during user-device interaction. This engine can identify the user's current emotional state through voice tone and facial expression analysis. For example, it can identify various emotions such as happiness, sadness, or interest.

[0525] The server incorporates emotional data obtained from the emotion engine into a generative model and dynamically adjusts the suggested content according to the user's emotional state. This adjustment makes it possible to provide information that is most appropriate to the user's needs and emotions at the right time.

[0526] Individual suggestions are notified to the user via their device. Since notifications are made in real time, users can receive the most appropriate suggestions based on their current emotional state. For example, if a user is feeling stressed, they may receive suggestions for relaxation products or services. Furthermore, users who receive suggestions can view detailed information and take relevant actions immediately.

[0527] This system further collects user feedback and uses it to refine the emotion engine and generative models. The server analyzes the feedback and improves the overall accuracy of the system to better meet user needs. Through this process, users can always receive information optimized for their own emotions and needs.

[0528] The following describes the processing flow.

[0529] Step 1:

[0530] The device displays an interface for users to enter or update their profile information when they log into the application. Here, users can enter or update information such as age, occupation, interests, and past purchase history.

[0531] Step 2:

[0532] The server receives profile information sent from the user via their device and stores it in a database. This builds the user's profile.

[0533] Step 3:

[0534] The server retrieves data via APIs from partner information sources. This data includes information from ranking sites and companies that provide trend information.

[0535] Step 4:

[0536] The server filters the retrieved data based on the user's profile, selecting the most relevant information. This process is crucial for prioritizing the most up-to-date and reliable information.

[0537] Step 5:

[0538] The device uses its built-in camera and microphone to analyze voice tone and facial expressions during user interaction, and its emotion engine recognizes the user's emotions in real time.

[0539] Step 6:

[0540] The emotion engine sends recognized emotion data to the server, and based on this, the generative AI model dynamically adjusts the suggested content. This adjustment provides information optimized for the user's emotions.

[0541] Step 7:

[0542] The server generates personalized suggestions based on sentiment data and filtered information, and notifies the device of these suggestions. Notifications are sent via push notifications or in-app messages.

[0543] Step 8:

[0544] Users can receive notifications on their devices and, by reviewing the details, take appropriate action regarding the suggested products and services.

[0545] Step 9:

[0546] If a user provides feedback after using a suggestion, the device sends that feedback to the server. The server uses this information to further refine the sentiment engine and generative model, improving the system's accuracy.

[0547] (Example 2)

[0548] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0549] Traditional systems had the challenge of not being able to appropriately provide personalized suggestions that responded to the user's emotional state in real time. Furthermore, the feedback loop for improving the accuracy and relevance of suggestions to the user was not functioning adequately, resulting in a limited user experience.

[0550] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0551] In this invention, the server includes means for collecting information including user attribute information and behavioral history, storing it in a database, and building a profile; means for analyzing voice and video data to recognize the user's emotional state in real time; and means for obtaining relevant information from external sources and filtering it based on the profile. This makes it possible to provide appropriate suggestions in real time that meet the individual needs of the user.

[0552] "User attribute information" refers to identifiable data about a user, including basic personal characteristics such as age, gender, and hobbies.

[0553] "Behavioral history" refers to records of specific actions and habits a user has taken in the past, and is data that shows what choices and actions they have taken over time.

[0554] A "database" is a part of a computer system for systematically storing and managing information, and it has a structure that enables efficient data retrieval and manipulation.

[0555] A "profile" is a data set formed by integrating a user's attribute information and behavioral history, and it includes information that reflects the user's characteristics and interests.

[0556] "Emotional state" refers to the user's mental and emotional responses and conditions, and is measured through real-time audio and video data.

[0557] "Real-time recognition" refers to a process that instantly processes data acquired from users and constantly updates the state changes and information at that moment.

[0558] "External information sources" refer to data providers or platforms that exist outside the system and are sources of information such as rankings and trends.

[0559] A "generative model" refers to algorithms and technologies that use artificial intelligence to create new information and suggestions based on input data.

[0560] "Filtering" refers to the process of selecting data based on conditions or specific criteria, excluding unnecessary information, and extracting useful information.

[0561] This invention is a system that provides personalized suggestions based on the user's emotional state. It consists of a server, a terminal, and an emotion recognition engine.

[0562] The server stores attribute information and past behavioral history provided by users in a database and uses it to build user profiles. It also retrieves trend information and rankings from external sources and filters this information based on the user profile. Relational database management systems are commonly used for the databases, and SQL queries are utilized for filtering.

[0563] The device uses a camera and microphone to record user interaction and provides audio and video data to an emotion engine. This data is analyzed in real time by the emotion engine to recognize the user's emotional state. For example, it is common to use the OpenCV library for computer vision and a speech recognition API for speech analysis.

[0564] The emotion engine recognizes the user's emotional state, which is then sent to a server and input into a generative AI model. This generative model generates optimal suggestions based on the user's current emotional state and relevant filtered information. The generative AI model employs advanced natural language processing to create text-based prompts. For example, a prompt might ask, "What kind of music should be suggested if the user wants to relax in the afternoon?"

[0565] As a concrete example, if a user indicates a desire to relax via the camera and microphone on their device after returning home from work, the server inputs data analyzed by the emotion engine into a generative model, which then suggests music and relaxation techniques best suited for relaxation. In this way, the system can provide real-time suggestions tailored to the individual needs of each user.

[0566] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0567] Step 1:

[0568] The server collects attribute information and behavioral history entered by users through their devices. This data includes age, hobbies, and past usage history, and is stored in a database. The server uses SQL to organize the data and build user profiles. These profiles form the basis of the information.

[0569] Step 2:

[0570] The device uses its camera and microphone to collect audio and video data obtained during user interaction. The input data is sent to the emotion engine in real time. The emotion engine uses a computer vision library for video and a speech analysis API for audio to analyze the user's emotional state. As a result, a specific emotional state, such as happy or sad, is output.

[0571] Step 3:

[0572] The server receives emotional state data output from the emotion engine and combines it with profile data and trend information obtained from external sources. This data is then fed into a filtering algorithm to extract the most relevant information for the user. The output is then the relevant information.

[0573] Step 4:

[0574] The server inputs filtered relevant information and emotional state data into a generating AI model. The server generates prompt sentences, and based on these prompts, generates suggestions. As a result, personalized suggestions based on the user's emotions and profile are output. For example, it might generate suggestions for "music to recommend for relaxation."

[0575] Step 5:

[0576] The device notifies the user in real time of generated suggestions received from the server. Push notification technology is used for notifications, allowing users to receive suggestions immediately. This process involves the user reviewing the notified suggestions and, if necessary, viewing detailed information or taking action directly.

[0577] Step 6:

[0578] Users provide feedback on the suggestions offered. This feedback is collected via the device and sent to the server. The server analyzes this feedback and uses it to further improve the emotion engine and generative AI model. As a result, the overall accuracy of the system and the user experience improve.

[0579] (Application Example 2)

[0580] Next, we will explain Application Example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0581] Modern users have diverse needs based on varying emotions and circumstances each day, making it difficult to provide them with timely and appropriate information and products. Many services are uniform, lacking personalized suggestions tailored to individual emotions and situations, thus increasing user satisfaction. Furthermore, providing emotionally-based suggestions, which offer services optimized for each individual user, has been difficult to achieve with conventional technologies.

[0582] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0583] In this invention, the server includes means for collecting user information, means for obtaining relevant information from multiple information sources, and means for analyzing emotional data. This makes it possible to suggest individually optimized products and services based on the user's emotions and to assist the user in purchasing products that suit their situation.

[0584] "User information" refers to data collected by the system about users to understand their preferences and attributes.

[0585] "Information source" refers to an external data provider that the system uses to obtain relevant information.

[0586] A "generative model" is a technical mechanism that uses collected data and information to organize it and present it to users in a meaningful way.

[0587] "Emotional data" refers to information used to analyze a user's current emotional state, and includes characteristics derived from facial expressions, voice, and other sources.

[0588] "Means of recommending the purchase of goods or services" refers to a function that presents suitable goods or services based on the user's emotions and circumstances.

[0589] "Means of assisting the purchase process" refers to a system designed to help users purchase proposed goods or services quickly and smoothly.

[0590] "Feedback" refers to the reactions and opinions received from users, which are used to improve the system.

[0591] "Partner providers" refers to external businesses or organizations that cooperate in providing information or products.

[0592] "Trend information and rankings" refer to information about current trends and popular topics, and are highly likely to attract users' interest.

[0593] "Filtering" is the process of selecting only the necessary information from collected data based on specific criteria.

[0594] The system implementing this invention runs via the user's smartphone, smart glasses, or other device. The server collects profile information provided by the user and builds a detailed user profile based on it. This profile includes personal attributes and past behavioral history. The server also utilizes APIs to obtain trend information and rankings from multiple partner sources, enabling the collection of relevant information.

[0595] The emotion recognition engine on the device uses the camera and microphone to recognize the user's emotions in real time. This engine analyzes the user's emotions from their voice tone and facial expressions, identifying states such as joy, sadness, and interest. This allows the device to determine the user's current emotional state.

[0596] Based on the user's emotions, the server uses a generative AI model to generate optimal suggestions. For example, if a user is feeling stressed, the system can recommend purchasing relaxation products or services. This suggestion also includes a function to assist with the purchase process so that the purchase can be made immediately.

[0597] As a concrete example, when the system determines that a user is feeling a little down, it suggests a discount coupon for a nearby spa and allows for immediate purchase. This process involves dynamic suggestion generation using a generative AI model.

[0598] An example of a prompt to input into the generation AI model is, "Generate specific suggestions for products and services that can help a user relax when they are tired."

[0599] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0600] Step 1:

[0601] The device sends user profile information to the server. This information includes user attribute data and behavioral history, and is used by the server to create a user profile. The input is the user's profile information, and the output is the user profile data within the server.

[0602] Step 2:

[0603] The server retrieves trend information and rankings from partner sources via API. This information is filtered based on the user profile to show only the most relevant data. The input is trend data from external sources, and the output is filtered information associated with the user profile.

[0604] Step 3:

[0605] The device's emotion recognition engine uses the camera and microphone to collect and analyze user emotion data in real time. The input is camera video and audio data, and the output is emotion data indicating the user's current emotional state.

[0606] Step 4:

[0607] The server uses a generative AI model to generate personalized suggestions based on the user's emotional data. This process takes the user profile and filtered information as input and outputs suggestions optimized for the user's emotions.

[0608] Step 5:

[0609] The terminal notifies the user of suggestions sent from the server. These notifications include information about products and services suitable for the user. The input is suggestion data from the server, and the output is a notification message to the user.

[0610] Step 6:

[0611] The user reviews the notified products and services and proceeds with the purchase if they decide to buy them. The server assists with the purchase process based on the results of the generated AI model. The input is the user's purchase intent data, and the output is confirmation data of the completed purchase.

[0612] Step 7:

[0613] The server collects user feedback and uses it to refine the generative model. This feedback is used to improve the accuracy of suggestions and sentiment recognition. The input is user feedback data, and the output is the refined generative model.

[0614] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0615] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0616] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0617] [Fourth Embodiment]

[0618] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0619] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0620] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0621] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0622] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0623] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0624] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0625] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0626] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0627] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0628] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0629] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0630] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0631] This invention is implemented as a system that allows users to efficiently receive information and services. This system provides users with optimized suggestions through the coordinated operation of the server, terminal, and user.

[0632] The server first collects user information. This includes data such as profile information and past behavioral history entered by the user through their device. The server then retrieves relevant information from various partner sources, extracting data that is particularly relevant to the user's profile. This includes data from ranking sites and companies that provide trend information.

[0633] Furthermore, the server utilizes a generative model to automatically organize and summarize the acquired information. This generative model is based on machine learning algorithms, learning patterns based on user interests and behaviors, and selecting the most relevant information. The summarized information is then structured as individual suggestions for each user.

[0634] The generated suggestions are notified to the user via their device. Notification methods include smartphone push notifications and in-app notifications, allowing users to quickly review the suggestions. After receiving the notification, users can view detailed information and take further action.

[0635] For example, if a user is interested in buying a home, the server can summarize suitable housing options and financial support plans based on the user's profile and notify them via their device. Based on this notification, the user can consider the suggested options and make the best choice.

[0636] This system collects user feedback, records and analyzes it on the server, and uses it to refine the generative model. The entire system is designed to be progressively improved so that future suggestions become more accurate. This process ensures that users always receive suggestions optimized for them.

[0637] The following describes the processing flow.

[0638] Step 1:

[0639] The device displays an interface where users can enter or update their profile information when they log into the application. Here, users can enter information such as their age, occupation, interests, and past purchase history.

[0640] Step 2:

[0641] The server receives user information sent from the terminal and securely stores it in an internal database. This database forms the basis for building user profiles.

[0642] Step 3:

[0643] The server retrieves data via APIs from partner information sources. This data includes information from ranking sites and companies that provide trend information.

[0644] Step 4:

[0645] The server filters the retrieved data based on the user's profile, selecting the most relevant information. In this process, it compares information from different data sources, prioritizing the most up-to-date and reliable data.

[0646] Step 5:

[0647] The server uses a generative AI model to organize and summarize the selected information. This generative model analyzes patterns in the information and generates summaries tailored to the user's interests.

[0648] Step 6:

[0649] The server then constructs individual suggestions based on the summarized information, which are stored as personalized content for each user.

[0650] Step 7:

[0651] The server notifies the device with personalized suggestions. These notifications are delivered via push notifications or in-app messaging, ensuring users can see them immediately.

[0652] Step 8:

[0653] Users can receive notifications on their devices and check their contents. By viewing the details, they can obtain more information about the suggested services and products.

[0654] Step 9:

[0655] When a user provides feedback, the device sends that information to the server. The server analyzes this information and uses it to improve the generated AI model, thereby increasing the accuracy of future suggestions.

[0656] (Example 1)

[0657] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0658] In recent years, in our information-saturated society, it has become difficult for users to efficiently acquire information and services that are relevant to them. To solve this problem, it is necessary to automatically generate personalized suggestions tailored to the user's preferences and behavioral patterns and deliver them at the appropriate time.

[0659] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0660] In this invention, the server includes means for collecting user information as product data, means for acquiring profile-related information from multiple information sources, and means for organizing and summarizing the acquired information using a generation AI model. This makes it possible to provide personalized information and services that meet the user's needs.

[0661] "Methods for accumulating user information as product data" refers to methods for collecting information that forms the basis for personalized recommendations by accumulating profile information and data on areas of interest entered by customers.

[0662] "Means of obtaining profile-related information from multiple sources" refers to methods of collecting information related to customer preferences and behavior using external databases or APIs.

[0663] "Methods for organizing and summarizing acquired information using a generative AI model" refers to a process of analyzing collected data using artificial intelligence technology to efficiently extract and summarize information that is beneficial to the customer.

[0664] "Means of constructing personalized service proposals" refers to a method of automatically generating recommendations for the most suitable products and services based on the client's profile information.

[0665] "Means of providing information to customers via terminals" refers to the process of notifying customers of collected and generated suggestions through terminal devices.

[0666] "A means of collecting customer evaluation data and adjusting the parameters of the generative model based on it" refers to a method of optimizing the generative AI model based on feedback provided by users to improve the accuracy of suggestions.

[0667] "A means of acquiring trend information and ranking data from external organizations and selecting information based on customer attribute information" refers to the process of obtaining the latest market trends and popularity rankings from partner companies and institutions and filtering them according to customer needs.

[0668] This invention provides a system that enables users to efficiently receive personalized information and services. The system operates through the coordinated roles of server, terminal, and user.

[0669] The server collects user information, including profile information and behavioral history entered by the user via their device. A relational database management system (RDBMS) is used for data management to ensure data consistency and security. The server also retrieves relevant information from other data sources via external APIs. This process involves collecting various types of information via RESTful APIs and parsing the data in JSON format. The collected information is then passed to a generative AI model. This AI model is a machine learning model based on the Transformer architecture, which learns from the user's past behavioral patterns and automatically summarizes relevant information.

[0670] The device is responsible for notifying the user of personalized suggestions received from the server. The application uses the smartphone's push notification function to deliver suggestions to the user in real time. By opening the notification, the user can view details of the suggested services or products and choose further action as needed.

[0671] Users input information into the system according to their circumstances and interests, and provide feedback on the suggestions and services they receive. This feedback is also analyzed on the server and used to improve the accuracy of future suggestions.

[0672] As a concrete example, when a user is considering purchasing a home, the server provides the user with the most suitable housing options and financial plans. An example of a prompt in this case would be the text, "I am currently considering purchasing a home. I would like to know about housing options and financial support plans that are right for me." Based on this prompt, the server collects the necessary data, summarizes the information using a generative AI model, and provides the user with optimized suggestions.

[0673] This system allows users to quickly receive personalized information and suggestions that always match their needs.

[0674] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0675] Step 1:

[0676] Users enter profile information and areas of interest through their devices. This includes age, product categories of interest, and past browsing history. This input data is sent to the server as foundational data to generate personalized suggestions for the user.

[0677] Step 2:

[0678] The server stores information submitted by users in a product database. The data is managed using an RDBMS to ensure security and data consistency. Input data is stored as a user profile and used for subsequent processing.

[0679] Step 3:

[0680] The server retrieves relevant information from multiple external sources. Specifically, it obtains data from trend information and ranking services via RESTful APIs and parses it in JSON format. This retrieved data is then processed to match each user's profile.

[0681] Step 4:

[0682] The server inputs the collected information into a generating AI model. This AI model uses a machine learning algorithm based on the Transformer architecture to analyze user behavior patterns and organize and summarize the information based on the results. Through this process, the input data is transformed into content that is useful to the user.

[0683] Step 5:

[0684] Based on the summary information output from the generative AI model, the server generates personalized recommendations. These include product lists and service options tailored to each user's interests. The generated recommendations are then prepared to be communicated to the user in the next step.

[0685] Step 6:

[0686] The device notifies the user of the suggestion information received from the server. Notifications are sent via smartphone push notifications or in-app notifications, allowing users to quickly review the suggestions.

[0687] Step 7:

[0688] Users can review the suggested information through their device and view detailed information. If necessary, they can take specific actions such as purchasing or inquiring about the suggested products or services.

[0689] Step 8:

[0690] Users submit feedback on the information and suggestions provided. The server collects this feedback and uses it as data to adjust the parameters of the generated AI model. This process improves the accuracy of future suggestions, allowing users to receive more relevant information.

[0691] (Application Example 1)

[0692] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0693] In urban life, there is a challenge in receiving new information and services quickly and individually. To solve this problem, a system is needed that efficiently delivers personalized suggestions based on the user's interests.

[0694] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0695] In this invention, the server includes means for collecting user information, means for obtaining relevant information from multiple information sources, and means for obtaining urban area information and providing local event information based on the user's interests. This makes it possible to provide personalized and rapid information and services optimized for the user.

[0696] "User information" refers to data that the system uses to optimize suggestions, including user profiles and behavioral data.

[0697] "Information sources" refer to external and internal data providers that partner organizations and systems refer to in order to collect data.

[0698] A "generative model" is a computer program that uses machine learning algorithms to organize and summarize information acquired according to the user's interests and behavior.

[0699] "Personalized suggestions" refer to customized information and service suggestions generated based on the user's profile and behavior.

[0700] A "smart city" is a general term for a city that collects and analyzes urban information and aims to provide services optimized for residents and visitors.

[0701] "Local event information" refers to data about events and activities held within a specific city or region.

[0702] This invention is implemented in smart cities as a system for users to receive optimized information and services. The server acquires data from devices such as smartphones and tablets to collect user information and stores that information on a cloud server. In terms of hardware, AWS or Microsoft Azure can be used for cloud services, and Amazon RDS or MongoDB can be used for databases.

[0703] The server retrieves relevant information from multiple sources, including data feeds from public institutions and trend information provided by private companies. This retrieved information is then organized and summarized on the server using generative AI models such as TensorFlow and PyTorch.

[0704] The AI ​​model analyzes user behavior patterns and interests, extracting the most relevant information and structuring it into personalized recommendations. These recommendations are notified in real time on the user's smart device, allowing the user to view detailed information and take further action as needed.

[0705] For example, if a user is interested in visiting art museums, the server analyzes the user's browsing history, organizes information on nearby art events, and provides it as the most suitable suggestion. Furthermore, by collecting feedback on this suggestion, the accuracy of future suggestions can be improved.

[0706] A concrete example of a prompt message would be: "Suggest information about art events currently taking place or scheduled in the area to users who are interested in art."

[0707] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0708] Step 1:

[0709] The device collects user profile and location information and sends it to the server. Input includes the user's basic data and current location, while output is the collected data sent to the server in JSON format.

[0710] Step 2:

[0711] The server stores received user information in a cloud database and simultaneously retrieves local event and general trend information from multiple publicly available sources. Inputs are user information and data from the sources, while output is a diverse list of retrieved information. Amazon RDS is commonly used for the database.

[0712] Step 3:

[0713] The server analyzes the collected information using a generating AI model and generates optimal suggestions based on the user's interests and behavior. In this step, user information and various event information stored in the database are handled as input, and the output is the generated, customized suggestions. TensorFlow is used for the AI ​​model.

[0714] Step 4:

[0715] The generated suggestions are sent from the server to the device via push notifications. The device receives this notification data as input and displays the suggestions visually to the user as output. The user interface is intuitive and is often developed using React Native.

[0716] Step 5:

[0717] The user reviews the suggested event information, selects those of interest, and learns more details. The input is the displayed list of suggestions, and the output is the details page for the selected event. Feedback based on this selection is also sent to the server.

[0718] Step 6:

[0719] The server records user feedback in a database and uses it to adjust the generating AI model to improve the accuracy of future suggestions. Inputs include user preferences and behavioral history, while output are updated model parameters.

[0720] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0721] The present invention is implemented as a system that recognizes a user's emotions and provides personalized suggestions based on those emotions. This system consists of a server, a terminal, and an emotion engine that analyzes the user's emotions.

[0722] The server collects profile information provided by the user through their device and builds a user profile based on this information. This profile includes personal attribute information and behavioral history. The server also obtains ranking and trend information from multiple partner sources via APIs. This data is filtered based on the user profile to select the most relevant information.

[0723] The emotion engine uses the camera and microphone to recognize the user's emotions in real time during user-device interaction. This engine can identify the user's current emotional state through voice tone and facial expression analysis. For example, it can identify various emotions such as happiness, sadness, or interest.

[0724] The server incorporates emotional data obtained from the emotion engine into a generative model and dynamically adjusts the suggested content according to the user's emotional state. This adjustment makes it possible to provide information that is most appropriate to the user's needs and emotions at the right time.

[0725] Individual suggestions are notified to the user via their device. Since notifications are made in real time, users can receive the most appropriate suggestions based on their current emotional state. For example, if a user is feeling stressed, they may receive suggestions for relaxation products or services. Furthermore, users who receive suggestions can view detailed information and take relevant actions immediately.

[0726] This system further collects user feedback and uses it to refine the emotion engine and generative models. The server analyzes the feedback and improves the overall accuracy of the system to better meet user needs. Through this process, users can always receive information optimized for their own emotions and needs.

[0727] The following describes the processing flow.

[0728] Step 1:

[0729] The device displays an interface for users to enter or update their profile information when they log into the application. Here, users can enter or update information such as age, occupation, interests, and past purchase history.

[0730] Step 2:

[0731] The server receives profile information sent from the user via their device and stores it in a database. This builds the user's profile.

[0732] Step 3:

[0733] The server retrieves data via APIs from partner information sources. This data includes information from ranking sites and companies that provide trend information.

[0734] Step 4:

[0735] The server filters the retrieved data based on the user's profile, selecting the most relevant information. This process is crucial for prioritizing the most up-to-date and reliable information.

[0736] Step 5:

[0737] The device uses its built-in camera and microphone to analyze voice tone and facial expressions during user interaction, and its emotion engine recognizes the user's emotions in real time.

[0738] Step 6:

[0739] The emotion engine sends recognized emotion data to the server, and based on this, the generative AI model dynamically adjusts the suggested content. This adjustment provides information optimized for the user's emotions.

[0740] Step 7:

[0741] The server generates personalized suggestions based on sentiment data and filtered information, and notifies the device of these suggestions. Notifications are sent via push notifications or in-app messages.

[0742] Step 8:

[0743] Users can receive notifications on their devices and, by reviewing the details, take appropriate action regarding the suggested products and services.

[0744] Step 9:

[0745] If a user provides feedback after using a suggestion, the device sends that feedback to the server. The server uses this information to further refine the sentiment engine and generative model, improving the system's accuracy.

[0746] (Example 2)

[0747] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0748] Traditional systems had the challenge of not being able to appropriately provide personalized suggestions that responded to the user's emotional state in real time. Furthermore, the feedback loop for improving the accuracy and relevance of suggestions to the user was not functioning adequately, resulting in a limited user experience.

[0749] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0750] In this invention, the server includes means for collecting information including user attribute information and behavioral history, storing it in a database, and building a profile; means for analyzing voice and video data to recognize the user's emotional state in real time; and means for obtaining relevant information from external sources and filtering it based on the profile. This makes it possible to provide appropriate suggestions in real time that meet the individual needs of the user.

[0751] "User attribute information" refers to identifiable data about a user, including basic personal characteristics such as age, gender, and hobbies.

[0752] "Behavioral history" refers to records of specific actions and habits a user has taken in the past, and is data that shows what choices and actions they have taken over time.

[0753] A "database" is a part of a computer system for systematically storing and managing information, and it has a structure that enables efficient data retrieval and manipulation.

[0754] A "profile" is a data set formed by integrating a user's attribute information and behavioral history, and it includes information that reflects the user's characteristics and interests.

[0755] "Emotional state" refers to the user's mental and emotional responses and conditions, and is measured through real-time audio and video data.

[0756] "Real-time recognition" refers to a process that instantly processes data acquired from users and constantly updates the state changes and information at that moment.

[0757] "External information sources" refer to data providers or platforms that exist outside the system and are sources of information such as rankings and trends.

[0758] A "generative model" refers to algorithms and technologies that use artificial intelligence to create new information and suggestions based on input data.

[0759] "Filtering" refers to the process of selecting data based on conditions or specific criteria, excluding unnecessary information, and extracting useful information.

[0760] This invention is a system that provides personalized suggestions based on the user's emotional state. It consists of a server, a terminal, and an emotion recognition engine.

[0761] The server stores attribute information and past behavioral history provided by users in a database and uses it to build user profiles. It also retrieves trend information and rankings from external sources and filters this information based on the user profile. Relational database management systems are commonly used for the databases, and SQL queries are utilized for filtering.

[0762] The device uses a camera and microphone to record user interaction and provides audio and video data to an emotion engine. This data is analyzed in real time by the emotion engine to recognize the user's emotional state. For example, it is common to use the OpenCV library for computer vision and a speech recognition API for speech analysis.

[0763] The emotion engine recognizes the user's emotional state, which is then sent to a server and input into a generative AI model. This generative model generates optimal suggestions based on the user's current emotional state and relevant filtered information. The generative AI model employs advanced natural language processing to create text-based prompts. For example, a prompt might ask, "What kind of music should be suggested if the user wants to relax in the afternoon?"

[0764] As a concrete example, if a user indicates a desire to relax via the camera and microphone on their device after returning home from work, the server inputs data analyzed by the emotion engine into a generative model, which then suggests music and relaxation techniques best suited for relaxation. In this way, the system can provide real-time suggestions tailored to the individual needs of each user.

[0765] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0766] Step 1:

[0767] The server collects attribute information and behavioral history entered by users through their devices. This data includes age, hobbies, and past usage history, and is stored in a database. The server uses SQL to organize the data and build user profiles. These profiles form the basis of the information.

[0768] Step 2:

[0769] The device uses its camera and microphone to collect audio and video data obtained during user interaction. The input data is sent to the emotion engine in real time. The emotion engine uses a computer vision library for video and a speech analysis API for audio to analyze the user's emotional state. As a result, a specific emotional state, such as happy or sad, is output.

[0770] Step 3:

[0771] The server receives emotional state data output from the emotion engine and combines it with profile data and trend information obtained from external sources. This data is then fed into a filtering algorithm to extract the most relevant information for the user. The output is then the relevant information.

[0772] Step 4:

[0773] The server inputs filtered relevant information and emotional state data into a generating AI model. The server generates prompt sentences, and based on these prompts, generates suggestions. As a result, personalized suggestions based on the user's emotions and profile are output. For example, it might generate suggestions for "music to recommend for relaxation."

[0774] Step 5:

[0775] The device notifies the user in real time of generated suggestions received from the server. Push notification technology is used for notifications, allowing users to receive suggestions immediately. This process involves the user reviewing the notified suggestions and, if necessary, viewing detailed information or taking action directly.

[0776] Step 6:

[0777] Users provide feedback on the suggestions offered. This feedback is collected via the device and sent to the server. The server analyzes this feedback and uses it to further improve the emotion engine and generative AI model. As a result, the overall accuracy of the system and the user experience improve.

[0778] (Application Example 2)

[0779] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0780] Modern users have diverse needs based on varying emotions and circumstances each day, making it difficult to provide them with timely and appropriate information and products. Many services are uniform, lacking personalized suggestions tailored to individual emotions and situations, thus increasing user satisfaction. Furthermore, providing emotionally-based suggestions, which offer services optimized for each individual user, has been difficult to achieve with conventional technologies.

[0781] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0782] In this invention, the server includes means for collecting user information, means for obtaining relevant information from multiple information sources, and means for analyzing emotional data. This makes it possible to suggest individually optimized products and services based on the user's emotions and to assist the user in purchasing products that suit their situation.

[0783] "User information" refers to data collected by the system about users to understand their preferences and attributes.

[0784] "Information source" refers to an external data provider that the system uses to obtain relevant information.

[0785] A "generative model" is a technical mechanism that uses collected data and information to organize it and present it to users in a meaningful way.

[0786] "Emotional data" refers to information used to analyze a user's current emotional state, and includes characteristics derived from facial expressions, voice, and other sources.

[0787] "Means of recommending the purchase of goods or services" refers to a function that presents suitable goods or services based on the user's emotions and circumstances.

[0788] "Means of assisting the purchase process" refers to a system designed to help users purchase proposed goods or services quickly and smoothly.

[0789] "Feedback" refers to the reactions and opinions received from users, which are used to improve the system.

[0790] "Partner providers" refers to external businesses or organizations that cooperate in providing information or products.

[0791] "Trend information and rankings" refer to information about current trends and popular topics, and are highly likely to attract users' interest.

[0792] "Filtering" is the process of selecting only the necessary information from collected data based on specific criteria.

[0793] The system implementing this invention runs via the user's smartphone, smart glasses, or other device. The server collects profile information provided by the user and builds a detailed user profile based on it. This profile includes personal attributes and past behavioral history. The server also utilizes APIs to obtain trend information and rankings from multiple partner sources, enabling the collection of relevant information.

[0794] The emotion recognition engine on the device uses the camera and microphone to recognize the user's emotions in real time. This engine analyzes the user's emotions from their voice tone and facial expressions, identifying states such as joy, sadness, and interest. This allows the device to determine the user's current emotional state.

[0795] Based on the user's emotions, the server uses a generative AI model to generate optimal suggestions. For example, if a user is feeling stressed, the system can recommend purchasing relaxation products or services. This suggestion also includes a function to assist with the purchase process so that the purchase can be made immediately.

[0796] As a concrete example, when the system determines that a user is feeling a little down, it suggests a discount coupon for a nearby spa and allows for immediate purchase. This process involves dynamic suggestion generation using a generative AI model.

[0797] An example of a prompt to input into the generation AI model is, "Generate specific suggestions for products and services that can help a user relax when they are tired."

[0798] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0799] Step 1:

[0800] The device sends user profile information to the server. This information includes user attribute data and behavioral history, and is used by the server to create a user profile. The input is the user's profile information, and the output is the user profile data within the server.

[0801] Step 2:

[0802] The server retrieves trend information and rankings from partner sources via API. This information is filtered based on the user profile to show only the most relevant data. The input is trend data from external sources, and the output is filtered information associated with the user profile.

[0803] Step 3:

[0804] The device's emotion recognition engine uses the camera and microphone to collect and analyze user emotion data in real time. The input is camera video and audio data, and the output is emotion data indicating the user's current emotional state.

[0805] Step 4:

[0806] The server uses a generative AI model to generate personalized suggestions based on the user's emotional data. This process takes the user profile and filtered information as input and outputs suggestions optimized for the user's emotions.

[0807] Step 5:

[0808] The terminal notifies the user of suggestions sent from the server. These notifications include information about products and services suitable for the user. The input is suggestion data from the server, and the output is a notification message to the user.

[0809] Step 6:

[0810] The user reviews the notified products and services and proceeds with the purchase if they decide to buy them. The server assists with the purchase process based on the results of the generated AI model. The input is the user's purchase intent data, and the output is confirmation data of the completed purchase.

[0811] Step 7:

[0812] The server collects user feedback and uses it to refine the generative model. This feedback is used to improve the accuracy of suggestions and sentiment recognition. The input is user feedback data, and the output is the refined generative model.

[0813] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0814] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0815] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0816] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0817] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0818] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0819] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0820] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0821] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0822] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0823] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0824] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0825] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0826] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0827] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0828] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0829] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0830] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0831] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0832] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0833] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0834] The following is further disclosed regarding the embodiments described above.

[0835] (Claim 1)

[0836] Means for collecting user information,

[0837] Means of obtaining relevant information from multiple sources,

[0838] A means of organizing and summarizing acquired information using a generative model,

[0839] A means of generating individual proposals based on summarized information,

[0840] A means of notifying the user of the generated suggestions,

[0841] A system that includes this.

[0842] (Claim 2)

[0843] The system according to claim 1, further comprising means for collecting user feedback and adjusting the generative model based thereon.

[0844] (Claim 3)

[0845] The system according to claim 1, further comprising means for obtaining trend information and rankings from partner companies and filtering them based on the user's profile.

[0846] "Example 1"

[0847] (Claim 1)

[0848] A means of collecting user information as product data,

[0849] Means for obtaining profile-related information from multiple sources,

[0850] A means of organizing and summarizing acquired information using a generative AI model,

[0851] Means for constructing personalized service proposals based on summarized information,

[0852] A means of providing the configured proposal to the customer via a terminal,

[0853] A system that includes this.

[0854] (Claim 2)

[0855] The system according to claim 1, further comprising means for collecting customer evaluation data and adjusting the parameters of a generative model based on that data.

[0856] (Claim 3)

[0857] The system according to claim 1, further comprising means for importing trend information and ranking data from external organizations and selecting information based on customer attribute information.

[0858] "Application Example 1"

[0859] (Claim 1)

[0860] Means for collecting user information,

[0861] Means of obtaining relevant information from multiple sources,

[0862] A means of organizing and summarizing acquired information using a generative model,

[0863] A means of generating individual proposals based on summarized information,

[0864] A means of notifying the user of the generated suggestions,

[0865] This method also acquires information about urban areas and provides local event information based on user interests.

[0866] A system that includes this.

[0867] (Claim 2)

[0868] The system according to claim 1, further comprising means for collecting user feedback and adjusting the generative model based thereon.

[0869] (Claim 3)

[0870] The system according to claim 1, further comprising means for obtaining trend information and rankings from partner organizations and filtering them based on the user's profile.

[0871] "Example 2 of combining an emotion engine"

[0872] (Claim 1)

[0873] A means of collecting information including user attribute information and behavioral history, storing it in a database, and building a profile,

[0874] A means of analyzing audio and video data to recognize the user's emotional state in real time,

[0875] A means of obtaining relevant information from external sources and filtering it based on a profile,

[0876] A means for inputting acquired information and emotional states into a generative model to generate dynamically adjusted individual suggestions,

[0877] A means of notifying the user's device in real time of the generated suggestions,

[0878] A system that includes this.

[0879] (Claim 2)

[0880] The system according to claim 1, further comprising means for collecting user feedback and utilizing it to adjust emotion recognition means and generative models.

[0881] (Claim 3)

[0882] The system according to claim 1, comprising means for obtaining trend information and rankings from external sources, and for evaluating and filtering relevance based on user profiles.

[0883] "Application example 2 of combining emotional engines"

[0884] (Claim 1)

[0885] Means for collecting user information,

[0886] Means of obtaining relevant information from multiple sources,

[0887] A means of organizing and summarizing acquired information using a generative model,

[0888] A means of generating individual proposals based on summarized information,

[0889] A means of notifying the user of the generated suggestions,

[0890] Methods for analyzing emotional data,

[0891] A means of recommending the purchase of a product or service based on emotional data,

[0892] Means to assist with the purchase process,

[0893] A system that includes this.

[0894] (Claim 2)

[0895] The system according to claim 1, further comprising means for collecting user feedback and adjusting the generative model based thereon, and means for improving the accuracy of suggestions based on sentiment data.

[0896] (Claim 3)

[0897] The system according to claim 1, further comprising means for obtaining trend information and rankings from partner providers, filtering them based on the user's profile, and generating suggestions that take into account the user's emotional state. [Explanation of Symbols]

[0898] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. Means for collecting user information, Means of obtaining relevant information from multiple sources, A means of organizing and summarizing acquired information using a generative model, A means of generating individual proposals based on summarized information, A means of notifying the user of the generated suggestions, A system that includes this.

2. The system according to claim 1, further comprising means for collecting user feedback and adjusting the generative model based thereon.

3. The system according to claim 1, further comprising means for obtaining trend information and rankings from partner companies and filtering them based on the user's profile.

Images