system

Abstract

Provide a system. A means for collecting user information; A means of integrating and preprocessing various data; A means of analyzing user preferences and behavior patterns; A means of recommending products and services suitable for users; a means for customizing the user interface; A means to provide real-time feedback on system usage and update the model; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In modern society, many users use different applications and services, which means that data collected by each service is scattered, making it difficult to consistently understand users' preferences and behavioral patterns across each service. Furthermore, there is a lack of appropriate recommendations to provide users with products and services they prefer, which results in users having to spend a lot of time finding the information and services they need. [Means for solving the problem]

[0005] The present invention solves the above problems by providing a system that includes a means for collecting user information, a means for integrating and preprocessing various data, a means for analyzing user preferences and behavioral patterns, a means for recommending products and services suitable for users, a means for customizing the user interface, and a means for providing real-time feedback on system usage and updating the model. Specifically, the system centrally collects communication data, purchase data, and search data, and analyzes user preferences and behavioral patterns using machine learning algorithms to recommend appropriate products and services and provide a dynamically updated customized user interface.

[0006] "User information" refers to personal data and behavioral data about users, and specifically includes communication history, purchase history, search history, location information, etc.

[0007] "Data integration" refers to the process of bringing together data collected from different formats and sources into a single database and making it consistent.

[0008] "Preprocessing" refers to processes such as deleting duplicate data, correcting inconsistent data, and filling in missing data, which are carried out to maintain data consistency.

[0009] "Analysis of preferences and behavioral patterns" refers to the process of analyzing a user's past behavior and interests using machine learning algorithms, etc., to derive the user's characteristics and hobby trends.

[0010] "Product and service recommendation" means proposing the most suitable products and services to each user based on the results of an analysis of the user's preferences and behavioral patterns.

[0011] "User interface customization" refers to dynamically changing the display content and application layout to optimize it for the user.

[0012] "Real-time feedback" refers to the process by which the system instantly collects the latest behavioral data of users and uses this data to improve the entire system.

[0013] "Model updating" refers to retraining a machine learning algorithm based on new data and applying the results to the system to improve recommendation accuracy and system performance. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0015] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0018] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0019] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0020] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0022] [First embodiment]

[0023] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0024] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0027] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0030] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0034] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0035] This system collects user information, integrates and preprocesses various data, analyzes user preferences and behavioral patterns, and recommends personalized products and services. Furthermore, it customizes the user interface based on the recommendation results, provides real-time feedback on system usage, and updates the model to provide the optimal experience for users.

[0036] User Data Collection

[0037] First, the server collects user information. This information is collected from a wide range of data sources, including communication data, purchase data, and search data. For example, communication data may include what applications the user is using and how much data traffic they are consuming. Purchase data may include what store the user purchased what and how much they paid. Search data may include what keywords the user used in their search and which web pages they viewed.

[0038] Data integration and preprocessing

[0039] The server centrally integrates the collected data into a database, converting data from different formats and sources into a consistent format, removing duplicate data, correcting inconsistencies, and properly filling in missing data.

[0040] Analysis of user preferences and behavioral patterns

[0041] The server then applies machine learning algorithms to the preprocessed data to analyze user preferences and behavioral patterns. Using past data, the server identifies the types of products and services a user is interested in and the behavioral patterns they exhibit. For example, it identifies patterns such as whether a user frequently goes to the movies on weekends or regularly purchases a particular brand of product.

[0042] Personalized product and service recommendations

[0043] The server then uses the analysis results to recommend the most suitable products and services for each user. When recommending products and services, the server takes into account the user's past behavioral data and preferences to select items that are likely to interest the user. For example, if a user is interested in movies, the server will recommend the latest movie tickets and movie-related goods.

[0044] Customizing the app's user interface

[0045] The device uses the recommendation list received from the server to customize how the app is displayed. Specifically, it adds the most relevant apps and links as shortcuts to the home screen. For example, it displays links to buy movie tickets or offers from nearby restaurants. It also rearranges apps so that when you open them, you can quickly find content that interests you.

[0046] Real-time feedback and model updates

[0047] The server monitors users' app usage in real time, collecting new data to retrain the machine learning model and improve the accuracy of the recommendation system. For example, if a user takes an action that indicates a new preference, that behavioral data is immediately collected and reflected in the next recommendation.

[0048] Specific Examples

[0049] For example, User A uses communication services, makes daily purchases using an electronic payment service, and uses a search engine to get the latest movie information. The server collects this data and analyzes the user's preferences and behavioral patterns. If User A is determined to have a high interest in movies, the server recommends the latest movie ticket information and coupons for theaters showing movies in the user's area. The user interface is customized accordingly, and movie-related links and apps are displayed on the home screen. Real-time feedback is also provided to immediately reflect any new content or services that User A has become interested in. In this way, users not only receive products and services that are optimized for them, but also use a convenient interface based on those products and services.

[0050] As described above, the present invention realizes a system that provides users with the most optimal experience by aggregating and analyzing a wide range of user data and recommending personalized products and services.

[0051] The processing flow will be explained below.

[0052] Step 1:

[0053] The server uses the means for collecting user information to obtain a wide range of data, including communication data, purchase data, and search data, through various APIs. Specifically, the server collects the user's communication history from the API of a communication service provider, purchase history from the API of an electronic payment service, and search history from the API of a search engine.

[0054] Step 2:

[0055] The server consolidates and pre-processes the collected data. First, it converts data from different formats into a consistent format. Second, it removes duplicate data, corrects inconsistent data, and imputes missing data where necessary. This creates a clean and consistent dataset.

[0056] Step 3:

[0057] The server stores the pre-processed data in a database, which is used for subsequent analysis and recommendation processing. The database stores data by category, such as communication data, purchase data, and search data.

[0058] Step 4:

[0059] The server uses the data stored in the database to run machine learning algorithms, such as clustering and classification algorithms, to analyze user preferences and behavioral patterns. For example, a user who frequently searches for movies might be classified as a movie lover.

[0060] Step 5:

[0061] The server then uses the results of the machine learning algorithm to create a user profile, which includes the user's interests, behavioral patterns, purchase history, etc. This profile is then used to make personalized recommendations of products and services.

[0062] Step 6:

[0063] The server then applies an algorithm to recommend products and services based on the created profile. For example, if a user has a profile for movie lovers, the server will recommend the latest movie tickets and movie-related merchandise. The recommendation results are generated in the form of a list and sent to the user's device.

[0064] Step 7:

[0065] The device analyzes the recommendation list received from the server and dynamically customizes the user interface, adding the most relevant apps and links as shortcuts to the home screen for quick access, such as a movie ticket purchasing app or nearby restaurant offers.

[0066] Step 8:

[0067] The server monitors users' app usage in real time and collects new data, including which recommended items users click on and which products they end up purchasing, to help refine future recommendations.

[0068] Step 9:

[0069] The server retrains the machine learning model based on the new data collected, improving the accuracy of the recommendation system. Any data that indicates new preferences or behavioral patterns is immediately reflected in the model and reflected in the next recommendation results.

[0070] Step 10:

[0071] The server then applies the updated model to the user profile and re-recommends products and services. In this way, the system continues to provide optimized services based on the latest user information.

[0072] By following these steps, users can easily find products and services that match their preferences and behavioral patterns, and can access the information and services they need with just one smartphone.

[0073] Example 1

[0074] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0075] In today's information society, consumers are bombarded with a vast amount of information, making it difficult to select optimal products and services. Furthermore, existing recommendation systems often cannot adequately reflect individual user preferences and behavioral patterns, and can only provide generalized recommendations. Furthermore, there is a need for highly accurate recommendations that reflect changes in user preferences and behavior in real time. An efficient and effective system is needed to solve these problems.

[0076] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0077] In this invention, the server includes a means for collecting user information, a means for integrating and preprocessing various data, and a means for analyzing user preferences and behavioral patterns, which enables highly accurate analysis of user preferences and behavioral patterns and the recommendation of optimal products and services to users.

[0078] "Means for collecting user information" refers to processes and devices for collecting various data related to users, such as communication data, purchase data, and search data.

[0079] "Means of integrating and pre-processing various data" refers to processes or devices that convert data collected from different data sources into a consistent format, remove duplicate data, correct inconsistent data, and fill in missing data.

[0080] "Means for analyzing user preferences and behavioral patterns" refers to processes or devices that apply machine learning algorithms or other analytical techniques to the collected and pre-processed data to identify user preferences and behavioral patterns.

[0081] "Means for recommending products and services suitable for users" refers to the process or device that selects and presents the most suitable products and services based on the analyzed preferences and behavioral patterns of users.

[0082] "Means for customizing user interfaces" refers to processes or devices that change the display content and operation methods of applications and systems based on user preferences and behavioral patterns, making them easier for users to use.

[0083] "Means for providing real-time feedback on system usage and updating the model" refers to the process or device that monitors and collects data on users' use of the system in real time, and uses that data to retrain the machine learning model, thereby improving the accuracy of the system's recommendations.

[0084] "Means for encrypting data received from a terminal and sending it to a server" refers to a process or device for encrypting data collected from a user's terminal and securely sending it to a server.

[0085] "Means of transforming data from different formats into a consistent format" refers to a process or device that transforms data provided in different formats (e.g., JSON, XML) into a unified format.

[0086] "Means for applying machine learning algorithms to analyze user preferences and behavioral patterns" refers to a process or device that uses machine learning techniques to identify user preferences and behavioral patterns from pre-processed data.

[0087] System Overview

[0088] This system collects user information, integrates and preprocesses various data, analyzes user preferences and behavioral patterns, and recommends personalized products and services. Furthermore, it customizes the user interface based on the recommendation results, provides real-time feedback on system usage, and updates the model to provide the optimal experience for users.

[0089] User Data Collection

[0090] First, the server collects user information. This information is collected from a wide range of data sources, including communication data, purchase data, and search data. For example, communication data may include what applications the user is using and how much data traffic they are consuming. Purchase data may include what store the user purchased what and how much they paid. Search data may include what keywords the user used in their search and which web pages they viewed.

[0091] Data integration and preprocessing

[0092] The server centrally integrates the collected data into a database (e.g., MongoDB). During this process, it converts data obtained from different data formats and sources into a consistent format (e.g., JSON), removes duplicate data, corrects inconsistent data, and appropriately complements any missing data. Data conversion is performed using a data conversion library (e.g., Pandas).

[0093] Analysis of user preferences and behavioral patterns

[0094] The server then applies machine learning algorithms (e.g., k-means clustering, random forest) to the preprocessed data to analyze user preferences and behavioral patterns. Using past data, the server identifies what products and services users are interested in and what behavioral patterns they exhibit. For example, it identifies patterns such as whether a user often goes to the movies on weekends or regularly purchases a particular brand of product.

[0095] Personalized product and service recommendations

[0096] The server then uses the analysis results to recommend the most suitable products and services for each user. When recommending products and services, the server takes into account the user's past behavioral data and preferences to select items that are likely to interest the user. For example, if a user is interested in movies, the server will recommend the latest movie tickets and movie-related goods.

[0097] Customizing the app's user interface

[0098] The device uses the recommendation list received from the server to customize how the app is displayed. Specifically, it adds the most relevant apps and links as shortcuts to the home screen. For example, it displays links to buy movie tickets or offers from nearby restaurants. It also rearranges apps so that when you open them, you can quickly find content that interests you.

[0099] Real-time feedback and model updates

[0100] The server monitors users' app usage in real time, collecting new data to retrain the machine learning model and improve the accuracy of the recommendation system. For example, if a user takes an action that indicates a new preference, that behavioral data is immediately collected and reflected in the next recommendation.

[0101] Specific Examples

[0102] For example, User A uses communication services, makes daily purchases using an electronic payment service, and uses a search engine to get the latest movie information. The server collects this data and analyzes the user's preferences and behavioral patterns. If User A is determined to have a high interest in movies, the server recommends the latest movie ticket information and coupons for theaters showing movies in the user's area. The user interface is customized accordingly, and movie-related links and apps are displayed on the home screen. Real-time feedback is also provided to immediately reflect any new content or services that User A has become interested in. In this way, users not only receive products and services that are optimized for them, but also use a convenient interface based on those products and services.

[0103] Prompt Sentence Examples

[0104] "User A uses a communication service and frequently searches for movie-related information. Please recommend the best movie-related products and services for this user."

[0105] As described above, the present invention is a system that provides users with an optimal experience by aggregating and analyzing a wide range of user data and recommending personalized products and services.

[0106] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0107] Step 1: Collect user data

[0108] The server collects communication data, purchase data, search data, etc. from the device used by the user. This data is acquired in real time while the user is actually using the application. Specifically, when a user makes a purchase, the purchase information (product ID, price, purchase date, etc.) is recorded. Communication data includes which app was used and for how long. This data is sent to the server in JSON format.

[0109] Input: Communication data, purchase data, search data collected from the device

[0110] Output: User data in JSON format

[0111] How it works: The device records user actions and usage in real time, encrypts this in JSON format, and sends it to the server.

[0112] Step 2: Data integration and preprocessing

[0113] The server stores the data received in step 1 in a database (e.g., MongoDB) and centrally consolidates it. During this process, it converts data from different formats (e.g., JSON, XML) into a consistent format (e.g., JSON), removes duplicate data, corrects inconsistent data, and fills in missing data.

[0114] Input: User data in JSON format

[0115] Output: Preprocessed integrated data

[0116] How it works: The server uses a data transformation library (e.g., Pandas) to convert data from each data source into a consistent format and cleanse the data.

[0117] Step 3: Analyzing user preferences and behavior patterns

[0118] The server then applies machine learning algorithms (e.g., k-means clustering, random forest) to the preprocessed data to analyze user preferences and behavioral patterns. Specifically, it extracts patterns such as a user often watching movies on weekends or regularly purchasing products from a particular brand.

[0119] Input: Preprocessed integrated data

[0120] Output: Analysis results based on user preferences and behavioral patterns

[0121] How it works: The server analyzes the data using machine learning models (e.g., scikit-learn) to identify user preferences and behavioral patterns.

[0122] Step 4: Personalized product and service recommendations

[0123] The server recommends the most suitable products and services to the user based on the analysis results of step 3. For example, if a user is interested in movies, the server will recommend the latest movie tickets and related goods. The recommendation results are generated in list form and stored in the server's database.

[0124] Input: Analysis results based on user preferences and behavioral patterns

[0125] Output: Recommendation list

[0126] How it works: The server uses collaborative filtering and content-based filtering to select the best products and services for the user and generate a list.

[0127] Step 5: Customizing the app's user interface

[0128] The device customizes the display of applications based on the recommendation list received from the server, dynamically changing the UI by adding links and shortcuts to the home screen that interest the user.

[0129] Input: Recommendation list

[0130] Output: Customized user interface

[0131] Operation: The device rearranges the home screen and menus based on the received recommendation list to provide a user-friendly interface.

[0132] Step 6: Real-time feedback and model updates

[0133] The server monitors user usage data sent from the device in real time, collecting new data and using it to retrain the machine learning model and improve the accuracy of the recommendation system.

[0134] Input: User usage data

[0135] Output: Updated machine learning model

[0136] How it works: The server uses continuously collected user data to retrain the machine learning model and improve recommendation accuracy.

[0137] Through the above processing steps, the system can provide optimized recommendations to users and provide a customized experience based on their latest preferences and behavioral patterns.

[0138] (Application example 1)

[0139] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0140] The problem that this invention aims to solve is to improve the user's shopping experience by utilizing a wide range of user behavioral data to improve the accuracy of personalized product recommendations and dynamically optimizing the user interface based on those recommendations. In particular, it is a challenge to immediately respond to new changes in user preferences through feedback based on real-time behavioral data and instantaneous system updates.

[0141] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0142] In this invention, the server includes means for collecting user information, means for integrating and preprocessing various data, means for analyzing user preferences and behavioral patterns, means for recommending products and services suitable for the user, means for customizing the user interface, means for providing real-time feedback on system usage status and updating the model, means for recommending optimal products and services based on the analysis results using real-time user behavioral data, and means for dynamically changing the interface based on the recommendation results. This makes it possible to precisely analyze user preferences and behavioral patterns, provide personalized products and services based on the analysis results in real time, and optimize the user interface.

[0143] "Means for collecting user information" refers to functions for collecting data on user behavior and preferences.

[0144] "Means for integrating and preprocessing various data" refers to the ability to centralize data collected from different formats and sources, convert it into a consistent format, and correct missing or inconsistent data.

[0145] "Means for analyzing user preferences and behavioral patterns" refers to a function for analyzing user interests and behavioral trends based on collected and pre-processed data.

[0146] "Means of recommending products and services suitable for users" is a function that suggests the most suitable products and services for each user based on the analysis results.

[0147] "Means for customizing the user interface" refers to a function that dynamically changes the display content of an application or website according to the needs and preferences of each user.

[0148] "Means of providing feedback on system usage in real time and updating the model" is a function that collects user behavior data in real time, updates the machine learning model based on the results, and improves the accuracy of recommendations.

[0149] "A means of recommending optimal products and services based on the analysis results using real-time user behavior data" is a function that analyzes user behavior data collected in real time and suggests optimal products and services based on the results.

[0150] The "means for dynamically changing the interface based on the recommendation result" is a function for dynamically changing the content and layout of the user interface based on the product recommendation result.

[0151] The present invention provides a system that collects a wide range of user behavior data, analyzes it to make personalized product recommendations, and dynamically optimizes the user interface based on the results. This system has the following main functions:

[0152] 1. Data Collection

[0153] The server collects user behavior data from a variety of sources. This behavior data includes, for example, communication data, purchase data, and search data. This data allows us to understand what applications users use, what they purchase at which stores, what keywords they searched for, and how much data they consume.

[0154] 2. Data integration and preprocessing

[0155] The server centrally integrates the collected data into a database, converts data from different formats and sources into a consistent format, removes duplicates, corrects inconsistencies, and appropriately completes missing data.

[0156] 3. Analysis of preferences and behavioral patterns

[0157] The server then applies machine learning algorithms to analyze user preferences and behavioral patterns based on the preprocessed data. Using past data, the server identifies the types of products and services users are interested in and their behavioral patterns. For example, it identifies patterns such as users who regularly purchase products from a particular brand or who are interested in movies.

[0158] 4. Product and service recommendations

[0159] Based on the analysis results, the server recommends the most suitable products and services for each user. For example, if a user is interested in movies, it will recommend the latest movie tickets and movie-related goods.

[0160] 5. Interface customization

[0161] The device customizes the user interface based on the recommendation list received from the server, adding relevant apps and links as shortcuts to the home screen and rearranging the content to make it easier to find what interests you.

[0162] 6. Real-time feedback and model updates

[0163] The server monitors users' app usage in real time, collecting new data to retrain the machine learning model and improve its accuracy. If a user exhibits behavior that indicates a new preference, that behavioral data is immediately collected and reflected in the next recommendation.

[0164] Specific examples

[0165] For example, if a user consumes a lot of movie-related data, the server will recommend the latest movie tickets and movie-related merchandise to the user. Furthermore, the user interface will be customized accordingly, with movie-related links and apps displayed on the home screen. As the user's preferences change, the system will instantly adapt and update the recommendations and interface accordingly.

[0166] Prompt Sentence Examples

[0167] "Based on the following user behavior data, please recommend the best movie-related services and tickets for this user. Consider their movie viewing history, purchase history, and search history to generate the best list to display movie tickets and related products for the weekend."

[0168] In this way, the present invention provides users with optimized products and services and dynamically changes the interface based on the recommendations, providing a more personalized shopping experience.

[0169] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0170] Step 1:

[0171] The server collects user behavior data from a variety of sources, including communication data, purchase data, and search data. Specifically, the server records what applications the user uses, what items they purchase at which stores, and what keywords they use to search. This data is stored in the server's database.

[0172] Step 2:

[0173] The server centrally integrates the collected data into a database and performs pre-processing. Data obtained from different data formats and sources is converted into a consistent format, duplicate data is removed, inconsistent data is corrected, and missing data is filled in. After processing, the data is saved back into the database in a clean state.

[0174] Step 3:

[0175] The server then applies machine learning algorithms to the preprocessed data to analyze user preferences and behavioral patterns. Specifically, it uses the collected historical data to train a model that predicts what products and services users are interested in. The trained model can then extract patterns, such as users' tendency to purchase certain brands of products on weekends.

[0176] Step 4:

[0177] The server then recommends the best products and services for each user based on the analysis results. Based on the predictions made by the machine learning model, it generates a list of products and services that the user may be interested in. This recommendation information is stored in the server's database and later sent to the device.

[0178] Step 5:

[0179] The device customizes the user interface based on the recommendation list received from the server. Specifically, it adds apps and links that are relevant to the user as shortcuts to the home screen and rearranges the layout so that users can quickly find content they are interested in. This customization allows users to use apps in an environment optimized for them.

[0180] Step 6:

[0181] The server monitors the user's app usage in real time and collects new data, which is used to retrain the machine learning model and improve the system's accuracy. For example, if a user takes an action that indicates a new preference, that behavioral data is immediately collected and reflected in the next recommendation. As a result, the product and service recommendations to the user are further refined.

[0182] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0183] This invention is a system that collects user information, integrates and preprocesses various data, analyzes user preferences and behavioral patterns, and recommends personalized products and services. In addition, by combining it with an emotion engine that recognizes user emotions, it is possible to achieve even greater accuracy and personalization, thereby providing users with the best possible experience.

[0184] User Data Collection

[0185] First, the server uses a means of collecting user information to obtain communication data, purchase data, search data, and even user emotion data through various APIs. Specifically, the server collects the user's communication history from the API of the communication service provider, purchase history from the API of the electronic payment service, and search history from the API of the search engine. Furthermore, to recognize the user's emotions, the emotion engine uses image processing and voice analysis technology to analyze facial expressions and tone of voice and obtain emotion data.

[0186] Data integration and preprocessing

[0187] The server consolidates and pre-processes the collected data. First, it converts data from different formats into a consistent format. Second, it removes duplicate data, corrects inconsistent data, and imputes missing data where necessary. This creates a clean and consistent dataset.

[0188] Analysis of user preferences and behavioral patterns

[0189] The server then runs machine learning algorithms based on the preprocessed data. Specifically, it uses clustering and classification algorithms to analyze user preferences and behavioral patterns. For example, a user who frequently searches for movies is classified into a cluster called movie lovers. By combining this with emotional data, it is possible to analyze the emotional state of a user when they behave in a particular way. For example, if a user looks happy while searching for a movie, it can be determined that the movie has a positive impact on the user.

[0190] Personalized product and service recommendations

[0191] The server then uses the analysis results to recommend the most suitable products and services for each user. When recommending products and services, the server takes into account the user's past behavioral data and preferences, as well as emotional data, to select items that are likely to interest the user. For example, if a user is interested in movies and looks happy, the server will recommend the latest movie tickets and movie-related merchandise.

[0192] Customizing the app's user interface

[0193] The device dynamically customizes the user interface based on the recommendation list received from the server. Specifically, it adds the most relevant apps and links as shortcuts to the home screen for quick access. For example, it displays a movie ticket purchasing app or offers from nearby restaurants. It also rearranges apps so that users can quickly find content they are interested in when they open them.

[0194] Real-time feedback and model updates

[0195] The server monitors users' app usage in real time and collects new data, including which recommended items users click on, which products they end up purchasing, and new emotional data from users, which will be used to refine future recommendations.

[0196] The server retrains the machine learning model based on the new data collected, improving the accuracy of the recommendation system. Any data indicating new preferences, behavioral patterns, or emotional data is immediately reflected in the model and reflected in the next recommendation results.

[0197] Specific Examples

[0198] For example, User A uses communication services, makes daily purchases using an electronic payment service, and uses a search engine to get the latest movie information. The server collects this data and analyzes the user's preferences, behavioral patterns, and emotional state. If User A is determined to have a high interest in movies, the server recommends the latest movie ticket information and coupons for theaters showing movies in the user's area. The user interface is customized accordingly, and movie-related links and apps are displayed on the home screen. Real-time feedback is also provided to immediately reflect any new content or services that User A has become interested in. In this way, users not only receive products and services that are optimized for them, but also use a convenient interface based on those products and services.

[0199] As described above, the present invention realizes a system that provides users with the optimal experience by aggregating and analyzing a wide range of user data and emotional data and recommending personalized products and services.

[0200] The processing flow will be explained below.

[0201] Step 1:

[0202] The server uses the means for collecting user information to obtain communication data, purchase data, search data, and user emotion data through various APIs. Communication history is obtained from the API of the communication service provider, purchase history from the API of the electronic payment service, and search history from the API of the search engine. Furthermore, using image processing and voice analysis technology, the emotion engine analyzes the user's facial expressions and tone of voice to collect emotion data.

[0203] Step 2:

[0204] The server consolidates and pre-processes the collected data. First, it converts data from different formats into a unified format. Second, it removes duplicate data, corrects inconsistent data, and imputes missing data. This process creates a clean and consistent dataset.

[0205] Step 3:

[0206] The server stores the preprocessed data in a database, where data such as communication data, purchase data, search data, and emotion data are stored by category.

[0207] Step 4:

[0208] The server uses the data stored in the database to run machine learning algorithms. Specifically, it applies clustering and classification algorithms to analyze user preferences and behavioral patterns. For example, a user who frequently searches for movies can be clustered as a movie lover. It also combines emotional data to analyze the emotional state of a user when they behave in a particular way.

[0209] Step 5:

[0210] The server creates a user profile based on the results of the machine learning algorithm, including the user's interests, behavioral patterns, emotional state, etc. This profile is then used for subsequent recommendation processing.

[0211] Step 6:

[0212] The server then recommends products and services based on the created profile. For example, if a user is a movie lover and looks happy, it can recommend the latest movie tickets and movie-related goods. By incorporating the user's emotional data, it can also recommend products that will help the user relax if they are feeling stressed.

[0213] Step 7:

[0214] The device analyzes the recommendation list received from the server and dynamically customizes the user interface, adding relevant apps and links as shortcuts to the home screen for quick access, such as a movie ticket purchasing app or nearby restaurant offers.

[0215] Step 8:

[0216] The server monitors users' app usage in real time and collects new data, including which recommended items users click on, which products they end up purchasing, and new emotional data from users, which is then reflected in the next recommendation.

[0217] Step 9:

[0218] The server retrains the machine learning model based on the new data collected, improving the accuracy of the recommendation system. If new preferences or behavioral patterns are discovered, they are immediately reflected in the model and applied to the next recommendation.

[0219] Step 10:

[0220] The server then applies the updated model to the user profile and re-recommends products and services. This process allows the system to constantly provide the most optimized services based on the latest user information and emotional data.

[0221] Through these steps, users can easily find products and services that match their preferences and emotional state, and can access the information and services they need with just their smartphone.

[0222] Example 2

[0223] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0224] Currently, many recommendation systems recommend products and services based on user behavioral data and purchase history, but these systems do not take emotional data into account, which means they cannot make recommendations that are optimized for the user's emotional state.In addition, due to insufficient real-time feedback and model updates, there is also the problem that the accuracy of recommendation results decreases when the user's interests and preferences change.

[0225] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for collecting user information, a means for integrating various data and performing preprocessing, a means for analyzing user preferences and behavior patterns, a means for recommending products and services suitable for the user, a means for acquiring and analyzing emotion data, a means for customizing the user interface, and a means for providing feedback on the system usage status in real time and updating the model. This not only enables the recommendation of products and services optimized for the user, but also enables highly accurate recommendations by updating the model in real time.

[0226] "Means for collecting user information" refers to devices and software for collecting data about users, including communication data, purchase data, search data, and emotional data, through various APIs and sensors.

[0227] "Means for integrating and pre-processing various types of data" refers to devices and software that convert the diverse data collected into a unified format and perform pre-processing such as deleting duplicates, correcting inconsistencies, and filling in missing data.

[0228] "Means for analyzing user preferences and behavioral patterns" refers to devices or software that use machine learning algorithms and statistical analysis methods based on preprocessed data to extract and classify user interests and behavioral patterns.

[0229] "Means for recommending products and services suitable for users" refers to devices or software that select products and services that match the user's preferences and behavioral patterns based on the analysis results and present them as a recommendation list.

[0230] "Means for acquiring and analyzing emotional data" refers to devices or software that use image processing technology or voice analysis technology to analyze a user's facial expressions and tone of voice and determine their emotional state.

[0231] "Means for customizing the user interface" refers to devices or software that dynamically change the layout and display content of the home screen based on the recommendation list, providing a UI that is easy for users to use.

[0232] "Means of providing feedback on system usage in real time and updating the model" refers to devices and software that monitor and collect user behavioral and emotional data in real time, and retrain the machine learning model based on this new data to improve recommendation accuracy.

[0233] This invention is a system that collects and analyzes user information and recommends personalized products and services with high accuracy. By adding emotional data acquisition and analysis functions, this system provides recommendations that also correspond to the user's emotional state, and by updating the model in real time, it achieves highly accurate recommendations.

[0234] User Data Collection

[0235] The server collects user information through various APIs. Communication data is obtained from the API of the communication service provider, purchase data from the API of the electronic payment service, and search data from the API of the search engine. In addition, a camera using image processing technology and a microphone using voice analysis technology are used to obtain emotional data from facial expressions and tone of voice. For example, the server can determine whether the user is smiling from facial expressions captured by the camera, and evaluate whether they are enjoying themselves from the tone of their voice recorded by the microphone.

[0236] Data integration and preprocessing

[0237] The server converts the collected data into a unified format and performs preprocessing such as deleting duplicates, correcting inconsistencies, and filling in missing data. Data obtained from each API is unified into formats such as JSON or CSV. Inconsistent data (for example, data with different date and time formats) is corrected, and missing data is inferred or filled in. This creates a consistent dataset.

[0238] Analysis of user preferences and behavioral patterns

[0239] The server then runs machine learning algorithms on the preprocessed data to analyze user preferences and behavioral patterns. Specifically, it uses clustering algorithms (e.g., K-means clustering) and classification algorithms (e.g., decision trees and random forests). For example, a user who frequently searches for movies might be classified into a cluster called a movie lover. It also combines emotional data to analyze behaviors under specific emotional states.

[0240] Personalized product and service recommendations

[0241] The server then uses the analysis results to recommend the most suitable products and services for each user. It selects the most suitable items by taking into consideration not only past behavioral data and preferences, but also emotional data. For example, if a user is interested in movies and looks happy, it will be recommended the latest movie tickets and movie-related merchandise.

[0242] Customizing the app's user interface

[0243] The device dynamically customizes the user interface based on the recommendation list received from the server, changing the home screen layout to prioritize relevant apps and links. For example, movie ticket purchasing apps and movie-related links are displayed on the home screen. The device also rearranges the layout to make new content that may be of interest easier to access.

[0244] Real-time feedback and model updates

[0245] The server monitors and collects user behavioral and emotional data in real time. This new data is used to retrain the machine learning model and improve the accuracy of the recommendation system. The model parameters are updated based on new preferences and behavioral patterns. For example, the system can immediately respond to new content or products that users have become interested in.

[0246] As described above, the present invention enables users to recommend products and services that are optimized for them, and provides highly accurate recommendations through real-time model updates. Specifically, the following prompt sentences can be used:

[0247] Example prompt sentence:

[0248] "A method to recommend optimal products and services to a user searching for the latest movie tickets, taking into account the user's emotional data."

[0249] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0250] Step 1:

[0251] The server collects communication history, purchase history, search history, and emotion data via various APIs and sensors. Communication data is obtained from the API of the communication service provider, purchase data from the API of the electronic payment service, and search data from the API of the search engine. Emotion data is obtained from a camera using image processing technology and a microphone using voice analysis technology. The input is data from each API and sensor, and the output is a dataset of collected user information. The server stores this data in a repository where it is temporarily saved.

[0252] Step 2:

[0253] The server integrates and preprocesses the collected data. Specifically, it converts each piece of collected data into JSON or CSV format. It then removes duplicate data, corrects inconsistent data, and fills in missing data. The input is a dataset of collected user information, and the output is a clean dataset that has been integrated and preprocessed. The server runs software processes to remove duplicates and convert formats, creating a highly consistent dataset.

[0254] Step 3:

[0255] The server runs a machine learning algorithm on the preprocessed data to analyze user preferences and behavioral patterns. Specifically, it uses a clustering algorithm (e.g., K-means) to group users, and then uses a classification algorithm (e.g., decision tree or random forest) to analyze individual behavioral patterns. The input is the preprocessed, clean dataset, and the output is a model of user preferences and behavioral patterns. The server stores the results of clustering and classification in a database.

[0256] Step 4:

[0257] The server recommends optimal products and services to users based on the analysis results. The server selects the most suitable items by taking into account past behavioral data, preference data, and emotional data. The input is a model of the user's preferences and behavioral patterns, and the output is a personalized recommendation list. The server generates the recommendation list and prepares data corresponding to each user.

[0258] Step 5:

[0259] The device dynamically customizes the user interface based on the recommendation list received from the server. Specifically, it changes the home screen layout and adds relevant apps and links. The input is the recommendation list from the server, and the output is the customized user interface. The device adjusts the screen layout and adds shortcuts.

[0260] Step 6:

[0261] The server monitors users' behavioral and emotional data in real time and collects new data. The input is real-time data from the user's device, and the output is the latest behavioral data set. The server retrains the machine learning model based on this new data to improve the accuracy of the recommendation system. Specifically, it monitors click data, purchase history, and fluctuations in emotional state, and updates the model accordingly.

[0262] (Application example 2)

[0263] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0264] Conventional recommendation systems tend to recommend products and services based on static data such as a user's purchase history and search history, and are insufficient in responding to the user's emotional state or real-time behavior. Therefore, there is a demand for more appropriate and effective advertisements and product / service recommendations for users. This invention aims to solve the problems of conventional systems by providing personalized, highly accurate recommendations that also take into account the user's emotional data, thereby providing users with an optimal experience.

[0265] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting user information, means for integrating various data and performing preprocessing, means for analyzing user preferences and behavioral patterns, means for recommending products and services suitable for the user, means for recognizing the user's emotions, means for customizing the user interface, means for providing feedback on system usage status in real time and updating the model, and means for recommending and displaying personalized advertisements based on emotional data. This makes it possible to recommend appropriate products and services and display advertisements in real time based on the user's emotional state.

[0266] "User Information" refers to various information about a user, including communication data, purchase data, search data, and emotional data.

[0267] "Means for integrating and pre-processing various data" refers to the part of the system that converts data of different formats into a consistent format, removes duplicate data, corrects inconsistent data, and completes missing data.

[0268] "Means for analyzing user preferences and behavioral patterns" refers to the part of the system that uses machine learning algorithms to analyze patterns based on users' interests and past behavioral history.

[0269] "Means for recommending products and services suitable for users" refers to the process of recommending the most relevant products and services based on the analysis of users' preferences and behavioral patterns and emotional data.

[0270] "Means for recognizing the user's emotions" refers to the part of the system that uses image processing and voice analysis technology to obtain and analyze emotional data from the user's facial expressions and tone of voice.

[0271] "Means for customizing the user interface" refers to the process of dynamically changing the interface layout and display content based on the analysis results and the list of recommended products and services.

[0272] "Means of providing real-time feedback on system usage and updating the model" refers to the process of collecting user operation status and new data in real time, and using this information to retrain the machine learning model and improve its accuracy.

[0273] "Means for recommending and displaying personalized advertisements based on emotional data" refers to the process of selecting and displaying the most appropriate advertisements individually based on the user's current emotional state.

[0274] This invention is a system that collects and analyzes a wide variety of user information, including emotional data, to recommend personalized and highly accurate products and services. In particular, we describe the recommendation and display of advertisements that utilize emotional data. This system mainly consists of a server that collects and analyzes user information, and a terminal that provides the user interface.

[0275] Data collection

[0276] First, the server collects user information such as communication data, purchase data, and search data via various APIs. Specific hardware used includes smartphones, smart glasses, and head-mounted displays. The server then uses the data acquired from these devices and an emotion engine to obtain emotional data from facial expressions and tone of voice. For example, the server uses the smartphone's camera and microphone to perform facial recognition and voice analysis to obtain the user's emotions in real time.

[0277] Data integration and preprocessing

[0278] The server consolidates and preprocesses the collected data. Specific software used is a cloud platform (such as AWS® or Google® Cloud). This unifies data in different formats, removes duplicates, corrects inconsistent data, and fills in missing data. This process generates a clean, consistent dataset, facilitating subsequent analysis.

[0279] Analysis of preferences and behavioral patterns

[0280] The server uses machine learning algorithms to analyze user preferences and behavioral patterns based on the preprocessed data. Specifically, it uses machine learning libraries such as TENSORFLOW (registered trademark) and PyTorch. This allows for clustering and classification based on users' interests and past behavior. Furthermore, since emotional data is also included in the analysis, it is possible to analyze the emotional state of users when they behave in a particular way.

[0281] Product and service recommendations

[0282] The server then uses the analysis results to recommend the most suitable products and services to the user. By combining this with emotional data, the system can identify and recommend items and advertisements that are likely to interest the user in real time. For example, if a user searches for something related to movies and shows an excited expression, an advertisement for movie tickets will be displayed.

[0283] User interface customization

[0284] The device dynamically changes the user interface based on the recommendation list received from the server, for example, adding a shortcut to a movie ticket purchasing app on the home screen or displaying related offers, allowing users to quickly access the information and services they need.

[0285] Real-time feedback and model updates

[0286] The server monitors user behavior in real time and collects new data, including information about ads clicked, products purchased, and even new emotional data, which it uses to retrain machine learning models and improve the accuracy of the recommendation system.

[0287] Examples of prompt statements

[0288] When a user searches for something related to movies, if a happy expression is recognized, the system will generate a prompt like the one below based on that emotional data and display an advertisement for movie tickets.

[0289] If a user searches for "movie updates" and then looks happy:

[0290] "Check out our latest offers on movie tickets and movie merchandise for great savings!"

[0291] In this way, by making recommendations that reflect real-time emotional data, a system can be created that provides users with the optimal experience.

[0292] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0293] Step 1:

[0294] The server collects communication data, purchase data, and search data via various APIs. Specifically, it obtains this data from devices such as smartphones and smart glasses, and collects emotional data from facial expressions and tone of voice through an emotion engine. The server uses this data to initialize the user's interests and concerns. The inputs are communication data, purchase data, search data, and emotional data, and the output is integrated user data.

[0295] Step 2:

[0296] The server converts the collected data into a unified format, removes duplicates, corrects inconsistent data, and fills in missing data. The specific software used is a cloud platform (AWS or Google Cloud). The input is consolidated user data, and the output is a clean, consistent dataset.

[0297] Step 3:

[0298] The server runs machine learning algorithms (such as TensorFlow or PyTorch) on the clean dataset to analyze user preferences and behavioral patterns. In this step, pattern analysis is performed using clustering and classification algorithms. Emotional data is also taken into account, and behavioral patterns in specific emotional states are also analyzed. The input is the clean dataset, and the output is the analyzed user preferences and behavioral patterns.

[0299] Step 4:

[0300] The server recommends optimal products, services, and advertisements for each user based on the analysis results. In particular, it utilizes emotional data to select items and advertisements that are likely to interest the user in real time. It also uses a generative AI model to generate prompts and use them to display advertisements. The input to this step is the analysis results, and the output is recommended products, services, and advertisements.

[0301] Step 5:

[0302] The device customizes the user interface based on the recommendation list obtained from the server. For example, it adds a shortcut to a movie ticket purchasing app on the home screen and displays related offers. This allows the user to quickly access the information and services they need. The input is the recommendation list, and the output is the customized user interface.

[0303] Step 6:

[0304] The server monitors user behavior in real time and collects new data, including information about ads clicked, products purchased, and even new emotional data. The server uses this data to retrain the machine learning model and improve the accuracy of the recommendation system. The input is real-time user behavior data, and the output is an updated machine learning model.

[0305] In this way, specific data processing and calculations are carried out at each step, resulting in a system that provides users with advertisements and services optimized for them in real time.

[0306] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0307] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0308] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0309] [Second embodiment]

[0310] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0311] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0312] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0313] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0314] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0315] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0316] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0317] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0318] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0319] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0320] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0321] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0322] This system collects user information, integrates and preprocesses various data, analyzes user preferences and behavioral patterns, and recommends personalized products and services. Furthermore, it customizes the user interface based on the recommendation results, provides real-time feedback on system usage, and updates the model to provide the optimal experience for users.

[0323] User Data Collection

[0324] First, the server collects user information. This information is collected from a wide range of data sources, including communication data, purchase data, and search data. For example, communication data may include what applications the user is using and how much data traffic they are consuming. Purchase data may include what store the user purchased what and how much they paid. Search data may include what keywords the user used in their search and which web pages they viewed.

[0325] Data integration and preprocessing

[0326] The server centrally integrates the collected data into a database, converting data from different formats and sources into a consistent format, removing duplicate data, correcting inconsistencies, and properly filling in missing data.

[0327] Analysis of user preferences and behavioral patterns

[0328] The server then applies machine learning algorithms to the preprocessed data to analyze user preferences and behavioral patterns. Using past data, the server identifies the types of products and services a user is interested in and the behavioral patterns they exhibit. For example, it identifies patterns such as whether a user frequently goes to the movies on weekends or regularly purchases a particular brand of product.

[0329] Personalized product and service recommendations

[0330] The server then uses the analysis results to recommend the most suitable products and services for each user. When recommending products and services, the server takes into account the user's past behavioral data and preferences to select items that are likely to interest the user. For example, if a user is interested in movies, the server will recommend the latest movie tickets and movie-related goods.

[0331] Customizing the app's user interface

[0332] The device uses the recommendation list received from the server to customize how the app is displayed. Specifically, it adds the most relevant apps and links as shortcuts to the home screen. For example, it displays links to buy movie tickets or offers from nearby restaurants. It also rearranges apps so that when you open them, you can quickly find content that interests you.

[0333] Real-time feedback and model updates

[0334] The server monitors users' app usage in real time, collecting new data to retrain the machine learning model and improve the accuracy of the recommendation system. For example, if a user takes an action that indicates a new preference, that behavioral data is immediately collected and reflected in the next recommendation.

[0335] Specific Examples

[0336] For example, User A uses communication services, makes daily purchases using an electronic payment service, and uses a search engine to get the latest movie information. The server collects this data and analyzes the user's preferences and behavioral patterns. If User A is determined to have a high interest in movies, the server recommends the latest movie ticket information and coupons for theaters showing movies in the user's area. The user interface is customized accordingly, and movie-related links and apps are displayed on the home screen. Real-time feedback is also provided to immediately reflect any new content or services that User A has become interested in. In this way, users not only receive products and services that are optimized for them, but also use a convenient interface based on those products and services.

[0337] As described above, the present invention realizes a system that provides users with the most optimal experience by aggregating and analyzing a wide range of user data and recommending personalized products and services.

[0338] The processing flow will be explained below.

[0339] Step 1:

[0340] The server uses the means for collecting user information to obtain a wide range of data, including communication data, purchase data, and search data, through various APIs. Specifically, the server collects the user's communication history from the API of a communication service provider, purchase history from the API of an electronic payment service, and search history from the API of a search engine.

[0341] Step 2:

[0342] The server consolidates and pre-processes the collected data. First, it converts data from different formats into a consistent format. Second, it removes duplicate data, corrects inconsistent data, and imputes missing data where necessary. This creates a clean and consistent dataset.

[0343] Step 3:

[0344] The server stores the pre-processed data in a database, which is used for subsequent analysis and recommendation processing. The database stores data by category, such as communication data, purchase data, and search data.

[0345] Step 4:

[0346] The server uses the data stored in the database to run machine learning algorithms, such as clustering and classification algorithms, to analyze user preferences and behavioral patterns. For example, a user who frequently searches for movies might be classified as a movie lover.

[0347] Step 5:

[0348] The server then uses the results of the machine learning algorithm to create a user profile, which includes the user's interests, behavioral patterns, purchase history, etc. This profile is then used to make personalized recommendations of products and services.

[0349] Step 6:

[0350] The server then applies an algorithm to recommend products and services based on the created profile. For example, if a user has a profile for movie lovers, the server will recommend the latest movie tickets and movie-related merchandise. The recommendation results are generated in the form of a list and sent to the user's device.

[0351] Step 7:

[0352] The device analyzes the recommendation list received from the server and dynamically customizes the user interface, adding the most relevant apps and links as shortcuts to the home screen for quick access, such as a movie ticket purchasing app or nearby restaurant offers.

[0353] Step 8:

[0354] The server monitors users' app usage in real time and collects new data, including which recommended items users click on and which products they end up purchasing, to help refine future recommendations.

[0355] Step 9:

[0356] The server retrains the machine learning model based on the new data collected, improving the accuracy of the recommendation system. Any data that indicates new preferences or behavioral patterns is immediately reflected in the model and reflected in the next recommendation results.

[0357] Step 10:

[0358] The server then applies the updated model to the user profile and re-recommends products and services. In this way, the system continues to provide optimized services based on the latest user information.

[0359] By following these steps, users can easily find products and services that match their preferences and behavioral patterns, and can access the information and services they need with just one smartphone.

[0360] Example 1

[0361] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0362] In today's information society, consumers are bombarded with a vast amount of information, making it difficult to select optimal products and services. Furthermore, existing recommendation systems often cannot adequately reflect individual user preferences and behavioral patterns, and can only provide generalized recommendations. Furthermore, there is a need for highly accurate recommendations that reflect changes in user preferences and behavior in real time. An efficient and effective system is needed to solve these problems.

[0363] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0364] In this invention, the server includes a means for collecting user information, a means for integrating and preprocessing various data, and a means for analyzing user preferences and behavioral patterns, which enables highly accurate analysis of user preferences and behavioral patterns and the recommendation of optimal products and services to users.

[0365] "Means for collecting user information" refers to processes and devices for collecting various data related to users, such as communication data, purchase data, and search data.

[0366] "Means of integrating and pre-processing various data" refers to processes or devices that convert data collected from different data sources into a consistent format, remove duplicate data, correct inconsistent data, and fill in missing data.

[0367] "Means for analyzing user preferences and behavioral patterns" refers to processes or devices that apply machine learning algorithms or other analytical techniques to the collected and pre-processed data to identify user preferences and behavioral patterns.

[0368] "Means for recommending products and services suitable for users" refers to the process or device that selects and presents the most suitable products and services based on the analyzed preferences and behavioral patterns of users.

[0369] "Means for customizing user interfaces" refers to processes or devices that change the display content and operation methods of applications and systems based on user preferences and behavioral patterns, making them easier for users to use.

[0370] "Means for providing real-time feedback on system usage and updating the model" refers to the process or device that monitors and collects data on users' use of the system in real time, and uses that data to retrain the machine learning model, thereby improving the accuracy of the system's recommendations.

[0371] "Means for encrypting data received from a terminal and sending it to a server" refers to a process or device for encrypting data collected from a user's terminal and securely sending it to a server.

[0372] "Means of transforming data from different formats into a consistent format" refers to a process or device that transforms data provided in different formats (e.g., JSON, XML) into a unified format.

[0373] "Means for applying machine learning algorithms to analyze user preferences and behavioral patterns" refers to a process or device that uses machine learning techniques to identify user preferences and behavioral patterns from pre-processed data.

[0374] System Overview

[0375] This system collects user information, integrates and preprocesses various data, analyzes user preferences and behavioral patterns, and recommends personalized products and services. Furthermore, it customizes the user interface based on the recommendation results, provides real-time feedback on system usage, and updates the model to provide the optimal experience for users.

[0376] User Data Collection

[0377] First, the server collects user information. This information is collected from a wide range of data sources, including communication data, purchase data, and search data. For example, communication data may include what applications the user is using and how much data traffic they are consuming. Purchase data may include what store the user purchased what and how much they paid. Search data may include what keywords the user used in their search and which web pages they viewed.

[0378] Data integration and preprocessing

[0379] The server centrally integrates the collected data into a database (e.g., MongoDB). During this process, it converts data obtained from different data formats and sources into a consistent format (e.g., JSON), removes duplicate data, corrects inconsistent data, and appropriately complements any missing data. Data conversion is performed using a data conversion library (e.g., Pandas).

[0380] Analysis of user preferences and behavioral patterns

[0381] The server then applies machine learning algorithms (e.g., k-means clustering, random forest) to the preprocessed data to analyze user preferences and behavioral patterns. Using past data, the server identifies what products and services users are interested in and what behavioral patterns they exhibit. For example, it identifies patterns such as whether a user often goes to the movies on weekends or regularly purchases a particular brand of product.

[0382] Personalized product and service recommendations

[0383] The server then uses the analysis results to recommend the most suitable products and services for each user. When recommending products and services, the server takes into account the user's past behavioral data and preferences to select items that are likely to interest the user. For example, if a user is interested in movies, the server will recommend the latest movie tickets and movie-related goods.

[0384] Customizing the app's user interface

[0385] The device uses the recommendation list received from the server to customize how the app is displayed. Specifically, it adds the most relevant apps and links as shortcuts to the home screen. For example, it displays links to buy movie tickets or offers from nearby restaurants. It also rearranges apps so that when you open them, you can quickly find content that interests you.

[0386] Real-time feedback and model updates

[0387] The server monitors users' app usage in real time, collecting new data to retrain the machine learning model and improve the accuracy of the recommendation system. For example, if a user takes an action that indicates a new preference, that behavioral data is immediately collected and reflected in the next recommendation.

[0388] Specific Examples

[0389] For example, User A uses communication services, makes daily purchases using an electronic payment service, and uses a search engine to get the latest movie information. The server collects this data and analyzes the user's preferences and behavioral patterns. If User A is determined to have a high interest in movies, the server recommends the latest movie ticket information and coupons for theaters showing movies in the user's area. The user interface is customized accordingly, and movie-related links and apps are displayed on the home screen. Real-time feedback is also provided to immediately reflect any new content or services that User A has become interested in. In this way, users not only receive products and services that are optimized for them, but also use a convenient interface based on those products and services.

[0390] Prompt Sentence Examples

[0391] "User A uses a communication service and frequently searches for movie-related information. Please recommend the best movie-related products and services for this user."

[0392] As described above, the present invention is a system that provides users with an optimal experience by aggregating and analyzing a wide range of user data and recommending personalized products and services.

[0393] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0394] Step 1: Collect user data

[0395] The server collects communication data, purchase data, search data, etc. from the device used by the user. This data is acquired in real time while the user is actually using the application. Specifically, when a user makes a purchase, the purchase information (product ID, price, purchase date, etc.) is recorded. Communication data includes which app was used and for how long. This data is sent to the server in JSON format.

[0396] Input: Communication data, purchase data, search data collected from the device

[0397] Output: User data in JSON format

[0398] How it works: The device records user actions and usage in real time, encrypts this in JSON format, and sends it to the server.

[0399] Step 2: Data integration and preprocessing

[0400] The server stores the data received in step 1 in a database (e.g., MongoDB) and centrally consolidates it. During this process, it converts data from different formats (e.g., JSON, XML) into a consistent format (e.g., JSON), removes duplicate data, corrects inconsistent data, and fills in missing data.

[0401] Input: User data in JSON format

[0402] Output: Preprocessed integrated data

[0403] How it works: The server uses a data transformation library (e.g., Pandas) to convert data from each data source into a consistent format and cleanse the data.

[0404] Step 3: Analyzing user preferences and behavior patterns

[0405] The server then applies machine learning algorithms (e.g., k-means clustering, random forest) to the preprocessed data to analyze user preferences and behavioral patterns. Specifically, it extracts patterns such as a user often watching movies on weekends or regularly purchasing products from a particular brand.

[0406] Input: Preprocessed integrated data

[0407] Output: Analysis results based on user preferences and behavioral patterns

[0408] How it works: The server analyzes the data using machine learning models (e.g., scikit-learn) to identify user preferences and behavioral patterns.

[0409] Step 4: Personalized product and service recommendations

[0410] The server recommends the most suitable products and services to the user based on the analysis results of step 3. For example, if a user is interested in movies, the server will recommend the latest movie tickets and related goods. The recommendation results are generated in list form and stored in the server's database.

[0411] Input: Analysis results based on user preferences and behavioral patterns

[0412] Output: Recommendation list

[0413] How it works: The server uses collaborative filtering and content-based filtering to select the best products and services for the user and generate a list.

[0414] Step 5: Customizing the app's user interface

[0415] The device customizes the display of applications based on the recommendation list received from the server, dynamically changing the UI by adding links and shortcuts to the home screen that interest the user.

[0416] Input: Recommendation list

[0417] Output: Customized user interface

[0418] Operation: The device rearranges the home screen and menus based on the received recommendation list to provide a user-friendly interface.

[0419] Step 6: Real-time feedback and model updates

[0420] The server monitors user usage data sent from the device in real time, collecting new data and using it to retrain the machine learning model and improve the accuracy of the recommendation system.

[0421] Input: User usage data

[0422] Output: Updated machine learning model

[0423] How it works: The server uses continuously collected user data to retrain the machine learning model and improve recommendation accuracy.

[0424] Through the above processing steps, the system can provide optimized recommendations to users and provide a customized experience based on their latest preferences and behavioral patterns.

[0425] (Application example 1)

[0426] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0427] The problem that this invention aims to solve is to improve the user's shopping experience by utilizing a wide range of user behavioral data to improve the accuracy of personalized product recommendations and dynamically optimizing the user interface based on those recommendations. In particular, it is a challenge to immediately respond to new changes in user preferences through feedback based on real-time behavioral data and instantaneous system updates.

[0428] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0429] In this invention, the server includes means for collecting user information, means for integrating and preprocessing various data, means for analyzing user preferences and behavioral patterns, means for recommending products and services suitable for the user, means for customizing the user interface, means for providing real-time feedback on system usage status and updating the model, means for recommending optimal products and services based on the analysis results using real-time user behavioral data, and means for dynamically changing the interface based on the recommendation results. This makes it possible to precisely analyze user preferences and behavioral patterns, provide personalized products and services based on the analysis results in real time, and optimize the user interface.

[0430] "Means for collecting user information" refers to functions for collecting data on user behavior and preferences.

[0431] "Means for integrating and preprocessing various data" refers to the ability to centralize data collected from different formats and sources, convert it into a consistent format, and correct missing or inconsistent data.

[0432] "Means for analyzing user preferences and behavioral patterns" refers to a function for analyzing user interests and behavioral trends based on collected and pre-processed data.

[0433] "Means of recommending products and services suitable for users" is a function that suggests the most suitable products and services for each user based on the analysis results.

[0434] "Means for customizing the user interface" refers to a function that dynamically changes the display content of an application or website according to the needs and preferences of each user.

[0435] "Means of providing feedback on system usage in real time and updating the model" is a function that collects user behavior data in real time, updates the machine learning model based on the results, and improves the accuracy of recommendations.

[0436] "A means of recommending optimal products and services based on the analysis results using real-time user behavior data" is a function that analyzes user behavior data collected in real time and suggests optimal products and services based on the results.

[0437] The "means for dynamically changing the interface based on the recommendation result" is a function for dynamically changing the content and layout of the user interface based on the product recommendation result.

[0438] The present invention provides a system that collects a wide range of user behavior data, analyzes it to make personalized product recommendations, and dynamically optimizes the user interface based on the results. This system has the following main functions:

[0439] 1. Data Collection

[0440] The server collects user behavior data from a variety of sources. This behavior data includes, for example, communication data, purchase data, and search data. This data allows us to understand what applications users use, what they purchase at which stores, what keywords they searched for, and how much data they consume.

[0441] 2. Data integration and preprocessing

[0442] The server centrally integrates the collected data into a database, converts data from different formats and sources into a consistent format, removes duplicates, corrects inconsistencies, and appropriately completes missing data.

[0443] 3. Analysis of preferences and behavioral patterns

[0444] The server then applies machine learning algorithms to analyze user preferences and behavioral patterns based on the preprocessed data. Using past data, the server identifies the types of products and services users are interested in and their behavioral patterns. For example, it identifies patterns such as users who regularly purchase products from a particular brand or who are interested in movies.

[0445] 4. Product and service recommendations

[0446] Based on the analysis results, the server recommends the most suitable products and services for each user. For example, if a user is interested in movies, it will recommend the latest movie tickets and movie-related goods.

[0447] 5. Interface customization

[0448] The device customizes the user interface based on the recommendation list received from the server, adding relevant apps and links as shortcuts to the home screen and rearranging the content to make it easier to find what interests you.

[0449] 6. Real-time feedback and model updates

[0450] The server monitors users' app usage in real time, collecting new data to retrain the machine learning model and improve its accuracy. If a user exhibits behavior that indicates a new preference, that behavioral data is immediately collected and reflected in the next recommendation.

[0451] Specific examples

[0452] For example, if a user consumes a lot of movie-related data, the server will recommend the latest movie tickets and movie-related merchandise to the user. Furthermore, the user interface will be customized accordingly, with movie-related links and apps displayed on the home screen. As the user's preferences change, the system will instantly adapt and update the recommendations and interface accordingly.

[0453] Prompt Sentence Examples

[0454] "Based on the following user behavior data, please recommend the best movie-related services and tickets for this user. Consider their movie viewing history, purchase history, and search history to generate the best list to display movie tickets and related products for the weekend."

[0455] In this way, the present invention provides users with optimized products and services and dynamically changes the interface based on the recommendations, providing a more personalized shopping experience.

[0456] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0457] Step 1:

[0458] The server collects user behavior data from a variety of sources, including communication data, purchase data, and search data. Specifically, the server records what applications the user uses, what items they purchase at which stores, and what keywords they use to search. This data is stored in the server's database.

[0459] Step 2:

[0460] The server centrally integrates the collected data into a database and performs pre-processing. Data obtained from different data formats and sources is converted into a consistent format, duplicate data is removed, inconsistent data is corrected, and missing data is filled in. After processing, the data is saved back into the database in a clean state.

[0461] Step 3:

[0462] The server then applies machine learning algorithms to the preprocessed data to analyze user preferences and behavioral patterns. Specifically, it uses the collected historical data to train a model that predicts what products and services users are interested in. The trained model can then extract patterns, such as users' tendency to purchase certain brands of products on weekends.

[0463] Step 4:

[0464] The server then recommends the best products and services for each user based on the analysis results. Based on the predictions made by the machine learning model, it generates a list of products and services that the user may be interested in. This recommendation information is stored in the server's database and later sent to the device.

[0465] Step 5:

[0466] The device customizes the user interface based on the recommendation list received from the server. Specifically, it adds apps and links that are relevant to the user as shortcuts to the home screen and rearranges the layout so that users can quickly find content they are interested in. This customization allows users to use apps in an environment optimized for them.

[0467] Step 6:

[0468] The server monitors the user's app usage in real time and collects new data, which is used to retrain the machine learning model and improve the system's accuracy. For example, if a user takes an action that indicates a new preference, that behavioral data is immediately collected and reflected in the next recommendation. As a result, the product and service recommendations to the user are further refined.

[0469] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0470] This invention is a system that collects user information, integrates and preprocesses various data, analyzes user preferences and behavioral patterns, and recommends personalized products and services. In addition, by combining it with an emotion engine that recognizes user emotions, it is possible to achieve even greater accuracy and personalization, thereby providing users with the best possible experience.

[0471] User Data Collection

[0472] First, the server uses a means of collecting user information to obtain communication data, purchase data, search data, and even user emotion data through various APIs. Specifically, the server collects the user's communication history from the API of the communication service provider, purchase history from the API of the electronic payment service, and search history from the API of the search engine. Furthermore, to recognize the user's emotions, the emotion engine uses image processing and voice analysis technology to analyze facial expressions and tone of voice and obtain emotion data.

[0473] Data integration and preprocessing

[0474] The server consolidates and pre-processes the collected data. First, it converts data from different formats into a consistent format. Second, it removes duplicate data, corrects inconsistent data, and imputes missing data where necessary. This creates a clean and consistent dataset.

[0475] Analysis of user preferences and behavioral patterns

[0476] The server then runs machine learning algorithms based on the preprocessed data. Specifically, it uses clustering and classification algorithms to analyze user preferences and behavioral patterns. For example, a user who frequently searches for movies is classified into a cluster called movie lovers. By combining this with emotional data, it is possible to analyze the emotional state of a user when they behave in a particular way. For example, if a user looks happy while searching for a movie, it can be determined that the movie has a positive impact on the user.

[0477] Personalized product and service recommendations

[0478] The server then uses the analysis results to recommend the most suitable products and services for each user. When recommending products and services, the server takes into account the user's past behavioral data and preferences, as well as emotional data, to select items that are likely to interest the user. For example, if a user is interested in movies and looks happy, the server will recommend the latest movie tickets and movie-related merchandise.

[0479] Customizing the app's user interface

[0480] The device dynamically customizes the user interface based on the recommendation list received from the server. Specifically, it adds the most relevant apps and links as shortcuts to the home screen for quick access. For example, it displays a movie ticket purchasing app or offers from nearby restaurants. It also rearranges apps so that users can quickly find content they are interested in when they open them.

[0481] Real-time feedback and model updates

[0482] The server monitors users' app usage in real time and collects new data, including which recommended items users click on, which products they end up purchasing, and new emotional data from users, which will be used to refine future recommendations.

[0483] The server retrains the machine learning model based on the new data collected, improving the accuracy of the recommendation system. Any data indicating new preferences, behavioral patterns, or emotional data is immediately reflected in the model and reflected in the next recommendation results.

[0484] Specific Examples

[0485] For example, User A uses communication services, makes daily purchases using an electronic payment service, and uses a search engine to get the latest movie information. The server collects this data and analyzes the user's preferences, behavioral patterns, and emotional state. If User A is determined to have a high interest in movies, the server recommends the latest movie ticket information and coupons for theaters showing movies in the user's area. The user interface is customized accordingly, and movie-related links and apps are displayed on the home screen. Real-time feedback is also provided to immediately reflect any new content or services that User A has become interested in. In this way, users not only receive products and services that are optimized for them, but also use a convenient interface based on those products and services.

[0486] As described above, the present invention realizes a system that provides users with the optimal experience by aggregating and analyzing a wide range of user data and emotional data and recommending personalized products and services.

[0487] The processing flow will be explained below.

[0488] Step 1:

[0489] The server uses the means for collecting user information to obtain communication data, purchase data, search data, and user emotion data through various APIs. Communication history is obtained from the API of the communication service provider, purchase history from the API of the electronic payment service, and search history from the API of the search engine. Furthermore, using image processing and voice analysis technology, the emotion engine analyzes the user's facial expressions and tone of voice to collect emotion data.

[0490] Step 2:

[0491] The server consolidates and pre-processes the collected data. First, it converts data from different formats into a unified format. Second, it removes duplicate data, corrects inconsistent data, and imputes missing data. This process creates a clean and consistent dataset.

[0492] Step 3:

[0493] The server stores the preprocessed data in a database, where data such as communication data, purchase data, search data, and emotion data are stored by category.

[0494] Step 4:

[0495] The server uses the data stored in the database to run machine learning algorithms. Specifically, it applies clustering and classification algorithms to analyze user preferences and behavioral patterns. For example, a user who frequently searches for movies can be clustered as a movie lover. It also combines emotional data to analyze the emotional state of a user when they behave in a particular way.

[0496] Step 5:

[0497] The server creates a user profile based on the results of the machine learning algorithm, including the user's interests, behavioral patterns, emotional state, etc. This profile is then used for subsequent recommendation processing.

[0498] Step 6:

[0499] The server then recommends products and services based on the created profile. For example, if a user is a movie lover and looks happy, it can recommend the latest movie tickets and movie-related goods. By incorporating the user's emotional data, it can also recommend products that will help the user relax if they are feeling stressed.

[0500] Step 7:

[0501] The device analyzes the recommendation list received from the server and dynamically customizes the user interface, adding relevant apps and links as shortcuts to the home screen for quick access, such as a movie ticket purchasing app or nearby restaurant offers.

[0502] Step 8:

[0503] The server monitors users' app usage in real time and collects new data, including which recommended items users click on, which products they end up purchasing, and new emotional data from users, which is then reflected in the next recommendation.

[0504] Step 9:

[0505] The server retrains the machine learning model based on the new data collected, improving the accuracy of the recommendation system. If new preferences or behavioral patterns are discovered, they are immediately reflected in the model and applied to the next recommendation.

[0506] Step 10:

[0507] The server then applies the updated model to the user profile and re-recommends products and services. This process allows the system to constantly provide the most optimized services based on the latest user information and emotional data.

[0508] Through these steps, users can easily find products and services that match their preferences and emotional state, and can access the information and services they need with just their smartphone.

[0509] Example 2

[0510] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0511] Currently, many recommendation systems recommend products and services based on user behavioral data and purchase history, but these systems do not take emotional data into account, which means they cannot make recommendations that are optimized for the user's emotional state.In addition, due to insufficient real-time feedback and model updates, there is also the problem that the accuracy of recommendation results decreases when the user's interests and preferences change.

[0512] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for collecting user information, a means for integrating various data and performing preprocessing, a means for analyzing user preferences and behavior patterns, a means for recommending products and services suitable for the user, a means for acquiring and analyzing emotion data, a means for customizing the user interface, and a means for providing feedback on the system usage status in real time and updating the model. This not only enables the recommendation of products and services optimized for the user, but also enables highly accurate recommendations by updating the model in real time.

[0513] "Means for collecting user information" refers to devices and software for collecting data about users, including communication data, purchase data, search data, and emotional data, through various APIs and sensors.

[0514] "Means for integrating and pre-processing various types of data" refers to devices and software that convert the diverse data collected into a unified format and perform pre-processing such as deleting duplicates, correcting inconsistencies, and filling in missing data.

[0515] "Means for analyzing user preferences and behavioral patterns" refers to devices or software that use machine learning algorithms and statistical analysis methods based on preprocessed data to extract and classify user interests and behavioral patterns.

[0516] "Means for recommending products and services suitable for users" refers to devices or software that select products and services that match the user's preferences and behavioral patterns based on the analysis results and present them as a recommendation list.

[0517] "Means for acquiring and analyzing emotional data" refers to devices or software that use image processing technology or voice analysis technology to analyze a user's facial expressions and tone of voice and determine their emotional state.

[0518] "Means for customizing the user interface" refers to devices or software that dynamically change the layout and display content of the home screen based on the recommendation list, providing a UI that is easy for users to use.

[0519] "Means of providing feedback on system usage in real time and updating the model" refers to devices and software that monitor and collect user behavioral and emotional data in real time, and retrain the machine learning model based on this new data to improve recommendation accuracy.

[0520] This invention is a system that collects and analyzes user information and recommends personalized products and services with high accuracy. By adding emotional data acquisition and analysis functions, this system provides recommendations that also correspond to the user's emotional state, and by updating the model in real time, it achieves highly accurate recommendations.

[0521] User Data Collection

[0522] The server collects user information through various APIs. Communication data is obtained from the API of the communication service provider, purchase data from the API of the electronic payment service, and search data from the API of the search engine. In addition, a camera using image processing technology and a microphone using voice analysis technology are used to obtain emotional data from facial expressions and tone of voice. For example, the server can determine whether the user is smiling from facial expressions captured by the camera, and evaluate whether they are enjoying themselves from the tone of their voice recorded by the microphone.

[0523] Data integration and preprocessing

[0524] The server converts the collected data into a unified format and performs preprocessing such as deleting duplicates, correcting inconsistencies, and filling in missing data. Data obtained from each API is unified into formats such as JSON or CSV. Inconsistent data (for example, data with different date and time formats) is corrected, and missing data is inferred or filled in. This creates a consistent dataset.

[0525] Analysis of user preferences and behavioral patterns

[0526] The server then runs machine learning algorithms on the preprocessed data to analyze user preferences and behavioral patterns. Specifically, it uses clustering algorithms (e.g., K-means clustering) and classification algorithms (e.g., decision trees and random forests). For example, a user who frequently searches for movies might be classified into a cluster called a movie lover. It also combines emotional data to analyze behaviors under specific emotional states.

[0527] Personalized product and service recommendations

[0528] The server then uses the analysis results to recommend the most suitable products and services for each user. It selects the most suitable items by taking into consideration not only past behavioral data and preferences, but also emotional data. For example, if a user is interested in movies and looks happy, it will be recommended the latest movie tickets and movie-related merchandise.

[0529] Customizing the app's user interface

[0530] The device dynamically customizes the user interface based on the recommendation list received from the server, changing the home screen layout to prioritize relevant apps and links. For example, movie ticket purchasing apps and movie-related links are displayed on the home screen. The device also rearranges the layout to make new content that may be of interest easier to access.

[0531] Real-time feedback and model updates

[0532] The server monitors and collects user behavioral and emotional data in real time. This new data is used to retrain the machine learning model and improve the accuracy of the recommendation system. The model parameters are updated based on new preferences and behavioral patterns. For example, the system can immediately respond to new content or products that users have become interested in.

[0533] As described above, the present invention enables users to recommend products and services that are optimized for them, and provides highly accurate recommendations through real-time model updates. Specifically, the following prompt sentences can be used:

[0534] Example prompt sentence:

[0535] "A method to recommend optimal products and services to a user searching for the latest movie tickets, taking into account the user's emotional data."

[0536] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0537] Step 1:

[0538] The server collects communication history, purchase history, search history, and emotion data via various APIs and sensors. Communication data is obtained from the API of the communication service provider, purchase data from the API of the electronic payment service, and search data from the API of the search engine. Emotion data is obtained from a camera using image processing technology and a microphone using voice analysis technology. The input is data from each API and sensor, and the output is a dataset of collected user information. The server stores this data in a repository where it is temporarily saved.

[0539] Step 2:

[0540] The server integrates and preprocesses the collected data. Specifically, it converts each piece of collected data into JSON or CSV format. It then removes duplicate data, corrects inconsistent data, and fills in missing data. The input is a dataset of collected user information, and the output is a clean dataset that has been integrated and preprocessed. The server runs software processes to remove duplicates and convert formats, creating a highly consistent dataset.

[0541] Step 3:

[0542] The server runs a machine learning algorithm on the preprocessed data to analyze user preferences and behavioral patterns. Specifically, it uses a clustering algorithm (e.g., K-means) to group users, and then uses a classification algorithm (e.g., decision tree or random forest) to analyze individual behavioral patterns. The input is the preprocessed, clean dataset, and the output is a model of user preferences and behavioral patterns. The server stores the results of clustering and classification in a database.

[0543] Step 4:

[0544] The server recommends optimal products and services to users based on the analysis results. The server selects the most suitable items by taking into account past behavioral data, preference data, and emotional data. The input is a model of the user's preferences and behavioral patterns, and the output is a personalized recommendation list. The server generates the recommendation list and prepares data corresponding to each user.

[0545] Step 5:

[0546] The device dynamically customizes the user interface based on the recommendation list received from the server. Specifically, it changes the home screen layout and adds relevant apps and links. The input is the recommendation list from the server, and the output is the customized user interface. The device adjusts the screen layout and adds shortcuts.

[0547] Step 6:

[0548] The server monitors users' behavioral and emotional data in real time and collects new data. The input is real-time data from the user's device, and the output is the latest behavioral data set. The server retrains the machine learning model based on this new data to improve the accuracy of the recommendation system. Specifically, it monitors click data, purchase history, and fluctuations in emotional state, and updates the model accordingly.

[0549] (Application example 2)

[0550] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0551] Conventional recommendation systems tend to recommend products and services based on static data such as a user's purchase history and search history, and are insufficient in responding to the user's emotional state or real-time behavior. Therefore, there is a demand for more appropriate and effective advertisements and product / service recommendations for users. This invention aims to solve the problems of conventional systems by providing personalized, highly accurate recommendations that also take into account the user's emotional data, thereby providing users with an optimal experience.

[0552] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting user information, means for integrating various data and performing preprocessing, means for analyzing user preferences and behavioral patterns, means for recommending products and services suitable for the user, means for recognizing the user's emotions, means for customizing the user interface, means for providing feedback on system usage status in real time and updating the model, and means for recommending and displaying personalized advertisements based on emotional data. This makes it possible to recommend appropriate products and services and display advertisements in real time based on the user's emotional state.

[0553] "User Information" refers to various information about a user, including communication data, purchase data, search data, and emotional data.

[0554] "Means for integrating and pre-processing various data" refers to the part of the system that converts data of different formats into a consistent format, removes duplicate data, corrects inconsistent data, and completes missing data.

[0555] "Means for analyzing user preferences and behavioral patterns" refers to the part of the system that uses machine learning algorithms to analyze patterns based on users' interests and past behavioral history.

[0556] "Means for recommending products and services suitable for users" refers to the process of recommending the most relevant products and services based on the analysis of users' preferences and behavioral patterns and emotional data.

[0557] "Means for recognizing the user's emotions" refers to the part of the system that uses image processing and voice analysis technology to obtain and analyze emotional data from the user's facial expressions and tone of voice.

[0558] "Means for customizing the user interface" refers to the process of dynamically changing the interface layout and display content based on the analysis results and the list of recommended products and services.

[0559] "Means of providing real-time feedback on system usage and updating the model" refers to the process of collecting user operation status and new data in real time, and using this information to retrain the machine learning model and improve its accuracy.

[0560] "Means for recommending and displaying personalized advertisements based on emotional data" refers to the process of selecting and displaying the most appropriate advertisements individually based on the user's current emotional state.

[0561] This invention is a system that collects and analyzes a wide variety of user information, including emotional data, to recommend personalized and highly accurate products and services. In particular, we describe the recommendation and display of advertisements that utilize emotional data. This system mainly consists of a server that collects and analyzes user information, and a terminal that provides the user interface.

[0562] Data collection

[0563] First, the server collects user information such as communication data, purchase data, and search data via various APIs. Specific hardware used includes smartphones, smart glasses, and head-mounted displays. The server then uses the data acquired from these devices and an emotion engine to obtain emotional data from facial expressions and tone of voice. For example, the server uses the smartphone's camera and microphone to perform facial recognition and voice analysis to obtain the user's emotions in real time.

[0564] Data integration and preprocessing

[0565] The server consolidates and preprocesses the collected data. Specifically, cloud platforms (such as AWS and Google Cloud) are used for this purpose. This process involves unifying data in different formats, removing duplicates, correcting inconsistent data, and filling in missing data. This process produces a clean, consistent dataset, facilitating subsequent analysis.

[0566] Analysis of preferences and behavioral patterns

[0567] The server uses machine learning algorithms to analyze user preferences and behavioral patterns based on the preprocessed data. Specifically, it uses machine learning libraries such as TensorFlow and PyTorch. This allows for clustering and classification based on users' interests and past behavior. Emotional data is also included in the analysis, making it possible to analyze the emotional state of users that leads them to take specific actions.

[0568] Product and service recommendations

[0569] The server then uses the analysis results to recommend the most suitable products and services to the user. By combining this with emotional data, the system can identify and recommend items and advertisements that are likely to interest the user in real time. For example, if a user searches for something related to movies and shows an excited expression, an advertisement for movie tickets will be displayed.

[0570] User interface customization

[0571] The device dynamically changes the user interface based on the recommendation list received from the server, for example, adding a shortcut to a movie ticket purchasing app on the home screen or displaying related offers, allowing users to quickly access the information and services they need.

[0572] Real-time feedback and model updates

[0573] The server monitors user behavior in real time and collects new data, including information about ads clicked, products purchased, and even new emotional data, which it uses to retrain machine learning models and improve the accuracy of the recommendation system.

[0574] Examples of prompt statements

[0575] When a user searches for something related to movies, if a happy expression is recognized, the system will generate a prompt like the one below based on that emotional data and display an advertisement for movie tickets.

[0576] If a user searches for "movie updates" and then looks happy:

[0577] "Check out our latest offers on movie tickets and movie merchandise for great savings!"

[0578] In this way, by making recommendations that reflect real-time emotional data, a system can be created that provides users with the optimal experience.

[0579] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0580] Step 1:

[0581] The server collects communication data, purchase data, and search data via various APIs. Specifically, it obtains this data from devices such as smartphones and smart glasses, and collects emotional data from facial expressions and tone of voice through an emotion engine. The server uses this data to initialize the user's interests and concerns. The inputs are communication data, purchase data, search data, and emotional data, and the output is integrated user data.

[0582] Step 2:

[0583] The server converts the collected data into a unified format, removes duplicates, corrects inconsistent data, and fills in missing data. The specific software used is a cloud platform (AWS or Google Cloud). The input is consolidated user data, and the output is a clean, consistent dataset.

[0584] Step 3:

[0585] The server runs machine learning algorithms (such as TensorFlow or PyTorch) on the clean dataset to analyze user preferences and behavioral patterns. In this step, pattern analysis is performed using clustering and classification algorithms. Emotional data is also taken into account, and behavioral patterns in specific emotional states are also analyzed. The input is the clean dataset, and the output is the analyzed user preferences and behavioral patterns.

[0586] Step 4:

[0587] The server recommends optimal products, services, and advertisements for each user based on the analysis results. In particular, it utilizes emotional data to select items and advertisements that are likely to interest the user in real time. It also uses a generative AI model to generate prompts and use them to display advertisements. The input to this step is the analysis results, and the output is recommended products, services, and advertisements.

[0588] Step 5:

[0589] The device customizes the user interface based on the recommendation list obtained from the server. For example, it adds a shortcut to a movie ticket purchasing app on the home screen and displays related offers. This allows the user to quickly access the information and services they need. The input is the recommendation list, and the output is the customized user interface.

[0590] Step 6:

[0591] The server monitors user behavior in real time and collects new data, including information about ads clicked, products purchased, and even new emotional data. The server uses this data to retrain the machine learning model and improve the accuracy of the recommendation system. The input is real-time user behavior data, and the output is an updated machine learning model.

[0592] In this way, specific data processing and calculations are carried out at each step, resulting in a system that provides users with advertisements and services optimized for them in real time.

[0593] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0594] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0595] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0596] [Third embodiment]

[0597] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0598] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0599] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0600] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0601] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0602] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0603] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0604] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0605] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0606] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0607] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0608] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0609] This system collects user information, integrates and preprocesses various data, analyzes user preferences and behavioral patterns, and recommends personalized products and services. Furthermore, it customizes the user interface based on the recommendation results, provides real-time feedback on system usage, and updates the model to provide the optimal experience for users.

[0610] User Data Collection

[0611] First, the server collects user information. This information is collected from a wide range of data sources, including communication data, purchase data, and search data. For example, communication data may include what applications the user is using and how much data traffic they are consuming. Purchase data may include what store the user purchased what and how much they paid. Search data may include what keywords the user used in their search and which web pages they viewed.

[0612] Data integration and preprocessing

[0613] The server centrally integrates the collected data into a database, converting data from different formats and sources into a consistent format, removing duplicate data, correcting inconsistencies, and properly filling in missing data.

[0614] Analysis of user preferences and behavioral patterns

[0615] The server then applies machine learning algorithms to the preprocessed data to analyze user preferences and behavioral patterns. Using past data, the server identifies the types of products and services a user is interested in and the behavioral patterns they exhibit. For example, it identifies patterns such as whether a user frequently goes to the movies on weekends or regularly purchases a particular brand of product.

[0616] Personalized product and service recommendations

[0617] The server then uses the analysis results to recommend the most suitable products and services for each user. When recommending products and services, the server takes into account the user's past behavioral data and preferences to select items that are likely to interest the user. For example, if a user is interested in movies, the server will recommend the latest movie tickets and movie-related goods.

[0618] Customizing the app's user interface

[0619] The device uses the recommendation list received from the server to customize how the app is displayed. Specifically, it adds the most relevant apps and links as shortcuts to the home screen. For example, it displays links to buy movie tickets or offers from nearby restaurants. It also rearranges apps so that when you open them, you can quickly find content that interests you.

[0620] Real-time feedback and model updates

[0621] The server monitors users' app usage in real time, collecting new data to retrain the machine learning model and improve the accuracy of the recommendation system. For example, if a user takes an action that indicates a new preference, that behavioral data is immediately collected and reflected in the next recommendation.

[0622] Specific Examples

[0623] For example, User A uses communication services, makes daily purchases using an electronic payment service, and uses a search engine to get the latest movie information. The server collects this data and analyzes the user's preferences and behavioral patterns. If User A is determined to have a high interest in movies, the server recommends the latest movie ticket information and coupons for theaters showing movies in the user's area. The user interface is customized accordingly, and movie-related links and apps are displayed on the home screen. Real-time feedback is also provided to immediately reflect any new content or services that User A has become interested in. In this way, users not only receive products and services that are optimized for them, but also use a convenient interface based on those products and services.

[0624] As described above, the present invention realizes a system that provides users with the most optimal experience by aggregating and analyzing a wide range of user data and recommending personalized products and services.

[0625] The processing flow will be explained below.

[0626] Step 1:

[0627] The server uses the means for collecting user information to obtain a wide range of data, including communication data, purchase data, and search data, through various APIs. Specifically, the server collects the user's communication history from the API of a communication service provider, purchase history from the API of an electronic payment service, and search history from the API of a search engine.

[0628] Step 2:

[0629] The server consolidates and pre-processes the collected data. First, it converts data from different formats into a consistent format. Second, it removes duplicate data, corrects inconsistent data, and imputes missing data where necessary. This creates a clean and consistent dataset.

[0630] Step 3:

[0631] The server stores the pre-processed data in a database, which is used for subsequent analysis and recommendation processing. The database stores data by category, such as communication data, purchase data, and search data.

[0632] Step 4:

[0633] The server uses the data stored in the database to run machine learning algorithms, such as clustering and classification algorithms, to analyze user preferences and behavioral patterns. For example, a user who frequently searches for movies might be classified as a movie lover.

[0634] Step 5:

[0635] The server then uses the results of the machine learning algorithm to create a user profile, which includes the user's interests, behavioral patterns, purchase history, etc. This profile is then used to make personalized recommendations of products and services.

[0636] Step 6:

[0637] The server then applies an algorithm to recommend products and services based on the created profile. For example, if a user has a profile for movie lovers, the server will recommend the latest movie tickets and movie-related merchandise. The recommendation results are generated in the form of a list and sent to the user's device.

[0638] Step 7:

[0639] The device analyzes the recommendation list received from the server and dynamically customizes the user interface, adding the most relevant apps and links as shortcuts to the home screen for quick access, such as a movie ticket purchasing app or nearby restaurant offers.

[0640] Step 8:

[0641] The server monitors users' app usage in real time and collects new data, including which recommended items users click on and which products they end up purchasing, to help refine future recommendations.

[0642] Step 9:

[0643] The server retrains the machine learning model based on the new data collected, improving the accuracy of the recommendation system. Any data that indicates new preferences or behavioral patterns is immediately reflected in the model and reflected in the next recommendation results.

[0644] Step 10:

[0645] The server then applies the updated model to the user profile and re-recommends products and services. In this way, the system continues to provide optimized services based on the latest user information.

[0646] By following these steps, users can easily find products and services that match their preferences and behavioral patterns, and can access the information and services they need with just one smartphone.

[0647] Example 1

[0648] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0649] In today's information society, consumers are bombarded with a vast amount of information, making it difficult to select optimal products and services. Furthermore, existing recommendation systems often cannot adequately reflect individual user preferences and behavioral patterns, and can only provide generalized recommendations. Furthermore, there is a need for highly accurate recommendations that reflect changes in user preferences and behavior in real time. An efficient and effective system is needed to solve these problems.

[0650] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0651] In this invention, the server includes a means for collecting user information, a means for integrating and preprocessing various data, and a means for analyzing user preferences and behavioral patterns, which enables highly accurate analysis of user preferences and behavioral patterns and the recommendation of optimal products and services to users.

[0652] "Means for collecting user information" refers to processes and devices for collecting various data related to users, such as communication data, purchase data, and search data.

[0653] "Means of integrating and pre-processing various data" refers to processes or devices that convert data collected from different data sources into a consistent format, remove duplicate data, correct inconsistent data, and fill in missing data.

[0654] "Means for analyzing user preferences and behavioral patterns" refers to processes or devices that apply machine learning algorithms or other analytical techniques to the collected and pre-processed data to identify user preferences and behavioral patterns.

[0655] "Means for recommending products and services suitable for users" refers to the process or device that selects and presents the most suitable products and services based on the analyzed preferences and behavioral patterns of users.

[0656] "Means for customizing user interfaces" refers to processes or devices that change the display content and operation methods of applications and systems based on user preferences and behavioral patterns, making them easier for users to use.

[0657] "Means for providing real-time feedback on system usage and updating the model" refers to the process or device that monitors and collects data on users' use of the system in real time, and uses that data to retrain the machine learning model, thereby improving the accuracy of the system's recommendations.

[0658] "Means for encrypting data received from a terminal and sending it to a server" refers to a process or device for encrypting data collected from a user's terminal and securely sending it to a server.

[0659] "Means of transforming data from different formats into a consistent format" refers to a process or device that transforms data provided in different formats (e.g., JSON, XML) into a unified format.

[0660] "Means for applying machine learning algorithms to analyze user preferences and behavioral patterns" refers to a process or device that uses machine learning techniques to identify user preferences and behavioral patterns from pre-processed data.

[0661] System Overview

[0662] This system collects user information, integrates and preprocesses various data, analyzes user preferences and behavioral patterns, and recommends personalized products and services. Furthermore, it customizes the user interface based on the recommendation results, provides real-time feedback on system usage, and updates the model to provide the optimal experience for users.

[0663] User Data Collection

[0664] First, the server collects user information. This information is collected from a wide range of data sources, including communication data, purchase data, and search data. For example, communication data may include what applications the user is using and how much data traffic they are consuming. Purchase data may include what store the user purchased what and how much they paid. Search data may include what keywords the user used in their search and which web pages they viewed.

[0665] Data integration and preprocessing

[0666] The server centrally integrates the collected data into a database (e.g., MongoDB). During this process, it converts data obtained from different data formats and sources into a consistent format (e.g., JSON), removes duplicate data, corrects inconsistent data, and appropriately complements any missing data. Data conversion is performed using a data conversion library (e.g., Pandas).

[0667] Analysis of user preferences and behavioral patterns

[0668] The server then applies machine learning algorithms (e.g., k-means clustering, random forest) to the preprocessed data to analyze user preferences and behavioral patterns. Using past data, the server identifies what products and services users are interested in and what behavioral patterns they exhibit. For example, it identifies patterns such as whether a user often goes to the movies on weekends or regularly purchases a particular brand of product.

[0669] Personalized product and service recommendations

[0670] The server then uses the analysis results to recommend the most suitable products and services for each user. When recommending products and services, the server takes into account the user's past behavioral data and preferences to select items that are likely to interest the user. For example, if a user is interested in movies, the server will recommend the latest movie tickets and movie-related goods.

[0671] Customizing the app's user interface

[0672] The device uses the recommendation list received from the server to customize how the app is displayed. Specifically, it adds the most relevant apps and links as shortcuts to the home screen. For example, it displays links to buy movie tickets or offers from nearby restaurants. It also rearranges apps so that when you open them, you can quickly find content that interests you.

[0673] Real-time feedback and model updates

[0674] The server monitors users' app usage in real time, collecting new data to retrain the machine learning model and improve the accuracy of the recommendation system. For example, if a user takes an action that indicates a new preference, that behavioral data is immediately collected and reflected in the next recommendation.

[0675] Specific Examples

[0676] For example, User A uses communication services, makes daily purchases using an electronic payment service, and uses a search engine to get the latest movie information. The server collects this data and analyzes the user's preferences and behavioral patterns. If User A is determined to have a high interest in movies, the server recommends the latest movie ticket information and coupons for theaters showing movies in the user's area. The user interface is customized accordingly, and movie-related links and apps are displayed on the home screen. Real-time feedback is also provided to immediately reflect any new content or services that User A has become interested in. In this way, users not only receive products and services that are optimized for them, but also use a convenient interface based on those products and services.

[0677] Prompt Sentence Examples

[0678] "User A uses a communication service and frequently searches for movie-related information. Please recommend the best movie-related products and services for this user."

[0679] As described above, the present invention is a system that provides users with an optimal experience by aggregating and analyzing a wide range of user data and recommending personalized products and services.

[0680] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0681] Step 1: Collect user data

[0682] The server collects communication data, purchase data, search data, etc. from the device used by the user. This data is acquired in real time while the user is actually using the application. Specifically, when a user makes a purchase, the purchase information (product ID, price, purchase date, etc.) is recorded. Communication data includes which app was used and for how long. This data is sent to the server in JSON format.

[0683] Input: Communication data, purchase data, search data collected from the device

[0684] Output: User data in JSON format

[0685] How it works: The device records user actions and usage in real time, encrypts this in JSON format, and sends it to the server.

[0686] Step 2: Data integration and preprocessing

[0687] The server stores the data received in step 1 in a database (e.g., MongoDB) and centrally consolidates it. During this process, it converts data from different formats (e.g., JSON, XML) into a consistent format (e.g., JSON), removes duplicate data, corrects inconsistent data, and fills in missing data.

[0688] Input: User data in JSON format

[0689] Output: Preprocessed integrated data

[0690] How it works: The server uses a data transformation library (e.g., Pandas) to convert data from each data source into a consistent format and cleanse the data.

[0691] Step 3: Analyzing user preferences and behavior patterns

[0692] The server then applies machine learning algorithms (e.g., k-means clustering, random forest) to the preprocessed data to analyze user preferences and behavioral patterns. Specifically, it extracts patterns such as a user often watching movies on weekends or regularly purchasing products from a particular brand.

[0693] Input: Preprocessed integrated data

[0694] Output: Analysis results based on user preferences and behavioral patterns

[0695] How it works: The server analyzes the data using machine learning models (e.g., scikit-learn) to identify user preferences and behavioral patterns.

[0696] Step 4: Personalized product and service recommendations

[0697] The server recommends the most suitable products and services to the user based on the analysis results of step 3. For example, if a user is interested in movies, the server will recommend the latest movie tickets and related goods. The recommendation results are generated in list form and stored in the server's database.

[0698] Input: Analysis results based on user preferences and behavioral patterns

[0699] Output: Recommendation list

[0700] How it works: The server uses collaborative filtering and content-based filtering to select the best products and services for the user and generate a list.

[0701] Step 5: Customizing the app's user interface

[0702] The device customizes the display of applications based on the recommendation list received from the server, dynamically changing the UI by adding links and shortcuts to the home screen that interest the user.

[0703] Input: Recommendation list

[0704] Output: Customized user interface

[0705] Operation: The device rearranges the home screen and menus based on the received recommendation list to provide a user-friendly interface.

[0706] Step 6: Real-time feedback and model updates

[0707] The server monitors user usage data sent from the device in real time, collecting new data and using it to retrain the machine learning model and improve the accuracy of the recommendation system.

[0708] Input: User usage data

[0709] Output: Updated machine learning model

[0710] How it works: The server uses continuously collected user data to retrain the machine learning model and improve recommendation accuracy.

[0711] Through the above processing steps, the system can provide optimized recommendations to users and provide a customized experience based on their latest preferences and behavioral patterns.

[0712] (Application example 1)

[0713] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0714] The problem that this invention aims to solve is to improve the user's shopping experience by utilizing a wide range of user behavioral data to improve the accuracy of personalized product recommendations and dynamically optimizing the user interface based on those recommendations. In particular, it is a challenge to immediately respond to new changes in user preferences through feedback based on real-time behavioral data and instantaneous system updates.

[0715] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0716] In this invention, the server includes means for collecting user information, means for integrating and preprocessing various data, means for analyzing user preferences and behavioral patterns, means for recommending products and services suitable for the user, means for customizing the user interface, means for providing real-time feedback on system usage status and updating the model, means for recommending optimal products and services based on the analysis results using real-time user behavioral data, and means for dynamically changing the interface based on the recommendation results. This makes it possible to precisely analyze user preferences and behavioral patterns, provide personalized products and services based on the analysis results in real time, and optimize the user interface.

[0717] "Means for collecting user information" refers to functions for collecting data on user behavior and preferences.

[0718] "Means for integrating and preprocessing various data" refers to the ability to centralize data collected from different formats and sources, convert it into a consistent format, and correct missing or inconsistent data.

[0719] "Means for analyzing user preferences and behavioral patterns" refers to a function for analyzing user interests and behavioral trends based on collected and pre-processed data.

[0720] "Means of recommending products and services suitable for users" is a function that suggests the most suitable products and services for each user based on the analysis results.

[0721] "Means for customizing the user interface" refers to a function that dynamically changes the display content of an application or website according to the needs and preferences of each user.

[0722] "Means of providing feedback on system usage in real time and updating the model" is a function that collects user behavior data in real time, updates the machine learning model based on the results, and improves the accuracy of recommendations.

[0723] "A means of recommending optimal products and services based on the analysis results using real-time user behavior data" is a function that analyzes user behavior data collected in real time and suggests optimal products and services based on the results.

[0724] The "means for dynamically changing the interface based on the recommendation result" is a function for dynamically changing the content and layout of the user interface based on the product recommendation result.

[0725] The present invention provides a system that collects a wide range of user behavior data, analyzes it to make personalized product recommendations, and dynamically optimizes the user interface based on the results. This system has the following main functions:

[0726] 1. Data Collection

[0727] The server collects user behavior data from a variety of sources. This behavior data includes, for example, communication data, purchase data, and search data. This data allows us to understand what applications users use, what they purchase at which stores, what keywords they searched for, and how much data they consume.

[0728] 2. Data integration and preprocessing

[0729] The server centrally integrates the collected data into a database, converts data from different formats and sources into a consistent format, removes duplicates, corrects inconsistencies, and appropriately completes missing data.

[0730] 3. Analysis of preferences and behavioral patterns

[0731] The server then applies machine learning algorithms to analyze user preferences and behavioral patterns based on the preprocessed data. Using past data, the server identifies the types of products and services users are interested in and their behavioral patterns. For example, it identifies patterns such as users who regularly purchase products from a particular brand or who are interested in movies.

[0732] 4. Product and service recommendations

[0733] Based on the analysis results, the server recommends the most suitable products and services for each user. For example, if a user is interested in movies, it will recommend the latest movie tickets and movie-related goods.

[0734] 5. Interface customization

[0735] The device customizes the user interface based on the recommendation list received from the server, adding relevant apps and links as shortcuts to the home screen and rearranging the content to make it easier to find what interests you.

[0736] 6. Real-time feedback and model updates

[0737] The server monitors users' app usage in real time, collecting new data to retrain the machine learning model and improve its accuracy. If a user exhibits behavior that indicates a new preference, that behavioral data is immediately collected and reflected in the next recommendation.

[0738] Specific examples

[0739] For example, if a user consumes a lot of movie-related data, the server will recommend the latest movie tickets and movie-related merchandise to the user. Furthermore, the user interface will be customized accordingly, with movie-related links and apps displayed on the home screen. As the user's preferences change, the system will instantly adapt and update the recommendations and interface accordingly.

[0740] Prompt Sentence Examples

[0741] "Based on the following user behavior data, please recommend the best movie-related services and tickets for this user. Consider their movie viewing history, purchase history, and search history to generate the best list to display movie tickets and related products for the weekend."

[0742] In this way, the present invention provides users with optimized products and services and dynamically changes the interface based on the recommendations, providing a more personalized shopping experience.

[0743] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0744] Step 1:

[0745] The server collects user behavior data from a variety of sources, including communication data, purchase data, and search data. Specifically, the server records what applications the user uses, what items they purchase at which stores, and what keywords they use to search. This data is stored in the server's database.

[0746] Step 2:

[0747] The server centrally integrates the collected data into a database and performs pre-processing. Data obtained from different data formats and sources is converted into a consistent format, duplicate data is removed, inconsistent data is corrected, and missing data is filled in. After processing, the data is saved back into the database in a clean state.

[0748] Step 3:

[0749] The server then applies machine learning algorithms to the preprocessed data to analyze user preferences and behavioral patterns. Specifically, it uses the collected historical data to train a model that predicts what products and services users are interested in. The trained model can then extract patterns, such as users' tendency to purchase certain brands of products on weekends.

[0750] Step 4:

[0751] The server then recommends the best products and services for each user based on the analysis results. Based on the predictions made by the machine learning model, it generates a list of products and services that the user may be interested in. This recommendation information is stored in the server's database and later sent to the device.

[0752] Step 5:

[0753] The device customizes the user interface based on the recommendation list received from the server. Specifically, it adds apps and links that are relevant to the user as shortcuts to the home screen and rearranges the layout so that users can quickly find content they are interested in. This customization allows users to use apps in an environment optimized for them.

[0754] Step 6:

[0755] The server monitors the user's app usage in real time and collects new data, which is used to retrain the machine learning model and improve the system's accuracy. For example, if a user takes an action that indicates a new preference, that behavioral data is immediately collected and reflected in the next recommendation. As a result, the product and service recommendations to the user are further refined.

[0756] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0757] This invention is a system that collects user information, integrates and preprocesses various data, analyzes user preferences and behavioral patterns, and recommends personalized products and services. In addition, by combining it with an emotion engine that recognizes user emotions, it is possible to achieve even greater accuracy and personalization, thereby providing users with the best possible experience.

[0758] User Data Collection

[0759] First, the server uses a means of collecting user information to obtain communication data, purchase data, search data, and even user emotion data through various APIs. Specifically, the server collects the user's communication history from the API of the communication service provider, purchase history from the API of the electronic payment service, and search history from the API of the search engine. Furthermore, to recognize the user's emotions, the emotion engine uses image processing and voice analysis technology to analyze facial expressions and tone of voice and obtain emotion data.

[0760] Data integration and preprocessing

[0761] The server consolidates and pre-processes the collected data. First, it converts data from different formats into a consistent format. Second, it removes duplicate data, corrects inconsistent data, and imputes missing data where necessary. This creates a clean and consistent dataset.

[0762] Analysis of user preferences and behavioral patterns

[0763] The server then runs machine learning algorithms based on the preprocessed data. Specifically, it uses clustering and classification algorithms to analyze user preferences and behavioral patterns. For example, a user who frequently searches for movies is classified into a cluster called movie lovers. By combining this with emotional data, it is possible to analyze the emotional state of a user when they behave in a particular way. For example, if a user looks happy while searching for a movie, it can be determined that the movie has a positive impact on the user.

[0764] Personalized product and service recommendations

[0765] The server then uses the analysis results to recommend the most suitable products and services for each user. When recommending products and services, the server takes into account the user's past behavioral data and preferences, as well as emotional data, to select items that are likely to interest the user. For example, if a user is interested in movies and looks happy, the server will recommend the latest movie tickets and movie-related merchandise.

[0766] Customizing the app's user interface

[0767] The device dynamically customizes the user interface based on the recommendation list received from the server. Specifically, it adds the most relevant apps and links as shortcuts to the home screen for quick access. For example, it displays a movie ticket purchasing app or offers from nearby restaurants. It also rearranges apps so that users can quickly find content they are interested in when they open them.

[0768] Real-time feedback and model updates

[0769] The server monitors users' app usage in real time and collects new data, including which recommended items users click on, which products they end up purchasing, and new emotional data from users, which will be used to refine future recommendations.

[0770] The server retrains the machine learning model based on the new data collected, improving the accuracy of the recommendation system. Any data indicating new preferences, behavioral patterns, or emotional data is immediately reflected in the model and reflected in the next recommendation results.

[0771] Specific Examples

[0772] For example, User A uses communication services, makes daily purchases using an electronic payment service, and uses a search engine to get the latest movie information. The server collects this data and analyzes the user's preferences, behavioral patterns, and emotional state. If User A is determined to have a high interest in movies, the server recommends the latest movie ticket information and coupons for theaters showing movies in the user's area. The user interface is customized accordingly, and movie-related links and apps are displayed on the home screen. Real-time feedback is also provided to immediately reflect any new content or services that User A has become interested in. In this way, users not only receive products and services that are optimized for them, but also use a convenient interface based on those products and services.

[0773] As described above, the present invention realizes a system that provides users with the optimal experience by aggregating and analyzing a wide range of user data and emotional data and recommending personalized products and services.

[0774] The processing flow will be explained below.

[0775] Step 1:

[0776] The server uses the means for collecting user information to obtain communication data, purchase data, search data, and user emotion data through various APIs. Communication history is obtained from the API of the communication service provider, purchase history from the API of the electronic payment service, and search history from the API of the search engine. Furthermore, using image processing and voice analysis technology, the emotion engine analyzes the user's facial expressions and tone of voice to collect emotion data.

[0777] Step 2:

[0778] The server consolidates and pre-processes the collected data. First, it converts data from different formats into a unified format. Second, it removes duplicate data, corrects inconsistent data, and imputes missing data. This process creates a clean and consistent dataset.

[0779] Step 3:

[0780] The server stores the preprocessed data in a database, where data such as communication data, purchase data, search data, and emotion data are stored by category.

[0781] Step 4:

[0782] The server uses the data stored in the database to run machine learning algorithms. Specifically, it applies clustering and classification algorithms to analyze user preferences and behavioral patterns. For example, a user who frequently searches for movies can be clustered as a movie lover. It also combines emotional data to analyze the emotional state of a user when they behave in a particular way.

[0783] Step 5:

[0784] The server creates a user profile based on the results of the machine learning algorithm, including the user's interests, behavioral patterns, emotional state, etc. This profile is then used for subsequent recommendation processing.

[0785] Step 6:

[0786] The server then recommends products and services based on the created profile. For example, if a user is a movie lover and looks happy, it can recommend the latest movie tickets and movie-related goods. By incorporating the user's emotional data, it can also recommend products that will help the user relax if they are feeling stressed.

[0787] Step 7:

[0788] The device analyzes the recommendation list received from the server and dynamically customizes the user interface, adding relevant apps and links as shortcuts to the home screen for quick access, such as a movie ticket purchasing app or nearby restaurant offers.

[0789] Step 8:

[0790] The server monitors users' app usage in real time and collects new data, including which recommended items users click on, which products they end up purchasing, and new emotional data from users, which is then reflected in the next recommendation.

[0791] Step 9:

[0792] The server retrains the machine learning model based on the new data collected, improving the accuracy of the recommendation system. If new preferences or behavioral patterns are discovered, they are immediately reflected in the model and applied to the next recommendation.

[0793] Step 10:

[0794] The server then applies the updated model to the user profile and re-recommends products and services. This process allows the system to constantly provide the most optimized services based on the latest user information and emotional data.

[0795] Through these steps, users can easily find products and services that match their preferences and emotional state, and can access the information and services they need with just their smartphone.

[0796] Example 2

[0797] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0798] Currently, many recommendation systems recommend products and services based on user behavioral data and purchase history, but these systems do not take emotional data into account, which means they cannot make recommendations that are optimized for the user's emotional state.In addition, due to insufficient real-time feedback and model updates, there is also the problem that the accuracy of recommendation results decreases when the user's interests and preferences change.

[0799] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for collecting user information, a means for integrating various data and performing preprocessing, a means for analyzing user preferences and behavior patterns, a means for recommending products and services suitable for the user, a means for acquiring and analyzing emotion data, a means for customizing the user interface, and a means for providing feedback on the system usage status in real time and updating the model. This not only enables the recommendation of products and services optimized for the user, but also enables highly accurate recommendations by updating the model in real time.

[0800] "Means for collecting user information" refers to devices and software for collecting data about users, including communication data, purchase data, search data, and emotional data, through various APIs and sensors.

[0801] "Means for integrating and pre-processing various types of data" refers to devices and software that convert the diverse data collected into a unified format and perform pre-processing such as deleting duplicates, correcting inconsistencies, and filling in missing data.

[0802] "Means for analyzing user preferences and behavioral patterns" refers to devices or software that use machine learning algorithms and statistical analysis methods based on preprocessed data to extract and classify user interests and behavioral patterns.

[0803] "Means for recommending products and services suitable for users" refers to devices or software that select products and services that match the user's preferences and behavioral patterns based on the analysis results and present them as a recommendation list.

[0804] "Means for acquiring and analyzing emotional data" refers to devices or software that use image processing technology or voice analysis technology to analyze a user's facial expressions and tone of voice and determine their emotional state.

[0805] "Means for customizing the user interface" refers to devices or software that dynamically change the layout and display content of the home screen based on the recommendation list, providing a UI that is easy for users to use.

[0806] "Means of providing feedback on system usage in real time and updating the model" refers to devices and software that monitor and collect user behavioral and emotional data in real time, and retrain the machine learning model based on this new data to improve recommendation accuracy.

[0807] This invention is a system that collects and analyzes user information and recommends personalized products and services with high accuracy. By adding emotional data acquisition and analysis functions, this system provides recommendations that also correspond to the user's emotional state, and by updating the model in real time, it achieves highly accurate recommendations.

[0808] User Data Collection

[0809] The server collects user information through various APIs. Communication data is obtained from the API of the communication service provider, purchase data from the API of the electronic payment service, and search data from the API of the search engine. In addition, a camera using image processing technology and a microphone using voice analysis technology are used to obtain emotional data from facial expressions and tone of voice. For example, the server can determine whether the user is smiling from facial expressions captured by the camera, and evaluate whether they are enjoying themselves from the tone of their voice recorded by the microphone.

[0810] Data integration and preprocessing

[0811] The server converts the collected data into a unified format and performs preprocessing such as deleting duplicates, correcting inconsistencies, and filling in missing data. Data obtained from each API is unified into formats such as JSON or CSV. Inconsistent data (for example, data with different date and time formats) is corrected, and missing data is inferred or filled in. This creates a consistent dataset.

[0812] Analysis of user preferences and behavioral patterns

[0813] The server then runs machine learning algorithms on the preprocessed data to analyze user preferences and behavioral patterns. Specifically, it uses clustering algorithms (e.g., K-means clustering) and classification algorithms (e.g., decision trees and random forests). For example, a user who frequently searches for movies might be classified into a cluster called a movie lover. It also combines emotional data to analyze behaviors under specific emotional states.

[0814] Personalized product and service recommendations

[0815] The server then uses the analysis results to recommend the most suitable products and services for each user. It selects the most suitable items by taking into consideration not only past behavioral data and preferences, but also emotional data. For example, if a user is interested in movies and looks happy, it will be recommended the latest movie tickets and movie-related merchandise.

[0816] Customizing the app's user interface

[0817] The device dynamically customizes the user interface based on the recommendation list received from the server, changing the home screen layout to prioritize relevant apps and links. For example, movie ticket purchasing apps and movie-related links are displayed on the home screen. The device also rearranges the layout to make new content that may be of interest easier to access.

[0818] Real-time feedback and model updates

[0819] The server monitors and collects user behavioral and emotional data in real time. This new data is used to retrain the machine learning model and improve the accuracy of the recommendation system. The model parameters are updated based on new preferences and behavioral patterns. For example, the system can immediately respond to new content or products that users have become interested in.

[0820] As described above, the present invention enables users to recommend products and services that are optimized for them, and provides highly accurate recommendations through real-time model updates. Specifically, the following prompt sentences can be used:

[0821] Example prompt sentence:

[0822] "A method to recommend optimal products and services to a user searching for the latest movie tickets, taking into account the user's emotional data."

[0823] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0824] Step 1:

[0825] The server collects communication history, purchase history, search history, and emotion data via various APIs and sensors. Communication data is obtained from the API of the communication service provider, purchase data from the API of the electronic payment service, and search data from the API of the search engine. Emotion data is obtained from a camera using image processing technology and a microphone using voice analysis technology. The input is data from each API and sensor, and the output is a dataset of collected user information. The server stores this data in a repository where it is temporarily saved.

[0826] Step 2:

[0827] The server integrates and preprocesses the collected data. Specifically, it converts each piece of collected data into JSON or CSV format. It then removes duplicate data, corrects inconsistent data, and fills in missing data. The input is a dataset of collected user information, and the output is a clean dataset that has been integrated and preprocessed. The server runs software processes to remove duplicates and convert formats, creating a highly consistent dataset.

[0828] Step 3:

[0829] The server runs a machine learning algorithm on the preprocessed data to analyze user preferences and behavioral patterns. Specifically, it uses a clustering algorithm (e.g., K-means) to group users, and then uses a classification algorithm (e.g., decision tree or random forest) to analyze individual behavioral patterns. The input is the preprocessed, clean dataset, and the output is a model of user preferences and behavioral patterns. The server stores the results of clustering and classification in a database.

[0830] Step 4:

[0831] The server recommends optimal products and services to users based on the analysis results. The server selects the most suitable items by taking into account past behavioral data, preference data, and emotional data. The input is a model of the user's preferences and behavioral patterns, and the output is a personalized recommendation list. The server generates the recommendation list and prepares data corresponding to each user.

[0832] Step 5:

[0833] The device dynamically customizes the user interface based on the recommendation list received from the server. Specifically, it changes the home screen layout and adds relevant apps and links. The input is the recommendation list from the server, and the output is the customized user interface. The device adjusts the screen layout and adds shortcuts.

[0834] Step 6:

[0835] The server monitors users' behavioral and emotional data in real time and collects new data. The input is real-time data from the user's device, and the output is the latest behavioral data set. The server retrains the machine learning model based on this new data to improve the accuracy of the recommendation system. Specifically, it monitors click data, purchase history, and fluctuations in emotional state, and updates the model accordingly.

[0836] (Application example 2)

[0837] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0838] Conventional recommendation systems tend to recommend products and services based on static data such as a user's purchase history and search history, and are insufficient in responding to the user's emotional state or real-time behavior. Therefore, there is a demand for more appropriate and effective advertisements and product / service recommendations for users. This invention aims to solve the problems of conventional systems by providing personalized, highly accurate recommendations that also take into account the user's emotional data, thereby providing users with an optimal experience.

[0839] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting user information, means for integrating various data and performing preprocessing, means for analyzing user preferences and behavioral patterns, means for recommending products and services suitable for the user, means for recognizing the user's emotions, means for customizing the user interface, means for providing feedback on system usage status in real time and updating the model, and means for recommending and displaying personalized advertisements based on emotional data. This makes it possible to recommend appropriate products and services and display advertisements in real time based on the user's emotional state.

[0840] "User Information" refers to various information about a user, including communication data, purchase data, search data, and emotional data.

[0841] "Means for integrating and pre-processing various data" refers to the part of the system that converts data of different formats into a consistent format, removes duplicate data, corrects inconsistent data, and completes missing data.

[0842] "Means for analyzing user preferences and behavioral patterns" refers to the part of the system that uses machine learning algorithms to analyze patterns based on users' interests and past behavioral history.

[0843] "Means for recommending products and services suitable for users" refers to the process of recommending the most relevant products and services based on the analysis of users' preferences and behavioral patterns and emotional data.

[0844] "Means for recognizing the user's emotions" refers to the part of the system that uses image processing and voice analysis technology to obtain and analyze emotional data from the user's facial expressions and tone of voice.

[0845] "Means for customizing the user interface" refers to the process of dynamically changing the interface layout and display content based on the analysis results and the list of recommended products and services.

[0846] "Means of providing real-time feedback on system usage and updating the model" refers to the process of collecting user operation status and new data in real time, and using this information to retrain the machine learning model and improve its accuracy.

[0847] "Means for recommending and displaying personalized advertisements based on emotional data" refers to the process of selecting and displaying the most appropriate advertisements individually based on the user's current emotional state.

[0848] This invention is a system that collects and analyzes a wide variety of user information, including emotional data, to recommend personalized and highly accurate products and services. In particular, we describe the recommendation and display of advertisements that utilize emotional data. This system mainly consists of a server that collects and analyzes user information, and a terminal that provides the user interface.

[0849] Data collection

[0850] First, the server collects user information such as communication data, purchase data, and search data via various APIs. Specific hardware used includes smartphones, smart glasses, and head-mounted displays. The server then uses the data acquired from these devices and an emotion engine to obtain emotional data from facial expressions and tone of voice. For example, the server uses the smartphone's camera and microphone to perform facial recognition and voice analysis to obtain the user's emotions in real time.

[0851] Data integration and preprocessing

[0852] The server consolidates and preprocesses the collected data. Specifically, cloud platforms (such as AWS and Google Cloud) are used for this purpose. This process involves unifying data in different formats, removing duplicates, correcting inconsistent data, and filling in missing data. This process produces a clean, consistent dataset, facilitating subsequent analysis.

[0853] Analysis of preferences and behavioral patterns

[0854] The server uses machine learning algorithms to analyze user preferences and behavioral patterns based on the preprocessed data. Specifically, it uses machine learning libraries such as TensorFlow and PyTorch. This allows for clustering and classification based on users' interests and past behavior. Emotional data is also included in the analysis, making it possible to analyze the emotional state of users that leads them to take specific actions.

[0855] Product and service recommendations

[0856] The server then uses the analysis results to recommend the most suitable products and services to the user. By combining this with emotional data, the system can identify and recommend items and advertisements that are likely to interest the user in real time. For example, if a user searches for something related to movies and shows an excited expression, an advertisement for movie tickets will be displayed.

[0857] User interface customization

[0858] The device dynamically changes the user interface based on the recommendation list received from the server, for example, adding a shortcut to a movie ticket purchasing app on the home screen or displaying related offers, allowing users to quickly access the information and services they need.

[0859] Real-time feedback and model updates

[0860] The server monitors user behavior in real time and collects new data, including information about ads clicked, products purchased, and even new emotional data, which it uses to retrain machine learning models and improve the accuracy of the recommendation system.

[0861] Examples of prompt statements

[0862] When a user searches for something related to movies, if a happy expression is recognized, the system will generate a prompt like the one below based on that emotional data and display an advertisement for movie tickets.

[0863] If a user searches for "movie updates" and then looks happy:

[0864] "Check out our latest offers on movie tickets and movie merchandise for great savings!"

[0865] In this way, by making recommendations that reflect real-time emotional data, a system can be created that provides users with the optimal experience.

[0866] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0867] Step 1:

[0868] The server collects communication data, purchase data, and search data via various APIs. Specifically, it obtains this data from devices such as smartphones and smart glasses, and collects emotional data from facial expressions and tone of voice through an emotion engine. The server uses this data to initialize the user's interests and concerns. The inputs are communication data, purchase data, search data, and emotional data, and the output is integrated user data.

[0869] Step 2:

[0870] The server converts the collected data into a unified format, removes duplicates, corrects inconsistent data, and fills in missing data. The specific software used is a cloud platform (AWS or Google Cloud). The input is consolidated user data, and the output is a clean, consistent dataset.

[0871] Step 3:

[0872] The server runs machine learning algorithms (such as TensorFlow or PyTorch) on the clean dataset to analyze user preferences and behavioral patterns. In this step, pattern analysis is performed using clustering and classification algorithms. Emotional data is also taken into account, and behavioral patterns in specific emotional states are also analyzed. The input is the clean dataset, and the output is the analyzed user preferences and behavioral patterns.

[0873] Step 4:

[0874] The server recommends optimal products, services, and advertisements for each user based on the analysis results. In particular, it utilizes emotional data to select items and advertisements that are likely to interest the user in real time. It also uses a generative AI model to generate prompts and use them to display advertisements. The input to this step is the analysis results, and the output is recommended products, services, and advertisements.

[0875] Step 5:

[0876] The device customizes the user interface based on the recommendation list obtained from the server. For example, it adds a shortcut to a movie ticket purchasing app on the home screen and displays related offers. This allows the user to quickly access the information and services they need. The input is the recommendation list, and the output is the customized user interface.

[0877] Step 6:

[0878] The server monitors user behavior in real time and collects new data, including information about ads clicked, products purchased, and even new emotional data. The server uses this data to retrain the machine learning model and improve the accuracy of the recommendation system. The input is real-time user behavior data, and the output is an updated machine learning model.

[0879] In this way, specific data processing and calculations are carried out at each step, resulting in a system that provides users with advertisements and services optimized for them in real time.

[0880] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0881] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0882] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[0883] [Fourth embodiment]

[0884] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0885] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0886] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0887] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0888] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0889] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0890] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0891] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0892] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0893] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0894] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0895] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0896] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[0897] This system collects user information, integrates and preprocesses various data, analyzes user preferences and behavioral patterns, and recommends personalized products and services. Furthermore, it customizes the user interface based on the recommendation results, provides real-time feedback on system usage, and updates the model to provide the optimal experience for users.

[0898] User Data Collection

[0899] First, the server collects user information. This information is collected from a wide range of data sources, including communication data, purchase data, and search data. For example, communication data may include what applications the user is using and how much data traffic they are consuming. Purchase data may include what store the user purchased what and how much they paid. Search data may include what keywords the user used in their search and which web pages they viewed.

[0900] Data integration and preprocessing

[0901] The server centrally integrates the collected data into a database, converting data from different formats and sources into a consistent format, removing duplicate data, correcting inconsistencies, and properly filling in missing data.

[0902] Analysis of user preferences and behavioral patterns

[0903] The server then applies machine learning algorithms to the preprocessed data to analyze user preferences and behavioral patterns. Using past data, the server identifies the types of products and services a user is interested in and the behavioral patterns they exhibit. For example, it identifies patterns such as whether a user frequently goes to the movies on weekends or regularly purchases a particular brand of product.

[0904] Personalized product and service recommendations

[0905] The server then uses the analysis results to recommend the most suitable products and services for each user. When recommending products and services, the server takes into account the user's past behavioral data and preferences to select items that are likely to interest the user. For example, if a user is interested in movies, the server will recommend the latest movie tickets and movie-related goods.

[0906] Customizing the app's user interface

[0907] The device uses the recommendation list received from the server to customize how the app is displayed. Specifically, it adds the most relevant apps and links as shortcuts to the home screen. For example, it displays links to buy movie tickets or offers from nearby restaurants. It also rearranges apps so that when you open them, you can quickly find content that interests you.

[0908] Real-time feedback and model updates

[0909] The server monitors users' app usage in real time, collecting new data to retrain the machine learning model and improve the accuracy of the recommendation system. For example, if a user takes an action that indicates a new preference, that behavioral data is immediately collected and reflected in the next recommendation.

[0910] Specific Examples

[0911] For example, User A uses communication services, makes daily purchases using an electronic payment service, and uses a search engine to get the latest movie information. The server collects this data and analyzes the user's preferences and behavioral patterns. If User A is determined to have a high interest in movies, the server recommends the latest movie ticket information and coupons for theaters showing movies in the user's area. The user interface is customized accordingly, and movie-related links and apps are displayed on the home screen. Real-time feedback is also provided to immediately reflect any new content or services that User A has become interested in. In this way, users not only receive products and services that are optimized for them, but also use a convenient interface based on those products and services.

[0912] As described above, the present invention realizes a system that provides users with the most optimal experience by aggregating and analyzing a wide range of user data and recommending personalized products and services.

[0913] The processing flow will be explained below.

[0914] Step 1:

[0915] The server uses the means for collecting user information to obtain a wide range of data, including communication data, purchase data, and search data, through various APIs. Specifically, the server collects the user's communication history from the API of a communication service provider, purchase history from the API of an electronic payment service, and search history from the API of a search engine.

[0916] Step 2:

[0917] The server consolidates and pre-processes the collected data. First, it converts data from different formats into a consistent format. Second, it removes duplicate data, corrects inconsistent data, and imputes missing data where necessary. This creates a clean and consistent dataset.

[0918] Step 3:

[0919] The server stores the pre-processed data in a database, which is used for subsequent analysis and recommendation processing. The database stores data by category, such as communication data, purchase data, and search data.

[0920] Step 4:

[0921] The server uses the data stored in the database to run machine learning algorithms, such as clustering and classification algorithms, to analyze user preferences and behavioral patterns. For example, a user who frequently searches for movies might be classified as a movie lover.

[0922] Step 5:

[0923] The server then uses the results of the machine learning algorithm to create a user profile, which includes the user's interests, behavioral patterns, purchase history, etc. This profile is then used to make personalized recommendations of products and services.

[0924] Step 6:

[0925] The server then applies an algorithm to recommend products and services based on the created profile. For example, if a user has a profile for movie lovers, the server will recommend the latest movie tickets and movie-related merchandise. The recommendation results are generated in the form of a list and sent to the user's device.

[0926] Step 7:

[0927] The device analyzes the recommendation list received from the server and dynamically customizes the user interface, adding the most relevant apps and links as shortcuts to the home screen for quick access, such as a movie ticket purchasing app or nearby restaurant offers.

[0928] Step 8:

[0929] The server monitors users' app usage in real time and collects new data, including which recommended items users click on and which products they end up purchasing, to help refine future recommendations.

[0930] Step 9:

[0931] The server retrains the machine learning model based on the new data collected, improving the accuracy of the recommendation system. Any data that indicates new preferences or behavioral patterns is immediately reflected in the model and reflected in the next recommendation results.

[0932] Step 10:

[0933] The server then applies the updated model to the user profile and re-recommends products and services. In this way, the system continues to provide optimized services based on the latest user information.

[0934] By following these steps, users can easily find products and services that match their preferences and behavioral patterns, and can access the information and services they need with just one smartphone.

[0935] Example 1

[0936] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[0937] In today's information society, consumers are bombarded with a vast amount of information, making it difficult to select optimal products and services. Furthermore, existing recommendation systems often cannot adequately reflect individual user preferences and behavioral patterns, and can only provide generalized recommendations. Furthermore, there is a need for highly accurate recommendations that reflect changes in user preferences and behavior in real time. An efficient and effective system is needed to solve these problems.

[0938] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0939] In this invention, the server includes a means for collecting user information, a means for integrating and preprocessing various data, and a means for analyzing user preferences and behavioral patterns, which enables highly accurate analysis of user preferences and behavioral patterns and the recommendation of optimal products and services to users.

[0940] "Means for collecting user information" refers to processes and devices for collecting various data related to users, such as communication data, purchase data, and search data.

[0941] "Means of integrating and pre-processing various data" refers to processes or devices that convert data collected from different data sources into a consistent format, remove duplicate data, correct inconsistent data, and fill in missing data.

[0942] "Means for analyzing user preferences and behavioral patterns" refers to processes or devices that apply machine learning algorithms or other analytical techniques to the collected and pre-processed data to identify user preferences and behavioral patterns.

[0943] "Means for recommending products and services suitable for users" refers to the process or device that selects and presents the most suitable products and services based on the analyzed preferences and behavioral patterns of users.

[0944] "Means for customizing user interfaces" refers to processes or devices that change the display content and operation methods of applications and systems based on user preferences and behavioral patterns, making them easier for users to use.

[0945] "Means for providing real-time feedback on system usage and updating the model" refers to the process or device that monitors and collects data on users' use of the system in real time, and uses that data to retrain the machine learning model, thereby improving the accuracy of the system's recommendations.

[0946] "Means for encrypting data received from a terminal and sending it to a server" refers to a process or device for encrypting data collected from a user's terminal and securely sending it to a server.

[0947] "Means of transforming data from different formats into a consistent format" refers to a process or device that transforms data provided in different formats (e.g., JSON, XML) into a unified format.

[0948] "Means for applying machine learning algorithms to analyze user preferences and behavioral patterns" refers to a process or device that uses machine learning techniques to identify user preferences and behavioral patterns from pre-processed data.

[0949] System Overview

[0950] This system collects user information, integrates and preprocesses various data, analyzes user preferences and behavioral patterns, and recommends personalized products and services. Furthermore, it customizes the user interface based on the recommendation results, provides real-time feedback on system usage, and updates the model to provide the optimal experience for users.

[0951] User Data Collection

[0952] First, the server collects user information. This information is collected from a wide range of data sources, including communication data, purchase data, and search data. For example, communication data may include what applications the user is using and how much data traffic they are consuming. Purchase data may include what store the user purchased what and how much they paid. Search data may include what keywords the user used in their search and which web pages they viewed.

[0953] Data integration and preprocessing

[0954] The server centrally integrates the collected data into a database (e.g., MongoDB). During this process, it converts data obtained from different data formats and sources into a consistent format (e.g., JSON), removes duplicate data, corrects inconsistent data, and appropriately complements any missing data. Data conversion is performed using a data conversion library (e.g., Pandas).

[0955] Analysis of user preferences and behavioral patterns

[0956] The server then applies machine learning algorithms (e.g., k-means clustering, random forest) to the preprocessed data to analyze user preferences and behavioral patterns. Using past data, the server identifies what products and services users are interested in and what behavioral patterns they exhibit. For example, it identifies patterns such as whether a user often goes to the movies on weekends or regularly purchases a particular brand of product.

[0957] Personalized product and service recommendations

[0958] The server then uses the analysis results to recommend the most suitable products and services for each user. When recommending products and services, the server takes into account the user's past behavioral data and preferences to select items that are likely to interest the user. For example, if a user is interested in movies, the server will recommend the latest movie tickets and movie-related goods.

[0959] Customizing the app's user interface

[0960] The device uses the recommendation list received from the server to customize how the app is displayed. Specifically, it adds the most relevant apps and links as shortcuts to the home screen. For example, it displays links to buy movie tickets or offers from nearby restaurants. It also rearranges apps so that when you open them, you can quickly find content that interests you.

[0961] Real-time feedback and model updates

[0962] The server monitors users' app usage in real time, collecting new data to retrain the machine learning model and improve the accuracy of the recommendation system. For example, if a user takes an action that indicates a new preference, that behavioral data is immediately collected and reflected in the next recommendation.

[0963] Specific Examples

[0964] For example, User A uses communication services, makes daily purchases using an electronic payment service, and uses a search engine to get the latest movie information. The server collects this data and analyzes the user's preferences and behavioral patterns. If User A is determined to have a high interest in movies, the server recommends the latest movie ticket information and coupons for theaters showing movies in the user's area. The user interface is customized accordingly, and movie-related links and apps are displayed on the home screen. Real-time feedback is also provided to immediately reflect any new content or services that User A has become interested in. In this way, users not only receive products and services that are optimized for them, but also use a convenient interface based on those products and services.

[0965] Prompt Sentence Examples

[0966] "User A uses a communication service and frequently searches for movie-related information. Please recommend the best movie-related products and services for this user."

[0967] As described above, the present invention is a system that provides users with an optimal experience by aggregating and analyzing a wide range of user data and recommending personalized products and services.

[0968] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0969] Step 1: Collect user data

[0970] The server collects communication data, purchase data, search data, etc. from the device used by the user. This data is acquired in real time while the user is actually using the application. Specifically, when a user makes a purchase, the purchase information (product ID, price, purchase date, etc.) is recorded. Communication data includes which app was used and for how long. This data is sent to the server in JSON format.

[0971] Input: Communication data, purchase data, search data collected from the device

[0972] Output: User data in JSON format

[0973] How it works: The device records user actions and usage in real time, encrypts this in JSON format, and sends it to the server.

[0974] Step 2: Data integration and preprocessing

[0975] The server stores the data received in step 1 in a database (e.g., MongoDB) and centrally consolidates it. During this process, it converts data from different formats (e.g., JSON, XML) into a consistent format (e.g., JSON), removes duplicate data, corrects inconsistent data, and fills in missing data.

[0976] Input: User data in JSON format

[0977] Output: Preprocessed integrated data

[0978] How it works: The server uses a data transformation library (e.g., Pandas) to convert data from each data source into a consistent format and cleanse the data.

[0979] Step 3: Analyzing user preferences and behavior patterns

[0980] The server then applies machine learning algorithms (e.g., k-means clustering, random forest) to the preprocessed data to analyze user preferences and behavioral patterns. Specifically, it extracts patterns such as a user often watching movies on weekends or regularly purchasing products from a particular brand.

[0981] Input: Preprocessed integrated data

[0982] Output: Analysis results based on user preferences and behavioral patterns

[0983] How it works: The server analyzes the data using machine learning models (e.g., scikit-learn) to identify user preferences and behavioral patterns.

[0984] Step 4: Personalized product and service recommendations

[0985] The server recommends the most suitable products and services to the user based on the analysis results of step 3. For example, if a user is interested in movies, the server will recommend the latest movie tickets and related goods. The recommendation results are generated in list form and stored in the server's database.

[0986] Input: Analysis results based on user preferences and behavioral patterns

[0987] Output: Recommendation list

[0988] How it works: The server uses collaborative filtering and content-based filtering to select the best products and services for the user and generate a list.

[0989] Step 5: Customizing the app's user interface

[0990] The device customizes the display of applications based on the recommendation list received from the server, dynamically changing the UI by adding links and shortcuts to the home screen that interest the user.

[0991] Input: Recommendation list

[0992] Output: Customized user interface

[0993] Operation: The device rearranges the home screen and menus based on the received recommendation list to provide a user-friendly interface.

[0994] Step 6: Real-time feedback and model updates

[0995] The server monitors user usage data sent from the device in real time, collecting new data and using it to retrain the machine learning model and improve the accuracy of the recommendation system.

[0996] Input: User usage data

[0997] Output: Updated machine learning model

[0998] How it works: The server uses continuously collected user data to retrain the machine learning model and improve recommendation accuracy.

[0999] Through the above processing steps, the system can provide optimized recommendations to users and provide a customized experience based on their latest preferences and behavioral patterns.

[1000] (Application example 1)

[1001] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1002] The problem that this invention aims to solve is to improve the user's shopping experience by utilizing a wide range of user behavioral data to improve the accuracy of personalized product recommendations and dynamically optimizing the user interface based on those recommendations. In particular, it is a challenge to immediately respond to new changes in user preferences through feedback based on real-time behavioral data and instantaneous system updates.

[1003] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1004] In this invention, the server includes means for collecting user information, means for integrating and preprocessing various data, means for analyzing user preferences and behavioral patterns, means for recommending products and services suitable for the user, means for customizing the user interface, means for providing real-time feedback on system usage status and updating the model, means for recommending optimal products and services based on the analysis results using real-time user behavioral data, and means for dynamically changing the interface based on the recommendation results. This makes it possible to precisely analyze user preferences and behavioral patterns, provide personalized products and services based on the analysis results in real time, and optimize the user interface.

[1005] "Means for collecting user information" refers to functions for collecting data on user behavior and preferences.

[1006] "Means for integrating and preprocessing various data" refers to the ability to centralize data collected from different formats and sources, convert it into a consistent format, and correct missing or inconsistent data.

[1007] "Means for analyzing user preferences and behavioral patterns" refers to a function for analyzing user interests and behavioral trends based on collected and pre-processed data.

[1008] "Means of recommending products and services suitable for users" is a function that suggests the most suitable products and services for each user based on the analysis results.

[1009] "Means for customizing the user interface" refers to a function that dynamically changes the display content of an application or website according to the needs and preferences of each user.

[1010] "Means of providing feedback on system usage in real time and updating the model" is a function that collects user behavior data in real time, updates the machine learning model based on the results, and improves the accuracy of recommendations.

[1011] "A means of recommending optimal products and services based on the analysis results using real-time user behavior data" is a function that analyzes user behavior data collected in real time and suggests optimal products and services based on the results.

[1012] The "means for dynamically changing the interface based on the recommendation result" is a function for dynamically changing the content and layout of the user interface based on the product recommendation result.

[1013] The present invention provides a system that collects a wide range of user behavior data, analyzes it to make personalized product recommendations, and dynamically optimizes the user interface based on the results. This system has the following main functions:

[1014] 1. Data Collection

[1015] The server collects user behavior data from a variety of sources. This behavior data includes, for example, communication data, purchase data, and search data. This data allows us to understand what applications users use, what they purchase at which stores, what keywords they searched for, and how much data they consume.

[1016] 2. Data integration and preprocessing

[1017] The server centrally integrates the collected data into a database, converts data from different formats and sources into a consistent format, removes duplicates, corrects inconsistencies, and appropriately completes missing data.

[1018] 3. Analysis of preferences and behavioral patterns

[1019] The server then applies machine learning algorithms to analyze user preferences and behavioral patterns based on the preprocessed data. Using past data, the server identifies the types of products and services users are interested in and their behavioral patterns. For example, it identifies patterns such as users who regularly purchase products from a particular brand or who are interested in movies.

[1020] 4. Product and service recommendations

[1021] Based on the analysis results, the server recommends the most suitable products and services for each user. For example, if a user is interested in movies, it will recommend the latest movie tickets and movie-related goods.

[1022] 5. Interface customization

[1023] The device customizes the user interface based on the recommendation list received from the server, adding relevant apps and links as shortcuts to the home screen and rearranging the content to make it easier to find what interests you.

[1024] 6. Real-time feedback and model updates

[1025] The server monitors users' app usage in real time, collecting new data to retrain the machine learning model and improve its accuracy. If a user exhibits behavior that indicates a new preference, that behavioral data is immediately collected and reflected in the next recommendation.

[1026] Specific examples

[1027] For example, if a user consumes a lot of movie-related data, the server will recommend the latest movie tickets and movie-related merchandise to the user. Furthermore, the user interface will be customized accordingly, with movie-related links and apps displayed on the home screen. As the user's preferences change, the system will instantly adapt and update the recommendations and interface accordingly.

[1028] Prompt Sentence Examples

[1029] "Based on the following user behavior data, please recommend the best movie-related services and tickets for this user. Consider their movie viewing history, purchase history, and search history to generate the best list to display movie tickets and related products for the weekend."

[1030] In this way, the present invention provides users with optimized products and services and dynamically changes the interface based on the recommendations, providing a more personalized shopping experience.

[1031] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1032] Step 1:

[1033] The server collects user behavior data from a variety of sources, including communication data, purchase data, and search data. Specifically, the server records what applications the user uses, what items they purchase at which stores, and what keywords they use to search. This data is stored in the server's database.

[1034] Step 2:

[1035] The server centrally integrates the collected data into a database and performs pre-processing. Data obtained from different data formats and sources is converted into a consistent format, duplicate data is removed, inconsistent data is corrected, and missing data is filled in. After processing, the data is saved back into the database in a clean state.

[1036] Step 3:

[1037] The server then applies machine learning algorithms to the preprocessed data to analyze user preferences and behavioral patterns. Specifically, it uses the collected historical data to train a model that predicts what products and services users are interested in. The trained model can then extract patterns, such as users' tendency to purchase certain brands of products on weekends.

[1038] Step 4:

[1039] The server then recommends the best products and services for each user based on the analysis results. Based on the predictions made by the machine learning model, it generates a list of products and services that the user may be interested in. This recommendation information is stored in the server's database and later sent to the device.

[1040] Step 5:

[1041] The device customizes the user interface based on the recommendation list received from the server. Specifically, it adds apps and links that are relevant to the user as shortcuts to the home screen and rearranges the layout so that users can quickly find content they are interested in. This customization allows users to use apps in an environment optimized for them.

[1042] Step 6:

[1043] The server monitors the user's app usage in real time and collects new data, which is used to retrain the machine learning model and improve the system's accuracy. For example, if a user takes an action that indicates a new preference, that behavioral data is immediately collected and reflected in the next recommendation. As a result, the product and service recommendations to the user are further refined.

[1044] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1045] This invention is a system that collects user information, integrates and preprocesses various data, analyzes user preferences and behavioral patterns, and recommends personalized products and services. In addition, by combining it with an emotion engine that recognizes user emotions, it is possible to achieve even greater accuracy and personalization, thereby providing users with the best possible experience.

[1046] User Data Collection

[1047] First, the server uses a means of collecting user information to obtain communication data, purchase data, search data, and even user emotion data through various APIs. Specifically, the server collects the user's communication history from the API of the communication service provider, purchase history from the API of the electronic payment service, and search history from the API of the search engine. Furthermore, to recognize the user's emotions, the emotion engine uses image processing and voice analysis technology to analyze facial expressions and tone of voice and obtain emotion data.

[1048] Data integration and preprocessing

[1049] The server consolidates and pre-processes the collected data. First, it converts data from different formats into a consistent format. Second, it removes duplicate data, corrects inconsistent data, and imputes missing data where necessary. This creates a clean and consistent dataset.

[1050] Analysis of user preferences and behavioral patterns

[1051] The server then runs machine learning algorithms based on the preprocessed data. Specifically, it uses clustering and classification algorithms to analyze user preferences and behavioral patterns. For example, a user who frequently searches for movies is classified into a cluster called movie lovers. By combining this with emotional data, it is possible to analyze the emotional state of a user when they behave in a particular way. For example, if a user looks happy while searching for a movie, it can be determined that the movie has a positive impact on the user.

[1052] Personalized product and service recommendations

[1053] The server then uses the analysis results to recommend the most suitable products and services for each user. When recommending products and services, the server takes into account the user's past behavioral data and preferences, as well as emotional data, to select items that are likely to interest the user. For example, if a user is interested in movies and looks happy, the server will recommend the latest movie tickets and movie-related merchandise.

[1054] Customizing the app's user interface

[1055] The device dynamically customizes the user interface based on the recommendation list received from the server. Specifically, it adds the most relevant apps and links as shortcuts to the home screen for quick access. For example, it displays a movie ticket purchasing app or offers from nearby restaurants. It also rearranges apps so that users can quickly find content they are interested in when they open them.

[1056] Real-time feedback and model updates

[1057] The server monitors users' app usage in real time and collects new data, including which recommended items users click on, which products they end up purchasing, and new emotional data from users, which will be used to refine future recommendations.

[1058] The server retrains the machine learning model based on the new data collected, improving the accuracy of the recommendation system. Any data indicating new preferences, behavioral patterns, or emotional data is immediately reflected in the model and reflected in the next recommendation results.

[1059] Specific Examples

[1060] For example, User A uses communication services, makes daily purchases using an electronic payment service, and uses a search engine to get the latest movie information. The server collects this data and analyzes the user's preferences, behavioral patterns, and emotional state. If User A is determined to have a high interest in movies, the server recommends the latest movie ticket information and coupons for theaters showing movies in the user's area. The user interface is customized accordingly, and movie-related links and apps are displayed on the home screen. Real-time feedback is also provided to immediately reflect any new content or services that User A has become interested in. In this way, users not only receive products and services that are optimized for them, but also use a convenient interface based on those products and services.

[1061] As described above, the present invention realizes a system that provides users with the optimal experience by aggregating and analyzing a wide range of user data and emotional data and recommending personalized products and services.

[1062] The processing flow will be explained below.

[1063] Step 1:

[1064] The server uses the means for collecting user information to obtain communication data, purchase data, search data, and user emotion data through various APIs. Communication history is obtained from the API of the communication service provider, purchase history from the API of the electronic payment service, and search history from the API of the search engine. Furthermore, using image processing and voice analysis technology, the emotion engine analyzes the user's facial expressions and tone of voice to collect emotion data.

[1065] Step 2:

[1066] The server consolidates and pre-processes the collected data. First, it converts data from different formats into a unified format. Second, it removes duplicate data, corrects inconsistent data, and imputes missing data. This process creates a clean and consistent dataset.

[1067] Step 3:

[1068] The server stores the preprocessed data in a database, where data such as communication data, purchase data, search data, and emotion data are stored by category.

[1069] Step 4:

[1070] The server uses the data stored in the database to run machine learning algorithms. Specifically, it applies clustering and classification algorithms to analyze user preferences and behavioral patterns. For example, a user who frequently searches for movies can be clustered as a movie lover. It also combines emotional data to analyze the emotional state of a user when they behave in a particular way.

[1071] Step 5:

[1072] The server creates a user profile based on the results of the machine learning algorithm, including the user's interests, behavioral patterns, emotional state, etc. This profile is then used for subsequent recommendation processing.

[1073] Step 6:

[1074] The server then recommends products and services based on the created profile. For example, if a user is a movie lover and looks happy, it can recommend the latest movie tickets and movie-related goods. By incorporating the user's emotional data, it can also recommend products that will help the user relax if they are feeling stressed.

[1075] Step 7:

[1076] The device analyzes the recommendation list received from the server and dynamically customizes the user interface, adding relevant apps and links as shortcuts to the home screen for quick access, such as a movie ticket purchasing app or nearby restaurant offers.

[1077] Step 8:

[1078] The server monitors users' app usage in real time and collects new data, including which recommended items users click on, which products they end up purchasing, and new emotional data from users, which is then reflected in the next recommendation.

[1079] Step 9:

[1080] The server retrains the machine learning model based on the new data collected, improving the accuracy of the recommendation system. If new preferences or behavioral patterns are discovered, they are immediately reflected in the model and applied to the next recommendation.

[1081] Step 10:

[1082] The server then applies the updated model to the user profile and re-recommends products and services. This process allows the system to constantly provide the most optimized services based on the latest user information and emotional data.

[1083] Through these steps, users can easily find products and services that match their preferences and emotional state, and can access the information and services they need with just their smartphone.

[1084] Example 2

[1085] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1086] Currently, many recommendation systems recommend products and services based on user behavioral data and purchase history, but these systems do not take emotional data into account, which means they cannot make recommendations that are optimized for the user's emotional state.In addition, due to insufficient real-time feedback and model updates, there is also the problem that the accuracy of recommendation results decreases when the user's interests and preferences change.

[1087] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for collecting user information, a means for integrating various data and performing preprocessing, a means for analyzing user preferences and behavior patterns, a means for recommending products and services suitable for the user, a means for acquiring and analyzing emotion data, a means for customizing the user interface, and a means for providing feedback on the system usage status in real time and updating the model. This not only enables the recommendation of products and services optimized for the user, but also enables highly accurate recommendations by updating the model in real time.

[1088] "Means for collecting user information" refers to devices and software for collecting data about users, including communication data, purchase data, search data, and emotional data, through various APIs and sensors.

[1089] "Means for integrating and pre-processing various types of data" refers to devices and software that convert the diverse data collected into a unified format and perform pre-processing such as deleting duplicates, correcting inconsistencies, and filling in missing data.

[1090] "Means for analyzing user preferences and behavioral patterns" refers to devices or software that use machine learning algorithms and statistical analysis methods based on preprocessed data to extract and classify user interests and behavioral patterns.

[1091] "Means for recommending products and services suitable for users" refers to devices or software that select products and services that match the user's preferences and behavioral patterns based on the analysis results and present them as a recommendation list.

[1092] "Means for acquiring and analyzing emotional data" refers to devices or software that use image processing technology or voice analysis technology to analyze a user's facial expressions and tone of voice and determine their emotional state.

[1093] "Means for customizing the user interface" refers to devices or software that dynamically change the layout and display content of the home screen based on the recommendation list, providing a UI that is easy for users to use.

[1094] "Means of providing feedback on system usage in real time and updating the model" refers to devices and software that monitor and collect user behavioral and emotional data in real time, and retrain the machine learning model based on this new data to improve recommendation accuracy.

[1095] This invention is a system that collects and analyzes user information and recommends personalized products and services with high accuracy. By adding emotional data acquisition and analysis functions, this system provides recommendations that also correspond to the user's emotional state, and by updating the model in real time, it achieves highly accurate recommendations.

[1096] User Data Collection

[1097] The server collects user information through various APIs. Communication data is obtained from the API of the communication service provider, purchase data from the API of the electronic payment service, and search data from the API of the search engine. In addition, a camera using image processing technology and a microphone using voice analysis technology are used to obtain emotional data from facial expressions and tone of voice. For example, the server can determine whether the user is smiling from facial expressions captured by the camera, and evaluate whether they are enjoying themselves from the tone of their voice recorded by the microphone.

[1098] Data integration and preprocessing

[1099] The server converts the collected data into a unified format and performs preprocessing such as deleting duplicates, correcting inconsistencies, and filling in missing data. Data obtained from each API is unified into formats such as JSON or CSV. Inconsistent data (for example, data with different date and time formats) is corrected, and missing data is inferred or filled in. This creates a consistent dataset.

[1100] Analysis of user preferences and behavioral patterns

[1101] The server then runs machine learning algorithms on the preprocessed data to analyze user preferences and behavioral patterns. Specifically, it uses clustering algorithms (e.g., K-means clustering) and classification algorithms (e.g., decision trees and random forests). For example, a user who frequently searches for movies might be classified into a cluster called a movie lover. It also combines emotional data to analyze behaviors under specific emotional states.

[1102] Personalized product and service recommendations

[1103] The server then uses the analysis results to recommend the most suitable products and services for each user. It selects the most suitable items by taking into consideration not only past behavioral data and preferences, but also emotional data. For example, if a user is interested in movies and looks happy, it will be recommended the latest movie tickets and movie-related merchandise.

[1104] Customizing the app's user interface

[1105] The device dynamically customizes the user interface based on the recommendation list received from the server, changing the home screen layout to prioritize relevant apps and links. For example, movie ticket purchasing apps and movie-related links are displayed on the home screen. The device also rearranges the layout to make new content that may be of interest easier to access.

[1106] Real-time feedback and model updates

[1107] The server monitors and collects user behavioral and emotional data in real time. This new data is used to retrain the machine learning model and improve the accuracy of the recommendation system. The model parameters are updated based on new preferences and behavioral patterns. For example, the system can immediately respond to new content or products that users have become interested in.

[1108] As described above, the present invention enables users to recommend products and services that are optimized for them, and provides highly accurate recommendations through real-time model updates. Specifically, the following prompt sentences can be used:

[1109] Example prompt sentence:

[1110] "A method to recommend optimal products and services to a user searching for the latest movie tickets, taking into account the user's emotional data."

[1111] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1112] Step 1:

[1113] The server collects communication history, purchase history, search history, and emotion data via various APIs and sensors. Communication data is obtained from the API of the communication service provider, purchase data from the API of the electronic payment service, and search data from the API of the search engine. Emotion data is obtained from a camera using image processing technology and a microphone using voice analysis technology. The input is data from each API and sensor, and the output is a dataset of collected user information. The server stores this data in a repository where it is temporarily saved.

[1114] Step 2:

[1115] The server integrates and preprocesses the collected data. Specifically, it converts each piece of collected data into JSON or CSV format. It then removes duplicate data, corrects inconsistent data, and fills in missing data. The input is a dataset of collected user information, and the output is a clean dataset that has been integrated and preprocessed. The server runs software processes to remove duplicates and convert formats, creating a highly consistent dataset.

[1116] Step 3:

[1117] The server runs a machine learning algorithm on the preprocessed data to analyze user preferences and behavioral patterns. Specifically, it uses a clustering algorithm (e.g., K-means) to group users, and then uses a classification algorithm (e.g., decision tree or random forest) to analyze individual behavioral patterns. The input is the preprocessed, clean dataset, and the output is a model of user preferences and behavioral patterns. The server stores the results of clustering and classification in a database.

[1118] Step 4:

[1119] The server recommends optimal products and services to users based on the analysis results. The server selects the most suitable items by taking into account past behavioral data, preference data, and emotional data. The input is a model of the user's preferences and behavioral patterns, and the output is a personalized recommendation list. The server generates the recommendation list and prepares data corresponding to each user.

[1120] Step 5:

[1121] The device dynamically customizes the user interface based on the recommendation list received from the server. Specifically, it changes the home screen layout and adds relevant apps and links. The input is the recommendation list from the server, and the output is the customized user interface. The device adjusts the screen layout and adds shortcuts.

[1122] Step 6:

[1123] The server monitors users' behavioral and emotional data in real time and collects new data. The input is real-time data from the user's device, and the output is the latest behavioral data set. The server retrains the machine learning model based on this new data to improve the accuracy of the recommendation system. Specifically, it monitors click data, purchase history, and fluctuations in emotional state, and updates the model accordingly.

[1124] (Application example 2)

[1125] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1126] Conventional recommendation systems tend to recommend products and services based on static data such as a user's purchase history and search history, and are insufficient in responding to the user's emotional state or real-time behavior. Therefore, there is a demand for more appropriate and effective advertisements and product / service recommendations for users. This invention aims to solve the problems of conventional systems by providing personalized, highly accurate recommendations that also take into account the user's emotional data, thereby providing users with an optimal experience.

[1127] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting user information, means for integrating various data and performing preprocessing, means for analyzing user preferences and behavioral patterns, means for recommending products and services suitable for the user, means for recognizing the user's emotions, means for customizing the user interface, means for providing feedback on system usage status in real time and updating the model, and means for recommending and displaying personalized advertisements based on emotional data. This makes it possible to recommend appropriate products and services and display advertisements in real time based on the user's emotional state.

[1128] "User Information" refers to various information about a user, including communication data, purchase data, search data, and emotional data.

[1129] "Means for integrating and pre-processing various data" refers to the part of the system that converts data of different formats into a consistent format, removes duplicate data, corrects inconsistent data, and completes missing data.

[1130] "Means for analyzing user preferences and behavioral patterns" refers to the part of the system that uses machine learning algorithms to analyze patterns based on users' interests and past behavioral history.

[1131] "Means for recommending products and services suitable for users" refers to the process of recommending the most relevant products and services based on the analysis of users' preferences and behavioral patterns and emotional data.

[1132] "Means for recognizing the user's emotions" refers to the part of the system that uses image processing and voice analysis technology to obtain and analyze emotional data from the user's facial expressions and tone of voice.

[1133] "Means for customizing the user interface" refers to the process of dynamically changing the interface layout and display content based on the analysis results and the list of recommended products and services.

[1134] "Means of providing real-time feedback on system usage and updating the model" refers to the process of collecting user operation status and new data in real time, and using this information to retrain the machine learning model and improve its accuracy.

[1135] "Means for recommending and displaying personalized advertisements based on emotional data" refers to the process of selecting and displaying the most appropriate advertisements individually based on the user's current emotional state.

[1136] This invention is a system that collects and analyzes a wide variety of user information, including emotional data, to recommend personalized and highly accurate products and services. In particular, we describe the recommendation and display of advertisements that utilize emotional data. This system mainly consists of a server that collects and analyzes user information, and a terminal that provides the user interface.

[1137] Data collection

[1138] First, the server collects user information such as communication data, purchase data, and search data via various APIs. Specific hardware used includes smartphones, smart glasses, and head-mounted displays. The server then uses the data acquired from these devices and an emotion engine to obtain emotional data from facial expressions and tone of voice. For example, the server uses the smartphone's camera and microphone to perform facial recognition and voice analysis to obtain the user's emotions in real time.

[1139] Data integration and preprocessing

[1140] The server consolidates and preprocesses the collected data. Specifically, cloud platforms (such as AWS and Google Cloud) are used for this purpose. This process involves unifying data in different formats, removing duplicates, correcting inconsistent data, and filling in missing data. This process produces a clean, consistent dataset, facilitating subsequent analysis.

[1141] Analysis of preferences and behavioral patterns

[1142] The server uses machine learning algorithms to analyze user preferences and behavioral patterns based on the preprocessed data. Specifically, it uses machine learning libraries such as TensorFlow and PyTorch. This allows for clustering and classification based on users' interests and past behavior. Emotional data is also included in the analysis, making it possible to analyze the emotional state of users that leads them to take specific actions.

[1143] Product and service recommendations

[1144] The server then uses the analysis results to recommend the most suitable products and services to the user. By combining this with emotional data, the system can identify and recommend items and advertisements that are likely to interest the user in real time. For example, if a user searches for something related to movies and shows an excited expression, an advertisement for movie tickets will be displayed.

[1145] User interface customization

[1146] The device dynamically changes the user interface based on the recommendation list received from the server, for example, adding a shortcut to a movie ticket purchasing app on the home screen or displaying related offers, allowing users to quickly access the information and services they need.

[1147] Real-time feedback and model updates

[1148] The server monitors user behavior in real time and collects new data, including information about ads clicked, products purchased, and even new emotional data, which it uses to retrain machine learning models and improve the accuracy of the recommendation system.

[1149] Examples of prompt statements

[1150] When a user searches for something related to movies, if a happy expression is recognized, the system will generate a prompt like the one below based on that emotional data and display an advertisement for movie tickets.

[1151] If a user searches for "movie updates" and then looks happy:

[1152] "Check out our latest offers on movie tickets and movie merchandise for great savings!"

[1153] In this way, by making recommendations that reflect real-time emotional data, a system can be created that provides users with the optimal experience.

[1154] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1155] Step 1:

[1156] The server collects communication data, purchase data, and search data via various APIs. Specifically, it obtains this data from devices such as smartphones and smart glasses, and collects emotional data from facial expressions and tone of voice through an emotion engine. The server uses this data to initialize the user's interests and concerns. The inputs are communication data, purchase data, search data, and emotional data, and the output is integrated user data.

[1157] Step 2:

[1158] The server converts the collected data into a unified format, removes duplicates, corrects inconsistent data, and fills in missing data. The specific software used is a cloud platform (AWS or Google Cloud). The input is consolidated user data, and the output is a clean, consistent dataset.

[1159] Step 3:

[1160] The server runs machine learning algorithms (such as TensorFlow or PyTorch) on the clean dataset to analyze user preferences and behavioral patterns. In this step, pattern analysis is performed using clustering and classification algorithms. Emotional data is also taken into account, and behavioral patterns in specific emotional states are also analyzed. The input is the clean dataset, and the output is the analyzed user preferences and behavioral patterns.

[1161] Step 4:

[1162] The server recommends optimal products, services, and advertisements for each user based on the analysis results. In particular, it utilizes emotional data to select items and advertisements that are likely to interest the user in real time. It also uses a generative AI model to generate prompts and use them to display advertisements. The input to this step is the analysis results, and the output is recommended products, services, and advertisements.

[1163] Step 5:

[1164] The device customizes the user interface based on the recommendation list obtained from the server. For example, it adds a shortcut to a movie ticket purchasing app on the home screen and displays related offers. This allows the user to quickly access the information and services they need. The input is the recommendation list, and the output is the customized user interface.

[1165] Step 6:

[1166] The server monitors user behavior in real time and collects new data, including information about ads clicked, products purchased, and even new emotional data. The server uses this data to retrain the machine learning model and improve the accuracy of the recommendation system. The input is real-time user behavior data, and the output is an updated machine learning model.

[1167] In this way, specific data processing and calculations are carried out at each step, resulting in a system that provides users with advertisements and services optimized for them in real time.

[1168] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1169] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1170] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1171] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1172] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1173] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1174] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1175] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1176] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1177] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1178] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1179] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1180] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1181] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1182] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1183] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1184] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1185] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1186] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1187] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1188] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1189] The following is further disclosed regarding the above embodiment.

[1190] (Claim 1)

[1191] The means by which user information is collected;

[1192] A means of integrating and preprocessing various data;

[1193] A means of analyzing user preferences and behavior patterns;

[1194] A means of recommending products and services suitable for users;

[1195] a means for customizing the user interface;

[1196] A means to provide real-time feedback on system usage and update the model;

[1197] A system including:

[1198] (Claim 2)

[1199] 10. The system of claim 1, further comprising means for collecting communication data, purchase data, and search data.

[1200] (Claim 3)

[1201] 10. The system of claim 1, further comprising means for analyzing user preferences and behavioral patterns using machine learning algorithms.

[1202] "Example 1"

[1203] (Claim 1)

[1204] The means by which user information is collected;

[1205] A means of integrating and preprocessing various data;

[1206] A means of analyzing user preferences and behavior patterns;

[1207] A means of recommending products and services suitable for users;

[1208] a means for customizing the user interface;

[1209] A means to provide real-time feedback on system usage and update the model;

[1210] means for encrypting data received from the terminal and transmitting the data to a server;

[1211] A means of converting data from different formats into a consistent format;

[1212] A means of applying machine learning algorithms to analyze user preferences and behavioral patterns;

[1213] A system including:

[1214] (Claim 2)

[1215] 10. The system of claim 1, further comprising means for collecting communication data, purchase data, and search data.

[1216] (Claim 3)

[1217] 10. The system of claim 1, further comprising means for analyzing user preferences and behavioral patterns using machine learning algorithms.

[1218] "Application Example 1"

[1219] (Claim 1)

[1220] The means by which user information is collected;

[1221] A means of integrating and preprocessing various data;

[1222] A means of analyzing user preferences and behavior patterns;

[1223] A means of recommending products and services suitable for users;

[1224] a means for customizing the user interface;

[1225] A means to provide real-time feedback on system usage and update the model;

[1226] A means of recommending optimal products and services based on the analysis results using real-time behavioral data of users,

[1227] A means for dynamically changing the interface based on the recommendation results;

[1228] A system including:

[1229] (Claim 2)

[1230] 10. The system of claim 1, further comprising means for collecting communication data, purchase data, and search data.

[1231] (Claim 3)

[1232] 10. The system of claim 1, further comprising means for analyzing user preferences and behavioral patterns using machine learning algorithms.

[1233] "Example 2: Combining Emotion Engines"

[1234] (Claim 1)

[1235] The means by which user information is collected;

[1236] A means of integrating and preprocessing various data;

[1237] A means of analyzing user preferences and behavior patterns;

[1238] A means of recommending products and services suitable for users;

[1239] A means for acquiring and analyzing emotion data;

[1240] a means for customizing the user interface;

[1241] A means to provide real-time feedback on system usage and update the model;

[1242] A system including:

[1243] (Claim 2)

[1244] 10. The system of claim 1, further comprising means for collecting communication data, purchase data, and search data.

[1245] (Claim 3)

[1246] 10. The system of claim 1, further comprising means for analyzing user preferences and behavioral patterns using machine learning algorithms.

[1247] "Application example 2 when combining emotion engines"

[1248] (Claim 1)

[1249] The means by which user information is collected;

[1250] A means of integrating and preprocessing various data;

[1251] A means of analyzing user preferences and behavior patterns;

[1252] A means of recommending products and services suitable for users;

[1253] a means for recognizing a user's emotion;

[1254] a means for customizing the user interface;

[1255] A means to provide real-time feedback on system usage and update the model;

[1256] A means for recommending and displaying personalized advertisements based on emotional data;

[1257] A system including:

[1258] (Claim 2)

[1259] 10. The system of claim 1, further comprising means for collecting communication data, purchase data, and search data.

[1260] (Claim 3)

[1261] 10. The system of claim 1, further comprising means for analyzing user preferences, behavioral patterns, and emotional data using machine learning algorithms. [Explanation of symbols]

[1262] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. The means by which user information is collected; A means of integrating and preprocessing various data; A means of analyzing user preferences and behavior patterns; A means of recommending products and services suitable for users; a means for customizing the user interface; A means to provide real-time feedback on system usage and update the model; A system including:

2. 10. The system of claim 1, further comprising means for collecting communication data, purchase data, and search data.

3. The system of claim 1 , further comprising means for analyzing user preferences and behavioral patterns using machine learning algorithms.

Images