system

Abstract

We provide the system. [Solution] Means of collecting user information, Means of collecting product information, Means for pre-processing the user information and product information, A means for training a model based on the aforementioned preprocessed data, A means of making recommendations to users using the aforementioned model, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The present invention relates to a shopping system that efficiently recommends fashion items optimized for individual preferences and body types. In conventional shopping methods, the user has to select each product one by one from a variety of options, which requires time and effort, and there is a problem that it is difficult to find a product that perfectly matches individual preferences. Therefore, there has been a demand to improve the user's purchase experience and provide more accurate product recommendations.

Means for Solving the Problems

[0005] To address this challenge, the present invention provides a system that collects user information, preprocesses the data by integrating it with product information, and then constructs a machine learning model based on that data. This model learns the user's preferences and purchase history, and as a result provides individually optimized fashion recommendations. Furthermore, by displaying the recommendation results on the user's terminal and utilizing user feedback to continuously optimize the model, the system provides a valuable shopping experience for the user.

[0006] "User information" refers to data about individual users, including purchase history, browsing history, preferences, body type information, and feedback.

[0007] "Product information" refers to data about a product, including product ID, category, size, color, price, and brand information.

[0008] "Preprocessing" refers to preparing collected data so that it can be effectively learned by the model. This process includes steps such as imputing missing values, sorting out duplicate data, encoding categorical data, and normalizing the data.

[0009] "Training a model" means using machine learning algorithms to extract trends and patterns based on preprocessed data, and then training the model to improve its predictive ability for new data.

[0010] "Making recommendations" means using the results of a trained model to select and present products that are best suited to each user's individual preferences and needs.

[0011] A "terminal" refers to a device operated by the user, such as a smartphone or computer that provides an interface for product search and recommendation displays.

[0012] "Feedback" refers to evaluations and opinions that users provide to the system, including comments on product satisfaction, suitability, and service. [Brief explanation of the drawing]

[0013] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0014] Next, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, the labeled processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0017] In the following embodiments, the labeled RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0018] In the following embodiments, the labeled storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0019] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0021] [First Embodiment]

[0022] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0023] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0026] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0029] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0033] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0034] The embodiments for carrying out the present invention will be described in detail below. This system consists of a server, a terminal, and a user.

[0035] The server is responsible for collecting and managing user information and product information. Collected user information includes past purchase history, browsing history, and data on preferred styles and sizes. Product information includes details such as product category, size, color, price, and brand. The server continuously updates this information to maintain the most up-to-date data.

[0036] The server preprocesses the collected data and trains a machine learning model. Data preprocessing includes procedures such as imputing missing values, removing duplicate data, and encoding categorical data. This step prepares the data in a format that is easy for the model to learn. During model training, the preprocessed data is used to learn user preferences and purchasing trends, improving its ability to generate new product recommendations.

[0037] The terminal is responsible for visually displaying recommended products to the user. When a user begins searching for or browsing products, a request is sent to the server. The recommended product list is generated by a trained model and customized based on the individual user's needs and preferences. The terminal displays this list along with images, descriptions, prices, and availability to provide information to the user.

[0038] Users can browse and purchase products by operating their devices. They can select items of interest from the visually displayed products. Furthermore, users can provide satisfaction ratings and product feedback after purchase. This feedback is collected on the server and used to further optimize the model.

[0039] As a concrete example, suppose a user is searching for sneakers on their device. Based on the user's past data collected, the server is likely to recommend blue sneakers using an appropriate model. From these recommended items displayed on the device, the user can select their preferred design and price range and make a purchase.

[0040] Thus, the present invention provides an efficient and personalized shopping experience by quickly recommending individually optimized products based on the user's preferences. As a result, it aims to improve user satisfaction and contribute to promoting the purchase of products handled.

[0041] The following describes the processing flow.

[0042] Step 1:

[0043] The server collects user information and product information from ZOZOTOWN's database. User information includes purchase history, browsing history, preferences, and body type data, while product information includes product ID, category, size, color, price, and brand information.

[0044] Step 2:

[0045] The server preprocesses the collected data. Specifically, it imputes or removes missing values, sorts out duplicate data, quantifies (encodes) categorical data, and normalizes the data to prepare it for effective learning by machine learning models.

[0046] Step 3:

[0047] The server trains a machine learning model based on pre-processed data. This training analyzes users' purchasing patterns and preferences to build recommendation logic that can be used in the future.

[0048] Step 4:

[0049] The device sends a request to the server when the user searches for or looks for a specific fashion item.

[0050] Step 5:

[0051] The server receives requests from the terminal and uses a trained model to generate a list of optimal product recommendations for the user. This list is personalized based on the user's preferences and purchase history.

[0052] Step 6:

[0053] The terminal displays a list of recommended products received from the server to the user. This list includes information such as product images, details, prices, and stock availability, and is displayed in a visually verifiable format.

[0054] Step 7:

[0055] Users can browse products displayed on their devices, add them to their favorites, or purchase them directly. After a purchase, users can provide feedback on the product, and this feedback is used for future model training.

[0056] (Example 1)

[0057] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0058] There is a need for a system that provides fast and accurate personalized product recommendations based on user preferences and purchase history. Conventional systems struggle to effectively process vast amounts of user and product information to generate recommendations, and their methods for appropriately utilizing user feedback are limited. A system that solves these problems is necessary.

[0059] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0060] In this invention, the server includes means for acquiring user information, means for acquiring product information, and means for pre-processing the user information and product information. This makes it possible to generate product recommendations based on the preferences of each user.

[0061] The definitions of important terms are listed below.

[0062] "User information" refers to data about individual users collected for the purpose of recommending products, and includes past purchase history and preferences.

[0063] "Product information" refers to information about products that are recommended to users, and includes data such as category, size, color, price, and brand.

[0064] "Preprocessing" is the process of preparing data to facilitate the training of machine learning models, and includes tasks such as imputing missing values, removing duplicate data, and encoding categorical data.

[0065] A "learning device" is a system that uses machine learning models to analyze user preferences and tendencies and is trained to generate product recommendations.

[0066] "Product recommendations" refer to information that presents personalized products based on the user's preferences and purchase history.

[0067] "Feedback" refers to the opinions and evaluations that users provide after using or purchasing a product, and is data used to further optimize the system.

[0068] This invention is a system that recommends products optimized based on user preferences, and consists of a server, a terminal, and a user.

[0069] The server is used as hardware to store and manage user information and item information. Common database management systems include SQL Server and MongoDB. The server collects data such as the user's past purchase history, browsing history, and preferences, and stores the information in an appropriate manner. Libraries such as Python's Pandas and Scikit-learn are used for data preprocessing.

[0070] The server trains a machine learning model using pre-processed data. Frameworks such as TENSORFLOW® and PyTorch are used for training, analyzing user preferences and purchasing trends to build a model for generating product recommendations.

[0071] The terminal provides users with an interface that visually presents products. It displays product lists via a web browser or native application, making it easy to access product information, including images, descriptions, and pricing.

[0072] Users can operate their devices to search for products, select items of interest, and purchase them. After purchase, they can provide feedback such as product ratings and opinions. This feedback information is collected by a server and used to further optimize machine learning models.

[0073] For example, if a user is looking for a spring coat, the server will recommend coats with spring-like colors and designs based on past purchase data and trends. An example of a prompt message would be, "What spring items would you recommend for someone in their 20s?" This system can provide users with an excellent shopping experience through fast and highly personalized product recommendations.

[0074] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0075] Step 1:

[0076] The server collects user information and item information as input data. This process includes inputting the user's past purchase and browsing history, as well as product-related information (category, size, color, price, etc.). This data is stored in a database such as SQL Server or MongoDB. Specifically, when a user logs into the site, their activity history is automatically recorded and updated on the server.

[0077] Step 2:

[0078] The server preprocesses the collected data. The input for preprocessing is the raw data collected in step 1. Libraries such as Pandas and Scikit-learn are used to process the data, including imputing missing values, removing duplicate data, and encoding categorical data. This results in output data in a format easily processed by machine learning models. Specifically, the server performs data cleaning to create a clean dataset ready for analysis.

[0079] Step 3:

[0080] The server trains a machine learning model based on preprocessed data. The model is trained using TensorFlow or PyTorch, based on the preprocessed data as input. The output of this step is a trained model that predicts user preferences and behavior and suggests the optimal product. Specifically, it iteratively processes a large amount of training data, allowing the model to learn purchase patterns and improve prediction accuracy.

[0081] Step 4:

[0082] The server generates product recommendations using a pre-trained model. Input is the user's current activity and past data, and output is a personalized product list. The server receives prompts such as "What are some recommended spring items for people in their 20s?" and lists products based on them. Specifically, the model analyzes available information and quickly selects the most suitable product candidates.

[0083] Step 5:

[0084] The terminal displays a list of recommended products sent from the server to the user. The input data is the product list received from the server, and the output is product information (images, descriptions, prices, etc.) presented visually to the user. The user can view the products on the terminal screen. Specifically, the application generates a product page and displays the product information through the user interface.

[0085] Step 6:

[0086] Users view, select, and purchase recommended products. After purchase, they can provide product ratings and reviews. This feedback is sent to the server as input data and used to further optimize the model. The expected output is an improvement in the accuracy of recommendations that continues to improve. Specifically, the system works by having users provide reviews, and their opinions are reflected in future recommendations.

[0087] (Application Example 1)

[0088] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0089] In modern e-commerce, personalized product recommendations based on users' diverse preferences and purchase history are in demand. However, conventional systems struggle with real-time recommendations, making it difficult for users to quickly discover necessary products. Furthermore, methods for efficiently collecting user feedback and incorporating it into system optimization are not yet well established.

[0090] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0091] In this invention, the server includes means for collecting user information, means for collecting product information, and means for generating personalized product recommendations in real time. This enables real-time product recommendations based on user preferences, allowing for rapid product discovery and system optimization.

[0092] "Collecting user information" means obtaining data about a user's past purchase history, browsing history, and preferred styles and sizes.

[0093] "Collecting product information" means obtaining details such as the product's category, size, color, price, and brand.

[0094] "Preprocessing" refers to preparing data, including imputing missing values, removing duplicate data, and encoding categorical data.

[0095] "Training a model" means using machine learning algorithms to analyze and predict data patterns based on pre-processed data.

[0096] "Making recommendations" means using a trained model to present highly relevant products to a specific user.

[0097] "Generating personalized product recommendations in real time" means processing that quickly suggests appropriate products based on the user's current behavior and preferences.

[0098] "Displaying recommendations" means visually showing the generated product list to the user's output device.

[0099] "Collecting feedback" means gathering data on user opinions and satisfaction levels.

[0100] "Using for optimization" means using the collected feedback to make improvements to enhance the system's performance and accuracy.

[0101] The system implementing this invention consists of a server, a terminal, and a user. The server is responsible for efficiently collecting and managing user information and product information. User information includes data on past purchase history, browsing history, and preferred styles and sizes. Product information includes details such as category, size, color, price, and brand, and this data is aggregated on the server.

[0102] The server uses Python to build machine learning models and employs software libraries such as Scikit-learn and TensorFlow to preprocess the data. Data preprocessing includes imputation of missing values, data deduplication, and encoding of categorical data. This preprocessed data is then processed by the machine learning model to learn user preferences. This model is hosted on Amazon Web Services (AWS®), and Amazon RDS is used for database management.

[0103] The device runs an application designed with React Native to provide users with personalized product recommendations in real time. When a user launches the app, recommended products are displayed based on data retrieved from the server. This allows users to quickly discover products that perfectly match their preferences.

[0104] Users can select items of interest from the presented products and provide their opinions and ratings. This feedback is collected again on the server and used to improve the accuracy of the model and optimize the system.

[0105] For example, if a user searches for "sneakers" on the app, blue sneakers could be recommended based on past data. This would allow users to quickly access the products they are looking for, resulting in a more fulfilling shopping experience.

[0106] An example of a prompt would be: "Generate a prompt that suggests recommended products to a user who searched for blue sneakers."

[0107] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0108] Step 1:

[0109] The server collects user information and product information. User information includes purchase history, browsing history, and preferred styles and sizes, while product information includes category, size, color, price, and brand. This data is stored on AWS in preparation for preprocessing in the next step. The input is raw user and product information, and the output is a processed dataset.

[0110] Step 2:

[0111] The server preprocesses the collected data. Specifically, it uses Python and Scikit-learn to impute missing values, remove data deduplication, encode categorical data, and convert it into a format suitable for machine learning. The input is the processed dataset from step 1, and the output is the preprocessed data.

[0112] Step 3:

[0113] The server trains a machine learning model based on pre-processed data. During this process, it uses TensorFlow to execute algorithms and analyze user preferences and purchasing trends. The model is optimized for real-time product recommendations and managed on AWS. Input is pre-processed data, and output is the trained recommendation model.

[0114] Step 4:

[0115] The device retrieves recommended products from a server via an AI model based on user actions. When a user launches the app or performs a search, a product list is generated in real time. The prompt is based on the instruction, "Generate a prompt that suggests recommended products to a user who searched for blue sneakers." The input is the user's search query and app launch information, and the output is a list of recommended products.

[0116] Step 5:

[0117] Users browse or purchase products of interest from a list displayed on their device. Users input their opinions and ratings about the products, which are sent to the server as feedback. Input consists of user selections and feedback information, while output is user satisfaction data.

[0118] Step 6:

[0119] The server retrains and optimizes the model based on the collected feedback. This information is used to improve the accuracy of product recommendations and reflect in future recommendations. The input is user feedback, and the output is the optimized recommendation model.

[0120] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0121] The following describes specific embodiments for carrying out the present invention. This system combines a server, a terminal, a user, and an emotion engine.

[0122] The server is responsible for collecting and managing user and product information. Collected user information includes past purchase history, browsing history, preferences, and body type information, while product information includes product ID, category, size, color, price, and brand details. The server continuously collects and updates this information, maintaining an up-to-date database, which provides a foundation for making appropriate recommendations to users.

[0123] The server trains a machine learning model based on preprocessed data. Data preprocessing includes imputing or removing missing values, converting categorical data to numerical values, and normalizing the data. This prepares the data for efficient model training. The trained model understands user preferences and purchasing trends, providing a foundation for new product recommendations.

[0124] The emotion engine can recognize the user's emotions and acquire emotional information. When a user browses products using their device, it uses cameras and sensors to infer emotions from facial expressions, tone of voice, mouse movements, etc., and sends that information to the server. Emotional information reflects emotional states such as joy, surprise, and dissatisfaction.

[0125] The terminal plays a role in visually displaying recommendations to the user. A list of recommendations from the server is sent to the terminal and displayed along with product images, details, prices, and inventory information. The recommendation results are adjusted to take into account the user's sentiment information, resulting in content optimized for the user.

[0126] Users can browse and purchase products via their devices. They can not only make purchases based on the suggested product recommendations, but also provide feedback on the products, which is then used to further optimize the model.

[0127] For example, if the emotion engine determines that a user is feeling stressed, the server can recommend products with relaxing colors or products that prioritize comfort. In this way, the aim is to provide a more personalized experience by offering individualized product suggestions based on the user's emotional state.

[0128] As described above, the present invention utilizes an emotion engine to provide product recommendations that take into account the user's emotions, thereby achieving a deeper level of personalization and contributing to improved user satisfaction and purchase intent.

[0129] The following describes the processing flow.

[0130] Step 1:

[0131] The server collects user and product information from the database. User information includes past purchase history, browsing history, preferences, and body type information, while product information includes product ID, category, size, color, price, and brand information. Based on this information, the server always maintains the most up-to-date data.

[0132] Step 2:

[0133] The server collects the user's emotions using an emotion engine. This emotion collection is performed by analyzing the user's facial expressions, voice tone, and operation patterns through the device's camera and microphone. The inferred emotional state is then sent to the server.

[0134] Step 3:

[0135] The server preprocesses the collected user information, product information, and sentiment information. It imputes missing values, encodes categorical data, normalizes the data, and converts it into a format suitable for model training.

[0136] Step 4:

[0137] The server uses pre-processed data to train a machine learning model. The model analyzes user preferences and purchasing behavior, and by incorporating sentiment data, it builds a more accurate recommendation logic.

[0138] Step 5:

[0139] When a user views or searches for products from their device, that request is sent to the server.

[0140] Step 6:

[0141] The server generates a list of product recommendations that take into account the user's preferences and emotions, based on the received request and the trained model. For users with a positive emotional state, it recommends products that match that state, and for users with a negative emotional state, it suggests items that will soothe their mood.

[0142] Step 7:

[0143] The device displays recommended items sent from the server to the user. These include product images, details, and pricing information, presented in a way that allows the user to easily compare and select items.

[0144] Step 8:

[0145] Users can browse the offered products and purchase selected items. Furthermore, they can provide feedback on the products and experience after purchase, contributing to improving the accuracy of future recommendations.

[0146] This processing flow allows users to receive product suggestions tailored to their emotional state, enabling them to enjoy a more personalized shopping experience.

[0147] (Example 2)

[0148] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0149] Traditional recommendation systems rely solely on users' past data, making it difficult to suggest products that take emotional changes into account. This prevents them from providing personalized recommendations based on users' real-time emotional states. As a result, improvements in user satisfaction are limited, and there are challenges in optimizing purchase intent.

[0150] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0151] In this invention, the server includes means for collecting user information, means for collecting product information, means for training a machine learning model based on preprocessed data, means for acquiring sentiment data, and means for recommending products to the user using the machine learning model and sentiment data. This enables highly accurate product recommendations that take into account the user's current emotional state.

[0152] "User information" refers to data related to individual users, such as their purchase history, browsing history, preferences, and body type information.

[0153] "Product information" refers to data related to a product, such as product ID, category, size, color, price, and brand details.

[0154] "Preprocessing" refers to the process of preparing data into a format suitable for analysis and model training, such as imputing or removing missing values, converting categorical data to numerical values, and normalizing data.

[0155] A "machine learning model" refers to a mathematical model that uses algorithms to analyze data and learn patterns to predict and classify future data.

[0156] "Emotional data" refers to information that digitally represents the emotional state inferred from a user's facial expressions, tone of voice, and behavior.

[0157] "Recommending products" means suggesting products that are likely to interest or appeal to individual users, based on their data.

[0158] "Model optimization" refers to the process of making adjustments and improvements to a machine learning model based on collected data and feedback in order to enhance its accuracy.

[0159] A description of embodiments for carrying out this invention will be given.

[0160] This system combines a server, terminals, users, and a data processing engine. The server collects and manages user information and product information. User information includes purchase history and preferences, while product information includes various details such as product ID and price. To integrate this information and store it in a database, the server efficiently retrieves external data using a REST API.

[0161] Next, the server preprocesses the data. This involves using Python to perform operations such as imputing missing values, converting data to numerical values, and normalizing the data. This prepares the data for training models using machine learning libraries like TensorFlow and PyTorch.

[0162] Furthermore, when a user uses a device to browse products, the device collects emotional data. The device's camera and microphone are used to analyze the user's facial expressions and tone of voice, and a data processing engine interprets this data to send real-time emotional status to the server.

[0163] The server combines trained machine learning models with collected sentiment data to generate product recommendations for the user. For example, if it determines that a user is stressed, it can recommend products that promote relaxation. This process provides a personalized experience and increases user satisfaction.

[0164] As a concrete example, a prompt might read, "Build an optimal model for recommending products that help users relax when they are experiencing stress." By inputting such prompts into the AI ​​model, a more accurate recommendation system can be achieved.

[0165] In this way, the present invention constructs a system that provides a more personalized purchasing experience by recommending products that take into account the emotional state of the user.

[0166] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0167] Step 1:

[0168] The server collects user information and product information. It uses data obtained from external databases and APIs as input, and outputs an updated database integrating this information. In this process, the server uses REST APIs to collect information and stores purchase history and product details in the database.

[0169] Step 2:

[0170] The server performs preprocessing of the collected data. It uses raw data as input and outputs data converted into a format suitable for machine learning. Specifically, it performs processes such as filling in missing values ​​with the mean, one-hot encoding to convert categorical data to numerical data, and data normalization. These data processing steps are performed using Python.

[0171] Step 3:

[0172] The server trains a machine learning model based on preprocessed data. It uses a preprocessed dataset as input and outputs a trained model. The server uses machine learning frameworks such as TensorFlow and PyTorch to build a model that predicts user purchasing trends.

[0173] Step 4:

[0174] The device collects emotional data when users browse products. It uses real-time data such as the user's facial expressions and voice as input, and outputs analyzed emotional information. The device uses its camera and microphone to analyze the user's facial expressions and voice tone, and transmits the emotional data to the server in real time.

[0175] Step 5:

[0176] The server generates product recommendations by combining machine learning models and sentiment data. It uses a trained model and sentiment data as input, and outputs individually optimized product lists. The server specifically creates recommendation lists that include relaxing products when it determines the user is experiencing stress.

[0177] Step 6:

[0178] The terminal visually displays recommendation information received from the server to the user. It uses a list of recommended products as input and outputs a product list that the user can view on the screen. The terminal displays images, detailed information, prices, etc., through a user interface to ensure easy understanding for the user.

[0179] (Application Example 2)

[0180] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0181] Online platforms face the challenge of providing product recommendations that take into account users' emotional needs. Traditional systems often rely solely on users' past purchase and browsing history, making it difficult to achieve personalization that reflects real-time emotional states. Therefore, there is a growing need for product recommendations that respond to users' real-time emotions.

[0182] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0183] In this invention, the server includes means for collecting user information, product information, and sentiment data; means for preprocessing this information; and means for training a machine learning model based on the preprocessed data. This enables product recommendations tailored to the user's emotional state.

[0184] "User information" refers to data that includes an individual's purchase history, browsing history, preferences, and other related information.

[0185] "Product information" refers to detailed data about a product, such as product ID, category, size, color, price, and brand.

[0186] "Emotional data" refers to information that indicates the emotional state of a user, inferred from their facial expressions, tone of voice, mouse movements, etc.

[0187] "Preprocessing" refers to the process of imputing or removing missing values, converting categorical data to numerical values, and normalizing data so that machine learning models can process raw data efficiently.

[0188] A "machine learning model" is a mathematical algorithm that is trained using data to make predictions and judgments about specific tasks.

[0189] "Product recommendation" is the process of presenting appropriate products based on a user's past behavior and current emotional state.

[0190] The system implementing this invention uses a combination of hardware and software to effectively utilize user information, product information, and sentiment data. The server uses a machine learning model built with Python and TensorFlow / Keras to perform personalized product recommendations to users based on the collected data. Three main components—the server, the terminal, and the user—interact to perform specific functions.

[0191] The server collects user and product information into a database and performs preprocessing. Preprocessing includes imputing missing values, converting categorical data to numerical values, and normalizing the data. A machine learning model learns from this preprocessed data, analyzing the user's past purchasing trends and emotional state to recommend products. This enables the provision of information tailored to the individual needs of each user.

[0192] The terminal consists of visual devices such as smartphones, and uses a camera and microphone to sense the user's facial expressions and voice. The emotion engine analyzes this data in real time and transmits the emotional state to the server.

[0193] Users visually review recommended products via their devices and consider those that interest them. For example, a user experiencing stress might be recommended products with relaxing effects, and the user can intuitively grasp this information. A generative AI model is used in this process to generate and analyze data according to prompt messages.

[0194] For example, using a prompt message such as, "Please input the user's facial expression and voice tone data, and output recommended product categories," the system recommends appropriate products that reflect the user's emotional state. In this way, it is possible to provide users with a personalized shopping experience based on emotional information obtained from cameras and microphones.

[0195] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0196] Step 1:

[0197] The server collects user and product information from the database. As input, it retrieves the user's past purchase and browsing history, product IDs, and category information, and stores them in the database. Based on this, the server formats the data in a way that makes it easy to use in subsequent processing.

[0198] Step 2:

[0199] The device uses a camera and microphone to capture the user's facial expressions and voice data in real time. The input consists of the user's facial image and voice, and this data is sent to an emotion engine for facial expression analysis and voice tone analysis. The output is information indicating the emotional state.

[0200] Step 3:

[0201] The server receives sentiment data sent from the sentiment engine and performs preprocessing. The input is user sentiment information, which is quantified and converted into a format that can be used by machine learning models. This includes data cleansing and normalization.

[0202] Step 4:

[0203] The server trains a machine learning model using pre-processed data. It takes clean user information, product information, and sentiment data as input and builds a predictive model through computation. The output is ready for product recommendations tailored to the sentiment of a specific user.

[0204] Step 5:

[0205] The server uses a pre-trained machine learning model to recommend appropriate products to the user. The input is a set of pre-processed information, and the output of the machine learning model generates a product list tailored to the user's emotional state. The recommendation results are then sent to the user's terminal.

[0206] Step 6:

[0207] The terminal visually displays product recommendations received from the server. The input is a list of recommended products, and the user interface displays product images and detailed descriptions. This allows the user to select a product.

[0208] Step 7:

[0209] Users review and select recommended products via their devices and provide purchase information as feedback as needed. This allows the server to receive feedback and further optimize its model.

[0210] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0211] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0212] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0213] [Second Embodiment]

[0214] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0215] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0216] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0217] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0218] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0219] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0220] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0221] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0222] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0223] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0224] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0225] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0226] The embodiments for carrying out the present invention will be described in detail below. This system consists of a server, a terminal, and a user.

[0227] The server is responsible for collecting and managing user information and product information. Collected user information includes past purchase history, browsing history, and data on preferred styles and sizes. Product information includes details such as product category, size, color, price, and brand. The server continuously updates this information to maintain the most up-to-date data.

[0228] The server preprocesses the collected data and trains a machine learning model. Data preprocessing includes procedures such as imputing missing values, removing duplicate data, and encoding categorical data. This step prepares the data in a format that is easy for the model to learn. During model training, the preprocessed data is used to learn user preferences and purchasing trends, improving its ability to generate new product recommendations.

[0229] The terminal is responsible for visually displaying recommended products to the user. When a user begins searching for or browsing products, a request is sent to the server. The recommended product list is generated by a trained model and customized based on the individual user's needs and preferences. The terminal displays this list along with images, descriptions, prices, and availability to provide information to the user.

[0230] Users can browse and purchase products by operating their devices. They can select items of interest from the visually displayed products. Furthermore, users can provide satisfaction ratings and product feedback after purchase. This feedback is collected on the server and used to further optimize the model.

[0231] As a concrete example, suppose a user is searching for sneakers on their device. Based on the user's past data collected, the server is likely to recommend blue sneakers using an appropriate model. From these recommended items displayed on the device, the user can select their preferred design and price range and make a purchase.

[0232] Thus, the present invention provides an efficient and personalized shopping experience by quickly recommending individually optimized products based on the user's preferences. As a result, it aims to improve user satisfaction and contribute to promoting the purchase of products handled.

[0233] The following describes the processing flow.

[0234] Step 1:

[0235] The server collects user information and product information from ZOZOTOWN's database. User information includes purchase history, browsing history, preferences, and body type data, while product information includes product ID, category, size, color, price, and brand information.

[0236] Step 2:

[0237] The server preprocesses the collected data. Specifically, it imputes or removes missing values, sorts out duplicate data, quantifies (encodes) categorical data, and normalizes the data to prepare it for effective learning by machine learning models.

[0238] Step 3:

[0239] The server trains a machine learning model based on pre-processed data. This training analyzes users' purchasing patterns and preferences to build recommendation logic that can be used in the future.

[0240] Step 4:

[0241] The device sends a request to the server when the user searches for or looks for a specific fashion item.

[0242] Step 5:

[0243] The server receives requests from the terminal and uses a trained model to generate a list of optimal product recommendations for the user. This list is personalized based on the user's preferences and purchase history.

[0244] Step 6:

[0245] The terminal displays a list of recommended products received from the server to the user. This list includes information such as product images, details, prices, and stock availability, and is displayed in a visually verifiable format.

[0246] Step 7:

[0247] Users can browse products displayed on their devices, add them to their favorites, or purchase them directly. After a purchase, users can provide feedback on the product, and this feedback is used for future model training.

[0248] (Example 1)

[0249] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0250] There is a need for a system that provides fast and accurate personalized product recommendations based on user preferences and purchase history. Conventional systems struggle to effectively process vast amounts of user and product information to generate recommendations, and their methods for appropriately utilizing user feedback are limited. A system that solves these problems is necessary.

[0251] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0252] In this invention, the server includes means for acquiring user information, means for acquiring product information, and means for pre-processing the user information and product information. This makes it possible to generate product recommendations based on the preferences of each user.

[0253] The definitions of important terms are listed below.

[0254] "User information" refers to data about individual users collected for the purpose of recommending products, and includes past purchase history and preferences.

[0255] "Product information" refers to information about products that are recommended to users, and includes data such as category, size, color, price, and brand.

[0256] "Preprocessing" is the process of preparing data to facilitate the training of machine learning models, and includes tasks such as imputing missing values, removing duplicate data, and encoding categorical data.

[0257] A "learning device" is a system that uses machine learning models to analyze user preferences and tendencies and is trained to generate product recommendations.

[0258] "Product recommendations" refer to information that presents personalized products based on the user's preferences and purchase history.

[0259] "Feedback" refers to the opinions and evaluations that users provide after using or purchasing a product, and is data used to further optimize the system.

[0260] This invention is a system that recommends products optimized based on user preferences, and consists of a server, a terminal, and a user.

[0261] The server is used as hardware to store and manage user information and item information. Common database management systems include SQL Server and MongoDB. The server collects data such as the user's past purchase history, browsing history, and preferences, and stores the information in an appropriate manner. Libraries such as Python's Pandas and Scikit-learn are used for data preprocessing.

[0262] The server trains machine learning models using pre-processed data. Frameworks such as TensorFlow and PyTorch are used for training, analyzing user preferences and purchasing trends to build models for generating product recommendations.

[0263] The terminal provides users with an interface that visually presents products. It displays product lists via a web browser or native application, making it easy to access product information, including images, descriptions, and pricing.

[0264] Users can operate their devices to search for products, select items of interest, and purchase them. After purchase, they can provide feedback such as product ratings and opinions. This feedback information is collected by a server and used to further optimize machine learning models.

[0265] For example, if a user is looking for a spring coat, the server will recommend coats with spring-like colors and designs based on past purchase data and trends. An example of a prompt message would be, "What spring items would you recommend for someone in their 20s?" This system can provide users with an excellent shopping experience through fast and highly personalized product recommendations.

[0266] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0267] Step 1:

[0268] The server collects user information and item information as input data. This process includes inputting the user's past purchase and browsing history, as well as product-related information (category, size, color, price, etc.). This data is stored in a database such as SQL Server or MongoDB. Specifically, when a user logs into the site, their activity history is automatically recorded and updated on the server.

[0269] Step 2:

[0270] The server preprocesses the collected data. The input for preprocessing is the raw data collected in step 1. Libraries such as Pandas and Scikit-learn are used to process the data, including imputing missing values, removing duplicate data, and encoding categorical data. This results in output data in a format easily processed by machine learning models. Specifically, the server performs data cleaning to create a clean dataset ready for analysis.

[0271] Step 3:

[0272] The server trains a machine learning model based on preprocessed data. The model is trained using TensorFlow or PyTorch, based on the preprocessed data as input. The output of this step is a trained model that predicts user preferences and behavior and suggests the optimal product. Specifically, it iteratively processes a large amount of training data, allowing the model to learn purchase patterns and improve prediction accuracy.

[0273] Step 4:

[0274] The server generates product recommendations using a pre-trained model. Input is the user's current activity and past data, and output is a personalized product list. The server receives prompts such as "What are some recommended spring items for people in their 20s?" and lists products based on them. Specifically, the model analyzes available information and quickly selects the most suitable product candidates.

[0275] Step 5:

[0276] The terminal displays a list of recommended products sent from the server to the user. The input data is the product list received from the server, and the output is product information (images, descriptions, prices, etc.) presented visually to the user. The user can view the products on the terminal screen. Specifically, the application generates a product page and displays the product information through the user interface.

[0277] Step 6:

[0278] The user views the recommended products, makes a selection, and purchases them. After the purchase, the user can provide evaluations and reviews of the products. These feedbacks are sent to the server as input data and used for further optimization of the model. As output, a continuous improvement in the recommendation accuracy is predicted. Specifically, there is a mechanism where the user provides a review and that opinion is reflected in the recommendations for subsequent times.

[0279] (Application Example 1)

[0280] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0281] In modern e-commerce, there is a demand for individualized product recommendations based on the diverse preferences and purchase histories of users. However, in conventional systems, it is difficult to provide real-time recommendations, and there is a problem that necessary products cannot be discovered quickly. Also, a method for efficiently collecting users' opinions and reflecting them in the optimization of the system has not been fully established.

[0282] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following respective means.

[0283] In this invention, the server includes means for collecting user information, means for collecting product information, and means for generating individualized product recommendations in real time. Thereby, real-time product recommendations based on users' preferences become possible, and rapid product discovery and system optimization can be realized.

[0284] "Collecting user information" means obtaining data on the user's past purchase history, browsing history, preferred styles and sizes.

[0285] "Collecting product information" means obtaining details such as the category, size, color, price, brand, etc. of the product.

[0286] "Preprocessing" means performing data grooming, including filling in missing data values, cleaning up duplicate data, and encoding categorical data.

[0287] "Training the model" means analyzing and predicting data patterns using machine learning algorithms based on the preprocessed data.

[0288] "Making recommendations" means presenting relevant products to a specific user using the trained model.

[0289] "Generating personalized product recommendations in real time" means performing a process of quickly suggesting appropriate products based on the user's current actions and preferences.

[0290] "Displaying recommendations" means visually presenting the generated product list to the user's output device.

[0291] "Collecting feedback" means gathering data on the user's opinions and satisfaction levels.

[0292] "Using for optimization" means making improvements to enhance the system's performance and accuracy based on the collected feedback.

[0293] The system for implementing this invention consists of a server, a terminal, and a user. The server is responsible for efficiently collecting and managing user information and product information. User information includes past purchase history, browsing history, and data related to preferred styles and sizes. Product information includes details such as category, size, color, price, brand, etc., and these data are aggregated on the server.

[0294] The server uses Python to build machine learning models and employs software libraries such as Scikit-learn and TensorFlow to preprocess the data. Data preprocessing includes imputation of missing values, data deduplication, and encoding of categorical data. This preprocessed data is then processed by the machine learning model to learn user preferences. This model is hosted on Amazon Web Services (AWS), and Amazon RDS is used for database management.

[0295] The device runs an application designed with React Native to provide users with personalized product recommendations in real time. When a user launches the app, recommended products are displayed based on data retrieved from the server. This allows users to quickly discover products that perfectly match their preferences.

[0296] Users can select items of interest from the presented products and provide their opinions and ratings. This feedback is collected again on the server and used to improve the accuracy of the model and optimize the system.

[0297] For example, if a user searches for "sneakers" on the app, blue sneakers could be recommended based on past data. This would allow users to quickly access the products they are looking for, resulting in a more fulfilling shopping experience.

[0298] An example of a prompt would be: "Generate a prompt that suggests recommended products to a user who searched for blue sneakers."

[0299] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0300] Step 1:

[0301] The server collects user information and product information. User information includes purchase history, browsing history, preferred styles and sizes, and product information includes categories, sizes, colors, prices, brands, etc. These data are saved in AWS to prepare for preprocessing in the next step. The input is raw user information and product information, and the output is a processed dataset.

[0302] Step 2:

[0303] The server preprocesses the collected data. Specifically, using Python and Scikit-learn, it performs imputation of missing values, removal of duplicate data, encoding of categorical data, and converts it into a form suitable for machine learning. The input is the processed dataset from Step 1, and the output is preprocessed data.

[0304] Step 3:

[0305] The server trains a machine learning model based on the preprocessed data. At this time, it uses TensorFlow to execute the algorithm and analyze the user's preferences and purchase tendencies. The model is optimized for real-time product recommendations and is managed on AWS. The input is the preprocessed data, and the output is a trained recommendation model.

[0306] Step 4:

[0307] The terminal obtains recommended products from the server via the generated AI model based on the user's actions. When the user launches the app or conducts a search, a real-time product list is generated. The prompt sentence is based on "Generate a prompt for recommending products to users who searched for blue sneakers." The input is the user's search query and app launch information, and the output is a list of recommended products.

[0308] Step 5:

[0309] Users browse or purchase products of interest from a list displayed on their device. Users input their opinions and ratings about the products, which are sent to the server as feedback. Input consists of user selections and feedback information, while output is user satisfaction data.

[0310] Step 6:

[0311] The server retrains and optimizes the model based on the collected feedback. This information is used to improve the accuracy of product recommendations and reflect in future recommendations. The input is user feedback, and the output is the optimized recommendation model.

[0312] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0313] The following describes specific embodiments for carrying out the present invention. This system combines a server, a terminal, a user, and an emotion engine.

[0314] The server is responsible for collecting and managing user and product information. Collected user information includes past purchase history, browsing history, preferences, and body type information, while product information includes product ID, category, size, color, price, and brand details. The server continuously collects and updates this information, maintaining an up-to-date database, which provides a foundation for making appropriate recommendations to users.

[0315] The server trains a machine learning model based on preprocessed data. Data preprocessing includes imputing or removing missing values, converting categorical data to numerical values, and normalizing the data. This prepares the data for efficient model training. The trained model understands user preferences and purchasing trends, providing a foundation for new product recommendations.

[0316] The emotion engine can recognize the user's emotions and acquire emotional information. When a user browses products using their device, it uses cameras and sensors to infer emotions from facial expressions, tone of voice, mouse movements, etc., and sends that information to the server. Emotional information reflects emotional states such as joy, surprise, and dissatisfaction.

[0317] The terminal plays a role in visually displaying recommendations to the user. A list of recommendations from the server is sent to the terminal and displayed along with product images, details, prices, and inventory information. The recommendation results are adjusted to take into account the user's sentiment information, resulting in content optimized for the user.

[0318] Users can browse and purchase products via their devices. They can not only make purchases based on the suggested product recommendations, but also provide feedback on the products, which is then used to further optimize the model.

[0319] For example, if the emotion engine determines that a user is feeling stressed, the server can recommend products with relaxing colors or products that prioritize comfort. In this way, the aim is to provide a more personalized experience by offering individualized product suggestions based on the user's emotional state.

[0320] As described above, the present invention utilizes an emotion engine to provide product recommendations that take into account the user's emotions, thereby achieving a deeper level of personalization and contributing to improved user satisfaction and purchase intent.

[0321] The following describes the processing flow.

[0322] Step 1:

[0323] The server collects user and product information from the database. User information includes past purchase history, browsing history, preferences, and body type information, while product information includes product ID, category, size, color, price, and brand information. Based on this information, the server always maintains the most up-to-date data.

[0324] Step 2:

[0325] The server collects the user's emotions using an emotion engine. This emotion collection is performed by analyzing the user's facial expressions, voice tone, and operation patterns through the device's camera and microphone. The inferred emotional state is then sent to the server.

[0326] Step 3:

[0327] The server preprocesses the collected user information, product information, and sentiment information. It imputes missing values, encodes categorical data, normalizes the data, and converts it into a format suitable for model training.

[0328] Step 4:

[0329] The server uses pre-processed data to train a machine learning model. The model analyzes user preferences and purchasing behavior, and by incorporating sentiment data, it builds a more accurate recommendation logic.

[0330] Step 5:

[0331] When a user views or searches for products from their device, that request is sent to the server.

[0332] Step 6:

[0333] The server generates a list of product recommendations that take into account the user's preferences and emotions, based on the received request and the trained model. For users with a positive emotional state, it recommends products that match that state, and for users with a negative emotional state, it suggests items that will soothe their mood.

[0334] Step 7:

[0335] The device displays recommended items sent from the server to the user. These include product images, details, and pricing information, presented in a way that allows the user to easily compare and select items.

[0336] Step 8:

[0337] Users can browse the offered products and purchase selected items. Furthermore, they can provide feedback on the products and experience after purchase, contributing to improving the accuracy of future recommendations.

[0338] This processing flow allows users to receive product suggestions tailored to their emotional state, enabling them to enjoy a more personalized shopping experience.

[0339] (Example 2)

[0340] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0341] Traditional recommendation systems rely solely on users' past data, making it difficult to suggest products that take emotional changes into account. This prevents them from providing personalized recommendations based on users' real-time emotional states. As a result, improvements in user satisfaction are limited, and there are challenges in optimizing purchase intent.

[0342] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0343] In this invention, the server includes means for collecting user information, means for collecting product information, means for training a machine learning model based on preprocessed data, means for acquiring sentiment data, and means for recommending products to the user using the machine learning model and sentiment data. This enables highly accurate product recommendations that take into account the user's current emotional state.

[0344] "User information" refers to data related to individual users, such as their purchase history, browsing history, preferences, and body type information.

[0345] "Product information" refers to data related to a product, such as product ID, category, size, color, price, and brand details.

[0346] "Preprocessing" refers to the process of preparing data into a format suitable for analysis and model training, such as imputing or removing missing values, converting categorical data to numerical values, and normalizing data.

[0347] A "machine learning model" refers to a mathematical model that uses algorithms to analyze data and learn patterns to predict and classify future data.

[0348] "Emotional data" refers to information that digitally represents the emotional state inferred from a user's facial expressions, tone of voice, and behavior.

[0349] "Recommending products" means suggesting products that are likely to interest or appeal to individual users, based on their data.

[0350] "Model optimization" refers to the process of making adjustments and improvements to a machine learning model based on collected data and feedback in order to enhance its accuracy.

[0351] A description of embodiments for carrying out this invention will be given.

[0352] This system combines a server, terminals, users, and a data processing engine. The server collects and manages user information and product information. User information includes purchase history and preferences, while product information includes various details such as product ID and price. To integrate this information and store it in a database, the server efficiently retrieves external data using a REST API.

[0353] Next, the server preprocesses the data. This involves using Python to perform operations such as imputing missing values, converting data to numerical values, and normalizing the data. This prepares the data for training models using machine learning libraries like TensorFlow and PyTorch.

[0354] Furthermore, when a user uses a device to browse products, the device collects emotional data. The device's camera and microphone are used to analyze the user's facial expressions and tone of voice, and a data processing engine interprets this data to send real-time emotional status to the server.

[0355] The server combines trained machine learning models with collected sentiment data to generate product recommendations for the user. For example, if it determines that a user is stressed, it can recommend products that promote relaxation. This process provides a personalized experience and increases user satisfaction.

[0356] As a concrete example, a prompt might read, "Build an optimal model for recommending products that help users relax when they are experiencing stress." By inputting such prompts into the AI ​​model, a more accurate recommendation system can be achieved.

[0357] In this way, the present invention constructs a system that provides a more personalized purchasing experience by recommending products that take into account the emotional state of the user.

[0358] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0359] Step 1:

[0360] The server collects user information and product information. It uses data obtained from external databases and APIs as input, and outputs an updated database integrating this information. In this process, the server uses REST APIs to collect information and stores purchase history and product details in the database.

[0361] Step 2:

[0362] The server performs preprocessing of the collected data. It uses raw data as input and outputs data converted into a format suitable for machine learning. Specifically, it performs processes such as filling in missing values ​​with the mean, one-hot encoding to convert categorical data to numerical data, and data normalization. These data processing steps are performed using Python.

[0363] Step 3:

[0364] The server trains a machine learning model based on preprocessed data. It uses a preprocessed dataset as input and outputs a trained model. The server uses machine learning frameworks such as TensorFlow and PyTorch to build a model that predicts user purchasing trends.

[0365] Step 4:

[0366] The device collects emotional data when users browse products. It uses real-time data such as the user's facial expressions and voice as input, and outputs analyzed emotional information. The device uses its camera and microphone to analyze the user's facial expressions and voice tone, and transmits the emotional data to the server in real time.

[0367] Step 5:

[0368] The server generates product recommendations by combining machine learning models and sentiment data. It uses a trained model and sentiment data as input, and outputs individually optimized product lists. The server specifically creates recommendation lists that include relaxing products when it determines the user is experiencing stress.

[0369] Step 6:

[0370] The terminal visually displays recommendation information received from the server to the user. It uses a list of recommended products as input and outputs a product list that the user can view on the screen. The terminal displays images, detailed information, prices, etc., through a user interface to ensure easy understanding for the user.

[0371] (Application Example 2)

[0372] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0373] Online platforms face the challenge of providing product recommendations that take into account users' emotional needs. Traditional systems often rely solely on users' past purchase and browsing history, making it difficult to achieve personalization that reflects real-time emotional states. Therefore, there is a growing need for product recommendations that respond to users' real-time emotions.

[0374] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0375] In this invention, the server includes means for collecting user information, product information, and sentiment data; means for preprocessing this information; and means for training a machine learning model based on the preprocessed data. This enables product recommendations tailored to the user's emotional state.

[0376] "User information" refers to data that includes an individual's purchase history, browsing history, preferences, and other related information.

[0377] "Product information" refers to detailed data about a product, such as product ID, category, size, color, price, and brand.

[0378] "Emotional data" refers to information that indicates the emotional state of a user, inferred from their facial expressions, tone of voice, mouse movements, etc.

[0379] "Preprocessing" refers to the process of imputing or removing missing values, converting categorical data to numerical values, and normalizing data so that machine learning models can process raw data efficiently.

[0380] A "machine learning model" is a mathematical algorithm that is trained using data to make predictions and judgments about specific tasks.

[0381] "Product recommendation" is the process of presenting appropriate products based on a user's past behavior and current emotional state.

[0382] The system implementing this invention uses a combination of hardware and software to effectively utilize user information, product information, and sentiment data. The server uses a machine learning model built with Python and TensorFlow / Keras to perform personalized product recommendations to users based on the collected data. Three main components—the server, the terminal, and the user—interact to perform specific functions.

[0383] The server collects user and product information into a database and performs preprocessing. Preprocessing includes imputing missing values, converting categorical data to numerical values, and normalizing the data. A machine learning model learns from this preprocessed data, analyzing the user's past purchasing trends and emotional state to recommend products. This enables the provision of information tailored to the individual needs of each user.

[0384] The terminal consists of visual devices such as smartphones, and uses a camera and microphone to sense the user's facial expressions and voice. The emotion engine analyzes this data in real time and transmits the emotional state to the server.

[0385] Users visually review recommended products via their devices and consider those that interest them. For example, a user experiencing stress might be recommended products with relaxing effects, and the user can intuitively grasp this information. A generative AI model is used in this process to generate and analyze data according to prompt messages.

[0386] For example, using a prompt message such as, "Please input the user's facial expression and voice tone data, and output recommended product categories," the system recommends appropriate products that reflect the user's emotional state. In this way, it is possible to provide users with a personalized shopping experience based on emotional information obtained from cameras and microphones.

[0387] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0388] Step 1:

[0389] The server collects user and product information from the database. As input, it retrieves the user's past purchase and browsing history, product IDs, and category information, and stores them in the database. Based on this, the server formats the data in a way that makes it easy to use in subsequent processing.

[0390] Step 2:

[0391] The device uses a camera and microphone to capture the user's facial expressions and voice data in real time. The input consists of the user's facial image and voice, and this data is sent to an emotion engine for facial expression analysis and voice tone analysis. The output is information indicating the emotional state.

[0392] Step 3:

[0393] The server receives sentiment data sent from the sentiment engine and performs preprocessing. The input is user sentiment information, which is quantified and converted into a format that can be used by machine learning models. This includes data cleansing and normalization.

[0394] Step 4:

[0395] The server trains a machine learning model using pre-processed data. It takes clean user information, product information, and sentiment data as input and builds a predictive model through computation. The output is ready for product recommendations tailored to the sentiment of a specific user.

[0396] Step 5:

[0397] The server uses a pre-trained machine learning model to recommend appropriate products to the user. The input is a set of pre-processed information, and the output of the machine learning model generates a product list tailored to the user's emotional state. The recommendation results are then sent to the user's terminal.

[0398] Step 6:

[0399] The terminal visually displays product recommendations received from the server. The input is a list of recommended products, and the user interface displays product images and detailed descriptions. This allows the user to select a product.

[0400] Step 7:

[0401] Users review and select recommended products via their devices and provide purchase information as feedback as needed. This allows the server to receive feedback and further optimize its model.

[0402] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0403] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0404] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0405] [Third Embodiment]

[0406] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0407] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0408] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0409] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0410] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0411] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0412] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0413] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0414] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0415] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0416] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0417] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0418] The embodiments for carrying out the present invention will be described in detail below. This system consists of a server, a terminal, and a user.

[0419] The server is responsible for collecting and managing user information and product information. Collected user information includes past purchase history, browsing history, and data on preferred styles and sizes. Product information includes details such as product category, size, color, price, and brand. The server continuously updates this information to maintain the most up-to-date data.

[0420] The server preprocesses the collected data and trains a machine learning model. Data preprocessing includes procedures such as imputing missing values, removing duplicate data, and encoding categorical data. This step prepares the data in a format that is easy for the model to learn. During model training, the preprocessed data is used to learn user preferences and purchasing trends, improving its ability to generate new product recommendations.

[0421] The terminal is responsible for visually displaying recommended products to the user. When a user begins searching for or browsing products, a request is sent to the server. The recommended product list is generated by a trained model and customized based on the individual user's needs and preferences. The terminal displays this list along with images, descriptions, prices, and availability to provide information to the user.

[0422] Users can browse and purchase products by operating their devices. They can select items of interest from the visually displayed products. Furthermore, users can provide satisfaction ratings and product feedback after purchase. This feedback is collected on the server and used to further optimize the model.

[0423] As a concrete example, suppose a user is searching for sneakers on their device. Based on the user's past data collected, the server is likely to recommend blue sneakers using an appropriate model. From these recommended items displayed on the device, the user can select their preferred design and price range and make a purchase.

[0424] Thus, the present invention provides an efficient and personalized shopping experience by quickly recommending individually optimized products based on the user's preferences. As a result, it aims to improve user satisfaction and contribute to promoting the purchase of products handled.

[0425] The following describes the processing flow.

[0426] Step 1:

[0427] The server collects user information and product information from ZOZOTOWN's database. User information includes purchase history, browsing history, preferences, and body type data, while product information includes product ID, category, size, color, price, and brand information.

[0428] Step 2:

[0429] The server preprocesses the collected data. Specifically, it imputes or removes missing values, sorts out duplicate data, quantifies (encodes) categorical data, and normalizes the data to prepare it for effective learning by machine learning models.

[0430] Step 3:

[0431] The server trains a machine learning model based on pre-processed data. This training analyzes users' purchasing patterns and preferences to build recommendation logic that can be used in the future.

[0432] Step 4:

[0433] The device sends a request to the server when the user searches for or looks for a specific fashion item.

[0434] Step 5:

[0435] The server receives requests from the terminal and uses a trained model to generate a list of optimal product recommendations for the user. This list is personalized based on the user's preferences and purchase history.

[0436] Step 6:

[0437] The terminal displays a list of recommended products received from the server to the user. This list includes information such as product images, details, prices, and stock availability, and is displayed in a visually verifiable format.

[0438] Step 7:

[0439] Users can browse products displayed on their devices, add them to their favorites, or purchase them directly. After a purchase, users can provide feedback on the product, and this feedback is used for future model training.

[0440] (Example 1)

[0441] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0442] There is a need for a system that provides fast and accurate personalized product recommendations based on user preferences and purchase history. Conventional systems struggle to effectively process vast amounts of user and product information to generate recommendations, and their methods for appropriately utilizing user feedback are limited. A system that solves these problems is necessary.

[0443] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0444] In this invention, the server includes means for acquiring user information, means for acquiring product information, and means for pre-processing the user information and product information. This makes it possible to generate product recommendations based on the preferences of each user.

[0445] The definitions of important terms are listed below.

[0446] "User information" refers to data about individual users collected for the purpose of recommending products, and includes past purchase history and preferences.

[0447] "Product information" refers to information about products that are recommended to users, and includes data such as category, size, color, price, and brand.

[0448] "Preprocessing" is the process of preparing data to facilitate the training of machine learning models, and includes tasks such as imputing missing values, removing duplicate data, and encoding categorical data.

[0449] A "learning device" is a system that uses machine learning models to analyze user preferences and tendencies and is trained to generate product recommendations.

[0450] "Product recommendations" refer to information that presents personalized products based on the user's preferences and purchase history.

[0451] "Feedback" refers to the opinions and evaluations that users provide after using or purchasing a product, and is data used to further optimize the system.

[0452] This invention is a system that recommends products optimized based on user preferences, and consists of a server, a terminal, and a user.

[0453] The server is used as hardware to store and manage user information and item information. Common database management systems include SQL Server and MongoDB. The server collects data such as the user's past purchase history, browsing history, and preferences, and stores the information in an appropriate manner. Libraries such as Python's Pandas and Scikit-learn are used for data preprocessing.

[0454] The server trains machine learning models using pre-processed data. Frameworks such as TensorFlow and PyTorch are used for training, analyzing user preferences and purchasing trends to build models for generating product recommendations.

[0455] The terminal provides users with an interface that visually presents products. It displays product lists via a web browser or native application, making it easy to access product information, including images, descriptions, and pricing.

[0456] Users can operate their devices to search for products, select items of interest, and purchase them. After purchase, they can provide feedback such as product ratings and opinions. This feedback information is collected by a server and used to further optimize machine learning models.

[0457] For example, if a user is looking for a spring coat, the server will recommend coats with spring-like colors and designs based on past purchase data and trends. An example of a prompt message would be, "What spring items would you recommend for someone in their 20s?" This system can provide users with an excellent shopping experience through fast and highly personalized product recommendations.

[0458] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0459] Step 1:

[0460] The server collects user information and item information as input data. This process includes inputting the user's past purchase and browsing history, as well as product-related information (category, size, color, price, etc.). This data is stored in a database such as SQL Server or MongoDB. Specifically, when a user logs into the site, their activity history is automatically recorded and updated on the server.

[0461] Step 2:

[0462] The server preprocesses the collected data. The input for preprocessing is the raw data collected in step 1. Libraries such as Pandas and Scikit-learn are used to process the data, including imputing missing values, removing duplicate data, and encoding categorical data. This results in output data in a format easily processed by machine learning models. Specifically, the server performs data cleaning to create a clean dataset ready for analysis.

[0463] Step 3:

[0464] The server trains a machine learning model based on preprocessed data. The model is trained using TensorFlow or PyTorch, based on the preprocessed data as input. The output of this step is a trained model that predicts user preferences and behavior and suggests the optimal product. Specifically, it iteratively processes a large amount of training data, allowing the model to learn purchase patterns and improve prediction accuracy.

[0465] Step 4:

[0466] The server generates product recommendations using a pre-trained model. Input is the user's current activity and past data, and output is a personalized product list. The server receives prompts such as "What are some recommended spring items for people in their 20s?" and lists products based on them. Specifically, the model analyzes available information and quickly selects the most suitable product candidates.

[0467] Step 5:

[0468] The terminal displays a list of recommended products sent from the server to the user. The input data is the product list received from the server, and the output is product information (images, descriptions, prices, etc.) presented visually to the user. The user can view the products on the terminal screen. Specifically, the application generates a product page and displays the product information through the user interface.

[0469] Step 6:

[0470] Users view, select, and purchase recommended products. After purchase, they can provide product ratings and reviews. This feedback is sent to the server as input data and used to further optimize the model. The expected output is an improvement in the accuracy of recommendations that continues to improve. Specifically, the system works by having users provide reviews, and their opinions are reflected in future recommendations.

[0471] (Application Example 1)

[0472] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0473] In modern e-commerce, personalized product recommendations based on users' diverse preferences and purchase history are in demand. However, conventional systems struggle with real-time recommendations, making it difficult for users to quickly discover necessary products. Furthermore, methods for efficiently collecting user feedback and incorporating it into system optimization are not yet well established.

[0474] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0475] In this invention, the server includes means for collecting user information, means for collecting product information, and means for generating personalized product recommendations in real time. This enables real-time product recommendations based on user preferences, allowing for rapid product discovery and system optimization.

[0476] "Collecting user information" means obtaining data about a user's past purchase history, browsing history, and preferred styles and sizes.

[0477] "Collecting product information" means obtaining details such as the product's category, size, color, price, and brand.

[0478] "Preprocessing" refers to preparing data, including imputing missing values, removing duplicate data, and encoding categorical data.

[0479] "Training a model" means using machine learning algorithms to analyze and predict data patterns based on pre-processed data.

[0480] "Making recommendations" means using a trained model to present highly relevant products to a specific user.

[0481] "Generating personalized product recommendations in real time" means processing that quickly suggests appropriate products based on the user's current behavior and preferences.

[0482] "Displaying recommendations" means visually showing the generated product list to the user's output device.

[0483] "Collecting feedback" means gathering data on user opinions and satisfaction levels.

[0484] "Using for optimization" means using the collected feedback to make improvements to enhance the system's performance and accuracy.

[0485] The system implementing this invention consists of a server, a terminal, and a user. The server is responsible for efficiently collecting and managing user information and product information. User information includes data on past purchase history, browsing history, and preferred styles and sizes. Product information includes details such as category, size, color, price, and brand, and this data is aggregated on the server.

[0486] The server uses Python to build machine learning models and employs software libraries such as Scikit-learn and TensorFlow to preprocess the data. Data preprocessing includes imputation of missing values, data deduplication, and encoding of categorical data. This preprocessed data is then processed by the machine learning model to learn user preferences. This model is hosted on Amazon Web Services (AWS), and Amazon RDS is used for database management.

[0487] The device runs an application designed with React Native to provide users with personalized product recommendations in real time. When a user launches the app, recommended products are displayed based on data retrieved from the server. This allows users to quickly discover products that perfectly match their preferences.

[0488] Users can select items of interest from the presented products and provide their opinions and ratings. This feedback is collected again on the server and used to improve the accuracy of the model and optimize the system.

[0489] For example, if a user searches for "sneakers" on the app, blue sneakers could be recommended based on past data. This would allow users to quickly access the products they are looking for, resulting in a more fulfilling shopping experience.

[0490] An example of a prompt would be: "Generate a prompt that suggests recommended products to a user who searched for blue sneakers."

[0491] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0492] Step 1:

[0493] The server collects user information and product information. User information includes purchase history, browsing history, and preferred styles and sizes, while product information includes category, size, color, price, and brand. This data is stored on AWS in preparation for preprocessing in the next step. The input is raw user and product information, and the output is a processed dataset.

[0494] Step 2:

[0495] The server preprocesses the collected data. Specifically, it uses Python and Scikit-learn to impute missing values, remove data deduplication, encode categorical data, and convert it into a format suitable for machine learning. The input is the processed dataset from step 1, and the output is the preprocessed data.

[0496] Step 3:

[0497] The server trains a machine learning model based on pre-processed data. During this process, it uses TensorFlow to execute algorithms and analyze user preferences and purchasing trends. The model is optimized for real-time product recommendations and managed on AWS. Input is pre-processed data, and output is the trained recommendation model.

[0498] Step 4:

[0499] The device retrieves recommended products from a server via an AI model based on user actions. When a user launches the app or performs a search, a product list is generated in real time. The prompt is based on the instruction, "Generate a prompt that suggests recommended products to a user who searched for blue sneakers." The input is the user's search query and app launch information, and the output is a list of recommended products.

[0500] Step 5:

[0501] Users browse or purchase products of interest from a list displayed on their device. Users input their opinions and ratings about the products, which are sent to the server as feedback. Input consists of user selections and feedback information, while output is user satisfaction data.

[0502] Step 6:

[0503] The server retrains and optimizes the model based on the collected feedback. This information is used to improve the accuracy of product recommendations and reflect in future recommendations. The input is user feedback, and the output is the optimized recommendation model.

[0504] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0505] The following describes specific embodiments for carrying out the present invention. This system combines a server, a terminal, a user, and an emotion engine.

[0506] The server is responsible for collecting and managing user and product information. Collected user information includes past purchase history, browsing history, preferences, and body type information, while product information includes product ID, category, size, color, price, and brand details. The server continuously collects and updates this information, maintaining an up-to-date database, which provides a foundation for making appropriate recommendations to users.

[0507] The server trains a machine learning model based on preprocessed data. Data preprocessing includes imputing or removing missing values, converting categorical data to numerical values, and normalizing the data. This prepares the data for efficient model training. The trained model understands user preferences and purchasing trends, providing a foundation for new product recommendations.

[0508] The emotion engine can recognize the user's emotions and acquire emotional information. When a user browses products using their device, it uses cameras and sensors to infer emotions from facial expressions, tone of voice, mouse movements, etc., and sends that information to the server. Emotional information reflects emotional states such as joy, surprise, and dissatisfaction.

[0509] The terminal plays a role in visually displaying recommendations to the user. A list of recommendations from the server is sent to the terminal and displayed along with product images, details, prices, and inventory information. The recommendation results are adjusted to take into account the user's sentiment information, resulting in content optimized for the user.

[0510] Users can browse and purchase products via their devices. They can not only make purchases based on the suggested product recommendations, but also provide feedback on the products, which is then used to further optimize the model.

[0511] For example, if the emotion engine determines that a user is feeling stressed, the server can recommend products with relaxing colors or products that prioritize comfort. In this way, the aim is to provide a more personalized experience by offering individualized product suggestions based on the user's emotional state.

[0512] As described above, the present invention utilizes an emotion engine to provide product recommendations that take into account the user's emotions, thereby achieving a deeper level of personalization and contributing to improved user satisfaction and purchase intent.

[0513] The following describes the processing flow.

[0514] Step 1:

[0515] The server collects user and product information from the database. User information includes past purchase history, browsing history, preferences, and body type information, while product information includes product ID, category, size, color, price, and brand information. Based on this information, the server always maintains the most up-to-date data.

[0516] Step 2:

[0517] The server collects the user's emotions using an emotion engine. This emotion collection is performed by analyzing the user's facial expressions, voice tone, and operation patterns through the device's camera and microphone. The inferred emotional state is then sent to the server.

[0518] Step 3:

[0519] The server preprocesses the collected user information, product information, and sentiment information. It imputes missing values, encodes categorical data, normalizes the data, and converts it into a format suitable for model training.

[0520] Step 4:

[0521] The server uses pre-processed data to train a machine learning model. The model analyzes user preferences and purchasing behavior, and by incorporating sentiment data, it builds a more accurate recommendation logic.

[0522] Step 5:

[0523] When a user views or searches for products from their device, that request is sent to the server.

[0524] Step 6:

[0525] The server generates a list of product recommendations that take into account the user's preferences and emotions, based on the received request and the trained model. For users with a positive emotional state, it recommends products that match that state, and for users with a negative emotional state, it suggests items that will soothe their mood.

[0526] Step 7:

[0527] The device displays recommended items sent from the server to the user. These include product images, details, and pricing information, presented in a way that allows the user to easily compare and select items.

[0528] Step 8:

[0529] Users can browse the offered products and purchase selected items. Furthermore, they can provide feedback on the products and experience after purchase, contributing to improving the accuracy of future recommendations.

[0530] This processing flow allows users to receive product suggestions tailored to their emotional state, enabling them to enjoy a more personalized shopping experience.

[0531] (Example 2)

[0532] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0533] Traditional recommendation systems rely solely on users' past data, making it difficult to suggest products that take emotional changes into account. This prevents them from providing personalized recommendations based on users' real-time emotional states. As a result, improvements in user satisfaction are limited, and there are challenges in optimizing purchase intent.

[0534] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0535] In this invention, the server includes means for collecting user information, means for collecting product information, means for training a machine learning model based on preprocessed data, means for acquiring sentiment data, and means for recommending products to the user using the machine learning model and sentiment data. This enables highly accurate product recommendations that take into account the user's current emotional state.

[0536] "User information" refers to data related to individual users, such as their purchase history, browsing history, preferences, and body type information.

[0537] "Product information" refers to data related to a product, such as product ID, category, size, color, price, and brand details.

[0538] "Preprocessing" refers to the process of preparing data into a format suitable for analysis and model training, such as imputing or removing missing values, converting categorical data to numerical values, and normalizing data.

[0539] A "machine learning model" refers to a mathematical model that uses algorithms to analyze data and learn patterns to predict and classify future data.

[0540] "Emotional data" refers to information that digitally represents the emotional state inferred from a user's facial expressions, tone of voice, and behavior.

[0541] "Recommending products" means suggesting products that are likely to interest or appeal to individual users, based on their data.

[0542] "Model optimization" refers to the process of making adjustments and improvements to a machine learning model based on collected data and feedback in order to enhance its accuracy.

[0543] A description of embodiments for carrying out this invention will be given.

[0544] This system combines a server, terminals, users, and a data processing engine. The server collects and manages user information and product information. User information includes purchase history and preferences, while product information includes various details such as product ID and price. To integrate this information and store it in a database, the server efficiently retrieves external data using a REST API.

[0545] Next, the server preprocesses the data. This involves using Python to perform operations such as imputing missing values, converting data to numerical values, and normalizing the data. This prepares the data for training models using machine learning libraries like TensorFlow and PyTorch.

[0546] Furthermore, when a user uses a device to browse products, the device collects emotional data. The device's camera and microphone are used to analyze the user's facial expressions and tone of voice, and a data processing engine interprets this data to send real-time emotional status to the server.

[0547] The server combines trained machine learning models with collected sentiment data to generate product recommendations for the user. For example, if it determines that a user is stressed, it can recommend products that promote relaxation. This process provides a personalized experience and increases user satisfaction.

[0548] As a concrete example, a prompt might read, "Build an optimal model for recommending products that help users relax when they are experiencing stress." By inputting such prompts into the AI ​​model, a more accurate recommendation system can be achieved.

[0549] In this way, the present invention constructs a system that provides a more personalized purchasing experience by recommending products that take into account the emotional state of the user.

[0550] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0551] Step 1:

[0552] The server collects user information and product information. It uses data obtained from external databases and APIs as input, and outputs an updated database integrating this information. In this process, the server uses REST APIs to collect information and stores purchase history and product details in the database.

[0553] Step 2:

[0554] The server performs preprocessing of the collected data. It uses raw data as input and outputs data converted into a format suitable for machine learning. Specifically, it performs processes such as filling in missing values ​​with the mean, one-hot encoding to convert categorical data to numerical data, and data normalization. These data processing steps are performed using Python.

[0555] Step 3:

[0556] The server trains a machine learning model based on preprocessed data. It uses a preprocessed dataset as input and outputs a trained model. The server uses machine learning frameworks such as TensorFlow and PyTorch to build a model that predicts user purchasing trends.

[0557] Step 4:

[0558] The device collects emotional data when users browse products. It uses real-time data such as the user's facial expressions and voice as input, and outputs analyzed emotional information. The device uses its camera and microphone to analyze the user's facial expressions and voice tone, and transmits the emotional data to the server in real time.

[0559] Step 5:

[0560] The server generates product recommendations by combining machine learning models and sentiment data. It uses a trained model and sentiment data as input, and outputs individually optimized product lists. The server specifically creates recommendation lists that include relaxing products when it determines the user is experiencing stress.

[0561] Step 6:

[0562] The terminal visually displays recommendation information received from the server to the user. It uses a list of recommended products as input and outputs a product list that the user can view on the screen. The terminal displays images, detailed information, prices, etc., through a user interface to ensure easy understanding for the user.

[0563] (Application Example 2)

[0564] Next, we will explain Application Example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0565] Online platforms face the challenge of providing product recommendations that take into account users' emotional needs. Traditional systems often rely solely on users' past purchase and browsing history, making it difficult to achieve personalization that reflects real-time emotional states. Therefore, there is a growing need for product recommendations that respond to users' real-time emotions.

[0566] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0567] In this invention, the server includes means for collecting user information, product information, and sentiment data; means for preprocessing this information; and means for training a machine learning model based on the preprocessed data. This enables product recommendations tailored to the user's emotional state.

[0568] "User information" refers to data that includes an individual's purchase history, browsing history, preferences, and other related information.

[0569] "Product information" refers to detailed data about a product, such as product ID, category, size, color, price, and brand.

[0570] "Emotional data" refers to information that indicates the emotional state of a user, inferred from their facial expressions, tone of voice, mouse movements, etc.

[0571] "Preprocessing" refers to the process of imputing or removing missing values, converting categorical data to numerical values, and normalizing data so that machine learning models can process raw data efficiently.

[0572] A "machine learning model" is a mathematical algorithm that is trained using data to make predictions and judgments about specific tasks.

[0573] "Product recommendation" is the process of presenting appropriate products based on a user's past behavior and current emotional state.

[0574] The system implementing this invention uses a combination of hardware and software to effectively utilize user information, product information, and sentiment data. The server uses a machine learning model built with Python and TensorFlow / Keras to perform personalized product recommendations to users based on the collected data. Three main components—the server, the terminal, and the user—interact to perform specific functions.

[0575] The server collects user and product information into a database and performs preprocessing. Preprocessing includes imputing missing values, converting categorical data to numerical values, and normalizing the data. A machine learning model learns from this preprocessed data, analyzing the user's past purchasing trends and emotional state to recommend products. This enables the provision of information tailored to the individual needs of each user.

[0576] The terminal consists of visual devices such as smartphones, and uses a camera and microphone to sense the user's facial expressions and voice. The emotion engine analyzes this data in real time and transmits the emotional state to the server.

[0577] Users visually review recommended products via their devices and consider those that interest them. For example, a user experiencing stress might be recommended products with relaxing effects, and the user can intuitively grasp this information. A generative AI model is used in this process to generate and analyze data according to prompt messages.

[0578] For example, using a prompt message such as, "Please input the user's facial expression and voice tone data, and output recommended product categories," the system recommends appropriate products that reflect the user's emotional state. In this way, it is possible to provide users with a personalized shopping experience based on emotional information obtained from cameras and microphones.

[0579] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0580] Step 1:

[0581] The server collects user and product information from the database. As input, it retrieves the user's past purchase and browsing history, product IDs, and category information, and stores them in the database. Based on this, the server formats the data in a way that makes it easy to use in subsequent processing.

[0582] Step 2:

[0583] The device uses a camera and microphone to capture the user's facial expressions and voice data in real time. The input consists of the user's facial image and voice, and this data is sent to an emotion engine for facial expression analysis and voice tone analysis. The output is information indicating the emotional state.

[0584] Step 3:

[0585] The server receives sentiment data sent from the sentiment engine and performs preprocessing. The input is user sentiment information, which is quantified and converted into a format that can be used by machine learning models. This includes data cleansing and normalization.

[0586] Step 4:

[0587] The server trains a machine learning model using pre-processed data. It takes clean user information, product information, and sentiment data as input and builds a predictive model through computation. The output is ready for product recommendations tailored to the sentiment of a specific user.

[0588] Step 5:

[0589] The server uses a pre-trained machine learning model to recommend appropriate products to the user. The input is a set of pre-processed information, and the output of the machine learning model generates a product list tailored to the user's emotional state. The recommendation results are then sent to the user's terminal.

[0590] Step 6:

[0591] The terminal visually displays product recommendations received from the server. The input is a list of recommended products, and the user interface displays product images and detailed descriptions. This allows the user to select a product.

[0592] Step 7:

[0593] Users review and select recommended products via their devices and provide purchase information as feedback as needed. This allows the server to receive feedback and further optimize its model.

[0594] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0595] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0596] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0597] [Fourth Embodiment]

[0598] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0599] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0600] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0601] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0602] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0603] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0604] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0605] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0606] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0607] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0608] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0609] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0610] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0611] The embodiments for carrying out the present invention will be described in detail below. This system consists of a server, a terminal, and a user.

[0612] The server is responsible for collecting and managing user information and product information. Collected user information includes past purchase history, browsing history, and data on preferred styles and sizes. Product information includes details such as product category, size, color, price, and brand. The server continuously updates this information to maintain the most up-to-date data.

[0613] The server preprocesses the collected data and trains a machine learning model. Data preprocessing includes procedures such as imputing missing values, removing duplicate data, and encoding categorical data. This step prepares the data in a format that is easy for the model to learn. During model training, the preprocessed data is used to learn user preferences and purchasing trends, improving its ability to generate new product recommendations.

[0614] The terminal is responsible for visually displaying recommended products to the user. When a user begins searching for or browsing products, a request is sent to the server. The recommended product list is generated by a trained model and customized based on the individual user's needs and preferences. The terminal displays this list along with images, descriptions, prices, and availability to provide information to the user.

[0615] Users can browse and purchase products by operating their devices. They can select items of interest from the visually displayed products. Furthermore, users can provide satisfaction ratings and product feedback after purchase. This feedback is collected on the server and used to further optimize the model.

[0616] As a concrete example, suppose a user is searching for sneakers on their device. Based on the user's past data collected, the server is likely to recommend blue sneakers using an appropriate model. From these recommended items displayed on the device, the user can select their preferred design and price range and make a purchase.

[0617] Thus, the present invention provides an efficient and personalized shopping experience by quickly recommending individually optimized products based on the user's preferences. As a result, it aims to improve user satisfaction and contribute to promoting the purchase of products handled.

[0618] The following describes the processing flow.

[0619] Step 1:

[0620] The server collects user information and product information from ZOZOTOWN's database. User information includes purchase history, browsing history, preferences, and body type data, while product information includes product ID, category, size, color, price, and brand information.

[0621] Step 2:

[0622] The server preprocesses the collected data. Specifically, it imputes or removes missing values, sorts out duplicate data, quantifies (encodes) categorical data, and normalizes the data to prepare it for effective learning by machine learning models.

[0623] Step 3:

[0624] The server trains a machine learning model based on pre-processed data. This training analyzes users' purchasing patterns and preferences to build recommendation logic that can be used in the future.

[0625] Step 4:

[0626] The device sends a request to the server when the user searches for or looks for a specific fashion item.

[0627] Step 5:

[0628] The server receives requests from the terminal and uses a trained model to generate a list of optimal product recommendations for the user. This list is personalized based on the user's preferences and purchase history.

[0629] Step 6:

[0630] The terminal displays a list of recommended products received from the server to the user. This list includes information such as product images, details, prices, and stock availability, and is displayed in a visually verifiable format.

[0631] Step 7:

[0632] Users can browse products displayed on their devices, add them to their favorites, or purchase them directly. After a purchase, users can provide feedback on the product, and this feedback is used for future model training.

[0633] (Example 1)

[0634] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0635] There is a need for a system that provides fast and accurate personalized product recommendations based on user preferences and purchase history. Conventional systems struggle to effectively process vast amounts of user and product information to generate recommendations, and their methods for appropriately utilizing user feedback are limited. A system that solves these problems is necessary.

[0636] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0637] In this invention, the server includes means for acquiring user information, means for acquiring product information, and means for pre-processing the user information and product information. This makes it possible to generate product recommendations based on the preferences of each user.

[0638] The definitions of important terms are listed below.

[0639] "User information" refers to data about individual users collected for the purpose of recommending products, and includes past purchase history and preferences.

[0640] "Product information" refers to information about products that are recommended to users, and includes data such as category, size, color, price, and brand.

[0641] "Preprocessing" is the process of preparing data to facilitate the training of machine learning models, and includes tasks such as imputing missing values, removing duplicate data, and encoding categorical data.

[0642] A "learning device" is a system that uses machine learning models to analyze user preferences and tendencies and is trained to generate product recommendations.

[0643] "Product recommendations" refer to information that presents personalized products based on the user's preferences and purchase history.

[0644] "Feedback" refers to the opinions and evaluations that users provide after using or purchasing a product, and is data used to further optimize the system.

[0645] This invention is a system that recommends products optimized based on user preferences, and consists of a server, a terminal, and a user.

[0646] The server is used as hardware to store and manage user information and item information. Common database management systems include SQL Server and MongoDB. The server collects data such as the user's past purchase history, browsing history, and preferences, and stores the information in an appropriate manner. Libraries such as Python's Pandas and Scikit-learn are used for data preprocessing.

[0647] The server trains machine learning models using pre-processed data. Frameworks such as TensorFlow and PyTorch are used for training, analyzing user preferences and purchasing trends to build models for generating product recommendations.

[0648] The terminal provides users with an interface that visually presents products. It displays product lists via a web browser or native application, making it easy to access product information, including images, descriptions, and pricing.

[0649] Users can operate their devices to search for products, select items of interest, and purchase them. After purchase, they can provide feedback such as product ratings and opinions. This feedback information is collected by a server and used to further optimize machine learning models.

[0650] For example, if a user is looking for a spring coat, the server will recommend coats with spring-like colors and designs based on past purchase data and trends. An example of a prompt message would be, "What spring items would you recommend for someone in their 20s?" This system can provide users with an excellent shopping experience through fast and highly personalized product recommendations.

[0651] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0652] Step 1:

[0653] The server collects user information and item information as input data. This process includes inputting the user's past purchase and browsing history, as well as product-related information (category, size, color, price, etc.). This data is stored in a database such as SQL Server or MongoDB. Specifically, when a user logs into the site, their activity history is automatically recorded and updated on the server.

[0654] Step 2:

[0655] The server preprocesses the collected data. The input for preprocessing is the raw data collected in step 1. Libraries such as Pandas and Scikit-learn are used to process the data, including imputing missing values, removing duplicate data, and encoding categorical data. This results in output data in a format easily processed by machine learning models. Specifically, the server performs data cleaning to create a clean dataset ready for analysis.

[0656] Step 3:

[0657] The server trains a machine learning model based on preprocessed data. The model is trained using TensorFlow or PyTorch, based on the preprocessed data as input. The output of this step is a trained model that predicts user preferences and behavior and suggests the optimal product. Specifically, it iteratively processes a large amount of training data, allowing the model to learn purchase patterns and improve prediction accuracy.

[0658] Step 4:

[0659] The server generates product recommendations using a pre-trained model. Input is the user's current activity and past data, and output is a personalized product list. The server receives prompts such as "What are some recommended spring items for people in their 20s?" and lists products based on them. Specifically, the model analyzes available information and quickly selects the most suitable product candidates.

[0660] Step 5:

[0661] The terminal displays a list of recommended products sent from the server to the user. The input data is the product list received from the server, and the output is product information (images, descriptions, prices, etc.) presented visually to the user. The user can view the products on the terminal screen. Specifically, the application generates a product page and displays the product information through the user interface.

[0662] Step 6:

[0663] Users view, select, and purchase recommended products. After purchase, they can provide product ratings and reviews. This feedback is sent to the server as input data and used to further optimize the model. The expected output is an improvement in the accuracy of recommendations that continues to improve. Specifically, the system works by having users provide reviews, and their opinions are reflected in future recommendations.

[0664] (Application Example 1)

[0665] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0666] In modern e-commerce, personalized product recommendations based on users' diverse preferences and purchase history are in demand. However, conventional systems struggle with real-time recommendations, making it difficult for users to quickly discover necessary products. Furthermore, methods for efficiently collecting user feedback and incorporating it into system optimization are not yet well established.

[0667] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0668] In this invention, the server includes means for collecting user information, means for collecting product information, and means for generating personalized product recommendations in real time. This enables real-time product recommendations based on user preferences, allowing for rapid product discovery and system optimization.

[0669] "Collecting user information" means obtaining data about a user's past purchase history, browsing history, and preferred styles and sizes.

[0670] "Collecting product information" means obtaining details such as the product's category, size, color, price, and brand.

[0671] "Preprocessing" refers to preparing data, including imputing missing values, removing duplicate data, and encoding categorical data.

[0672] "Training a model" means using machine learning algorithms to analyze and predict data patterns based on pre-processed data.

[0673] "Making recommendations" means using a trained model to present highly relevant products to a specific user.

[0674] "Generating personalized product recommendations in real time" means processing that quickly suggests appropriate products based on the user's current behavior and preferences.

[0675] "Displaying recommendations" means visually showing the generated product list to the user's output device.

[0676] "Collecting feedback" means gathering data on user opinions and satisfaction levels.

[0677] "Using for optimization" means using the collected feedback to make improvements to enhance the system's performance and accuracy.

[0678] The system implementing this invention consists of a server, a terminal, and a user. The server is responsible for efficiently collecting and managing user information and product information. User information includes data on past purchase history, browsing history, and preferred styles and sizes. Product information includes details such as category, size, color, price, and brand, and this data is aggregated on the server.

[0679] The server uses Python to build machine learning models and employs software libraries such as Scikit-learn and TensorFlow to preprocess the data. Data preprocessing includes imputation of missing values, data deduplication, and encoding of categorical data. This preprocessed data is then processed by the machine learning model to learn user preferences. This model is hosted on Amazon Web Services (AWS), and Amazon RDS is used for database management.

[0680] The device runs an application designed with React Native to provide users with personalized product recommendations in real time. When a user launches the app, recommended products are displayed based on data retrieved from the server. This allows users to quickly discover products that perfectly match their preferences.

[0681] Users can select items of interest from the presented products and provide their opinions and ratings. This feedback is collected again on the server and used to improve the accuracy of the model and optimize the system.

[0682] For example, if a user searches for "sneakers" on the app, blue sneakers could be recommended based on past data. This would allow users to quickly access the products they are looking for, resulting in a more fulfilling shopping experience.

[0683] An example of a prompt would be: "Generate a prompt that suggests recommended products to a user who searched for blue sneakers."

[0684] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0685] Step 1:

[0686] The server collects user information and product information. User information includes purchase history, browsing history, and preferred styles and sizes, while product information includes category, size, color, price, and brand. This data is stored on AWS in preparation for preprocessing in the next step. The input is raw user and product information, and the output is a processed dataset.

[0687] Step 2:

[0688] The server preprocesses the collected data. Specifically, it uses Python and Scikit-learn to impute missing values, remove data deduplication, encode categorical data, and convert it into a format suitable for machine learning. The input is the processed dataset from step 1, and the output is the preprocessed data.

[0689] Step 3:

[0690] The server trains a machine learning model based on pre-processed data. During this process, it uses TensorFlow to execute algorithms and analyze user preferences and purchasing trends. The model is optimized for real-time product recommendations and managed on AWS. Input is pre-processed data, and output is the trained recommendation model.

[0691] Step 4:

[0692] The device retrieves recommended products from a server via an AI model based on user actions. When a user launches the app or performs a search, a product list is generated in real time. The prompt is based on the instruction, "Generate a prompt that suggests recommended products to a user who searched for blue sneakers." The input is the user's search query and app launch information, and the output is a list of recommended products.

[0693] Step 5:

[0694] Users browse or purchase products of interest from a list displayed on their device. Users input their opinions and ratings about the products, which are sent to the server as feedback. Input consists of user selections and feedback information, while output is user satisfaction data.

[0695] Step 6:

[0696] The server retrains and optimizes the model based on the collected feedback. This information is used to improve the accuracy of product recommendations and reflect in future recommendations. The input is user feedback, and the output is the optimized recommendation model.

[0697] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0698] The following describes specific embodiments for carrying out the present invention. This system combines a server, a terminal, a user, and an emotion engine.

[0699] The server is responsible for collecting and managing user and product information. Collected user information includes past purchase history, browsing history, preferences, and body type information, while product information includes product ID, category, size, color, price, and brand details. The server continuously collects and updates this information, maintaining an up-to-date database, which provides a foundation for making appropriate recommendations to users.

[0700] The server trains a machine learning model based on preprocessed data. Data preprocessing includes imputing or removing missing values, converting categorical data to numerical values, and normalizing the data. This prepares the data for efficient model training. The trained model understands user preferences and purchasing trends, providing a foundation for new product recommendations.

[0701] The emotion engine can recognize the user's emotions and acquire emotional information. When a user browses products using their device, it uses cameras and sensors to infer emotions from facial expressions, tone of voice, mouse movements, etc., and sends that information to the server. Emotional information reflects emotional states such as joy, surprise, and dissatisfaction.

[0702] The terminal plays a role in visually displaying recommendations to the user. A list of recommendations from the server is sent to the terminal and displayed along with product images, details, prices, and inventory information. The recommendation results are adjusted to take into account the user's sentiment information, resulting in content optimized for the user.

[0703] Users can browse and purchase products via their devices. They can not only make purchases based on the suggested product recommendations, but also provide feedback on the products, which is then used to further optimize the model.

[0704] For example, if the emotion engine determines that a user is feeling stressed, the server can recommend products with relaxing colors or products that prioritize comfort. In this way, the aim is to provide a more personalized experience by offering individualized product suggestions based on the user's emotional state.

[0705] As described above, the present invention utilizes an emotion engine to provide product recommendations that take into account the user's emotions, thereby achieving a deeper level of personalization and contributing to improved user satisfaction and purchase intent.

[0706] The following describes the processing flow.

[0707] Step 1:

[0708] The server collects user and product information from the database. User information includes past purchase history, browsing history, preferences, and body type information, while product information includes product ID, category, size, color, price, and brand information. Based on this information, the server always maintains the most up-to-date data.

[0709] Step 2:

[0710] The server collects the user's emotions using an emotion engine. This emotion collection is performed by analyzing the user's facial expressions, voice tone, and operation patterns through the device's camera and microphone. The inferred emotional state is then sent to the server.

[0711] Step 3:

[0712] The server preprocesses the collected user information, product information, and sentiment information. It imputes missing values, encodes categorical data, normalizes the data, and converts it into a format suitable for model training.

[0713] Step 4:

[0714] The server uses pre-processed data to train a machine learning model. The model analyzes user preferences and purchasing behavior, and by incorporating sentiment data, it builds a more accurate recommendation logic.

[0715] Step 5:

[0716] When a user views or searches for products from their device, that request is sent to the server.

[0717] Step 6:

[0718] The server generates a list of product recommendations that take into account the user's preferences and emotions, based on the received request and the trained model. For users with a positive emotional state, it recommends products that match that state, and for users with a negative emotional state, it suggests items that will soothe their mood.

[0719] Step 7:

[0720] The device displays recommended items sent from the server to the user. These include product images, details, and pricing information, presented in a way that allows the user to easily compare and select items.

[0721] Step 8:

[0722] Users can browse the offered products and purchase selected items. Furthermore, they can provide feedback on the products and experience after purchase, contributing to improving the accuracy of future recommendations.

[0723] This processing flow allows users to receive product suggestions tailored to their emotional state, enabling them to enjoy a more personalized shopping experience.

[0724] (Example 2)

[0725] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0726] Traditional recommendation systems rely solely on users' past data, making it difficult to suggest products that take emotional changes into account. This prevents them from providing personalized recommendations based on users' real-time emotional states. As a result, improvements in user satisfaction are limited, and there are challenges in optimizing purchase intent.

[0727] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0728] In this invention, the server includes means for collecting user information, means for collecting product information, means for training a machine learning model based on preprocessed data, means for acquiring sentiment data, and means for recommending products to the user using the machine learning model and sentiment data. This enables highly accurate product recommendations that take into account the user's current emotional state.

[0729] "User information" refers to data related to individual users, such as their purchase history, browsing history, preferences, and body type information.

[0730] "Product information" refers to data related to a product, such as product ID, category, size, color, price, and brand details.

[0731] "Preprocessing" refers to the process of preparing data into a format suitable for analysis and model training, such as imputing or removing missing values, converting categorical data to numerical values, and normalizing data.

[0732] A "machine learning model" refers to a mathematical model that uses algorithms to analyze data and learn patterns to predict and classify future data.

[0733] "Emotional data" refers to information that digitally represents the emotional state inferred from a user's facial expressions, tone of voice, and behavior.

[0734] "Recommending products" means suggesting products that are likely to interest or appeal to individual users, based on their data.

[0735] "Model optimization" refers to the process of making adjustments and improvements to a machine learning model based on collected data and feedback in order to enhance its accuracy.

[0736] A description of embodiments for carrying out this invention will be given.

[0737] This system combines a server, terminals, users, and a data processing engine. The server collects and manages user information and product information. User information includes purchase history and preferences, while product information includes various details such as product ID and price. To integrate this information and store it in a database, the server efficiently retrieves external data using a REST API.

[0738] Next, the server preprocesses the data. This involves using Python to perform operations such as imputing missing values, converting data to numerical values, and normalizing the data. This prepares the data for training models using machine learning libraries like TensorFlow and PyTorch.

[0739] Furthermore, when a user uses a device to browse products, the device collects emotional data. The device's camera and microphone are used to analyze the user's facial expressions and tone of voice, and a data processing engine interprets this data to send real-time emotional status to the server.

[0740] The server combines trained machine learning models with collected sentiment data to generate product recommendations for the user. For example, if it determines that a user is stressed, it can recommend products that promote relaxation. This process provides a personalized experience and increases user satisfaction.

[0741] As a concrete example, a prompt might read, "Build an optimal model for recommending products that help users relax when they are experiencing stress." By inputting such prompts into the AI ​​model, a more accurate recommendation system can be achieved.

[0742] In this way, the present invention constructs a system that provides a more personalized purchasing experience by recommending products that take into account the emotional state of the user.

[0743] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0744] Step 1:

[0745] The server collects user information and product information. It uses data obtained from external databases and APIs as input, and outputs an updated database integrating this information. In this process, the server uses REST APIs to collect information and stores purchase history and product details in the database.

[0746] Step 2:

[0747] The server performs preprocessing of the collected data. It uses raw data as input and outputs data converted into a format suitable for machine learning. Specifically, it performs processes such as filling in missing values ​​with the mean, one-hot encoding to convert categorical data to numerical data, and data normalization. These data processing steps are performed using Python.

[0748] Step 3:

[0749] The server trains a machine learning model based on preprocessed data. It uses a preprocessed dataset as input and outputs a trained model. The server uses machine learning frameworks such as TensorFlow and PyTorch to build a model that predicts user purchasing trends.

[0750] Step 4:

[0751] The device collects emotional data when users browse products. It uses real-time data such as the user's facial expressions and voice as input, and outputs analyzed emotional information. The device uses its camera and microphone to analyze the user's facial expressions and voice tone, and transmits the emotional data to the server in real time.

[0752] Step 5:

[0753] The server generates product recommendations by combining machine learning models and sentiment data. It uses a trained model and sentiment data as input, and outputs individually optimized product lists. The server specifically creates recommendation lists that include relaxing products when it determines the user is experiencing stress.

[0754] Step 6:

[0755] The terminal visually displays recommendation information received from the server to the user. It uses a list of recommended products as input and outputs a product list that the user can view on the screen. The terminal displays images, detailed information, prices, etc., through a user interface to ensure easy understanding for the user.

[0756] (Application Example 2)

[0757] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0758] Online platforms face the challenge of providing product recommendations that take into account users' emotional needs. Traditional systems often rely solely on users' past purchase and browsing history, making it difficult to achieve personalization that reflects real-time emotional states. Therefore, there is a growing need for product recommendations that respond to users' real-time emotions.

[0759] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0760] In this invention, the server includes means for collecting user information, product information, and sentiment data; means for preprocessing this information; and means for training a machine learning model based on the preprocessed data. This enables product recommendations tailored to the user's emotional state.

[0761] "User information" refers to data that includes an individual's purchase history, browsing history, preferences, and other related information.

[0762] "Product information" refers to detailed data about a product, such as product ID, category, size, color, price, and brand.

[0763] "Emotional data" refers to information that indicates the emotional state of a user, inferred from their facial expressions, tone of voice, mouse movements, etc.

[0764] "Preprocessing" refers to the process of imputing or removing missing values, converting categorical data to numerical values, and normalizing data so that machine learning models can process raw data efficiently.

[0765] A "machine learning model" is a mathematical algorithm that is trained using data to make predictions and judgments about specific tasks.

[0766] "Product recommendation" is the process of presenting appropriate products based on a user's past behavior and current emotional state.

[0767] The system implementing this invention uses a combination of hardware and software to effectively utilize user information, product information, and sentiment data. The server uses a machine learning model built with Python and TensorFlow / Keras to perform personalized product recommendations to users based on the collected data. Three main components—the server, the terminal, and the user—interact to perform specific functions.

[0768] The server collects user and product information into a database and performs preprocessing. Preprocessing includes imputing missing values, converting categorical data to numerical values, and normalizing the data. A machine learning model learns from this preprocessed data, analyzing the user's past purchasing trends and emotional state to recommend products. This enables the provision of information tailored to the individual needs of each user.

[0769] The terminal consists of visual devices such as smartphones, and uses a camera and microphone to sense the user's facial expressions and voice. The emotion engine analyzes this data in real time and transmits the emotional state to the server.

[0770] Users visually review recommended products via their devices and consider those that interest them. For example, a user experiencing stress might be recommended products with relaxing effects, and the user can intuitively grasp this information. A generative AI model is used in this process to generate and analyze data according to prompt messages.

[0771] For example, using a prompt message such as, "Please input the user's facial expression and voice tone data, and output recommended product categories," the system recommends appropriate products that reflect the user's emotional state. In this way, it is possible to provide users with a personalized shopping experience based on emotional information obtained from cameras and microphones.

[0772] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0773] Step 1:

[0774] The server collects user and product information from the database. As input, it retrieves the user's past purchase and browsing history, product IDs, and category information, and stores them in the database. Based on this, the server formats the data in a way that makes it easy to use in subsequent processing.

[0775] Step 2:

[0776] The device uses a camera and microphone to capture the user's facial expressions and voice data in real time. The input consists of the user's facial image and voice, and this data is sent to an emotion engine for facial expression analysis and voice tone analysis. The output is information indicating the emotional state.

[0777] Step 3:

[0778] The server receives sentiment data sent from the sentiment engine and performs preprocessing. The input is user sentiment information, which is quantified and converted into a format that can be used by machine learning models. This includes data cleansing and normalization.

[0779] Step 4:

[0780] The server trains a machine learning model using pre-processed data. It takes clean user information, product information, and sentiment data as input and builds a predictive model through computation. The output is ready for product recommendations tailored to the sentiment of a specific user.

[0781] Step 5:

[0782] The server uses a pre-trained machine learning model to recommend appropriate products to the user. The input is a set of pre-processed information, and the output of the machine learning model generates a product list tailored to the user's emotional state. The recommendation results are then sent to the user's terminal.

[0783] Step 6:

[0784] The terminal visually displays product recommendations received from the server. The input is a list of recommended products, and the user interface displays product images and detailed descriptions. This allows the user to select a product.

[0785] Step 7:

[0786] Users review and select recommended products via their devices and provide purchase information as feedback as needed. This allows the server to receive feedback and further optimize its model.

[0787] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0788] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0789] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0790] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0791] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0792] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0793] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0794] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0795] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0796] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0797] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0798] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0799] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0800] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0801] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0802] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0803] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0804] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0805] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0806] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0807] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0808] The following is further disclosed regarding the embodiments described above.

[0809] (Claim 1)

[0810] Means of collecting user information,

[0811] Means of collecting product information,

[0812] Means for pre-processing the user information and product information,

[0813] A means for training a model based on the aforementioned preprocessed data,

[0814] A means of making recommendations to users using the aforementioned model,

[0815] A system that includes this.

[0816] (Claim 2)

[0817] The system according to claim 1, further comprising means for displaying the aforementioned recommendation on the user's terminal.

[0818] (Claim 3)

[0819] The system according to claim 1, comprising means for collecting user feedback and using it to optimize the model.

[0820] "Example 1"

[0821] (Claim 1)

[0822] Means of obtaining user information,

[0823] Means for obtaining item information,

[0824] Means for pre-processing the user information and item information,

[0825] A means for training a learning device based on the aforementioned preprocessed data,

[0826] Means for generating product recommendations using the aforementioned learning device,

[0827] A system that includes means for recording feedback based on user instructions.

[0828] (Claim 2)

[0829] The system according to claim 1, which visually displays the aforementioned product recommendations on the user's display device.

[0830] (Claim 3)

[0831] The system according to claim 1, which optimizes the learning device using the aforementioned feedback.

[0832] "Application Example 1"

[0833] (Claim 1)

[0834] Means of collecting user information,

[0835] Means of collecting product information,

[0836] Means for pre-processing the user information and product information,

[0837] A means for training a model based on the aforementioned preprocessed data,

[0838] A means of making recommendations to users using the aforementioned model,

[0839] A means of generating personalized product recommendations in real time,

[0840] A system that includes this.

[0841] (Claim 2)

[0842] The system according to claim 1, further comprising means for displaying the recommendation on the user's output device.

[0843] (Claim 3)

[0844] The system according to claim 1, comprising means for collecting user feedback and using it to optimize the model.

[0845] "Example 2 of combining an emotion engine"

[0846] (Claim 1)

[0847] Means of collecting user information,

[0848] Means of collecting product information,

[0849] Means for pre-processing the user information and product information,

[0850] A means for training a machine learning model based on the aforementioned preprocessed data,

[0851] Means of acquiring emotional data,

[0852] A means for recommending products to users using the aforementioned machine learning model and sentiment data,

[0853] A system that includes this.

[0854] (Claim 2)

[0855] The system according to claim 1, comprising means for displaying the aforementioned recommendation on the user's device.

[0856] (Claim 3)

[0857] The system according to claim 1, which collects feedback from users and uses it to optimize the model.

[0858] "Application example 2 when combining with an emotional engine"

[0859] (Claim 1)

[0860] Means of collecting user information,

[0861] Means of collecting product information,

[0862] Means for preprocessing the aforementioned user information, product information, and sentiment data,

[0863] A means for training a machine learning model based on the aforementioned preprocessed data,

[0864] A means of providing product recommendations based on the user's emotional state using the aforementioned model,

[0865] A system that includes this.

[0866] (Claim 2)

[0867] The system according to claim 1, which displays the aforementioned recommendation on a visual device.

[0868] (Claim 3)

[0869] The system according to claim 1, which distributes emotional data and uses it for model training and optimization. [Explanation of Symbols]

[0870] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. Means of collecting user information, Means of collecting product information, Means for pre-processing the user information and product information, A means for training a model based on the aforementioned preprocessed data, A means of making recommendations to users using the aforementioned model, A system that includes this.

2. The system according to claim 1, further comprising means for displaying the aforementioned recommendation on the user's terminal.

3. The system according to claim 1, comprising means for collecting feedback from users and using it to optimize the model.

Images