System

Abstract

A system is provided.SOLUTION: A personalized shopping assistant system comprising: means for inputting a purchase history, a use history, and a use impression of a user; means for receiving and storing the purchase history, the use history, and the use impression in a database; machine learning means for learning a tendency of the user based on data stored in the database; means for selecting an appropriate product based on the tendency of the user learned by the machine learning means; and means for receiving feedback of the selected product and storing the feedback in the database.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Today's consumers spend a lot of time and effort selecting interior and home appliances that fit their lifestyles and preferences. Furthermore, product selection requires consideration of a wide range of factors, including design, cost, durability, and functionality, making the selection process extremely complex. This can make choosing the best product difficult and stressful. Therefore, there is a growing need for systems that can learn users' preferences and usage patterns and provide personalized product recommendations based on them. [Means for solving the problem]

[0005] The present invention provides a means for inputting a user's purchase history, usage history, and usage impressions, and a means for receiving and storing said purchase history, usage history, and usage impressions in a database. A system is constructed that includes a means for learning user trends using a machine learning algorithm based on this data. The system also includes a means for selecting appropriate products based on the learning results and a means for notifying the user of information about said products. The accuracy of the system is also improved by providing a means for receiving user feedback and storing it in a database. This allows users to easily select the optimal interior and home appliances that suit their lifestyle and preferences.

[0006] "User" refers to an individual or entity equivalent to an individual who uses this system to purchase home appliances and interior goods and provide feedback.

[0007] "Purchase history" refers to detailed information about home appliances and interior items a user has purchased in the past, including product name, brand, purchase date, price, and place of purchase.

[0008] "Usage history" refers to historical data about how a user has used the home appliances and interior items they purchased, including frequency of use and usage conditions.

[0009] "Usage impressions" refers to the impressions and evaluations that users have of the home appliances and interior items they have purchased, and specifically includes feedback such as satisfaction after use, dissatisfaction, and areas for improvement.

[0010] The "database" is a data repository used in this system to accumulate and manage data such as users' purchase history, usage history, and usage impressions.

[0011] "Machine learning methods" refer to methods that use algorithms to learn and analyze user trends and preferences based on collected data.

[0012] The "selection means" is a means for selecting the most suitable product based on the user's tendencies and preferences learned by the machine learning means.

[0013] "Notification means" refers to the method of informing users of information about the selected product, and specifically includes in-app notifications and push notifications.

[0014] The "feedback means" is a means for receiving user feedback and evaluations and storing them in a database.

[0015] A "personalized shopping assistant system" refers to an individualized shopping support system that learns a user's purchase history, usage history, and usage impressions, and then suggests the most suitable products based on that information. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9]1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] This invention is a personalized shopping assistant system that uses AI to learn the user's style and preferences based on their purchase and usage history and impressions of interior and home appliances, and then suggests the most suitable interior and home appliances.

[0038] System Overview

[0039] This system mainly consists of a server and a terminal (user device). The server stores data, learns, selects products, and makes suggestions, while the terminal is responsible for inputting data and receiving suggestions.

[0040] Data collection

[0041] Enter your purchase history

[0042] User: Through the interface, enter detailed information about the home appliance or interior item purchased (product name, brand, price, purchase location, purchase date).

[0043] Device: Purchase information is stored locally and sent to the server over the network.

[0044] Entering usage history and impressions

[0045] Users: Regularly enter usage, feedback, and satisfaction with purchased products.

[0046] Device: Usage history and user feedback data is stored locally and sent to the server.

[0047] Data storage and analysis

[0048] Receiving and storing data

[0049] Server: Stores purchase history, usage history, and user feedback sent from user devices in a database.

[0050] Data preprocessing and feature extraction

[0051] Server: Preprocesses the received data and extracts user features (preferred brands, price range, features, etc.).

[0052] Machine learning model updates

[0053] Server: Periodically analyzes data using machine learning algorithms to adapt the learning model to reflect user style and preferences.

[0054] Product selection and proposal

[0055] Selecting the right product

[0056] Server: Based on the learning model, the server selects products that match the user's preferences. This selection process also takes into account price, functionality, design, and vendor information.

[0057] Proposal generation and submission

[0058] Server: Generates detailed information about the selected product (product name, key features, price, where it can be purchased, and the best time to purchase).

[0059] Server: Sends the proposals to the user's device.

[0060] Receiving suggestions and feedback

[0061] Receiving Proposals

[0062] Device: Receives suggestions from the server and notifies the user. Notification methods include in-app notifications and push notifications.

[0063] Enter and submit feedback

[0064] User: Purchases the suggested product and enters their impressions and ratings.

[0065] Device: Feedback is stored locally and periodically sent to the server.

[0066] Server: The feedback is stored in a database and used as the basis for retraining.

[0067] Specific examples

[0068] For example, if a user buys a new smart refrigerator and enters details and user experience into the application, the server can:

[0069] 1. Server: Stores purchase data in a database and performs preprocessing together with past data.

[0070] 2. Server: A machine learning model learns user preferences and determines whether a newly released smart oven is suitable for the user.

[0071] 3. Server: Generates detailed information about the smart oven (price, features, vendor, best time to purchase) and sends it to the device.

[0072] 4. On the device: Notify the user of the offer and encourage them to purchase.

[0073] 5. User: Purchases the proposed smart oven and enters their experience using it into the terminal.

[0074] 6. Device: Sends usage information to the server and reflects it in the next learning session.

[0075] In this way, the system continuously learns user trends and continues to suggest appropriate products.

[0076] The processing flow will be explained below.

[0077] Step 1:

[0078] User: When purchasing home appliances or interior items, the user accesses the application and enters their purchase history, including the product name, brand, price, purchase location, and purchase date.

[0079] Step 2:

[0080] Terminal: Purchase information is stored in a local database and sent to a server over the network.

[0081] Step 3:

[0082] Server: Receives purchase information sent from the device and stores it in a central database.

[0083] Step 4:

[0084] User: Enters information about the usage and impressions of the purchased product into the application, such as frequency of use, ease of use, and satisfaction level.

[0085] Step 5:

[0086] Terminal: Usage history and user feedback data is stored in a local database and sent to a server via the network.

[0087] Step 6:

[0088] Server: Receives usage history and feedback sent from the device and stores it in a central database.

[0089] Step 7:

[0090] Server: Preprocesses the purchase history, usage history, and usage impressions stored in the database to make them easier to analyze. Specifically, it handles missing values ​​and standardizes the format of the data.

[0091] Step 8:

[0092] Server: Extracts features from the preprocessed data to identify user preferences and trends, such as preferred brands, price ranges, and features.

[0093] Step 9:

[0094] Server: Uses the features to update the machine learning model and learn about the user's style and preferences.

[0095] Step 10:

[0096] Server: Using a learning model, the server selects product candidates that match the user's preferences, taking into account price, functionality, design, and other factors.

[0097] Step 11:

[0098] Server: Generates detailed information about the selected product, including the product name, key features, price, where it is available, and the best time to purchase.

[0099] Step 12:

[0100] Server: Sends the selected products and their details to the user's device.

[0101] Step 13:

[0102] On the device, the user is notified of the suggestions received from the server via in-app notifications and push notifications.

[0103] Step 14:

[0104] User: Review the suggested products and purchase them if they are interested.

[0105] Step 15:

[0106] Users: Enter post-purchase feedback into the application, including experience and satisfaction.

[0107] Step 16:

[0108] Terminal: The feedback is stored in a local database and sent to a server over the network.

[0109] Step 17:

[0110] Server: Receives feedback sent from devices and stores it in a central database for future analysis and learning.

[0111] Step 18:

[0112] Server: Using the feedback data, the machine learning model is updated again to improve the accuracy of the suggestions.

[0113] Example 1

[0114] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0115] Conventional shopping assistant systems have had the challenge of making recommendations that fully reflect a user's individual preferences and past purchase history. It is also difficult to absorb user feedback in real time and use it in future recommendations. Furthermore, they are unable to provide sufficient recommendations on the best time and place to purchase, limiting the user experience.

[0116] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0117] In this invention, the server includes means for inputting a user's purchase history, usage history, and usage impressions, means for receiving and storing the purchase history, usage history, and usage impressions in a database, means for preprocessing the data stored in the database, means for extracting features from the preprocessed data, machine learning means for learning user tendencies based on the data stored in the database and the extracted features, means for selecting appropriate products based on the user tendencies learned by the machine learning means, means for sending and notifying the user of information about the selected products to the user's terminal, and means for receiving and storing feedback on the notified products in the database. This enables personalized suggestions that reflect the user's individual preferences and past purchase history.

[0118] "Users" are individuals or corporations who use this system to purchase interior goods and home appliances, manage their usage, etc.

[0119] "Purchase history" refers to data on detailed information about interior and home appliance products purchased by a user (product name, brand, price, purchase location, purchase date, etc.).

[0120] "Usage history" refers to data about how users use interior and home appliances they have purchased.

[0121] "Usage impressions" is data that represents the evaluations and opinions that users have about the interior design and home appliances they have used.

[0122] "Means" are the specific methods or tools used to realize the system.

[0123] A "database" is a storage device for centrally managing and storing various data such as purchase history, usage history, and usage impressions.

[0124] "Preprocessing" is the process of correcting or removing inconsistencies or missing data and preparing the data for analysis before data analysis.

[0125] "Features" refers to the numerical representation of user preferences and behavioral tendencies extracted from a database, which are used as input data in machine learning.

[0126] "Machine learning" is an algorithm that allows a computer to learn from past data and make predictions and decisions based on future data.

[0127] "Selection" is the process of using a machine learning model to determine the product candidates that best suit the user's preferences.

[0128] "Terminal" means the device (e.g., smartphone, tablet, PC) that a user uses to interact with the system.

[0129] "Notification" refers to the act of transmitting information from a server to a user, and includes in-app notifications and push notifications.

[0130] "Feedback" refers to information about users' impressions and evaluations of the proposed product after using it.

[0131] This invention is a personalized shopping assistant system in which a generative AI learns a user's style and preferences based on their purchase history, usage history, and impressions of interior and home appliances, and suggests the most suitable interior and home appliances. This system is mainly composed of a server and a terminal (user device).

[0132] composition

[0133] server

[0134] The server serves as a central location for receiving, storing, pre-processing, and machine learning data. It has the following main functions:

[0135] Purchase history, usage history, and usage feedback are received and stored in a database.

[0136] Preprocess the received data and extract features.

[0137] Based on the features, machine learning algorithms (e.g., random forests, neural networks) are used to learn user tendencies.

[0138] Based on the learning results, the system selects the most suitable product for the user and generates detailed information about it.

[0139] The selected product information is sent to the user's device.

[0140] Terminal

[0141] A terminal is a device that a user uses to enter data and receive notifications from a server. Its main functions are:

[0142] Provides an interface for inputting purchase history, usage history, and usage impressions.

[0143] The user is notified of the proposal information received from the server.

[0144] Data processing and calculation

[0145] Data collection

[0146] User: Using a dedicated app or web portal on a smartphone or PC, enter your purchase history (product name, brand, price, purchase location, purchase date). Example: Value Range Refrigerator, AB manufacturer, 50,000 yen, XYZ store, October 1, 2023.

[0147] Terminal: The information entered by the user is temporarily stored in a local database and sent to a server via the Internet.

[0148] User: Enter usage status, impressions, and satisfaction level of the purchased product. Example: Good cooling performance, excellent design.

[0149] Data storage and analysis

[0150] Server: Stores the received purchase history, usage history, and user feedback in a database.

[0151] Server: Preprocesses the received data, corrects missing data, and removes inconsistent data. Then, extracts features based on user preferences and behavioral trends.

[0152] Server: Uses machine learning algorithms based on features to learn user trends.

[0153] Product selection and proposal

[0154] Server: Selects the best product for the user based on learned trends, taking into account information such as price, functionality, design, and vendor.

[0155] Server: Generates detailed information about the selected product (product name, key features, price, where it can be purchased, and the best time to purchase) and sends it to the user's device.

[0156] On the device: Notify the user of the received recommendation. Example: Display an in-app notification saying, "Our new smart oven is perfect for you!"

[0157] Enter and submit feedback

[0158] User: Purchases the suggested product and enters their impressions and ratings into the device.

[0159] Device: Stores feedback data locally and sends it to the server.

[0160] Server: Receives feedback and stores it in a database for future machine learning.

[0161] Specific examples

[0162] Prompt Sentence Examples

[0163] "Please tell us the details of the home appliance you recently purchased (product name, brand, price, place of purchase, purchase date)."

[0164] "Please tell us about your experience using and satisfaction with the home appliances you purchased."

[0165] "Tell me about the next interior or home appliance you're considering purchasing."

[0166] These prompts can be used to gather the necessary data from the user and provide optimal product recommendations.

[0167] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0168] Step 1:

[0169] Input: User enters purchase history through application or web portal. For example: product name, brand, price, purchase location, purchase date.

[0170] How it works: Users use the interface on their smartphone or computer to enter detailed information about the home appliances and interior items they have purchased.

[0171] Output: Purchase information is stored on the device and data is generated that is later sent to a server.

[0172] Step 2:

[0173] Input: Purchase history data and details entered by the user.

[0174] How it works: The device temporarily stores the purchase information in a local database and then sends it over the internet to a server, periodically retrying the transmission until it is successful.

[0175] Output: Purchase history data sent to the server.

[0176] Step 3:

[0177] Input: Purchase history data received by the server.

[0178] How it works: The server receives the purchase history sent from the device and stores it in a database.

[0179] Output: Purchase history stored in a database.

[0180] Step 4:

[0181] Input: User-entered usage history data such as usage status, impressions, and satisfaction.

[0182] How it works: Users periodically enter their impressions and satisfaction with the device. Examples include "This refrigerator has great cooling performance" and "It has a nice design."

[0183] Output: Usage history and impressions are stored on the device and data is generated that is later sent to the server.

[0184] Step 5:

[0185] Input: Usage history and user feedback data.

[0186] How it works: The device stores this data locally and sends it to the server.

[0187] Output: Usage history and user feedback data sent to the server.

[0188] Step 6:

[0189] Input: Usage history and user feedback data received by the server.

[0190] Operation: The server receives the usage history and usage impressions sent from the device and stores them in a database.

[0191] Output: Usage history and user feedback data stored in a database.

[0192] Step 7:

[0193] Input: Purchase history, usage history, and user feedback data stored in the database.

[0194] How it works: The server preprocesses the received data, correcting or removing inconsistencies and missing data, and then extracts features that quantify user preferences and behavioral trends.

[0195] Output: Preprocessed data and extracted features.

[0196] Step 8:

[0197] Input: Preprocessed data and extracted features.

[0198] How it works: Using machine learning algorithms based on features, it learns user tendencies.

[0199] Output: An updated machine learning model.

[0200] Step 9:

[0201] Input: The machine learning model obtained by training.

[0202] How it works: The server uses a learning model to recommend products that match the user's preferences. Example: "The new smart oven is perfect for you."

[0203] Output: Suitable products and their details (price, key features, where available, best time to buy, etc.).

[0204] Step 10:

[0205] Input: Detailed information about the selected product.

[0206] How it works: The server generates detailed information about the selected product and sends it to the user's device.

[0207] Output: The suggestions sent to the user's device.

[0208] Step 11:

[0209] Input: Proposal notification.

[0210] How it works: The device receives the offer from the server and notifies the user. Notification methods include in-app notifications and push notifications. Example: "Our new smart oven is perfect for you!"

[0211] Output: A notification showing the suggestion.

[0212] Step 12:

[0213] Input: User's impressions and ratings of the proposed product.

[0214] How it works: The user uses the suggested product and enters their impressions and ratings.

[0215] Output: Feedback data is generated and saved to the device.

[0216] Step 13:

[0217] Input: Feedback data.

[0218] How it works: The device stores the feedback data locally and sends it to the server.

[0219] Output: Feedback data sent to the server.

[0220] Step 14:

[0221] Input: The feedback data received by the server.

[0222] How it works: The server receives the feedback data and stores it in a database. It is used for retraining.

[0223] Output: Feedback data stored in a database.

[0224] (Application example 1)

[0225] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0226] Conventional personalized shopping assistant systems have had problems such as limited product suggestions based on users' purchase history and usage impressions, being available only in specific locations, or having poor visual suggestions. Furthermore, there was a lack of means for users to check the experience of the suggested products in advance, and there were limitations on providing information on the optimal time and place to purchase, which could lead to a decrease in user satisfaction.

[0227] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0228] In this invention, the server includes a means for inputting a user's purchase history, usage history, and usage impressions, a means for receiving the purchase history, usage history, and usage impressions and storing them in a database, and a means for preprocessing the data stored in the database and extracting features. This makes it possible to learn user trends, suggest optimal products, visually display them in a virtual store, and suggest where they can be purchased and the optimal time to purchase them.

[0229] "User" means any individual or legal entity using a particular product or service.

[0230] "Purchase history" refers to information about products and services a user has purchased in the past.

[0231] "Usage history" refers to data about how a user uses a purchased product or service.

[0232] "Usage impressions" refers to the impressions and evaluations that users have after using a product or service.

[0233] "Input means" refers to the interface or function that allows users to input data into the system.

[0234] A "database" refers to a system that systematically stores information and makes it available for searching and use as needed.

[0235] "Preprocessing" refers to a series of steps that convert collected data into an analyzable format.

[0236] "Features" refer to useful attributes or parameters extracted from data.

[0237] "Machine learning methods" refer to techniques that allow computers to learn from data and recognize patterns.

[0238] "Goods" means any products or services intended for purchase or use by Users.

[0239] A "virtual store" refers to an online shopping environment created using virtual reality technology.

[0240] "Visual presentation means" refers to techniques or methods for presenting information graphically to a user.

[0241] "Feedback" refers to the ratings and opinions users provide about products and services.

[0242] The "optimal time to purchase" refers to the time period that encourages users to make a purchase at the most advantageous time.

[0243] This invention is a personalized shopping assistant system that uses AI to learn the user's style and preferences based on their purchase and usage history and impressions of interior and home appliances, and then suggests the most suitable products. The system is mainly composed of a server and a terminal (user device).

[0244] System Overview

[0245] Data collection

[0246] Users enter their purchase history and usage impressions via the terminal. They enter detailed information about the purchased product, usage status, impressions, and satisfaction level, and the terminal collects this information. The collected data is sent to the server via the network.

[0247] Data storage and analysis

[0248] The server stores the received purchase history, usage history, and usage impressions in a database. The stored data is preprocessed to extract user features (preferred brands, price ranges, features, etc.). The features are analyzed using a machine learning model to learn the user's style and preferences.

[0249] Product selection and proposal

[0250] The server selects products that match the user's preferences based on the learning model. This selection process also takes into account price, functionality, design, and vendor information. The selected product information is visually displayed in a virtual store.

[0251] Program processing

[0252] 1. Data Acquisition and Transmission

[0253] The device (such as the user's VR headset or smart glasses) collects purchase history and usage impressions and sends them to the server. The hardware used includes Oculus as a VR headset and Google® Glass® as smart glasses.

[0254] 2. Data analysis and proposal generation

[0255] The server uses the Python Flask framework to analyze the received data. It uses machine learning models such as TENSORFLOW (registered trademark) to learn the user's preferences and select appropriate products. This selected product information is stored in a database and then sent back to the device.

[0256] 3. Virtual store display

[0257] The device uses software such as Unity or Unreal Engine to visually display product information in a virtual environment, allowing users to move around the virtual store and examine the suggested products in detail.

[0258] Specific examples

[0259] For example, if a user purchases a new smart refrigerator and enters details and user experience into an application, the flow would be as follows:

[0260] Users enter information about the smart refrigerator they have purchased and their experience using it into the application.

[0261] The terminal collects this data and sends it to the server.

[0262] The server receives and stores the data and uses machine learning models to learn user preferences.

[0263] The server then selects a new smart oven that is suitable for the user and generates detailed information about it.

[0264] The terminal visually presents the proposed smart oven to the user in a virtual reality environment.

[0265] Users can view suggested smart ovens in a virtual store and consider purchasing them.

[0266] Prompt Sentence Examples

[0267] user_id: "12345"

[0268] purchased_items:

[0269] item: "Smart Refrigerator"

[0270] brand: "BrandA"

[0271] price: 50000

[0272] purchase_date: "2023-08-01"

[0273] item: "Washing machine"

[0274] brand: "BrandB"

[0275] price: 30000

[0276] purchase_date: "2023-01-15"

[0277] user_feedback:

[0278] item: "Smart Refrigerator"

[0279] Rating: 5

[0280] comment: "Very useful"

[0281] item: "Washing machine"

[0282] Rating: 4

[0283] comment: "Very satisfied"

[0284] In this way, the system suggests optimal products based on the user's purchase history and usage impressions, enabling a visual experience in a virtual store.

[0285] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0286] Step 1:

[0287] Users enter their purchase history and usage experience, including details such as product name, brand, price, purchase location, and purchase date. This data is collected and stored locally on the device (e.g., smart glasses or VR headset). The input data format is as follows:

[0288] Input: Product name, brand, price, purchase location, purchase date

[0289] Output: Locally stored purchase and usage data

[0290] Step 2:

[0291] The device sends the collected purchase history and usage feedback to a server via the network, using the Oculus SDK and Google Glass API as interfaces.

[0292] Input: Locally stored purchase and usage data

[0293] Output: Data sent to the server

[0294] Step 3:

[0295] The server stores the received purchase history, usage history, and usage feedback in a database. After storing this data, it is preprocessed for analysis.

[0296] Input: Submitted purchase history, usage history, and user impressions

[0297] Output: Data stored in the database

[0298] Step 4:

[0299] The server preprocesses the data and extracts user features, including cleaning, normalizing, handling missing values, and performing feature engineering.

[0300] Input: Raw data stored in a database

[0301] Output: Extracted features

[0302] Step 5:

[0303] The server inputs the features into a machine learning model (e.g., TensorFlow) to learn the user's style and preferences. The model recognizes and learns patterns based on the user's past data.

[0304] Input: Extracted features

[0305] Output: A learned model containing user preference patterns

[0306] Step 6:

[0307] The server then uses the learning model to select products that suit the user's preferences, taking into account product price, functionality, design, and vendor information.

[0308] Input: User preference patterns

[0309] Output: A list of recommended products

[0310] Step 7:

[0311] The server generates detailed information about the selected product (product name, main features, price, distributor, best time to purchase) and sends it to the terminal.

[0312] Input: A list of recommended products

[0313] Output: Proposal to the device

[0314] Step 8:

[0315] The device visually displays the received recommendations in a virtual store. Users can view the suggested products in the virtual space using a VR headset or smart glasses. The system is displayed using Unity or Unreal Engine.

[0316] Input: Proposal

[0317] Output: Visual representation of the virtual store

[0318] Step 9:

[0319] Users can view the proposed products in the virtual store and provide feedback, which is collected at the terminal and sent to the server.

[0320] Input: User feedback

[0321] Output: Feedback data to the server

[0322] Step 10:

[0323] The server stores the received feedback in a database and uses it to update the machine learning model, thereby improving the accuracy of the suggestions.

[0324] Input: User feedback data

[0325] Output: Updated training model

[0326] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0327] This is a personalized shopping assistant system that uses a generation AI to learn the user's style and preferences based on their purchase and usage history, impressions, and emotional data on interior and home appliances, and then suggests the most suitable interior and home appliances. In particular, by combining it with an emotion engine, it is possible to suggest products that take into account the user's emotional state, providing a more personalized service.

[0328] System Overview

[0329] This system is mainly composed of a server, a terminal (user device), and an emotion engine. The server stores data, learns, selects products, and makes recommendations, while the terminal is responsible for inputting data and receiving recommendations. The emotion engine collects and analyzes user emotion data and sends it to the server.

[0330] Data collection

[0331] Enter your purchase history

[0332] User: Through the interface, the user enters detailed information about the home appliance or interior item they purchased, such as the product name, brand, price, purchase location, and purchase date.

[0333] Terminal: Purchase information is stored in a local database and sent to a server over the network.

[0334] Entering usage history and impressions

[0335] Users: Periodically enter information about the usage, impressions, and satisfaction of purchased products, such as frequency of use, ease of use, and satisfaction level.

[0336] Device: Usage history and user feedback data is stored in a local database and sent to the server.

[0337] Collecting Emotional Data

[0338] On-device: Collects real-time emotional data from the user's facial expressions, tone of voice, and input text. This data is analyzed through an emotion engine.

[0339] Emotion engine: Analyzes collected emotion data to identify the user's emotional state, specifically identifying emotions such as joy, sadness, anger, and surprise.

[0340] Emotion engine: Sends the emotion analysis results to the server.

[0341] Data storage and analysis

[0342] Receiving and storing data

[0343] Server: Stores the purchase history, usage history, usage impressions, and emotion data sent from the device and emotion engine in a database.

[0344] Data preprocessing and feature extraction

[0345] Server: Preprocesses the received data to make it easier to analyze. Specifically, it handles missing values ​​and standardizes the format of the data. Emotion data is also preprocessed.

[0346] Server: Extract user features (preferred brands, price range, features, emotional state, etc.) from the preprocessed data.

[0347] Machine learning model updates

[0348] Server: Periodically analyzes the data using machine learning algorithms to adapt the learning model to reflect user style and preferences. Emotional data is also incorporated into the learning process.

[0349] Product selection and proposal

[0350] Selecting the right product

[0351] Server: Based on the learning model, the server selects product candidates that match the user's preferences. This selection process takes into account price, functionality, design, and emotional data.

[0352] Proposal generation and submission

[0353] Server: Generates detailed information about the selected product, including the product name, key features, price, where it is available, and the best time to purchase.

[0354] Server: Sends the selected products and their details to the user's device.

[0355] Receiving suggestions and feedback

[0356] Receiving Proposals

[0357] Device: Receives suggestions from the server and notifies the user via in-app notifications and push notifications.

[0358] Enter and submit feedback

[0359] User: Review the suggested products, purchase them if they are interested, and enter feedback into the application after purchase, including their experience and satisfaction.

[0360] Terminal: The feedback is stored in a local database and sent to a server over the network.

[0361] Server: The feedback sent from the device is stored in a database and used as the basis for retraining.

[0362] Server: Using the feedback data, the machine learning model is updated again to improve the accuracy of the suggestions.

[0363] Specific examples

[0364] For example, a user purchases a new smart refrigerator and enters its details and their experience in the application. Furthermore, the emotion engine collects emotional data such as the user's happiness and satisfaction while using the refrigerator. Based on this data, the server performs the following actions:

[0365] 1. Server: Stores purchase data and emotion data in a database and performs preprocessing together with past data.

[0366] 2. Server: A machine learning model learns the user's preferences and emotional state and determines whether a newly released smart oven is suitable for the user.

[0367] 3. Server: Generates detailed information about the smart oven (price, features, vendor, best time to purchase) and sends it to the device.

[0368] 4. On the device: Notify the user of the offer and encourage them to purchase.

[0369] 5. User: Purchases the proposed smart oven and enters their experience using it into the terminal.

[0370] 6. Device: Sends usage information to the server and reflects it in the next learning session.

[0371] In this way, the system can make more accurate suggestions based on the user's emotional data, allowing the user to select the most appropriate product based on their emotional state.

[0372] The processing flow will be explained below.

[0373] Step 1:

[0374] User: When purchasing home appliances or interior items, the user accesses the application and enters their purchase history, including the product name, brand, price, purchase location, and purchase date.

[0375] Step 2:

[0376] Terminal: Purchase information is stored in a local database and sent to a server over the network.

[0377] Step 3:

[0378] Server: Receives purchase information sent from the device and stores it in a central database.

[0379] Step 4:

[0380] User: Enters information about the usage and impressions of the purchased product into the application, such as frequency of use, ease of use, and satisfaction level.

[0381] Step 5:

[0382] Terminal: Usage history and user feedback data is stored in a local database and sent to a server via the network.

[0383] Step 6:

[0384] Server: Receives usage history and feedback sent from the device and stores it in a central database.

[0385] Step 7:

[0386] On-device: Collects real-time emotional data from the user's facial expressions, tone of voice, and input text. Analyzes emotions using facial recognition and voice analysis algorithms.

[0387] Step 8:

[0388] Emotion engine: Analyzes collected emotion data to identify the user's emotional state, specifically identifying emotions such as joy, sadness, anger, and surprise.

[0389] Step 9:

[0390] Emotion engine: Sends the emotion analysis results to the server.

[0391] Step 10:

[0392] Server: Stores the purchase history, usage history, usage impressions, and emotion data sent from the device and emotion engine in a database.

[0393] Step 11:

[0394] Server: Preprocesses the received data to make it easier to analyze, specifically by handling missing values ​​and standardizing the data format.

[0395] Step 12:

[0396] Server: Extract user features (preferred brands, price range, features, emotional state, etc.) from the preprocessed data.

[0397] Step 13:

[0398] Server: Uses the features to update the machine learning model to learn about the user's style and preferences, incorporating sentiment data into the learning process.

[0399] Step 14:

[0400] Server: Based on the learning model, it selects product candidates that match the user's preferences, taking into account price, functionality, design, and emotional data.

[0401] Step 15:

[0402] Server: Generates detailed information about the selected product, including the product name, key features, price, where it is available, and the best time to purchase.

[0403] Step 16:

[0404] Server: Sends the selected products and their details to the user's device.

[0405] Step 17:

[0406] On the device, the user is notified of the suggestions received from the server via in-app notifications and push notifications.

[0407] Step 18:

[0408] User: Review the suggested products and purchase them if they are interested.

[0409] Step 19:

[0410] Users: Enter post-purchase feedback into the application, including experience and satisfaction.

[0411] Step 20:

[0412] Terminal: The feedback is stored in a local database and sent to a server over the network.

[0413] Step 21:

[0414] Server: Receives feedback sent from devices and stores it in a central database for future analysis and learning.

[0415] Step 22:

[0416] Server: Using the feedback data, the machine learning model is updated again to improve the accuracy of the suggestions.

[0417] In this way, the system can make more accurate suggestions based on the user's emotional data, allowing the user to select the most appropriate product based on their emotional state.

[0418] Example 2

[0419] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0420] Traditional personalized shopping assistant systems suggest products based on a user's purchase history, usage history, and usage impressions. However, these systems do not take into account the user's emotional state, making it difficult to make optimal product suggestions based on each individual user's emotions. Furthermore, the process by which users provide feedback on suggested products is not sufficiently considered, limiting the improvement of suggestion accuracy. This makes it difficult to make effective product suggestions that increase user satisfaction.

[0421] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0422] In this invention, the server includes means for receiving and storing a user's purchase history, usage history, usage impressions, and user emotional data in a database, machine learning means for learning user tendencies based on the data stored in the database, means for selecting appropriate products based on the user tendencies and emotional state learned by the machine learning means, and means for receiving the notified product feedback and emotional data and storing them in the database. This enables personalized product suggestions that take the user's emotional state into consideration, and further enables the accuracy of suggestions to be continuously improved based on feedback data from the user.

[0423] "Purchase history" refers to detailed information about products purchased by a user in the past, including product name, brand, price, purchase location, and purchase date.

[0424] "Usage history" refers to information about how a user uses a purchased product, including, for example, frequency of use, duration of use, and purpose of use.

[0425] "Usage impressions" refers to the impressions and evaluations that users have of the products they have purchased, including satisfaction, ease of use, and evaluation of design.

[0426] "Emotional data" refers to data that represents a user's emotional state, including facial expressions, tone of voice, and emotional information derived from input text.

[0427] "Means of storage" refers to the technology and methods used to record input data in a database.

[0428] "Machine learning methods" refer to algorithms that analyze data and learn patterns and trends based on the results, which can then predict user preferences and behavioral patterns.

[0429] "Means of selection" refers to the technology or method of selecting the optimal product from multiple options based on the results obtained through machine learning.

[0430] "Means of Notification" refers to the technology or method used to notify users of information about the selected product, including in-app notifications and push notifications.

[0431] "Means for receiving feedback" refers to the technology and methods for receiving product ratings and impressions provided by users.

[0432] "Preprocessing means" refers to techniques and methods for initial processing of collected data to make it easier to analyze. Specifically, this includes imputing missing values ​​and standardizing data formats.

[0433] "Means for extracting features" refers to techniques and methods for extracting important features and patterns from preprocessed data that can be used for analysis.

[0434] This invention is a personalized shopping assistant system that uses a generation AI to learn the user's style and preferences based on the user's purchase history, usage history, usage impressions, and emotional data for interior and home appliances, and then suggests the most suitable interior and home appliances.The invention uses a server, a terminal (user device), and an emotion engine.

[0435] System Overview

[0436] The system's main components are a server, a terminal, and an emotion engine. The server stores data, learns, selects products, and makes suggestions. The terminal is responsible for inputting data and receiving suggestions from users, while the emotion engine collects and analyzes users' emotion data and sends it to the server.

[0437] Data collection

[0438] 1. Enter your purchase history

[0439] Using the interface, users input detailed information about the home appliances and interior items they have purchased, such as the product name, brand, price, purchase location, and purchase date. The device stores this data in a local database and transmits it to a server via the network.

[0440] 2. Enter your usage history and impressions

[0441] Users periodically enter information about the usage, impressions, and satisfaction of purchased products, such as frequency of use, ease of use, and satisfaction level. The device stores this data in a local database and transmits it to the server.

[0442] 3. Collecting Emotional Data

[0443] The device collects real-time emotional data from the user's facial expressions, tone of voice, and input text. The emotion engine analyzes this data to identify the user's emotional state. Specifically, it identifies emotions such as joy, sadness, anger, and surprise. The analysis results are sent to the server.

[0444] Data storage and analysis

[0445] 1. Data storage

[0446] The server stores the purchase history, use history, use impressions, and emotion data transmitted from the terminal and the emotion engine in a database.

[0447] 2. Data preprocessing and feature extraction

[0448] The server preprocesses the data it receives to make it easier to analyze. Specifically, it complements missing data and standardizes data formats. From the preprocessed data, it extracts user features (preferred brands, price ranges, features, emotional state, etc.).

[0449] 3. Updating the machine learning model

[0450] The server analyzes the data using machine learning algorithms, incorporating user style and preferences into the learning model, with emotional data also being incorporated into the learning process.

[0451] Product selection and proposal

[0452] 1. Selecting the right product

[0453] The server uses the learning model to select product candidates that match the user's preferences. This selection process takes into account price, functionality, design, and emotional data.

[0454] 2. Generate and submit proposals

[0455] The server generates detailed information about the selected product, including the product name, key features, price, where it can be purchased, and the best time to purchase it. The server then sends this information to the user's device.

[0456] Receiving feedback and relearning

[0457] 1. Receiving Proposals

[0458] The device receives the suggestion from the server and notifies the user via in-app notifications or push notifications.

[0459] 2. Enter and submit your feedback

[0460] If the user checks the suggested product and purchases it, they input feedback such as their experience with it and their level of satisfaction. The device stores this feedback in a local database and sends it to the server.

[0461] 3. Relearn

[0462] The server stores the feedback data in a database and uses it as the basis for re-learning.The server then uses the feedback data to update the machine learning model again and improve the accuracy of the suggestions.

[0463] Specific examples

[0464] For example, if a user purchases a new smart refrigerator and enters its details and their experience in the application, the emotion engine will collect emotions such as joy and satisfaction during use. Based on this data, the server will:

[0465] 1. The server stores purchase data and emotion data in a database and performs preprocessing together with past data.

[0466] 2. The server uses a machine learning model to learn the user's preferences and emotional state and determines that a newly released smart oven is suitable for the user.

[0467] 3. The server generates detailed information about the smart oven (price, features, vendor, best time to purchase) and sends it to the device.

[0468] 4. The device notifies the user of the offer and encourages them to make a purchase.

[0469] 5. The user purchases the recommended smart oven and enters their experience using it into the terminal.

[0470] 6. The device sends the usage experience to the server and it is reflected in the next learning session.

[0471] In this way, the system can make more accurate suggestions based on the user's emotional data, allowing the user to select the most suitable product based on their emotional state.

[0472] Prompt Sentence Examples

[0473] "After you purchase your new smart refrigerator, please tell us your experience and satisfaction. We will use that information to recommend your next product."

[0474] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0475] The flow of this system's program processing

[0476] Step 1:

[0477] Input: The user enters detailed information about the home appliance or interior item they purchased (product name, brand, price, purchase location, purchase date, etc.) into the interface.

[0478] Processing: The device stores these purchase information in a local database and sends it to the server over the network.

[0479] Output: The server stores the received purchase data in a database.

[0480] Step 2:

[0481] Input: Users periodically input information about their usage and impressions of the products they have purchased (frequency of use, ease of use, satisfaction, etc.).

[0482] Processing: The device stores this usage history and usage impression data in a local database and sends it to the server.

[0483] Output: The server stores the received usage history and user feedback data in a database.

[0484] Step 3:

[0485] Input: Obtain emotional data such as facial expressions, tone of voice, and input text when users use home appliances and interior items.

[0486] Processing: The device collects these emotion data in real time and sends them to the emotion engine, which analyzes the collected emotion data and identifies the user's emotional state.

[0487] Output: The emotion engine sends the analysis results to the server, which stores them in a database.

[0488] Step 4:

[0489] Input: The server stores the purchase history, usage history, usage impressions, and emotion data received from the device and emotion engine in a database.

[0490] Processing: The server preprocesses the data and standardizes it into a format that is easy to analyze. Specifically, it complements missing data and reformats the data format.

[0491] Output: Save the preprocessed data to a database.

[0492] Step 5:

[0493] Input: Preprocessed purchase history, usage history, usage impressions, and sentiment data.

[0494] Processing: The server extracts features (preferred brand, price range, features, emotional state, etc.) from the preprocessed data.

[0495] Output: Extracted feature data.

[0496] Step 6:

[0497] Input: Feature data.

[0498] Processing: The server uses machine learning algorithms to learn about the user's style and preferences and update the model. Emotional data is also incorporated into the learning process.

[0499] Output: The updated training model.

[0500] Step 7:

[0501] Input: The updated training model.

[0502] Processing: The server uses the learning model to select product candidates that match the user's preferences. This selection process takes into account price, functionality, design, and emotional data.

[0503] Output: Selected product information.

[0504] Step 8:

[0505] Input: Selected product information.

[0506] Processing: The server generates detailed information about the selected product (product name, key features, price, where it is available, best time to buy, etc.).

[0507] Output: Detailed product information.

[0508] Step 9:

[0509] Input: Product details.

[0510] Processing: The server sends product information to the user's device. The device receives the offer from the server and notifies the user. Notification methods include in-app notifications and push notifications.

[0511] Output: Notification to the user.

[0512] Step 10:

[0513] Input: Notification from the server.

[0514] Processing: If the user checks the suggested product and purchases it, they enter feedback into the application, such as their experience and satisfaction with it. The device stores this feedback in a local database and sends it to the server.

[0515] Output: Feedback data.

[0516] Step 11:

[0517] Input: Feedback data.

[0518] Processing: The server saves the feedback data in a database and uses it as the basis for re-learning. The server then uses the newly input feedback data to update the machine learning model again, improving the accuracy of the suggestions.

[0519] Output: The updated training model.

[0520] (Application example 2)

[0521] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0522] Conventional shopping assistant systems typically suggest products based on a user's purchase and usage history of interior design and home appliances. However, this approach fails to consider the user's emotional state and does not necessarily suggest products that best fit the user's current needs and preferences. Furthermore, the accuracy of product selection is limited, creating a demand for more personalized suggestions. To address this, product suggestions that take the user's emotional state into account are necessary. Therefore, the present invention aims to provide more accurate personalized product suggestions by analyzing the user's emotional data and making product suggestions based on that information.

[0523] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for inputting a user's purchase history, usage history, and usage impressions; means for receiving and storing the purchase history, usage history, and usage impressions in a database; machine learning means for learning user tendencies based on the data stored in the database; means for selecting appropriate products based on the user tendencies learned by the machine learning means; means for collecting and analyzing emotional data from the user's facial expressions, tone of voice, and input text; an emotion analysis engine for identifying the user's emotional state based on the collected emotional data; means for incorporating the results of the emotion analysis engine into the machine learning means and suggesting products taking the user's emotional state into consideration; means for notifying the user of information about the selected products; and means for receiving and storing feedback on the notified products in the database. This enables highly accurate personalized product suggestions based on the user's emotional state.

[0524] "Purchase history" refers to information about products and services purchased by a user in the past.

[0525] "Usage history" is a detailed record of how a user uses purchased products and services.

[0526] "Usage impressions" refers to the impressions and evaluations that users have about the products or services they have purchased.

[0527] A "database" is a system for efficiently organizing, managing, and storing data within a computer system.

[0528] "Machine learning methods" are techniques that use computer programs to learn patterns from data and then make predictions or classifications based on that knowledge.

[0529] "Emotional data" is data that represents the user's emotional state, and is information collected from facial expressions, tone of voice, input text, etc.

[0530] An "emotion analysis engine" is software or a system for analyzing collected emotional data and identifying a user's emotional state.

[0531] "Product suggestion" is the act of selecting the most suitable product based on user data and recommending it to the user.

[0532] "Feedback" refers to the evaluations and impressions that users give about the products and services provided.

[0533] "Notification means" refers to a method or device for conveying information about the selected product to the user.

[0534] "Preprocessing" refers to the initial processing used to convert data into a form that is easier to analyze.

[0535] A "feature" is an important element in the data input into a machine learning model for making predictions and classifications.

[0536] "Style" refers to the design and functional characteristics that users choose according to their particular trends and preferences.

[0537] "Push notifications" are a technology that allows you to communicate information to users in real time.

[0538] This invention is a personalized shopping assistant system that analyzes emotional data based on a user's purchase history, usage history, and usage impressions, and suggests optimal products to the user. This system is primarily composed of a server, a terminal (user device), and an emotion analysis engine.

[0539] System Configuration

[0540] Terminal

[0541] A terminal is a user device, such as a smartphone, that uses the following hardware and software:

[0542] Smartphone (using camera and microphone)

[0543] SQLite (as a local database)

[0544] Flutter (registered trademark) (mobile application framework)

[0545] Emotion API (for emotion recognition)

[0546] server

[0547] The server processes and learns from the data using the following software:

[0548] Flask (backend framework)

[0549] TensorFlow (for machine learning algorithms)

[0550] Sentiment Analysis Engine

[0551] The sentiment analysis engine uses the Emotion API to analyze the collected emotional data.

[0552] Data processing and proposal flow

[0553] Data collection

[0554] Users enter detailed information about the home appliances and interior items they have purchased into the device, such as the product name, brand, price, purchase location, and purchase date. They also enter their usage history and impressions. The device stores this data in a local database and periodically sends it to the server.

[0555] In addition, the device's built-in camera and microphone are used to capture the user's facial expressions and tone of voice in real time, and emotional data is collected through the Emotion API.

[0556] Data storage and preprocessing

[0557] The server receives data sent from the device using Flask and stores it in a database. It also preprocesses the data to handle missing values ​​and standardize the format.

[0558] Data Analysis and Learning

[0559] The server uses TensorFlow to periodically update the machine learning model and select product candidates based on the user's preferences and emotional state. For example, the following prompt sentence is used:

[0560] Prompt Sentence Examples

[0561] def get_personalized_recommendations(user_data, emotion_data):

[0562] """

[0563] Personalized product recommendations based on user and sentiment data

[0564] """

[0565] from recommendation_engine import RecommendationEngine

[0566] from emotion_analysis import EmotionEngine

[0567] Analyze user and sentiment data

[0568] user_preferences = RecommendationEngine.analyze_user_data(user_data)

[0569] user_emotion = EmotionEngine.analyze_emotion_data(emotion_data)

[0570] Select the best product candidates for the user

[0571] recommended_items = RecommendationEngine.get_recommendations(user_preferences, user_emotion)

[0572] return recommended_items

[0573] For example, it suggests smart ovens based on user preferences and emotions.

[0574] user_data = {

[0575] 'purchase_history': [{'item': 'smart fridge', 'brand': 'BrandA', 'price': 80000}],

[0576] 'usage_feedback': [{'item': 'smart fridge', 'satisfaction': 5}]

[0577] }

[0578] emotion_data = {'joy': 0.8, 'satisfaction': 0.9}

[0579] recommendations = get_personalized_recommendations(user_data, emotion_data)

[0580] print(recommendations)

[0581] Product suggestions and feedback

[0582] The server generates detailed information about the selected product and sends it to the device. The device then uses push notifications to suggest the product to the user and displays the product detail page. The user then purchases the product and enters their usage experience and impressions into the app again. This allows the system to relearn the user's feedback and improve the accuracy of its next suggestions.

[0583] This invention enables highly accurate personalized product suggestions that correspond to the user's emotional state, allowing the user to have a more satisfying shopping experience.

[0584] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0585] Step 1:

[0586] Users enter detailed information about the home appliances and interior items they have purchased into the device. Specifically, they enter data such as the product name, brand, price, purchase location, and purchase date. The entered data is saved in the device's SQLite database, which records the user's purchase history.

[0587] Step 2:

[0588] Users enter their usage history and impressions of the purchased product into the terminal. For example, they enter data such as frequency of use, ease of use, and satisfaction level. This data is also stored in an SQLite database. This records the user's usage history and impressions.

[0589] Step 3:

[0590] The device uses a built-in camera and microphone to capture the user's facial expressions and tone of voice in real time. The captured data is analyzed using the Emotion API to generate emotion data, which records the user's emotional state.

[0591] Step 4:

[0592] The device transmits purchase history, usage history, usage impressions, and emotion data to the server via the network. This data is received by the server-side Flask backend framework and stored in a database.

[0593] Step 5:

[0594] The server preprocesses the received data. Specifically, it processes missing values, standardizes the format, and converts the data into a format that is easy to analyze. This preprocessed data is used to extract user features.

[0595] Step 6:

[0596] The server updates the machine learning model using TensorFlow to select product candidates based on the user's preferences and emotional state. Using the preprocessed data and features as input, the machine learning model learns the user's tendencies and emotional state and generates optimal product candidates.

[0597] Step 7:

[0598] The server generates detailed information about the selected product, including the product name, main features, price, where it can be purchased, and the best time to purchase it. The generated information is sent to the device.

[0599] Step 8:

[0600] The device notifies the user of the received product suggestions, and the user can receive a push notification or an in-app notification to view the details page of the suggested product.

[0601] Step 9:

[0602] The user checks the suggested products and, if they are interested, purchases them. After purchasing, they again enter their impressions and feedback into the device. This feedback data is also stored in the SQLite database.

[0603] Step 10:

[0604] The device sends feedback data to the server, which receives the feedback data and stores it in a database. The stored feedback data is used to retrain the machine learning model and improve the accuracy of the next suggestion.

[0605] This allows for highly accurate product suggestions based on the user's emotional state, allowing the user to have a more satisfying shopping experience.

[0606] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0607] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0608] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0609] [Second embodiment]

[0610] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0611] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0612] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0613] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0614] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0615] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0616] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0617] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0618] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0619] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0620] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0621] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0622] This invention is a personalized shopping assistant system that uses AI to learn the user's style and preferences based on their purchase and usage history and impressions of interior and home appliances, and then suggests the most suitable interior and home appliances.

[0623] System Overview

[0624] This system mainly consists of a server and a terminal (user device). The server stores data, learns, selects products, and makes suggestions, while the terminal is responsible for inputting data and receiving suggestions.

[0625] Data collection

[0626] Enter your purchase history

[0627] User: Through the interface, enter detailed information about the home appliance or interior item purchased (product name, brand, price, purchase location, purchase date).

[0628] Device: Purchase information is stored locally and sent to the server over the network.

[0629] Entering usage history and impressions

[0630] Users: Regularly enter usage, feedback, and satisfaction with purchased products.

[0631] Device: Usage history and user feedback data is stored locally and sent to the server.

[0632] Data storage and analysis

[0633] Receiving and storing data

[0634] Server: Stores purchase history, usage history, and user feedback sent from user devices in a database.

[0635] Data preprocessing and feature extraction

[0636] Server: Preprocesses the received data and extracts user features (preferred brands, price range, features, etc.).

[0637] Machine learning model updates

[0638] Server: Periodically analyzes data using machine learning algorithms to adapt the learning model to reflect user style and preferences.

[0639] Product selection and proposal

[0640] Selecting the right product

[0641] Server: Based on the learning model, the server selects products that match the user's preferences. This selection process also takes into account price, functionality, design, and vendor information.

[0642] Proposal generation and submission

[0643] Server: Generates detailed information about the selected product (product name, key features, price, where it can be purchased, and the best time to purchase).

[0644] Server: Sends the proposals to the user's device.

[0645] Receiving suggestions and feedback

[0646] Receiving Proposals

[0647] Device: Receives suggestions from the server and notifies the user. Notification methods include in-app notifications and push notifications.

[0648] Enter and submit feedback

[0649] User: Purchases the suggested product and enters their impressions and ratings.

[0650] Device: Feedback is stored locally and periodically sent to the server.

[0651] Server: The feedback is stored in a database and used as the basis for retraining.

[0652] Specific examples

[0653] For example, if a user buys a new smart refrigerator and enters details and user experience into the application, the server can:

[0654] 1. Server: Stores purchase data in a database and performs preprocessing together with past data.

[0655] 2. Server: A machine learning model learns user preferences and determines whether a newly released smart oven is suitable for the user.

[0656] 3. Server: Generates detailed information about the smart oven (price, features, vendor, best time to purchase) and sends it to the device.

[0657] 4. On the device: Notify the user of the offer and encourage them to purchase.

[0658] 5. User: Purchases the proposed smart oven and enters their experience using it into the terminal.

[0659] 6. Device: Sends usage information to the server and reflects it in the next learning session.

[0660] In this way, the system continuously learns user trends and continues to suggest appropriate products.

[0661] The processing flow will be explained below.

[0662] Step 1:

[0663] User: When purchasing home appliances or interior items, the user accesses the application and enters their purchase history, including the product name, brand, price, purchase location, and purchase date.

[0664] Step 2:

[0665] Terminal: Purchase information is stored in a local database and sent to a server over the network.

[0666] Step 3:

[0667] Server: Receives purchase information sent from the device and stores it in a central database.

[0668] Step 4:

[0669] User: Enters information about the usage and impressions of the purchased product into the application, such as frequency of use, ease of use, and satisfaction level.

[0670] Step 5:

[0671] Terminal: Usage history and user feedback data is stored in a local database and sent to a server via the network.

[0672] Step 6:

[0673] Server: Receives usage history and feedback sent from the device and stores it in a central database.

[0674] Step 7:

[0675] Server: Preprocesses the purchase history, usage history, and usage impressions stored in the database to make them easier to analyze. Specifically, it handles missing values ​​and standardizes the format of the data.

[0676] Step 8:

[0677] Server: Extracts features from the preprocessed data to identify user preferences and trends, such as preferred brands, price ranges, and features.

[0678] Step 9:

[0679] Server: Uses the features to update the machine learning model and learn about the user's style and preferences.

[0680] Step 10:

[0681] Server: Using a learning model, the server selects product candidates that match the user's preferences, taking into account price, functionality, design, and other factors.

[0682] Step 11:

[0683] Server: Generates detailed information about the selected product, including the product name, key features, price, where it is available, and the best time to purchase.

[0684] Step 12:

[0685] Server: Sends the selected products and their details to the user's device.

[0686] Step 13:

[0687] On the device, the user is notified of the suggestions received from the server via in-app notifications and push notifications.

[0688] Step 14:

[0689] User: Review the suggested products and purchase them if they are interested.

[0690] Step 15:

[0691] Users: Enter post-purchase feedback into the application, including experience and satisfaction.

[0692] Step 16:

[0693] Terminal: The feedback is stored in a local database and sent to a server over the network.

[0694] Step 17:

[0695] Server: Receives feedback sent from devices and stores it in a central database for future analysis and learning.

[0696] Step 18:

[0697] Server: Using the feedback data, the machine learning model is updated again to improve the accuracy of the suggestions.

[0698] Example 1

[0699] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0700] Conventional shopping assistant systems have had the challenge of making recommendations that fully reflect a user's individual preferences and past purchase history. It is also difficult to absorb user feedback in real time and use it in future recommendations. Furthermore, they are unable to provide sufficient recommendations on the best time and place to purchase, limiting the user experience.

[0701] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0702] In this invention, the server includes means for inputting a user's purchase history, usage history, and usage impressions, means for receiving and storing the purchase history, usage history, and usage impressions in a database, means for preprocessing the data stored in the database, means for extracting features from the preprocessed data, machine learning means for learning user tendencies based on the data stored in the database and the extracted features, means for selecting appropriate products based on the user tendencies learned by the machine learning means, means for sending and notifying the user of information about the selected products to the user's terminal, and means for receiving and storing feedback on the notified products in the database. This enables personalized suggestions that reflect the user's individual preferences and past purchase history.

[0703] "Users" are individuals or corporations who use this system to purchase interior goods and home appliances, manage their usage, etc.

[0704] "Purchase history" refers to data on detailed information about interior and home appliance products purchased by a user (product name, brand, price, purchase location, purchase date, etc.).

[0705] "Usage history" refers to data about how users use interior and home appliances they have purchased.

[0706] "Usage impressions" is data that represents the evaluations and opinions that users have about the interior design and home appliances they have used.

[0707] "Means" are the specific methods or tools used to realize the system.

[0708] A "database" is a storage device for centrally managing and storing various data such as purchase history, usage history, and usage impressions.

[0709] "Preprocessing" is the process of correcting or removing inconsistencies or missing data and preparing the data for analysis before data analysis.

[0710] "Features" refers to the numerical representation of user preferences and behavioral tendencies extracted from a database, which are used as input data in machine learning.

[0711] "Machine learning" is an algorithm that allows a computer to learn from past data and make predictions and decisions based on future data.

[0712] "Selection" is the process of using a machine learning model to determine the product candidates that best suit the user's preferences.

[0713] "Terminal" means the device (e.g., smartphone, tablet, PC) that a user uses to interact with the system.

[0714] "Notification" refers to the act of transmitting information from a server to a user, and includes in-app notifications and push notifications.

[0715] "Feedback" refers to information about users' impressions and evaluations of the proposed product after using it.

[0716] This invention is a personalized shopping assistant system in which a generative AI learns a user's style and preferences based on their purchase history, usage history, and impressions of interior and home appliances, and suggests the most suitable interior and home appliances. This system is mainly composed of a server and a terminal (user device).

[0717] composition

[0718] server

[0719] The server serves as a central location for receiving, storing, pre-processing, and machine learning data. It has the following main functions:

[0720] Purchase history, usage history, and usage feedback are received and stored in a database.

[0721] Preprocess the received data and extract features.

[0722] Based on the features, machine learning algorithms (e.g., random forests, neural networks) are used to learn user tendencies.

[0723] Based on the learning results, the system selects the most suitable product for the user and generates detailed information about it.

[0724] The selected product information is sent to the user's device.

[0725] Terminal

[0726] A terminal is a device that a user uses to enter data and receive notifications from a server. Its main functions are:

[0727] Provides an interface for inputting purchase history, usage history, and usage impressions.

[0728] The user is notified of the proposal information received from the server.

[0729] Data processing and calculation

[0730] Data collection

[0731] User: Using a dedicated app or web portal on a smartphone or PC, enter your purchase history (product name, brand, price, purchase location, purchase date). Example: Value Range Refrigerator, AB manufacturer, 50,000 yen, XYZ store, October 1, 2023.

[0732] Terminal: The information entered by the user is temporarily stored in a local database and sent to a server via the Internet.

[0733] User: Enter usage status, impressions, and satisfaction level of the purchased product. Example: Good cooling performance, excellent design.

[0734] Data storage and analysis

[0735] Server: Stores the received purchase history, usage history, and user feedback in a database.

[0736] Server: Preprocesses the received data, corrects missing data, and removes inconsistent data. Then, extracts features based on user preferences and behavioral trends.

[0737] Server: Uses machine learning algorithms based on features to learn user trends.

[0738] Product selection and proposal

[0739] Server: Selects the best product for the user based on learned trends, taking into account information such as price, functionality, design, and vendor.

[0740] Server: Generates detailed information about the selected product (product name, key features, price, where it can be purchased, and the best time to purchase) and sends it to the user's device.

[0741] On the device: Notify the user of the received recommendation. Example: Display an in-app notification saying, "Our new smart oven is perfect for you!"

[0742] Enter and submit feedback

[0743] User: Purchases the suggested product and enters their impressions and ratings into the device.

[0744] Device: Stores feedback data locally and sends it to the server.

[0745] Server: Receives feedback and stores it in a database for future machine learning.

[0746] Specific examples

[0747] Prompt Sentence Examples

[0748] "Please tell us the details of the home appliance you recently purchased (product name, brand, price, place of purchase, purchase date)."

[0749] "Please tell us about your experience using and satisfaction with the home appliances you purchased."

[0750] "Tell me about the next interior or home appliance you're considering purchasing."

[0751] These prompts can be used to gather the necessary data from the user and provide optimal product recommendations.

[0752] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0753] Step 1:

[0754] Input: User enters purchase history through application or web portal. For example: product name, brand, price, purchase location, purchase date.

[0755] How it works: Users use the interface on their smartphone or computer to enter detailed information about the home appliances and interior items they have purchased.

[0756] Output: Purchase information is stored on the device and data is generated that is later sent to a server.

[0757] Step 2:

[0758] Input: Purchase history data and details entered by the user.

[0759] How it works: The device temporarily stores the purchase information in a local database and then sends it over the internet to a server, periodically retrying the transmission until it is successful.

[0760] Output: Purchase history data sent to the server.

[0761] Step 3:

[0762] Input: Purchase history data received by the server.

[0763] How it works: The server receives the purchase history sent from the device and stores it in a database.

[0764] Output: Purchase history stored in a database.

[0765] Step 4:

[0766] Input: User-entered usage history data such as usage status, impressions, and satisfaction.

[0767] How it works: Users periodically enter their impressions and satisfaction with the device. Examples include "This refrigerator has great cooling performance" and "It has a nice design."

[0768] Output: Usage history and impressions are stored on the device and data is generated that is later sent to the server.

[0769] Step 5:

[0770] Input: Usage history and user feedback data.

[0771] How it works: The device stores this data locally and sends it to the server.

[0772] Output: Usage history and user feedback data sent to the server.

[0773] Step 6:

[0774] Input: Usage history and user feedback data received by the server.

[0775] Operation: The server receives the usage history and usage impressions sent from the device and stores them in a database.

[0776] Output: Usage history and user feedback data stored in a database.

[0777] Step 7:

[0778] Input: Purchase history, usage history, and user feedback data stored in the database.

[0779] How it works: The server preprocesses the received data, correcting or removing inconsistencies and missing data, and then extracts features that quantify user preferences and behavioral trends.

[0780] Output: Preprocessed data and extracted features.

[0781] Step 8:

[0782] Input: Preprocessed data and extracted features.

[0783] How it works: Using machine learning algorithms based on features, it learns user tendencies.

[0784] Output: An updated machine learning model.

[0785] Step 9:

[0786] Input: The machine learning model obtained by training.

[0787] How it works: The server uses a learning model to recommend products that match the user's preferences. Example: "The new smart oven is perfect for you."

[0788] Output: Suitable products and their details (price, key features, where available, best time to buy, etc.).

[0789] Step 10:

[0790] Input: Detailed information about the selected product.

[0791] How it works: The server generates detailed information about the selected product and sends it to the user's device.

[0792] Output: The suggestions sent to the user's device.

[0793] Step 11:

[0794] Input: Proposal notification.

[0795] How it works: The device receives the offer from the server and notifies the user. Notification methods include in-app notifications and push notifications. Example: "Our new smart oven is perfect for you!"

[0796] Output: A notification showing the suggestion.

[0797] Step 12:

[0798] Input: User's impressions and ratings of the proposed product.

[0799] How it works: The user uses the suggested product and enters their impressions and ratings.

[0800] Output: Feedback data is generated and saved to the device.

[0801] Step 13:

[0802] Input: Feedback data.

[0803] How it works: The device stores the feedback data locally and sends it to the server.

[0804] Output: Feedback data sent to the server.

[0805] Step 14:

[0806] Input: The feedback data received by the server.

[0807] How it works: The server receives the feedback data and stores it in a database. It is used for retraining.

[0808] Output: Feedback data stored in a database.

[0809] (Application example 1)

[0810] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0811] Conventional personalized shopping assistant systems have had problems such as limited product suggestions based on users' purchase history and usage impressions, being available only in specific locations, or having poor visual suggestions. Furthermore, there was a lack of means for users to check the experience of the suggested products in advance, and there were limitations on providing information on the optimal time and place to purchase, which could lead to a decrease in user satisfaction.

[0812] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0813] In this invention, the server includes a means for inputting a user's purchase history, usage history, and usage impressions, a means for receiving the purchase history, usage history, and usage impressions and storing them in a database, and a means for preprocessing the data stored in the database and extracting features. This makes it possible to learn user trends, suggest optimal products, visually display them in a virtual store, and suggest where they can be purchased and the optimal time to purchase them.

[0814] "User" means any individual or legal entity using a particular product or service.

[0815] "Purchase history" refers to information about products and services a user has purchased in the past.

[0816] "Usage history" refers to data about how a user uses a purchased product or service.

[0817] "Usage impressions" refers to the impressions and evaluations that users have after using a product or service.

[0818] "Input means" refers to the interface or function that allows users to input data into the system.

[0819] A "database" refers to a system that systematically stores information and makes it available for searching and use as needed.

[0820] "Preprocessing" refers to a series of steps that convert collected data into an analyzable format.

[0821] "Features" refer to useful attributes or parameters extracted from data.

[0822] "Machine learning methods" refer to techniques that allow computers to learn from data and recognize patterns.

[0823] "Goods" means any products or services intended for purchase or use by Users.

[0824] A "virtual store" refers to an online shopping environment created using virtual reality technology.

[0825] "Visual presentation means" refers to techniques or methods for presenting information graphically to a user.

[0826] "Feedback" refers to the ratings and opinions users provide about products and services.

[0827] The "optimal time to purchase" refers to the time period that encourages users to make a purchase at the most advantageous time.

[0828] This invention is a personalized shopping assistant system that uses AI to learn the user's style and preferences based on their purchase and usage history and impressions of interior and home appliances, and then suggests the most suitable products. The system is mainly composed of a server and a terminal (user device).

[0829] System Overview

[0830] Data collection

[0831] Users enter their purchase history and usage impressions via the terminal. They enter detailed information about the purchased product, usage status, impressions, and satisfaction level, and the terminal collects this information. The collected data is sent to the server via the network.

[0832] Data storage and analysis

[0833] The server stores the received purchase history, usage history, and usage impressions in a database. The stored data is preprocessed to extract user features (preferred brands, price ranges, features, etc.). The features are analyzed using a machine learning model to learn the user's style and preferences.

[0834] Product selection and proposal

[0835] The server selects products that match the user's preferences based on the learning model. This selection process also takes into account price, functionality, design, and vendor information. The selected product information is visually displayed in a virtual store.

[0836] Program processing

[0837] 1. Data Acquisition and Transmission

[0838] The device (such as the user's VR headset or smart glasses) collects purchase history and usage impressions and sends them to the server. The hardware used includes Oculus as a VR headset and Google Glass as smart glasses.

[0839] 2. Data analysis and proposal generation

[0840] The server uses Python's Flask framework to analyze the received data, and uses machine learning models such as TensorFlow to learn the user's preferences and select appropriate products. This selected product information is stored in a database and then sent back to the device.

[0841] 3. Virtual store display

[0842] The device uses software such as Unity or Unreal Engine to visually display product information in a virtual environment, allowing users to move around the virtual store and examine the suggested products in detail.

[0843] Specific examples

[0844] For example, if a user purchases a new smart refrigerator and enters details and user experience into an application, the flow would be as follows:

[0845] Users enter information about the smart refrigerator they have purchased and their experience using it into the application.

[0846] The terminal collects this data and sends it to the server.

[0847] The server receives and stores the data and uses machine learning models to learn user preferences.

[0848] The server then selects a new smart oven that is suitable for the user and generates detailed information about it.

[0849] The terminal visually presents the proposed smart oven to the user in a virtual reality environment.

[0850] Users can view suggested smart ovens in a virtual store and consider purchasing them.

[0851] Prompt Sentence Examples

[0852] user_id: "12345"

[0853] purchased_items:

[0854] item: "Smart Refrigerator"

[0855] brand: "BrandA"

[0856] price: 50000

[0857] purchase_date: "2023-08-01"

[0858] item: "Washing machine"

[0859] brand: "BrandB"

[0860] price: 30000

[0861] purchase_date: "2023-01-15"

[0862] user_feedback:

[0863] item: "Smart Refrigerator"

[0864] Rating: 5

[0865] comment: "Very useful"

[0866] item: "Washing machine"

[0867] Rating: 4

[0868] comment: "Very satisfied"

[0869] In this way, the system suggests optimal products based on the user's purchase history and usage impressions, enabling a visual experience in a virtual store.

[0870] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0871] Step 1:

[0872] Users enter their purchase history and usage experience, including details such as product name, brand, price, purchase location, and purchase date. This data is collected and stored locally on the device (e.g., smart glasses or VR headset). The input data format is as follows:

[0873] Input: Product name, brand, price, purchase location, purchase date

[0874] Output: Locally stored purchase and usage data

[0875] Step 2:

[0876] The device sends the collected purchase history and usage feedback to a server via the network, using the Oculus SDK and Google Glass API as interfaces.

[0877] Input: Locally stored purchase and usage data

[0878] Output: Data sent to the server

[0879] Step 3:

[0880] The server stores the received purchase history, usage history, and usage feedback in a database. After storing this data, it is preprocessed for analysis.

[0881] Input: Submitted purchase history, usage history, and user impressions

[0882] Output: Data stored in the database

[0883] Step 4:

[0884] The server preprocesses the data and extracts user features, including cleaning, normalizing, handling missing values, and performing feature engineering.

[0885] Input: Raw data stored in a database

[0886] Output: Extracted features

[0887] Step 5:

[0888] The server inputs the features into a machine learning model (e.g., TensorFlow) to learn the user's style and preferences. The model recognizes and learns patterns based on the user's past data.

[0889] Input: Extracted features

[0890] Output: A learned model containing user preference patterns

[0891] Step 6:

[0892] The server then uses the learning model to select products that suit the user's preferences, taking into account product price, functionality, design, and vendor information.

[0893] Input: User preference patterns

[0894] Output: A list of recommended products

[0895] Step 7:

[0896] The server generates detailed information about the selected product (product name, main features, price, distributor, best time to purchase) and sends it to the terminal.

[0897] Input: A list of recommended products

[0898] Output: Proposal to the device

[0899] Step 8:

[0900] The device visually displays the received recommendations in a virtual store. Users can view the suggested products in the virtual space using a VR headset or smart glasses. The system is displayed using Unity or Unreal Engine.

[0901] Input: Proposal

[0902] Output: Visual representation of the virtual store

[0903] Step 9:

[0904] Users can view the proposed products in the virtual store and provide feedback, which is collected at the terminal and sent to the server.

[0905] Input: User feedback

[0906] Output: Feedback data to the server

[0907] Step 10:

[0908] The server stores the received feedback in a database and uses it to update the machine learning model, thereby improving the accuracy of the suggestions.

[0909] Input: User feedback data

[0910] Output: Updated training model

[0911] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0912] This is a personalized shopping assistant system that uses a generation AI to learn the user's style and preferences based on their purchase and usage history, impressions, and emotional data on interior and home appliances, and then suggests the most suitable interior and home appliances. In particular, by combining it with an emotion engine, it is possible to suggest products that take into account the user's emotional state, providing a more personalized service.

[0913] System Overview

[0914] This system is mainly composed of a server, a terminal (user device), and an emotion engine. The server stores data, learns, selects products, and makes recommendations, while the terminal is responsible for inputting data and receiving recommendations. The emotion engine collects and analyzes user emotion data and sends it to the server.

[0915] Data collection

[0916] Enter your purchase history

[0917] User: Through the interface, the user enters detailed information about the home appliance or interior item they purchased, such as the product name, brand, price, purchase location, and purchase date.

[0918] Terminal: Purchase information is stored in a local database and sent to a server over the network.

[0919] Entering usage history and impressions

[0920] Users: Periodically enter information about the usage, impressions, and satisfaction of purchased products, such as frequency of use, ease of use, and satisfaction level.

[0921] Device: Usage history and user feedback data is stored in a local database and sent to the server.

[0922] Collecting Emotional Data

[0923] On-device: Collects real-time emotional data from the user's facial expressions, tone of voice, and input text. This data is analyzed through an emotion engine.

[0924] Emotion engine: Analyzes collected emotion data to identify the user's emotional state, specifically identifying emotions such as joy, sadness, anger, and surprise.

[0925] Emotion engine: Sends the emotion analysis results to the server.

[0926] Data storage and analysis

[0927] Receiving and storing data

[0928] Server: Stores the purchase history, usage history, usage impressions, and emotion data sent from the device and emotion engine in a database.

[0929] Data preprocessing and feature extraction

[0930] Server: Preprocesses the received data to make it easier to analyze. Specifically, it handles missing values ​​and standardizes the format of the data. Emotion data is also preprocessed.

[0931] Server: Extract user features (preferred brands, price range, features, emotional state, etc.) from the preprocessed data.

[0932] Machine learning model updates

[0933] Server: Periodically analyzes the data using machine learning algorithms to adapt the learning model to reflect user style and preferences. Emotional data is also incorporated into the learning process.

[0934] Product selection and proposal

[0935] Selecting the right product

[0936] Server: Based on the learning model, the server selects product candidates that match the user's preferences. This selection process takes into account price, functionality, design, and emotional data.

[0937] Proposal generation and submission

[0938] Server: Generates detailed information about the selected product, including the product name, key features, price, where it is available, and the best time to purchase.

[0939] Server: Sends the selected products and their details to the user's device.

[0940] Receiving suggestions and feedback

[0941] Receiving Proposals

[0942] Device: Receives suggestions from the server and notifies the user via in-app notifications and push notifications.

[0943] Enter and submit feedback

[0944] User: Review the suggested products, purchase them if they are interested, and enter feedback into the application after purchase, including their experience and satisfaction.

[0945] Terminal: The feedback is stored in a local database and sent to a server over the network.

[0946] Server: The feedback sent from the device is stored in a database and used as the basis for retraining.

[0947] Server: Using the feedback data, the machine learning model is updated again to improve the accuracy of the suggestions.

[0948] Specific examples

[0949] For example, a user purchases a new smart refrigerator and enters its details and their experience in the application. Furthermore, the emotion engine collects emotional data such as the user's happiness and satisfaction while using the refrigerator. Based on this data, the server performs the following actions:

[0950] 1. Server: Stores purchase data and emotion data in a database and performs preprocessing together with past data.

[0951] 2. Server: A machine learning model learns the user's preferences and emotional state and determines whether a newly released smart oven is suitable for the user.

[0952] 3. Server: Generates detailed information about the smart oven (price, features, vendor, best time to purchase) and sends it to the device.

[0953] 4. On the device: Notify the user of the offer and encourage them to purchase.

[0954] 5. User: Purchases the proposed smart oven and enters their experience using it into the terminal.

[0955] 6. Device: Sends usage information to the server and reflects it in the next learning session.

[0956] In this way, the system can make more accurate suggestions based on the user's emotional data, allowing the user to select the most appropriate product based on their emotional state.

[0957] The processing flow will be explained below.

[0958] Step 1:

[0959] User: When purchasing home appliances or interior items, the user accesses the application and enters their purchase history, including the product name, brand, price, purchase location, and purchase date.

[0960] Step 2:

[0961] Terminal: Purchase information is stored in a local database and sent to a server over the network.

[0962] Step 3:

[0963] Server: Receives purchase information sent from the device and stores it in a central database.

[0964] Step 4:

[0965] User: Enters information about the usage and impressions of the purchased product into the application, such as frequency of use, ease of use, and satisfaction level.

[0966] Step 5:

[0967] Terminal: Usage history and user feedback data is stored in a local database and sent to a server via the network.

[0968] Step 6:

[0969] Server: Receives usage history and feedback sent from the device and stores it in a central database.

[0970] Step 7:

[0971] On-device: Collects real-time emotional data from the user's facial expressions, tone of voice, and input text. Analyzes emotions using facial recognition and voice analysis algorithms.

[0972] Step 8:

[0973] Emotion engine: Analyzes collected emotion data to identify the user's emotional state, specifically identifying emotions such as joy, sadness, anger, and surprise.

[0974] Step 9:

[0975] Emotion engine: Sends the emotion analysis results to the server.

[0976] Step 10:

[0977] Server: Stores the purchase history, usage history, usage impressions, and emotion data sent from the device and emotion engine in a database.

[0978] Step 11:

[0979] Server: Preprocesses the received data to make it easier to analyze, specifically by handling missing values ​​and standardizing the data format.

[0980] Step 12:

[0981] Server: Extract user features (preferred brands, price range, features, emotional state, etc.) from the preprocessed data.

[0982] Step 13:

[0983] Server: Uses the features to update the machine learning model to learn about the user's style and preferences, incorporating sentiment data into the learning process.

[0984] Step 14:

[0985] Server: Based on the learning model, it selects product candidates that match the user's preferences, taking into account price, functionality, design, and emotional data.

[0986] Step 15:

[0987] Server: Generates detailed information about the selected product, including the product name, key features, price, where it is available, and the best time to purchase.

[0988] Step 16:

[0989] Server: Sends the selected products and their details to the user's device.

[0990] Step 17:

[0991] On the device, the user is notified of the suggestions received from the server via in-app notifications and push notifications.

[0992] Step 18:

[0993] User: Review the suggested products and purchase them if they are interested.

[0994] Step 19:

[0995] Users: Enter post-purchase feedback into the application, including experience and satisfaction.

[0996] Step 20:

[0997] Terminal: The feedback is stored in a local database and sent to a server over the network.

[0998] Step 21:

[0999] Server: Receives feedback sent from devices and stores it in a central database for future analysis and learning.

[1000] Step 22:

[1001] Server: Using the feedback data, the machine learning model is updated again to improve the accuracy of the suggestions.

[1002] In this way, the system can make more accurate suggestions based on the user's emotional data, allowing the user to select the most appropriate product based on their emotional state.

[1003] Example 2

[1004] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1005] Traditional personalized shopping assistant systems suggest products based on a user's purchase history, usage history, and usage impressions. However, these systems do not take into account the user's emotional state, making it difficult to make optimal product suggestions based on each individual user's emotions. Furthermore, the process by which users provide feedback on suggested products is not sufficiently considered, limiting the improvement of suggestion accuracy. This makes it difficult to make effective product suggestions that increase user satisfaction.

[1006] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1007] In this invention, the server includes means for receiving and storing a user's purchase history, usage history, usage impressions, and user emotional data in a database, machine learning means for learning user tendencies based on the data stored in the database, means for selecting appropriate products based on the user tendencies and emotional state learned by the machine learning means, and means for receiving the notified product feedback and emotional data and storing them in the database. This enables personalized product suggestions that take the user's emotional state into consideration, and further enables the accuracy of suggestions to be continuously improved based on feedback data from the user.

[1008] "Purchase history" refers to detailed information about products purchased by a user in the past, including product name, brand, price, purchase location, and purchase date.

[1009] "Usage history" refers to information about how a user uses a purchased product, including, for example, frequency of use, duration of use, and purpose of use.

[1010] "Usage impressions" refers to the impressions and evaluations that users have of the products they have purchased, including satisfaction, ease of use, and evaluation of design.

[1011] "Emotional data" refers to data that represents a user's emotional state, including facial expressions, tone of voice, and emotional information derived from input text.

[1012] "Means of storage" refers to the technology and methods used to record input data in a database.

[1013] "Machine learning methods" refer to algorithms that analyze data and learn patterns and trends based on the results, which can then predict user preferences and behavioral patterns.

[1014] "Means of selection" refers to the technology or method of selecting the optimal product from multiple options based on the results obtained through machine learning.

[1015] "Means of Notification" refers to the technology or method used to notify users of information about the selected product, including in-app notifications and push notifications.

[1016] "Means for receiving feedback" refers to the technology and methods for receiving product ratings and impressions provided by users.

[1017] "Preprocessing means" refers to techniques and methods for initial processing of collected data to make it easier to analyze. Specifically, this includes imputing missing values ​​and standardizing data formats.

[1018] "Means for extracting features" refers to techniques and methods for extracting important features and patterns from preprocessed data that can be used for analysis.

[1019] This invention is a personalized shopping assistant system that uses a generation AI to learn the user's style and preferences based on the user's purchase history, usage history, usage impressions, and emotional data for interior and home appliances, and then suggests the most suitable interior and home appliances.The invention uses a server, a terminal (user device), and an emotion engine.

[1020] System Overview

[1021] The system's main components are a server, a terminal, and an emotion engine. The server stores data, learns, selects products, and makes suggestions. The terminal is responsible for inputting data and receiving suggestions from users, while the emotion engine collects and analyzes users' emotion data and sends it to the server.

[1022] Data collection

[1023] 1. Enter your purchase history

[1024] Using the interface, users input detailed information about the home appliances and interior items they have purchased, such as the product name, brand, price, purchase location, and purchase date. The device stores this data in a local database and transmits it to a server via the network.

[1025] 2. Enter your usage history and impressions

[1026] Users periodically enter information about the usage, impressions, and satisfaction of purchased products, such as frequency of use, ease of use, and satisfaction level. The device stores this data in a local database and transmits it to the server.

[1027] 3. Collecting Emotional Data

[1028] The device collects real-time emotional data from the user's facial expressions, tone of voice, and input text. The emotion engine analyzes this data to identify the user's emotional state. Specifically, it identifies emotions such as joy, sadness, anger, and surprise. The analysis results are sent to the server.

[1029] Data storage and analysis

[1030] 1. Data storage

[1031] The server stores the purchase history, use history, use impressions, and emotion data transmitted from the terminal and the emotion engine in a database.

[1032] 2. Data preprocessing and feature extraction

[1033] The server preprocesses the data it receives to make it easier to analyze. Specifically, it complements missing data and standardizes data formats. From the preprocessed data, it extracts user features (preferred brands, price ranges, features, emotional state, etc.).

[1034] 3. Updating the machine learning model

[1035] The server analyzes the data using machine learning algorithms, incorporating user style and preferences into the learning model, with emotional data also being incorporated into the learning process.

[1036] Product selection and proposal

[1037] 1. Selecting the right product

[1038] The server uses the learning model to select product candidates that match the user's preferences. This selection process takes into account price, functionality, design, and emotional data.

[1039] 2. Generate and submit proposals

[1040] The server generates detailed information about the selected product, including the product name, key features, price, where it can be purchased, and the best time to purchase it. The server then sends this information to the user's device.

[1041] Receiving feedback and relearning

[1042] 1. Receiving Proposals

[1043] The device receives the suggestion from the server and notifies the user via in-app notifications or push notifications.

[1044] 2. Enter and submit your feedback

[1045] If the user checks the suggested product and purchases it, they input feedback such as their experience with it and their level of satisfaction. The device stores this feedback in a local database and sends it to the server.

[1046] 3. Relearn

[1047] The server stores the feedback data in a database and uses it as the basis for re-learning.The server then uses the feedback data to update the machine learning model again and improve the accuracy of the suggestions.

[1048] Specific examples

[1049] For example, if a user purchases a new smart refrigerator and enters its details and their experience in the application, the emotion engine will collect emotions such as joy and satisfaction during use. Based on this data, the server will:

[1050] 1. The server stores purchase data and emotion data in a database and performs preprocessing together with past data.

[1051] 2. The server uses a machine learning model to learn the user's preferences and emotional state and determines that a newly released smart oven is suitable for the user.

[1052] 3. The server generates detailed information about the smart oven (price, features, vendor, best time to purchase) and sends it to the device.

[1053] 4. The device notifies the user of the offer and encourages them to make a purchase.

[1054] 5. The user purchases the recommended smart oven and enters their experience using it into the terminal.

[1055] 6. The device sends the usage experience to the server and it is reflected in the next learning session.

[1056] In this way, the system can make more accurate suggestions based on the user's emotional data, allowing the user to select the most suitable product based on their emotional state.

[1057] Prompt Sentence Examples

[1058] "After you purchase your new smart refrigerator, please tell us your experience and satisfaction. We will use that information to recommend your next product."

[1059] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1060] The flow of this system's program processing

[1061] Step 1:

[1062] Input: The user enters detailed information about the home appliance or interior item they purchased (product name, brand, price, purchase location, purchase date, etc.) into the interface.

[1063] Processing: The device stores these purchase information in a local database and sends it to the server over the network.

[1064] Output: The server stores the received purchase data in a database.

[1065] Step 2:

[1066] Input: Users periodically input information about their usage and impressions of the products they have purchased (frequency of use, ease of use, satisfaction, etc.).

[1067] Processing: The device stores this usage history and usage impression data in a local database and sends it to the server.

[1068] Output: The server stores the received usage history and user feedback data in a database.

[1069] Step 3:

[1070] Input: Obtain emotional data such as facial expressions, tone of voice, and input text when users use home appliances and interior items.

[1071] Processing: The device collects these emotion data in real time and sends them to the emotion engine, which analyzes the collected emotion data and identifies the user's emotional state.

[1072] Output: The emotion engine sends the analysis results to the server, which stores them in a database.

[1073] Step 4:

[1074] Input: The server stores the purchase history, usage history, usage impressions, and emotion data received from the device and emotion engine in a database.

[1075] Processing: The server preprocesses the data and standardizes it into a format that is easy to analyze. Specifically, it complements missing data and reformats the data format.

[1076] Output: Save the preprocessed data to a database.

[1077] Step 5:

[1078] Input: Preprocessed purchase history, usage history, usage impressions, and sentiment data.

[1079] Processing: The server extracts features (preferred brand, price range, features, emotional state, etc.) from the preprocessed data.

[1080] Output: Extracted feature data.

[1081] Step 6:

[1082] Input: Feature data.

[1083] Processing: The server uses machine learning algorithms to learn about the user's style and preferences and update the model. Emotional data is also incorporated into the learning process.

[1084] Output: The updated training model.

[1085] Step 7:

[1086] Input: The updated training model.

[1087] Processing: The server uses the learning model to select product candidates that match the user's preferences. This selection process takes into account price, functionality, design, and emotional data.

[1088] Output: Selected product information.

[1089] Step 8:

[1090] Input: Selected product information.

[1091] Processing: The server generates detailed information about the selected product (product name, key features, price, where it is available, best time to buy, etc.).

[1092] Output: Detailed product information.

[1093] Step 9:

[1094] Input: Product details.

[1095] Processing: The server sends product information to the user's device. The device receives the offer from the server and notifies the user. Notification methods include in-app notifications and push notifications.

[1096] Output: Notification to the user.

[1097] Step 10:

[1098] Input: Notification from the server.

[1099] Processing: If the user checks the suggested product and purchases it, they enter feedback into the application, such as their experience and satisfaction with it. The device stores this feedback in a local database and sends it to the server.

[1100] Output: Feedback data.

[1101] Step 11:

[1102] Input: Feedback data.

[1103] Processing: The server saves the feedback data in a database and uses it as the basis for re-learning. The server then uses the newly input feedback data to update the machine learning model again, improving the accuracy of the suggestions.

[1104] Output: The updated training model.

[1105] (Application example 2)

[1106] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1107] Conventional shopping assistant systems typically suggest products based on a user's purchase and usage history of interior design and home appliances. However, this approach fails to consider the user's emotional state and does not necessarily suggest products that best fit the user's current needs and preferences. Furthermore, the accuracy of product selection is limited, creating a demand for more personalized suggestions. To address this, product suggestions that take the user's emotional state into account are necessary. Therefore, the present invention aims to provide more accurate personalized product suggestions by analyzing the user's emotional data and making product suggestions based on that information.

[1108] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for inputting a user's purchase history, usage history, and usage impressions; means for receiving and storing the purchase history, usage history, and usage impressions in a database; machine learning means for learning user tendencies based on the data stored in the database; means for selecting appropriate products based on the user tendencies learned by the machine learning means; means for collecting and analyzing emotional data from the user's facial expressions, tone of voice, and input text; an emotion analysis engine for identifying the user's emotional state based on the collected emotional data; means for incorporating the results of the emotion analysis engine into the machine learning means and suggesting products taking the user's emotional state into consideration; means for notifying the user of information about the selected products; and means for receiving and storing feedback on the notified products in the database. This enables highly accurate personalized product suggestions based on the user's emotional state.

[1109] "Purchase history" refers to information about products and services purchased by a user in the past.

[1110] "Usage history" is a detailed record of how a user uses purchased products and services.

[1111] "Usage impressions" refers to the impressions and evaluations that users have about the products or services they have purchased.

[1112] A "database" is a system for efficiently organizing, managing, and storing data within a computer system.

[1113] "Machine learning methods" are techniques that use computer programs to learn patterns from data and then make predictions or classifications based on that knowledge.

[1114] "Emotional data" is data that represents the user's emotional state, and is information collected from facial expressions, tone of voice, input text, etc.

[1115] An "emotion analysis engine" is software or a system for analyzing collected emotional data and identifying a user's emotional state.

[1116] "Product suggestion" is the act of selecting the most suitable product based on user data and recommending it to the user.

[1117] "Feedback" refers to the evaluations and impressions that users give about the products and services provided.

[1118] "Notification means" refers to a method or device for conveying information about the selected product to the user.

[1119] "Preprocessing" refers to the initial processing used to convert data into a form that is easier to analyze.

[1120] A "feature" is an important element in the data input into a machine learning model for making predictions and classifications.

[1121] "Style" refers to the design and functional characteristics that users choose according to their particular trends and preferences.

[1122] "Push notifications" are a technology that allows you to communicate information to users in real time.

[1123] This invention is a personalized shopping assistant system that analyzes emotional data based on a user's purchase history, usage history, and usage impressions, and suggests optimal products to the user. This system is primarily composed of a server, a terminal (user device), and an emotion analysis engine.

[1124] System Configuration

[1125] Terminal

[1126] A terminal is a user device, such as a smartphone, that uses the following hardware and software:

[1127] Smartphone (using camera and microphone)

[1128] SQLite (as a local database)

[1129] Flutter (mobile application framework)

[1130] Emotion API (for emotion recognition)

[1131] server

[1132] The server processes and learns from the data using the following software:

[1133] Flask (backend framework)

[1134] TensorFlow (for machine learning algorithms)

[1135] Sentiment Analysis Engine

[1136] The sentiment analysis engine uses the Emotion API to analyze the collected emotional data.

[1137] Data processing and proposal flow

[1138] Data collection

[1139] Users enter detailed information about the home appliances and interior items they have purchased into the device, such as the product name, brand, price, purchase location, and purchase date. They also enter their usage history and impressions. The device stores this data in a local database and periodically sends it to the server.

[1140] In addition, the device's built-in camera and microphone are used to capture the user's facial expressions and tone of voice in real time, and emotional data is collected through the Emotion API.

[1141] Data storage and preprocessing

[1142] The server receives data sent from the device using Flask and stores it in a database. It also preprocesses the data to handle missing values ​​and standardize the format.

[1143] Data Analysis and Learning

[1144] The server uses TensorFlow to periodically update the machine learning model and select product candidates based on the user's preferences and emotional state. For example, the following prompt sentence is used:

[1145] Prompt Sentence Examples

[1146] def get_personalized_recommendations(user_data, emotion_data):

[1147] """

[1148] Personalized product recommendations based on user and sentiment data

[1149] """

[1150] from recommendation_engine import RecommendationEngine

[1151] from emotion_analysis import EmotionEngine

[1152] Analyze user and sentiment data

[1153] user_preferences = RecommendationEngine.analyze_user_data(user_data)

[1154] user_emotion = EmotionEngine.analyze_emotion_data(emotion_data)

[1155] Select the best product candidates for the user

[1156] recommended_items = RecommendationEngine.get_recommendations(user_preferences, user_emotion)

[1157] return recommended_items

[1158] For example, it suggests smart ovens based on user preferences and emotions.

[1159] user_data = {

[1160] 'purchase_history': [{'item': 'smart fridge', 'brand': 'BrandA', 'price': 80000}],

[1161] 'usage_feedback': [{'item': 'smart fridge', 'satisfaction': 5}]

[1162] }

[1163] emotion_data = {'joy': 0.8, 'satisfaction': 0.9}

[1164] recommendations = get_personalized_recommendations(user_data, emotion_data)

[1165] print(recommendations)

[1166] Product suggestions and feedback

[1167] The server generates detailed information about the selected product and sends it to the device. The device then uses push notifications to suggest the product to the user and displays the product detail page. The user then purchases the product and enters their usage experience and impressions into the app again. This allows the system to relearn the user's feedback and improve the accuracy of its next suggestions.

[1168] This invention enables highly accurate personalized product suggestions that correspond to the user's emotional state, allowing the user to have a more satisfying shopping experience.

[1169] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1170] Step 1:

[1171] Users enter detailed information about the home appliances and interior items they have purchased into the device. Specifically, they enter data such as the product name, brand, price, purchase location, and purchase date. The entered data is saved in the device's SQLite database, which records the user's purchase history.

[1172] Step 2:

[1173] Users enter their usage history and impressions of the purchased product into the terminal. For example, they enter data such as frequency of use, ease of use, and satisfaction level. This data is also stored in an SQLite database. This records the user's usage history and impressions.

[1174] Step 3:

[1175] The device uses a built-in camera and microphone to capture the user's facial expressions and tone of voice in real time. The captured data is analyzed using the Emotion API to generate emotion data, which records the user's emotional state.

[1176] Step 4:

[1177] The device transmits purchase history, usage history, usage impressions, and emotion data to the server via the network. This data is received by the server-side Flask backend framework and stored in a database.

[1178] Step 5:

[1179] The server preprocesses the received data. Specifically, it processes missing values, standardizes the format, and converts the data into a format that is easy to analyze. This preprocessed data is used to extract user features.

[1180] Step 6:

[1181] The server updates the machine learning model using TensorFlow to select product candidates based on the user's preferences and emotional state. Using the preprocessed data and features as input, the machine learning model learns the user's tendencies and emotional state and generates optimal product candidates.

[1182] Step 7:

[1183] The server generates detailed information about the selected product, including the product name, main features, price, where it can be purchased, and the best time to purchase it. The generated information is sent to the device.

[1184] Step 8:

[1185] The device notifies the user of the received product suggestions, and the user can receive a push notification or an in-app notification to view the details page of the suggested product.

[1186] Step 9:

[1187] The user checks the suggested products and, if they are interested, purchases them. After purchasing, they again enter their impressions and feedback into the device. This feedback data is also stored in the SQLite database.

[1188] Step 10:

[1189] The device sends feedback data to the server, which receives the feedback data and stores it in a database. The stored feedback data is used to retrain the machine learning model and improve the accuracy of the next suggestion.

[1190] This allows for highly accurate product suggestions based on the user's emotional state, allowing the user to have a more satisfying shopping experience.

[1191] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1192] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1193] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1194] [Third embodiment]

[1195] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1196] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1197] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1198] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1199] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1200] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1201] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1202] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1203] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1204] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1205] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1206] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1207] This invention is a personalized shopping assistant system that uses AI to learn the user's style and preferences based on their purchase and usage history and impressions of interior and home appliances, and then suggests the most suitable interior and home appliances.

[1208] System Overview

[1209] This system mainly consists of a server and a terminal (user device). The server stores data, learns, selects products, and makes suggestions, while the terminal is responsible for inputting data and receiving suggestions.

[1210] Data collection

[1211] Enter your purchase history

[1212] User: Through the interface, enter detailed information about the home appliance or interior item purchased (product name, brand, price, purchase location, purchase date).

[1213] Device: Purchase information is stored locally and sent to the server over the network.

[1214] Entering usage history and impressions

[1215] Users: Regularly enter usage, feedback, and satisfaction with purchased products.

[1216] Device: Usage history and user feedback data is stored locally and sent to the server.

[1217] Data storage and analysis

[1218] Receiving and storing data

[1219] Server: Stores purchase history, usage history, and user feedback sent from user devices in a database.

[1220] Data preprocessing and feature extraction

[1221] Server: Preprocesses the received data and extracts user features (preferred brands, price range, features, etc.).

[1222] Machine learning model updates

[1223] Server: Periodically analyzes data using machine learning algorithms to adapt the learning model to reflect user style and preferences.

[1224] Product selection and proposal

[1225] Selecting the right product

[1226] Server: Based on the learning model, the server selects products that match the user's preferences. This selection process also takes into account price, functionality, design, and vendor information.

[1227] Proposal generation and submission

[1228] Server: Generates detailed information about the selected product (product name, key features, price, where it can be purchased, and the best time to purchase).

[1229] Server: Sends the proposals to the user's device.

[1230] Receiving suggestions and feedback

[1231] Receiving Proposals

[1232] Device: Receives suggestions from the server and notifies the user. Notification methods include in-app notifications and push notifications.

[1233] Enter and submit feedback

[1234] User: Purchases the suggested product and enters their impressions and ratings.

[1235] Device: Feedback is stored locally and periodically sent to the server.

[1236] Server: The feedback is stored in a database and used as the basis for retraining.

[1237] Specific examples

[1238] For example, if a user buys a new smart refrigerator and enters details and user experience into the application, the server can:

[1239] 1. Server: Stores purchase data in a database and performs preprocessing together with past data.

[1240] 2. Server: A machine learning model learns user preferences and determines whether a newly released smart oven is suitable for the user.

[1241] 3. Server: Generates detailed information about the smart oven (price, features, vendor, best time to purchase) and sends it to the device.

[1242] 4. On the device: Notify the user of the offer and encourage them to purchase.

[1243] 5. User: Purchases the proposed smart oven and enters their experience using it into the terminal.

[1244] 6. Device: Sends usage information to the server and reflects it in the next learning session.

[1245] In this way, the system continuously learns user trends and continues to suggest appropriate products.

[1246] The processing flow will be explained below.

[1247] Step 1:

[1248] User: When purchasing home appliances or interior items, the user accesses the application and enters their purchase history, including the product name, brand, price, purchase location, and purchase date.

[1249] Step 2:

[1250] Terminal: Purchase information is stored in a local database and sent to a server over the network.

[1251] Step 3:

[1252] Server: Receives purchase information sent from the device and stores it in a central database.

[1253] Step 4:

[1254] User: Enters information about the usage and impressions of the purchased product into the application, such as frequency of use, ease of use, and satisfaction level.

[1255] Step 5:

[1256] Terminal: Usage history and user feedback data is stored in a local database and sent to a server via the network.

[1257] Step 6:

[1258] Server: Receives usage history and feedback sent from the device and stores it in a central database.

[1259] Step 7:

[1260] Server: Preprocesses the purchase history, usage history, and usage impressions stored in the database to make them easier to analyze. Specifically, it handles missing values ​​and standardizes the format of the data.

[1261] Step 8:

[1262] Server: Extracts features from the preprocessed data to identify user preferences and trends, such as preferred brands, price ranges, and features.

[1263] Step 9:

[1264] Server: Uses the features to update the machine learning model and learn about the user's style and preferences.

[1265] Step 10:

[1266] Server: Using a learning model, the server selects product candidates that match the user's preferences, taking into account price, functionality, design, and other factors.

[1267] Step 11:

[1268] Server: Generates detailed information about the selected product, including the product name, key features, price, where it is available, and the best time to purchase.

[1269] Step 12:

[1270] Server: Sends the selected products and their details to the user's device.

[1271] Step 13:

[1272] On the device, the user is notified of the suggestions received from the server via in-app notifications and push notifications.

[1273] Step 14:

[1274] User: Review the suggested products and purchase them if they are interested.

[1275] Step 15:

[1276] Users: Enter post-purchase feedback into the application, including experience and satisfaction.

[1277] Step 16:

[1278] Terminal: The feedback is stored in a local database and sent to a server over the network.

[1279] Step 17:

[1280] Server: Receives feedback sent from devices and stores it in a central database for future analysis and learning.

[1281] Step 18:

[1282] Server: Using the feedback data, the machine learning model is updated again to improve the accuracy of the suggestions.

[1283] Example 1

[1284] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1285] Conventional shopping assistant systems have had the challenge of making recommendations that fully reflect a user's individual preferences and past purchase history. It is also difficult to absorb user feedback in real time and use it in future recommendations. Furthermore, they are unable to provide sufficient recommendations on the best time and place to purchase, limiting the user experience.

[1286] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1287] In this invention, the server includes means for inputting a user's purchase history, usage history, and usage impressions, means for receiving and storing the purchase history, usage history, and usage impressions in a database, means for preprocessing the data stored in the database, means for extracting features from the preprocessed data, machine learning means for learning user tendencies based on the data stored in the database and the extracted features, means for selecting appropriate products based on the user tendencies learned by the machine learning means, means for sending and notifying the user of information about the selected products to the user's terminal, and means for receiving and storing feedback on the notified products in the database. This enables personalized suggestions that reflect the user's individual preferences and past purchase history.

[1288] "Users" are individuals or corporations who use this system to purchase interior goods and home appliances, manage their usage, etc.

[1289] "Purchase history" refers to data on detailed information about interior and home appliance products purchased by a user (product name, brand, price, purchase location, purchase date, etc.).

[1290] "Usage history" refers to data about how users use interior and home appliances they have purchased.

[1291] "Usage impressions" is data that represents the evaluations and opinions that users have about the interior design and home appliances they have used.

[1292] "Means" are the specific methods or tools used to realize the system.

[1293] A "database" is a storage device for centrally managing and storing various data such as purchase history, usage history, and usage impressions.

[1294] "Preprocessing" is the process of correcting or removing inconsistencies or missing data and preparing the data for analysis before data analysis.

[1295] "Features" refers to the numerical representation of user preferences and behavioral tendencies extracted from a database, which are used as input data in machine learning.

[1296] "Machine learning" is an algorithm that allows a computer to learn from past data and make predictions and decisions based on future data.

[1297] "Selection" is the process of using a machine learning model to determine the product candidates that best suit the user's preferences.

[1298] "Terminal" means the device (e.g., smartphone, tablet, PC) that a user uses to interact with the system.

[1299] "Notification" refers to the act of transmitting information from a server to a user, and includes in-app notifications and push notifications.

[1300] "Feedback" refers to information about users' impressions and evaluations of the proposed product after using it.

[1301] This invention is a personalized shopping assistant system in which a generative AI learns a user's style and preferences based on their purchase history, usage history, and impressions of interior and home appliances, and suggests the most suitable interior and home appliances. This system is mainly composed of a server and a terminal (user device).

[1302] composition

[1303] server

[1304] The server serves as a central location for receiving, storing, pre-processing, and machine learning data. It has the following main functions:

[1305] Purchase history, usage history, and usage feedback are received and stored in a database.

[1306] Preprocess the received data and extract features.

[1307] Based on the features, machine learning algorithms (e.g., random forests, neural networks) are used to learn user tendencies.

[1308] Based on the learning results, the system selects the most suitable product for the user and generates detailed information about it.

[1309] The selected product information is sent to the user's device.

[1310] Terminal

[1311] A terminal is a device that a user uses to enter data and receive notifications from a server. Its main functions are:

[1312] Provides an interface for inputting purchase history, usage history, and usage impressions.

[1313] The user is notified of the proposal information received from the server.

[1314] Data processing and calculation

[1315] Data collection

[1316] User: Using a dedicated app or web portal on a smartphone or PC, enter your purchase history (product name, brand, price, purchase location, purchase date). Example: Value Range Refrigerator, AB manufacturer, 50,000 yen, XYZ store, October 1, 2023.

[1317] Terminal: The information entered by the user is temporarily stored in a local database and sent to a server via the Internet.

[1318] User: Enter usage status, impressions, and satisfaction level of the purchased product. Example: Good cooling performance, excellent design.

[1319] Data storage and analysis

[1320] Server: Stores the received purchase history, usage history, and user feedback in a database.

[1321] Server: Preprocesses the received data, corrects missing data, and removes inconsistent data. Then, extracts features based on user preferences and behavioral trends.

[1322] Server: Uses machine learning algorithms based on features to learn user trends.

[1323] Product selection and proposal

[1324] Server: Selects the best product for the user based on learned trends, taking into account information such as price, functionality, design, and vendor.

[1325] Server: Generates detailed information about the selected product (product name, key features, price, where it can be purchased, and the best time to purchase) and sends it to the user's device.

[1326] On the device: Notify the user of the received recommendation. Example: Display an in-app notification saying, "Our new smart oven is perfect for you!"

[1327] Enter and submit feedback

[1328] User: Purchases the suggested product and enters their impressions and ratings into the device.

[1329] Device: Stores feedback data locally and sends it to the server.

[1330] Server: Receives feedback and stores it in a database for future machine learning.

[1331] Specific examples

[1332] Prompt Sentence Examples

[1333] "Please tell us the details of the home appliance you recently purchased (product name, brand, price, place of purchase, purchase date)."

[1334] "Please tell us about your experience using and satisfaction with the home appliances you purchased."

[1335] "Tell me about the next interior or home appliance you're considering purchasing."

[1336] These prompts can be used to gather the necessary data from the user and provide optimal product recommendations.

[1337] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1338] Step 1:

[1339] Input: User enters purchase history through application or web portal. For example: product name, brand, price, purchase location, purchase date.

[1340] How it works: Users use the interface on their smartphone or computer to enter detailed information about the home appliances and interior items they have purchased.

[1341] Output: Purchase information is stored on the device and data is generated that is later sent to a server.

[1342] Step 2:

[1343] Input: Purchase history data and details entered by the user.

[1344] How it works: The device temporarily stores the purchase information in a local database and then sends it over the internet to a server, periodically retrying the transmission until it is successful.

[1345] Output: Purchase history data sent to the server.

[1346] Step 3:

[1347] Input: Purchase history data received by the server.

[1348] How it works: The server receives the purchase history sent from the device and stores it in a database.

[1349] Output: Purchase history stored in a database.

[1350] Step 4:

[1351] Input: User-entered usage history data such as usage status, impressions, and satisfaction.

[1352] How it works: Users periodically enter their impressions and satisfaction with the device. Examples include "This refrigerator has great cooling performance" and "It has a nice design."

[1353] Output: Usage history and impressions are stored on the device and data is generated that is later sent to the server.

[1354] Step 5:

[1355] Input: Usage history and user feedback data.

[1356] How it works: The device stores this data locally and sends it to the server.

[1357] Output: Usage history and user feedback data sent to the server.

[1358] Step 6:

[1359] Input: Usage history and user feedback data received by the server.

[1360] Operation: The server receives the usage history and usage impressions sent from the device and stores them in a database.

[1361] Output: Usage history and user feedback data stored in a database.

[1362] Step 7:

[1363] Input: Purchase history, usage history, and user feedback data stored in the database.

[1364] How it works: The server preprocesses the received data, correcting or removing inconsistencies and missing data, and then extracts features that quantify user preferences and behavioral trends.

[1365] Output: Preprocessed data and extracted features.

[1366] Step 8:

[1367] Input: Preprocessed data and extracted features.

[1368] How it works: Using machine learning algorithms based on features, it learns user tendencies.

[1369] Output: An updated machine learning model.

[1370] Step 9:

[1371] Input: The machine learning model obtained by training.

[1372] How it works: The server uses a learning model to recommend products that match the user's preferences. Example: "The new smart oven is perfect for you."

[1373] Output: Suitable products and their details (price, key features, where available, best time to buy, etc.).

[1374] Step 10:

[1375] Input: Detailed information about the selected product.

[1376] How it works: The server generates detailed information about the selected product and sends it to the user's device.

[1377] Output: The suggestions sent to the user's device.

[1378] Step 11:

[1379] Input: Proposal notification.

[1380] How it works: The device receives the offer from the server and notifies the user. Notification methods include in-app notifications and push notifications. Example: "Our new smart oven is perfect for you!"

[1381] Output: A notification showing the suggestion.

[1382] Step 12:

[1383] Input: User's impressions and ratings of the proposed product.

[1384] How it works: The user uses the suggested product and enters their impressions and ratings.

[1385] Output: Feedback data is generated and saved to the device.

[1386] Step 13:

[1387] Input: Feedback data.

[1388] How it works: The device stores the feedback data locally and sends it to the server.

[1389] Output: Feedback data sent to the server.

[1390] Step 14:

[1391] Input: The feedback data received by the server.

[1392] How it works: The server receives the feedback data and stores it in a database. It is used for retraining.

[1393] Output: Feedback data stored in a database.

[1394] (Application example 1)

[1395] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1396] Conventional personalized shopping assistant systems have had problems such as limited product suggestions based on users' purchase history and usage impressions, being available only in specific locations, or having poor visual suggestions. Furthermore, there was a lack of means for users to check the experience of the suggested products in advance, and there were limitations on providing information on the optimal time and place to purchase, which could lead to a decrease in user satisfaction.

[1397] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1398] In this invention, the server includes a means for inputting a user's purchase history, usage history, and usage impressions, a means for receiving the purchase history, usage history, and usage impressions and storing them in a database, and a means for preprocessing the data stored in the database and extracting features. This makes it possible to learn user trends, suggest optimal products, visually display them in a virtual store, and suggest where they can be purchased and the optimal time to purchase them.

[1399] "User" means any individual or legal entity using a particular product or service.

[1400] "Purchase history" refers to information about products and services a user has purchased in the past.

[1401] "Usage history" refers to data about how a user uses a purchased product or service.

[1402] "Usage impressions" refers to the impressions and evaluations that users have after using a product or service.

[1403] "Input means" refers to the interface or function that allows users to input data into the system.

[1404] A "database" refers to a system that systematically stores information and makes it available for searching and use as needed.

[1405] "Preprocessing" refers to a series of steps that convert collected data into an analyzable format.

[1406] "Features" refer to useful attributes or parameters extracted from data.

[1407] "Machine learning methods" refer to techniques that allow computers to learn from data and recognize patterns.

[1408] "Goods" means any products or services intended for purchase or use by Users.

[1409] A "virtual store" refers to an online shopping environment created using virtual reality technology.

[1410] "Visual presentation means" refers to techniques or methods for presenting information graphically to a user.

[1411] "Feedback" refers to the ratings and opinions users provide about products and services.

[1412] The "optimal time to purchase" refers to the time period that encourages users to make a purchase at the most advantageous time.

[1413] This invention is a personalized shopping assistant system that uses AI to learn the user's style and preferences based on their purchase and usage history and impressions of interior and home appliances, and then suggests the most suitable products. The system is mainly composed of a server and a terminal (user device).

[1414] System Overview

[1415] Data collection

[1416] Users enter their purchase history and usage impressions via the terminal. They enter detailed information about the purchased product, usage status, impressions, and satisfaction level, and the terminal collects this information. The collected data is sent to the server via the network.

[1417] Data storage and analysis

[1418] The server stores the received purchase history, usage history, and usage impressions in a database. The stored data is preprocessed to extract user features (preferred brands, price ranges, features, etc.). The features are analyzed using a machine learning model to learn the user's style and preferences.

[1419] Product selection and proposal

[1420] The server selects products that match the user's preferences based on the learning model. This selection process also takes into account price, functionality, design, and vendor information. The selected product information is visually displayed in a virtual store.

[1421] Program processing

[1422] 1. Data Acquisition and Transmission

[1423] The device (such as the user's VR headset or smart glasses) collects purchase history and usage impressions and sends them to the server. The hardware used includes Oculus as a VR headset and Google Glass as smart glasses.

[1424] 2. Data analysis and proposal generation

[1425] The server uses Python's Flask framework to analyze the received data, and uses machine learning models such as TensorFlow to learn the user's preferences and select appropriate products. This selected product information is stored in a database and then sent back to the device.

[1426] 3. Virtual store display

[1427] The device uses software such as Unity or Unreal Engine to visually display product information in a virtual environment, allowing users to move around the virtual store and examine the suggested products in detail.

[1428] Specific examples

[1429] For example, if a user purchases a new smart refrigerator and enters details and user experience into an application, the flow would be as follows:

[1430] Users enter information about the smart refrigerator they have purchased and their experience using it into the application.

[1431] The terminal collects this data and sends it to the server.

[1432] The server receives and stores the data and uses machine learning models to learn user preferences.

[1433] The server then selects a new smart oven that is suitable for the user and generates detailed information about it.

[1434] The terminal visually presents the proposed smart oven to the user in a virtual reality environment.

[1435] Users can view suggested smart ovens in a virtual store and consider purchasing them.

[1436] Prompt Sentence Examples

[1437] user_id: "12345"

[1438] purchased_items:

[1439] item: "Smart Refrigerator"

[1440] brand: "BrandA"

[1441] price: 50000

[1442] purchase_date: "2023-08-01"

[1443] item: "Washing machine"

[1444] brand: "BrandB"

[1445] price: 30000

[1446] purchase_date: "2023-01-15"

[1447] user_feedback:

[1448] item: "Smart Refrigerator"

[1449] Rating: 5

[1450] comment: "Very useful"

[1451] item: "Washing machine"

[1452] Rating: 4

[1453] comment: "Very satisfied"

[1454] In this way, the system suggests optimal products based on the user's purchase history and usage impressions, enabling a visual experience in a virtual store.

[1455] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1456] Step 1:

[1457] Users enter their purchase history and usage experience, including details such as product name, brand, price, purchase location, and purchase date. This data is collected and stored locally on the device (e.g., smart glasses or VR headset). The input data format is as follows:

[1458] Input: Product name, brand, price, purchase location, purchase date

[1459] Output: Locally stored purchase and usage data

[1460] Step 2:

[1461] The device sends the collected purchase history and usage feedback to a server via the network, using the Oculus SDK and Google Glass API as interfaces.

[1462] Input: Locally stored purchase and usage data

[1463] Output: Data sent to the server

[1464] Step 3:

[1465] The server stores the received purchase history, usage history, and usage feedback in a database. After storing this data, it is preprocessed for analysis.

[1466] Input: Submitted purchase history, usage history, and user impressions

[1467] Output: Data stored in the database

[1468] Step 4:

[1469] The server preprocesses the data and extracts user features, including cleaning, normalizing, handling missing values, and performing feature engineering.

[1470] Input: Raw data stored in a database

[1471] Output: Extracted features

[1472] Step 5:

[1473] The server inputs the features into a machine learning model (e.g., TensorFlow) to learn the user's style and preferences. The model recognizes and learns patterns based on the user's past data.

[1474] Input: Extracted features

[1475] Output: A learned model containing user preference patterns

[1476] Step 6:

[1477] The server then uses the learning model to select products that suit the user's preferences, taking into account product price, functionality, design, and vendor information.

[1478] Input: User preference patterns

[1479] Output: A list of recommended products

[1480] Step 7:

[1481] The server generates detailed information about the selected product (product name, main features, price, distributor, best time to purchase) and sends it to the terminal.

[1482] Input: A list of recommended products

[1483] Output: Proposal to the device

[1484] Step 8:

[1485] The device visually displays the received recommendations in a virtual store. Users can view the suggested products in the virtual space using a VR headset or smart glasses. The system is displayed using Unity or Unreal Engine.

[1486] Input: Proposal

[1487] Output: Visual representation of the virtual store

[1488] Step 9:

[1489] Users can view the proposed products in the virtual store and provide feedback, which is collected at the terminal and sent to the server.

[1490] Input: User feedback

[1491] Output: Feedback data to the server

[1492] Step 10:

[1493] The server stores the received feedback in a database and uses it to update the machine learning model, thereby improving the accuracy of the suggestions.

[1494] Input: User feedback data

[1495] Output: Updated training model

[1496] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1497] This is a personalized shopping assistant system that uses a generation AI to learn the user's style and preferences based on their purchase and usage history, impressions, and emotional data on interior and home appliances, and then suggests the most suitable interior and home appliances. In particular, by combining it with an emotion engine, it is possible to suggest products that take into account the user's emotional state, providing a more personalized service.

[1498] System Overview

[1499] This system is mainly composed of a server, a terminal (user device), and an emotion engine. The server stores data, learns, selects products, and makes recommendations, while the terminal is responsible for inputting data and receiving recommendations. The emotion engine collects and analyzes user emotion data and sends it to the server.

[1500] Data collection

[1501] Enter your purchase history

[1502] User: Through the interface, the user enters detailed information about the home appliance or interior item they purchased, such as the product name, brand, price, purchase location, and purchase date.

[1503] Terminal: Purchase information is stored in a local database and sent to a server over the network.

[1504] Entering usage history and impressions

[1505] Users: Periodically enter information about the usage, impressions, and satisfaction of purchased products, such as frequency of use, ease of use, and satisfaction level.

[1506] Device: Usage history and user feedback data is stored in a local database and sent to the server.

[1507] Collecting Emotional Data

[1508] On-device: Collects real-time emotional data from the user's facial expressions, tone of voice, and input text. This data is analyzed through an emotion engine.

[1509] Emotion engine: Analyzes collected emotion data to identify the user's emotional state, specifically identifying emotions such as joy, sadness, anger, and surprise.

[1510] Emotion engine: Sends the emotion analysis results to the server.

[1511] Data storage and analysis

[1512] Receiving and storing data

[1513] Server: Stores the purchase history, usage history, usage impressions, and emotion data sent from the device and emotion engine in a database.

[1514] Data preprocessing and feature extraction

[1515] Server: Preprocesses the received data to make it easier to analyze. Specifically, it handles missing values ​​and standardizes the format of the data. Emotion data is also preprocessed.

[1516] Server: Extract user features (preferred brands, price range, features, emotional state, etc.) from the preprocessed data.

[1517] Machine learning model updates

[1518] Server: Periodically analyzes the data using machine learning algorithms to adapt the learning model to reflect user style and preferences. Emotional data is also incorporated into the learning process.

[1519] Product selection and proposal

[1520] Selecting the right product

[1521] Server: Based on the learning model, the server selects product candidates that match the user's preferences. This selection process takes into account price, functionality, design, and emotional data.

[1522] Proposal generation and submission

[1523] Server: Generates detailed information about the selected product, including the product name, key features, price, where it is available, and the best time to purchase.

[1524] Server: Sends the selected products and their details to the user's device.

[1525] Receiving suggestions and feedback

[1526] Receiving Proposals

[1527] Device: Receives suggestions from the server and notifies the user via in-app notifications and push notifications.

[1528] Enter and submit feedback

[1529] User: Review the suggested products, purchase them if they are interested, and enter feedback into the application after purchase, including their experience and satisfaction.

[1530] Terminal: The feedback is stored in a local database and sent to a server over the network.

[1531] Server: The feedback sent from the device is stored in a database and used as the basis for retraining.

[1532] Server: Using the feedback data, the machine learning model is updated again to improve the accuracy of the suggestions.

[1533] Specific examples

[1534] For example, a user purchases a new smart refrigerator and enters its details and their experience in the application. Furthermore, the emotion engine collects emotional data such as the user's happiness and satisfaction while using the refrigerator. Based on this data, the server performs the following actions:

[1535] 1. Server: Stores purchase data and emotion data in a database and performs preprocessing together with past data.

[1536] 2. Server: A machine learning model learns the user's preferences and emotional state and determines whether a newly released smart oven is suitable for the user.

[1537] 3. Server: Generates detailed information about the smart oven (price, features, vendor, best time to purchase) and sends it to the device.

[1538] 4. On the device: Notify the user of the offer and encourage them to purchase.

[1539] 5. User: Purchases the proposed smart oven and enters their experience using it into the terminal.

[1540] 6. Device: Sends usage information to the server and reflects it in the next learning session.

[1541] In this way, the system can make more accurate suggestions based on the user's emotional data, allowing the user to select the most appropriate product based on their emotional state.

[1542] The processing flow will be explained below.

[1543] Step 1:

[1544] User: When purchasing home appliances or interior items, the user accesses the application and enters their purchase history, including the product name, brand, price, purchase location, and purchase date.

[1545] Step 2:

[1546] Terminal: Purchase information is stored in a local database and sent to a server over the network.

[1547] Step 3:

[1548] Server: Receives purchase information sent from the device and stores it in a central database.

[1549] Step 4:

[1550] User: Enters information about the usage and impressions of the purchased product into the application, such as frequency of use, ease of use, and satisfaction level.

[1551] Step 5:

[1552] Terminal: Usage history and user feedback data is stored in a local database and sent to a server via the network.

[1553] Step 6:

[1554] Server: Receives usage history and feedback sent from the device and stores it in a central database.

[1555] Step 7:

[1556] On-device: Collects real-time emotional data from the user's facial expressions, tone of voice, and input text. Analyzes emotions using facial recognition and voice analysis algorithms.

[1557] Step 8:

[1558] Emotion engine: Analyzes collected emotion data to identify the user's emotional state, specifically identifying emotions such as joy, sadness, anger, and surprise.

[1559] Step 9:

[1560] Emotion engine: Sends the emotion analysis results to the server.

[1561] Step 10:

[1562] Server: Stores the purchase history, usage history, usage impressions, and emotion data sent from the device and emotion engine in a database.

[1563] Step 11:

[1564] Server: Preprocesses the received data to make it easier to analyze, specifically by handling missing values ​​and standardizing the data format.

[1565] Step 12:

[1566] Server: Extract user features (preferred brands, price range, features, emotional state, etc.) from the preprocessed data.

[1567] Step 13:

[1568] Server: Uses the features to update the machine learning model to learn about the user's style and preferences, incorporating sentiment data into the learning process.

[1569] Step 14:

[1570] Server: Based on the learning model, it selects product candidates that match the user's preferences, taking into account price, functionality, design, and emotional data.

[1571] Step 15:

[1572] Server: Generates detailed information about the selected product, including the product name, key features, price, where it is available, and the best time to purchase.

[1573] Step 16:

[1574] Server: Sends the selected products and their details to the user's device.

[1575] Step 17:

[1576] On the device, the user is notified of the suggestions received from the server via in-app notifications and push notifications.

[1577] Step 18:

[1578] User: Review the suggested products and purchase them if they are interested.

[1579] Step 19:

[1580] Users: Enter post-purchase feedback into the application, including experience and satisfaction.

[1581] Step 20:

[1582] Terminal: The feedback is stored in a local database and sent to a server over the network.

[1583] Step 21:

[1584] Server: Receives feedback sent from devices and stores it in a central database for future analysis and learning.

[1585] Step 22:

[1586] Server: Using the feedback data, the machine learning model is updated again to improve the accuracy of the suggestions.

[1587] In this way, the system can make more accurate suggestions based on the user's emotional data, allowing the user to select the most appropriate product based on their emotional state.

[1588] Example 2

[1589] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1590] Traditional personalized shopping assistant systems suggest products based on a user's purchase history, usage history, and usage impressions. However, these systems do not take into account the user's emotional state, making it difficult to make optimal product suggestions based on each individual user's emotions. Furthermore, the process by which users provide feedback on suggested products is not sufficiently considered, limiting the improvement of suggestion accuracy. This makes it difficult to make effective product suggestions that increase user satisfaction.

[1591] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1592] In this invention, the server includes means for receiving and storing a user's purchase history, usage history, usage impressions, and user emotional data in a database, machine learning means for learning user tendencies based on the data stored in the database, means for selecting appropriate products based on the user tendencies and emotional state learned by the machine learning means, and means for receiving the notified product feedback and emotional data and storing them in the database. This enables personalized product suggestions that take the user's emotional state into consideration, and further enables the accuracy of suggestions to be continuously improved based on feedback data from the user.

[1593] "Purchase history" refers to detailed information about products purchased by a user in the past, including product name, brand, price, purchase location, and purchase date.

[1594] "Usage history" refers to information about how a user uses a purchased product, including, for example, frequency of use, duration of use, and purpose of use.

[1595] "Usage impressions" refers to the impressions and evaluations that users have of the products they have purchased, including satisfaction, ease of use, and evaluation of design.

[1596] "Emotional data" refers to data that represents a user's emotional state, including facial expressions, tone of voice, and emotional information derived from input text.

[1597] "Means of storage" refers to the technology and methods used to record input data in a database.

[1598] "Machine learning methods" refer to algorithms that analyze data and learn patterns and trends based on the results, which can then predict user preferences and behavioral patterns.

[1599] "Means of selection" refers to the technology or method of selecting the optimal product from multiple options based on the results obtained through machine learning.

[1600] "Means of Notification" refers to the technology or method used to notify users of information about the selected product, including in-app notifications and push notifications.

[1601] "Means for receiving feedback" refers to the technology and methods for receiving product ratings and impressions provided by users.

[1602] "Preprocessing means" refers to techniques and methods for initial processing of collected data to make it easier to analyze. Specifically, this includes imputing missing values ​​and standardizing data formats.

[1603] "Means for extracting features" refers to techniques and methods for extracting important features and patterns from preprocessed data that can be used for analysis.

[1604] This invention is a personalized shopping assistant system that uses a generation AI to learn the user's style and preferences based on the user's purchase history, usage history, usage impressions, and emotional data for interior and home appliances, and then suggests the most suitable interior and home appliances.The invention uses a server, a terminal (user device), and an emotion engine.

[1605] System Overview

[1606] The system's main components are a server, a terminal, and an emotion engine. The server stores data, learns, selects products, and makes suggestions. The terminal is responsible for inputting data and receiving suggestions from users, while the emotion engine collects and analyzes users' emotion data and sends it to the server.

[1607] Data collection

[1608] 1. Enter your purchase history

[1609] Using the interface, users input detailed information about the home appliances and interior items they have purchased, such as the product name, brand, price, purchase location, and purchase date. The device stores this data in a local database and transmits it to a server via the network.

[1610] 2. Enter your usage history and impressions

[1611] Users periodically enter information about the usage, impressions, and satisfaction of purchased products, such as frequency of use, ease of use, and satisfaction level. The device stores this data in a local database and transmits it to the server.

[1612] 3. Collecting Emotional Data

[1613] The device collects real-time emotional data from the user's facial expressions, tone of voice, and input text. The emotion engine analyzes this data to identify the user's emotional state. Specifically, it identifies emotions such as joy, sadness, anger, and surprise. The analysis results are sent to the server.

[1614] Data storage and analysis

[1615] 1. Data storage

[1616] The server stores the purchase history, use history, use impressions, and emotion data transmitted from the terminal and the emotion engine in a database.

[1617] 2. Data preprocessing and feature extraction

[1618] The server preprocesses the data it receives to make it easier to analyze. Specifically, it complements missing data and standardizes data formats. From the preprocessed data, it extracts user features (preferred brands, price ranges, features, emotional state, etc.).

[1619] 3. Updating the machine learning model

[1620] The server analyzes the data using machine learning algorithms, incorporating user style and preferences into the learning model, with emotional data also being incorporated into the learning process.

[1621] Product selection and proposal

[1622] 1. Selecting the right product

[1623] The server uses the learning model to select product candidates that match the user's preferences. This selection process takes into account price, functionality, design, and emotional data.

[1624] 2. Generate and submit proposals

[1625] The server generates detailed information about the selected product, including the product name, key features, price, where it can be purchased, and the best time to purchase it. The server then sends this information to the user's device.

[1626] Receiving feedback and relearning

[1627] 1. Receiving Proposals

[1628] The device receives the suggestion from the server and notifies the user via in-app notifications or push notifications.

[1629] 2. Enter and submit your feedback

[1630] If the user checks the suggested product and purchases it, they input feedback such as their experience with it and their level of satisfaction. The device stores this feedback in a local database and sends it to the server.

[1631] 3. Relearn

[1632] The server stores the feedback data in a database and uses it as the basis for re-learning.The server then uses the feedback data to update the machine learning model again and improve the accuracy of the suggestions.

[1633] Specific examples

[1634] For example, if a user purchases a new smart refrigerator and enters its details and their experience in the application, the emotion engine will collect emotions such as joy and satisfaction during use. Based on this data, the server will:

[1635] 1. The server stores purchase data and emotion data in a database and performs preprocessing together with past data.

[1636] 2. The server uses a machine learning model to learn the user's preferences and emotional state and determines that a newly released smart oven is suitable for the user.

[1637] 3. The server generates detailed information about the smart oven (price, features, vendor, best time to purchase) and sends it to the device.

[1638] 4. The device notifies the user of the offer and encourages them to make a purchase.

[1639] 5. The user purchases the recommended smart oven and enters their experience using it into the terminal.

[1640] 6. The device sends the usage experience to the server and it is reflected in the next learning session.

[1641] In this way, the system can make more accurate suggestions based on the user's emotional data, allowing the user to select the most suitable product based on their emotional state.

[1642] Prompt Sentence Examples

[1643] "After you purchase your new smart refrigerator, please tell us your experience and satisfaction. We will use that information to recommend your next product."

[1644] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1645] The flow of this system's program processing

[1646] Step 1:

[1647] Input: The user enters detailed information about the home appliance or interior item they purchased (product name, brand, price, purchase location, purchase date, etc.) into the interface.

[1648] Processing: The device stores these purchase information in a local database and sends it to the server over the network.

[1649] Output: The server stores the received purchase data in a database.

[1650] Step 2:

[1651] Input: Users periodically input information about their usage and impressions of the products they have purchased (frequency of use, ease of use, satisfaction, etc.).

[1652] Processing: The device stores this usage history and usage impression data in a local database and sends it to the server.

[1653] Output: The server stores the received usage history and user feedback data in a database.

[1654] Step 3:

[1655] Input: Obtain emotional data such as facial expressions, tone of voice, and input text when users use home appliances and interior items.

[1656] Processing: The device collects these emotion data in real time and sends them to the emotion engine, which analyzes the collected emotion data and identifies the user's emotional state.

[1657] Output: The emotion engine sends the analysis results to the server, which stores them in a database.

[1658] Step 4:

[1659] Input: The server stores the purchase history, usage history, usage impressions, and emotion data received from the device and emotion engine in a database.

[1660] Processing: The server preprocesses the data and standardizes it into a format that is easy to analyze. Specifically, it complements missing data and reformats the data format.

[1661] Output: Save the preprocessed data to a database.

[1662] Step 5:

[1663] Input: Preprocessed purchase history, usage history, usage impressions, and sentiment data.

[1664] Processing: The server extracts features (preferred brand, price range, features, emotional state, etc.) from the preprocessed data.

[1665] Output: Extracted feature data.

[1666] Step 6:

[1667] Input: Feature data.

[1668] Processing: The server uses machine learning algorithms to learn about the user's style and preferences and update the model. Emotional data is also incorporated into the learning process.

[1669] Output: The updated training model.

[1670] Step 7:

[1671] Input: The updated training model.

[1672] Processing: The server uses the learning model to select product candidates that match the user's preferences. This selection process takes into account price, functionality, design, and emotional data.

[1673] Output: Selected product information.

[1674] Step 8:

[1675] Input: Selected product information.

[1676] Processing: The server generates detailed information about the selected product (product name, key features, price, where it is available, best time to buy, etc.).

[1677] Output: Detailed product information.

[1678] Step 9:

[1679] Input: Product details.

[1680] Processing: The server sends product information to the user's device. The device receives the offer from the server and notifies the user. Notification methods include in-app notifications and push notifications.

[1681] Output: Notification to the user.

[1682] Step 10:

[1683] Input: Notification from the server.

[1684] Processing: If the user checks the suggested product and purchases it, they enter feedback into the application, such as their experience and satisfaction with it. The device stores this feedback in a local database and sends it to the server.

[1685] Output: Feedback data.

[1686] Step 11:

[1687] Input: Feedback data.

[1688] Processing: The server saves the feedback data in a database and uses it as the basis for re-learning. The server then uses the newly input feedback data to update the machine learning model again, improving the accuracy of the suggestions.

[1689] Output: The updated training model.

[1690] (Application example 2)

[1691] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1692] Conventional shopping assistant systems typically suggest products based on a user's purchase and usage history of interior design and home appliances. However, this approach fails to consider the user's emotional state and does not necessarily suggest products that best fit the user's current needs and preferences. Furthermore, the accuracy of product selection is limited, creating a demand for more personalized suggestions. To address this, product suggestions that take the user's emotional state into account are necessary. Therefore, the present invention aims to provide more accurate personalized product suggestions by analyzing the user's emotional data and making product suggestions based on that information.

[1693] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for inputting a user's purchase history, usage history, and usage impressions; means for receiving and storing the purchase history, usage history, and usage impressions in a database; machine learning means for learning user tendencies based on the data stored in the database; means for selecting appropriate products based on the user tendencies learned by the machine learning means; means for collecting and analyzing emotional data from the user's facial expressions, tone of voice, and input text; an emotion analysis engine for identifying the user's emotional state based on the collected emotional data; means for incorporating the results of the emotion analysis engine into the machine learning means and suggesting products taking the user's emotional state into consideration; means for notifying the user of information about the selected products; and means for receiving and storing feedback on the notified products in the database. This enables highly accurate personalized product suggestions based on the user's emotional state.

[1694] "Purchase history" refers to information about products and services purchased by a user in the past.

[1695] "Usage history" is a detailed record of how a user uses purchased products and services.

[1696] "Usage impressions" refers to the impressions and evaluations that users have about the products or services they have purchased.

[1697] A "database" is a system for efficiently organizing, managing, and storing data within a computer system.

[1698] "Machine learning methods" are techniques that use computer programs to learn patterns from data and then make predictions or classifications based on that knowledge.

[1699] "Emotional data" is data that represents the user's emotional state, and is information collected from facial expressions, tone of voice, input text, etc.

[1700] An "emotion analysis engine" is software or a system for analyzing collected emotional data and identifying a user's emotional state.

[1701] "Product suggestion" is the act of selecting the most suitable product based on user data and recommending it to the user.

[1702] "Feedback" refers to the evaluations and impressions that users give about the products and services provided.

[1703] "Notification means" refers to a method or device for conveying information about the selected product to the user.

[1704] "Preprocessing" refers to the initial processing used to convert data into a form that is easier to analyze.

[1705] A "feature" is an important element in the data input into a machine learning model for making predictions and classifications.

[1706] "Style" refers to the design and functional characteristics that users choose according to their particular trends and preferences.

[1707] "Push notifications" are a technology that allows you to communicate information to users in real time.

[1708] This invention is a personalized shopping assistant system that analyzes emotional data based on a user's purchase history, usage history, and usage impressions, and suggests optimal products to the user. This system is primarily composed of a server, a terminal (user device), and an emotion analysis engine.

[1709] System Configuration

[1710] Terminal

[1711] A terminal is a user device, such as a smartphone, that uses the following hardware and software:

[1712] Smartphone (using camera and microphone)

[1713] SQLite (as a local database)

[1714] Flutter (mobile application framework)

[1715] Emotion API (for emotion recognition)

[1716] server

[1717] The server processes and learns from the data using the following software:

[1718] Flask (backend framework)

[1719] TensorFlow (for machine learning algorithms)

[1720] Sentiment Analysis Engine

[1721] The sentiment analysis engine uses the Emotion API to analyze the collected emotional data.

[1722] Data processing and proposal flow

[1723] Data collection

[1724] Users enter detailed information about the home appliances and interior items they have purchased into the device, such as the product name, brand, price, purchase location, and purchase date. They also enter their usage history and impressions. The device stores this data in a local database and periodically sends it to the server.

[1725] In addition, the device's built-in camera and microphone are used to capture the user's facial expressions and tone of voice in real time, and emotional data is collected through the Emotion API.

[1726] Data storage and preprocessing

[1727] The server receives data sent from the device using Flask and stores it in a database. It also preprocesses the data to handle missing values ​​and standardize the format.

[1728] Data Analysis and Learning

[1729] The server uses TensorFlow to periodically update the machine learning model and select product candidates based on the user's preferences and emotional state. For example, the following prompt sentence is used:

[1730] Prompt Sentence Examples

[1731] def get_personalized_recommendations(user_data, emotion_data):

[1732] """

[1733] Personalized product recommendations based on user and sentiment data

[1734] """

[1735] from recommendation_engine import RecommendationEngine

[1736] from emotion_analysis import EmotionEngine

[1737] Analyze user and sentiment data

[1738] user_preferences = RecommendationEngine.analyze_user_data(user_data)

[1739] user_emotion = EmotionEngine.analyze_emotion_data(emotion_data)

[1740] Select the best product candidates for the user

[1741] recommended_items = RecommendationEngine.get_recommendations(user_preferences, user_emotion)

[1742] return recommended_items

[1743] For example, it suggests smart ovens based on user preferences and emotions.

[1744] user_data = {

[1745] 'purchase_history': [{'item': 'smart fridge', 'brand': 'BrandA', 'price': 80000}],

[1746] 'usage_feedback': [{'item': 'smart fridge', 'satisfaction': 5}]

[1747] }

[1748] emotion_data = {'joy': 0.8, 'satisfaction': 0.9}

[1749] recommendations = get_personalized_recommendations(user_data, emotion_data)

[1750] print(recommendations)

[1751] Product suggestions and feedback

[1752] The server generates detailed information about the selected product and sends it to the device. The device then uses push notifications to suggest the product to the user and displays the product detail page. The user then purchases the product and enters their usage experience and impressions into the app again. This allows the system to relearn the user's feedback and improve the accuracy of its next suggestions.

[1753] This invention enables highly accurate personalized product suggestions that correspond to the user's emotional state, allowing the user to have a more satisfying shopping experience.

[1754] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1755] Step 1:

[1756] Users enter detailed information about the home appliances and interior items they have purchased into the device. Specifically, they enter data such as the product name, brand, price, purchase location, and purchase date. The entered data is saved in the device's SQLite database, which records the user's purchase history.

[1757] Step 2:

[1758] Users enter their usage history and impressions of the purchased product into the terminal. For example, they enter data such as frequency of use, ease of use, and satisfaction level. This data is also stored in an SQLite database. This records the user's usage history and impressions.

[1759] Step 3:

[1760] The device uses a built-in camera and microphone to capture the user's facial expressions and tone of voice in real time. The captured data is analyzed using the Emotion API to generate emotion data, which records the user's emotional state.

[1761] Step 4:

[1762] The device transmits purchase history, usage history, usage impressions, and emotion data to the server via the network. This data is received by the server-side Flask backend framework and stored in a database.

[1763] Step 5:

[1764] The server preprocesses the received data. Specifically, it processes missing values, standardizes the format, and converts the data into a format that is easy to analyze. This preprocessed data is used to extract user features.

[1765] Step 6:

[1766] The server updates the machine learning model using TensorFlow to select product candidates based on the user's preferences and emotional state. Using the preprocessed data and features as input, the machine learning model learns the user's tendencies and emotional state and generates optimal product candidates.

[1767] Step 7:

[1768] The server generates detailed information about the selected product, including the product name, main features, price, where it can be purchased, and the best time to purchase it. The generated information is sent to the device.

[1769] Step 8:

[1770] The device notifies the user of the received product suggestions, and the user can receive a push notification or an in-app notification to view the details page of the suggested product.

[1771] Step 9:

[1772] The user checks the suggested products and, if they are interested, purchases them. After purchasing, they again enter their impressions and feedback into the device. This feedback data is also stored in the SQLite database.

[1773] Step 10:

[1774] The device sends feedback data to the server, which receives the feedback data and stores it in a database. The stored feedback data is used to retrain the machine learning model and improve the accuracy of the next suggestion.

[1775] This allows for highly accurate product suggestions based on the user's emotional state, allowing the user to have a more satisfying shopping experience.

[1776] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1777] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1778] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1779] [Fourth embodiment]

[1780] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1781] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1782] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1783] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1784] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1785] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1786] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1787] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1788] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1789] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1790] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1791] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1792] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1793] This invention is a personalized shopping assistant system that uses AI to learn the user's style and preferences based on their purchase and usage history and impressions of interior and home appliances, and then suggests the most suitable interior and home appliances.

[1794] System Overview

[1795] This system mainly consists of a server and a terminal (user device). The server stores data, learns, selects products, and makes suggestions, while the terminal is responsible for inputting data and receiving suggestions.

[1796] Data collection

[1797] Enter your purchase history

[1798] User: Through the interface, enter detailed information about the home appliance or interior item purchased (product name, brand, price, purchase location, purchase date).

[1799] Device: Purchase information is stored locally and sent to the server over the network.

[1800] Entering usage history and impressions

[1801] Users: Regularly enter usage, feedback, and satisfaction with purchased products.

[1802] Device: Usage history and user feedback data is stored locally and sent to the server.

[1803] Data storage and analysis

[1804] Receiving and storing data

[1805] Server: Stores purchase history, usage history, and user feedback sent from user devices in a database.

[1806] Data preprocessing and feature extraction

[1807] Server: Preprocesses the received data and extracts user features (preferred brands, price range, features, etc.).

[1808] Machine learning model updates

[1809] Server: Periodically analyzes data using machine learning algorithms to adapt the learning model to reflect user style and preferences.

[1810] Product selection and proposal

[1811] Selecting the right product

[1812] Server: Based on the learning model, the server selects products that match the user's preferences. This selection process also takes into account price, functionality, design, and vendor information.

[1813] Proposal generation and submission

[1814] Server: Generates detailed information about the selected product (product name, key features, price, where it can be purchased, and the best time to purchase).

[1815] Server: Sends the proposals to the user's device.

[1816] Receiving suggestions and feedback

[1817] Receiving Proposals

[1818] Device: Receives suggestions from the server and notifies the user. Notification methods include in-app notifications and push notifications.

[1819] Enter and submit feedback

[1820] User: Purchases the suggested product and enters their impressions and ratings.

[1821] Device: Feedback is stored locally and periodically sent to the server.

[1822] Server: The feedback is stored in a database and used as the basis for retraining.

[1823] Specific examples

[1824] For example, if a user buys a new smart refrigerator and enters details and user experience into the application, the server can:

[1825] 1. Server: Stores purchase data in a database and performs preprocessing together with past data.

[1826] 2. Server: A machine learning model learns user preferences and determines whether a newly released smart oven is suitable for the user.

[1827] 3. Server: Generates detailed information about the smart oven (price, features, vendor, best time to purchase) and sends it to the device.

[1828] 4. On the device: Notify the user of the offer and encourage them to purchase.

[1829] 5. User: Purchases the proposed smart oven and enters their experience using it into the terminal.

[1830] 6. Device: Sends usage information to the server and reflects it in the next learning session.

[1831] In this way, the system continuously learns user trends and continues to suggest appropriate products.

[1832] The processing flow will be explained below.

[1833] Step 1:

[1834] User: When purchasing home appliances or interior items, the user accesses the application and enters their purchase history, including the product name, brand, price, purchase location, and purchase date.

[1835] Step 2:

[1836] Terminal: Purchase information is stored in a local database and sent to a server over the network.

[1837] Step 3:

[1838] Server: Receives purchase information sent from the device and stores it in a central database.

[1839] Step 4:

[1840] User: Enters information about the usage and impressions of the purchased product into the application, such as frequency of use, ease of use, and satisfaction level.

[1841] Step 5:

[1842] Terminal: Usage history and user feedback data is stored in a local database and sent to a server via the network.

[1843] Step 6:

[1844] Server: Receives usage history and feedback sent from the device and stores it in a central database.

[1845] Step 7:

[1846] Server: Preprocesses the purchase history, usage history, and usage impressions stored in the database to make them easier to analyze. Specifically, it handles missing values ​​and standardizes the format of the data.

[1847] Step 8:

[1848] Server: Extracts features from the preprocessed data to identify user preferences and trends, such as preferred brands, price ranges, and features.

[1849] Step 9:

[1850] Server: Uses the features to update the machine learning model and learn about the user's style and preferences.

[1851] Step 10:

[1852] Server: Using a learning model, the server selects product candidates that match the user's preferences, taking into account price, functionality, design, and other factors.

[1853] Step 11:

[1854] Server: Generates detailed information about the selected product, including the product name, key features, price, where it is available, and the best time to purchase.

[1855] Step 12:

[1856] Server: Sends the selected products and their details to the user's device.

[1857] Step 13:

[1858] On the device, the user is notified of the suggestions received from the server via in-app notifications and push notifications.

[1859] Step 14:

[1860] User: Review the suggested products and purchase them if they are interested.

[1861] Step 15:

[1862] Users: Enter post-purchase feedback into the application, including experience and satisfaction.

[1863] Step 16:

[1864] Terminal: The feedback is stored in a local database and sent to a server over the network.

[1865] Step 17:

[1866] Server: Receives feedback sent from devices and stores it in a central database for future analysis and learning.

[1867] Step 18:

[1868] Server: Using the feedback data, the machine learning model is updated again to improve the accuracy of the suggestions.

[1869] Example 1

[1870] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1871] Conventional shopping assistant systems have had the challenge of making recommendations that fully reflect a user's individual preferences and past purchase history. It is also difficult to absorb user feedback in real time and use it in future recommendations. Furthermore, they are unable to provide sufficient recommendations on the best time and place to purchase, limiting the user experience.

[1872] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1873] In this invention, the server includes means for inputting a user's purchase history, usage history, and usage impressions, means for receiving and storing the purchase history, usage history, and usage impressions in a database, means for preprocessing the data stored in the database, means for extracting features from the preprocessed data, machine learning means for learning user tendencies based on the data stored in the database and the extracted features, means for selecting appropriate products based on the user tendencies learned by the machine learning means, means for sending and notifying the user of information about the selected products to the user's terminal, and means for receiving and storing feedback on the notified products in the database. This enables personalized suggestions that reflect the user's individual preferences and past purchase history.

[1874] "Users" are individuals or corporations who use this system to purchase interior goods and home appliances, manage their usage, etc.

[1875] "Purchase history" refers to data on detailed information about interior and home appliance products purchased by a user (product name, brand, price, purchase location, purchase date, etc.).

[1876] "Usage history" refers to data about how users use interior and home appliances they have purchased.

[1877] "Usage impressions" is data that represents the evaluations and opinions that users have about the interior design and home appliances they have used.

[1878] "Means" are the specific methods or tools used to realize the system.

[1879] A "database" is a storage device for centrally managing and storing various data such as purchase history, usage history, and usage impressions.

[1880] "Preprocessing" is the process of correcting or removing inconsistencies or missing data and preparing the data for analysis before data analysis.

[1881] "Features" refers to the numerical representation of user preferences and behavioral tendencies extracted from a database, which are used as input data in machine learning.

[1882] "Machine learning" is an algorithm that allows a computer to learn from past data and make predictions and decisions based on future data.

[1883] "Selection" is the process of using a machine learning model to determine the product candidates that best suit the user's preferences.

[1884] "Terminal" means the device (e.g., smartphone, tablet, PC) that a user uses to interact with the system.

[1885] "Notification" refers to the act of transmitting information from a server to a user, and includes in-app notifications and push notifications.

[1886] "Feedback" refers to information about users' impressions and evaluations of the proposed product after using it.

[1887] This invention is a personalized shopping assistant system in which a generative AI learns a user's style and preferences based on their purchase history, usage history, and impressions of interior and home appliances, and suggests the most suitable interior and home appliances. This system is mainly composed of a server and a terminal (user device).

[1888] composition

[1889] server

[1890] The server serves as a central location for receiving, storing, pre-processing, and machine learning data. It has the following main functions:

[1891] Purchase history, usage history, and usage feedback are received and stored in a database.

[1892] Preprocess the received data and extract features.

[1893] Based on the features, machine learning algorithms (e.g., random forests, neural networks) are used to learn user tendencies.

[1894] Based on the learning results, the system selects the most suitable product for the user and generates detailed information about it.

[1895] The selected product information is sent to the user's device.

[1896] Terminal

[1897] A terminal is a device that a user uses to enter data and receive notifications from a server. Its main functions are:

[1898] Provides an interface for inputting purchase history, usage history, and usage impressions.

[1899] The user is notified of the proposal information received from the server.

[1900] Data processing and calculation

[1901] Data collection

[1902] User: Using a dedicated app or web portal on a smartphone or PC, enter your purchase history (product name, brand, price, purchase location, purchase date). Example: Value Range Refrigerator, AB manufacturer, 50,000 yen, XYZ store, October 1, 2023.

[1903] Terminal: The information entered by the user is temporarily stored in a local database and sent to a server via the Internet.

[1904] User: Enter usage status, impressions, and satisfaction level of the purchased product. Example: Good cooling performance, excellent design.

[1905] Data storage and analysis

[1906] Server: Stores the received purchase history, usage history, and user feedback in a database.

[1907] Server: Preprocesses the received data, corrects missing data, and removes inconsistent data. Then, extracts features based on user preferences and behavioral trends.

[1908] Server: Uses machine learning algorithms based on features to learn user trends.

[1909] Product selection and proposal

[1910] Server: Selects the best product for the user based on learned trends, taking into account information such as price, functionality, design, and vendor.

[1911] Server: Generates detailed information about the selected product (product name, key features, price, where it can be purchased, and the best time to purchase) and sends it to the user's device.

[1912] On the device: Notify the user of the received recommendation. Example: Display an in-app notification saying, "Our new smart oven is perfect for you!"

[1913] Enter and submit feedback

[1914] User: Purchases the suggested product and enters their impressions and ratings into the device.

[1915] Device: Stores feedback data locally and sends it to the server.

[1916] Server: Receives feedback and stores it in a database for future machine learning.

[1917] Specific examples

[1918] Prompt Sentence Examples

[1919] "Please tell us the details of the home appliance you recently purchased (product name, brand, price, place of purchase, purchase date)."

[1920] "Please tell us about your experience using and satisfaction with the home appliances you purchased."

[1921] "Tell me about the next interior or home appliance you're considering purchasing."

[1922] These prompts can be used to gather the necessary data from the user and provide optimal product recommendations.

[1923] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1924] Step 1:

[1925] Input: User enters purchase history through application or web portal. For example: product name, brand, price, purchase location, purchase date.

[1926] How it works: Users use the interface on their smartphone or computer to enter detailed information about the home appliances and interior items they have purchased.

[1927] Output: Purchase information is stored on the device and data is generated that is later sent to a server.

[1928] Step 2:

[1929] Input: Purchase history data and details entered by the user.

[1930] How it works: The device temporarily stores the purchase information in a local database and then sends it over the internet to a server, periodically retrying the transmission until it is successful.

[1931] Output: Purchase history data sent to the server.

[1932] Step 3:

[1933] Input: Purchase history data received by the server.

[1934] How it works: The server receives the purchase history sent from the device and stores it in a database.

[1935] Output: Purchase history stored in a database.

[1936] Step 4:

[1937] Input: User-entered usage history data such as usage status, impressions, and satisfaction.

[1938] How it works: Users periodically enter their impressions and satisfaction with the device. Examples include "This refrigerator has great cooling performance" and "It has a nice design."

[1939] Output: Usage history and impressions are stored on the device and data is generated that is later sent to the server.

[1940] Step 5:

[1941] Input: Usage history and user feedback data.

[1942] How it works: The device stores this data locally and sends it to the server.

[1943] Output: Usage history and user feedback data sent to the server.

[1944] Step 6:

[1945] Input: Usage history and user feedback data received by the server.

[1946] Operation: The server receives the usage history and usage impressions sent from the device and stores them in a database.

[1947] Output: Usage history and user feedback data stored in a database.

[1948] Step 7:

[1949] Input: Purchase history, usage history, and user feedback data stored in the database.

[1950] How it works: The server preprocesses the received data, correcting or removing inconsistencies and missing data, and then extracts features that quantify user preferences and behavioral trends.

[1951] Output: Preprocessed data and extracted features.

[1952] Step 8:

[1953] Input: Preprocessed data and extracted features.

[1954] How it works: Using machine learning algorithms based on features, it learns user tendencies.

[1955] Output: An updated machine learning model.

[1956] Step 9:

[1957] Input: The machine learning model obtained by training.

[1958] How it works: The server uses a learning model to recommend products that match the user's preferences. Example: "The new smart oven is perfect for you."

[1959] Output: Suitable products and their details (price, key features, where available, best time to buy, etc.).

[1960] Step 10:

[1961] Input: Detailed information about the selected product.

[1962] How it works: The server generates detailed information about the selected product and sends it to the user's device.

[1963] Output: The suggestions sent to the user's device.

[1964] Step 11:

[1965] Input: Proposal notification.

[1966] How it works: The device receives the offer from the server and notifies the user. Notification methods include in-app notifications and push notifications. Example: "Our new smart oven is perfect for you!"

[1967] Output: A notification showing the suggestion.

[1968] Step 12:

[1969] Input: User's impressions and ratings of the proposed product.

[1970] How it works: The user uses the suggested product and enters their impressions and ratings.

[1971] Output: Feedback data is generated and saved to the device.

[1972] Step 13:

[1973] Input: Feedback data.

[1974] How it works: The device stores the feedback data locally and sends it to the server.

[1975] Output: Feedback data sent to the server.

[1976] Step 14:

[1977] Input: The feedback data received by the server.

[1978] How it works: The server receives the feedback data and stores it in a database. It is used for retraining.

[1979] Output: Feedback data stored in a database.

[1980] (Application example 1)

[1981] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1982] Conventional personalized shopping assistant systems have had problems such as limited product suggestions based on users' purchase history and usage impressions, being available only in specific locations, or having poor visual suggestions. Furthermore, there was a lack of means for users to check the experience of the suggested products in advance, and there were limitations on providing information on the optimal time and place to purchase, which could lead to a decrease in user satisfaction.

[1983] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1984] In this invention, the server includes a means for inputting a user's purchase history, usage history, and usage impressions, a means for receiving the purchase history, usage history, and usage impressions and storing them in a database, and a means for preprocessing the data stored in the database and extracting features. This makes it possible to learn user trends, suggest optimal products, visually display them in a virtual store, and suggest where they can be purchased and the optimal time to purchase them.

[1985] "User" means any individual or legal entity using a particular product or service.

[1986] "Purchase history" refers to information about products and services a user has purchased in the past.

[1987] "Usage history" refers to data about how a user uses a purchased product or service.

[1988] "Usage impressions" refers to the impressions and evaluations that users have after using a product or service.

[1989] "Input means" refers to the interface or function that allows users to input data into the system.

[1990] A "database" refers to a system that systematically stores information and makes it available for searching and use as needed.

[1991] "Preprocessing" refers to a series of steps that convert collected data into an analyzable format.

[1992] "Features" refer to useful attributes or parameters extracted from data.

[1993] "Machine learning methods" refer to techniques that allow computers to learn from data and recognize patterns.

[1994] "Goods" means any products or services intended for purchase or use by Users.

[1995] A "virtual store" refers to an online shopping environment created using virtual reality technology.

[1996] "Visual presentation means" refers to techniques or methods for presenting information graphically to a user.

[1997] "Feedback" refers to the ratings and opinions users provide about products and services.

[1998] The "optimal time to purchase" refers to the time period that encourages users to make a purchase at the most advantageous time.

[1999] This invention is a personalized shopping assistant system that uses AI to learn the user's style and preferences based on their purchase and usage history and impressions of interior and home appliances, and then suggests the most suitable products. The system is mainly composed of a server and a terminal (user device).

[2000] System Overview

[2001] Data collection

[2002] Users enter their purchase history and usage impressions via the terminal. They enter detailed information about the purchased product, usage status, impressions, and satisfaction level, and the terminal collects this information. The collected data is sent to the server via the network.

[2003] Data storage and analysis

[2004] The server stores the received purchase history, usage history, and usage impressions in a database. The stored data is preprocessed to extract user features (preferred brands, price ranges, features, etc.). The features are analyzed using a machine learning model to learn the user's style and preferences.

[2005] Product selection and proposal

[2006] The server selects products that match the user's preferences based on the learning model. This selection process also takes into account price, functionality, design, and vendor information. The selected product information is visually displayed in a virtual store.

[2007] Program processing

[2008] 1. Data Acquisition and Transmission

[2009] The device (such as the user's VR headset or smart glasses) collects purchase history and usage impressions and sends them to the server. The hardware used includes Oculus as a VR headset and Google Glass as smart glasses.

[2010] 2. Data analysis and proposal generation

[2011] The server uses Python's Flask framework to analyze the received data, and uses machine learning models such as TensorFlow to learn the user's preferences and select appropriate products. This selected product information is stored in a database and then sent back to the device.

[2012] 3. Virtual store display

[2013] The device uses software such as Unity or Unreal Engine to visually display product information in a virtual environment, allowing users to move around the virtual store and examine the suggested products in detail.

[2014] Specific examples

[2015] For example, if a user purchases a new smart refrigerator and enters details and user experience into an application, the flow would be as follows:

[2016] Users enter information about the smart refrigerator they have purchased and their experience using it into the application.

[2017] The terminal collects this data and sends it to the server.

[2018] The server receives and stores the data and uses machine learning models to learn user preferences.

[2019] The server then selects a new smart oven that is suitable for the user and generates detailed information about it.

[2020] The terminal visually presents the proposed smart oven to the user in a virtual reality environment.

[2021] Users can view suggested smart ovens in a virtual store and consider purchasing them.

[2022] Prompt Sentence Examples

[2023] user_id: "12345"

[2024] purchased_items:

[2025] item: "Smart Refrigerator"

[2026] brand: "BrandA"

[2027] price: 50000

[2028] purchase_date: "2023-08-01"

[2029] item: "Washing machine"

[2030] brand: "BrandB"

[2031] price: 30000

[2032] purchase_date: "2023-01-15"

[2033] user_feedback:

[2034] item: "Smart Refrigerator"

[2035] Rating: 5

[2036] comment: "Very useful"

[2037] item: "Washing machine"

[2038] Rating: 4

[2039] comment: "Very satisfied"

[2040] In this way, the system suggests optimal products based on the user's purchase history and usage impressions, enabling a visual experience in a virtual store.

[2041] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2042] Step 1:

[2043] Users enter their purchase history and usage experience, including details such as product name, brand, price, purchase location, and purchase date. This data is collected and stored locally on the device (e.g., smart glasses or VR headset). The input data format is as follows:

[2044] Input: Product name, brand, price, purchase location, purchase date

[2045] Output: Locally stored purchase and usage data

[2046] Step 2:

[2047] The device sends the collected purchase history and usage feedback to a server via the network, using the Oculus SDK and Google Glass API as interfaces.

[2048] Input: Locally stored purchase and usage data

[2049] Output: Data sent to the server

[2050] Step 3:

[2051] The server stores the received purchase history, usage history, and usage feedback in a database. After storing this data, it is preprocessed for analysis.

[2052] Input: Submitted purchase history, usage history, and user impressions

[2053] Output: Data stored in the database

[2054] Step 4:

[2055] The server preprocesses the data and extracts user features, including cleaning, normalizing, handling missing values, and performing feature engineering.

[2056] Input: Raw data stored in a database

[2057] Output: Extracted features

[2058] Step 5:

[2059] The server inputs the features into a machine learning model (e.g., TensorFlow) to learn the user's style and preferences. The model recognizes and learns patterns based on the user's past data.

[2060] Input: Extracted features

[2061] Output: A learned model containing user preference patterns

[2062] Step 6:

[2063] The server then uses the learning model to select products that suit the user's preferences, taking into account product price, functionality, design, and vendor information.

[2064] Input: User preference patterns

[2065] Output: A list of recommended products

[2066] Step 7:

[2067] The server generates detailed information about the selected product (product name, main features, price, distributor, best time to purchase) and sends it to the terminal.

[2068] Input: A list of recommended products

[2069] Output: Proposal to the device

[2070] Step 8:

[2071] The device visually displays the received recommendations in a virtual store. Users can view the suggested products in the virtual space using a VR headset or smart glasses. The system is displayed using Unity or Unreal Engine.

[2072] Input: Proposal

[2073] Output: Visual representation of the virtual store

[2074] Step 9:

[2075] Users can view the proposed products in the virtual store and provide feedback, which is collected at the terminal and sent to the server.

[2076] Input: User feedback

[2077] Output: Feedback data to the server

[2078] Step 10:

[2079] The server stores the received feedback in a database and uses it to update the machine learning model, thereby improving the accuracy of the suggestions.

[2080] Input: User feedback data

[2081] Output: Updated training model

[2082] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2083] This is a personalized shopping assistant system that uses a generation AI to learn the user's style and preferences based on their purchase and usage history, impressions, and emotional data on interior and home appliances, and then suggests the most suitable interior and home appliances. In particular, by combining it with an emotion engine, it is possible to suggest products that take into account the user's emotional state, providing a more personalized service.

[2084] System Overview

[2085] This system is mainly composed of a server, a terminal (user device), and an emotion engine. The server stores data, learns, selects products, and makes recommendations, while the terminal is responsible for inputting data and receiving recommendations. The emotion engine collects and analyzes user emotion data and sends it to the server.

[2086] Data collection

[2087] Enter your purchase history

[2088] User: Through the interface, the user enters detailed information about the home appliance or interior item they purchased, such as the product name, brand, price, purchase location, and purchase date.

[2089] Terminal: Purchase information is stored in a local database and sent to a server over the network.

[2090] Entering usage history and impressions

[2091] Users: Periodically enter information about the usage, impressions, and satisfaction of purchased products, such as frequency of use, ease of use, and satisfaction level.

[2092] Device: Usage history and user feedback data is stored in a local database and sent to the server.

[2093] Collecting Emotional Data

[2094] On-device: Collects real-time emotional data from the user's facial expressions, tone of voice, and input text. This data is analyzed through an emotion engine.

[2095] Emotion engine: Analyzes collected emotion data to identify the user's emotional state, specifically identifying emotions such as joy, sadness, anger, and surprise.

[2096] Emotion engine: Sends the emotion analysis results to the server.

[2097] Data storage and analysis

[2098] Receiving and storing data

[2099] Server: Stores the purchase history, usage history, usage impressions, and emotion data sent from the device and emotion engine in a database.

[2100] Data preprocessing and feature extraction

[2101] Server: Preprocesses the received data to make it easier to analyze. Specifically, it handles missing values ​​and standardizes the format of the data. Emotion data is also preprocessed.

[2102] Server: Extract user features (preferred brands, price range, features, emotional state, etc.) from the preprocessed data.

[2103] Machine learning model updates

[2104] Server: Periodically analyzes the data using machine learning algorithms to adapt the learning model to reflect user style and preferences. Emotional data is also incorporated into the learning process.

[2105] Product selection and proposal

[2106] Selecting the right product

[2107] Server: Based on the learning model, the server selects product candidates that match the user's preferences. This selection process takes into account price, functionality, design, and emotional data.

[2108] Proposal generation and submission

[2109] Server: Generates detailed information about the selected product, including the product name, key features, price, where it is available, and the best time to purchase.

[2110] Server: Sends the selected products and their details to the user's device.

[2111] Receiving suggestions and feedback

[2112] Receiving Proposals

[2113] Device: Receives suggestions from the server and notifies the user via in-app notifications and push notifications.

[2114] Enter and submit feedback

[2115] User: Review the suggested products, purchase them if they are interested, and enter feedback into the application after purchase, including their experience and satisfaction.

[2116] Terminal: The feedback is stored in a local database and sent to a server over the network.

[2117] Server: The feedback sent from the device is stored in a database and used as the basis for retraining.

[2118] Server: Using the feedback data, the machine learning model is updated again to improve the accuracy of the suggestions.

[2119] Specific examples

[2120] For example, a user purchases a new smart refrigerator and enters its details and their experience in the application. Furthermore, the emotion engine collects emotional data such as the user's happiness and satisfaction while using the refrigerator. Based on this data, the server performs the following actions:

[2121] 1. Server: Stores purchase data and emotion data in a database and performs preprocessing together with past data.

[2122] 2. Server: A machine learning model learns the user's preferences and emotional state and determines whether a newly released smart oven is suitable for the user.

[2123] 3. Server: Generates detailed information about the smart oven (price, features, vendor, best time to purchase) and sends it to the device.

[2124] 4. On the device: Notify the user of the offer and encourage them to purchase.

[2125] 5. User: Purchases the proposed smart oven and enters their experience using it into the terminal.

[2126] 6. Device: Sends usage information to the server and reflects it in the next learning session.

[2127] In this way, the system can make more accurate suggestions based on the user's emotional data, allowing the user to select the most appropriate product based on their emotional state.

[2128] The processing flow will be explained below.

[2129] Step 1:

[2130] User: When purchasing home appliances or interior items, the user accesses the application and enters their purchase history, including the product name, brand, price, purchase location, and purchase date.

[2131] Step 2:

[2132] Terminal: Purchase information is stored in a local database and sent to a server over the network.

[2133] Step 3:

[2134] Server: Receives purchase information sent from the device and stores it in a central database.

[2135] Step 4:

[2136] User: Enters information about the usage and impressions of the purchased product into the application, such as frequency of use, ease of use, and satisfaction level.

[2137] Step 5:

[2138] Terminal: Usage history and user feedback data is stored in a local database and sent to a server via the network.

[2139] Step 6:

[2140] Server: Receives usage history and feedback sent from the device and stores it in a central database.

[2141] Step 7:

[2142] On-device: Collects real-time emotional data from the user's facial expressions, tone of voice, and input text. Analyzes emotions using facial recognition and voice analysis algorithms.

[2143] Step 8:

[2144] Emotion engine: Analyzes collected emotion data to identify the user's emotional state, specifically identifying emotions such as joy, sadness, anger, and surprise.

[2145] Step 9:

[2146] Emotion engine: Sends the emotion analysis results to the server.

[2147] Step 10:

[2148] Server: Stores the purchase history, usage history, usage impressions, and emotion data sent from the device and emotion engine in a database.

[2149] Step 11:

[2150] Server: Preprocesses the received data to make it easier to analyze, specifically by handling missing values ​​and standardizing the data format.

[2151] Step 12:

[2152] Server: Extract user features (preferred brands, price range, features, emotional state, etc.) from the preprocessed data.

[2153] Step 13:

[2154] Server: Uses the features to update the machine learning model to learn about the user's style and preferences, incorporating sentiment data into the learning process.

[2155] Step 14:

[2156] Server: Based on the learning model, it selects product candidates that match the user's preferences, taking into account price, functionality, design, and emotional data.

[2157] Step 15:

[2158] Server: Generates detailed information about the selected product, including the product name, key features, price, where it is available, and the best time to purchase.

[2159] Step 16:

[2160] Server: Sends the selected products and their details to the user's device.

[2161] Step 17:

[2162] On the device, the user is notified of the suggestions received from the server via in-app notifications and push notifications.

[2163] Step 18:

[2164] User: Review the suggested products and purchase them if they are interested.

[2165] Step 19:

[2166] Users: Enter post-purchase feedback into the application, including experience and satisfaction.

[2167] Step 20:

[2168] Terminal: The feedback is stored in a local database and sent to a server over the network.

[2169] Step 21:

[2170] Server: Receives feedback sent from devices and stores it in a central database for future analysis and learning.

[2171] Step 22:

[2172] Server: Using the feedback data, the machine learning model is updated again to improve the accuracy of the suggestions.

[2173] In this way, the system can make more accurate suggestions based on the user's emotional data, allowing the user to select the most appropriate product based on their emotional state.

[2174] Example 2

[2175] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2176] Traditional personalized shopping assistant systems suggest products based on a user's purchase history, usage history, and usage impressions. However, these systems do not take into account the user's emotional state, making it difficult to make optimal product suggestions based on each individual user's emotions. Furthermore, the process by which users provide feedback on suggested products is not sufficiently considered, limiting the improvement of suggestion accuracy. This makes it difficult to make effective product suggestions that increase user satisfaction.

[2177] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2178] In this invention, the server includes means for receiving and storing a user's purchase history, usage history, usage impressions, and user emotional data in a database, machine learning means for learning user tendencies based on the data stored in the database, means for selecting appropriate products based on the user tendencies and emotional state learned by the machine learning means, and means for receiving the notified product feedback and emotional data and storing them in the database. This enables personalized product suggestions that take the user's emotional state into consideration, and further enables the accuracy of suggestions to be continuously improved based on feedback data from the user.

[2179] "Purchase history" refers to detailed information about products purchased by a user in the past, including product name, brand, price, purchase location, and purchase date.

[2180] "Usage history" refers to information about how a user uses a purchased product, including, for example, frequency of use, duration of use, and purpose of use.

[2181] "Usage impressions" refers to the impressions and evaluations that users have of the products they have purchased, including satisfaction, ease of use, and evaluation of design.

[2182] "Emotional data" refers to data that represents a user's emotional state, including facial expressions, tone of voice, and emotional information derived from input text.

[2183] "Means of storage" refers to the technology and methods used to record input data in a database.

[2184] "Machine learning methods" refer to algorithms that analyze data and learn patterns and trends based on the results, which can then predict user preferences and behavioral patterns.

[2185] "Means of selection" refers to the technology or method of selecting the optimal product from multiple options based on the results obtained through machine learning.

[2186] "Means of Notification" refers to the technology or method used to notify users of information about the selected product, including in-app notifications and push notifications.

[2187] "Means for receiving feedback" refers to the technology and methods for receiving product ratings and impressions provided by users.

[2188] "Preprocessing means" refers to techniques and methods for initial processing of collected data to make it easier to analyze. Specifically, this includes imputing missing values ​​and standardizing data formats.

[2189] "Means for extracting features" refers to techniques and methods for extracting important features and patterns from preprocessed data that can be used for analysis.

[2190] This invention is a personalized shopping assistant system that uses a generation AI to learn the user's style and preferences based on the user's purchase history, usage history, usage impressions, and emotional data for interior and home appliances, and then suggests the most suitable interior and home appliances.The invention uses a server, a terminal (user device), and an emotion engine.

[2191] System Overview

[2192] The system's main components are a server, a terminal, and an emotion engine. The server stores data, learns, selects products, and makes suggestions. The terminal is responsible for inputting data and receiving suggestions from users, while the emotion engine collects and analyzes users' emotion data and sends it to the server.

[2193] Data collection

[2194] 1. Enter your purchase history

[2195] Using the interface, users input detailed information about the home appliances and interior items they have purchased, such as the product name, brand, price, purchase location, and purchase date. The device stores this data in a local database and transmits it to a server via the network.

[2196] 2. Enter your usage history and impressions

[2197] Users periodically enter information about the usage, impressions, and satisfaction of purchased products, such as frequency of use, ease of use, and satisfaction level. The device stores this data in a local database and transmits it to the server.

[2198] 3. Collecting Emotional Data

[2199] The device collects real-time emotional data from the user's facial expressions, tone of voice, and input text. The emotion engine analyzes this data to identify the user's emotional state. Specifically, it identifies emotions such as joy, sadness, anger, and surprise. The analysis results are sent to the server.

[2200] Data storage and analysis

[2201] 1. Data storage

[2202] The server stores the purchase history, use history, use impressions, and emotion data transmitted from the terminal and the emotion engine in a database.

[2203] 2. Data preprocessing and feature extraction

[2204] The server preprocesses the data it receives to make it easier to analyze. Specifically, it complements missing data and standardizes data formats. From the preprocessed data, it extracts user features (preferred brands, price ranges, features, emotional state, etc.).

[2205] 3. Updating the machine learning model

[2206] The server analyzes the data using machine learning algorithms, incorporating user style and preferences into the learning model, with emotional data also being incorporated into the learning process.

[2207] Product selection and proposal

[2208] 1. Selecting the right product

[2209] The server uses the learning model to select product candidates that match the user's preferences. This selection process takes into account price, functionality, design, and emotional data.

[2210] 2. Generate and submit proposals

[2211] The server generates detailed information about the selected product, including the product name, key features, price, where it can be purchased, and the best time to purchase it. The server then sends this information to the user's device.

[2212] Receiving feedback and relearning

[2213] 1. Receiving Proposals

[2214] The device receives the suggestion from the server and notifies the user via in-app notifications or push notifications.

[2215] 2. Enter and submit your feedback

[2216] If the user checks the suggested product and purchases it, they input feedback such as their experience with it and their level of satisfaction. The device stores this feedback in a local database and sends it to the server.

[2217] 3. Relearn

[2218] The server stores the feedback data in a database and uses it as the basis for re-learning.The server then uses the feedback data to update the machine learning model again and improve the accuracy of the suggestions.

[2219] Specific examples

[2220] For example, if a user purchases a new smart refrigerator and enters its details and their experience in the application, the emotion engine will collect emotions such as joy and satisfaction during use. Based on this data, the server will:

[2221] 1. The server stores purchase data and emotion data in a database and performs preprocessing together with past data.

[2222] 2. The server uses a machine learning model to learn the user's preferences and emotional state and determines that a newly released smart oven is suitable for the user.

[2223] 3. The server generates detailed information about the smart oven (price, features, vendor, best time to purchase) and sends it to the device.

[2224] 4. The device notifies the user of the offer and encourages them to make a purchase.

[2225] 5. The user purchases the recommended smart oven and enters their experience using it into the terminal.

[2226] 6. The device sends the usage experience to the server and it is reflected in the next learning session.

[2227] In this way, the system can make more accurate suggestions based on the user's emotional data, allowing the user to select the most suitable product based on their emotional state.

[2228] Prompt Sentence Examples

[2229] "After you purchase your new smart refrigerator, please tell us your experience and satisfaction. We will use that information to recommend your next product."

[2230] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2231] The flow of this system's program processing

[2232] Step 1:

[2233] Input: The user enters detailed information about the home appliance or interior item they purchased (product name, brand, price, purchase location, purchase date, etc.) into the interface.

[2234] Processing: The device stores these purchase information in a local database and sends it to the server over the network.

[2235] Output: The server stores the received purchase data in a database.

[2236] Step 2:

[2237] Input: Users periodically input information about their usage and impressions of the products they have purchased (frequency of use, ease of use, satisfaction, etc.).

[2238] Processing: The device stores this usage history and usage impression data in a local database and sends it to the server.

[2239] Output: The server stores the received usage history and user feedback data in a database.

[2240] Step 3:

[2241] Input: Obtain emotional data such as facial expressions, tone of voice, and input text when users use home appliances and interior items.

[2242] Processing: The device collects these emotion data in real time and sends them to the emotion engine, which analyzes the collected emotion data and identifies the user's emotional state.

[2243] Output: The emotion engine sends the analysis results to the server, which stores them in a database.

[2244] Step 4:

[2245] Input: The server stores the purchase history, usage history, usage impressions, and emotion data received from the device and emotion engine in a database.

[2246] Processing: The server preprocesses the data and standardizes it into a format that is easy to analyze. Specifically, it complements missing data and reformats the data format.

[2247] Output: Save the preprocessed data to a database.

[2248] Step 5:

[2249] Input: Preprocessed purchase history, usage history, usage impressions, and sentiment data.

[2250] Processing: The server extracts features (preferred brand, price range, features, emotional state, etc.) from the preprocessed data.

[2251] Output: Extracted feature data.

[2252] Step 6:

[2253] Input: Feature data.

[2254] Processing: The server uses machine learning algorithms to learn about the user's style and preferences and update the model. Emotional data is also incorporated into the learning process.

[2255] Output: The updated training model.

[2256] Step 7:

[2257] Input: The updated training model.

[2258] Processing: The server uses the learning model to select product candidates that match the user's preferences. This selection process takes into account price, functionality, design, and emotional data.

[2259] Output: Selected product information.

[2260] Step 8:

[2261] Input: Selected product information.

[2262] Processing: The server generates detailed information about the selected product (product name, key features, price, where it is available, best time to buy, etc.).

[2263] Output: Detailed product information.

[2264] Step 9:

[2265] Input: Product details.

[2266] Processing: The server sends product information to the user's device. The device receives the offer from the server and notifies the user. Notification methods include in-app notifications and push notifications.

[2267] Output: Notification to the user.

[2268] Step 10:

[2269] Input: Notification from the server.

[2270] Processing: If the user checks the suggested product and purchases it, they enter feedback into the application, such as their experience and satisfaction with it. The device stores this feedback in a local database and sends it to the server.

[2271] Output: Feedback data.

[2272] Step 11:

[2273] Input: Feedback data.

[2274] Processing: The server saves the feedback data in a database and uses it as the basis for re-learning. The server then uses the newly input feedback data to update the machine learning model again, improving the accuracy of the suggestions.

[2275] Output: The updated training model.

[2276] (Application example 2)

[2277] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2278] Conventional shopping assistant systems typically suggest products based on a user's purchase and usage history of interior design and home appliances. However, this approach fails to consider the user's emotional state and does not necessarily suggest products that best fit the user's current needs and preferences. Furthermore, the accuracy of product selection is limited, creating a demand for more personalized suggestions. To address this, product suggestions that take the user's emotional state into account are necessary. Therefore, the present invention aims to provide more accurate personalized product suggestions by analyzing the user's emotional data and making product suggestions based on that information.

[2279] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for inputting a user's purchase history, usage history, and usage impressions; means for receiving and storing the purchase history, usage history, and usage impressions in a database; machine learning means for learning user tendencies based on the data stored in the database; means for selecting appropriate products based on the user tendencies learned by the machine learning means; means for collecting and analyzing emotional data from the user's facial expressions, tone of voice, and input text; an emotion analysis engine for identifying the user's emotional state based on the collected emotional data; means for incorporating the results of the emotion analysis engine into the machine learning means and suggesting products taking the user's emotional state into consideration; means for notifying the user of information about the selected products; and means for receiving and storing feedback on the notified products in the database. This enables highly accurate personalized product suggestions based on the user's emotional state.

[2280] "Purchase history" refers to information about products and services purchased by a user in the past.

[2281] "Usage history" is a detailed record of how a user uses purchased products and services.

[2282] "Usage impressions" refers to the impressions and evaluations that users have about the products or services they have purchased.

[2283] A "database" is a system for efficiently organizing, managing, and storing data within a computer system.

[2284] "Machine learning methods" are techniques that use computer programs to learn patterns from data and then make predictions or classifications based on that knowledge.

[2285] "Emotional data" is data that represents the user's emotional state, and is information collected from facial expressions, tone of voice, input text, etc.

[2286] An "emotion analysis engine" is software or a system for analyzing collected emotional data and identifying a user's emotional state.

[2287] "Product suggestion" is the act of selecting the most suitable product based on user data and recommending it to the user.

[2288] "Feedback" refers to the evaluations and impressions that users give about the products and services provided.

[2289] "Notification means" refers to a method or device for conveying information about the selected product to the user.

[2290] "Preprocessing" refers to the initial processing used to convert data into a form that is easier to analyze.

[2291] A "feature" is an important element in the data input into a machine learning model for making predictions and classifications.

[2292] "Style" refers to the design and functional characteristics that users choose according to their particular trends and preferences.

[2293] "Push notifications" are a technology that allows you to communicate information to users in real time.

[2294] This invention is a personalized shopping assistant system that analyzes emotional data based on a user's purchase history, usage history, and usage impressions, and suggests optimal products to the user. This system is primarily composed of a server, a terminal (user device), and an emotion analysis engine.

[2295] System Configuration

[2296] Terminal

[2297] A terminal is a user device, such as a smartphone, that uses the following hardware and software:

[2298] Smartphone (using camera and microphone)

[2299] SQLite (as a local database)

[2300] Flutter (mobile application framework)

[2301] Emotion API (for emotion recognition)

[2302] server

[2303] The server processes and learns from the data using the following software:

[2304] Flask (backend framework)

[2305] TensorFlow (for machine learning algorithms)

[2306] Sentiment Analysis Engine

[2307] The sentiment analysis engine uses the Emotion API to analyze the collected emotional data.

[2308] Data processing and proposal flow

[2309] Data collection

[2310] Users enter detailed information about the home appliances and interior items they have purchased into the device, such as the product name, brand, price, purchase location, and purchase date. They also enter their usage history and impressions. The device stores this data in a local database and periodically sends it to the server.

[2311] In addition, the device's built-in camera and microphone are used to capture the user's facial expressions and tone of voice in real time, and emotional data is collected through the Emotion API.

[2312] Data storage and preprocessing

[2313] The server receives data sent from the device using Flask and stores it in a database. It also preprocesses the data to handle missing values ​​and standardize the format.

[2314] Data Analysis and Learning

[2315] The server uses TensorFlow to periodically update the machine learning model and select product candidates based on the user's preferences and emotional state. For example, the following prompt sentence is used:

[2316] Prompt Sentence Examples

[2317] def get_personalized_recommendations(user_data, emotion_data):

[2318] """

[2319] Personalized product recommendations based on user and sentiment data

[2320] """

[2321] from recommendation_engine import RecommendationEngine

[2322] from emotion_analysis import EmotionEngine

[2323] Analyze user and sentiment data

[2324] user_preferences = RecommendationEngine.analyze_user_data(user_data)

[2325] user_emotion = EmotionEngine.analyze_emotion_data(emotion_data)

[2326] Select the best product candidates for the user

[2327] recommended_items = RecommendationEngine.get_recommendations(user_preferences, user_emotion)

[2328] return recommended_items

[2329] For example, it suggests smart ovens based on user preferences and emotions.

[2330] user_data = {

[2331] 'purchase_history': [{'item': 'smart fridge', 'brand': 'BrandA', 'price': 80000}],

[2332] 'usage_feedback': [{'item': 'smart fridge', 'satisfaction': 5}]

[2333] }

[2334] emotion_data = {'joy': 0.8, 'satisfaction': 0.9}

[2335] recommendations = get_personalized_recommendations(user_data, emotion_data)

[2336] print(recommendations)

[2337] Product suggestions and feedback

[2338] The server generates detailed information about the selected product and sends it to the device. The device then uses push notifications to suggest the product to the user and displays the product detail page. The user then purchases the product and enters their usage experience and impressions into the app again. This allows the system to relearn the user's feedback and improve the accuracy of its next suggestions.

[2339] This invention enables highly accurate personalized product suggestions that correspond to the user's emotional state, allowing the user to have a more satisfying shopping experience.

[2340] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2341] Step 1:

[2342] Users enter detailed information about the home appliances and interior items they have purchased into the device. Specifically, they enter data such as the product name, brand, price, purchase location, and purchase date. The entered data is saved in the device's SQLite database, which records the user's purchase history.

[2343] Step 2:

[2344] Users enter their usage history and impressions of the purchased product into the terminal. For example, they enter data such as frequency of use, ease of use, and satisfaction level. This data is also stored in an SQLite database. This records the user's usage history and impressions.

[2345] Step 3:

[2346] The device uses a built-in camera and microphone to capture the user's facial expressions and tone of voice in real time. The captured data is analyzed using the Emotion API to generate emotion data, which records the user's emotional state.

[2347] Step 4:

[2348] The device transmits purchase history, usage history, usage impressions, and emotion data to the server via the network. This data is received by the server-side Flask backend framework and stored in a database.

[2349] Step 5:

[2350] The server preprocesses the received data. Specifically, it processes missing values, standardizes the format, and converts the data into a format that is easy to analyze. This preprocessed data is used to extract user features.

[2351] Step 6:

[2352] The server updates the machine learning model using TensorFlow to select product candidates based on the user's preferences and emotional state. Using the preprocessed data and features as input, the machine learning model learns the user's tendencies and emotional state and generates optimal product candidates.

[2353] Step 7:

[2354] The server generates detailed information about the selected product, including the product name, main features, price, where it can be purchased, and the best time to purchase it. The generated information is sent to the device.

[2355] Step 8:

[2356] The device notifies the user of the received product suggestions, and the user can receive a push notification or an in-app notification to view the details page of the suggested product.

[2357] Step 9:

[2358] The user checks the suggested products and, if they are interested, purchases them. After purchasing, they again enter their impressions and feedback into the device. This feedback data is also stored in the SQLite database.

[2359] Step 10:

[2360] The device sends feedback data to the server, which receives the feedback data and stores it in a database. The stored feedback data is used to retrain the machine learning model and improve the accuracy of the next suggestion.

[2361] This allows for highly accurate product suggestions based on the user's emotional state, allowing the user to have a more satisfying shopping experience.

[2362] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2363] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2364] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2365] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2366] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2367] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2368] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2369] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2370] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2371] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2372] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2373] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2374] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2375] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2376] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2377] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with...

Claims

1. A means for inputting the user's purchase history, usage history, and usage impressions; means for receiving and storing the purchase history, usage history, and usage impressions in a database; a machine learning means for learning user tendencies based on the data stored in the database; A means for selecting an appropriate product based on the user's tendency learned by the machine learning means; means for notifying the user of information regarding the selected product; A means for receiving feedback on the notified product and storing the feedback in the database. A personalized shopping assistant system including.

2. A means of collecting purchase history, usage history, and usage feedback; means for pre-processing the collected data; means for extracting features from the preprocessed data; Includes machine learning methods to learn user style and preferences based on extracted features The personalized shopping assistant system of claim 1 .

3. Includes a means to suggest where selected products are available and the best time to purchase them The personalized shopping assistant system of claim 1 .

Images