system

Abstract

We provide the system. [Solution] In order to suggest the most suitable items based on the user's preferences, characteristics, budget, and lifestyle, a means of collecting purchase history, browsing history, and social networking data to generate user characteristics, A means for selecting and proposing the most suitable items for the user using artificial intelligence processing based on the user's characteristics, A means of virtually trying out or placing selected items in a user's characteristics or environment using augmented reality technology, A means of comparing market values, obtaining the most economical purchasing options and available discount information, and presenting them to the user. A means of interpreting user inquiries using natural language processing and providing relevant information, A means of providing a more intuitive purchasing experience through virtual try-ons of items that users are interested in, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, the method including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] With the spread of online shopping, it has become difficult for consumers to select products suitable for themselves from a vast amount of information. In particular, the occurrence of returns due to selecting products that do not fit in terms of size or preferences, and the anxiety caused by opaque prices are damaging the consumer experience. Also, consumers cannot quickly respond to a wide variety of products, and there is a lack of means to intuitively select products that fit their body type and room, which is also a problem. As a result, consumer satisfaction has decreased, and the convenience of online shopping has not been fully enjoyed.

Means for Solving the Problems

[0005] This invention collects consumer purchase history, web browsing history, and social networking data to generate user profiles, thereby gaining a deep understanding of each consumer's preferences, body type, budget, and lifestyle. It also uses artificial intelligence algorithms to select the most suitable products based on these profiles. Furthermore, augmented reality technology enables virtual try-ons and room placement simulations of the products, allowing consumers to make intuitive and realistic product selections. By comparing market prices and presenting the most economical purchase options, it provides consumers with safe and cost-effective choices. Additionally, it utilizes natural language processing to quickly and accurately interpret and respond to consumer questions, resulting in improved customer support.

[0006] A "user profile" is a dataset that represents individual characteristics of consumers, such as their preferences, body type, budget, and lifestyle.

[0007] An "artificial intelligence algorithm" is a program in which a computer system analyzes data, discovers patterns, and suggests the most suitable products to the user.

[0008] Augmented reality technology is a technology that overlays digital information onto the real world, allowing users to virtually try on or place products in a real-world environment.

[0009] "Purchase history" refers to a record of goods and services that a consumer has purchased in the past.

[0010] "Web browsing history" refers to a record of the web pages that consumers have viewed on the internet.

[0011] "Social networking data" refers to records of consumers' interests, concerns, and interactions expressed on social media.

[0012] "Natural language processing" is a technology that enables computers to understand and process human language appropriately. [Brief explanation of the drawing]

[0013] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0014] Next, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, a labeled processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0017] In the following embodiments, a labeled RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0018] In the following embodiments, a labeled storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0019] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0021] [First Embodiment]

[0022] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0023] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0026] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0029] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0033] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0034] This invention is a system for providing consumers with the optimal shopping experience, and is implemented as follows.

[0035] First, when a user logs into an online shopping platform, the device collects purchase history, web browsing history, and social networking data from that device with the user's consent. This collected data forms a profile representing the user's preferences, body type, budget, lifestyle, and other characteristics.

[0036] The server uses profile data received from the terminal to execute an artificial intelligence algorithm and select products that match the user's preferences. This algorithm analyzes a vast dataset to identify products that the user is likely to be interested in.

[0037] The selected products are suggested to the user via the device. The user can choose items of interest from these suggestions and virtually try them on or place them in their home. Specifically, augmented reality technology is used to display an image on the device's screen showing how the product fits the user's body type and how it would look placed in their home room.

[0038] The server also collects market prices for the user's selected product from numerous internet resources and compares them. Furthermore, it gathers information on available coupons, identifies the most advantageous purchase option, and presents it to the user via the terminal.

[0039] In addition, if a user has questions about a specific product or service, they can ask them in natural language. The device receives this question, analyzes it using a natural language processing engine, and the server retrieves the appropriate answer from a database or external information source before providing it to the user through the device.

[0040] For example, if a user wants to buy a new jacket, jackets that match their preferences are suggested based on their profile. They can then choose one they like, virtually try it on, and confirm the appropriate size and style. Furthermore, the system presents the most cost-effective way to purchase the jacket, taking into account the suggested price and any coupon information. This ensures a satisfying shopping experience for the user.

[0041] The following describes the processing flow.

[0042] Step 1:

[0043] A user logs into an online shopping platform. At this point, information collection begins with the user's consent.

[0044] Step 2:

[0045] The device collects the user's purchase history, web browsing history, and social networking data, and temporarily stores it in local storage.

[0046] Step 3:

[0047] The device sends the collected data to the server. The server analyzes the received data and generates a user profile.

[0048] Step 4:

[0049] The server uses the generated user profile to execute an artificial intelligence algorithm and select products from the database that match the user's preferences.

[0050] Step 5:

[0051] The server sends the selected product list to the terminal. The terminal displays this product list to the user.

[0052] Step 6:

[0053] The user selects an item of interest from the displayed product list and virtually tries it on or places it in the room. The device uses augmented reality technology to visually provide the user with the opportunity to try on or place the item.

[0054] Step 7:

[0055] The server collects and compares market prices for selected products from multiple online stores. It also searches for available coupon information.

[0056] Step 8:

[0057] The server sends the cheapest purchase option and available coupons to the terminal and presents them to the user.

[0058] Step 9:

[0059] When a user has a question about a product or service, they ask it in natural language. The device receives this question and interprets it using an NLP engine.

[0060] Step 10:

[0061] The server retrieves answers from databases and external sources based on the interpreted question and provides them to the user through the terminal.

[0062] Step 11:

[0063] Users complete their purchase after experiencing a satisfying shopping experience.

[0064] (Example 1)

[0065] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0066] Modern consumers face the challenge of finding products online that suit their preferences, lifestyles, and budgets. Choosing the most economical and appropriate item from a diverse range of options is also a burden. Furthermore, it's difficult to see how a product will look in their home environment before purchasing it.

[0067] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0068] In this invention, the server includes means for suggesting the most suitable products based on the user's characteristic information, means for selecting and suggesting products using an intelligent algorithm, and means for virtually trying out or arranging products using virtual technology. This allows users to easily find economical and appropriate products that match their preferences and lifestyle, and to check the appearance of those products before purchasing them.

[0069] "User characteristic information" refers to a profile that includes information such as the user's preferences, appearance, budget, and lifestyle.

[0070] "Commercial products" refers to the general term for goods or services offered to users.

[0071] "Transaction history" refers to a record of purchases made by a user in the past.

[0072] "Web visit information" refers to information about the websites a user has accessed and the content they have viewed on the internet.

[0073] "Social data" refers to information and records of interactions shared by users on social networking sites.

[0074] An "intelligent algorithm" is a set of computational procedures that use artificial intelligence technology to analyze data and derive results that align with a specific purpose.

[0075] "Virtualization technology" is a technique that simulates objects and environments that do not actually exist on a computer.

[0076] "Language processing" is the technology that enables computers to understand and process human language appropriately.

[0077] "Discount" refers to any discount or benefit applied at the time of purchase.

[0078] This system, as a form of implementing the invention, aims to provide consumers with the best possible shopping experience. The following illustrates how this system works.

[0079] First, the device operates on the user's device under certain conditions. When the user logs into an online shopping platform, the device collects transaction history, web visit information, and social data with the user's consent. This generates user characteristic information, which includes preferences, appearance, budget, and lifestyle.

[0080] The server executes an intelligent algorithm based on user characteristic information received from the terminal. This algorithm analyzes a wide range of datasets and has a process for selecting products suitable for the user. The software used includes a generative AI model that can process large amounts of data quickly and enable recommendations based on individual user preferences.

[0081] The selected products are then presented to the user via a terminal. Virtual technology is used to allow the user to virtually try on or place the products in their own environment or home space for visual confirmation. Any device with a standard display can be used for this purpose.

[0082] Furthermore, the server researches market prices for selected products via the internet and identifies the most economical purchase option. In addition, the server uses natural language processing technology to analyze user inquiries and provide appropriate answers and information. This further enhances the user's purchasing experience.

[0083] For example, if a user has a request to "find a jacket that suits a casual style," they can input a prompt such as "Please suggest jackets that suit a casual style for men in their 30s." Based on this prompt, the system will select and suggest appropriate jackets based on the user's profile. In addition, it is possible to visualize how the jacket will look beforehand by utilizing the virtual try-on function.

[0084] This format allows users to have a more satisfying shopping experience.

[0085] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0086] Step 1:

[0087] The device detects when a user logs into an online shopping platform. Once the user logs in, the device, with the user's consent, collects transaction history, web visit information, and social data. Input data includes the user's past purchase history, viewed pages, and social network activity records. This data is acquired and output as user characteristic information.

[0088] Step 2:

[0089] The terminal sends user characteristic information it has acquired to the server. The server receives this information as input and analyzes the data using intelligent algorithms. This analysis creates and outputs a list of products best suited to the user. This process involves analyzing a large dataset to select products that match the user's preferences and budget.

[0090] Step 3:

[0091] The server sends the created product list to the terminal. The terminal uses this list to generate an interface that allows the user to easily compare options. The user can view this interface and select products that interest them. The output includes images and detailed information about the products presented to the user.

[0092] Step 4:

[0093] When a user selects items of interest from a curated list, the device uses virtual technology to virtually try out or place those items. This process involves displaying the selected items to fit the user's appearance or generating an image of them placed in a room. The input is the user-selected items, and the output is a virtually visualized image.

[0094] Step 5:

[0095] The server collects and compares market prices for selected products from internet sources. It then identifies the most economical option and available discounts, and sends them to the terminal. The input is market price information and discount information, and the output is the optimal purchase option.

[0096] Step 6:

[0097] If a user has questions about product details or purchasing, they can make inquiries using natural language. The terminal receives these inquiries and parses them via a language processing engine. The server retrieves solutions and relevant information from its database and provides them to the user through the terminal. The input is the user's inquiry, and the output is the relevant answer or information.

[0098] (Application Example 1)

[0099] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0100] Traditional online shopping systems have several drawbacks, including low accuracy in recommending products to users, the inability to virtually try on or view physical items, and the difficulty in easily selecting economical purchasing options. Furthermore, their ability to respond quickly to user inquiries has been limited. There is a need to address these challenges and provide users with an intuitive and personalized shopping experience.

[0101] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0102] In this invention, the server includes means for using artificial intelligence processing to generate user characteristics and select items, means for enabling virtual trial or placement using augmented reality technology, and means for collecting market value and discount information to present economical purchasing options. This makes it possible to provide the user with personalized product suggestions and a trial experience similar to that of actual items, and to present the most economical purchasing options.

[0103] "Preference" is a concept that refers to the individual tendencies or preferences of a user regarding specific goods or services they like.

[0104] "Characteristics" refer to elements that indicate the unique characteristics of each individual user, such as their body type and behavioral patterns.

[0105] "Budget" refers to the range of funds a user can allocate to purchases.

[0106] "Lifestyle" refers to patterns of habits and behaviors based on a user's way of life and values.

[0107] "Goods" refers to the general term for products and services offered to users.

[0108] "Purchase history" refers to a series of records related to products that a user has purchased in the past.

[0109] "Browsing history" refers to the history of web pages a user has visited on the internet.

[0110] "Social networking data" refers to information obtained from the social media platforms that users utilize.

[0111] "Artificial intelligence processing" is an information processing technology used by computers to analyze user characteristics and select appropriate items.

[0112] Augmented reality technology is a technique that overlays computer-generated visual information onto the real world.

[0113] "Market value" refers to the price information at which an item is currently offered in the market.

[0114] "Discount information" refers to economic incentives or promotional information applicable to goods.

[0115] "Natural language processing" is a technology that enables computers to understand human language and generate appropriate responses.

[0116] "Virtual trial" refers to the act of using a computer to simulate the experience of a user trying out an item they have selected.

[0117] "Personalized product recommendations" refer to suggestions for the most suitable items selected based on multiple characteristics of a given user.

[0118] This invention is a system for providing users with a more sophisticated shopping experience, and is implemented in the following manner.

[0119] First, when a user logs into the e-commerce platform, the server, with their authorization, collects purchase history, browsing history, and social networking data from their device. This collected data constitutes characteristics representing the user's preferences, traits, budget, and lifestyle. At this stage, a database system (e.g., MySQL® or MongoDB) is used to securely store the data.

[0120] Next, the server uses the characteristic data received from the terminal to perform artificial intelligence processing. The algorithm analyzes a vast dataset and selects items that the user is likely to be interested in. This process utilizes machine learning frameworks (e.g., TENSORFLOW® or PyTorch).

[0121] Selected items are presented to the user via a device. The user can use the device to select items of interest and virtually try them on using augmented reality technology. Specifically, ARKit (iOS) or ARCore (Android®) is installed on the device, allowing the items to be virtually tried on according to the user's characteristics and environment. For example, a user can virtually try on a jacket they have selected using AR.

[0122] Furthermore, the server collects market value data for the items selected by the user from various internet resources and indexes the most economical purchasing options. Web scraping techniques using Python's Beautiful Soup and Selenium are used for this information gathering.

[0123] In addition, if a user has questions about a particular item or service, they can ask them in natural language. The terminal receives this question, analyzes it using a natural language processing engine, and the server retrieves relevant answers from a database or external sources and provides them to the user. At this stage, OpenAI® generative AI models play a role in generating natural responses. A concrete example of a prompt would be, "What material is this jacket made of?"

[0124] This allows the system to simultaneously provide users with intuitive and personalized product recommendations, economical purchasing options, and a trial experience using augmented reality.

[0125] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0126] Step 1:

[0127] When a user logs into the e-commerce platform, the server collects purchase history, browsing history, and social networking data from the user's device with the user's permission. This data forms the basis for generating user characteristics. The input data is classified according to its type and stored in a database. The output is the basic data used to construct the user profile.

[0128] Step 2:

[0129] The server performs artificial intelligence processing based on the stored data. This process generates user profiles using machine learning models. The input is data about the user's purchase history and preferences, and by analyzing this data, it identifies items that the user is predicted to be interested in. Specifically, models built with TensorFlow or PyTorch run, and the output is a list of individualized items.

[0130] Step 3:

[0131] On the device, selected items are suggested to the user. The user selects an item and uses the device's AR function to virtually try on the selected item on their body or in their environment. The input is the data of the selected item, and the output is a visual trial experience generated using AR. ARKit (iOS) or ARCore (Android) is used for operation.

[0132] Step 4:

[0133] The server collects market value and discount information for selected items from the internet. The input is the item's identification information, and web scraping is performed using Python's Beautiful Soup or Selenium to obtain the most economical price information. The output provides the price and discount information to be presented to the user.

[0134] Step 5:

[0135] The terminal receives user inquiries about goods and services in natural language. The input is the user's question. The received question is analyzed by a natural language processing engine, and the server generates an appropriate answer via an AI model. For example, if the prompt "What material is this jacket made of?" is entered, the answer will output material information.

[0136] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0137] The present invention is a system that combines an emotion engine to enhance the user's personalized shopping experience, and is implemented as follows.

[0138] First, when a user logs into the application, the device retrieves purchase history, web browsing history, and social networking data. Based on this data, a profile is generated that reflects the user's preferences, body type, budget, and lifestyle.

[0139] Next, the server analyzes the generated user profile and uses an artificial intelligence algorithm to select the most suitable product for the user. This selection process also incorporates an emotion engine that analyzes the user's facial recognition, voice tone, body movements, and other factors.

[0140] The emotion engine recognizes the user's emotions in real time from their facial expressions and voice, and sends that data to the server. This allows the artificial intelligence algorithm to make product recommendations tailored to the user's emotional state.

[0141] The selected products are presented to the user via the terminal. If the user shows interest in a product, they can try it on or place it using augmented reality technology. During this process, the emotion engine continues to monitor the user's emotional changes and update the data accordingly.

[0142] Furthermore, the server compares market prices while taking sentiment data into account, and provides users with the most advantageous purchase options and available coupon information.

[0143] As a concrete example, while a user is using an apparel app on their smartphone, the emotion engine recognizes the user's face and analyzes their emotions. If it detects that the user's emotions towards the selected clothing are positive, related accessories and additional items are suggested. Conversely, if it determines that the user is losing interest, items in different styles or colors are presented, and measures are taken to rekindle the user's interest.

[0144] This allows users' emotional states to be reflected in their product choices, enabling a better shopping experience.

[0145] The following describes the processing flow.

[0146] Step 1:

[0147] The user logs into an online shopping platform. Once the application launches, information collection begins with the user's consent.

[0148] Step 2:

[0149] The device collects the user's purchase history, web browsing history, and social networking data. This data is stored locally.

[0150] Step 3:

[0151] The device sends the collected data to the server. The server analyzes it and generates a profile based on the user's preferences, body type, budget, and lifestyle.

[0152] Step 4:

[0153] The server uses the generated user profile to execute an artificial intelligence algorithm and select product candidates.

[0154] Step 5:

[0155] Using the camera and microphone built into the device, the emotion engine analyzes the user's facial expressions and voice tone in real time and generates emotion data.

[0156] Step 6:

[0157] The device sends emotional data to the server. Based on this data, the server dynamically adjusts product recommendations and selects the item that best suits the user's current emotional state.

[0158] Step 7:

[0159] The server sends a product list containing the selected items to the terminal. The terminal displays this list on the user's screen.

[0160] Step 8:

[0161] When a user selects an item they are interested in, the device uses augmented reality technology to provide visual feedback, such as trying on the selected item or placing it in the room.

[0162] Step 9:

[0163] The server checks the market price of the currently selected product and compares it to prices at other stores. It also collects information on available coupons.

[0164] Step 10:

[0165] The server sends the most favorable pricing information and available coupons to the terminal and presents them to the user.

[0166] Step 11:

[0167] The device continues to monitor the user's facial expressions and voice, tracking changes in their emotions. Based on this data, the server makes new product suggestions and adjusts the interface to maintain the user's interest.

[0168] Step 12:

[0169] Based on the information presented, the user decides to purchase the product and completes the purchase process.

[0170] (Example 2)

[0171] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0172] Traditional online shopping systems have the problem of failing to adequately reflect users' preferences and emotional states in product recommendations, resulting in a lack of personalized shopping experiences. Furthermore, virtual try-ons and visualizations of products users are interested in are insufficient, lacking means to increase purchasing intent. Finding the most economical purchase option from market prices is also not done efficiently. Therefore, there is a problem in that these systems fail to provide users with the optimal shopping experience.

[0173] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0174] This invention includes a server that provides opportunities to suggest optimal products based on the user's preferences, body type, budget, and lifestyle; a server that analyzes the user profile using a machine learning algorithm and provides real-time product recommendations including emotional data; and a server that allows the user to virtually experience selected products using augmented reality technology. This enables the user to enjoy a personalized shopping experience and receive product suggestions that respond to their emotions.

[0175] A "user profile" is a collection of information that reflects the individual characteristics of a user, such as their preferences, body type, budget, and lifestyle.

[0176] A "machine learning algorithm" is a computational method that learns patterns from large amounts of data and uses that knowledge to make predictions and decisions.

[0177] "Emotional data" refers to emotional information analyzed from a user's facial expressions, tone of voice, and other factors, and is acquired in real time.

[0178] Augmented reality technology refers to a technology that overlays digital information onto the real world and presents it to the user, allowing them to virtually experience products.

[0179] "Social network data" is a general term for digital information that includes users' activity history and interests on social media and online platforms.

[0180] "Natural language processing" is a computational technique for analyzing and understanding human language, and it provides a function to interpret user questions.

[0181] This invention is a system designed to enhance the user's personalized shopping experience. The system is implemented using the user's terminal, a server, an emotion engine, and associated software. Details are provided below.

[0182] The device activates when a user logs into a dedicated application and collects user data. This data includes purchase history, web browsing history, and social network data. Collection and storage are performed using a database API and securely accessed from cloud-based data storage.

[0183] The server processes the collected data to create user profiles. These profiles are analyzed using artificial intelligence algorithms and used to select and suggest the most suitable products for each user. Machine learning libraries (such as Python's Pandas and Scikit-learn) are utilized in this process.

[0184] The system also incorporates an emotion recognition engine that analyzes the user's facial and voice data in real time. OpenCV and TensorFlow can be used for this analysis. This allows the user's instantaneous emotional state to be transmitted to the server, which is then reflected in appropriate product recommendations.

[0185] Products that a user shows interest in are visualized on the device using augmented reality technology. This involves using ARKit or ARCore to enable virtual try-ons and product placement. The server also compares prices using market data and presents the user with the most economical purchase options and discount information.

[0186] As a concrete example, when a user interacts with an apparel app, the emotion engine scans the user's face and analyzes emotional data. If a positive response is detected, the server recommends related accessories and items.

[0187] An example of a prompt to a generative AI model is, "Use the user's sentiment data to create a list of recommended apparel items." In this way, the system can provide the user with a personalized, emotion-driven shopping experience.

[0188] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0189] Step 1:

[0190] When a user logs into the dedicated application, the device retrieves purchase history, web browsing history, and social network data. User authentication information is used as input, and the retrieved data is collected from cloud storage via a database API. This allows for the aggregation of diverse user activity histories.

[0191] Step 2:

[0192] The server analyzes the collected data to generate user profiles. Historical data is used as input, and machine learning algorithms are executed for analysis. Data processing is performed using Python's Pandas library, and the output is a profile that reflects the user's preferences and lifestyle.

[0193] Step 3:

[0194] The emotion engine scans the user's face and voice in real time and extracts emotion data. The input at this stage is raw data from the camera and microphone, and emotion analysis is performed using OpenCV or TensorFlow. The output is sent to the server as analyzed emotion data.

[0195] Step 4:

[0196] The server selects the optimal product using user profiles and sentiment data. A generative AI model processes the profile data and sentiment data as input and generates a list of recommended products as output. Libraries such as Scikit-learn are useful for this data processing.

[0197] Step 5:

[0198] The terminal uses a product list received from the server to make suggestions to the user. It takes a product list as input and provides a virtual try-on of the products using augmented reality as output. This operation uses ARKit or ARCore to enable virtual try-on and placement of selected products.

[0199] Step 6:

[0200] The server compares user data with market price data and presents the most economical purchase options and discount information. Using market data and user conditions as input, the algorithm performs optimization. As output, the user receives the most suitable price information on their terminal.

[0201] (Application Example 2)

[0202] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0203] Modern consumers have access to a wide variety of products, increasing the burden of choosing the best option for themselves. Furthermore, traditional online and offline shopping processes lack dynamic product recommendations based on consumer emotions and preferences. This makes it difficult to provide an efficient and satisfying shopping experience.

[0204] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0205] In this invention, the server includes means for collecting user information data and generating user information using a knowledge-based system, means for virtually trying on or arranging products using augmented reality technology, and means for analyzing the user's emotional state and adjusting product suggestions in real time using the knowledge-based system. This enables personalized product suggestions that respond to the user's emotions and preferences, providing an efficient and satisfying shopping experience.

[0206] "User preferences" refer to the characteristics and distinctive tendencies of products and services that individual users like.

[0207] "Body type" refers to the size and shape of the user's body.

[0208] "Budget" refers to the range of money a user plans to spend on a product or service.

[0209] "Lifestyle" is a concept that describes how a user spends their daily life, their values, habits, and preferences.

[0210] "Purchase history" refers to a record of products and services that a user has purchased in the past.

[0211] "Information browsing history" refers to the history of a user's browsing activity on websites and online platforms.

[0212] "Information sharing data" refers to information related to user activities and interests that is shared through social media and other communication tools.

[0213] "User information" refers to profile information generated based on the user's individual preferences, body type, budget, lifestyle, etc.

[0214] A "knowledge-based system" refers to a system that analyzes user information data and executes artificial intelligence algorithms to provide optimal product recommendations.

[0215] Augmented reality technology is a technique that overlays virtual information and objects onto the real world environment.

[0216] "Emotional state" refers to the real-time emotional stage analyzed from the user's facial expressions and voice.

[0217] "Discount information" refers to coupons and promotional information designed to lower the prices of products offered in the market.

[0218] The system of this invention includes a program that analyzes various data to suggest the optimal product based on the user's preferences, body type, budget, and lifestyle. The terminal first acquires the user's purchase history, information browsing history, and information sharing data, and generates user information based on this. The server uses a knowledge-based system to analyze this user information and select the most suitable product for the user. Here, the user's facial expressions and voice information are collected in real time for sentiment analysis. Based on this data, the server dynamically adjusts the product suggestions.

[0219] Utilizing augmented reality technology, the device provides users with virtual try-on and placement functions for selected products. This allows users to visually examine products and make purchase decisions. This technology is implemented using augmented reality software such as Unity and ARKit. Furthermore, by presenting the most advantageous discount information available on the market, users can make economical choices.

[0220] As a concrete example, in an application using a smart mirror in a physical store, the system automatically starts operating when a user stands in front of the mirror. The mirror suggests products optimized for the user in real time, and selected products can be virtually tried on using augmented reality technology. An example of a prompt would be, "Please show me a demo of a smart mirror app that uses emotion analysis technology to improve the customer experience in a physical store. Please suggest products optimized based on the user's facial expressions, and allow the user to virtually try on items they are interested in using augmented reality."

[0221] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0222] Step 1:

[0223] The device collects purchase history, browsing history, and shared information data when a user logs into an application. Based on this data, the device generates user information that reflects the user's preferences, body type, budget, and lifestyle. The input to this process is the user's past behavioral data, and the output is the generated user information.

[0224] Step 2:

[0225] The server receives the generated user information and performs analysis using a knowledge-based system. The server applies artificial intelligence algorithms as data processing to select the most suitable product for the user, and outputs a list of the resulting products. In this process, user information is used as input, and a list of optimal products is obtained as output.

[0226] Step 3:

[0227] The device captures the user's facial expressions using a camera sensor and sends them to the server. The server performs emotion analysis based on this data to understand the user's emotional state. The emotional state obtained from data calculations by the emotion analysis engine, using the user's facial expression data as input, is output.

[0228] Step 4:

[0229] The server updates the algorithms within the knowledge base system based on the results of the sentiment analysis, dynamically adjusting the product recommendations. Here, the analyzed sentiment state and the optimal product list are taken as input, and the newly adjusted product recommendations are output.

[0230] Step 5:

[0231] The device uses augmented reality software to visually present virtual try-ons and placements of products the user has expressed interest in. The input to this process is tailored product suggestions, and the output is a video of the virtual try-on or placement presented to the user.

[0232] Step 6:

[0233] The server collects market price information and provides users with purchase options, including the most advantageous discounts. The input needed to attract user interest is a tailored product suggestion, and the output is a list of discount information and purchase options.

[0234] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0235] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0236] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0237] [Second Embodiment]

[0238] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0239] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0240] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0241] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0242] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0243] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0244] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0245] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0246] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0247] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0248] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0249] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0250] This invention is a system for providing consumers with the optimal shopping experience, and is implemented as follows.

[0251] First, when a user logs into an online shopping platform, the device collects purchase history, web browsing history, and social networking data from that device with the user's consent. This collected data forms a profile representing the user's preferences, body type, budget, lifestyle, and other characteristics.

[0252] The server uses profile data received from the terminal to execute an artificial intelligence algorithm and select products that match the user's preferences. This algorithm analyzes a vast dataset to identify products that the user is likely to be interested in.

[0253] The selected products are suggested to the user via the device. The user can choose items of interest from these suggestions and virtually try them on or place them in their home. Specifically, augmented reality technology is used to display an image on the device's screen showing how the product fits the user's body type and how it would look placed in their home room.

[0254] The server also collects market prices for the user's selected product from numerous internet resources and compares them. Furthermore, it gathers information on available coupons, identifies the most advantageous purchase option, and presents it to the user via the terminal.

[0255] In addition, if a user has questions about a specific product or service, they can ask them in natural language. The device receives this question, analyzes it using a natural language processing engine, and the server retrieves the appropriate answer from a database or external information source before providing it to the user through the device.

[0256] For example, if a user wants to buy a new jacket, jackets that match their preferences are suggested based on their profile. They can then choose one they like, virtually try it on, and confirm the appropriate size and style. Furthermore, the system presents the most cost-effective way to purchase the jacket, taking into account the suggested price and any coupon information. This ensures a satisfying shopping experience for the user.

[0257] The following describes the processing flow.

[0258] Step 1:

[0259] A user logs into an online shopping platform. At this point, information collection begins with the user's consent.

[0260] Step 2:

[0261] The device collects the user's purchase history, web browsing history, and social networking data, and temporarily stores it in local storage.

[0262] Step 3:

[0263] The device sends the collected data to the server. The server analyzes the received data and generates a user profile.

[0264] Step 4:

[0265] The server uses the generated user profile to execute an artificial intelligence algorithm and select products from the database that match the user's preferences.

[0266] Step 5:

[0267] The server sends the selected product list to the terminal. The terminal displays this product list to the user.

[0268] Step 6:

[0269] The user selects an item of interest from the displayed product list and virtually tries it on or places it in the room. The device uses augmented reality technology to visually provide the user with the opportunity to try on or place the item.

[0270] Step 7:

[0271] The server collects and compares market prices for selected products from multiple online stores. It also searches for available coupon information.

[0272] Step 8:

[0273] The server sends the cheapest purchase option and available coupons to the terminal and presents them to the user.

[0274] Step 9:

[0275] When a user has a question about a product or service, they ask it in natural language. The device receives this question and interprets it using an NLP engine.

[0276] Step 10:

[0277] The server retrieves answers from databases and external sources based on the interpreted question and provides them to the user through the terminal.

[0278] Step 11:

[0279] Users complete their purchase after experiencing a satisfying shopping experience.

[0280] (Example 1)

[0281] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0282] Modern consumers face the challenge of finding products online that suit their preferences, lifestyles, and budgets. Choosing the most economical and appropriate item from a diverse range of options is also a burden. Furthermore, it's difficult to see how a product will look in their home environment before purchasing it.

[0283] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0284] In this invention, the server includes means for proposing an optimal commercial item based on the user's characteristic information, means for selecting and proposing a commercial item using an intelligent algorithm, and means for virtually trying out or arranging a commercial item using virtual technology. As a result, the user can easily find an economical and appropriate commercial item that matches their preferences and lifestyle, and can confirm the appearance of the commercial item before purchase.

[0285] The "user's characteristic information" is a profile including information such as the user's preferences, appearance, budget, and lifestyle.

[0286] The "commercial item" is a general term for products or services proposed to the user.

[0287] The "transaction history" is a record of the user's past purchases.

[0288] The "web access information" is information regarding the websites accessed by the user on the Internet and the content viewed.

[0289] The "social data" is information shared by the user on social networking sites and records of exchanges.

[0290] The "intelligent algorithm" is a computational procedure for analyzing data using artificial intelligence technology and deriving results in line with a specific purpose.

[0291] The "virtual technology" is a technology for simulating objects and environments that do not actually exist on a computer.

[0292] The "language processing" is a technology for enabling a computer to understand and appropriately process human language.

[0293] The "discount" refers to discounts and benefits applied at the time of purchase.

[0294] This system, as a form of implementing the invention, aims to provide consumers with the best possible shopping experience. The following illustrates how this system works.

[0295] First, the device operates on the user's device under certain conditions. When the user logs into an online shopping platform, the device collects transaction history, web visit information, and social data with the user's consent. This generates user characteristic information, which includes preferences, appearance, budget, and lifestyle.

[0296] The server executes an intelligent algorithm based on user characteristic information received from the terminal. This algorithm analyzes a wide range of datasets and has a process for selecting products suitable for the user. The software used includes a generative AI model that can process large amounts of data quickly and enable recommendations based on individual user preferences.

[0297] The selected products are then presented to the user via a terminal. Virtual technology is used to allow the user to virtually try on or place the products in their own environment or home space for visual confirmation. Any device with a standard display can be used for this purpose.

[0298] Furthermore, the server researches market prices for selected products via the internet and identifies the most economical purchase option. In addition, the server uses natural language processing technology to analyze user inquiries and provide appropriate answers and information. This further enhances the user's purchasing experience.

[0299] For example, if a user has a request to "find a jacket that suits a casual style," they can input a prompt such as "Please suggest jackets that suit a casual style for men in their 30s." Based on this prompt, the system will select and suggest appropriate jackets based on the user's profile. In addition, it is possible to visualize how the jacket will look beforehand by utilizing the virtual try-on function.

[0300] This format allows users to have a more satisfying shopping experience.

[0301] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0302] Step 1:

[0303] The device detects when a user logs into an online shopping platform. Once the user logs in, the device, with the user's consent, collects transaction history, web visit information, and social data. Input data includes the user's past purchase history, viewed pages, and social network activity records. This data is acquired and output as user characteristic information.

[0304] Step 2:

[0305] The terminal sends user characteristic information it has acquired to the server. The server receives this information as input and analyzes the data using intelligent algorithms. This analysis creates and outputs a list of products best suited to the user. This process involves analyzing a large dataset to select products that match the user's preferences and budget.

[0306] Step 3:

[0307] The server sends the created product list to the terminal. Based on this list, the terminal generates an interface that allows the user to easily compare options. The user can view this interface and select the products of interest. What is output is the images and detailed information of the products presented to the user.

[0308] Step 4:

[0309] When the user selects the products of interest from the selected products, the terminal uses virtual technology to try out or place the products. In this process, the selected products are displayed to fit the user's appearance, or an image of placing them in the room is generated. The input is the products selected by the user, and the output is the virtualized visual image.

[0310] Step 5:

[0311] The server collects the market prices of the selected products from information sources on the Internet and makes comparisons. Next, it identifies the most economical options and available discounts and sends them to the terminal. The input is the market price information and discount information, and the output is the optimal purchase option.

[0312] Step 6:

[0313] When the user has questions about the product details or purchase, they can make an inquiry in natural language. The terminal receives this inquiry and analyzes it through a language processing engine. The server obtains solutions and relevant information to the user's questions from the database and provides them to the user through the terminal. The input is the user's inquiry, and the output is the relevant answers and information.

[0314] (Application Example 1)

[0315] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0316] Traditional online shopping systems have several drawbacks, including low accuracy in recommending products to users, the inability to virtually try on or view physical items, and the difficulty in easily selecting economical purchasing options. Furthermore, their ability to respond quickly to user inquiries has been limited. There is a need to address these challenges and provide users with an intuitive and personalized shopping experience.

[0317] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0318] In this invention, the server includes means for using artificial intelligence processing to generate user characteristics and select items, means for enabling virtual trial or placement using augmented reality technology, and means for collecting market value and discount information to present economical purchasing options. This makes it possible to provide the user with personalized product suggestions and a trial experience similar to that of actual items, and to present the most economical purchasing options.

[0319] "Preference" is a concept that refers to the individual tendencies or preferences of a user regarding specific goods or services they like.

[0320] "Characteristics" refer to elements that indicate the unique characteristics of each individual user, such as their body type and behavioral patterns.

[0321] "Budget" refers to the range of funds a user can allocate to purchases.

[0322] "Lifestyle" refers to patterns of habits and behaviors based on a user's way of life and values.

[0323] "Goods" refers to the general term for products and services offered to users.

[0324] "Purchase history" refers to a series of records related to products that a user has purchased in the past.

[0325] "Browsing history" refers to the history of web pages a user has visited on the internet.

[0326] "Social networking data" refers to information obtained from the social media platforms that users utilize.

[0327] "Artificial intelligence processing" is an information processing technology used by computers to analyze user characteristics and select appropriate items.

[0328] Augmented reality technology is a technique that overlays computer-generated visual information onto the real world.

[0329] "Market value" refers to the price information at which an item is currently offered in the market.

[0330] "Discount information" refers to economic incentives or promotional information applicable to goods.

[0331] "Natural language processing" is a technology that enables computers to understand human language and generate appropriate responses.

[0332] "Virtual trial" refers to the act of using a computer to simulate the experience of a user trying out an item they have selected.

[0333] "Personalized product recommendations" refer to suggestions for the most suitable items selected based on multiple characteristics of a given user.

[0334] This invention is a system for providing users with a more sophisticated shopping experience, and is implemented in the following manner.

[0335] First, when a user logs into the e-commerce platform, the server, with their authorization, collects purchase history, browsing history, and social networking data from their device. This collected data constitutes characteristics representing the user's preferences, traits, budget, and lifestyle. At this stage, a database system (e.g., MySQL or MongoDB) is used to securely store the data.

[0336] Next, the server uses the characteristic data received from the terminal to perform artificial intelligence processing. The algorithm analyzes a vast dataset and selects items that the user is likely to be interested in. This process utilizes machine learning frameworks (such as TensorFlow or PyTorch).

[0337] Selected items are presented to the user via a device. The user can use the device to select items of interest and virtually try them on using augmented reality technology. Specifically, ARKit (iOS) or ARCore (Android) is installed on the device, allowing the items to be virtually tried on according to the user's characteristics and environment. For example, a user can virtually try on a jacket they have selected using AR.

[0338] Furthermore, the server collects market value data for the items selected by the user from various internet resources and indexes the most economical purchasing options. Web scraping techniques using Python's Beautiful Soup and Selenium are used for this information gathering.

[0339] In addition, if a user has questions about a particular item or service, they can ask them in natural language. The terminal receives this question, analyzes it using a natural language processing engine, and the server retrieves relevant answers from a database or external sources and provides them to the user. At this stage, OpenAI's generative AI model plays a role in generating natural-sounding responses. A concrete example of a prompt would be, "What material is this jacket made of?"

[0340] This allows the system to simultaneously provide users with intuitive and personalized product recommendations, economical purchasing options, and a trial experience using augmented reality.

[0341] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0342] Step 1:

[0343] When a user logs into the e-commerce platform, the server collects purchase history, browsing history, and social networking data from the user's device with the user's permission. This data forms the basis for generating user characteristics. The input data is classified according to its type and stored in a database. The output is the basic data used to construct the user profile.

[0344] Step 2:

[0345] The server performs artificial intelligence processing based on the stored data. This process generates user profiles using machine learning models. The input is data about the user's purchase history and preferences, and by analyzing this data, it identifies items that the user is predicted to be interested in. Specifically, models built with TensorFlow or PyTorch run, and the output is a list of individualized items.

[0346] Step 3:

[0347] On the device, selected items are suggested to the user. The user selects an item and uses the device's AR function to virtually try on the selected item on their body or in their environment. The input is the data of the selected item, and the output is a visual trial experience generated using AR. ARKit (iOS) or ARCore (Android) is used for operation.

[0348] Step 4:

[0349] The server collects market value and discount information for selected items from the internet. The input is the item's identification information, and web scraping is performed using Python's Beautiful Soup or Selenium to obtain the most economical price information. The output provides the price and discount information to be presented to the user.

[0350] Step 5:

[0351] The terminal receives user inquiries about goods and services in natural language. The input is the user's question. The received question is analyzed by a natural language processing engine, and the server generates an appropriate answer via an AI model. For example, if the prompt "What material is this jacket made of?" is entered, the answer will output material information.

[0352] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0353] The present invention is a system that combines an emotion engine to enhance the user's personalized shopping experience, and is implemented as follows.

[0354] First, when a user logs into the application, the device retrieves purchase history, web browsing history, and social networking data. Based on this data, a profile is generated that reflects the user's preferences, body type, budget, and lifestyle.

[0355] Next, the server analyzes the generated user profile and uses an artificial intelligence algorithm to select the most suitable product for the user. This selection process also incorporates an emotion engine that analyzes the user's facial recognition, voice tone, body movements, and other factors.

[0356] The emotion engine recognizes the user's emotions in real time from their facial expressions and voice, and sends that data to the server. This allows the artificial intelligence algorithm to make product recommendations tailored to the user's emotional state.

[0357] The selected products are presented to the user via the terminal. If the user shows interest in a product, they can try it on or place it using augmented reality technology. During this process, the emotion engine continues to monitor the user's emotional changes and update the data accordingly.

[0358] Furthermore, the server compares market prices while taking sentiment data into account, and provides users with the most advantageous purchase options and available coupon information.

[0359] As a concrete example, while a user is using an apparel app on their smartphone, the emotion engine recognizes the user's face and analyzes their emotions. If it detects that the user's emotions towards the selected clothing are positive, related accessories and additional items are suggested. Conversely, if it determines that the user is losing interest, items in different styles or colors are presented, and measures are taken to rekindle the user's interest.

[0360] This allows users' emotional states to be reflected in their product choices, enabling a better shopping experience.

[0361] The following describes the processing flow.

[0362] Step 1:

[0363] The user logs into an online shopping platform. Once the application launches, information collection begins with the user's consent.

[0364] Step 2:

[0365] The device collects the user's purchase history, web browsing history, and social networking data. This data is stored locally.

[0366] Step 3:

[0367] The device sends the collected data to the server. The server analyzes it and generates a profile based on the user's preferences, body type, budget, and lifestyle.

[0368] Step 4:

[0369] The server uses the generated user profile to execute an artificial intelligence algorithm and select product candidates.

[0370] Step 5:

[0371] Using the camera and microphone built into the device, the emotion engine analyzes the user's facial expressions and voice tone in real time and generates emotion data.

[0372] Step 6:

[0373] The device sends emotional data to the server. Based on this data, the server dynamically adjusts product recommendations and selects the item that best suits the user's current emotional state.

[0374] Step 7:

[0375] The server sends a product list containing the selected items to the terminal. The terminal displays this list on the user's screen.

[0376] Step 8:

[0377] When a user selects an item they are interested in, the device uses augmented reality technology to provide visual feedback, such as trying on the selected item or placing it in the room.

[0378] Step 9:

[0379] The server checks the market price of the currently selected product and compares it to prices at other stores. It also collects information on available coupons.

[0380] Step 10:

[0381] The server sends the most favorable pricing information and available coupons to the terminal and presents them to the user.

[0382] Step 11:

[0383] The device continues to monitor the user's facial expressions and voice, tracking changes in their emotions. Based on this data, the server makes new product suggestions and adjusts the interface to maintain the user's interest.

[0384] Step 12:

[0385] Based on the information presented, the user decides to purchase the product and completes the purchase process.

[0386] (Example 2)

[0387] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0388] Traditional online shopping systems have the problem of failing to adequately reflect users' preferences and emotional states in product recommendations, resulting in a lack of personalized shopping experiences. Furthermore, virtual try-ons and visualizations of products users are interested in are insufficient, lacking means to increase purchasing intent. Finding the most economical purchase option from market prices is also not done efficiently. Therefore, there is a problem in that these systems fail to provide users with the optimal shopping experience.

[0389] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0390] This invention includes a server that provides opportunities to suggest optimal products based on the user's preferences, body type, budget, and lifestyle; a server that analyzes the user profile using a machine learning algorithm and provides real-time product recommendations including emotional data; and a server that allows the user to virtually experience selected products using augmented reality technology. This enables the user to enjoy a personalized shopping experience and receive product suggestions that respond to their emotions.

[0391] A "user profile" is a collection of information that reflects the individual characteristics of a user, such as their preferences, body type, budget, and lifestyle.

[0392] A "machine learning algorithm" is a computational method that learns patterns from large amounts of data and uses that knowledge to make predictions and decisions.

[0393] "Emotional data" refers to emotional information analyzed from a user's facial expressions, tone of voice, and other factors, and is acquired in real time.

[0394] Augmented reality technology refers to a technology that overlays digital information onto the real world and presents it to the user, allowing them to virtually experience products.

[0395] "Social network data" is a general term for digital information that includes users' activity history and interests on social media and online platforms.

[0396] "Natural language processing" is a computational technique for analyzing and understanding human language, and it provides a function to interpret user questions.

[0397] This invention is a system designed to enhance the user's personalized shopping experience. The system is implemented using the user's terminal, a server, an emotion engine, and associated software. Details are provided below.

[0398] The device activates when a user logs into a dedicated application and collects user data. This data includes purchase history, web browsing history, and social network data. Collection and storage are performed using a database API and securely accessed from cloud-based data storage.

[0399] The server processes the collected data to create user profiles. These profiles are analyzed using artificial intelligence algorithms and used to select and suggest the most suitable products for each user. Machine learning libraries (such as Python's Pandas and Scikit-learn) are utilized in this process.

[0400] The system also incorporates an emotion recognition engine that analyzes the user's facial and voice data in real time. OpenCV and TensorFlow can be used for this analysis. This allows the user's instantaneous emotional state to be transmitted to the server, which is then reflected in appropriate product recommendations.

[0401] Products that a user shows interest in are visualized on the device using augmented reality technology. This involves using ARKit or ARCore to enable virtual try-ons and product placement. The server also compares prices using market data and presents the user with the most economical purchase options and discount information.

[0402] As a concrete example, when a user interacts with an apparel app, the emotion engine scans the user's face and analyzes emotional data. If a positive response is detected, the server recommends related accessories and items.

[0403] An example of a prompt to a generative AI model is, "Use the user's sentiment data to create a list of recommended apparel items." In this way, the system can provide the user with a personalized, emotion-driven shopping experience.

[0404] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0405] Step 1:

[0406] When a user logs into the dedicated application, the device retrieves purchase history, web browsing history, and social network data. User authentication information is used as input, and the retrieved data is collected from cloud storage via a database API. This allows for the aggregation of diverse user activity histories.

[0407] Step 2:

[0408] The server analyzes the collected data to generate user profiles. Historical data is used as input, and machine learning algorithms are executed for analysis. Data processing is performed using Python's Pandas library, and the output is a profile that reflects the user's preferences and lifestyle.

[0409] Step 3:

[0410] The emotion engine scans the user's face and voice in real time and extracts emotion data. The input at this stage is raw data from the camera and microphone, and emotion analysis is performed using OpenCV or TensorFlow. The output is sent to the server as analyzed emotion data.

[0411] Step 4:

[0412] The server selects the optimal product using user profiles and sentiment data. A generative AI model processes the profile data and sentiment data as input and generates a list of recommended products as output. Libraries such as Scikit-learn are useful for this data processing.

[0413] Step 5:

[0414] The terminal uses a product list received from the server to make suggestions to the user. It takes a product list as input and provides a virtual try-on of the products using augmented reality as output. This operation uses ARKit or ARCore to enable virtual try-on and placement of selected products.

[0415] Step 6:

[0416] The server compares user data with market price data and presents the most economical purchase options and discount information. Using market data and user conditions as input, the algorithm performs optimization. As output, the user receives the most suitable price information on their terminal.

[0417] (Application Example 2)

[0418] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0419] Modern consumers have access to a wide variety of products, increasing the burden of choosing the best option for themselves. Furthermore, traditional online and offline shopping processes lack dynamic product recommendations based on consumer emotions and preferences. This makes it difficult to provide an efficient and satisfying shopping experience.

[0420] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0421] In this invention, the server includes means for collecting user information data and generating user information using a knowledge-based system, means for virtually trying on or arranging products using augmented reality technology, and means for analyzing the user's emotional state and adjusting product suggestions in real time using the knowledge-based system. This enables personalized product suggestions that respond to the user's emotions and preferences, providing an efficient and satisfying shopping experience.

[0422] "User preferences" refer to the characteristics and distinctive tendencies of products and services that individual users like.

[0423] "Body type" refers to the size and shape of the user's body.

[0424] "Budget" refers to the range of money a user plans to spend on a product or service.

[0425] "Lifestyle" is a concept that describes how a user spends their daily life, their values, habits, and preferences.

[0426] "Purchase history" refers to a record of products and services that a user has purchased in the past.

[0427] "Information browsing history" refers to the history of a user's browsing activity on websites and online platforms.

[0428] "Information sharing data" refers to information related to user activities and interests that is shared through social media and other communication tools.

[0429] "User information" refers to profile information generated based on the user's individual preferences, body type, budget, lifestyle, etc.

[0430] A "knowledge-based system" refers to a system that analyzes user information data and executes artificial intelligence algorithms to provide optimal product recommendations.

[0431] Augmented reality technology is a technique that overlays virtual information and objects onto the real world environment.

[0432] "Emotional state" refers to the real-time emotional stage analyzed from the user's facial expressions and voice.

[0433] "Discount information" refers to coupons and promotional information designed to lower the prices of products offered in the market.

[0434] The system of this invention includes a program that analyzes various data to suggest the optimal product based on the user's preferences, body type, budget, and lifestyle. The terminal first acquires the user's purchase history, information browsing history, and information sharing data, and generates user information based on this. The server uses a knowledge-based system to analyze this user information and select the most suitable product for the user. Here, the user's facial expressions and voice information are collected in real time for sentiment analysis. Based on this data, the server dynamically adjusts the product suggestions.

[0435] Utilizing augmented reality technology, the device provides users with virtual try-on and placement functions for selected products. This allows users to visually examine products and make purchase decisions. This technology is implemented using augmented reality software such as Unity and ARKit. Furthermore, by presenting the most advantageous discount information available on the market, users can make economical choices.

[0436] As a concrete example, in an application using a smart mirror in a physical store, the system automatically starts operating when a user stands in front of the mirror. The mirror suggests products optimized for the user in real time, and selected products can be virtually tried on using augmented reality technology. An example of a prompt would be, "Please show me a demo of a smart mirror app that uses emotion analysis technology to improve the customer experience in a physical store. Please suggest products optimized based on the user's facial expressions, and allow the user to virtually try on items they are interested in using augmented reality."

[0437] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0438] Step 1:

[0439] The device collects purchase history, browsing history, and shared information data when a user logs into an application. Based on this data, the device generates user information that reflects the user's preferences, body type, budget, and lifestyle. The input to this process is the user's past behavioral data, and the output is the generated user information.

[0440] Step 2:

[0441] The server receives the generated user information and performs analysis using a knowledge-based system. The server applies artificial intelligence algorithms as data processing to select the most suitable product for the user, and outputs a list of the resulting products. In this process, user information is used as input, and a list of optimal products is obtained as output.

[0442] Step 3:

[0443] The device captures the user's facial expressions using a camera sensor and sends them to the server. The server performs emotion analysis based on this data to understand the user's emotional state. The emotional state obtained from data calculations by the emotion analysis engine, using the user's facial expression data as input, is output.

[0444] Step 4:

[0445] The server updates the algorithms within the knowledge base system based on the results of the sentiment analysis, dynamically adjusting the product recommendations. Here, the analyzed sentiment state and the optimal product list are taken as input, and the newly adjusted product recommendations are output.

[0446] Step 5:

[0447] The device uses augmented reality software to visually present virtual try-ons and placements of products the user has expressed interest in. The input to this process is tailored product suggestions, and the output is a video of the virtual try-on or placement presented to the user.

[0448] Step 6:

[0449] The server collects market price information and provides users with purchase options, including the most advantageous discounts. The input needed to attract user interest is a tailored product suggestion, and the output is a list of discount information and purchase options.

[0450] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0451] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0452] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0453] [Third Embodiment]

[0454] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0455] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0456] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0457] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0458] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0459] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0460] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0461] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0462] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0463] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0464] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0465] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0466] This invention is a system for providing consumers with the optimal shopping experience, and is implemented as follows.

[0467] First, when a user logs into an online shopping platform, the device collects purchase history, web browsing history, and social networking data from that device with the user's consent. This collected data forms a profile representing the user's preferences, body type, budget, lifestyle, and other characteristics.

[0468] The server uses profile data received from the terminal to execute an artificial intelligence algorithm and select products that match the user's preferences. This algorithm analyzes a vast dataset to identify products that the user is likely to be interested in.

[0469] The selected products are suggested to the user via the device. The user can choose items of interest from these suggestions and virtually try them on or place them in their home. Specifically, augmented reality technology is used to display an image on the device's screen showing how the product fits the user's body type and how it would look placed in their home room.

[0470] The server also collects market prices for the user's selected product from numerous internet resources and compares them. Furthermore, it gathers information on available coupons, identifies the most advantageous purchase option, and presents it to the user via the terminal.

[0471] In addition, if a user has questions about a specific product or service, they can ask them in natural language. The device receives this question, analyzes it using a natural language processing engine, and the server retrieves the appropriate answer from a database or external information source before providing it to the user through the device.

[0472] For example, if a user wants to buy a new jacket, jackets that match their preferences are suggested based on their profile. They can then choose one they like, virtually try it on, and confirm the appropriate size and style. Furthermore, the system presents the most cost-effective way to purchase the jacket, taking into account the suggested price and any coupon information. This ensures a satisfying shopping experience for the user.

[0473] The following describes the processing flow.

[0474] Step 1:

[0475] A user logs into an online shopping platform. At this point, information collection begins with the user's consent.

[0476] Step 2:

[0477] The device collects the user's purchase history, web browsing history, and social networking data, and temporarily stores it in local storage.

[0478] Step 3:

[0479] The device sends the collected data to the server. The server analyzes the received data and generates a user profile.

[0480] Step 4:

[0481] The server uses the generated user profile to execute an artificial intelligence algorithm and select products from the database that match the user's preferences.

[0482] Step 5:

[0483] The server sends the selected product list to the terminal. The terminal displays this product list to the user.

[0484] Step 6:

[0485] The user selects an item of interest from the displayed product list and virtually tries it on or places it in the room. The device uses augmented reality technology to visually provide the user with the opportunity to try on or place the item.

[0486] Step 7:

[0487] The server collects and compares market prices for selected products from multiple online stores. It also searches for available coupon information.

[0488] Step 8:

[0489] The server sends the cheapest purchase option and available coupons to the terminal and presents them to the user.

[0490] Step 9:

[0491] When a user has a question about a product or service, they ask it in natural language. The device receives this question and interprets it using an NLP engine.

[0492] Step 10:

[0493] The server retrieves answers from databases and external sources based on the interpreted question and provides them to the user through the terminal.

[0494] Step 11:

[0495] Users complete their purchase after experiencing a satisfying shopping experience.

[0496] (Example 1)

[0497] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0498] Modern consumers face the challenge of finding products online that suit their preferences, lifestyles, and budgets. Choosing the most economical and appropriate item from a diverse range of options is also a burden. Furthermore, it's difficult to see how a product will look in their home environment before purchasing it.

[0499] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0500] In this invention, the server includes means for suggesting the most suitable products based on the user's characteristic information, means for selecting and suggesting products using an intelligent algorithm, and means for virtually trying out or arranging products using virtual technology. This allows users to easily find economical and appropriate products that match their preferences and lifestyle, and to check the appearance of those products before purchasing them.

[0501] "User characteristic information" refers to a profile that includes information such as the user's preferences, appearance, budget, and lifestyle.

[0502] "Commercial products" refers to the general term for goods or services offered to users.

[0503] "Transaction history" refers to a record of purchases made by a user in the past.

[0504] "Web visit information" refers to information about the websites a user has accessed and the content they have viewed on the internet.

[0505] "Social data" refers to information and records of interactions shared by users on social networking sites.

[0506] An "intelligent algorithm" is a set of computational procedures that use artificial intelligence technology to analyze data and derive results that align with a specific purpose.

[0507] "Virtualization technology" is a technique that simulates objects and environments that do not actually exist on a computer.

[0508] "Language processing" is the technology that enables computers to understand and process human language appropriately.

[0509] "Discount" refers to any discount or benefit applied at the time of purchase.

[0510] This system, as a form of implementing the invention, aims to provide consumers with the best possible shopping experience. The following illustrates how this system works.

[0511] First, the device operates on the user's device under certain conditions. When the user logs into an online shopping platform, the device collects transaction history, web visit information, and social data with the user's consent. This generates user characteristic information, which includes preferences, appearance, budget, and lifestyle.

[0512] The server executes an intelligent algorithm based on user characteristic information received from the terminal. This algorithm analyzes a wide range of datasets and has a process for selecting products suitable for the user. The software used includes a generative AI model that can process large amounts of data quickly and enable recommendations based on individual user preferences.

[0513] The selected products are then presented to the user via a terminal. Virtual technology is used to allow the user to virtually try on or place the products in their own environment or home space for visual confirmation. Any device with a standard display can be used for this purpose.

[0514] Furthermore, the server researches market prices for selected products via the internet and identifies the most economical purchase option. In addition, the server uses natural language processing technology to analyze user inquiries and provide appropriate answers and information. This further enhances the user's purchasing experience.

[0515] For example, if a user has a request to "find a jacket that suits a casual style," they can input a prompt such as "Please suggest jackets that suit a casual style for men in their 30s." Based on this prompt, the system will select and suggest appropriate jackets based on the user's profile. In addition, it is possible to visualize how the jacket will look beforehand by utilizing the virtual try-on function.

[0516] This format allows users to have a more satisfying shopping experience.

[0517] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0518] Step 1:

[0519] The device detects when a user logs into an online shopping platform. Once the user logs in, the device, with the user's consent, collects transaction history, web visit information, and social data. Input data includes the user's past purchase history, viewed pages, and social network activity records. This data is acquired and output as user characteristic information.

[0520] Step 2:

[0521] The terminal sends user characteristic information it has acquired to the server. The server receives this information as input and analyzes the data using intelligent algorithms. This analysis creates and outputs a list of products best suited to the user. This process involves analyzing a large dataset to select products that match the user's preferences and budget.

[0522] Step 3:

[0523] The server sends the created product list to the terminal. The terminal uses this list to generate an interface that allows the user to easily compare options. The user can view this interface and select products that interest them. The output includes images and detailed information about the products presented to the user.

[0524] Step 4:

[0525] When a user selects items of interest from a curated list, the device uses virtual technology to virtually try out or place those items. This process involves displaying the selected items to fit the user's appearance or generating an image of them placed in a room. The input is the user-selected items, and the output is a virtually visualized image.

[0526] Step 5:

[0527] The server collects and compares market prices for selected products from internet sources. It then identifies the most economical option and available discounts, and sends them to the terminal. The input is market price information and discount information, and the output is the optimal purchase option.

[0528] Step 6:

[0529] If a user has questions about product details or purchasing, they can make inquiries using natural language. The terminal receives these inquiries and parses them via a language processing engine. The server retrieves solutions and relevant information from its database and provides them to the user through the terminal. The input is the user's inquiry, and the output is the relevant answer or information.

[0530] (Application Example 1)

[0531] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0532] Traditional online shopping systems have several drawbacks, including low accuracy in recommending products to users, the inability to virtually try on or view physical items, and the difficulty in easily selecting economical purchasing options. Furthermore, their ability to respond quickly to user inquiries has been limited. There is a need to address these challenges and provide users with an intuitive and personalized shopping experience.

[0533] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0534] In this invention, the server includes means for using artificial intelligence processing to generate user characteristics and select items, means for enabling virtual trial or placement using augmented reality technology, and means for collecting market value and discount information to present economical purchasing options. This makes it possible to provide the user with personalized product suggestions and a trial experience similar to that of actual items, and to present the most economical purchasing options.

[0535] "Preference" is a concept that refers to the individual tendencies or preferences of a user regarding specific goods or services they like.

[0536] "Characteristics" refer to elements that indicate the unique characteristics of each individual user, such as their body type and behavioral patterns.

[0537] "Budget" refers to the range of funds a user can allocate to purchases.

[0538] "Lifestyle" refers to patterns of habits and behaviors based on a user's way of life and values.

[0539] "Goods" refers to the general term for products and services offered to users.

[0540] "Purchase history" refers to a series of records related to products that a user has purchased in the past.

[0541] "Browsing history" refers to the history of web pages a user has visited on the internet.

[0542] "Social networking data" refers to information obtained from the social media platforms that users utilize.

[0543] "Artificial intelligence processing" is an information processing technology used by computers to analyze user characteristics and select appropriate items.

[0544] Augmented reality technology is a technique that overlays computer-generated visual information onto the real world.

[0545] "Market value" refers to the price information at which an item is currently offered in the market.

[0546] "Discount information" refers to economic incentives or promotional information applicable to goods.

[0547] "Natural language processing" is a technology that enables computers to understand human language and generate appropriate responses.

[0548] "Virtual trial" refers to the act of using a computer to simulate the experience of a user trying out an item they have selected.

[0549] "Personalized product recommendations" refer to suggestions for the most suitable items selected based on multiple characteristics of a given user.

[0550] This invention is a system for providing users with a more sophisticated shopping experience, and is implemented in the following manner.

[0551] First, when a user logs into the e-commerce platform, the server, with their authorization, collects purchase history, browsing history, and social networking data from their device. This collected data constitutes characteristics representing the user's preferences, traits, budget, and lifestyle. At this stage, a database system (e.g., MySQL or MongoDB) is used to securely store the data.

[0552] Next, the server uses the characteristic data received from the terminal to perform artificial intelligence processing. The algorithm analyzes a vast dataset and selects items that the user is likely to be interested in. This process utilizes machine learning frameworks (such as TensorFlow or PyTorch).

[0553] Selected items are presented to the user via a device. The user can use the device to select items of interest and virtually try them on using augmented reality technology. Specifically, ARKit (iOS) or ARCore (Android) is installed on the device, allowing the items to be virtually tried on according to the user's characteristics and environment. For example, a user can virtually try on a jacket they have selected using AR.

[0554] Furthermore, the server collects market value data for the items selected by the user from various internet resources and indexes the most economical purchasing options. Web scraping techniques using Python's Beautiful Soup and Selenium are used for this information gathering.

[0555] In addition, if a user has questions about a particular item or service, they can ask them in natural language. The terminal receives this question, analyzes it using a natural language processing engine, and the server retrieves relevant answers from a database or external sources and provides them to the user. At this stage, OpenAI's generative AI model plays a role in generating natural-sounding responses. A concrete example of a prompt would be, "What material is this jacket made of?"

[0556] This allows the system to simultaneously provide users with intuitive and personalized product recommendations, economical purchasing options, and a trial experience using augmented reality.

[0557] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0558] Step 1:

[0559] When a user logs into the e-commerce platform, the server collects purchase history, browsing history, and social networking data from the user's device with the user's permission. This data forms the basis for generating user characteristics. The input data is classified according to its type and stored in a database. The output is the basic data used to construct the user profile.

[0560] Step 2:

[0561] The server performs artificial intelligence processing based on the stored data. This process generates user profiles using machine learning models. The input is data about the user's purchase history and preferences, and by analyzing this data, it identifies items that the user is predicted to be interested in. Specifically, models built with TensorFlow or PyTorch run, and the output is a list of individualized items.

[0562] Step 3:

[0563] On the device, selected items are suggested to the user. The user selects an item and uses the device's AR function to virtually try on the selected item on their body or in their environment. The input is the data of the selected item, and the output is a visual trial experience generated using AR. ARKit (iOS) or ARCore (Android) is used for operation.

[0564] Step 4:

[0565] The server collects market value and discount information for selected items from the internet. The input is the item's identification information, and web scraping is performed using Python's Beautiful Soup or Selenium to obtain the most economical price information. The output provides the price and discount information to be presented to the user.

[0566] Step 5:

[0567] The terminal receives user inquiries about goods and services in natural language. The input is the user's question. The received question is analyzed by a natural language processing engine, and the server generates an appropriate answer via an AI model. For example, if the prompt "What material is this jacket made of?" is entered, the answer will output material information.

[0568] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0569] The present invention is a system that combines an emotion engine to enhance the user's personalized shopping experience, and is implemented as follows.

[0570] First, when a user logs into the application, the device retrieves purchase history, web browsing history, and social networking data. Based on this data, a profile is generated that reflects the user's preferences, body type, budget, and lifestyle.

[0571] Next, the server analyzes the generated user profile and uses an artificial intelligence algorithm to select the most suitable product for the user. This selection process also incorporates an emotion engine that analyzes the user's facial recognition, voice tone, body movements, and other factors.

[0572] The emotion engine recognizes the user's emotions in real time from their facial expressions and voice, and sends that data to the server. This allows the artificial intelligence algorithm to make product recommendations tailored to the user's emotional state.

[0573] The selected products are presented to the user via the terminal. If the user shows interest in a product, they can try it on or place it using augmented reality technology. During this process, the emotion engine continues to monitor the user's emotional changes and update the data accordingly.

[0574] Furthermore, the server compares market prices while taking sentiment data into account, and provides users with the most advantageous purchase options and available coupon information.

[0575] As a concrete example, while a user is using an apparel app on their smartphone, the emotion engine recognizes the user's face and analyzes their emotions. If it detects that the user's emotions towards the selected clothing are positive, related accessories and additional items are suggested. Conversely, if it determines that the user is losing interest, items in different styles or colors are presented, and measures are taken to rekindle the user's interest.

[0576] This allows users' emotional states to be reflected in their product choices, enabling a better shopping experience.

[0577] The following describes the processing flow.

[0578] Step 1:

[0579] The user logs into an online shopping platform. Once the application launches, information collection begins with the user's consent.

[0580] Step 2:

[0581] The device collects the user's purchase history, web browsing history, and social networking data. This data is stored locally.

[0582] Step 3:

[0583] The device sends the collected data to the server. The server analyzes it and generates a profile based on the user's preferences, body type, budget, and lifestyle.

[0584] Step 4:

[0585] The server uses the generated user profile to execute an artificial intelligence algorithm and select product candidates.

[0586] Step 5:

[0587] Using the camera and microphone built into the device, the emotion engine analyzes the user's facial expressions and voice tone in real time and generates emotion data.

[0588] Step 6:

[0589] The device sends emotional data to the server. Based on this data, the server dynamically adjusts product recommendations and selects the item that best suits the user's current emotional state.

[0590] Step 7:

[0591] The server sends a product list containing the selected items to the terminal. The terminal displays this list on the user's screen.

[0592] Step 8:

[0593] When a user selects an item they are interested in, the device uses augmented reality technology to provide visual feedback, such as trying on the selected item or placing it in the room.

[0594] Step 9:

[0595] The server checks the market price of the currently selected product and compares it to prices at other stores. It also collects information on available coupons.

[0596] Step 10:

[0597] The server sends the most favorable pricing information and available coupons to the terminal and presents them to the user.

[0598] Step 11:

[0599] The device continues to monitor the user's facial expressions and voice, tracking changes in their emotions. Based on this data, the server makes new product suggestions and adjusts the interface to maintain the user's interest.

[0600] Step 12:

[0601] Based on the information presented, the user decides to purchase the product and completes the purchase process.

[0602] (Example 2)

[0603] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0604] Traditional online shopping systems have the problem of failing to adequately reflect users' preferences and emotional states in product recommendations, resulting in a lack of personalized shopping experiences. Furthermore, virtual try-ons and visualizations of products users are interested in are insufficient, lacking means to increase purchasing intent. Finding the most economical purchase option from market prices is also not done efficiently. Therefore, there is a problem in that these systems fail to provide users with the optimal shopping experience.

[0605] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0606] This invention includes a server that provides opportunities to suggest optimal products based on the user's preferences, body type, budget, and lifestyle; a server that analyzes the user profile using a machine learning algorithm and provides real-time product recommendations including emotional data; and a server that allows the user to virtually experience selected products using augmented reality technology. This enables the user to enjoy a personalized shopping experience and receive product suggestions that respond to their emotions.

[0607] A "user profile" is a collection of information that reflects the individual characteristics of a user, such as their preferences, body type, budget, and lifestyle.

[0608] A "machine learning algorithm" is a computational method that learns patterns from large amounts of data and uses that knowledge to make predictions and decisions.

[0609] "Emotional data" refers to emotional information analyzed from a user's facial expressions, tone of voice, and other factors, and is acquired in real time.

[0610] Augmented reality technology refers to a technology that overlays digital information onto the real world and presents it to the user, allowing them to virtually experience products.

[0611] "Social network data" is a general term for digital information that includes users' activity history and interests on social media and online platforms.

[0612] "Natural language processing" is a computational technique for analyzing and understanding human language, and it provides a function to interpret user questions.

[0613] This invention is a system designed to enhance the user's personalized shopping experience. The system is implemented using the user's terminal, a server, an emotion engine, and associated software. Details are provided below.

[0614] The device activates when a user logs into a dedicated application and collects user data. This data includes purchase history, web browsing history, and social network data. Collection and storage are performed using a database API and securely accessed from cloud-based data storage.

[0615] The server processes the collected data to create user profiles. These profiles are analyzed using artificial intelligence algorithms and used to select and suggest the most suitable products for each user. Machine learning libraries (such as Python's Pandas and Scikit-learn) are utilized in this process.

[0616] The system also incorporates an emotion recognition engine that analyzes the user's facial and voice data in real time. OpenCV and TensorFlow can be used for this analysis. This allows the user's instantaneous emotional state to be transmitted to the server, which is then reflected in appropriate product recommendations.

[0617] Products that a user shows interest in are visualized on the device using augmented reality technology. This involves using ARKit or ARCore to enable virtual try-ons and product placement. The server also compares prices using market data and presents the user with the most economical purchase options and discount information.

[0618] As a concrete example, when a user interacts with an apparel app, the emotion engine scans the user's face and analyzes emotional data. If a positive response is detected, the server recommends related accessories and items.

[0619] An example of a prompt to a generative AI model is, "Use the user's sentiment data to create a list of recommended apparel items." In this way, the system can provide the user with a personalized, emotion-driven shopping experience.

[0620] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0621] Step 1:

[0622] When a user logs into the dedicated application, the device retrieves purchase history, web browsing history, and social network data. User authentication information is used as input, and the retrieved data is collected from cloud storage via a database API. This allows for the aggregation of diverse user activity histories.

[0623] Step 2:

[0624] The server analyzes the collected data to generate user profiles. Historical data is used as input, and machine learning algorithms are executed for analysis. Data processing is performed using Python's Pandas library, and the output is a profile that reflects the user's preferences and lifestyle.

[0625] Step 3:

[0626] The emotion engine scans the user's face and voice in real time and extracts emotion data. The input at this stage is raw data from the camera and microphone, and emotion analysis is performed using OpenCV or TensorFlow. The output is sent to the server as analyzed emotion data.

[0627] Step 4:

[0628] The server selects the optimal product using user profiles and sentiment data. A generative AI model processes the profile data and sentiment data as input and generates a list of recommended products as output. Libraries such as Scikit-learn are useful for this data processing.

[0629] Step 5:

[0630] The terminal uses a product list received from the server to make suggestions to the user. It takes a product list as input and provides a virtual try-on of the products using augmented reality as output. This operation uses ARKit or ARCore to enable virtual try-on and placement of selected products.

[0631] Step 6:

[0632] The server compares user data with market price data and presents the most economical purchase options and discount information. Using market data and user conditions as input, the algorithm performs optimization. As output, the user receives the most suitable price information on their terminal.

[0633] (Application Example 2)

[0634] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0635] Modern consumers have access to a wide variety of products, increasing the burden of choosing the best option for themselves. Furthermore, traditional online and offline shopping processes lack dynamic product recommendations based on consumer emotions and preferences. This makes it difficult to provide an efficient and satisfying shopping experience.

[0636] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0637] In this invention, the server includes means for collecting user information data and generating user information using a knowledge-based system, means for virtually trying on or arranging products using augmented reality technology, and means for analyzing the user's emotional state and adjusting product suggestions in real time using the knowledge-based system. This enables personalized product suggestions that respond to the user's emotions and preferences, providing an efficient and satisfying shopping experience.

[0638] "User preferences" refer to the characteristics and distinctive tendencies of products and services that individual users like.

[0639] "Body type" refers to the size and shape of the user's body.

[0640] "Budget" refers to the range of money a user plans to spend on a product or service.

[0641] "Lifestyle" is a concept that describes how a user spends their daily life, their values, habits, and preferences.

[0642] "Purchase history" refers to a record of products and services that a user has purchased in the past.

[0643] "Information browsing history" refers to the history of a user's browsing activity on websites and online platforms.

[0644] "Information sharing data" refers to information related to user activities and interests that is shared through social media and other communication tools.

[0645] "User information" refers to profile information generated based on the user's individual preferences, body type, budget, lifestyle, etc.

[0646] A "knowledge-based system" refers to a system that analyzes user information data and executes artificial intelligence algorithms to provide optimal product recommendations.

[0647] Augmented reality technology is a technique that overlays virtual information and objects onto the real world environment.

[0648] "Emotional state" refers to the real-time emotional stage analyzed from the user's facial expressions and voice.

[0649] "Discount information" refers to coupons and promotional information designed to lower the prices of products offered in the market.

[0650] The system of this invention includes a program that analyzes various data to suggest the optimal product based on the user's preferences, body type, budget, and lifestyle. The terminal first acquires the user's purchase history, information browsing history, and information sharing data, and generates user information based on this. The server uses a knowledge-based system to analyze this user information and select the most suitable product for the user. Here, the user's facial expressions and voice information are collected in real time for sentiment analysis. Based on this data, the server dynamically adjusts the product suggestions.

[0651] Utilizing augmented reality technology, the device provides users with virtual try-on and placement functions for selected products. This allows users to visually examine products and make purchase decisions. This technology is implemented using augmented reality software such as Unity and ARKit. Furthermore, by presenting the most advantageous discount information available on the market, users can make economical choices.

[0652] As a concrete example, in an application using a smart mirror in a physical store, the system automatically starts operating when a user stands in front of the mirror. The mirror suggests products optimized for the user in real time, and selected products can be virtually tried on using augmented reality technology. An example of a prompt would be, "Please show me a demo of a smart mirror app that uses emotion analysis technology to improve the customer experience in a physical store. Please suggest products optimized based on the user's facial expressions, and allow the user to virtually try on items they are interested in using augmented reality."

[0653] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0654] Step 1:

[0655] The device collects purchase history, browsing history, and shared information data when a user logs into an application. Based on this data, the device generates user information that reflects the user's preferences, body type, budget, and lifestyle. The input to this process is the user's past behavioral data, and the output is the generated user information.

[0656] Step 2:

[0657] The server receives the generated user information and performs analysis using a knowledge-based system. The server applies artificial intelligence algorithms as data processing to select the most suitable product for the user, and outputs a list of the resulting products. In this process, user information is used as input, and a list of optimal products is obtained as output.

[0658] Step 3:

[0659] The device captures the user's facial expressions using a camera sensor and sends them to the server. The server performs emotion analysis based on this data to understand the user's emotional state. The emotional state obtained from data calculations by the emotion analysis engine, using the user's facial expression data as input, is output.

[0660] Step 4:

[0661] The server updates the algorithms within the knowledge base system based on the results of the sentiment analysis, dynamically adjusting the product recommendations. Here, the analyzed sentiment state and the optimal product list are taken as input, and the newly adjusted product recommendations are output.

[0662] Step 5:

[0663] The device uses augmented reality software to visually present virtual try-ons and placements of products the user has expressed interest in. The input to this process is tailored product suggestions, and the output is a video of the virtual try-on or placement presented to the user.

[0664] Step 6:

[0665] The server collects market price information and provides users with purchase options, including the most advantageous discounts. The input needed to attract user interest is a tailored product suggestion, and the output is a list of discount information and purchase options.

[0666] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0667] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0668] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0669] [Fourth Embodiment]

[0670] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0671] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0672] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0673] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0674] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0675] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0676] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0677] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0678] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0679] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0680] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0681] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0682] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0683] This invention is a system for providing consumers with the optimal shopping experience, and is implemented as follows.

[0684] First, when a user logs into an online shopping platform, the device collects purchase history, web browsing history, and social networking data from that device with the user's consent. This collected data forms a profile representing the user's preferences, body type, budget, lifestyle, and other characteristics.

[0685] The server uses profile data received from the terminal to execute an artificial intelligence algorithm and select products that match the user's preferences. This algorithm analyzes a vast dataset to identify products that the user is likely to be interested in.

[0686] The selected products are suggested to the user via the device. The user can choose items of interest from these suggestions and virtually try them on or place them in their home. Specifically, augmented reality technology is used to display an image on the device's screen showing how the product fits the user's body type and how it would look placed in their home room.

[0687] The server also collects market prices for the user's selected product from numerous internet resources and compares them. Furthermore, it gathers information on available coupons, identifies the most advantageous purchase option, and presents it to the user via the terminal.

[0688] In addition, if a user has questions about a specific product or service, they can ask them in natural language. The device receives this question, analyzes it using a natural language processing engine, and the server retrieves the appropriate answer from a database or external information source before providing it to the user through the device.

[0689] For example, if a user wants to buy a new jacket, jackets that match their preferences are suggested based on their profile. They can then choose one they like, virtually try it on, and confirm the appropriate size and style. Furthermore, the system presents the most cost-effective way to purchase the jacket, taking into account the suggested price and any coupon information. This ensures a satisfying shopping experience for the user.

[0690] The following describes the processing flow.

[0691] Step 1:

[0692] A user logs into an online shopping platform. At this point, information collection begins with the user's consent.

[0693] Step 2:

[0694] The device collects the user's purchase history, web browsing history, and social networking data, and temporarily stores it in local storage.

[0695] Step 3:

[0696] The device sends the collected data to the server. The server analyzes the received data and generates a user profile.

[0697] Step 4:

[0698] The server uses the generated user profile to execute an artificial intelligence algorithm and select products from the database that match the user's preferences.

[0699] Step 5:

[0700] The server sends the selected product list to the terminal. The terminal displays this product list to the user.

[0701] Step 6:

[0702] The user selects an item of interest from the displayed product list and virtually tries it on or places it in the room. The device uses augmented reality technology to visually provide the user with the opportunity to try on or place the item.

[0703] Step 7:

[0704] The server collects and compares market prices for selected products from multiple online stores. It also searches for available coupon information.

[0705] Step 8:

[0706] The server sends the cheapest purchase option and available coupons to the terminal and presents them to the user.

[0707] Step 9:

[0708] When a user has a question about a product or service, they ask it in natural language. The device receives this question and interprets it using an NLP engine.

[0709] Step 10:

[0710] The server retrieves answers from databases and external sources based on the interpreted question and provides them to the user through the terminal.

[0711] Step 11:

[0712] Users complete their purchase after experiencing a satisfying shopping experience.

[0713] (Example 1)

[0714] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0715] Modern consumers face the challenge of finding products online that suit their preferences, lifestyles, and budgets. Choosing the most economical and appropriate item from a diverse range of options is also a burden. Furthermore, it's difficult to see how a product will look in their home environment before purchasing it.

[0716] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0717] In this invention, the server includes means for suggesting the most suitable products based on the user's characteristic information, means for selecting and suggesting products using an intelligent algorithm, and means for virtually trying out or arranging products using virtual technology. This allows users to easily find economical and appropriate products that match their preferences and lifestyle, and to check the appearance of those products before purchasing them.

[0718] "User characteristic information" refers to a profile that includes information such as the user's preferences, appearance, budget, and lifestyle.

[0719] "Commercial products" refers to the general term for goods or services offered to users.

[0720] "Transaction history" refers to a record of purchases made by a user in the past.

[0721] "Web visit information" refers to information about the websites a user has accessed and the content they have viewed on the internet.

[0722] "Social data" refers to information and records of interactions shared by users on social networking sites.

[0723] An "intelligent algorithm" is a set of computational procedures that use artificial intelligence technology to analyze data and derive results that align with a specific purpose.

[0724] "Virtualization technology" is a technique that simulates objects and environments that do not actually exist on a computer.

[0725] "Language processing" is the technology that enables computers to understand and process human language appropriately.

[0726] "Discount" refers to any discount or benefit applied at the time of purchase.

[0727] This system, as a form of implementing the invention, aims to provide consumers with the best possible shopping experience. The following illustrates how this system works.

[0728] First, the device operates on the user's device under certain conditions. When the user logs into an online shopping platform, the device collects transaction history, web visit information, and social data with the user's consent. This generates user characteristic information, which includes preferences, appearance, budget, and lifestyle.

[0729] The server executes an intelligent algorithm based on user characteristic information received from the terminal. This algorithm analyzes a wide range of datasets and has a process for selecting products suitable for the user. The software used includes a generative AI model that can process large amounts of data quickly and enable recommendations based on individual user preferences.

[0730] The selected products are then presented to the user via a terminal. Virtual technology is used to allow the user to virtually try on or place the products in their own environment or home space for visual confirmation. Any device with a standard display can be used for this purpose.

[0731] Furthermore, the server researches market prices for selected products via the internet and identifies the most economical purchase option. In addition, the server uses natural language processing technology to analyze user inquiries and provide appropriate answers and information. This further enhances the user's purchasing experience.

[0732] For example, if a user has a request to "find a jacket that suits a casual style," they can input a prompt such as "Please suggest jackets that suit a casual style for men in their 30s." Based on this prompt, the system will select and suggest appropriate jackets based on the user's profile. In addition, it is possible to visualize how the jacket will look beforehand by utilizing the virtual try-on function.

[0733] This format allows users to have a more satisfying shopping experience.

[0734] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0735] Step 1:

[0736] The device detects when a user logs into an online shopping platform. Once the user logs in, the device, with the user's consent, collects transaction history, web visit information, and social data. Input data includes the user's past purchase history, viewed pages, and social network activity records. This data is acquired and output as user characteristic information.

[0737] Step 2:

[0738] The terminal sends user characteristic information it has acquired to the server. The server receives this information as input and analyzes the data using intelligent algorithms. This analysis creates and outputs a list of products best suited to the user. This process involves analyzing a large dataset to select products that match the user's preferences and budget.

[0739] Step 3:

[0740] The server sends the created product list to the terminal. The terminal uses this list to generate an interface that allows the user to easily compare options. The user can view this interface and select products that interest them. The output includes images and detailed information about the products presented to the user.

[0741] Step 4:

[0742] When a user selects items of interest from a curated list, the device uses virtual technology to virtually try out or place those items. This process involves displaying the selected items to fit the user's appearance or generating an image of them placed in a room. The input is the user-selected items, and the output is a virtually visualized image.

[0743] Step 5:

[0744] The server collects and compares market prices for selected products from internet sources. It then identifies the most economical option and available discounts, and sends them to the terminal. The input is market price information and discount information, and the output is the optimal purchase option.

[0745] Step 6:

[0746] If a user has questions about product details or purchasing, they can make inquiries using natural language. The terminal receives these inquiries and parses them via a language processing engine. The server retrieves solutions and relevant information from its database and provides them to the user through the terminal. The input is the user's inquiry, and the output is the relevant answer or information.

[0747] (Application Example 1)

[0748] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0749] Traditional online shopping systems have several drawbacks, including low accuracy in recommending products to users, the inability to virtually try on or view physical items, and the difficulty in easily selecting economical purchasing options. Furthermore, their ability to respond quickly to user inquiries has been limited. There is a need to address these challenges and provide users with an intuitive and personalized shopping experience.

[0750] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0751] In this invention, the server includes means for using artificial intelligence processing to generate user characteristics and select items, means for enabling virtual trial or placement using augmented reality technology, and means for collecting market value and discount information to present economical purchasing options. This makes it possible to provide the user with personalized product suggestions and a trial experience similar to that of actual items, and to present the most economical purchasing options.

[0752] "Preference" is a concept that refers to the individual tendencies or preferences of a user regarding specific goods or services they like.

[0753] "Characteristics" refer to elements that indicate the unique characteristics of each individual user, such as their body type and behavioral patterns.

[0754] "Budget" refers to the range of funds a user can allocate to purchases.

[0755] "Lifestyle" refers to patterns of habits and behaviors based on a user's way of life and values.

[0756] "Goods" refers to the general term for products and services offered to users.

[0757] "Purchase history" refers to a series of records related to products that a user has purchased in the past.

[0758] "Browsing history" refers to the history of web pages a user has visited on the internet.

[0759] "Social networking data" refers to information obtained from the social media platforms that users utilize.

[0760] "Artificial intelligence processing" is an information processing technology used by computers to analyze user characteristics and select appropriate items.

[0761] Augmented reality technology is a technique that overlays computer-generated visual information onto the real world.

[0762] "Market value" refers to the price information at which an item is currently offered in the market.

[0763] "Discount information" refers to economic incentives or promotional information applicable to goods.

[0764] "Natural language processing" is a technology that enables computers to understand human language and generate appropriate responses.

[0765] "Virtual trial" refers to the act of using a computer to simulate the experience of a user trying out an item they have selected.

[0766] "Personalized product recommendations" refer to suggestions for the most suitable items selected based on multiple characteristics of a given user.

[0767] This invention is a system for providing users with a more sophisticated shopping experience, and is implemented in the following manner.

[0768] First, when a user logs into the e-commerce platform, the server, with their authorization, collects purchase history, browsing history, and social networking data from their device. This collected data constitutes characteristics representing the user's preferences, traits, budget, and lifestyle. At this stage, a database system (e.g., MySQL or MongoDB) is used to securely store the data.

[0769] Next, the server uses the characteristic data received from the terminal to perform artificial intelligence processing. The algorithm analyzes a vast dataset and selects items that the user is likely to be interested in. This process utilizes machine learning frameworks (such as TensorFlow or PyTorch).

[0770] Selected items are presented to the user via a device. The user can use the device to select items of interest and virtually try them on using augmented reality technology. Specifically, ARKit (iOS) or ARCore (Android) is installed on the device, allowing the items to be virtually tried on according to the user's characteristics and environment. For example, a user can virtually try on a jacket they have selected using AR.

[0771] Furthermore, the server collects market value data for the items selected by the user from various internet resources and indexes the most economical purchasing options. Web scraping techniques using Python's Beautiful Soup and Selenium are used for this information gathering.

[0772] In addition, if a user has questions about a particular item or service, they can ask them in natural language. The terminal receives this question, analyzes it using a natural language processing engine, and the server retrieves relevant answers from a database or external sources and provides them to the user. At this stage, OpenAI's generative AI model plays a role in generating natural-sounding responses. A concrete example of a prompt would be, "What material is this jacket made of?"

[0773] This allows the system to simultaneously provide users with intuitive and personalized product recommendations, economical purchasing options, and a trial experience using augmented reality.

[0774] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0775] Step 1:

[0776] When a user logs into the e-commerce platform, the server collects purchase history, browsing history, and social networking data from the user's device with the user's permission. This data forms the basis for generating user characteristics. The input data is classified according to its type and stored in a database. The output is the basic data used to construct the user profile.

[0777] Step 2:

[0778] The server performs artificial intelligence processing based on the stored data. This process generates user profiles using machine learning models. The input is data about the user's purchase history and preferences, and by analyzing this data, it identifies items that the user is predicted to be interested in. Specifically, models built with TensorFlow or PyTorch run, and the output is a list of individualized items.

[0779] Step 3:

[0780] On the device, selected items are suggested to the user. The user selects an item and uses the device's AR function to virtually try on the selected item on their body or in their environment. The input is the data of the selected item, and the output is a visual trial experience generated using AR. ARKit (iOS) or ARCore (Android) is used for operation.

[0781] Step 4:

[0782] The server collects market value and discount information for selected items from the internet. The input is the item's identification information, and web scraping is performed using Python's Beautiful Soup or Selenium to obtain the most economical price information. The output provides the price and discount information to be presented to the user.

[0783] Step 5:

[0784] The terminal receives user inquiries about goods and services in natural language. The input is the user's question. The received question is analyzed by a natural language processing engine, and the server generates an appropriate answer via an AI model. For example, if the prompt "What material is this jacket made of?" is entered, the answer will output material information.

[0785] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0786] The present invention is a system that combines an emotion engine to enhance the user's personalized shopping experience, and is implemented as follows.

[0787] First, when a user logs into the application, the device retrieves purchase history, web browsing history, and social networking data. Based on this data, a profile is generated that reflects the user's preferences, body type, budget, and lifestyle.

[0788] Next, the server analyzes the generated user profile and uses an artificial intelligence algorithm to select the most suitable product for the user. This selection process also incorporates an emotion engine that analyzes the user's facial recognition, voice tone, body movements, and other factors.

[0789] The emotion engine recognizes the user's emotions in real time from their facial expressions and voice, and sends that data to the server. This allows the artificial intelligence algorithm to make product recommendations tailored to the user's emotional state.

[0790] The selected products are presented to the user via the terminal. If the user shows interest in a product, they can try it on or place it using augmented reality technology. During this process, the emotion engine continues to monitor the user's emotional changes and update the data accordingly.

[0791] Furthermore, the server compares market prices while taking sentiment data into account, and provides users with the most advantageous purchase options and available coupon information.

[0792] As a concrete example, while a user is using an apparel app on their smartphone, the emotion engine recognizes the user's face and analyzes their emotions. If it detects that the user's emotions towards the selected clothing are positive, related accessories and additional items are suggested. Conversely, if it determines that the user is losing interest, items in different styles or colors are presented, and measures are taken to rekindle the user's interest.

[0793] This allows users' emotional states to be reflected in their product choices, enabling a better shopping experience.

[0794] The following describes the processing flow.

[0795] Step 1:

[0796] The user logs into an online shopping platform. Once the application launches, information collection begins with the user's consent.

[0797] Step 2:

[0798] The device collects the user's purchase history, web browsing history, and social networking data. This data is stored locally.

[0799] Step 3:

[0800] The device sends the collected data to the server. The server analyzes it and generates a profile based on the user's preferences, body type, budget, and lifestyle.

[0801] Step 4:

[0802] The server uses the generated user profile to execute an artificial intelligence algorithm and select product candidates.

[0803] Step 5:

[0804] Using the camera and microphone built into the device, the emotion engine analyzes the user's facial expressions and voice tone in real time and generates emotion data.

[0805] Step 6:

[0806] The device sends emotional data to the server. Based on this data, the server dynamically adjusts product recommendations and selects the item that best suits the user's current emotional state.

[0807] Step 7:

[0808] The server sends a product list containing the selected items to the terminal. The terminal displays this list on the user's screen.

[0809] Step 8:

[0810] When a user selects an item they are interested in, the device uses augmented reality technology to provide visual feedback, such as trying on the selected item or placing it in the room.

[0811] Step 9:

[0812] The server checks the market price of the currently selected product and compares it to prices at other stores. It also collects information on available coupons.

[0813] Step 10:

[0814] The server sends the most favorable pricing information and available coupons to the terminal and presents them to the user.

[0815] Step 11:

[0816] The device continues to monitor the user's facial expressions and voice, tracking changes in their emotions. Based on this data, the server makes new product suggestions and adjusts the interface to maintain the user's interest.

[0817] Step 12:

[0818] Based on the information presented, the user decides to purchase the product and completes the purchase process.

[0819] (Example 2)

[0820] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0821] Traditional online shopping systems have the problem of failing to adequately reflect users' preferences and emotional states in product recommendations, resulting in a lack of personalized shopping experiences. Furthermore, virtual try-ons and visualizations of products users are interested in are insufficient, lacking means to increase purchasing intent. Finding the most economical purchase option from market prices is also not done efficiently. Therefore, there is a problem in that these systems fail to provide users with the optimal shopping experience.

[0822] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0823] This invention includes a server that provides opportunities to suggest optimal products based on the user's preferences, body type, budget, and lifestyle; a server that analyzes the user profile using a machine learning algorithm and provides real-time product recommendations including emotional data; and a server that allows the user to virtually experience selected products using augmented reality technology. This enables the user to enjoy a personalized shopping experience and receive product suggestions that respond to their emotions.

[0824] A "user profile" is a collection of information that reflects the individual characteristics of a user, such as their preferences, body type, budget, and lifestyle.

[0825] A "machine learning algorithm" is a computational method that learns patterns from large amounts of data and uses that knowledge to make predictions and decisions.

[0826] "Emotional data" refers to emotional information analyzed from a user's facial expressions, tone of voice, and other factors, and is acquired in real time.

[0827] Augmented reality technology refers to a technology that overlays digital information onto the real world and presents it to the user, allowing them to virtually experience products.

[0828] "Social network data" is a general term for digital information that includes users' activity history and interests on social media and online platforms.

[0829] "Natural language processing" is a computational technique for analyzing and understanding human language, and it provides a function to interpret user questions.

[0830] This invention is a system designed to enhance the user's personalized shopping experience. The system is implemented using the user's terminal, a server, an emotion engine, and associated software. Details are provided below.

[0831] The device activates when a user logs into a dedicated application and collects user data. This data includes purchase history, web browsing history, and social network data. Collection and storage are performed using a database API and securely accessed from cloud-based data storage.

[0832] The server processes the collected data to create user profiles. These profiles are analyzed using artificial intelligence algorithms and used to select and suggest the most suitable products for each user. Machine learning libraries (such as Python's Pandas and Scikit-learn) are utilized in this process.

[0833] The system also incorporates an emotion recognition engine that analyzes the user's facial and voice data in real time. OpenCV and TensorFlow can be used for this analysis. This allows the user's instantaneous emotional state to be transmitted to the server, which is then reflected in appropriate product recommendations.

[0834] Products that a user shows interest in are visualized on the device using augmented reality technology. This involves using ARKit or ARCore to enable virtual try-ons and product placement. The server also compares prices using market data and presents the user with the most economical purchase options and discount information.

[0835] As a concrete example, when a user interacts with an apparel app, the emotion engine scans the user's face and analyzes emotional data. If a positive response is detected, the server recommends related accessories and items.

[0836] An example of a prompt to a generative AI model is, "Use the user's sentiment data to create a list of recommended apparel items." In this way, the system can provide the user with a personalized, emotion-driven shopping experience.

[0837] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0838] Step 1:

[0839] When a user logs into the dedicated application, the device retrieves purchase history, web browsing history, and social network data. User authentication information is used as input, and the retrieved data is collected from cloud storage via a database API. This allows for the aggregation of diverse user activity histories.

[0840] Step 2:

[0841] The server analyzes the collected data to generate user profiles. Historical data is used as input, and machine learning algorithms are executed for analysis. Data processing is performed using Python's Pandas library, and the output is a profile that reflects the user's preferences and lifestyle.

[0842] Step 3:

[0843] The emotion engine scans the user's face and voice in real time and extracts emotion data. The input at this stage is raw data from the camera and microphone, and emotion analysis is performed using OpenCV or TensorFlow. The output is sent to the server as analyzed emotion data.

[0844] Step 4:

[0845] The server selects the optimal product using user profiles and sentiment data. A generative AI model processes the profile data and sentiment data as input and generates a list of recommended products as output. Libraries such as Scikit-learn are useful for this data processing.

[0846] Step 5:

[0847] The terminal uses a product list received from the server to make suggestions to the user. It takes a product list as input and provides a virtual try-on of the products using augmented reality as output. This operation uses ARKit or ARCore to enable virtual try-on and placement of selected products.

[0848] Step 6:

[0849] The server compares user data with market price data and presents the most economical purchase options and discount information. Using market data and user conditions as input, the algorithm performs optimization. As output, the user receives the most suitable price information on their terminal.

[0850] (Application Example 2)

[0851] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0852] Modern consumers have access to a wide variety of products, increasing the burden of choosing the best option for themselves. Furthermore, traditional online and offline shopping processes lack dynamic product recommendations based on consumer emotions and preferences. This makes it difficult to provide an efficient and satisfying shopping experience.

[0853] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0854] In this invention, the server includes means for collecting user information data and generating user information using a knowledge-based system, means for virtually trying on or arranging products using augmented reality technology, and means for analyzing the user's emotional state and adjusting product suggestions in real time using the knowledge-based system. This enables personalized product suggestions that respond to the user's emotions and preferences, providing an efficient and satisfying shopping experience.

[0855] "User preferences" refer to the characteristics and distinctive tendencies of products and services that individual users like.

[0856] "Body type" refers to the size and shape of the user's body.

[0857] "Budget" refers to the range of money a user plans to spend on a product or service.

[0858] "Lifestyle" is a concept that describes how a user spends their daily life, their values, habits, and preferences.

[0859] "Purchase history" refers to a record of products and services that a user has purchased in the past.

[0860] "Information browsing history" refers to the history of a user's browsing activity on websites and online platforms.

[0861] "Information sharing data" refers to information related to user activities and interests that is shared through social media and other communication tools.

[0862] "User information" refers to profile information generated based on the user's individual preferences, body type, budget, lifestyle, etc.

[0863] A "knowledge-based system" refers to a system that analyzes user information data and executes artificial intelligence algorithms to provide optimal product recommendations.

[0864] Augmented reality technology is a technique that overlays virtual information and objects onto the real world environment.

[0865] "Emotional state" refers to the real-time emotional stage analyzed from the user's facial expressions and voice.

[0866] "Discount information" refers to coupons and promotional information designed to lower the prices of products offered in the market.

[0867] The system of this invention includes a program that analyzes various data to suggest the optimal product based on the user's preferences, body type, budget, and lifestyle. The terminal first acquires the user's purchase history, information browsing history, and information sharing data, and generates user information based on this. The server uses a knowledge-based system to analyze this user information and select the most suitable product for the user. Here, the user's facial expressions and voice information are collected in real time for sentiment analysis. Based on this data, the server dynamically adjusts the product suggestions.

[0868] Utilizing augmented reality technology, the device provides users with virtual try-on and placement functions for selected products. This allows users to visually examine products and make purchase decisions. This technology is implemented using augmented reality software such as Unity and ARKit. Furthermore, by presenting the most advantageous discount information available on the market, users can make economical choices.

[0869] As a concrete example, in an application using a smart mirror in a physical store, the system automatically starts operating when a user stands in front of the mirror. The mirror suggests products optimized for the user in real time, and selected products can be virtually tried on using augmented reality technology. An example of a prompt would be, "Please show me a demo of a smart mirror app that uses emotion analysis technology to improve the customer experience in a physical store. Please suggest products optimized based on the user's facial expressions, and allow the user to virtually try on items they are interested in using augmented reality."

[0870] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0871] Step 1:

[0872] The device collects purchase history, browsing history, and shared information data when a user logs into an application. Based on this data, the device generates user information that reflects the user's preferences, body type, budget, and lifestyle. The input to this process is the user's past behavioral data, and the output is the generated user information.

[0873] Step 2:

[0874] The server receives the generated user information and performs analysis using a knowledge-based system. The server applies artificial intelligence algorithms as data processing to select the most suitable product for the user, and outputs a list of the resulting products. In this process, user information is used as input, and a list of optimal products is obtained as output.

[0875] Step 3:

[0876] The device captures the user's facial expressions using a camera sensor and sends them to the server. The server performs emotion analysis based on this data to understand the user's emotional state. The emotional state obtained from data calculations by the emotion analysis engine, using the user's facial expression data as input, is output.

[0877] Step 4:

[0878] The server updates the algorithms within the knowledge base system based on the results of the sentiment analysis, dynamically adjusting the product recommendations. Here, the analyzed sentiment state and the optimal product list are taken as input, and the newly adjusted product recommendations are output.

[0879] Step 5:

[0880] The device uses augmented reality software to visually present virtual try-ons and placements of products the user has expressed interest in. The input to this process is tailored product suggestions, and the output is a video of the virtual try-on or placement presented to the user.

[0881] Step 6:

[0882] The server collects market price information and provides users with purchase options, including the most advantageous discounts. The input needed to attract user interest is a tailored product suggestion, and the output is a list of discount information and purchase options.

[0883] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0884] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0885] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0886] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0887] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0888] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0889] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0890] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0891] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0892] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0893] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0894] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0895] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0896] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0897] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0898] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0899] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0900] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0901] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0902] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0903] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[0904] The following is further disclosed regarding the embodiments described above.

[0905] (Claim 1)

[0906] In order to suggest the most suitable products based on the user's preferences, body type, budget, and lifestyle, a means of collecting purchase history, web browsing history, and social networking data to generate a user profile is provided.

[0907] Based on the aforementioned user profile, a means of selecting and proposing the most suitable product for the user using an artificial intelligence algorithm,

[0908] A means of using augmented reality technology to virtually try on or place selected products on the user's body or in their room,

[0909] A means of comparing market prices, obtaining the cheapest purchase option and available coupons, and presenting them to the user.

[0910] A means of interpreting user questions using natural language processing and providing relevant information,

[0911] A system that includes this.

[0912] (Claim 2)

[0913] The system according to claim 1, which identifies a user's preferences and lifestyle by analyzing social networking data and web browsing history.

[0914] (Claim 3)

[0915] The system according to claim 1, wherein an artificial intelligence algorithm updates product suggestions in real time using user profile data.

[0916] "Example 1"

[0917] (Claim 1)

[0918] In order to propose the most suitable products based on user characteristic information, a means for collecting transaction history, web visit information, and social data to generate user characteristic information,

[0919] Based on the aforementioned user characteristic information, a means of selecting and proposing the most suitable product for the user using an intelligent algorithm,

[0920] A means of virtually trying out or placing selected products in the user's appearance or space using virtual technology,

[0921] A means of comparing market prices, obtaining the most economical purchase options and available discounts, and presenting them to the user.

[0922] A means of interpreting user questions using language processing and providing relevant information,

[0923] A system that includes this.

[0924] (Claim 2)

[0925] The system according to claim 1, which identifies user characteristic information by analyzing social data and web visit information.

[0926] (Claim 3)

[0927] The system according to claim 1, wherein an intelligent algorithm instantly updates product recommendations using user characteristic information.

[0928] "Application Example 1"

[0929] (Claim 1)

[0930] In order to suggest the most suitable items based on the user's preferences, characteristics, budget, and lifestyle, a means of collecting purchase history, browsing history, and social networking data to generate user characteristics,

[0931] A means for selecting and proposing the most suitable items for the user using artificial intelligence processing based on the user's characteristics,

[0932] A means of virtually trying out or placing selected items in a user's characteristics or environment using augmented reality technology,

[0933] A means of comparing market values, obtaining the most economical purchasing options and available discount information, and presenting them to the user.

[0934] A means of interpreting user inquiries using natural language processing and providing relevant information,

[0935] A means of providing a more intuitive purchasing experience through virtual try-ons of items that users are interested in,

[0936] A system that includes this.

[0937] (Claim 2)

[0938] The system according to claim 1, which identifies a user's preferences and lifestyle by analyzing social networking data and web browsing information.

[0939] (Claim 3)

[0940] The system according to claim 1, wherein artificial intelligence processing instantly updates product suggestions using user characteristic data.

[0941] "Example 2 of combining an emotion engine"

[0942] (Claim 1)

[0943] In order to suggest the most suitable products based on the user's preferences, body type, budget, and lifestyle, a means of collecting purchase history, web browsing history, and social network data to generate a user profile is provided.

[0944] Based on the aforementioned user profile, a means of selecting and proposing the most suitable product for the user using a machine learning algorithm,

[0945] A means of analyzing a user's emotions in real time from their face, voice, and body movements, and adjusting product recommendations based on the analysis results,

[0946] A means of virtually trying on or placing selected products on a user's body or in their location using augmented reality technology,

[0947] A means of comparing market prices and presenting users with the most economical purchase options and available discount information,

[0948] A means of interpreting user questions using natural language processing and providing relevant information,

[0949] A system that includes this.

[0950] (Claim 2)

[0951] The system according to claim 1, which identifies a user's preferences and lifestyle by analyzing social network data and web browsing history.

[0952] (Claim 3)

[0953] The system according to claim 1, wherein the machine learning algorithm updates product suggestions in real time using the user's profile data and emotional state.

[0954] "Application example 2 when combining with an emotional engine"

[0955] (Claim 1)

[0956] In order to suggest the most suitable products based on the user's preferences, body type, budget, and lifestyle, a means of collecting purchase history, information browsing history, and information sharing data to generate user information is provided.

[0957] Based on the aforementioned user information, a means of selecting and proposing the most suitable product for the user using a knowledge-based system,

[0958] A means of virtually trying on or placing selected products on the user's body or in a space using augmented reality technology,

[0959] A means of comparing market prices, obtaining the most advantageous purchase options and available discount information, and presenting them to the user.

[0960] A means of interpreting user questions using natural language processing and providing relevant information,

[0961] A means of recognizing the user's facial expressions, analyzing their emotional state in real time, and dynamically changing product recommendations based on the analysis results,

[0962] A system that includes this.

[0963] (Claim 2)

[0964] The system according to claim 1, which identifies a user's preferences and lifestyle by analyzing information sharing data and information browsing history.

[0965] (Claim 3)

[0966] The system according to claim 1, wherein the knowledge-based system updates product suggestions in real time using user information data and adjusts the suggestion content using sentiment analysis. [Explanation of Symbols]

[0967] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. In order to suggest the most suitable items based on the user's preferences, characteristics, budget, and lifestyle, a means of collecting purchase history, browsing history, and social networking data to generate user characteristics, A means for selecting and proposing the most suitable items for the user using artificial intelligence processing based on the user's characteristics, A means of virtually trying out or placing selected items in a user's characteristics or environment using augmented reality technology, A means of comparing market values, obtaining the most economical purchasing options and available discount information, and presenting them to the user. A means of interpreting user inquiries using natural language processing and providing relevant information, A means of providing a more intuitive purchasing experience through virtual try-ons of items that users are interested in, A system that includes this.

2. The system according to claim 1, which identifies a user's preferences and lifestyle by analyzing social networking data and web browsing information.

3. The system according to claim 1, wherein artificial intelligence processing instantly updates product suggestions using user characteristic data.

Images