System

Abstract

A system is provided.SOLUTION: A system, comprising: means for receiving and analyzing text or images entered by a consumer; means for searching a database for relevant products based on the analysis; and means for displaying the search results to the consumer.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, consumer purchasing experiences in the digital realm have evolved rapidly, but the sheer variety of information makes it difficult for consumers to find the products that best suit them. Furthermore, typical recommendation systems relying solely on past history have difficulty responding to consumer needs and questions that change in real time. Furthermore, there is a lack of ways to quickly and accurately resolve the anxieties and questions consumers have when making purchasing decisions. Thus, there is a need to resolve these three issues - information overload, lack of personalization, and insufficient support for purchasing decisions - and further enrich the consumer purchasing experience. [Means for solving the problem]

[0005] To solve the above problems, the present invention provides the following means: A system including means for receiving and analyzing text or images input by a consumer, means for searching a database for related products based on the analysis results, and means for displaying the search results to the consumer. The system further includes means for recommending related products based on an event, and means for generating personalized product suggestions based on the consumer's past purchase history and browsing history. This system can personalize the consumer's purchasing experience and provide real-time support.

[0006] "Consumer" refers to the ultimate user who purchases and uses a particular product or service.

[0007] "Input" refers to the act of a consumer providing information, such as text or images, to a system.

[0008] "Analysis" refers to the process of using machine learning and algorithms to understand and extract meaning from input text or images.

[0009] "Related products" refers to products or services that are deemed most suitable based on the information entered by the consumer and the results of analysis.

[0010] "Database" refers to an information storage system that systematically stores and makes available information about related products.

[0011] "Display" refers to the act of making search results and related information visible in a consumer's visual interface.

[0012] "Event" refers to a special situation or occasion, such as a particular day or purpose, that influences gift selection and other behaviors.

[0013] "Recommendation" refers to the act of a system suggesting a particular product or service to a consumer.

[0014] "Purchase history" refers to a record of products and services previously purchased by a consumer.

[0015] "Browser history" refers to a record of products and services that a consumer has previously viewed.

[0016] "Personalization" refers to tailoring experiences and information to individual consumers based on their preferences and behavior.

[0017] A "means" refers to a method, process, or device for achieving a particular purpose.

[0018] A "system" refers to a collection of multiple elements that work together to achieve a specific function or purpose. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0020] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0023] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] The present invention relates to a system for personalizing a consumer's purchasing experience and providing highly relevant information and products. A specific embodiment of this system will be described below.

[0041] Intelligent Search Function

[0042] Users can enter text or upload an image into the search bar, and the system will analyze the input and search the database for relevant product information to display. This intelligent search function allows consumers to efficiently obtain the product information they are looking for.

[0043] Program processing

[0044] The user types "What's the latest version of the refrigerator in this picture?" into the app's search bar, or uploads an image of the refrigerator. The device sends the input data to the server, which uses natural language processing (NLP) algorithms to analyze the text and understand the user's intent. In the case of an image, the server uses image recognition algorithms to extract product features. Based on the analysis, the server searches a product database to identify relevant products. The search results are sent to the device, which displays them to the user.

[0045] Specific examples

[0046] The user uploads an image of their refrigerator, and the device sends the image to the server. The server uses image recognition technology to identify the refrigerator model and searches the database for the latest information. The search results are sent to the device, and the latest model information is displayed to the user.

[0047] Event-based gift recommendation engine

[0048] Users can enter gift-related questions via text, and the system will analyze the event and target information and recommend gift items based on that information, allowing consumers to find the perfect gift for a special event or occasion.

[0049] Program processing

[0050] The user types, "What gift would you recommend for my mother's birthday?" The device sends the question to the server. The server analyzes the question and extracts the event target information "birthday" and "mother." The server uses an event-based gift recommendation engine to search for relevant products and sends the recommendation results to the device. The device displays the recommended gift items to the user.

[0051] Specific examples

[0052] When a user types "What would you recommend for my mother's birthday?", the device sends the question to the server. The server analyzes the "birthday" and "mother" and recommends gift items based on that. The recommendation results are sent to the device, and suitable gift candidates are displayed to the user.

[0053] Personalized Shopping Experience

[0054] When a user logs in to the app, the system will suggest products tailored to their individual preferences based on their past purchase and browsing history. This personalization feature allows users to quickly search for product information based on their interests and preferences.

[0055] Program processing

[0056] A user logs in to the app. The device sends the user's past purchase and browsing history to the server. The server analyzes the user's historical data and identifies individual preferences and interests. The server generates personalized product suggestions based on the analysis results and sends a list of suggested products to the device. The device displays the list of suggested products to the user.

[0057] Specific examples

[0058] When a user logs in to the app, their device sends their past purchase history to the server. The server analyzes the data and suggests products that match the user's preferences. A list of suggested products is sent to the device and displayed to the user.

[0059] Purchasing support function

[0060] When users ask questions about specific products, the system provides quick answers using the FAQ section and AI chatbots, helping users to resolve any doubts or concerns they may have about their purchase.

[0061] Program processing

[0062] The user types a question, such as "How long is the warranty on this refrigerator?" The device sends the question to the server, which uses an FAQ section or an AI chatbot to generate an appropriate answer to the question. The generated answer is sent to the device, which then displays the appropriate answer to the user.

[0063] Specific examples

[0064] When a user asks about the warranty period of a refrigerator, the device sends the question to the server, which parses the question and retrieves the warranty period information from the FAQ section. The retrieved information is sent to the device and displayed to the user.

[0065] Community Features

[0066] Users can post reviews and opinions about products, and other users can view that information. This feature allows consumers to make purchasing decisions while also taking into account the opinions of other users.

[0067] Program processing

[0068] The user enters a review or opinion on a product. The device sends the review to the server. The server stores the received review in a database. When another user visits a specific product page, the server searches the database for reviews of that product and sends the search results to the device. The device displays the review to the user.

[0069] Specific examples

[0070] A user posts a review about a refrigerator, and the device sends it to the server. The server stores the review in a database. When another user visits the refrigerator's product page, the server retrieves the stored review and sends it to the device. The device displays the review to the user.

[0071] In this way, the system according to the claims can enhance the consumer's buying experience and provide personalized services.

[0072] The processing flow will be explained below.

[0073] Intelligent Search Function

[0074] Program processing

[0075] Step 1:

[0076] A user types "What's the latest version of the refrigerator in this picture?" into the app's search bar or uploads an image of the refrigerator.

[0077] Step 2:

[0078] The device sends the entered text and images to the server.

[0079] Step 3:

[0080] The server analyzes the submitted text using natural language processing (NLP) algorithms to understand the user's intent.

[0081] Step 4:

[0082] The server analyzes the transmitted image using an image recognition algorithm and extracts its features.

[0083] Step 5:

[0084] The server searches a product database based on the analysis results to identify highly relevant products.

[0085] Step 6:

[0086] The server transmits the search results to the terminal.

[0087] Step 7:

[0088] The terminal displays the search results to the user.

[0089] Event-based gift recommendation engine

[0090] Program processing

[0091] Step 1:

[0092] The user texts in, "What do you recommend for my mom's birthday?"

[0093] Step 2:

[0094] The terminal sends the text to the server.

[0095] Step 3:

[0096] The server analyzes the submitted text using natural language processing (NLP) to identify the event "birthday" and the target "mother."

[0097] Step 4:

[0098] The server uses an event-based gift recommendation engine to search for relevant gift items.

[0099] Step 5:

[0100] The server transmits the recommendation results to the terminal.

[0101] Step 6:

[0102] The terminal displays a list of recommended products to the user.

[0103] Personalized Shopping Experience

[0104] Program processing

[0105] Step 1:

[0106] The user logs in to the app.

[0107] Step 2:

[0108] The terminal sends the logged-in user's past purchase history and browsing history to the server and loads that data.

[0109] Step 3:

[0110] The server analyzes the received historical data to identify the user's preferences and interests.

[0111] Step 4:

[0112] The server generates personalized product suggestions based on the analysis results.

[0113] Step 5:

[0114] The server transmits the generated list of suggested products to the terminal.

[0115] Step 6:

[0116] The terminal displays a list of suggested products to the user.

[0117] Purchasing support function

[0118] Program processing

[0119] Step 1:

[0120] A user types a question: "How long is the warranty on this refrigerator?"

[0121] Step 2:

[0122] The terminal transmits the entered question to the server.

[0123] Step 3:

[0124] The server analyzes the submitted questions and uses an FAQ section or AI chatbot to generate appropriate answers.

[0125] Step 4:

[0126] The server sends the generated answer to the terminal.

[0127] Step 5:

[0128] The terminal displays the generated answer to the user.

[0129] Community Features

[0130] Program processing

[0131] Step 1:

[0132] The user enters reviews and opinions about the purchased product.

[0133] Step 2:

[0134] The device sends the entered reviews and opinions to the server.

[0135] Step 3:

[0136] The server stores the received reviews in a database.

[0137] Step 4:

[0138] When another user visits a particular product page, the server searches the database for reviews of that product.

[0139] Step 5:

[0140] The server sends the review of the search results to the terminal.

[0141] Step 6:

[0142] The terminal displays the searched reviews to the user.

[0143] Example 1

[0144] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0145] Conventional online shopping systems make it difficult for consumers to efficiently find the best products for them from the vast amount of product information available, resulting in a cumbersome shopping experience. Furthermore, product recommendations based on special events or individual preferences are not adequately implemented, which can lead to low consumer satisfaction. Furthermore, the inability to easily obtain detailed product information or reviews can make it difficult to resolve concerns or questions about purchasing.

[0146] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0147] In this invention, the server includes a means for analyzing text or images entered by a consumer, a means for analyzing the text using a natural language processing algorithm to understand the user's intent, a means for analyzing the image using an image recognition algorithm to extract product features, a means for searching a database for related products based on the analysis results, and a means for displaying the search results to the consumer. This allows consumers to efficiently find the most suitable product. In addition, personalized recommendations based on events and past history improve consumer satisfaction and provide a comfortable shopping experience.

[0148] "Consumer" refers to a general user of the system for the purpose of purchasing goods or services.

[0149] "Text" refers to sentences or characters entered by consumers, and is textual information that is understood by the system.

[0150] "Images" refers to visual information such as photographs and illustrations uploaded by consumers.

[0151] "Means of analysis" refers to methods and techniques for processing input text or images with a computer program and understanding their content.

[0152] A "natural language processing algorithm" refers to a computational method for analyzing text and understanding the meaning and intent of human language.

[0153] "Means for understanding user intent" refers to methods that use natural language processing algorithms to analyze the meaning and purpose of text entered by a user.

[0154] An "image recognition algorithm" refers to a computational method for analyzing an image and identifying the objects and features contained within it.

[0155] "Means for extracting product features" refers to the use of image recognition algorithms to identify specific products and their attributes from uploaded images.

[0156] "Related Products" refers to products and services available for purchase that are suggested based on information entered or uploaded by the consumer.

[0157] A "database" refers to a collection of information that systematically stores related products and other information and makes it easy to search and retrieve.

[0158] "Searching means" refers to the methods and technologies used to search the database based on the analysis results and find relevant product information.

[0159] "Means for displaying search results to consumers" refers to methods and technologies for displaying searched product information on a user interface so that consumers can check it.

[0160] "Event" refers to a specific situation or occasion, such as a birthday or anniversary, and is the criterion for recommending products related to that situation.

[0161] "Personalized product suggestions" refers to a method of recommending products that match a consumer's individual preferences based on individual data such as their past purchase history and browsing history.

[0162] The present invention relates to a system for personalizing a consumer's shopping experience and providing highly relevant information and products. The system has multiple functions for analyzing user-entered text and images and efficiently providing relevant product information.

[0163] Intelligent Search Function

[0164] Users can enter text or upload an image into the search bar, and the system will analyze the input and search the database for relevant product information to display. This intelligent search function allows consumers to efficiently obtain the product information they are looking for.

[0165] Hardware and software used

[0166] Server: A computer with high processing power (e.g., a Linux server)

[0167] Terminal: The device that provides the user interface (e.g., smartphone, tablet, PC)

[0168] Natural Language Processing Algorithms: Google Cloud Natural Language API

[0169] Image recognition algorithm: Amazon Rekognition

[0170] Database: MySQL

[0171] Specific actions

[0172] The user types the text "What is the latest version of the refrigerator in this image?" into the app's search bar or uploads an image of the refrigerator.

[0173] The terminal sends the input data to the server.

[0174] The server uses the Google Cloud Natural Language API to analyze the text and understand the user's intent.

[0175] In the case of images, the server uses Amazon Rekognition to perform image recognition and extract the product's features.

[0176] The server searches a MySQL database based on the analysis results to identify relevant products.

[0177] The search results are sent to the terminal, which displays the results to the user.

[0178] Event-based gift recommendation engine

[0179] Users can enter gift-related questions via text, and the system will analyze the event and target information and recommend gift items based on that information, allowing consumers to find the perfect gift for a special event or occasion.

[0180] Specific actions

[0181] User types, "What do you recommend for my mom's birthday?"

[0182] The terminal sends a question to the server.

[0183] The server parses the question using the Google Cloud Natural Language API and extracts the event target information "birthday" and "mother."

[0184] The server uses an event-based gift recommendation engine to search for relevant products, and the recommendation results are sent to the terminal.

[0185] The terminal displays the recommended gift items to the user.

[0186] Personalized Shopping Experience

[0187] When a user logs in to the app, the system will suggest products tailored to their individual preferences based on their past purchase and browsing history. This personalization feature allows users to quickly search for product information based on their interests and preferences.

[0188] Specific actions

[0189] A user logs in to the app.

[0190] The device sends the user's past purchase history and browsing history to the server.

[0191] The server analyzes the received historical data to identify the user's preferences and interests.

[0192] The server generates personalized product suggestions based on the analysis results, and a list of suggested products is sent to the terminal.

[0193] The terminal displays a list of suggested products to the user.

[0194] Purchasing support function

[0195] When users ask questions about specific products, the system provides quick answers using the FAQ section and AI chatbots, helping users to resolve any doubts or concerns they may have about their purchase.

[0196] Specific actions

[0197] A user types a question: "How long is the warranty on this refrigerator?"

[0198] The terminal sends a question to the server.

[0199] The server uses Dialogflow to analyze the question and generate an appropriate answer from the FAQ database.

[0200] The generated answer is sent to the terminal, which displays the appropriate answer to the user.

[0201] Community Features

[0202] Users can post reviews and opinions about products, and other users can view that information. This feature allows consumers to make purchasing decisions while also taking into account the opinions of other users.

[0203] Specific actions

[0204] Users enter product reviews and opinions.

[0205] The device sends the review to the server.

[0206] The server stores the received reviews in a MySQL database.

[0207] When another user visits a specific product page, the server searches the database for reviews of that product and sends the search results to the terminal.

[0208] The device displays the review to the user.

[0209] With the above functions, the system of the present invention can improve the consumer's purchasing experience and provide personalized services.

[0210] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0211] Intelligent Search Function

[0212] Step 1:

[0213] A user types the text "What is the latest version of the refrigerator in this image?" into the search bar or uploads an image of a refrigerator.

[0214] Input: Text or image from the user

[0215] Output: Text or image sent to the terminal

[0216] Step 2:

[0217] The terminal sends the input data to the server.

[0218] Input: Text or Image

[0219] Output: HTTP request sent to the server

[0220] Step 3:

[0221] The server branches the process depending on the input data. For text, the server analyzes the text using the Google Cloud Natural Language API to understand the user's intent. For images, the server extracts features using Amazon Rekognition.

[0222] Input: Text or image sent to the server

[0223] Output: Analysis results (user intent for text, feature data for images)

[0224] Step 4:

[0225] Based on the analysis results, the server searches the MySQL database to identify relevant products.

[0226] Specifically, an SQL query is generated based on the extracted keywords and features and issued to the database.

[0227] Input: Analysis results (user intent or feature data)

[0228] Output: Related product information

[0229] Step 5:

[0230] The server formats the search results in JSON format and sends them to the terminal as an HTTP response.

[0231] Input: Relevant product information

[0232] Output: Search results in JSON format

[0233] Step 6:

[0234] The terminal parses the received search results and displays them on the user interface.

[0235] Input: JSON format search results

[0236] Output: Product information displayed in the user interface

[0237] Event-based gift recommendation engine

[0238] Step 1:

[0239] User types, "What do you recommend for my mom's birthday?"

[0240] Input: User question text

[0241] Output: Question data sent to the terminal

[0242] Step 2:

[0243] The terminal transmits the question data to the server.

[0244] Input: Question data

[0245] Output: HTTP request sent to the server

[0246] Step 3:

[0247] The server parses the question using the Google Cloud Natural Language API and extracts the event target information "birthday" and "mother."

[0248] Input: Query data sent to the server

[0249] Output: Extracted event and target information

[0250] Step 4:

[0251] The server searches for related products using a gift recommendation engine based on the event information and target information, and obtains recommendation results.

[0252] Input: Event information, target information

[0253] Output: Gift recommendation results

[0254] Step 5:

[0255] The server formats the recommendation results in JSON format and sends them to the terminal as an HTTP response.

[0256] Input: Gift recommendation results

[0257] Output: Recommendation results in JSON format

[0258] Step 6:

[0259] The device parses the received recommendation results and displays them on the user interface.

[0260] Input: Recommendation results in JSON format

[0261] Output: Gift information displayed in the user interface

[0262] Personalized Shopping Experience

[0263] Step 1:

[0264] A user logs in to the app.

[0265] Input: Username and Password

[0266] Output: Login request

[0267] Step 2:

[0268] The terminal transmits the user's past history data to the server.

[0269] Input: Historical data

[0270] Output: HTTP request sent to the server

[0271] Step 3:

[0272] The server analyzes the historical data to identify the user's preferences and interests.

[0273] Input: Historical data sent to the server

[0274] Output: Analysis results (user preferences and interests)

[0275] Step 4:

[0276] The server generates personalized product suggestions based on the analysis results using a Recommendation Engine.

[0277] Input: Analysis results (user preferences and interests)

[0278] Output: Personalized product recommendations

[0279] Step 5:

[0280] The server formats the list of suggested products in JSON format and sends it to the terminal as an HTTP response.

[0281] Input: Product suggestion list

[0282] Output: A list of suggestions in JSON format

[0283] Step 6:

[0284] The terminal parses the received proposal list and displays it on the user interface.

[0285] Input: A list of suggestions in JSON format

[0286] Output: A list of suggested products displayed in the user interface

[0287] Purchasing support function

[0288] Step 1:

[0289] A user enters a question about a particular product.

[0290] Input: Question text

[0291] Output: Question data sent to the terminal

[0292] Step 2:

[0293] The terminal sends a question to the server.

[0294] Input: Question data

[0295] Output: HTTP request sent to the server

[0296] Step 3:

[0297] The server uses Dialogflow to analyze the question and generate an appropriate answer.

[0298] Input: Query data sent to the server

[0299] Output: Correct answer data

[0300] Step 4:

[0301] The server searches the FAQ database and retrieves the corresponding information.

[0302] Input: Question data

[0303] Output: FAQ information

[0304] Step 5:

[0305] The server formats the acquired FAQ information into JSON format and sends it to the terminal as an HTTP response.

[0306] Input: FAQ information

[0307] Output: JSON formatted response data

[0308] Step 6:

[0309] The terminal parses the received response data and displays it on the user interface.

[0310] Input: JSON formatted response data

[0311] Output: Answer information displayed in the user interface

[0312] Community Features

[0313] Step 1:

[0314] Users enter product reviews and opinions.

[0315] Input: Review or opinion text

[0316] Output: Review sent to device

[0317] Step 2:

[0318] The device sends the review to the server.

[0319] Input: Review text

[0320] Output: HTTP request sent to the server

[0321] Step 3:

[0322] The server stores the received reviews in a MySQL database.

[0323] Input: The review sent to the server

[0324] Output: Reviews stored in a database

[0325] Step 4:

[0326] Another user visits a specific product page.

[0327] Input: Product page request

[0328] Output: HTTP request to the server

[0329] Step 5:

[0330] The server searches the database for product reviews.

[0331] Input: Product page request

[0332] Output: Review information as search results

[0333] Step 6:

[0334] The server formats the search results in JSON format and sends them to the terminal as an HTTP response.

[0335] Input: Review information as search results

[0336] Output: Review information in JSON format

[0337] Step 7:

[0338] The terminal parses the received review information and displays it on the user interface.

[0339] Input: JSON formatted review information

[0340] Output: Review information displayed in the user interface

[0341] (Application example 1)

[0342] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0343] Traditional purchasing experiences often only offer consumers a uniform selection of products, making it difficult to provide personalized suggestions based on individual consumer preferences and past history. Product searches using text and images also have low accuracy, preventing consumers from efficiently finding the products they are looking for. Furthermore, responses to consumer questions about specific products are often delayed, hindering purchasing decisions. Therefore, there is a need for a system that can more individually tailor the consumer purchasing experience and provide relevant information quickly and appropriately.

[0344] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0345] In this invention, the server includes means for receiving and analyzing text or images entered by a consumer, means for searching a database for related products based on the analysis results, means for displaying the search results to the consumer, means for analyzing purchase history and browsing history based on a consumer ID and generating personalized product suggestions, means including an integrated natural language processing algorithm and an image recognition algorithm, and means for generating appropriate answers to end-user questions using artificial intelligence, thereby enabling consumers to efficiently receive product suggestions that match their preferences and the information they are looking for.

[0346] A "consumer ID" is an identifier used to uniquely identify a consumer.

[0347] "Purchase history" is a record of products and services that a consumer has purchased in the past.

[0348] "Browser history" is a record of the products and pages a consumer has previously viewed on a website or app.

[0349] "Personalized product recommendations" are recommendations for products or services that are customized for a specific consumer based on that consumer's past behavior and preferences.

[0350] "Text analysis" is the process of analyzing input text data and understanding its meaning and content.

[0351] "Image analysis" is the process of analyzing input image data and recognizing objects and features within it.

[0352] A database is a collection of information that organizes and stores related data so that it can be quickly searched.

[0353] A "natural language processing algorithm" is a computational method for understanding and analyzing text data.

[0354] An "image recognition algorithm" is a computational method for analyzing image data and identifying specific objects or features.

[0355] "Artificial intelligence" is a technology that gives computers the ability to learn and reason like human intelligence.

[0356] The "means for generating an appropriate answer to a question" is a process for analyzing the question entered by the consumer and generating an answer based on related information.

[0357] "Related product search" is the process of locating related products from a database based on consumer input.

[0358] "Product proposal generation" is the process of making product proposals to individual consumers based on their past behavioral data.

[0359] A "display means" is a method or device that visually presents information, such as search results or suggested products, to consumers.

[0360] This invention is a system that allows consumers to input text or images through an interface, and then analyzes the input to provide related products. In particular, it has a personalized product suggestion function based on the consumer's purchase history and browsing history, and a question-answering function using artificial intelligence.

[0361] Program Overview

[0362] This system mainly consists of the following hardware and software:

[0363] 1. Hardware:

[0364] Server: AWS EC2

[0365] Database: Amazon RDS (MySQL)

[0366] User devices: smartphones and computers

[0367] 2. Software:

[0368] Natural Language Processing (NLP) Algorithm: Google's BERT Model

[0369] Image recognition algorithm: Google Cloud Vision API

[0370] Server-side program: Python (Flask)

[0371] System Operation Overview

[0372] Intelligent Search Function

[0373] A user enters text into the device's search bar or uploads an image. The device sends the input data to the server, which analyzes the text input with a natural language processing algorithm or the image input with an image recognition algorithm. The server then searches the database to identify relevant products. The search results are sent to the device and displayed to the user.

[0374] Personalized Shopping Experience

[0375] When a user logs in to the app, the device sends the user's ID to the server, which analyzes their purchase and browsing history to generate a personalized product recommendation list, which is then sent to the device and displayed to the user.

[0376] Purchasing support function

[0377] A user enters a question about a particular product. The device sends the question to the server, which uses natural language processing algorithms and an FAQ database to generate an appropriate answer. The answer is then sent to the device and displayed to the user.

[0378] Specific examples

[0379] Intelligent Search: The user uploads a picture of their refrigerator and sends it to the server. The server uses image recognition technology to identify the refrigerator model and retrieves the latest information from the database. The search results are sent to the device, and the latest model information is displayed to the user.

[0380] Personalized shopping experience: When a user logs into the app and a previously purchased smartwatch is recorded, the server suggests new smartwatch models and displays the list to the user.

[0381] Purchasing support function: When a user asks, "How long is the warranty period for this refrigerator?", the server refers to the FAQ database and provides the user with information about the warranty period.

[0382] Prompt Sentence Examples

[0383] If a user types "What is the latest iPhone model?", the database will be searched for and displayed as relevant latest iPhone models.

[0384] Input: What is the latest model of iPhone?

[0385] Output: The iPhone 13 is currently the latest model and comes with the latest features.

[0386] If you log in with user ID "12345", we will recommend new smartwatch models based on your past purchase history.

[0387] Input: I would like products recommended based on the purchasing history of user ID "12345".

[0388] Output: We recommend the new Apple Watch Series 6.

[0389] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0390] Intelligent Search Processing Steps

[0391] Step 1: User Input

[0392] The user enters text or uploads an image into the device's input interface.

[0393] Input: Text or image

[0394] Output: Text or image data

[0395] Step 2: Send data

[0396] The device sends input data (text or image) to the server using HTTP POST as the transmission protocol.

[0397] Input: Text or image data entered by the user

[0398] Output: Text or image data sent to the server

[0399] Step 3: Data analysis

[0400] The server analyzes the received data. For text data, it uses a natural language processing algorithm (Google's BERT) to analyze the intent, and for image data, it uses an image recognition algorithm (Google Cloud Vision API) to extract features.

[0401] Input: Text or image data sent to the server

[0402] Output: Text analysis results or image feature data

[0403] Step 4: Database Search

[0404] The server searches the product database based on the analysis results or extracted features and executes SQL queries to retrieve relevant product information.

[0405] Input: Text analysis results or image feature data

[0406] Output: Product information

[0407] Step 5: Submit search results

[0408] The server sends the search results to the device in JSON format.

[0409] Input: JSON data of search results

[0410] Output: Product information sent to the device

[0411] Step 6: View the results

[0412] The product information received by the terminal is displayed on a user interface.

[0413] Input: Product information sent to the terminal

[0414] Output: The product list displayed to the user

[0415] Processing steps for a personalized shopping experience

[0416] Step 1: User Login

[0417] The user enters their credentials to log in to the app.

[0418] Input: User ID and password

[0419] Output: Authentication token

[0420] Step 2: Send past history

[0421] The device sends the user ID to the server, which retrieves relevant purchase and browsing history from its internal database.

[0422] Input: User ID

[0423] Output: Purchase history and browsing history data

[0424] Step 3: Historical data analysis

[0425] The server analyzes the historical data and generates a personalized product list.

[0426] Input: Purchase history and browsing history data

[0427] Output: A personalized product list

[0428] Step 4: Submit your proposal list

[0429] The server transmits the generated product proposal list to the terminal.

[0430] Input: Personalized product list

[0431] Output: Suggestion list sent to the device

[0432] Step 5: List View

[0433] The terminal displays the product suggestion list on the user interface.

[0434] Input: Suggestion list sent to the device

[0435] Output: A list of suggested products displayed to the user

[0436] Purchasing Support Function Processing Steps

[0437] Step 1: Enter your question

[0438] The user inputs a question about the product they want to purchase.

[0439] Input: Question text

[0440] Output: The question text entered

[0441] Step 2: Submit your question

[0442] The terminal sends the question text to the server.

[0443] Input: The entered question text

[0444] Output: The question text sent to the server

[0445] Step 3: Question analysis

[0446] The server analyzes the question text using natural language processing algorithms.

[0447] Input: Question text sent to the server

[0448] Output: Question analysis results

[0449] Step 4: Generate appropriate answers

[0450] The server searches the FAQ database for relevant answers and generates an appropriate answer.

[0451] Input: Question analysis results

[0452] Output: Correct answer

[0453] Step 5: Submit your response

[0454] The server sends the generated response to the terminal.

[0455] Input: Correct Answer

[0456] Output: Answer sent to the terminal

[0457] Step 6: View Answers

[0458] The terminal displays the generated answer on a user interface.

[0459] Input: Answer sent to the terminal

[0460] Output: The answer that is displayed to the user

[0461] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0462] This invention relates to a system that personalizes the consumer's purchasing experience and provides highly relevant information and products. In particular, by combining it with an emotion engine that recognizes the user's emotions, a higher level of personalization is achieved. Specific embodiments of this system are as follows:

[0463] Intelligent search function with emotion engine

[0464] Users can enter text into the search bar or upload an image, and the system analyzes the input, searches the database for relevant product information, and displays it. Utilizing an emotion engine, the system provides search results that take the user's emotions into account.

[0465] Program processing

[0466] The user types "What's the latest version of the refrigerator in this picture?" into the app's search bar, or uploads an image of the refrigerator. The device sends the input data, along with the user's facial expressions and tone of voice captured by the camera and microphone, to the server. The server analyzes the text with natural language processing (NLP) algorithms and the image with image recognition algorithms, while analyzing the user's emotions with an emotion engine. Based on the analysis results, it searches a product database to identify related products. The related products are sent to the device, which displays the results to the user.

[0467] Specific examples

[0468] The user uploads an image of their refrigerator, which the device sends to the server. The server uses image recognition technology to identify the refrigerator model, and if it detects a positive emotion in the user's facial expression, it searches the database for information on the latest, more expensive, and highly rated models. The search results are sent to the device, and the information on the latest model is displayed to the user.

[0469] Event-based gift recommendation engine

[0470] When a user enters a gift-related question in text, the system analyzes the event and target information and recommends gift items based on that information. By combining this function with an emotion engine, it becomes possible to make recommendations based on the consumer's emotions.

[0471] Program processing

[0472] The user types "What gift would you recommend for my mother's birthday?" and emotional information is also collected via the camera and microphone. The device sends the text and emotional information to the server. The server analyzes the question and identifies the event / target information of "birthday" and "mother," while simultaneously analyzing the user's emotions using an emotional engine. The server uses an event-based gift recommendation engine to search for related products, and the recommendation results are sent to the device. The device then displays the recommended gift items to the user.

[0473] Specific examples

[0474] If a user looks hesitant while typing "What would you recommend for my mother's birthday?", the server will recognize this hesitation using its emotion engine and prioritize recommending reliable and popular gift items. The recommended items are then sent to the device and displayed to the user.

[0475] Personalized Shopping Experience

[0476] When a user logs in to the app, the system will suggest products tailored to their individual preferences based on their past purchase and browsing history. Adding an emotion engine to this personalization function will enable even more advanced personalization.

[0477] Program processing

[0478] When a user logs in to the app, the device sends emotional information such as facial expression and tone of voice at the time of login, along with past purchase history and browsing history, to the server. The server analyzes the received history and emotional data to identify the user's preferences and interests. Based on the analysis results, the server generates personalized product suggestions that also take emotional information into account, and a list of suggested products is sent to the device. The device then displays the list of suggested products to the user.

[0479] Specific examples

[0480] When a user logs in to the app, the emotion engine recognizes their happy facial expression. The server takes this happy emotion into account along with their past purchase history to suggest new products that the user is likely to enjoy. The list of suggestions is sent to the device and displayed to the user.

[0481] Purchasing support function

[0482] When a user asks a question about a specific product, the system provides a quick answer using the FAQ section or an AI chatbot. The emotion engine also analyzes emotions, allowing the system to provide a more accurate answer based on the user's emotions.

[0483] Program processing

[0484] The user asks, "How long is the warranty period for this refrigerator?" and emotional information is also collected. The device sends the question and emotional information to the server. The server analyzes the question, recognizes the user's emotions using an emotional engine, and uses an FAQ section or AI chatbot to generate an appropriate answer. The generated answer is sent to the device, which displays it to the user.

[0485] Specific examples

[0486] If a user asks about the warranty period for a refrigerator and shows a worried expression, the server will recognize the user's anxiety using its emotion engine, provide a detailed explanation of the warranty, and provide information on related reliable products. The answer will be sent to the terminal and displayed to the user.

[0487] Community Features

[0488] Users can post reviews and opinions about products, which can then be viewed by other users. By combining this feature with an emotion engine, it is possible to understand the emotional tone of the reviews.

[0489] Program processing

[0490] Users input reviews and opinions, and emotional information is also collected. The device sends the reviews and emotional information to the server. The server stores the reviews and emotional information in a database. When another user visits a specific product page, the server searches the database for reviews of that product and sends the review results, which also reflect the emotional tone, to the device. The device displays the reviews to the user.

[0491] Specific examples

[0492] When a user posts a review about a refrigerator, the emotion engine analyzes the happy facial expression. The server stores the review and positive emotion in a database. When another user visits the refrigerator's product page, the review is displayed on their device as a positive emotion review.

[0493] In this way, the invention that combines the emotion engine can further personalize the consumer's purchasing experience and provide detailed emotional support.

[0494] The processing flow will be explained below.

[0495] Intelligent search function with emotion engine

[0496] Program processing

[0497] Step 1:

[0498] A user types "What's the latest version of the refrigerator in this picture?" into the app's search bar or uploads an image of the refrigerator.

[0499] Step 2:

[0500] The device sends the entered text and images, as well as the user's facial expressions and tone of voice captured through the camera and microphone, to the server.

[0501] Step 3:

[0502] The server analyzes the submitted text using natural language processing (NLP) algorithms to understand the user's intent.

[0503] Step 4:

[0504] The server analyzes the transmitted image using an image recognition algorithm and extracts its features.

[0505] Step 5:

[0506] The server uses an emotion engine to analyze the user's facial expressions and tone of voice to identify the user's emotions.

[0507] Step 6:

[0508] The server searches a product database based on the analysis results to identify highly relevant products.

[0509] Step 7:

[0510] The server transmits the search results together with supplementary information according to the user's feelings to the terminal.

[0511] Step 8:

[0512] The terminal displays the search results to the user.

[0513] Specific examples

[0514] The user uploads an image of the refrigerator, and the image, along with facial expression and tone of voice data, is sent from the device to the server.

[0515] The server uses image recognition technology to identify the refrigerator model, and if it detects a positive emotion in the user's facial expression, it searches the database for information on the latest, more luxurious and highly rated models.

[0516] The search results are sent to the device, and the user is shown information about the latest models and recommendations based on positive sentiment.

[0517] Event-based gift recommendation engine

[0518] Program processing

[0519] Step 1:

[0520] The user texts in, "What do you recommend for my mom's birthday?"

[0521] Step 2:

[0522] The device sends text and emotional information obtained through the camera and microphone to a server.

[0523] Step 3:

[0524] The server analyzes the submitted text using natural language processing (NLP) to identify the event "birthday" and the target "mother."

[0525] Step 4:

[0526] The server analyzes the user's emotions using an emotion engine and identifies the emotions.

[0527] Step 5:

[0528] The server uses an event-based gift recommendation engine to search for relevant gift items.

[0529] Step 6:

[0530] The server sends the search results and supplemental information based on the user's emotions to the terminal.

[0531] Step 7:

[0532] The terminal displays a list of recommended products to the user.

[0533] Specific examples

[0534] If a user types "What would you recommend for my mother's birthday?" and looks confused, the device sends the text and emotional information to the server.

[0535] The server identifies the "birthday" and "mother," and uses an emotion engine to recognize the customer's uncertainty and prioritize searching for reliable and popular gift items.

[0536] The recommended products are sent to the terminal, and a list of reliable products and the reasons for their selection are displayed to the user.

[0537] Personalized Shopping Experience

[0538] Program processing

[0539] Step 1:

[0540] The user logs in to the app.

[0541] Step 2:

[0542] The device sends past purchase history and browsing history to the server along with the user's facial expression and tone of voice when logging in.

[0543] Step 3:

[0544] The server analyzes the received history and emotion data to identify the user's preferences and interests.

[0545] Step 4:

[0546] A server generates personalized product suggestions based on preferences and interests.

[0547] Step 5:

[0548] The server transmits a list of suggested products that also takes emotional information into consideration to the terminal.

[0549] Step 6:

[0550] The terminal displays a list of suggested products to the user.

[0551] Specific examples

[0552] When a user logs in to the app, their happy facial expression is recognized by the emotion engine. The device then sends past purchase history and emotional information to the server.

[0553] The server analyzes the data and generates product suggestions that take into account the user's preferences and pleasant emotions. The suggestion list is sent to the terminal and displayed to the user.

[0554] Purchasing support function

[0555] Program processing

[0556] Step 1:

[0557] A user types a question: "How long is the warranty on this refrigerator?"

[0558] Step 2:

[0559] The device transmits the text and emotion information to the server.

[0560] Step 3:

[0561] The server analyzes the submitted questions and uses an FAQ section or AI chatbot to generate appropriate answers.

[0562] Step 4:

[0563] The server uses an emotion engine to analyze the user's emotions and generate a response based on those emotions.

[0564] Step 5:

[0565] The server sends the generated answer to the terminal.

[0566] Step 6:

[0567] The terminal displays the answer to the user.

[0568] Specific examples

[0569] If a user asks about the warranty period for a refrigerator and looks worried, the server will recognize the anxiety using its emotion engine, provide a detailed explanation of the warranty, and also provide information on related, more reliable products.

[0570] The answer is sent to the terminal and displayed to the user.

[0571] Community Features

[0572] Program processing

[0573] Step 1:

[0574] The user enters reviews and opinions about the purchased product.

[0575] Step 2:

[0576] The device transmits the input reviews, opinions, and sentiment information to the server.

[0577] Step 3:

[0578] The server stores the received reviews and sentiment information in a database.

[0579] Step 4:

[0580] When another user visits a particular product page, the server searches the database for reviews of that product.

[0581] Step 5:

[0582] The server sends the review of the search results and information reflecting the emotional tone to the device.

[0583] Step 6:

[0584] The terminal displays the searched reviews to the user.

[0585] Specific examples

[0586] When a user posts a review about a refrigerator, the emotion engine also analyzes the happy facial expression they express.

[0587] The server stores the review and positive sentiment in a database, and when another user visits the refrigerator's product page, it is displayed on their device as a positive sentiment review.

[0588] In this way, the invention that combines the emotion engine can further personalize the consumer's purchasing experience and provide detailed emotional support.

[0589] Example 2

[0590] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0591] Conventional shopping systems typically search for and recommend products based on consumer input, but lack advanced personalization that takes into account the consumer's emotional information. This makes it difficult to achieve the high relevance and satisfaction desired by consumers. Furthermore, even when recommending products based on events or the consumer's past history, further accuracy could be improved by utilizing emotional information, but such functionality is currently lacking.

[0592] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for receiving and analyzing text or images input by a consumer, means for collecting and analyzing emotional information, and means for searching a database for related products based on the analysis results. This enables highly personalized and highly relevant product searches that take into account the consumer's emotional information. Furthermore, by including means for recommending related products based on an event and means for generating recommendation results taking into account the consumer's emotional information, highly accurate event-based product recommendations can be realized. In addition, by including means for generating personalized product proposals based on the consumer's past history and means for generating recommendation results taking into account the consumer's emotional information, consumer satisfaction can be further increased.

[0593] "Consumer" refers to an individual who purchases or uses a product or service.

[0594] "Input text or image" refers to text data or image data provided by a consumer through an application or website interface.

[0595] "Means of analysis" refers to algorithms or programs that understand input data and extract meaning and features.

[0596] "Emotional information" refers to data about emotions extracted from consumers' facial expressions, tone of voice, gestures, etc.

[0597] "Collect" refers to obtaining data using sensor devices or input devices.

[0598] "Means for analyzing" refers to a system that includes algorithms or programs for interpreting collected emotional information and identifying specific emotional states.

[0599] A "database" refers to a data storage system in which multiple product information items are stored in an organized manner and can be easily searched and referenced.

[0600] A "search method" refers to an algorithm or program that efficiently searches through information in a database and finds entries that match a condition.

[0601] "Related products" refers to products that are relevant to a consumer's interests or needs based on data entered by the consumer and analysis results.

[0602] An "event" refers to a specific date, time, or occasion, such as a birthday, anniversary, or sales event.

[0603] "Past history" refers to a record of a consumer's previous purchases and browsing.

[0604] "Personalized product recommendations" refers to a list of products and services that are specifically suggested to a consumer based on their individual interests, history, and even emotional information.

[0605] "Recommendation results" refers to a list of products selected by the system and presented to the consumer.

[0606] This invention relates to a system that personalizes consumer purchasing experiences and provides highly relevant information and products. In particular, by combining it with an emotion engine that analyzes consumer emotion information, a more advanced level of personalization is achieved.

[0607] Intelligent search function with emotion engine

[0608] Users can enter text or upload an image into the application's search bar, and the system analyzes the input, searches the database for relevant product information, and displays it. By utilizing an emotion engine, the system also takes the user's emotions into account when providing search results.

[0609] The server analyzes the user's input data using natural language processing (NLP) algorithms and images using image recognition algorithms. It also analyzes the user's emotions using an emotion engine. Based on the analysis results, the server searches a database to identify relevant products. The search results are sent to the device, which displays them to the user.

[0610] Specific examples

[0611] The user uploads an image of their refrigerator, which the device sends to the server. The server uses image recognition technology to identify the refrigerator model, and if it recognizes a positive emotion in the user's facial expression, it searches the database for information on the latest, more expensive, and highly rated model. The search results are sent to the device, and the information on the latest model is displayed to the user.

[0612] Example prompt sentence:

[0613] "Browse photos of the latest refrigerator models. Users seem happy."

[0614] Event-based gift recommendation engine

[0615] When a user enters a gift-related question in text, the system analyzes the event and target information and recommends gift items based on that information. This function can be combined with an emotion engine to make recommendations based on the consumer's emotions.

[0616] The server analyzes the user's question and identifies event target information such as "birthday" and "mother." It then analyzes the user's emotions using an emotion engine. Based on the analysis results, it uses an event-based gift recommendation engine to search for relevant products. The recommendation results are sent to the device, which then displays the gift products to the user.

[0617] Specific examples

[0618] If a user types in "What gift would you recommend for my mother's birthday?" and shows a confused expression, the server will recognize the user's indecision using its emotion engine and prioritize recommending reliable and popular gift items. The recommended items are then sent to the device and displayed to the user.

[0619] Example prompt sentence:

[0620] "Users seem to be confused about finding the perfect gift for their mother's birthday."

[0621] Personalized Shopping Experience

[0622] When a user logs in to the application, the system will suggest products tailored to their individual preferences based on their past purchase and browsing history. Adding an emotion engine to this personalization function will enable even more advanced personalization.

[0623] The server analyzes the received history data and emotional data to identify the user's preferences and interests. Based on the analysis results, the server generates personalized product suggestions that take into account the emotional information and sends the list of suggested products to the terminal. The terminal then displays the list of suggested products to the user.

[0624] Specific examples

[0625] When a user logs in to the app, the emotion engine recognizes their happy facial expression. The server takes this happy emotion into account along with their past purchase history to suggest new products that the user is likely to enjoy. The list of suggestions is sent to the device and displayed to the user.

[0626] Example prompt sentence:

[0627] "Suggest new products to users who are in a good mood. Also consider past purchase history."

[0628] Purchasing support function

[0629] When a user asks a question about a specific product, the system provides a quick answer using the FAQ section or an AI chatbot. The emotion engine also analyzes emotions, allowing the system to provide a more accurate answer based on the user's emotions.

[0630] The server analyzes the question and recognizes the user's emotions using an emotion engine. Based on the analysis results, it generates an appropriate answer using an FAQ section or an AI chatbot and sends the answer to the device. The device then displays the answer to the user.

[0631] Specific examples

[0632] If a user asks about the warranty period for a refrigerator and shows a worried expression, the server will recognize the user's anxiety using its emotion engine, provide a detailed explanation of the warranty, and provide information on related reliable products. The answer will be sent to the terminal and displayed to the user.

[0633] Example prompt sentence:

[0634] "We will inform anxious users about the warranty period of the refrigerator and also suggest related trusted products."

[0635] Community Features

[0636] Users can post reviews and opinions about products, which can then be viewed by other users. By combining this feature with an emotion engine, it is possible to understand the emotional tone of the reviews.

[0637] The server stores the reviews and emotional information in a database, and when another user visits a specific product page, it provides review results that also reflect the emotional tone.The terminal displays the reviews to the user.

[0638] Specific examples

[0639] When a user posts a review about a refrigerator, the emotion engine analyzes the happy facial expression. The server stores the review and positive emotion in a database. When another user visits the refrigerator's product page, the review is displayed on their device as a positive emotion review.

[0640] Example prompt sentence:

[0641] "We store users' positive reviews in a database and make them available to other users."

[0642] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0643] Intelligent search function with emotion engine

[0644] Program processing

[0645] Step 1:

[0646] A user enters text into the app's search bar or uploads an image, and the input data is captured on the device.

[0647] Input: Text entered into the search bar or an uploaded image

[0648] Output: Input data is stored in the terminal

[0649] Step 2:

[0650] The device collects the user's facial expressions and tone of voice using a camera and microphone, and the collected emotional information is stored as data on the device.

[0651] Input: User's facial expression, tone of voice

[0652] Output: Emotional information data (facial expression recognition results, voice tone analysis results)

[0653] Step 3:

[0654] The device sends the input data and emotion information to the server, where they are packaged as data packets.

[0655] Input: Input data and emotion information data

[0656] Output: sent as data packets to the server

[0657] Step 4:

[0658] The server analyzes the input data: for text input, it uses natural language processing (NLP) algorithms; for images, it uses image recognition algorithms.

[0659] Input: User-entered data (text or image)

[0660] Output: Analysis results (e.g., text semantic analysis results, image recognition results)

[0661] Step 5:

[0662] The server analyzes the user's emotional information using an emotion engine, and the analysis results include specific emotions (e.g., positive, negative, etc.).

[0663] Input: Emotional information data

[0664] Output: Sentiment analysis result (e.g., positive, negative)

[0665] Step 6:

[0666] The server searches a product database based on the analysis results, and highly relevant product information is identified.

[0667] Input: Text / image analysis results, sentiment analysis results

[0668] Output: Related product information

[0669] Step 7:

[0670] The server sends the search results to the device in an appropriate format (e.g., JSON format).

[0671] Input: Related product information

[0672] Output: Related product information sent

[0673] Step 8:

[0674] The terminal displays the search results to the user using a graphical user interface (GUI).

[0675] Input: Related product information sent from the server

[0676] Output: Product information is displayed visually to the user

[0677] Event-based gift recommendation engine

[0678] Program processing

[0679] Step 1:

[0680] The user enters a question about the gift in text, and the input data is stored on the device.

[0681] Input: Text question (e.g., "What would you recommend for my mother's birthday?")

[0682] Output: Input data is stored in the terminal

[0683] Step 2:

[0684] The device collects the user's emotional information through the camera and microphone, and the collected data is stored on the device.

[0685] Input: User's facial expression, tone of voice

[0686] Output: Emotional information data (facial expression recognition results, voice tone analysis results)

[0687] Step 3:

[0688] The device sends the question and emotion information to the server. The data is sent in packets.

[0689] Input: Text questions, emotion information data

[0690] Output: sent as data packets to the server

[0691] Step 4:

[0692] The server parses the text question and extracts event target information such as "birthday" or "mother."

[0693] Input: Text question

[0694] Output: Analysis results (event and target information)

[0695] Step 5:

[0696] The server analyzes the user's emotional information using an emotion engine, and the analysis results include specific emotions.

[0697] Input: Emotional information data

[0698] Output: Emotion analysis results

[0699] Step 6:

[0700] Based on the analysis results, the server searches for related products using an event-based gift recommendation engine.

[0701] Input: Event and target information, emotion analysis results

[0702] Output: Relevant gift product information

[0703] Step 7:

[0704] The server sends the recommendation results to the device in an appropriate format (e.g., JSON format).

[0705] Input: Related gift product information

[0706] Output: Gift item information sent

[0707] Step 8:

[0708] The device displays the recommended gift items to the user through a GUI.

[0709] Input: Gift item information sent from the server

[0710] Output: Product information is displayed visually to the user

[0711] Personalized Shopping Experience

[0712] Program processing

[0713] Step 1:

[0714] A user logs in to the app, and their authentication data is stored on the device.

[0715] Input: User ID, Password

[0716] Output: Authentication result and user login history

[0717] Step 2:

[0718] The device uses the camera and microphone to collect emotional information such as facial expressions and tone of voice when logging in. The collected data is stored on the device.

[0719] Input: User's facial expression, tone of voice

[0720] Output: Emotional information data (facial expression recognition results, voice tone analysis results)

[0721] Step 3:

[0722] The device sends past purchase history, browsing history, and even emotional information to a server. The data is sent in packets.

[0723] Input: purchase history, browsing history, emotional information data

[0724] Output: sent as data packets to the server

[0725] Step 4:

[0726] The server analyzes the received historical and emotional data to identify the user's preferences and interests.

[0727] Input: history data, emotion data

[0728] Output: Preference analysis results

[0729] Step 5:

[0730] Based on the analysis results, the server generates personalized product suggestions that take emotional information into consideration.

[0731] Input: Preference analysis results, emotional information

[0732] Output: Personalized product recommendations

[0733] Step 6:

[0734] The list of suggested products is sent to the terminal, and the data is in the appropriate format.

[0735] Input: Personalized product suggestions

[0736] Output: Product suggestion information sent

[0737] Step 7:

[0738] The terminal displays a list of suggested products to the user through a GUI.

[0739] Input: Product suggestion information sent from the server

[0740] Output: A list of product suggestions is visually displayed to the user

[0741] Purchasing support function

[0742] Program processing

[0743] Step 1:

[0744] The user enters a question about a specific product, and the input data is saved on the device.

[0745] Input: Text question (e.g., "How long is the warranty on this refrigerator?")

[0746] Output: Input data is stored in the terminal

[0747] Step 2:

[0748] The device collects the user's emotional information through the camera and microphone, and the collected data is stored on the device.

[0749] Input: User's facial expression, tone of voice

[0750] Output: Emotional information data (facial expression recognition results, voice tone analysis results)

[0751] Step 3:

[0752] The device sends the question and emotion information to the server. The data is sent in packets.

[0753] Input: Text questions, emotion information data

[0754] Output: sent as data packets to the server

[0755] Step 4:

[0756] The server analyzes the question text and determines the meaning of the question.

[0757] Input: Text question

[0758] Output: Question analysis results (e.g. warranty period extraction)

[0759] Step 5:

[0760] The server analyzes the user's emotional information using an emotion engine, and the analysis results include specific emotions.

[0761] Input: Emotional information data

[0762] Output: Emotion analysis results (e.g., anxiety recognition)

[0763] Step 6:

[0764] The server utilizes a FAQ section and AI chatbot to generate appropriate answers.

[0765] Input: Question analysis results, sentiment analysis results

[0766] Output: Generated answer (e.g. warranty period details)

[0767] Step 7:

[0768] The generated response is sent to the device, with the data in the appropriate format.

[0769] Input: Generated Answer

[0770] Output: Submitted response

[0771] Step 8:

[0772] The terminal displays the answers to the user through a GUI.

[0773] Input: The answer sent by the server

[0774] Output: The answer is displayed visually to the user

[0775] Community Features

[0776] Program processing

[0777] Step 1:

[0778] Users input reviews and opinions about products. The input data is saved on the device.

[0779] Input: Review or opinion text

[0780] Output: Input data is stored in the terminal

[0781] Step 2:

[0782] The device collects the user's emotional information through the camera and microphone, and the collected data is stored on the device.

[0783] Input: User's facial expression, tone of voice

[0784] Output: Emotional information data (facial expression recognition results, voice tone analysis results)

[0785] Step 3:

[0786] The device sends reviews and sentiment information to the server in the form of packets.

[0787] Input: Review text and sentiment data

[0788] Output: sent as data packets to the server

[0789] Step 4:

[0790] The server stores the reviews and sentiment information in a database, and the data is stored in a suitable format.

[0791] Input: Review text, emotional information data

[0792] Output: Saved data

[0793] Step 5:

[0794] When another user visits a particular product page, the server searches the database for reviews of that product, and the results reflect the emotional tone of the reviews.

[0795] Input: Product page visit request

[0796] Output: Search results (reviews and sentiment tone)

[0797] Step 6:

[0798] The server sends the search results to the device in the appropriate format.

[0799] Input: Search results

[0800] Output: Submitted search results

[0801] Step 7:

[0802] The device displays the reviews to the user through a GUI.

[0803] Input: Reviews sent from the server

[0804] Output: The review is displayed visually to the user

[0805] (Application example 2)

[0806] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0807] Conventional purchasing experience systems suggest related products by analyzing text and images entered by consumers, but do not take into account consumer circumstances such as emotions and behavior, which limits the degree of personalization. Furthermore, recommendations based solely on events or purchase history make it difficult to accurately grasp consumer needs in real time, posing challenges to improving consumer satisfaction.

[0808] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving and analyzing text or images entered by a consumer, means for searching a database for related products based on the analysis results, means for displaying the search results to the consumer, means for collecting the consumer's facial expressions, movements, and voice using cameras and sensors in the store, means for analyzing the collected data to recognize the consumer's emotions, and means for suggesting related products based on the recognized emotions. This enables more advanced and precise personalized product suggestions that take into account the consumer's emotions and real-time situation.

[0809] "Consumer" refers to any member of the public who intends to purchase a service or product.

[0810] "Input text or image" refers to written or visual information provided by the consumer to the system.

[0811] "Means of analysis" refers to the technology that understands input text or images and extracts information based on them.

[0812] "Means for searching related products from a database" refers to technology that searches for appropriate product information from a database based on the analysis results.

[0813] "Means for displaying search results to consumers" refers to the technology for displaying information retrieved from a database on a device used by a consumer.

[0814] "Cameras and sensors in the store" refers to monitoring and sensing devices used to capture consumer movements, facial expressions, and voices within the store.

[0815] "Means of collection" refers to technology that uses cameras and sensors to acquire data such as consumer movements, facial expressions, and voice.

[0816] "Means for recognizing consumer emotions by analyzing collected data" refers to technology that analyzes acquired data and identifies the emotions consumers are feeling.

[0817] "Means for suggesting related products based on recognized emotions" refers to technology that selects and suggests products suitable for consumers based on the results of emotion analysis.

[0818] This invention is a system for improving consumer purchasing experiences, and aims to analyze consumer sentiment and provide personalized product recommendations, particularly in brick-and-mortar stores. This system is implemented based on the following configuration and processing steps.

[0819] System configuration

[0820] 1. Hardware Configuration

[0821] Cameras and sensors: These are installed in physical stores to collect consumers' facial expressions, movements, and voices. For example, we use standard surveillance cameras and voice recognition sensors.

[0822] Devices: Consumers' smartphones, smart glasses, head-mounted displays (HMDs), etc. These devices send collected data to a server and display information from the server to the consumer.

[0823] Server: A server for data analysis and product proposals.

[0824] 2. Software Configuration

[0825] Natural Language Processing (NLP): Analyzing consumer-supplied text or speech, for example using common natural language processing algorithms.

[0826] Image recognition algorithm: Analyzes images uploaded by consumers and identifies related products. As a concrete example, we will use a common image recognition algorithm.

[0827] Emotion engine: Recognizes consumer emotions based on collected facial and voice data. As a concrete example, a general emotion recognition algorithm is used.

[0828] Database: A database for storing product information. Specific examples include SQL databases and NoSQL databases.

[0829] Processing flow

[0830] 1. Data Collection

[0831] Cameras and sensors are used to collect the consumer's facial expressions, movements, and voice, and the data is sent to a server via the device.

[0832] 2. Data Analysis

[0833] The data received by the server is analyzed using the following software:

[0834] Natural Language Processing Algorithms

[0835] Image Recognition Algorithm

[0836] Emotion Engine

[0837] 3. Product proposal

[0838] The server searches the product database based on the analysis results, identifies related products, and sends them to the terminal, which then displays the obtained product information to the consumer.

[0839] Specific examples

[0840] Scenario 1: Entering the store

[0841] A consumer enters a store and uses a device (e.g., smart glasses) to navigate the store. When the camera captures the consumer's interested expression, the emotion engine recognizes the interest and the server sends a guide message based on that, showing new product sections and recommended products.

[0842] Scenario 2: Product selection support

[0843] If a consumer appears to be unsure about a particular product, the camera captures the situation and the emotion engine recognizes the emotion of "indecision." Based on this, the server suggests other highly rated products and related articles, and displays this information on the device.

[0844] Prompt sentence for generative AI model

[0845] A customer enters a physical store and uses a smart device. The in-store camera captures the customer's facial expressions and behavior. If the customer shows interest, output a product suggestion method based on that interest. Also, explain how to suggest alternative products if the customer is unsure.

[0846] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0847] Step 1:

[0848] A user enters a store and uses a device such as a smartphone, smart glasses, or head-mounted display (HMD). Cameras and sensors collect the user's facial expressions, movements, and voice. This data is then input into the device.

[0849] Step 2:

[0850] The facial expression, movement, and voice data collected by the device are sent to a server in real time. Data input includes image data and voice data.

[0851] Step 3:

[0852] The server analyzes the received data. Specifically, it performs the following processes:

[0853] Natural language processing (NLP) algorithms analyze voice data and user-entered text.

[0854] Image recognition algorithms analyze the collected image data to identify product categories of interest.

[0855] The emotion engine analyzes facial and voice data to identify the user's emotions (e.g., interest, uncertainty, anxiety).

[0856] Step 4:

[0857] The server combines the user's emotional information obtained as a result of the analysis with the results of NLP and image recognition to search for related products in a database. The input includes emotional data, voice and text data, and image data, and generates a list of related products as an output.

[0858] Step 5:

[0859] The server generates a related product list and sends it to the device. The device receives it and displays it to the user. The displayed product list is updated in real time based on the user's facial expressions and movements.

[0860] Step 6:

[0861] The user can review the displayed product list and request more information about products they are interested in. Inputs at the device include touch and voice commands, and the details are again requested from the server.

[0862] Step 7:

[0863] The server retrieves detailed information from the database based on the user's request and sends it to the terminal, which then displays the information to the user, allowing the user to view more detailed product information.

[0864] Through the above steps, real-time personalized product suggestions that take user emotions into consideration are realized.

[0865] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0866] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0867] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0868] [Second embodiment]

[0869] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0870] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0871] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0872] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0873] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0874] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0875] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0876] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0877] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0878] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0879] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0880] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0881] The present invention relates to a system for personalizing a consumer's purchasing experience and providing highly relevant information and products. A specific embodiment of this system will be described below.

[0882] Intelligent Search Function

[0883] Users can enter text or upload an image into the search bar, and the system will analyze the input and search the database for relevant product information to display. This intelligent search function allows consumers to efficiently obtain the product information they are looking for.

[0884] Program processing

[0885] The user types "What's the latest version of the refrigerator in this picture?" into the app's search bar, or uploads an image of the refrigerator. The device sends the input data to the server, which uses natural language processing (NLP) algorithms to analyze the text and understand the user's intent. In the case of an image, the server uses image recognition algorithms to extract product features. Based on the analysis, the server searches a product database to identify relevant products. The search results are sent to the device, which displays them to the user.

[0886] Specific examples

[0887] The user uploads an image of their refrigerator, and the device sends the image to the server. The server uses image recognition technology to identify the refrigerator model and searches the database for the latest information. The search results are sent to the device, and the latest model information is displayed to the user.

[0888] Event-based gift recommendation engine

[0889] Users can enter gift-related questions via text, and the system will analyze the event and target information and recommend gift items based on that information, allowing consumers to find the perfect gift for a special event or occasion.

[0890] Program processing

[0891] The user types, "What gift would you recommend for my mother's birthday?" The device sends the question to the server. The server analyzes the question and extracts the event target information "birthday" and "mother." The server uses an event-based gift recommendation engine to search for relevant products and sends the recommendation results to the device. The device displays the recommended gift items to the user.

[0892] Specific examples

[0893] When a user types "What would you recommend for my mother's birthday?", the device sends the question to the server. The server analyzes the "birthday" and "mother" and recommends gift items based on that. The recommendation results are sent to the device, and suitable gift candidates are displayed to the user.

[0894] Personalized Shopping Experience

[0895] When a user logs in to the app, the system will suggest products tailored to their individual preferences based on their past purchase and browsing history. This personalization feature allows users to quickly search for product information based on their interests and preferences.

[0896] Program processing

[0897] A user logs in to the app. The device sends the user's past purchase and browsing history to the server. The server analyzes the user's historical data and identifies individual preferences and interests. The server generates personalized product suggestions based on the analysis results and sends a list of suggested products to the device. The device displays the list of suggested products to the user.

[0898] Specific examples

[0899] When a user logs in to the app, their device sends their past purchase history to the server. The server analyzes the data and suggests products that match the user's preferences. A list of suggested products is sent to the device and displayed to the user.

[0900] Purchasing support function

[0901] When users ask questions about specific products, the system provides quick answers using the FAQ section and AI chatbots, helping users to resolve any doubts or concerns they may have about their purchase.

[0902] Program processing

[0903] The user types a question, such as "How long is the warranty on this refrigerator?" The device sends the question to the server, which uses an FAQ section or an AI chatbot to generate an appropriate answer to the question. The generated answer is sent to the device, which then displays the appropriate answer to the user.

[0904] Specific examples

[0905] When a user asks about the warranty period of a refrigerator, the device sends the question to the server, which parses the question and retrieves the warranty period information from the FAQ section. The retrieved information is sent to the device and displayed to the user.

[0906] Community Features

[0907] Users can post reviews and opinions about products, and other users can view that information. This feature allows consumers to make purchasing decisions while also taking into account the opinions of other users.

[0908] Program processing

[0909] The user enters a review or opinion on a product. The device sends the review to the server. The server stores the received review in a database. When another user visits a specific product page, the server searches the database for reviews of that product and sends the search results to the device. The device displays the review to the user.

[0910] Specific examples

[0911] A user posts a review about a refrigerator, and the device sends it to the server. The server stores the review in a database. When another user visits the refrigerator's product page, the server retrieves the stored review and sends it to the device. The device displays the review to the user.

[0912] In this way, the system according to the claims can enhance the consumer's buying experience and provide personalized services.

[0913] The processing flow will be explained below.

[0914] Intelligent Search Function

[0915] Program processing

[0916] Step 1:

[0917] A user types "What's the latest version of the refrigerator in this picture?" into the app's search bar or uploads an image of the refrigerator.

[0918] Step 2:

[0919] The device sends the entered text and images to the server.

[0920] Step 3:

[0921] The server analyzes the submitted text using natural language processing (NLP) algorithms to understand the user's intent.

[0922] Step 4:

[0923] The server analyzes the transmitted image using an image recognition algorithm and extracts its features.

[0924] Step 5:

[0925] The server searches a product database based on the analysis results to identify highly relevant products.

[0926] Step 6:

[0927] The server transmits the search results to the terminal.

[0928] Step 7:

[0929] The terminal displays the search results to the user.

[0930] Event-based gift recommendation engine

[0931] Program processing

[0932] Step 1:

[0933] The user texts in, "What do you recommend for my mom's birthday?"

[0934] Step 2:

[0935] The terminal sends the text to the server.

[0936] Step 3:

[0937] The server analyzes the submitted text using natural language processing (NLP) to identify the event "birthday" and the target "mother."

[0938] Step 4:

[0939] The server uses an event-based gift recommendation engine to search for relevant gift items.

[0940] Step 5:

[0941] The server transmits the recommendation results to the terminal.

[0942] Step 6:

[0943] The terminal displays a list of recommended products to the user.

[0944] Personalized Shopping Experience

[0945] Program processing

[0946] Step 1:

[0947] The user logs in to the app.

[0948] Step 2:

[0949] The terminal sends the logged-in user's past purchase history and browsing history to the server and loads that data.

[0950] Step 3:

[0951] The server analyzes the received historical data to identify the user's preferences and interests.

[0952] Step 4:

[0953] The server generates personalized product suggestions based on the analysis results.

[0954] Step 5:

[0955] The server transmits the generated list of suggested products to the terminal.

[0956] Step 6:

[0957] The terminal displays a list of suggested products to the user.

[0958] Purchasing support function

[0959] Program processing

[0960] Step 1:

[0961] A user types a question: "How long is the warranty on this refrigerator?"

[0962] Step 2:

[0963] The terminal transmits the entered question to the server.

[0964] Step 3:

[0965] The server analyzes the submitted questions and uses an FAQ section or AI chatbot to generate appropriate answers.

[0966] Step 4:

[0967] The server sends the generated answer to the terminal.

[0968] Step 5:

[0969] The terminal displays the generated answer to the user.

[0970] Community Features

[0971] Program processing

[0972] Step 1:

[0973] The user enters reviews and opinions about the purchased product.

[0974] Step 2:

[0975] The device sends the entered reviews and opinions to the server.

[0976] Step 3:

[0977] The server stores the received reviews in a database.

[0978] Step 4:

[0979] When another user visits a particular product page, the server searches the database for reviews of that product.

[0980] Step 5:

[0981] The server sends the review of the search results to the terminal.

[0982] Step 6:

[0983] The terminal displays the searched reviews to the user.

[0984] Example 1

[0985] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0986] Conventional online shopping systems make it difficult for consumers to efficiently find the best products for them from the vast amount of product information available, resulting in a cumbersome shopping experience. Furthermore, product recommendations based on special events or individual preferences are not adequately implemented, which can lead to low consumer satisfaction. Furthermore, the inability to easily obtain detailed product information or reviews can make it difficult to resolve concerns or questions about purchasing.

[0987] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0988] In this invention, the server includes a means for analyzing text or images entered by a consumer, a means for analyzing the text using a natural language processing algorithm to understand the user's intent, a means for analyzing the image using an image recognition algorithm to extract product features, a means for searching a database for related products based on the analysis results, and a means for displaying the search results to the consumer. This allows consumers to efficiently find the most suitable product. In addition, personalized recommendations based on events and past history improve consumer satisfaction and provide a comfortable shopping experience.

[0989] "Consumer" refers to a general user of the system for the purpose of purchasing goods or services.

[0990] "Text" refers to sentences or characters entered by consumers, and is textual information that is understood by the system.

[0991] "Images" refers to visual information such as photographs and illustrations uploaded by consumers.

[0992] "Means of analysis" refers to methods and techniques for processing input text or images with a computer program and understanding their content.

[0993] A "natural language processing algorithm" refers to a computational method for analyzing text and understanding the meaning and intent of human language.

[0994] "Means for understanding user intent" refers to methods that use natural language processing algorithms to analyze the meaning and purpose of text entered by a user.

[0995] An "image recognition algorithm" refers to a computational method for analyzing an image and identifying the objects and features contained within it.

[0996] "Means for extracting product features" refers to the use of image recognition algorithms to identify specific products and their attributes from uploaded images.

[0997] "Related Products" refers to products and services available for purchase that are suggested based on information entered or uploaded by the consumer.

[0998] A "database" refers to a collection of information that systematically stores related products and other information and makes it easy to search and retrieve.

[0999] "Searching means" refers to the methods and technologies used to search the database based on the analysis results and find relevant product information.

[1000] "Means for displaying search results to consumers" refers to methods and technologies for displaying searched product information on a user interface so that consumers can check it.

[1001] "Event" refers to a specific situation or occasion, such as a birthday or anniversary, and is the criterion for recommending products related to that situation.

[1002] "Personalized product suggestions" refers to a method of recommending products that match a consumer's individual preferences based on individual data such as their past purchase history and browsing history.

[1003] The present invention relates to a system for personalizing a consumer's shopping experience and providing highly relevant information and products. The system has multiple functions for analyzing user-entered text and images and efficiently providing relevant product information.

[1004] Intelligent Search Function

[1005] Users can enter text or upload an image into the search bar, and the system will analyze the input and search the database for relevant product information to display. This intelligent search function allows consumers to efficiently obtain the product information they are looking for.

[1006] Hardware and software used

[1007] Server: A computer with high processing power (e.g., a Linux server)

[1008] Terminal: The device that provides the user interface (e.g., smartphone, tablet, PC)

[1009] Natural Language Processing Algorithms: Google Cloud Natural Language API

[1010] Image recognition algorithm: Amazon Rekognition

[1011] Database: MySQL

[1012] Specific actions

[1013] The user types the text "What is the latest version of the refrigerator in this image?" into the app's search bar or uploads an image of the refrigerator.

[1014] The terminal sends the input data to the server.

[1015] The server uses the Google Cloud Natural Language API to analyze the text and understand the user's intent.

[1016] In the case of images, the server uses Amazon Rekognition to perform image recognition and extract the product's features.

[1017] The server searches a MySQL database based on the analysis results to identify relevant products.

[1018] The search results are sent to the terminal, which displays the results to the user.

[1019] Event-based gift recommendation engine

[1020] Users can enter gift-related questions via text, and the system will analyze the event and target information and recommend gift items based on that information, allowing consumers to find the perfect gift for a special event or occasion.

[1021] Specific actions

[1022] User types, "What do you recommend for my mom's birthday?"

[1023] The terminal sends a question to the server.

[1024] The server parses the question using the Google Cloud Natural Language API and extracts the event target information "birthday" and "mother."

[1025] The server uses an event-based gift recommendation engine to search for relevant products, and the recommendation results are sent to the terminal.

[1026] The terminal displays the recommended gift items to the user.

[1027] Personalized Shopping Experience

[1028] When a user logs in to the app, the system will suggest products tailored to their individual preferences based on their past purchase and browsing history. This personalization feature allows users to quickly search for product information based on their interests and preferences.

[1029] Specific actions

[1030] A user logs in to the app.

[1031] The device sends the user's past purchase history and browsing history to the server.

[1032] The server analyzes the received historical data to identify the user's preferences and interests.

[1033] The server generates personalized product suggestions based on the analysis results, and a list of suggested products is sent to the terminal.

[1034] The terminal displays a list of suggested products to the user.

[1035] Purchasing support function

[1036] When users ask questions about specific products, the system provides quick answers using the FAQ section and AI chatbots, helping users to resolve any doubts or concerns they may have about their purchase.

[1037] Specific actions

[1038] A user types a question: "How long is the warranty on this refrigerator?"

[1039] The terminal sends a question to the server.

[1040] The server uses Dialogflow to analyze the question and generate an appropriate answer from the FAQ database.

[1041] The generated answer is sent to the terminal, which displays the appropriate answer to the user.

[1042] Community Features

[1043] Users can post reviews and opinions about products, and other users can view that information. This feature allows consumers to make purchasing decisions while also taking into account the opinions of other users.

[1044] Specific actions

[1045] Users enter product reviews and opinions.

[1046] The device sends the review to the server.

[1047] The server stores the received reviews in a MySQL database.

[1048] When another user visits a specific product page, the server searches the database for reviews of that product and sends the search results to the terminal.

[1049] The device displays the review to the user.

[1050] With the above functions, the system of the present invention can improve the consumer's purchasing experience and provide personalized services.

[1051] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1052] Intelligent Search Function

[1053] Step 1:

[1054] A user types the text "What is the latest version of the refrigerator in this image?" into the search bar or uploads an image of a refrigerator.

[1055] Input: Text or image from the user

[1056] Output: Text or image sent to the terminal

[1057] Step 2:

[1058] The terminal sends the input data to the server.

[1059] Input: Text or Image

[1060] Output: HTTP request sent to the server

[1061] Step 3:

[1062] The server branches the process depending on the input data. For text, the server analyzes the text using the Google Cloud Natural Language API to understand the user's intent. For images, the server extracts features using Amazon Rekognition.

[1063] Input: Text or image sent to the server

[1064] Output: Analysis results (user intent for text, feature data for images)

[1065] Step 4:

[1066] Based on the analysis results, the server searches the MySQL database to identify relevant products.

[1067] Specifically, an SQL query is generated based on the extracted keywords and features and issued to the database.

[1068] Input: Analysis results (user intent or feature data)

[1069] Output: Related product information

[1070] Step 5:

[1071] The server formats the search results in JSON format and sends them to the terminal as an HTTP response.

[1072] Input: Relevant product information

[1073] Output: Search results in JSON format

[1074] Step 6:

[1075] The terminal parses the received search results and displays them on the user interface.

[1076] Input: JSON format search results

[1077] Output: Product information displayed in the user interface

[1078] Event-based gift recommendation engine

[1079] Step 1:

[1080] User types, "What do you recommend for my mom's birthday?"

[1081] Input: User question text

[1082] Output: Question data sent to the terminal

[1083] Step 2:

[1084] The terminal transmits the question data to the server.

[1085] Input: Question data

[1086] Output: HTTP request sent to the server

[1087] Step 3:

[1088] The server parses the question using the Google Cloud Natural Language API and extracts the event target information "birthday" and "mother."

[1089] Input: Query data sent to the server

[1090] Output: Extracted event and target information

[1091] Step 4:

[1092] The server searches for related products using a gift recommendation engine based on the event information and target information, and obtains recommendation results.

[1093] Input: Event information, target information

[1094] Output: Gift recommendation results

[1095] Step 5:

[1096] The server formats the recommendation results in JSON format and sends them to the terminal as an HTTP response.

[1097] Input: Gift recommendation results

[1098] Output: Recommendation results in JSON format

[1099] Step 6:

[1100] The device parses the received recommendation results and displays them on the user interface.

[1101] Input: Recommendation results in JSON format

[1102] Output: Gift information displayed in the user interface

[1103] Personalized Shopping Experience

[1104] Step 1:

[1105] A user logs in to the app.

[1106] Input: Username and Password

[1107] Output: Login request

[1108] Step 2:

[1109] The terminal transmits the user's past history data to the server.

[1110] Input: Historical data

[1111] Output: HTTP request sent to the server

[1112] Step 3:

[1113] The server analyzes the historical data to identify the user's preferences and interests.

[1114] Input: Historical data sent to the server

[1115] Output: Analysis results (user preferences and interests)

[1116] Step 4:

[1117] The server generates personalized product suggestions based on the analysis results using a Recommendation Engine.

[1118] Input: Analysis results (user preferences and interests)

[1119] Output: Personalized product recommendations

[1120] Step 5:

[1121] The server formats the list of suggested products in JSON format and sends it to the terminal as an HTTP response.

[1122] Input: Product suggestion list

[1123] Output: A list of suggestions in JSON format

[1124] Step 6:

[1125] The terminal parses the received proposal list and displays it on the user interface.

[1126] Input: A list of suggestions in JSON format

[1127] Output: A list of suggested products displayed in the user interface

[1128] Purchasing support function

[1129] Step 1:

[1130] A user enters a question about a particular product.

[1131] Input: Question text

[1132] Output: Question data sent to the terminal

[1133] Step 2:

[1134] The terminal sends a question to the server.

[1135] Input: Question data

[1136] Output: HTTP request sent to the server

[1137] Step 3:

[1138] The server uses Dialogflow to analyze the question and generate an appropriate answer.

[1139] Input: Query data sent to the server

[1140] Output: Correct answer data

[1141] Step 4:

[1142] The server searches the FAQ database and retrieves the corresponding information.

[1143] Input: Question data

[1144] Output: FAQ information

[1145] Step 5:

[1146] The server formats the acquired FAQ information into JSON format and sends it to the terminal as an HTTP response.

[1147] Input: FAQ information

[1148] Output: JSON formatted response data

[1149] Step 6:

[1150] The terminal parses the received response data and displays it on the user interface.

[1151] Input: JSON formatted response data

[1152] Output: Answer information displayed in the user interface

[1153] Community Features

[1154] Step 1:

[1155] Users enter product reviews and opinions.

[1156] Input: Review or opinion text

[1157] Output: Review sent to device

[1158] Step 2:

[1159] The device sends the review to the server.

[1160] Input: Review text

[1161] Output: HTTP request sent to the server

[1162] Step 3:

[1163] The server stores the received reviews in a MySQL database.

[1164] Input: The review sent to the server

[1165] Output: Reviews stored in a database

[1166] Step 4:

[1167] Another user visits a specific product page.

[1168] Input: Product page request

[1169] Output: HTTP request to the server

[1170] Step 5:

[1171] The server searches the database for product reviews.

[1172] Input: Product page request

[1173] Output: Review information as search results

[1174] Step 6:

[1175] The server formats the search results in JSON format and sends them to the terminal as an HTTP response.

[1176] Input: Review information as search results

[1177] Output: Review information in JSON format

[1178] Step 7:

[1179] The terminal parses the received review information and displays it on the user interface.

[1180] Input: JSON formatted review information

[1181] Output: Review information displayed in the user interface

[1182] (Application example 1)

[1183] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1184] Traditional purchasing experiences often only offer consumers a uniform selection of products, making it difficult to provide personalized suggestions based on individual consumer preferences and past history. Product searches using text and images also have low accuracy, preventing consumers from efficiently finding the products they are looking for. Furthermore, responses to consumer questions about specific products are often delayed, hindering purchasing decisions. Therefore, there is a need for a system that can more individually tailor the consumer purchasing experience and provide relevant information quickly and appropriately.

[1185] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1186] In this invention, the server includes means for receiving and analyzing text or images entered by a consumer, means for searching a database for related products based on the analysis results, means for displaying the search results to the consumer, means for analyzing purchase history and browsing history based on a consumer ID and generating personalized product suggestions, means including an integrated natural language processing algorithm and an image recognition algorithm, and means for generating appropriate answers to end-user questions using artificial intelligence, thereby enabling consumers to efficiently receive product suggestions that match their preferences and the information they are looking for.

[1187] A "consumer ID" is an identifier used to uniquely identify a consumer.

[1188] "Purchase history" is a record of products and services that a consumer has purchased in the past.

[1189] "Browser history" is a record of the products and pages a consumer has previously viewed on a website or app.

[1190] "Personalized product recommendations" are recommendations for products or services that are customized for a specific consumer based on that consumer's past behavior and preferences.

[1191] "Text analysis" is the process of analyzing input text data and understanding its meaning and content.

[1192] "Image analysis" is the process of analyzing input image data and recognizing objects and features within it.

[1193] A database is a collection of information that organizes and stores related data so that it can be quickly searched.

[1194] A "natural language processing algorithm" is a computational method for understanding and analyzing text data.

[1195] An "image recognition algorithm" is a computational method for analyzing image data and identifying specific objects or features.

[1196] "Artificial intelligence" is a technology that gives computers the ability to learn and reason like human intelligence.

[1197] The "means for generating an appropriate answer to a question" is a process for analyzing the question entered by the consumer and generating an answer based on related information.

[1198] "Related product search" is the process of locating related products from a database based on consumer input.

[1199] "Product proposal generation" is the process of making product proposals to individual consumers based on their past behavioral data.

[1200] A "display means" is a method or device that visually presents information, such as search results or suggested products, to consumers.

[1201] This invention is a system that allows consumers to input text or images through an interface, and then analyzes the input to provide related products. In particular, it has a personalized product suggestion function based on the consumer's purchase history and browsing history, and a question-answering function using artificial intelligence.

[1202] Program Overview

[1203] This system mainly consists of the following hardware and software:

[1204] 1. Hardware:

[1205] Server: AWS EC2

[1206] Database: Amazon RDS (MySQL)

[1207] User devices: smartphones and computers

[1208] 2. Software:

[1209] Natural Language Processing (NLP) Algorithm: Google's BERT Model

[1210] Image recognition algorithm: Google Cloud Vision API

[1211] Server-side program: Python (Flask)

[1212] System Operation Overview

[1213] Intelligent Search Function

[1214] A user enters text into the device's search bar or uploads an image. The device sends the input data to the server, which analyzes the text input with a natural language processing algorithm or the image input with an image recognition algorithm. The server then searches the database to identify relevant products. The search results are sent to the device and displayed to the user.

[1215] Personalized Shopping Experience

[1216] When a user logs in to the app, the device sends the user's ID to the server, which analyzes their purchase and browsing history to generate a personalized product recommendation list, which is then sent to the device and displayed to the user.

[1217] Purchasing support function

[1218] A user enters a question about a particular product. The device sends the question to the server, which uses natural language processing algorithms and an FAQ database to generate an appropriate answer. The answer is then sent to the device and displayed to the user.

[1219] Specific examples

[1220] Intelligent Search: The user uploads a picture of their refrigerator and sends it to the server. The server uses image recognition technology to identify the refrigerator model and retrieves the latest information from the database. The search results are sent to the device, and the latest model information is displayed to the user.

[1221] Personalized shopping experience: When a user logs into the app and a previously purchased smartwatch is recorded, the server suggests new smartwatch models and displays the list to the user.

[1222] Purchasing support function: When a user asks, "How long is the warranty period for this refrigerator?", the server refers to the FAQ database and provides the user with information about the warranty period.

[1223] Prompt Sentence Examples

[1224] If a user types "What is the latest iPhone model?", the database will be searched for and displayed as relevant latest iPhone models.

[1225] Input: What is the latest model of iPhone?

[1226] Output: The iPhone 13 is currently the latest model and comes with the latest features.

[1227] If you log in with user ID "12345", we will recommend new smartwatch models based on your past purchase history.

[1228] Input: I would like products recommended based on the purchasing history of user ID "12345".

[1229] Output: We recommend the new Apple Watch Series 6.

[1230] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1231] Intelligent Search Processing Steps

[1232] Step 1: User Input

[1233] The user enters text or uploads an image into the device's input interface.

[1234] Input: Text or image

[1235] Output: Text or image data

[1236] Step 2: Send data

[1237] The device sends input data (text or image) to the server using HTTP POST as the transmission protocol.

[1238] Input: Text or image data entered by the user

[1239] Output: Text or image data sent to the server

[1240] Step 3: Data analysis

[1241] The server analyzes the received data. For text data, it uses a natural language processing algorithm (Google's BERT) to analyze the intent, and for image data, it uses an image recognition algorithm (Google Cloud Vision API) to extract features.

[1242] Input: Text or image data sent to the server

[1243] Output: Text analysis results or image feature data

[1244] Step 4: Database Search

[1245] The server searches the product database based on the analysis results or extracted features and executes SQL queries to retrieve relevant product information.

[1246] Input: Text analysis results or image feature data

[1247] Output: Product information

[1248] Step 5: Submit search results

[1249] The server sends the search results to the device in JSON format.

[1250] Input: JSON data of search results

[1251] Output: Product information sent to the device

[1252] Step 6: View the results

[1253] The product information received by the terminal is displayed on a user interface.

[1254] Input: Product information sent to the terminal

[1255] Output: The product list displayed to the user

[1256] Processing steps for a personalized shopping experience

[1257] Step 1: User Login

[1258] The user enters their credentials to log in to the app.

[1259] Input: User ID and password

[1260] Output: Authentication token

[1261] Step 2: Send past history

[1262] The device sends the user ID to the server, which retrieves relevant purchase and browsing history from its internal database.

[1263] Input: User ID

[1264] Output: Purchase history and browsing history data

[1265] Step 3: Historical data analysis

[1266] The server analyzes the historical data and generates a personalized product list.

[1267] Input: Purchase history and browsing history data

[1268] Output: Personalized product list

[1269] Step 4: Submit your proposal list

[1270] The server transmits the generated product proposal list to the terminal.

[1271] Input: Personalized Product List

[1272] Output: Suggestion list sent to the device

[1273] Step 5: List View

[1274] The terminal displays the product suggestion list on the user interface.

[1275] Input: Suggestion list sent to the device

[1276] Output: A list of suggested products displayed to the user

[1277] Purchasing Support Function Processing Steps

[1278] Step 1: Enter your question

[1279] The user inputs a question about the product they want to purchase.

[1280] Input: Question text

[1281] Output: The question text entered

[1282] Step 2: Submit your question

[1283] The terminal sends the question text to the server.

[1284] Input: The entered question text

[1285] Output: The question text sent to the server

[1286] Step 3: Question analysis

[1287] The server analyzes the question text using natural language processing algorithms.

[1288] Input: Question text sent to the server

[1289] Output: Question analysis results

[1290] Step 4: Generate appropriate answers

[1291] The server searches the FAQ database for relevant answers and generates an appropriate answer.

[1292] Input: Question analysis results

[1293] Output: Correct answer

[1294] Step 5: Submit your response

[1295] The server sends the generated response to the terminal.

[1296] Input: Correct Answer

[1297] Output: Answer sent to the terminal

[1298] Step 6: View Answers

[1299] The terminal displays the generated answer on a user interface.

[1300] Input: Answer sent to the terminal

[1301] Output: The answer that is displayed to the user

[1302] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1303] This invention relates to a system that personalizes the consumer's purchasing experience and provides highly relevant information and products. In particular, by combining it with an emotion engine that recognizes the user's emotions, a higher level of personalization is achieved. Specific embodiments of this system are as follows:

[1304] Intelligent search function with emotion engine

[1305] Users can enter text into the search bar or upload an image, and the system analyzes the input, searches the database for relevant product information, and displays it. Utilizing an emotion engine, the system provides search results that take the user's emotions into account.

[1306] Program processing

[1307] The user types "What's the latest version of the refrigerator in this picture?" into the app's search bar, or uploads an image of the refrigerator. The device sends the input data, along with the user's facial expressions and tone of voice captured by the camera and microphone, to the server. The server analyzes the text with natural language processing (NLP) algorithms and the image with image recognition algorithms, while analyzing the user's emotions with an emotion engine. Based on the analysis results, it searches a product database to identify related products. The related products are sent to the device, which displays the results to the user.

[1308] Specific examples

[1309] The user uploads an image of their refrigerator, which the device sends to the server. The server uses image recognition technology to identify the refrigerator model, and if it detects a positive emotion in the user's facial expression, it searches the database for information on the latest, more expensive, and highly rated models. The search results are sent to the device, and the information on the latest model is displayed to the user.

[1310] Event-based gift recommendation engine

[1311] When a user enters a gift-related question in text, the system analyzes the event and target information and recommends gift items based on that information. By combining this function with an emotion engine, it becomes possible to make recommendations based on the consumer's emotions.

[1312] Program processing

[1313] The user types "What gift would you recommend for my mother's birthday?" and emotional information is also collected via the camera and microphone. The device sends the text and emotional information to the server. The server analyzes the question and identifies the event / target information of "birthday" and "mother," while simultaneously analyzing the user's emotions using an emotional engine. The server uses an event-based gift recommendation engine to search for related products, and the recommendation results are sent to the device. The device then displays the recommended gift items to the user.

[1314] Specific examples

[1315] If a user looks hesitant while typing "What would you recommend for my mother's birthday?", the server will recognize this hesitation using its emotion engine and prioritize recommending reliable and popular gift items. The recommended items are then sent to the device and displayed to the user.

[1316] Personalized Shopping Experience

[1317] When a user logs in to the app, the system will suggest products tailored to their individual preferences based on their past purchase and browsing history. Adding an emotion engine to this personalization function will enable even more advanced personalization.

[1318] Program processing

[1319] When a user logs in to the app, the device sends emotional information such as facial expression and tone of voice at the time of login, along with past purchase history and browsing history, to the server. The server analyzes the received history and emotional data to identify the user's preferences and interests. Based on the analysis results, the server generates personalized product suggestions that also take emotional information into account, and a list of suggested products is sent to the device. The device then displays the list of suggested products to the user.

[1320] Specific examples

[1321] When a user logs in to the app, the emotion engine recognizes their happy facial expression. The server takes this happy emotion into account along with their past purchase history to suggest new products that the user is likely to enjoy. The list of suggestions is sent to the device and displayed to the user.

[1322] Purchasing support function

[1323] When a user asks a question about a specific product, the system provides a quick answer using the FAQ section or an AI chatbot. The emotion engine also analyzes emotions, allowing the system to provide a more accurate answer based on the user's emotions.

[1324] Program processing

[1325] The user asks, "How long is the warranty period for this refrigerator?" and emotional information is also collected. The device sends the question and emotional information to the server. The server analyzes the question, recognizes the user's emotions using an emotional engine, and uses an FAQ section or AI chatbot to generate an appropriate answer. The generated answer is sent to the device, which displays it to the user.

[1326] Specific examples

[1327] If a user asks about the warranty period for a refrigerator and shows a worried expression, the server will recognize the user's anxiety using its emotion engine, provide a detailed explanation of the warranty, and provide information on related reliable products. The answer will be sent to the terminal and displayed to the user.

[1328] Community Features

[1329] Users can post reviews and opinions about products, which can then be viewed by other users. By combining this feature with an emotion engine, it is possible to understand the emotional tone of the reviews.

[1330] Program processing

[1331] Users input reviews and opinions, and emotional information is also collected. The device sends the reviews and emotional information to the server. The server stores the reviews and emotional information in a database. When another user visits a specific product page, the server searches the database for reviews of that product and sends the review results, which also reflect the emotional tone, to the device. The device displays the reviews to the user.

[1332] Specific examples

[1333] When a user posts a review about a refrigerator, the emotion engine analyzes the happy facial expression. The server stores the review and positive emotion in a database. When another user visits the refrigerator's product page, the review is displayed on their device as a positive emotion review.

[1334] In this way, the invention that combines the emotion engine can further personalize the consumer's purchasing experience and provide detailed emotional support.

[1335] The processing flow will be explained below.

[1336] Intelligent search function with emotion engine

[1337] Program processing

[1338] Step 1:

[1339] A user types "What's the latest version of the refrigerator in this picture?" into the app's search bar or uploads an image of the refrigerator.

[1340] Step 2:

[1341] The device sends the entered text and images, as well as the user's facial expressions and tone of voice captured through the camera and microphone, to the server.

[1342] Step 3:

[1343] The server analyzes the submitted text using natural language processing (NLP) algorithms to understand the user's intent.

[1344] Step 4:

[1345] The server analyzes the transmitted image using an image recognition algorithm and extracts its features.

[1346] Step 5:

[1347] The server uses an emotion engine to analyze the user's facial expressions and tone of voice to identify the user's emotions.

[1348] Step 6:

[1349] The server searches a product database based on the analysis results to identify highly relevant products.

[1350] Step 7:

[1351] The server transmits the search results together with supplementary information according to the user's feelings to the terminal.

[1352] Step 8:

[1353] The terminal displays the search results to the user.

[1354] Specific examples

[1355] The user uploads an image of the refrigerator, and the image, along with facial expression and tone of voice data, is sent from the device to the server.

[1356] The server uses image recognition technology to identify the refrigerator model, and if it detects a positive emotion in the user's facial expression, it searches the database for information on the latest, more luxurious and highly rated models.

[1357] The search results are sent to the device, and the user is shown information about the latest models and recommendations based on positive sentiment.

[1358] Event-based gift recommendation engine

[1359] Program processing

[1360] Step 1:

[1361] The user texts in, "What do you recommend for my mom's birthday?"

[1362] Step 2:

[1363] The device sends text and emotional information obtained through the camera and microphone to a server.

[1364] Step 3:

[1365] The server analyzes the submitted text using natural language processing (NLP) to identify the event "birthday" and the target "mother."

[1366] Step 4:

[1367] The server analyzes the user's emotions using an emotion engine and identifies the emotions.

[1368] Step 5:

[1369] The server uses an event-based gift recommendation engine to search for relevant gift items.

[1370] Step 6:

[1371] The server sends the search results and supplemental information based on the user's emotions to the terminal.

[1372] Step 7:

[1373] The terminal displays a list of recommended products to the user.

[1374] Specific examples

[1375] If a user types "What would you recommend for my mother's birthday?" and looks confused, the device sends the text and emotional information to the server.

[1376] The server identifies the "birthday" and "mother," and uses an emotion engine to recognize the customer's uncertainty and prioritize searching for reliable and popular gift items.

[1377] The recommended products are sent to the terminal, and a list of reliable products and the reasons for their selection are displayed to the user.

[1378] Personalized Shopping Experience

[1379] Program processing

[1380] Step 1:

[1381] The user logs in to the app.

[1382] Step 2:

[1383] The device sends past purchase history and browsing history to the server along with the user's facial expression and tone of voice when logging in.

[1384] Step 3:

[1385] The server analyzes the received history and emotion data to identify the user's preferences and interests.

[1386] Step 4:

[1387] A server generates personalized product suggestions based on preferences and interests.

[1388] Step 5:

[1389] The server transmits a list of suggested products that also takes emotional information into consideration to the terminal.

[1390] Step 6:

[1391] The terminal displays a list of suggested products to the user.

[1392] Specific examples

[1393] When a user logs in to the app, their happy facial expression is recognized by the emotion engine. The device then sends past purchase history and emotional information to the server.

[1394] The server analyzes the data and generates product suggestions that take into account the user's preferences and pleasant emotions. The suggestion list is sent to the terminal and displayed to the user.

[1395] Purchasing support function

[1396] Program processing

[1397] Step 1:

[1398] A user types a question: "How long is the warranty on this refrigerator?"

[1399] Step 2:

[1400] The device transmits the text and emotion information to the server.

[1401] Step 3:

[1402] The server analyzes the submitted questions and uses an FAQ section or AI chatbot to generate appropriate answers.

[1403] Step 4:

[1404] The server uses an emotion engine to analyze the user's emotions and generate a response based on those emotions.

[1405] Step 5:

[1406] The server sends the generated answer to the terminal.

[1407] Step 6:

[1408] The terminal displays the answer to the user.

[1409] Specific examples

[1410] If a user asks about the warranty period for a refrigerator and looks worried, the server will recognize the anxiety using its emotion engine, provide a detailed explanation of the warranty, and also provide information on related, more reliable products.

[1411] The answer is sent to the terminal and displayed to the user.

[1412] Community Features

[1413] Program processing

[1414] Step 1:

[1415] The user enters reviews and opinions about the purchased product.

[1416] Step 2:

[1417] The device transmits the input reviews, opinions, and sentiment information to the server.

[1418] Step 3:

[1419] The server stores the received reviews and sentiment information in a database.

[1420] Step 4:

[1421] When another user visits a particular product page, the server searches the database for reviews of that product.

[1422] Step 5:

[1423] The server sends the review of the search results and information reflecting the emotional tone to the device.

[1424] Step 6:

[1425] The terminal displays the searched reviews to the user.

[1426] Specific examples

[1427] When a user posts a review about a refrigerator, the emotion engine also analyzes the happy facial expression they express.

[1428] The server stores the review and positive sentiment in a database, and when another user visits the refrigerator's product page, it is displayed on their device as a positive sentiment review.

[1429] In this way, the invention that combines the emotion engine can further personalize the consumer's purchasing experience and provide detailed emotional support.

[1430] Example 2

[1431] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1432] Conventional shopping systems typically search for and recommend products based on consumer input, but lack advanced personalization that takes into account the consumer's emotional information. This makes it difficult to achieve the high relevance and satisfaction desired by consumers. Furthermore, even when recommending products based on events or the consumer's past history, further accuracy could be improved by utilizing emotional information, but such functionality is currently lacking.

[1433] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for receiving and analyzing text or images input by a consumer, means for collecting and analyzing emotional information, and means for searching a database for related products based on the analysis results. This enables highly personalized and highly relevant product searches that take into account the consumer's emotional information. Furthermore, by including means for recommending related products based on an event and means for generating recommendation results taking into account the consumer's emotional information, highly accurate event-based product recommendations can be realized. In addition, by including means for generating personalized product proposals based on the consumer's past history and means for generating recommendation results taking into account the consumer's emotional information, consumer satisfaction can be further increased.

[1434] "Consumer" refers to an individual who purchases or uses a product or service.

[1435] "Input text or image" refers to text data or image data provided by a consumer through an application or website interface.

[1436] "Means of analysis" refers to algorithms or programs that understand input data and extract meaning and features.

[1437] "Emotional information" refers to data about emotions extracted from consumers' facial expressions, tone of voice, gestures, etc.

[1438] "Collect" refers to obtaining data using sensor devices or input devices.

[1439] "Means for analyzing" refers to a system that includes algorithms or programs for interpreting collected emotional information and identifying specific emotional states.

[1440] A "database" refers to a data storage system in which multiple product information items are stored in an organized manner and can be easily searched and referenced.

[1441] A "search method" refers to an algorithm or program that efficiently searches through information in a database and finds entries that match a condition.

[1442] "Related products" refers to products that are relevant to a consumer's interests or needs based on data entered by the consumer and analysis results.

[1443] An "event" refers to a specific date, time, or occasion, such as a birthday, anniversary, or sales event.

[1444] "Past history" refers to a record of a consumer's previous purchases and browsing.

[1445] "Personalized product recommendations" refers to a list of products and services that are specifically suggested to a consumer based on their individual interests, history, and even emotional information.

[1446] "Recommendation results" refers to a list of products selected by the system and presented to the consumer.

[1447] This invention relates to a system that personalizes consumer purchasing experiences and provides highly relevant information and products. In particular, by combining it with an emotion engine that analyzes consumer emotion information, a more advanced level of personalization is achieved.

[1448] Intelligent search function with emotion engine

[1449] Users can enter text or upload an image into the application's search bar, and the system analyzes the input, searches the database for relevant product information, and displays it. By utilizing an emotion engine, the system also takes the user's emotions into account when providing search results.

[1450] The server analyzes the user's input data using natural language processing (NLP) algorithms and images using image recognition algorithms. It also analyzes the user's emotions using an emotion engine. Based on the analysis results, the server searches a database to identify relevant products. The search results are sent to the device, which displays them to the user.

[1451] Specific examples

[1452] The user uploads an image of their refrigerator, which the device sends to the server. The server uses image recognition technology to identify the refrigerator model, and if it recognizes a positive emotion in the user's facial expression, it searches the database for information on the latest, more expensive, and highly rated model. The search results are sent to the device, and the information on the latest model is displayed to the user.

[1453] Example prompt sentence:

[1454] "Browse photos of the latest refrigerator models. Users seem happy."

[1455] Event-based gift recommendation engine

[1456] When a user enters a gift-related question in text, the system analyzes the event and target information and recommends gift items based on that information. This function can be combined with an emotion engine to make recommendations based on the consumer's emotions.

[1457] The server analyzes the user's question and identifies event target information such as "birthday" and "mother." It then analyzes the user's emotions using an emotion engine. Based on the analysis results, it uses an event-based gift recommendation engine to search for relevant products. The recommendation results are sent to the device, which then displays the gift products to the user.

[1458] Specific examples

[1459] If a user types in "What gift would you recommend for my mother's birthday?" and shows a confused expression, the server will recognize the user's indecision using its emotion engine and prioritize recommending reliable and popular gift items. The recommended items are then sent to the device and displayed to the user.

[1460] Example prompt sentence:

[1461] "Users seem to be confused about finding the perfect gift for their mother's birthday."

[1462] Personalized Shopping Experience

[1463] When a user logs in to the application, the system will suggest products tailored to their individual preferences based on their past purchase and browsing history. Adding an emotion engine to this personalization function will enable even more advanced personalization.

[1464] The server analyzes the received history data and emotional data to identify the user's preferences and interests. Based on the analysis results, the server generates personalized product suggestions that take into account the emotional information and sends the list of suggested products to the terminal. The terminal then displays the list of suggested products to the user.

[1465] Specific examples

[1466] When a user logs in to the app, the emotion engine recognizes their happy facial expression. The server takes this happy emotion into account along with their past purchase history to suggest new products that the user is likely to enjoy. The list of suggestions is sent to the device and displayed to the user.

[1467] Example prompt sentence:

[1468] "Suggest new products to users who are in a good mood. Also consider past purchase history."

[1469] Purchasing support function

[1470] When a user asks a question about a specific product, the system provides a quick answer using the FAQ section or an AI chatbot. The emotion engine also analyzes emotions, allowing the system to provide a more accurate answer based on the user's emotions.

[1471] The server analyzes the question and recognizes the user's emotions using an emotion engine. Based on the analysis results, it generates an appropriate answer using an FAQ section or an AI chatbot and sends the answer to the device. The device then displays the answer to the user.

[1472] Specific examples

[1473] If a user asks about the warranty period for a refrigerator and shows a worried expression, the server will recognize the user's anxiety using its emotion engine, provide a detailed explanation of the warranty, and provide information on related reliable products. The answer will be sent to the terminal and displayed to the user.

[1474] Example prompt sentence:

[1475] "We will inform anxious users about the warranty period of the refrigerator and also suggest related trusted products."

[1476] Community Features

[1477] Users can post reviews and opinions about products, which can then be viewed by other users. By combining this feature with an emotion engine, it is possible to understand the emotional tone of the reviews.

[1478] The server stores the reviews and emotional information in a database, and when another user visits a specific product page, it provides review results that also reflect the emotional tone.The terminal displays the reviews to the user.

[1479] Specific examples

[1480] When a user posts a review about a refrigerator, the emotion engine analyzes the happy facial expression. The server stores the review and positive emotion in a database. When another user visits the refrigerator's product page, the review is displayed on their device as a positive emotion review.

[1481] Example prompt sentence:

[1482] "We store users' positive reviews in a database and make them available to other users."

[1483] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1484] Intelligent search function with emotion engine

[1485] Program processing

[1486] Step 1:

[1487] A user enters text into the app's search bar or uploads an image, and the input data is captured on the device.

[1488] Input: Text entered into the search bar or an uploaded image

[1489] Output: Input data is stored in the terminal

[1490] Step 2:

[1491] The device collects the user's facial expressions and tone of voice using a camera and microphone, and the collected emotional information is stored as data on the device.

[1492] Input: User's facial expression, tone of voice

[1493] Output: Emotional information data (facial expression recognition results, voice tone analysis results)

[1494] Step 3:

[1495] The device sends the input data and emotion information to the server, where they are packaged as data packets.

[1496] Input: Input data and emotion information data

[1497] Output: sent as data packets to the server

[1498] Step 4:

[1499] The server analyzes the input data: for text input, it uses natural language processing (NLP) algorithms; for images, it uses image recognition algorithms.

[1500] Input: User-entered data (text or image)

[1501] Output: Analysis results (e.g., text semantic analysis results, image recognition results)

[1502] Step 5:

[1503] The server analyzes the user's emotional information using an emotion engine, and the analysis results include specific emotions (e.g., positive, negative, etc.).

[1504] Input: Emotional information data

[1505] Output: Sentiment analysis result (e.g., positive, negative)

[1506] Step 6:

[1507] The server searches a product database based on the analysis results, and highly relevant product information is identified.

[1508] Input: Text / image analysis results, sentiment analysis results

[1509] Output: Related product information

[1510] Step 7:

[1511] The server sends the search results to the device in an appropriate format (e.g., JSON format).

[1512] Input: Related product information

[1513] Output: Related product information sent

[1514] Step 8:

[1515] The terminal displays the search results to the user using a graphical user interface (GUI).

[1516] Input: Related product information sent from the server

[1517] Output: Product information is displayed visually to the user

[1518] Event-based gift recommendation engine

[1519] Program processing

[1520] Step 1:

[1521] The user enters a question about the gift in text, and the input data is stored on the device.

[1522] Input: Text question (e.g., "What would you recommend for my mother's birthday?")

[1523] Output: Input data is stored in the terminal

[1524] Step 2:

[1525] The device collects the user's emotional information through the camera and microphone, and the collected data is stored on the device.

[1526] Input: User's facial expression, tone of voice

[1527] Output: Emotional information data (facial expression recognition results, voice tone analysis results)

[1528] Step 3:

[1529] The device sends the question and emotion information to the server. The data is sent in packets.

[1530] Input: Text questions, emotion information data

[1531] Output: sent as data packets to the server

[1532] Step 4:

[1533] The server parses the text question and extracts event target information such as "birthday" or "mother."

[1534] Input: Text question

[1535] Output: Analysis results (event and target information)

[1536] Step 5:

[1537] The server analyzes the user's emotional information using an emotion engine, and the analysis results include specific emotions.

[1538] Input: Emotional information data

[1539] Output: Emotion analysis results

[1540] Step 6:

[1541] Based on the analysis results, the server searches for related products using an event-based gift recommendation engine.

[1542] Input: Event and target information, emotion analysis results

[1543] Output: Relevant gift product information

[1544] Step 7:

[1545] The server sends the recommendation results to the device in an appropriate format (e.g., JSON format).

[1546] Input: Related gift product information

[1547] Output: Gift item information sent

[1548] Step 8:

[1549] The device displays the recommended gift items to the user through a GUI.

[1550] Input: Gift item information sent from the server

[1551] Output: Product information is displayed visually to the user

[1552] Personalized Shopping Experience

[1553] Program processing

[1554] Step 1:

[1555] A user logs in to the app, and their authentication data is stored on the device.

[1556] Input: User ID, Password

[1557] Output: Authentication result and user login history

[1558] Step 2:

[1559] The device uses the camera and microphone to collect emotional information such as facial expressions and tone of voice when logging in. The collected data is stored on the device.

[1560] Input: User's facial expression, tone of voice

[1561] Output: Emotional information data (facial expression recognition results, voice tone analysis results)

[1562] Step 3:

[1563] The device sends past purchase history, browsing history, and even emotional information to a server. The data is sent in packets.

[1564] Input: purchase history, browsing history, emotional information data

[1565] Output: sent as data packets to the server

[1566] Step 4:

[1567] The server analyzes the received historical and emotional data to identify the user's preferences and interests.

[1568] Input: history data, emotion data

[1569] Output: Preference analysis results

[1570] Step 5:

[1571] Based on the analysis results, the server generates personalized product suggestions that take emotional information into consideration.

[1572] Input: Preference analysis results, emotional information

[1573] Output: Personalized product recommendations

[1574] Step 6:

[1575] The list of suggested products is sent to the terminal, and the data is in the appropriate format.

[1576] Input: Personalized product suggestions

[1577] Output: Product suggestion information sent

[1578] Step 7:

[1579] The terminal displays a list of suggested products to the user through a GUI.

[1580] Input: Product suggestion information sent from the server

[1581] Output: A list of product suggestions is visually displayed to the user

[1582] Purchasing support function

[1583] Program processing

[1584] Step 1:

[1585] The user enters a question about a specific product, and the input data is saved on the device.

[1586] Input: Text question (e.g., "How long is the warranty on this refrigerator?")

[1587] Output: Input data is stored in the terminal

[1588] Step 2:

[1589] The device collects the user's emotional information through the camera and microphone, and the collected data is stored on the device.

[1590] Input: User's facial expression, tone of voice

[1591] Output: Emotional information data (facial expression recognition results, voice tone analysis results)

[1592] Step 3:

[1593] The device sends the question and emotion information to the server. The data is sent in packets.

[1594] Input: Text questions, emotion information data

[1595] Output: sent as data packets to the server

[1596] Step 4:

[1597] The server analyzes the question text and determines the meaning of the question.

[1598] Input: Text question

[1599] Output: Question analysis results (e.g. warranty period extraction)

[1600] Step 5:

[1601] The server analyzes the user's emotional information using an emotion engine, and the analysis results include specific emotions.

[1602] Input: Emotional information data

[1603] Output: Emotion analysis results (e.g., anxiety recognition)

[1604] Step 6:

[1605] The server utilizes a FAQ section and AI chatbot to generate appropriate answers.

[1606] Input: Question analysis results, sentiment analysis results

[1607] Output: Generated answer (e.g. warranty period details)

[1608] Step 7:

[1609] The generated response is sent to the device, with the data in the appropriate format.

[1610] Input: Generated Answer

[1611] Output: Submitted response

[1612] Step 8:

[1613] The terminal displays the answers to the user through a GUI.

[1614] Input: The answer sent by the server

[1615] Output: The answer is displayed visually to the user

[1616] Community Features

[1617] Program processing

[1618] Step 1:

[1619] Users input reviews and opinions about products. The input data is saved on the device.

[1620] Input: Review or opinion text

[1621] Output: Input data is stored in the terminal

[1622] Step 2:

[1623] The device collects the user's emotional information through the camera and microphone, and the collected data is stored on the device.

[1624] Input: User's facial expression, tone of voice

[1625] Output: Emotional information data (facial expression recognition results, voice tone analysis results)

[1626] Step 3:

[1627] The device sends reviews and sentiment information to the server in the form of packets.

[1628] Input: Review text and sentiment data

[1629] Output: sent as data packets to the server

[1630] Step 4:

[1631] The server stores the reviews and sentiment information in a database, and the data is stored in a suitable format.

[1632] Input: Review text, emotional information data

[1633] Output: Saved data

[1634] Step 5:

[1635] When another user visits a particular product page, the server searches the database for reviews of that product, and the results reflect the emotional tone of the reviews.

[1636] Input: Product page visit request

[1637] Output: Search results (reviews and sentiment tone)

[1638] Step 6:

[1639] The server sends the search results to the device in the appropriate format.

[1640] Input: Search results

[1641] Output: Submitted search results

[1642] Step 7:

[1643] The device displays the reviews to the user through a GUI.

[1644] Input: Reviews sent from the server

[1645] Output: The review is displayed visually to the user

[1646] (Application example 2)

[1647] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1648] Conventional purchasing experience systems suggest related products by analyzing text and images entered by consumers, but do not take into account consumer circumstances such as emotions and behavior, which limits the degree of personalization. Furthermore, recommendations based solely on events or purchase history make it difficult to accurately grasp consumer needs in real time, posing challenges to improving consumer satisfaction.

[1649] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving and analyzing text or images entered by a consumer, means for searching a database for related products based on the analysis results, means for displaying the search results to the consumer, means for collecting the consumer's facial expressions, movements, and voice using cameras and sensors in the store, means for analyzing the collected data to recognize the consumer's emotions, and means for suggesting related products based on the recognized emotions. This enables more advanced and precise personalized product suggestions that take into account the consumer's emotions and real-time situation.

[1650] "Consumer" refers to any member of the public who intends to purchase a service or product.

[1651] "Input text or image" refers to written or visual information provided by the consumer to the system.

[1652] "Means of analysis" refers to the technology that understands input text or images and extracts information based on them.

[1653] "Means for searching related products from a database" refers to technology that searches for appropriate product information from a database based on the analysis results.

[1654] "Means for displaying search results to consumers" refers to the technology for displaying information retrieved from a database on a device used by a consumer.

[1655] "Cameras and sensors in the store" refers to monitoring and sensing devices used to capture consumer movements, facial expressions, and voices within the store.

[1656] "Means of collection" refers to technology that uses cameras and sensors to acquire data such as consumer movements, facial expressions, and voice.

[1657] "Means for recognizing consumer emotions by analyzing collected data" refers to technology that analyzes acquired data and identifies the emotions consumers are feeling.

[1658] "Means for suggesting related products based on recognized emotions" refers to technology that selects and suggests products suitable for consumers based on the results of emotion analysis.

[1659] This invention is a system for improving consumer purchasing experiences, and aims to analyze consumer sentiment and provide personalized product recommendations, particularly in brick-and-mortar stores. This system is implemented based on the following configuration and processing steps.

[1660] System configuration

[1661] 1. Hardware Configuration

[1662] Cameras and sensors: These are installed in physical stores to collect consumers' facial expressions, movements, and voices. For example, we use standard surveillance cameras and voice recognition sensors.

[1663] Devices: Consumers' smartphones, smart glasses, head-mounted displays (HMDs), etc. These devices send collected data to a server and display information from the server to the consumer.

[1664] Server: A server for data analysis and product proposals.

[1665] 2. Software Configuration

[1666] Natural Language Processing (NLP): Analyzing consumer-supplied text or speech, for example using common natural language processing algorithms.

[1667] Image recognition algorithm: Analyzes images uploaded by consumers and identifies related products. As a concrete example, we will use a common image recognition algorithm.

[1668] Emotion engine: Recognizes consumer emotions based on collected facial and voice data. As a concrete example, a general emotion recognition algorithm is used.

[1669] Database: A database for storing product information. Specific examples include SQL databases and NoSQL databases.

[1670] Processing flow

[1671] 1. Data Collection

[1672] Cameras and sensors are used to collect the consumer's facial expressions, movements, and voice, and the data is sent to a server via the device.

[1673] 2. Data Analysis

[1674] The data received by the server is analyzed using the following software:

[1675] Natural Language Processing Algorithms

[1676] Image Recognition Algorithm

[1677] Emotion Engine

[1678] 3. Product proposal

[1679] The server searches the product database based on the analysis results, identifies related products, and sends them to the terminal, which then displays the obtained product information to the consumer.

[1680] Specific examples

[1681] Scenario 1: Entering the store

[1682] A consumer enters a store and uses a device (e.g., smart glasses) to navigate the store. When the camera captures the consumer's interested expression, the emotion engine recognizes the interest and the server sends a guide message based on that, showing new product sections and recommended products.

[1683] Scenario 2: Product selection support

[1684] If a consumer appears to be unsure about a particular product, the camera captures the situation and the emotion engine recognizes the emotion of "indecision." Based on this, the server suggests other highly rated products and related articles, and displays this information on the device.

[1685] Prompt sentence for generative AI model

[1686] A customer enters a physical store and uses a smart device. The in-store camera captures the customer's facial expressions and behavior. If the customer shows interest, output a product suggestion method based on that interest. Also, explain how to suggest alternative products if the customer is unsure.

[1687] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1688] Step 1:

[1689] A user enters a store and uses a device such as a smartphone, smart glasses, or head-mounted display (HMD). Cameras and sensors collect the user's facial expressions, movements, and voice. This data is then input into the device.

[1690] Step 2:

[1691] The facial expression, movement, and voice data collected by the device are sent to a server in real time. Data input includes image data and voice data.

[1692] Step 3:

[1693] The server analyzes the received data. Specifically, it performs the following processes:

[1694] Natural language processing (NLP) algorithms analyze voice data and user-entered text.

[1695] Image recognition algorithms analyze the collected image data to identify product categories of interest.

[1696] The emotion engine analyzes facial and voice data to identify the user's emotions (e.g., interest, uncertainty, anxiety).

[1697] Step 4:

[1698] The server combines the user's emotional information obtained as a result of the analysis with the results of NLP and image recognition to search for related products in a database. The input includes emotional data, voice and text data, and image data, and generates a list of related products as an output.

[1699] Step 5:

[1700] The server generates a related product list and sends it to the device. The device receives it and displays it to the user. The displayed product list is updated in real time based on the user's facial expressions and movements.

[1701] Step 6:

[1702] The user can review the displayed product list and request more information about products they are interested in. Inputs at the device include touch and voice commands, and the details are again requested from the server.

[1703] Step 7:

[1704] The server retrieves detailed information from the database based on the user's request and sends it to the terminal, which then displays the information to the user, allowing the user to view more detailed product information.

[1705] Through the above steps, real-time personalized product suggestions that take user emotions into consideration are realized.

[1706] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1707] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1708] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1709] [Third embodiment]

[1710] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1711] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1712] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1713] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1714] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1715] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1716] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1717] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1718] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1719] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1720] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1721] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1722] The present invention relates to a system for personalizing a consumer's purchasing experience and providing highly relevant information and products. A specific embodiment of this system will be described below.

[1723] Intelligent Search Function

[1724] Users can enter text or upload an image into the search bar, and the system will analyze the input and search the database for relevant product information to display. This intelligent search function allows consumers to efficiently obtain the product information they are looking for.

[1725] Program processing

[1726] The user types "What's the latest version of the refrigerator in this picture?" into the app's search bar, or uploads an image of the refrigerator. The device sends the input data to the server, which uses natural language processing (NLP) algorithms to analyze the text and understand the user's intent. In the case of an image, the server uses image recognition algorithms to extract product features. Based on the analysis, the server searches a product database to identify relevant products. The search results are sent to the device, which displays them to the user.

[1727] Specific examples

[1728] The user uploads an image of their refrigerator, and the device sends the image to the server. The server uses image recognition technology to identify the refrigerator model and searches the database for the latest information. The search results are sent to the device, and the latest model information is displayed to the user.

[1729] Event-based gift recommendation engine

[1730] Users can enter gift-related questions via text, and the system will analyze the event and target information and recommend gift items based on that information, allowing consumers to find the perfect gift for a special event or occasion.

[1731] Program processing

[1732] The user types, "What gift would you recommend for my mother's birthday?" The device sends the question to the server. The server analyzes the question and extracts the event target information "birthday" and "mother." The server uses an event-based gift recommendation engine to search for relevant products and sends the recommendation results to the device. The device displays the recommended gift items to the user.

[1733] Specific examples

[1734] When a user types "What would you recommend for my mother's birthday?", the device sends the question to the server. The server analyzes the "birthday" and "mother" and recommends gift items based on that. The recommendation results are sent to the device, and suitable gift candidates are displayed to the user.

[1735] Personalized Shopping Experience

[1736] When a user logs in to the app, the system will suggest products tailored to their individual preferences based on their past purchase and browsing history. This personalization feature allows users to quickly search for product information based on their interests and preferences.

[1737] Program processing

[1738] A user logs in to the app. The device sends the user's past purchase and browsing history to the server. The server analyzes the user's historical data and identifies individual preferences and interests. The server generates personalized product suggestions based on the analysis results and sends a list of suggested products to the device. The device displays the list of suggested products to the user.

[1739] Specific examples

[1740] When a user logs in to the app, their device sends their past purchase history to the server. The server analyzes the data and suggests products that match the user's preferences. A list of suggested products is sent to the device and displayed to the user.

[1741] Purchasing support function

[1742] When users ask questions about specific products, the system provides quick answers using the FAQ section and AI chatbots, helping users to resolve any doubts or concerns they may have about their purchase.

[1743] Program processing

[1744] The user types a question, such as "How long is the warranty on this refrigerator?" The device sends the question to the server, which uses an FAQ section or an AI chatbot to generate an appropriate answer to the question. The generated answer is sent to the device, which then displays the appropriate answer to the user.

[1745] Specific examples

[1746] When a user asks about the warranty period of a refrigerator, the device sends the question to the server, which parses the question and retrieves the warranty period information from the FAQ section. The retrieved information is sent to the device and displayed to the user.

[1747] Community Features

[1748] Users can post reviews and opinions about products, and other users can view that information. This feature allows consumers to make purchasing decisions while also taking into account the opinions of other users.

[1749] Program processing

[1750] The user enters a review or opinion on a product. The device sends the review to the server. The server stores the received review in a database. When another user visits a specific product page, the server searches the database for reviews of that product and sends the search results to the device. The device displays the review to the user.

[1751] Specific examples

[1752] A user posts a review about a refrigerator, and the device sends it to the server. The server stores the review in a database. When another user visits the refrigerator's product page, the server retrieves the stored review and sends it to the device. The device displays the review to the user.

[1753] In this way, the system according to the claims can enhance the consumer's buying experience and provide personalized services.

[1754] The processing flow will be explained below.

[1755] Intelligent Search Function

[1756] Program processing

[1757] Step 1:

[1758] A user types "What's the latest version of the refrigerator in this picture?" into the app's search bar or uploads an image of the refrigerator.

[1759] Step 2:

[1760] The device sends the entered text and images to the server.

[1761] Step 3:

[1762] The server analyzes the submitted text using natural language processing (NLP) algorithms to understand the user's intent.

[1763] Step 4:

[1764] The server analyzes the transmitted image using an image recognition algorithm and extracts its features.

[1765] Step 5:

[1766] The server searches a product database based on the analysis results to identify highly relevant products.

[1767] Step 6:

[1768] The server transmits the search results to the terminal.

[1769] Step 7:

[1770] The terminal displays the search results to the user.

[1771] Event-based gift recommendation engine

[1772] Program processing

[1773] Step 1:

[1774] The user texts in, "What do you recommend for my mom's birthday?"

[1775] Step 2:

[1776] The terminal sends the text to the server.

[1777] Step 3:

[1778] The server analyzes the submitted text using natural language processing (NLP) to identify the event "birthday" and the target "mother."

[1779] Step 4:

[1780] The server uses an event-based gift recommendation engine to search for relevant gift items.

[1781] Step 5:

[1782] The server transmits the recommendation results to the terminal.

[1783] Step 6:

[1784] The terminal displays a list of recommended products to the user.

[1785] Personalized Shopping Experience

[1786] Program processing

[1787] Step 1:

[1788] The user logs in to the app.

[1789] Step 2:

[1790] The terminal sends the logged-in user's past purchase history and browsing history to the server and loads that data.

[1791] Step 3:

[1792] The server analyzes the received historical data to identify the user's preferences and interests.

[1793] Step 4:

[1794] The server generates personalized product suggestions based on the analysis results.

[1795] Step 5:

[1796] The server transmits the generated list of suggested products to the terminal.

[1797] Step 6:

[1798] The terminal displays a list of suggested products to the user.

[1799] Purchasing support function

[1800] Program processing

[1801] Step 1:

[1802] A user types a question: "How long is the warranty on this refrigerator?"

[1803] Step 2:

[1804] The terminal transmits the entered question to the server.

[1805] Step 3:

[1806] The server analyzes the submitted questions and uses an FAQ section or AI chatbot to generate appropriate answers.

[1807] Step 4:

[1808] The server sends the generated answer to the terminal.

[1809] Step 5:

[1810] The terminal displays the generated answer to the user.

[1811] Community Features

[1812] Program processing

[1813] Step 1:

[1814] The user enters reviews and opinions about the purchased product.

[1815] Step 2:

[1816] The device sends the entered reviews and opinions to the server.

[1817] Step 3:

[1818] The server stores the received reviews in a database.

[1819] Step 4:

[1820] When another user visits a particular product page, the server searches the database for reviews of that product.

[1821] Step 5:

[1822] The server sends the review of the search results to the terminal.

[1823] Step 6:

[1824] The terminal displays the searched reviews to the user.

[1825] Example 1

[1826] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1827] Conventional online shopping systems make it difficult for consumers to efficiently find the best products for them from the vast amount of product information available, resulting in a cumbersome shopping experience. Furthermore, product recommendations based on special events or individual preferences are not adequately implemented, which can lead to low consumer satisfaction. Furthermore, the inability to easily obtain detailed product information or reviews can make it difficult to resolve concerns or questions about purchasing.

[1828] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1829] In this invention, the server includes a means for analyzing text or images entered by a consumer, a means for analyzing the text using a natural language processing algorithm to understand the user's intent, a means for analyzing the image using an image recognition algorithm to extract product features, a means for searching a database for related products based on the analysis results, and a means for displaying the search results to the consumer. This allows consumers to efficiently find the most suitable product. In addition, personalized recommendations based on events and past history improve consumer satisfaction and provide a comfortable shopping experience.

[1830] "Consumer" refers to a general user of the system for the purpose of purchasing goods or services.

[1831] "Text" refers to sentences or characters entered by consumers, and is textual information that is understood by the system.

[1832] "Images" refers to visual information such as photographs and illustrations uploaded by consumers.

[1833] "Means of analysis" refers to methods and techniques for processing input text or images with a computer program and understanding their content.

[1834] A "natural language processing algorithm" refers to a computational method for analyzing text and understanding the meaning and intent of human language.

[1835] "Means for understanding user intent" refers to methods that use natural language processing algorithms to analyze the meaning and purpose of text entered by a user.

[1836] An "image recognition algorithm" refers to a computational method for analyzing an image and identifying the objects and features contained within it.

[1837] "Means for extracting product features" refers to the use of image recognition algorithms to identify specific products and their attributes from uploaded images.

[1838] "Related Products" refers to products and services available for purchase that are suggested based on information entered or uploaded by the consumer.

[1839] A "database" refers to a collection of information that systematically stores related products and other information and makes it easy to search and retrieve.

[1840] "Searching means" refers to the methods and technologies used to search the database based on the analysis results and find relevant product information.

[1841] "Means for displaying search results to consumers" refers to methods and technologies for displaying searched product information on a user interface so that consumers can check it.

[1842] "Event" refers to a specific situation or occasion, such as a birthday or anniversary, and is the criterion for recommending products related to that situation.

[1843] "Personalized product suggestions" refers to a method of recommending products that match a consumer's individual preferences based on individual data such as their past purchase history and browsing history.

[1844] The present invention relates to a system for personalizing a consumer's shopping experience and providing highly relevant information and products. The system has multiple functions for analyzing user-entered text and images and efficiently providing relevant product information.

[1845] Intelligent Search Function

[1846] Users can enter text or upload an image into the search bar, and the system will analyze the input and search the database for relevant product information to display. This intelligent search function allows consumers to efficiently obtain the product information they are looking for.

[1847] Hardware and software used

[1848] Server: A computer with high processing power (e.g., a Linux server)

[1849] Terminal: The device that provides the user interface (e.g., smartphone, tablet, PC)

[1850] Natural Language Processing Algorithms: Google Cloud Natural Language API

[1851] Image recognition algorithm: Amazon Rekognition

[1852] Database: MySQL

[1853] Specific actions

[1854] The user types the text "What is the latest version of the refrigerator in this image?" into the app's search bar or uploads an image of the refrigerator.

[1855] The terminal sends the input data to the server.

[1856] The server uses the Google Cloud Natural Language API to analyze the text and understand the user's intent.

[1857] In the case of images, the server uses Amazon Rekognition to perform image recognition and extract the product's features.

[1858] The server searches a MySQL database based on the analysis results to identify relevant products.

[1859] The search results are sent to the terminal, which displays the results to the user.

[1860] Event-based gift recommendation engine

[1861] Users can enter gift-related questions via text, and the system will analyze the event and target information and recommend gift items based on that information, allowing consumers to find the perfect gift for a special event or occasion.

[1862] Specific actions

[1863] User types, "What do you recommend for my mom's birthday?"

[1864] The terminal sends a question to the server.

[1865] The server parses the question using the Google Cloud Natural Language API and extracts the event target information "birthday" and "mother."

[1866] The server uses an event-based gift recommendation engine to search for relevant products, and the recommendation results are sent to the terminal.

[1867] The terminal displays the recommended gift items to the user.

[1868] Personalized Shopping Experience

[1869] When a user logs in to the app, the system will suggest products tailored to their individual preferences based on their past purchase and browsing history. This personalization feature allows users to quickly search for product information based on their interests and preferences.

[1870] Specific actions

[1871] A user logs in to the app.

[1872] The device sends the user's past purchase history and browsing history to the server.

[1873] The server analyzes the received historical data to identify the user's preferences and interests.

[1874] The server generates personalized product suggestions based on the analysis results, and a list of suggested products is sent to the terminal.

[1875] The terminal displays a list of suggested products to the user.

[1876] Purchasing support function

[1877] When users ask questions about specific products, the system provides quick answers using the FAQ section and AI chatbots, helping users to resolve any doubts or concerns they may have about their purchase.

[1878] Specific actions

[1879] A user types a question: "How long is the warranty on this refrigerator?"

[1880] The terminal sends a question to the server.

[1881] The server uses Dialogflow to analyze the question and generate an appropriate answer from the FAQ database.

[1882] The generated answer is sent to the terminal, which displays the appropriate answer to the user.

[1883] Community Features

[1884] Users can post reviews and opinions about products, and other users can view that information. This feature allows consumers to make purchasing decisions while also taking into account the opinions of other users.

[1885] Specific actions

[1886] Users enter product reviews and opinions.

[1887] The device sends the review to the server.

[1888] The server stores the received reviews in a MySQL database.

[1889] When another user visits a specific product page, the server searches the database for reviews of that product and sends the search results to the terminal.

[1890] The device displays the review to the user.

[1891] With the above functions, the system of the present invention can improve the consumer's purchasing experience and provide personalized services.

[1892] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1893] Intelligent Search Function

[1894] Step 1:

[1895] A user types the text "What is the latest version of the refrigerator in this image?" into the search bar or uploads an image of a refrigerator.

[1896] Input: Text or image from the user

[1897] Output: Text or image sent to the terminal

[1898] Step 2:

[1899] The terminal sends the input data to the server.

[1900] Input: Text or Image

[1901] Output: HTTP request sent to the server

[1902] Step 3:

[1903] The server branches the process depending on the input data. For text, the server analyzes the text using the Google Cloud Natural Language API to understand the user's intent. For images, the server extracts features using Amazon Rekognition.

[1904] Input: Text or image sent to the server

[1905] Output: Analysis results (user intent for text, feature data for images)

[1906] Step 4:

[1907] Based on the analysis results, the server searches the MySQL database to identify relevant products.

[1908] Specifically, an SQL query is generated based on the extracted keywords and features and issued to the database.

[1909] Input: Analysis results (user intent or feature data)

[1910] Output: Related product information

[1911] Step 5:

[1912] The server formats the search results in JSON format and sends them to the terminal as an HTTP response.

[1913] Input: Relevant product information

[1914] Output: Search results in JSON format

[1915] Step 6:

[1916] The terminal parses the received search results and displays them on the user interface.

[1917] Input: JSON format search results

[1918] Output: Product information displayed in the user interface

[1919] Event-based gift recommendation engine

[1920] Step 1:

[1921] User types, "What do you recommend for my mom's birthday?"

[1922] Input: User question text

[1923] Output: Question data sent to the terminal

[1924] Step 2:

[1925] The terminal transmits the question data to the server.

[1926] Input: Question data

[1927] Output: HTTP request sent to the server

[1928] Step 3:

[1929] The server parses the question using the Google Cloud Natural Language API and extracts the event target information "birthday" and "mother."

[1930] Input: Query data sent to the server

[1931] Output: Extracted event and target information

[1932] Step 4:

[1933] The server searches for related products using a gift recommendation engine based on the event information and target information, and obtains recommendation results.

[1934] Input: Event information, target information

[1935] Output: Gift recommendation results

[1936] Step 5:

[1937] The server formats the recommendation results in JSON format and sends them to the terminal as an HTTP response.

[1938] Input: Gift recommendation results

[1939] Output: Recommendation results in JSON format

[1940] Step 6:

[1941] The device parses the received recommendation results and displays them on the user interface.

[1942] Input: Recommendation results in JSON format

[1943] Output: Gift information displayed in the user interface

[1944] Personalized Shopping Experience

[1945] Step 1:

[1946] A user logs in to the app.

[1947] Input: Username and Password

[1948] Output: Login request

[1949] Step 2:

[1950] The terminal transmits the user's past history data to the server.

[1951] Input: Historical data

[1952] Output: HTTP request sent to the server

[1953] Step 3:

[1954] The server analyzes the historical data to identify the user's preferences and interests.

[1955] Input: Historical data sent to the server

[1956] Output: Analysis results (user preferences and interests)

[1957] Step 4:

[1958] The server generates personalized product suggestions based on the analysis results using a Recommendation Engine.

[1959] Input: Analysis results (user preferences and interests)

[1960] Output: Personalized product recommendations

[1961] Step 5:

[1962] The server formats the list of suggested products in JSON format and sends it to the terminal as an HTTP response.

[1963] Input: Product suggestion list

[1964] Output: A list of suggestions in JSON format

[1965] Step 6:

[1966] The terminal parses the received proposal list and displays it on the user interface.

[1967] Input: A list of suggestions in JSON format

[1968] Output: A list of suggested products displayed in the user interface

[1969] Purchasing support function

[1970] Step 1:

[1971] A user enters a question about a particular product.

[1972] Input: Question text

[1973] Output: Question data sent to the terminal

[1974] Step 2:

[1975] The terminal sends a question to the server.

[1976] Input: Question data

[1977] Output: HTTP request sent to the server

[1978] Step 3:

[1979] The server uses Dialogflow to analyze the question and generate an appropriate answer.

[1980] Input: Query data sent to the server

[1981] Output: Correct answer data

[1982] Step 4:

[1983] The server searches the FAQ database and retrieves the corresponding information.

[1984] Input: Question data

[1985] Output: FAQ information

[1986] Step 5:

[1987] The server formats the acquired FAQ information into JSON format and sends it to the terminal as an HTTP response.

[1988] Input: FAQ information

[1989] Output: JSON formatted response data

[1990] Step 6:

[1991] The terminal parses the received response data and displays it on the user interface.

[1992] Input: JSON formatted response data

[1993] Output: Answer information displayed in the user interface

[1994] Community Features

[1995] Step 1:

[1996] Users enter product reviews and opinions.

[1997] Input: Review or opinion text

[1998] Output: Review sent to device

[1999] Step 2:

[2000] The device sends the review to the server.

[2001] Input: Review text

[2002] Output: HTTP request sent to the server

[2003] Step 3:

[2004] The server stores the received reviews in a MySQL database.

[2005] Input: The review sent to the server

[2006] Output: Reviews stored in a database

[2007] Step 4:

[2008] Another user visits a specific product page.

[2009] Input: Product page request

[2010] Output: HTTP request to the server

[2011] Step 5:

[2012] The server searches the database for product reviews.

[2013] Input: Product page request

[2014] Output: Review information as search results

[2015] Step 6:

[2016] The server formats the search results in JSON format and sends them to the terminal as an HTTP response.

[2017] Input: Review information as search results

[2018] Output: Review information in JSON format

[2019] Step 7:

[2020] The terminal parses the received review information and displays it on the user interface.

[2021] Input: JSON formatted review information

[2022] Output: Review information displayed in the user interface

[2023] (Application example 1)

[2024] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2025] Traditional purchasing experiences often only offer consumers a uniform selection of products, making it difficult to provide personalized suggestions based on individual consumer preferences and past history. Product searches using text and images also have low accuracy, preventing consumers from efficiently finding the products they are looking for. Furthermore, responses to consumer questions about specific products are often delayed, hindering purchasing decisions. Therefore, there is a need for a system that can more individually tailor the consumer purchasing experience and provide relevant information quickly and appropriately.

[2026] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2027] In this invention, the server includes means for receiving and analyzing text or images entered by a consumer, means for searching a database for related products based on the analysis results, means for displaying the search results to the consumer, means for analyzing purchase history and browsing history based on a consumer ID and generating personalized product suggestions, means including an integrated natural language processing algorithm and an image recognition algorithm, and means for generating appropriate answers to end-user questions using artificial intelligence, thereby enabling consumers to efficiently receive product suggestions that match their preferences and the information they are looking for.

[2028] A "consumer ID" is an identifier used to uniquely identify a consumer.

[2029] "Purchase history" is a record of products and services that a consumer has purchased in the past.

[2030] "Browser history" is a record of the products and pages a consumer has previously viewed on a website or app.

[2031] "Personalized product recommendations" are recommendations for products or services that are customized for a specific consumer based on that consumer's past behavior and preferences.

[2032] "Text analysis" is the process of analyzing input text data and understanding its meaning and content.

[2033] "Image analysis" is the process of analyzing input image data and recognizing objects and features within it.

[2034] A database is a collection of information that organizes and stores related data so that it can be quickly searched.

[2035] A "natural language processing algorithm" is a computational method for understanding and analyzing text data.

[2036] An "image recognition algorithm" is a computational method for analyzing image data and identifying specific objects or features.

[2037] "Artificial intelligence" is a technology that gives computers the ability to learn and reason like human intelligence.

[2038] The "means for generating an appropriate answer to a question" is a process for analyzing the question entered by the consumer and generating an answer based on related information.

[2039] "Related product search" is the process of locating related products from a database based on consumer input.

[2040] "Product proposal generation" is the process of making product proposals to individual consumers based on their past behavioral data.

[2041] A "display means" is a method or device that visually presents information, such as search results or suggested products, to consumers.

[2042] This invention is a system that allows consumers to input text or images through an interface, and then analyzes the input to provide related products. In particular, it has a personalized product suggestion function based on the consumer's purchase history and browsing history, and a question-answering function using artificial intelligence.

[2043] Program Overview

[2044] This system mainly consists of the following hardware and software:

[2045] 1. Hardware:

[2046] Server: AWS EC2

[2047] Database: Amazon RDS (MySQL)

[2048] User devices: smartphones and computers

[2049] 2. Software:

[2050] Natural Language Processing (NLP) Algorithm: Google's BERT Model

[2051] Image recognition algorithm: Google Cloud Vision API

[2052] Server-side program: Python (Flask)

[2053] System Operation Overview

[2054] Intelligent Search Function

[2055] A user enters text into the device's search bar or uploads an image. The device sends the input data to the server, which analyzes the text input with a natural language processing algorithm or the image input with an image recognition algorithm. The server then searches the database to identify relevant products. The search results are sent to the device and displayed to the user.

[2056] Personalized Shopping Experience

[2057] When a user logs in to the app, the device sends the user's ID to the server, which analyzes their purchase and browsing history to generate a personalized product recommendation list, which is then sent to the device and displayed to the user.

[2058] Purchasing support function

[2059] A user enters a question about a particular product. The device sends the question to the server, which uses natural language processing algorithms and an FAQ database to generate an appropriate answer. The answer is then sent to the device and displayed to the user.

[2060] Specific examples

[2061] Intelligent Search: The user uploads a picture of their refrigerator and sends it to the server. The server uses image recognition technology to identify the refrigerator model and retrieves the latest information from the database. The search results are sent to the device, and the latest model information is displayed to the user.

[2062] Personalized shopping experience: When a user logs into the app and a previously purchased smartwatch is recorded, the server suggests new smartwatch models and displays the list to the user.

[2063] Purchasing support function: When a user asks, "How long is the warranty period for this refrigerator?", the server refers to the FAQ database and provides the user with information about the warranty period.

[2064] Prompt Sentence Examples

[2065] If a user types "What is the latest iPhone model?", the database will be searched for and displayed as relevant latest iPhone models.

[2066] Input: What is the latest model of iPhone?

[2067] Output: The iPhone 13 is currently the latest model and comes with the latest features.

[2068] If you log in with user ID "12345", we will recommend new smartwatch models based on your past purchase history.

[2069] Input: I would like products recommended based on the purchasing history of user ID "12345".

[2070] Output: We recommend the new Apple Watch Series 6.

[2071] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2072] Intelligent Search Processing Steps

[2073] Step 1: User Input

[2074] The user enters text or uploads an image into the device's input interface.

[2075] Input: Text or image

[2076] Output: Text or image data

[2077] Step 2: Send data

[2078] The device sends input data (text or image) to the server using HTTP POST as the transmission protocol.

[2079] Input: Text or image data entered by the user

[2080] Output: Text or image data sent to the server

[2081] Step 3: Data analysis

[2082] The server analyzes the received data. For text data, it uses a natural language processing algorithm (Google's BERT) to analyze the intent, and for image data, it uses an image recognition algorithm (Google Cloud Vision API) to extract features.

[2083] Input: Text or image data sent to the server

[2084] Output: Text analysis results or image feature data

[2085] Step 4: Database Search

[2086] The server searches the product database based on the analysis results or extracted features and executes SQL queries to retrieve relevant product information.

[2087] Input: Text analysis results or image feature data

[2088] Output: Product information

[2089] Step 5: Submit search results

[2090] The server sends the search results to the device in JSON format.

[2091] Input: JSON data of search results

[2092] Output: Product information sent to the device

[2093] Step 6: View the results

[2094] The product information received by the terminal is displayed on a user interface.

[2095] Input: Product information sent to the terminal

[2096] Output: The product list displayed to the user

[2097] Processing steps for a personalized shopping experience

[2098] Step 1: User Login

[2099] The user enters their credentials to log in to the app.

[2100] Input: User ID and password

[2101] Output: Authentication token

[2102] Step 2: Send past history

[2103] The device sends the user ID to the server, which retrieves relevant purchase and browsing history from its internal database.

[2104] Input: User ID

[2105] Output: Purchase history and browsing history data

[2106] Step 3: Historical data analysis

[2107] The server analyzes the historical data and generates a personalized product list.

[2108] Input: Purchase history and browsing history data

[2109] Output: Personalized product list

[2110] Step 4: Submit your proposal list

[2111] The server transmits the generated product proposal list to the terminal.

[2112] Input: Personalized Product List

[2113] Output: Suggestion list sent to the device

[2114] Step 5: List View

[2115] The terminal displays the product suggestion list on the user interface.

[2116] Input: Suggestion list sent to the device

[2117] Output: A list of suggested products displayed to the user

[2118] Purchasing Support Function Processing Steps

[2119] Step 1: Enter your question

[2120] The user inputs a question about the product they want to purchase.

[2121] Input: Question text

[2122] Output: The question text entered

[2123] Step 2: Submit your question

[2124] The terminal sends the question text to the server.

[2125] Input: The entered question text

[2126] Output: The question text sent to the server

[2127] Step 3: Question analysis

[2128] The server analyzes the question text using natural language processing algorithms.

[2129] Input: Question text sent to the server

[2130] Output: Question analysis results

[2131] Step 4: Generate appropriate answers

[2132] The server searches the FAQ database for relevant answers and generates an appropriate answer.

[2133] Input: Question analysis results

[2134] Output: Correct answer

[2135] Step 5: Submit your response

[2136] The server sends the generated response to the terminal.

[2137] Input: Correct Answer

[2138] Output: Answer sent to the terminal

[2139] Step 6: View Answers

[2140] The terminal displays the generated answer on a user interface.

[2141] Input: Answer sent to the terminal

[2142] Output: The answer that is displayed to the user

[2143] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2144] This invention relates to a system that personalizes the consumer's purchasing experience and provides highly relevant information and products. In particular, by combining it with an emotion engine that recognizes the user's emotions, a higher level of personalization is achieved. Specific embodiments of this system are as follows:

[2145] Intelligent search function with emotion engine

[2146] Users can enter text into the search bar or upload an image, and the system analyzes the input, searches the database for relevant product information, and displays it. Utilizing an emotion engine, the system provides search results that take the user's emotions into account.

[2147] Program processing

[2148] The user types "What's the latest version of the refrigerator in this picture?" into the app's search bar, or uploads an image of the refrigerator. The device sends the input data, along with the user's facial expressions and tone of voice captured by the camera and microphone, to the server. The server analyzes the text with natural language processing (NLP) algorithms and the image with image recognition algorithms, while analyzing the user's emotions with an emotion engine. Based on the analysis results, it searches a product database to identify related products. The related products are sent to the device, which displays the results to the user.

[2149] Specific examples

[2150] The user uploads an image of their refrigerator, which the device sends to the server. The server uses image recognition technology to identify the refrigerator model, and if it detects a positive emotion in the user's facial expression, it searches the database for information on the latest, more expensive, and highly rated models. The search results are sent to the device, and the information on the latest model is displayed to the user.

[2151] Event-based gift recommendation engine

[2152] When a user enters a gift-related question in text, the system analyzes the event and target information and recommends gift items based on that information. By combining this function with an emotion engine, it becomes possible to make recommendations based on the consumer's emotions.

[2153] Program processing

[2154] The user types "What gift would you recommend for my mother's birthday?" and emotional information is also collected via the camera and microphone. The device sends the text and emotional information to the server. The server analyzes the question and identifies the event / target information of "birthday" and "mother," while simultaneously analyzing the user's emotions using an emotional engine. The server uses an event-based gift recommendation engine to search for related products, and the recommendation results are sent to the device. The device then displays the recommended gift items to the user.

[2155] Specific examples

[2156] If a user looks hesitant while typing "What would you recommend for my mother's birthday?", the server will recognize this hesitation using its emotion engine and prioritize recommending reliable and popular gift items. The recommended items are then sent to the device and displayed to the user.

[2157] Personalized Shopping Experience

[2158] When a user logs in to the app, the system will suggest products tailored to their individual preferences based on their past purchase and browsing history. Adding an emotion engine to this personalization function will enable even more advanced personalization.

[2159] Program processing

[2160] When a user logs in to the app, the device sends emotional information such as facial expression and tone of voice at the time of login, along with past purchase history and browsing history, to the server. The server analyzes the received history and emotional data to identify the user's preferences and interests. Based on the analysis results, the server generates personalized product suggestions that also take emotional information into account, and a list of suggested products is sent to the device. The device then displays the list of suggested products to the user.

[2161] Specific examples

[2162] When a user logs in to the app, the emotion engine recognizes their happy facial expression. The server takes this happy emotion into account along with their past purchase history to suggest new products that the user is likely to enjoy. The list of suggestions is sent to the device and displayed to the user.

[2163] Purchasing support function

[2164] When a user asks a question about a specific product, the system provides a quick answer using the FAQ section or an AI chatbot. The emotion engine also analyzes emotions, allowing the system to provide a more accurate answer based on the user's emotions.

[2165] Program processing

[2166] The user asks, "How long is the warranty period for this refrigerator?" and emotional information is also collected. The device sends the question and emotional information to the server. The server analyzes the question, recognizes the user's emotions using an emotional engine, and uses an FAQ section or AI chatbot to generate an appropriate answer. The generated answer is sent to the device, which displays it to the user.

[2167] Specific examples

[2168] If a user asks about the warranty period for a refrigerator and shows a worried expression, the server will recognize the user's anxiety using its emotion engine, provide a detailed explanation of the warranty, and provide information on related reliable products. The answer will be sent to the terminal and displayed to the user.

[2169] Community Features

[2170] Users can post reviews and opinions about products, which can then be viewed by other users. By combining this feature with an emotion engine, it is possible to understand the emotional tone of the reviews.

[2171] Program processing

[2172] Users input reviews and opinions, and emotional information is also collected. The device sends the reviews and emotional information to the server. The server stores the reviews and emotional information in a database. When another user visits a specific product page, the server searches the database for reviews of that product and sends the review results, which also reflect the emotional tone, to the device. The device displays the reviews to the user.

[2173] Specific examples

[2174] When a user posts a review about a refrigerator, the emotion engine analyzes the happy facial expression. The server stores the review and positive emotion in a database. When another user visits the refrigerator's product page, the review is displayed on their device as a positive emotion review.

[2175] In this way, the invention that combines the emotion engine can further personalize the consumer's purchasing experience and provide detailed emotional support.

[2176] The processing flow will be explained below.

[2177] Intelligent search function with emotion engine

[2178] Program processing

[2179] Step 1:

[2180] A user types "What's the latest version of the refrigerator in this picture?" into the app's search bar or uploads an image of the refrigerator.

[2181] Step 2:

[2182] The device sends the entered text and images, as well as the user's facial expressions and tone of voice captured through the camera and microphone, to the server.

[2183] Step 3:

[2184] The server analyzes the submitted text using natural language processing (NLP) algorithms to understand the user's intent.

[2185] Step 4:

[2186] The server analyzes the transmitted image using an image recognition algorithm and extracts its features.

[2187] Step 5:

[2188] The server uses an emotion engine to analyze the user's facial expressions and tone of voice to identify the user's emotions.

[2189] Step 6:

[2190] The server searches a product database based on the analysis results to identify highly relevant products.

[2191] Step 7:

[2192] The server transmits the search results together with supplementary information according to the user's feelings to the terminal.

[2193] Step 8:

[2194] The terminal displays the search results to the user.

[2195] Specific examples

[2196] The user uploads an image of the refrigerator, and the image, along with facial expression and tone of voice data, is sent from the device to the server.

[2197] The server uses image recognition technology to identify the refrigerator model, and if it detects a positive emotion in the user's facial expression, it searches the database for information on the latest, more luxurious and highly rated models.

[2198] The search results are sent to the device, and the user is shown information about the latest models and recommendations based on positive sentiment.

[2199] Event-based gift recommendation engine

[2200] Program processing

[2201] Step 1:

[2202] The user texts in, "What do you recommend for my mom's birthday?"

[2203] Step 2:

[2204] The device sends text and emotional information obtained through the camera and microphone to a server.

[2205] Step 3:

[2206] The server analyzes the submitted text using natural language processing (NLP) to identify the event "birthday" and the target "mother."

[2207] Step 4:

[2208] The server analyzes the user's emotions using an emotion engine and identifies the emotions.

[2209] Step 5:

[2210] The server uses an event-based gift recommendation engine to search for relevant gift items.

[2211] Step 6:

[2212] The server sends the search results and supplemental information based on the user's emotions to the terminal.

[2213] Step 7:

[2214] The terminal displays a list of recommended products to the user.

[2215] Specific examples

[2216] If a user types "What would you recommend for my mother's birthday?" and looks confused, the device sends the text and emotional information to the server.

[2217] The server identifies the "birthday" and "mother," and uses an emotion engine to recognize the customer's uncertainty and prioritize searching for reliable and popular gift items.

[2218] The recommended products are sent to the terminal, and a list of reliable products and the reasons for their selection are displayed to the user.

[2219] Personalized Shopping Experience

[2220] Program processing

[2221] Step 1:

[2222] The user logs in to the app.

[2223] Step 2:

[2224] The device sends past purchase history and browsing history to the server along with the user's facial expression and tone of voice when logging in.

[2225] Step 3:

[2226] The server analyzes the received history and emotion data to identify the user's preferences and interests.

[2227] Step 4:

[2228] A server generates personalized product suggestions based on preferences and interests.

[2229] Step 5:

[2230] The server transmits a list of suggested products that also takes emotional information into consideration to the terminal.

[2231] Step 6:

[2232] The terminal displays a list of suggested products to the user.

[2233] Specific examples

[2234] When a user logs in to the app, their happy facial expression is recognized by the emotion engine. The device then sends past purchase history and emotional information to the server.

[2235] The server analyzes the data and generates product suggestions that take into account the user's preferences and pleasant emotions. The suggestion list is sent to the terminal and displayed to the user.

[2236] Purchasing support function

[2237] Program processing

[2238] Step 1:

[2239] A user types a question: "How long is the warranty on this refrigerator?"

[2240] Step 2:

[2241] The device transmits the text and emotion information to the server.

[2242] Step 3:

[2243] The server analyzes the submitted questions and uses an FAQ section or AI chatbot to generate appropriate answers.

[2244] Step 4:

[2245] The server uses an emotion engine to analyze the user's emotions and generate a response based on those emotions.

[2246] Step 5:

[2247] The server sends the generated answer to the terminal.

[2248] Step 6:

[2249] The terminal displays the answer to the user.

[2250] Specific examples

[2251] If a user asks about the warranty period for a refrigerator and looks worried, the server will recognize the anxiety using its emotion engine, provide a detailed explanation of the warranty, and also provide information on related, more reliable products.

[2252] The answer is sent to the terminal and displayed to the user.

[2253] Community Features

[2254] Program processing

[2255] Step 1:

[2256] The user enters reviews and opinions about the purchased product.

[2257] Step 2:

[2258] The device transmits the input reviews, opinions, and sentiment information to the server.

[2259] Step 3:

[2260] The server stores the received reviews and sentiment information in a database.

[2261] Step 4:

[2262] When another user visits a particular product page, the server searches the database for reviews of that product.

[2263] Step 5:

[2264] The server sends the review of the search results and information reflecting the emotional tone to the device.

[2265] Step 6:

[2266] The terminal displays the searched reviews to the user.

[2267] Specific examples

[2268] When a user posts a review about a refrigerator, the emotion engine also analyzes the happy facial expression they express.

[2269] The server stores the review and positive sentiment in a database, and when another user visits the refrigerator's product page, it is displayed on their device as a positive sentiment review.

[2270] In this way, the invention that combines the emotion engine can further personalize the consumer's purchasing experience and provide detailed emotional support.

[2271] Example 2

[2272] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2273] Conventional shopping systems typically search for and recommend products based on consumer input, but lack advanced personalization that takes into account the consumer's emotional information. This makes it difficult to achieve the high relevance and satisfaction desired by consumers. Furthermore, even when recommending products based on events or the consumer's past history, further accuracy could be improved by utilizing emotional information, but such functionality is currently lacking.

[2274] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for receiving and analyzing text or images input by a consumer, means for collecting and analyzing emotional information, and means for searching a database for related products based on the analysis results. This enables highly personalized and highly relevant product searches that take into account the consumer's emotional information. Furthermore, by including means for recommending related products based on an event and means for generating recommendation results taking into account the consumer's emotional information, highly accurate event-based product recommendations can be realized. In addition, by including means for generating personalized product proposals based on the consumer's past history and means for generating recommendation results taking into account the consumer's emotional information, consumer satisfaction can be further increased.

[2275] "Consumer" refers to an individual who purchases or uses a product or service.

[2276] "Input text or image" refers to text data or image data provided by a consumer through an application or website interface.

[2277] "Means of analysis" refers to algorithms or programs that understand input data and extract meaning and features.

[2278] "Emotional information" refers to data about emotions extracted from consumers' facial expressions, tone of voice, gestures, etc.

[2279] "Collect" refers to obtaining data using sensor devices or input devices.

[2280] "Means for analyzing" refers to a system that includes algorithms or programs for interpreting collected emotional information and identifying specific emotional states.

[2281] A "database" refers to a data storage system in which multiple product information items are stored in an organized manner and can be easily searched and referenced.

[2282] A "search method" refers to an algorithm or program that efficiently searches through information in a database and finds entries that match a condition.

[2283] "Related products" refers to products that are relevant to a consumer's interests or needs based on data entered by the consumer and analysis results.

[2284] An "event" refers to a specific date, time, or occasion, such as a birthday, anniversary, or sales event.

[2285] "Past history" refers to a record of a consumer's previous purchases and browsing.

[2286] "Personalized product recommendations" refers to a list of products and services that are specifically suggested to a consumer based on their individual interests, history, and even emotional information.

[2287] "Recommendation results" refers to a list of products selected by the system and presented to the consumer.

[2288] This invention relates to a system that personalizes consumer purchasing experiences and provides highly relevant information and products. In particular, by combining it with an emotion engine that analyzes consumer emotion information, a more advanced level of personalization is achieved.

[2289] Intelligent search function with emotion engine

[2290] Users can enter text or upload an image into the application's search bar, and the system analyzes the input, searches the database for relevant product information, and displays it. By utilizing an emotion engine, the system also takes the user's emotions into account when providing search results.

[2291] The server analyzes the user's input data using natural language processing (NLP) algorithms and images using image recognition algorithms. It also analyzes the user's emotions using an emotion engine. Based on the analysis results, the server searches a database to identify relevant products. The search results are sent to the device, which displays them to the user.

[2292] Specific examples

[2293] The user uploads an image of their refrigerator, which the device sends to the server. The server uses image recognition technology to identify the refrigerator model, and if it recognizes a positive emotion in the user's facial expression, it searches the database for information on the latest, more expensive, and highly rated model. The search results are sent to the device, and the information on the latest model is displayed to the user.

[2294] Example prompt sentence:

[2295] "Browse photos of the latest refrigerator models. Users seem happy."

[2296] Event-based gift recommendation engine

[2297] When a user enters a gift-related question in text, the system analyzes the event and target information and recommends gift items based on that information. This function can be combined with an emotion engine to make recommendations based on the consumer's emotions.

[2298] The server analyzes the user's question and identifies event target information such as "birthday" and "mother." It then analyzes the user's emotions using an emotion engine. Based on the analysis results, it uses an event-based gift recommendation engine to search for relevant products. The recommendation results are sent to the device, which then displays the gift products to the user.

[2299] Specific examples

[2300] If a user types in "What gift would you recommend for my mother's birthday?" and shows a confused expression, the server will recognize the user's indecision using its emotion engine and prioritize recommending reliable and popular gift items. The recommended items are then sent to the device and displayed to the user.

[2301] Example prompt sentence:

[2302] "Users seem to be confused about finding the perfect gift for their mother's birthday."

[2303] Personalized Shopping Experience

[2304] When a user logs in to the application, the system will suggest products tailored to their individual preferences based on their past purchase and browsing history. Adding an emotion engine to this personalization function will enable even more advanced personalization.

[2305] The server analyzes the received history data and emotional data to identify the user's preferences and interests. Based on the analysis results, the server generates personalized product suggestions that take into account the emotional information and sends the list of suggested products to the terminal. The terminal then displays the list of suggested products to the user.

[2306] Specific examples

[2307] When a user logs in to the app, the emotion engine recognizes their happy facial expression. The server takes this happy emotion into account along with their past purchase history to suggest new products that the user is likely to enjoy. The list of suggestions is sent to the device and displayed to the user.

[2308] Example prompt sentence:

[2309] "Suggest new products to users who are in a good mood. Also consider past purchase history."

[2310] Purchasing support function

[2311] When a user asks a question about a specific product, the system provides a quick answer using the FAQ section or an AI chatbot. The emotion engine also analyzes emotions, allowing the system to provide a more accurate answer based on the user's emotions.

[2312] The server analyzes the question and recognizes the user's emotions using an emotion engine. Based on the analysis results, it generates an appropriate answer using an FAQ section or an AI chatbot and sends the answer to the device. The device then displays the answer to the user.

[2313] Specific examples

[2314] If a user asks about the warranty period for a refrigerator and shows a worried expression, the server will recognize the user's anxiety using its emotion engine, provide a detailed explanation of the warranty, and provide information on related reliable products. The answer will be sent to the terminal and displayed to the user.

[2315] Example prompt sentence:

[2316] "We will inform anxious users about the warranty period of the refrigerator and also suggest related trusted products."

[2317] Community Features

[2318] Users can post reviews and opinions about products, which can then be viewed by other users. By combining this feature with an emotion engine, it is possible to understand the emotional tone of the reviews.

[2319] The server stores the reviews and emotional information in a database, and when another user visits a specific product page, it provides review results that also reflect the emotional tone.The terminal displays the reviews to the user.

[2320] Specific examples

[2321] When a user posts a review about a refrigerator, the emotion engine analyzes the happy facial expression. The server stores the review and positive emotion in a database. When another user visits the refrigerator's product page, the review is displayed on their device as a positive emotion review.

[2322] Example prompt sentence:

[2323] "We store users' positive reviews in a database and make them available to other users."

[2324] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2325] Intelligent search function with emotion engine

[2326] Program processing

[2327] Step 1:

[2328] A user enters text into the app's search bar or uploads an image, and the input data is captured on the device.

[2329] Input: Text entered into the search bar or an uploaded image

[2330] Output: Input data is stored in the terminal

[2331] Step 2:

[2332] The device collects the user's facial expressions and tone of voice using a camera and microphone, and the collected emotional information is stored as data on the device.

[2333] Input: User's facial expression, tone of voice

[2334] Output: Emotional information data (facial expression recognition results, voice tone analysis results)

[2335] Step 3:

[2336] The device sends the input data and emotion information to the server, where they are packaged as data packets.

[2337] Input: Input data and emotion information data

[2338] Output: sent as data packets to the server

[2339] Step 4:

[2340] The server analyzes the input data: for text input, it uses natural language processing (NLP) algorithms; for images, it uses image recognition algorithms.

[2341] Input: User-entered data (text or image)

[2342] Output: Analysis results (e.g., text semantic analysis results, image recognition results)

[2343] Step 5:

[2344] The server analyzes the user's emotional information using an emotion engine, and the analysis results include specific emotions (e.g., positive, negative, etc.).

[2345] Input: Emotional information data

[2346] Output: Sentiment analysis result (e.g., positive, negative)

[2347] Step 6:

[2348] The server searches a product database based on the analysis results, and highly relevant product information is identified.

[2349] Input: Text / image analysis results, sentiment analysis results

[2350] Output: Related product information

[2351] Step 7:

[2352] The server sends the search results to the device in an appropriate format (e.g., JSON format).

[2353] Input: Related product information

[2354] Output: Related product information sent

[2355] Step 8:

[2356] The terminal displays the search results to the user using a graphical user interface (GUI).

[2357] Input: Related product information sent from the server

[2358] Output: Product information is displayed visually to the user

[2359] Event-based gift recommendation engine

[2360] Program processing

[2361] Step 1:

[2362] The user enters a question about the gift in text, and the input data is stored on the device.

[2363] Input: Text question (e.g., "What would you recommend for my mother's birthday?")

[2364] Output: Input data is stored in the terminal

[2365] Step 2:

[2366] The device collects the user's emotional information through the camera and microphone, and the collected data is stored on the device.

[2367] Input: User's facial expression, tone of voice

[2368] Output: Emotional information data (facial expression recognition results, voice tone analysis results)

[2369] Step 3:

[2370] The device sends the question and emotion information to the server. The data is sent in packets.

[2371] Input: Text questions, emotion information data

[2372] Output: sent as data packets to the server

[2373] Step 4:

[2374] The server parses the text question and extracts event target information such as "birthday" or "mother."

[2375] Input: Text question

[2376] Output: Analysis results (event and target information)

[2377] Step 5:

[2378] The server analyzes the user's emotional information using an emotion engine, and the analysis results include specific emotions.

[2379] Input: Emotional information data

[2380] Output: Emotion analysis results

[2381] Step 6:

[2382] Based on the analysis results, the server searches for related products using an event-based gift recommendation engine.

[2383] Input: Event and target information, emotion analysis results

[2384] Output: Relevant gift product information

[2385] Step 7:

[2386] The server sends the recommendation results to the device in an appropriate format (e.g., JSON format).

[2387] Input: Related gift product information

[2388] Output: Gift item information sent

[2389] Step 8:

[2390] The device displays the recommended gift items to the user through a GUI.

[2391] Input: Gift item information sent from the server

[2392] Output: Product information is displayed visually to the user

[2393] Personalized Shopping Experience

[2394] Program processing

[2395] Step 1:

[2396] A user logs in to the app, and their authentication data is stored on the device.

[2397] Input: User ID, Password

[2398] Output: Authentication result and user login history

[2399] Step 2:

[2400] The device uses the camera and microphone to collect emotional information such as facial expressions and tone of voice when logging in. The collected data is stored on the device.

[2401] Input: User's facial expression, tone of voice

[2402] Output: Emotional information data (facial expression recognition results, voice tone analysis results)

[2403] Step 3:

[2404] The device sends past purchase history, browsing history, and even emotional information to a server. The data is sent in packets.

[2405] Input: purchase history, browsing history, emotional information data

[2406] Output: sent as data packets to the server

[2407] Step 4:

[2408] The server analyzes the received historical and emotional data to identify the user's preferences and interests.

[2409] Input: history data, emotion data

[2410] Output: Preference analysis results

[2411] Step 5:

[2412] Based on the analysis results, the server generates personalized product suggestions that take emotional information into consideration.

[2413] Input: Preference analysis results, emotional information

[2414] Output: Personalized product recommendations

[2415] Step 6:

[2416] The list of suggested products is sent to the terminal, and the data is in the appropriate format.

[2417] Input: Personalized product suggestions

[2418] Output: Product suggestion information sent

[2419] Step 7:

[2420] The terminal displays a list of suggested products to the user through a GUI.

[2421] Input: Product suggestion information sent from the server

[2422] Output: A list of product suggestions is visually displayed to the user

[2423] Purchasing support function

[2424] Program processing

[2425] Step 1:

[2426] The user enters a question about a specific product, and the input data is saved on the device.

[2427] Input: Text question (e.g., "How long is the warranty on this refrigerator?")

[2428] Output: Input data is stored in the terminal

[2429] Step 2:

[2430] The device collects the user's emotional information through the camera and microphone, and the collected data is stored on the device.

[2431] Input: User's facial expression, tone of voice

[2432] Output: Emotional information data (facial expression recognition results, voice tone analysis results)

[2433] Step 3:

[2434] The device sends the question and emotion information to the server. The data is sent in packets.

[2435] Input: Text questions, emotion information data

[2436] Output: sent as data packets to the server

[2437] Step 4:

[2438] The server analyzes the question text and determines the meaning of the question.

[2439] Input: Text question

[2440] Output: Question analysis results (e.g. warranty period extraction)

[2441] Step 5:

[2442] The server analyzes the user's emotional information using an emotion engine, and the analysis results include specific emotions.

[2443] Input: Emotional information data

[2444] Output: Emotion analysis results (e.g., anxiety recognition)

[2445] Step 6:

[2446] The server utilizes a FAQ section and AI chatbot to generate appropriate answers.

[2447] Input: Question analysis results, sentiment analysis results

[2448] Output: Generated answer (e.g. warranty period details)

[2449] Step 7:

[2450] The generated response is sent to the device, with the data in the appropriate format.

[2451] Input: Generated Answer

[2452] Output: Submitted response

[2453] Step 8:

[2454] The terminal displays the answers to the user through a GUI.

[2455] Input: The answer sent by the server

[2456] Output: The answer is displayed visually to the user

[2457] Community Features

[2458] Program processing

[2459] Step 1:

[2460] Users input reviews and opinions about products. The input data is saved on the device.

[2461] Input: Review or opinion text

[2462] Output: Input data is stored in the terminal

[2463] Step 2:

[2464] The device collects the user's emotional information through the camera and microphone, and the collected data is stored on the device.

[2465] Input: User's facial expression, tone of voice

[2466] Output: Emotional information data (facial expression recognition results, voice tone analysis results)

[2467] Step 3:

[2468] The device sends reviews and sentiment information to the server in the form of packets.

[2469] Input: Review text and sentiment data

[2470] Output: sent as data packets to the server

[2471] Step 4:

[2472] The server stores the reviews and sentiment information in a database, and the data is stored in a suitable format.

[2473] Input: Review text, emotional information data

[2474] Output: Saved data

[2475] Step 5:

[2476] When another user visits a particular product page, the server searches the database for reviews of that product, and the results reflect the emotional tone of the reviews.

[2477] Input: Product page visit request

[2478] Output: Search results (reviews and sentiment tone)

[2479] Step 6:

[2480] The server sends the search results to the device in the appropriate format.

[2481] Input: Search results

[2482] Output: Submitted search results

[2483] Step 7:

[2484] The device displays the reviews to the user through a GUI.

[2485] Input: Reviews sent from the server

[2486] Output: The review is displayed visually to the user

[2487] (Application example 2)

[2488] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2489] Conventional purchasing experience systems suggest related products by analyzing text and images entered by consumers, but do not take into account consumer circumstances such as emotions and behavior, which limits the degree of personalization. Furthermore, recommendations based solely on events or purchase history make it difficult to accurately grasp consumer needs in real time, posing challenges to improving consumer satisfaction.

[2490] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving and analyzing text or images entered by a consumer, means for searching a database for related products based on the analysis results, means for displaying the search results to the consumer, means for collecting the consumer's facial expressions, movements, and voice using cameras and sensors in the store, means for analyzing the collected data to recognize the consumer's emotions, and means for suggesting related products based on the recognized emotions. This enables more advanced and precise personalized product suggestions that take into account the consumer's emotions and real-time situation.

[2491] "Consumer" refers to any member of the public who intends to purchase a service or product.

[2492] "Input text or image" refers to written or visual information provided by the consumer to the system.

[2493] "Means of analysis" refers to the technology that understands input text or images and extracts information based on them.

[2494] "Means for searching related products from a database" refers to technology that searches for appropriate product information from a database based on the analysis results.

[2495] "Means for displaying search results to consumers" refers to the technology for displaying information retrieved from a database on a device used by a consumer.

[2496] "Cameras and sensors in the store" refers to monitoring and sensing devices used to capture consumer movements, facial expressions, and voices within the store.

[2497] "Means of collection" refers to technology that uses cameras and sensors to acquire data such as consumer movements, facial expressions, and voice.

[2498] "Means for recognizing consumer emotions by analyzing collected data" refers to technology that analyzes acquired data and identifies the emotions consumers are feeling.

[2499] "Means for suggesting related products based on recognized emotions" refers to technology that selects and suggests products suitable for consumers based on the results of emotion analysis.

[2500] This invention is a system for improving consumer purchasing experiences, and aims to analyze consumer sentiment and provide personalized product recommendations, particularly in brick-and-mortar stores. This system is implemented based on the following configuration and processing steps.

[2501] System configuration

[2502] 1. Hardware Configuration

[2503] Cameras and sensors: These are installed in physical stores to collect consumers' facial expressions, movements, and voices. For example, we use standard surveillance cameras and voice recognition sensors.

[2504] Devices: Consumers' smartphones, smart glasses, head-mounted displays (HMDs), etc. These devices send collected data to a server and display information from the server to the consumer.

[2505] Server: A server for data analysis and product proposals.

[2506] 2. Software Configuration

[2507] Natural Language Processing (NLP): Analyzing consumer-supplied text or speech, for example using common natural language processing algorithms.

[2508] Image recognition algorithm: Analyzes images uploaded by consumers and identifies related products. As a concrete example, we will use a common image recognition algorithm.

[2509] Emotion engine: Recognizes consumer emotions based on collected facial and voice data. As a concrete example, a general emotion recognition algorithm is used.

[2510] Database: A database for storing product information. Specific examples include SQL databases and NoSQL databases.

[2511] Processing flow

[2512] 1. Data Collection

[2513] Cameras and sensors are used to collect the consumer's facial expressions, movements, and voice, and the data is sent to a server via the device.

[2514] 2. Data Analysis

[2515] The data received by the server is analyzed using the following software:

[2516] Natural Language Processing Algorithms

[2517] Image Recognition Algorithm

[2518] Emotion Engine

[2519] 3. Product proposal

[2520] The server searches the product database based on the analysis results, identifies related products, and sends them to the terminal, which then displays the obtained product information to the consumer.

[2521] Specific examples

[2522] Scenario 1: Entering the store

[2523] A consumer enters a store and uses a device (e.g., smart glasses) to navigate the store. When the camera captures the consumer's interested expression, the emotion engine recognizes the interest and the server sends a guide message based on that, showing new product sections and recommended products.

[2524] Scenario 2: Product selection support

[2525] If a consumer appears to be unsure about a particular product, the camera captures the situation and the emotion engine recognizes the emotion of "indecision." Based on this, the server suggests other highly rated products and related articles, and displays this information on the device.

[2526] Prompt sentence for generative AI model

[2527] A customer enters a physical store and uses a smart device. The in-store camera captures the customer's facial expressions and behavior. If the customer shows interest, output a product suggestion method based on that interest. Also, explain how to suggest alternative products if the customer is unsure.

[2528] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2529] Step 1:

[2530] A user enters a store and uses a device such as a smartphone, smart glasses, or head-mounted display (HMD). Cameras and sensors collect the user's facial expressions, movements, and voice. This data is then input into the device.

[2531] Step 2:

[2532] The facial expression, movement, and voice data collected by the device are sent to a server in real time. Data input includes image data and voice data.

[2533] Step 3:

[2534] The server analyzes the received data. Specifically, it performs the following processes:

[2535] Natural language processing (NLP) algorithms analyze voice data and user-entered text.

[2536] Image recognition algorithms analyze the collected image data to identify product categories of interest.

[2537] The emotion engine analyzes facial and voice data to identify the user's emotions (e.g., interest, uncertainty, anxiety).

[2538] Step 4:

[2539] The server combines the user's emotional information obtained as a result of the analysis with the results of NLP and image recognition to search for related products in a database. The input includes emotional data, voice and text data, and image data, and generates a list of related products as an output.

[2540] Step 5:

[2541] The server generates a related product list and sends it to the device. The device receives it and displays it to the user. The displayed product list is updated in real time based on the user's facial expressions and movements.

[2542] Step 6:

[2543] The user can review the displayed product list and request more information about products they are interested in. Inputs at the device include touch and voice commands, and the details are again requested from the server.

[2544] Step 7:

[2545] The server retrieves detailed information from the database based on the user's request and sends it to the terminal, which then displays the information to the user, allowing the user to view more detailed product information.

[2546] Through the above steps, real-time personalized product suggestions that take user emotions into consideration are realized.

[2547] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[2548] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2549] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[2550] [Fourth embodiment]

[2551] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[2552] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[2553] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[2554] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[2555] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[2556] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[2557] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[2558] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[2559] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[2560] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[2561] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[2562] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[2563] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2564] The present invention relates to a system for personalizing a consumer's purchasing experience and providing highly relevant information and products. A specific embodiment of this system will be described below.

[2565] Intelligent Search Function

[2566] Users can enter text or upload an image into the search bar, and the system will analyze the input and search the database for relevant product information to display. This intelligent search function allows consumers to efficiently obtain the product information they are looking for.

[2567] Program processing

[2568] The user types "What's the latest version of the refrigerator in this picture?" into the app's search bar, or uploads an image of the refrigerator. The device sends the input data to the server, which uses natural language processing (NLP) algorithms to analyze the text and understand the user's intent. In the case of an image, the server uses image recognition algorithms to extract product features. Based on the analysis, the server searches a product database to identify relevant products. The search results are sent to the device, which displays them to the user.

[2569] Specific examples

[2570] The user uploads an image of their refrigerator, and the device sends the image to the server. The server uses image recognition technology to identify the refrigerator model and searches the database for the latest information. The search results are sent to the device, and the latest model information is displayed to the user.

[2571] Event-based gift recommendation engine

[2572] Users can enter gift-related questions via text, and the system will analyze the event and target information and recommend gift items based on that information, allowing consumers to find the perfect gift for a special event or occasion.

[2573] Program processing

[2574] The user types, "What gift would you recommend for my mother's birthday?" The device sends the question to the server. The server analyzes the question and extracts the event target information "birthday" and "mother." The server uses an event-based gift recommendation engine to search for relevant products and sends the recommendation results to the device. The device displays the recommended gift items to the user.

[2575] Specific examples

[2576] When a user types "What would you recommend for my mother's birthday?", the device sends the question to the server. The server analyzes the "birthday" and "mother" and recommends gift items based on that. The recommendation results are sent to the device, and suitable gift candidates are displayed to the user.

[2577] Personalized Shopping Experience

[2578] When a user logs in to the app, the system will suggest products tailored to their individual preferences based on their past purchase and browsing history. This personalization feature allows users to quickly search for product information based on their interests and preferences.

[2579] Program processing

[2580] A user logs in to the app. The device sends the user's past purchase and browsing history to the server. The server analyzes the user's historical data and identifies individual preferences and interests. The server generates personalized product suggestions based on the analysis results and sends a list of suggested products to the device. The device displays the list of suggested products to the user.

[2581] Specific examples

[2582] When a user logs in to the app, their device sends their past purchase history to the server. The server analyzes the data and suggests products that match the user's preferences. A list of suggested products is sent to the device and displayed to the user.

[2583] Purchasing support function

[2584] When users ask questions about specific products, the system provides quick answers using the FAQ section and AI chatbots, helping users to resolve any doubts or concerns they may have about their purchase.

[2585] Program processing

[2586] The user types a question, such as "How long is the warranty on this refrigerator?" The device sends the question to the server, which uses an FAQ section or an AI chatbot to generate an appropriate answer to the question. The generated answer is sent to the device, which then displays the appropriate answer to the user.

[2587] Specific examples

[2588] When a user asks about the warranty period of a refrigerator, the device sends the question to the server, which parses the question and retrieves the warranty period information from the FAQ section. The retrieved information is sent to the device and displayed to the user.

[2589] Community Features

[2590] Users can post reviews and opinions about products, and other users can view that information. This feature allows consumers to make purchasing decisions while also taking into account the opinions of other users.

[2591] Program processing

[2592] The user enters a review or opinion on a product. The device sends the review to the server. The server stores the received review in a database. When another user visits a specific product page, the server searches the database for reviews of that product and sends the search results to the device. The device displays the review to the user.

[2593] Specific examples

[2594] A user posts a review about a refrigerator, and the device sends it to the server. The server stores the review in a database. When another user visits the refrigerator's product page, the server retrieves the stored review and sends it to the device. The device displays the review to the user.

[2595] In this way, the system according to the claims can enhance the consumer's buying experience and provide personalized services.

[2596] The processing flow will be explained below.

[2597] Intelligent Search Function

[2598] Program processing

[2599] Step 1:

[2600] A user types "What's the latest version of the refrigerator in this picture?" into the app's search bar or uploads an image of the refrigerator.

[2601] Step 2:

[2602] The device sends the entered text and images to the server.

[2603] Step 3:

[2604] The server analyzes the submitted text using natural language processing (NLP) algorithms to understand the user's intent.

[2605] Step 4:

[2606] The server analyzes the transmitted image using an image recognition algorithm and extracts its features.

[2607] Step 5:

[2608] The server searches a product database based on the analysis results to identify highly relevant products.

[2609] Step 6:

[2610] The server transmits the search results to the terminal.

[2611] Step 7:

[2612] The terminal displays the search results to the user.

[2613] Event-based gift recommendation engine

[2614] Program processing

[2615] Step 1:

[2616] The user texts in, "What do you recommend for my mom's birthday?"

[2617] Step 2:

[2618] The terminal sends the text to the server.

[2619] Step 3:

[2620] The server analyzes the submitted text using natural language processing (NLP) to identify the event "birthday" and the target "mother."

[2621] Step 4:

[2622] The server uses an event-based gift recommendation engine to search for relevant gift items.

[2623] Step 5:

[2624] The server transmits the recommendation results to the terminal.

[2625] Step 6:

[2626] The terminal displays a list of recommended products to the user.

[2627] Personalized Shopping Experience

[2628] Program processing

[2629] Step 1:

[2630] The user logs in to the app.

[2631] Step 2:

[2632] The terminal sends the logged-in user's past purchase history and browsing history to the server and loads that data.

[2633] Step 3:

[2634] The server analyzes the received historical data to identify the user's preferences and interests.

[2635] Step 4:

[2636] The server generates personalized product suggestions based on the analysis results.

[2637] Step 5:

[2638] The server transmits the generated list of suggested products to the terminal.

[2639] Step 6:

[2640] The terminal displays a list of suggested products to the user.

[2641] Purchasing support function

[2642] Program processing

[2643] Step 1:

[2644] A user types a question: "How long is the warranty on this refrigerator?"

[2645] Step 2:

[2646] The terminal transmits the entered question to the server.

[2647] Step 3:

[2648] The server analyzes the submitted questions and uses an FAQ section or AI chatbot to generate appropriate answers.

[2649] Step 4:

[2650] The server sends the generated answer to the terminal.

[2651] Step 5:

[2652] The terminal displays the generated answer to the user.

[2653] Community Features

[2654] Program processing

[2655] Step 1:

[2656] The user enters reviews and opinions about the purchased product.

[2657] Step 2:

[2658] The device sends the entered reviews and opinions to the server.

[2659] Step 3:

[2660] The server stores the received reviews in a database.

[2661] Step 4:

[2662] When another user visits a particular product page, the server searches the database for reviews of that product.

[2663] Step 5:

[2664] The server sends the review of the search results to the terminal.

[2665] Step 6:

[2666] The terminal displays the searched reviews to the user.

[2667] Example 1

[2668] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2669] Conventional online shopping systems make it difficult for consumers to efficiently find the best products for them from the vast amount of product information available, resulting in a cumbersome shopping experience. Furthermore, product recommendations based on special events or individual preferences are not adequately implemented, which can lead to low consumer satisfaction. Furthermore, the inability to easily obtain detailed product information or reviews can make it difficult to resolve concerns or questions about purchasing.

[2670] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2671] In this invention, the server includes a means for analyzing text or images entered by a consumer, a means for analyzing the text using a natural language processing algorithm to understand the user's intent, a means for analyzing the image using an image recognition algorithm to extract product features, a means for searching a database for related products based on the analysis results, and a means for displaying the search results to the consumer. This allows consumers to efficiently find the most suitable product. In addition, personalized recommendations based on events and past history improve consumer satisfaction and provide a comfortable shopping experience.

[2672] "Consumer" refers to a general user of the system for the purpose of purchasing goods or services.

[2673] "Text" refers to sentences or characters entered by consumers, and is textual information that is understood by the system.

[2674] "Images" refers to visual information such as photographs and illustrations uploaded by consumers.

[2675] "Means of analysis" refers to methods and techniques for processing input text or images with a computer program and understanding their content.

[2676] A "natural language processing algorithm" refers to a computational method for analyzing text and understanding the meaning and intent of human language.

[2677] "Means for understanding user intent" refers to methods that use natural language processing algorithms to analyze the meaning and purpose of text entered by a user.

[2678] An "image recognition algorithm" refers to a computational method for analyzing an image and identifying the objects and features contained within it.

[2679] "Means for extracting product features" refers to the use of image recognition algorithms to identify specific products and their attributes from uploaded images.

[2680] "Related Products" refers to products and services available for purchase that are suggested based on information entered or uploaded by the consumer.

[2681] A "database" refers to a collection of information that systematically stores related products and other information and makes it easy to search and retrieve.

[2682] "Searching means" refers to the methods and technologies used to search the database based on the analysis results and find relevant product information.

[2683] "Means for displaying search results to consumers" refers to methods and technologies for displaying searched product information on a user interface so that consumers can check it.

[2684] "Event" refers to a specific situation or occasion, such as a birthday or anniversary, and is the criterion for recommending products related to that situation.

[2685] "Personalized product suggestions" refers to a method of recommending products that match a consumer's individual preferences based on individual data such as their past purchase history and browsing history.

[2686] The present invention relates to a system for personalizing a consumer's shopping experience and providing highly relevant information and products. The system has multiple functions for analyzing user-entered text and images and efficiently providing relevant product information.

[2687] Intelligent Search Function

[2688] Users can enter text or upload an image into the search bar, and the system will analyze the input and search the database for relevant product information to display. This intelligent search function allows consumers to efficiently obtain the product information they are looking for.

[2689] Hardware and software used

[2690] Server: A computer with high processing power (e.g., a Linux server)

[2691] Terminal: The device that provides the user interface (e.g., smartphone, tablet, PC)

[2692] Natural Language Processing Algorithms: Google Cloud Natural Language API

[2693] Image recognition algorithm: Amazon Rekognition

[2694] Database: MySQL

[2695] Specific actions

[2696] The user types the text "What is the latest version of the refrigerator in this image?" into the app's search bar or uploads an image of the refrigerator.

[2697] The terminal sends the input data to the server.

[2698] The server uses the Google Cloud Natural Language API to analyze the text and understand the user's intent.

[2699] In the case of images, the server uses Amazon Rekognition to perform image recognition and extract the product's features.

[2700] The server searches a MySQL database based on the analysis results to identify relevant products.

[2701] The search results are sent to the terminal, which displays the results to the user.

[2702] Event-based gift recommendation engine

[2703] Users can enter gift-related questions via text, and the system will analyze the event and target information and recommend gift items based on that information, allowing consumers to find the perfect gift for a special event or occasion.

[2704] Specific actions

[2705] User types, "What do you recommend for my mom's birthday?"

[2706] The terminal sends a question to the server.

[2707] The server parses the question using the Google Cloud Natural Language API and extracts the event target information "birthday" and "mother."

[2708] The server uses an event-based gift recommendation engine to search for relevant products, and the recommendation results are sent to the terminal.

[2709] The terminal displays the recommended gift items to the user.

[2710] Personalized Shopping Experience

[2711] When a user logs in to the app, the system will suggest products tailored to their individual preferences based on their past purchase and browsing history. This personalization feature allows users to quickly search for product information based on their interests and preferences.

[2712] Specific actions

[2713] A user logs in to the app.

[2714] The device sends the user's past purchase history and browsing history to the server.

[2715] The server analyzes the received historical data to identify the user's preferences and interests.

[2716] The server generates personalized product suggestions based on the analysis results, and a list of suggested products is sent to the terminal.

[2717] The terminal displays a list of suggested products to the user.

[2718] Purchasing support function

[2719] When users ask questions about specific products, the system provides quick answers using the FAQ section and AI chatbots, helping users to resolve any doubts or concerns they may have about their purchase.

[2720] Specific actions

[2721] A user types a question: "How long is the warranty on this refrigerator?"

[2722] The terminal sends a question to the server.

[2723] The server uses Dialogflow to analyze the question and generate an appropriate answer from the FAQ database.

[2724] The generated answer is sent to the terminal, which displays the appropriate answer to the user.

[2725] Community Features

[2726] Users can post reviews and opinions about products, and other users can view that information. This feature allows consumers to make purchasing decisions while also taking into account the opinions of other users.

[2727] Specific actions

[2728] Users enter product reviews and opinions.

[2729] The device sends the review to the server.

[2730] The server stores the received reviews in a MySQL database.

[2731] When another user visits a specific product page, the server searches the database for reviews of that product and sends the search results to the terminal.

[2732] The device displays the review to the user.

[2733] With the above functions, the system of the present invention can improve the consumer's purchasing experience and provide personalized services.

[2734] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2735] Intelligent Search Function

[2736] Step 1:

[2737] A user types the text "What is the latest version of the refrigerator in this image?" into the search bar or uploads an image of a refrigerator.

[2738] Input: Text or image from the user

[2739] Output: Text or image sent to the terminal

[2740] Step 2:

[2741] The terminal sends the input data to the server.

[2742] Input: Text or Image

[2743] Output: HTTP request sent to the server

[2744] Step 3:

[2745] The server branches the process depending on the input data. For text, the server analyzes the text using the Google Cloud Natural Language API to understand the user's intent. For images, the server extracts features using Amazon Rekognition.

[2746] Input: Text or image sent to the server

[2747] Output: Analysis results (user intent for text, feature data for images)

[2748] Step 4:

[2749] Based on the analysis results, the server searches the MySQL database to identify relevant products.

[2750] Specifically, an SQL query is generated based on the extracted keywords and features and issued to the database.

[2751] Input: Analysis results (user intent or feature data)

[2752] Output: Related product information

[2753] Step 5:

[2754] The server formats the search results in JSON format and sends them to the terminal as an HTTP response.

[2755] Input: Relevant product information

[2756] Output: Search results in JSON format

[2757] Step 6:

[2758] The terminal parses the received search results and displays them on the user interface.

[2759] Input: JSON format search results

[2760] Output: Product information displayed in the user interface

[2761] Event-based gift recommendation engine

[2762] Step 1:

[2763] User types, "What do you recommend for my mom's birthday?"

[2764] Input: User question text

[2765] Output: Question data sent to the terminal

[2766] Step 2:

[2767] The terminal transmits the question data to the server.

[2768] Input: Question data

[2769] Output: HTTP request sent to the server

[2770] Step 3: ...

Claims

1. means for receiving and analyzing consumer-entered text or images; A means for searching a database for related products based on the analysis results; a means for displaying search results to consumers; and A system including:

2. The system of claim 1 , further comprising: means for recommending related products based on the event.

3. The system of claim 1 , further comprising: means for generating personalized product suggestions based on a consumer's past purchasing and browsing history.

Images