system

Abstract

We provide the system. [Solution] A means of receiving voice or text input from the user, A means for analyzing the aforementioned input data and searching a database of related home appliances, A method for selecting the optimal product by analyzing product reviews and evaluations based on acquired data, A means for generating and providing to the user detailed information and purchase links for the aforementioned product, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Modern consumers have the problem that they have to investigate a huge amount of information when making an optimal product selection because there are a variety of household appliances in the market. Therefore, it is difficult for users to find products that match their own lifestyles and specific needs, resulting in problems that require time and effort.

Means for Solving the Problems

[0005] This invention provides a means for receiving voice or text input from a user, analyzing this input data, and searching a database of related home appliances. Furthermore, it supports the user's purchasing decision by analyzing product reviews and ratings based on the acquired data to select the optimal product, and generating and providing the user with detailed product information and purchase links. This system allows users to quickly and efficiently choose the product that best suits them without having to gather information in a cumbersome manner.

[0006] A "user" refers to an individual or legal entity that uses the system to obtain information about home appliances.

[0007] "Voice or text input" refers to the act of a user providing information to a system using speech recognition technology or text input.

[0008] "Input data" refers to the collective information that a user sends to the system, either as voice or text.

[0009] "Analysis" refers to the process of extracting meaning from received data and searching for related information.

[0010] A "home appliance database" refers to a collection of information that stores detailed information about various types of home appliances.

[0011] "Reviews and ratings" refer to records of opinions and evaluations made by consumers or third parties about a particular product.

[0012] "Selecting the optimal product" refers to the process of determining the product that best suits the user's requirements and needs, based on the analysis results.

[0013] "Detailed information" refers to information such as specifications, price, and features of the selected product.

[0014] The "purchase link" refers to the means of accessing an online store or sales site for purchasing the selected product.

[0015] "Provide" refers to the act of the system presenting or supplying information to the user.

Brief Explanation of Drawings

[0016] [Figure 1] It is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which multiple emotions are mapped. [Figure 10] It shows an emotion map to which multiple emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13]It is a sequence diagram showing the processing flow of a data processing system in Embodiment 2 when combined with an emotion engine. [Figure 14] It is a sequence diagram showing the processing flow of a data processing system in Application Example 2 when combined with an emotion engine.

Mode for Carrying Out the Invention

[0017] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0020] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0021] In the following embodiments, a numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0022] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0024] [First Embodiment]

[0025] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0026] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0029] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0032] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0036] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0037] This invention provides a system for streamlining the process by which users search for information about home appliances using voice or text. Users can input questions about home appliances using voice or text via a terminal such as a smartphone or personal computer. This input is converted into text data through voice recognition or text analysis on the terminal and transmitted to a server.

[0038] The server analyzes the received input data using natural language processing technology to search for relevant home appliances. From the resulting list of products, it further collects and analyzes reviews and ratings for each product to select the product that best suits the user's needs. This selection process considers the overall product evaluation and cost-effectiveness.

[0039] For selected products, the server generates detailed information and provides it to the user, including a purchase link. The terminal displays this received information on the user's screen, allowing the user to easily understand and make a purchase decision. For example, if the user enters "I want the latest smart refrigerator," the system evaluates several smart refrigerators on the market and displays the user detailed information on the highest-rated model along with a purchase link.

[0040] Furthermore, user behavior data and feedback are accumulated on the server, and the AI ​​model is continuously trained to improve the accuracy of suggestions. This allows users to confidently and efficiently choose home appliances that meet their needs.

[0041] The following describes the processing flow.

[0042] Step 1:

[0043] Users can ask questions about home appliances using their smartphones or computers via voice or text input. For example, they might ask, "Tell me about the latest smart refrigerators."

[0044] Step 2:

[0045] If the device receives voice input, it uses speech recognition technology to convert it into text data. If it receives text input, it is acquired as text data as is.

[0046] Step 3:

[0047] The terminal sends the generated text data to the server. In this case, the data is sent to a server in the cloud via an internet connection.

[0048] Step 4:

[0049] The server analyzes the received text data using natural language processing techniques. Specifically, it extracts keywords from the input questions and performs contextual analysis to understand the user's intent.

[0050] Step 5:

[0051] The server searches the database for a list of related home appliances based on the analysis results. Here, the list of potential home appliances is narrowed down based on the extracted keywords.

[0052] Step 6:

[0053] The server collects reviews and ratings of relevant products from online review sites for a list of home appliances. Furthermore, it uses natural language processing to analyze the sentiment of the reviews and calculate an overall rating score.

[0054] Step 7:

[0055] The server selects the most suitable product for the user based on the calculated evaluation score. Selection criteria include evaluation score, price, and features.

[0056] Step 8:

[0057] The server generates detailed information and purchase links for the selected products. This information includes product features, pricing, and retailer links.

[0058] Step 9:

[0059] The server generates information and sends it to the terminal. The terminal receives this information and displays it in a user-friendly format.

[0060] Step 10:

[0061] Users can review the information displayed on their device and, if necessary, proceed with the purchase process via the displayed link.

[0062] Step 11:

[0063] The server updates the AI ​​model based on user feedback and purchase history. This learning process will improve the accuracy of suggestions in the future.

[0064] (Example 1)

[0065] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0066] Conventional consumer electronics information retrieval systems struggle to quickly and accurately suggest the most suitable products to users. Furthermore, they lack mechanisms for efficiently collecting and utilizing feedback to improve the accuracy of suggestions based on user needs. Therefore, there is a need for the development of a comprehensive system that enhances the user experience.

[0067] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0068] In this invention, the server includes a device for receiving voice or text input data from a user, a device for analyzing the input data and searching a storage device that records information on related equipment, a device for analyzing product evaluation information and quality based on the acquired information and selecting the optimal equipment, and a device for collecting user operation information and adaptively improving search accuracy with a learning model. This makes it possible to quickly provide products suitable for the user and improve the accuracy of suggestions through continuous learning.

[0069] "Users" refer to those who obtain information through the system and select products.

[0070] A "device that accepts input data in the form of voice or text" refers to a device that has the function of receiving information in voice or text format from a user and preparing it for necessary processing.

[0071] A "device that analyzes input data and searches a storage device that records information about related equipment" refers to a device that analyzes received text data and has the function of extracting related equipment information from a database.

[0072] A "device that analyzes product evaluation information and quality to select the optimal equipment" refers to a device equipped with the function to select the equipment that best suits the user's needs based on the acquired evaluation data.

[0073] "A device that generates and provides detailed information and purchase routes to users" refers to a device that has the function of structuring detailed information and purchase methods for selected products and presenting them to users.

[0074] A "device that collects user operation information and adaptively improves search accuracy using a learning model" refers to a device that collects the user's operation history, updates an automated learning model based on that information, and has the function of improving future suggestion accuracy.

[0075] This system is an information processing device that enables users to efficiently search for home appliances and make the best selection. Users can input questions about home appliances using voice or text via devices such as smartphones or personal computers.

[0076] For voice input, the device uses speech recognition software to convert the speech into text data. Google's Speech-to-Text API or similar speech recognition technologies can be used. For text input, the input text is preprocessed using natural language processing software for analysis.

[0077] The processed data is sent to a server via the internet. The server uses natural language processing, implementing large-scale language models (e.g., BERT or GPT-3®), to analyze the received text data. This allows for the rapid extraction of relevant product information from the database.

[0078] The server uses the acquired information to select the most suitable home appliance based on product ratings and reviews. It also utilizes APIs from Amazon and other e-commerce platforms to collect data on product ratings and cost-effectiveness.

[0079] Detailed information and purchase links for selected products are generated by the server and sent to the device. Users can easily compare and purchase products based on the information displayed on their device screen.

[0080] For example, if a user enters "I'm looking for the latest eco-friendly air conditioner," the system will search for air conditioners with high energy efficiency ratings and provide detailed information and a purchase link for the highest-rated product.

[0081] An example of a prompt using a generative AI model is: "List some of the top-rated latest smart refrigerators that the user is interested in, and enumerate the most frequently mentioned advantages from their reviews. In addition, generate purchase links." This is used on the server and is designed to provide more relevant suggestions to the user.

[0082] User interaction history and feedback are used to improve the accuracy of the generated AI model through continuous learning. In this way, the accuracy of suggestions improves over time, supporting users in making satisfactory choices.

[0083] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0084] Step 1:

[0085] Users input questions about home appliances using a device such as a smartphone or computer, either by voice or text. If voice input is used, the input voice data is converted into text format by speech recognition software (e.g., Google Speech-to-Text API) installed on the device. This conversion ensures that the user's request is output as clear text data. For example, the user might say to the device, "I'm looking for an energy-efficient air conditioner."

[0086] Step 2:

[0087] The terminal uses natural language processing software to preprocess text data. For text input, the system performs morphological analysis and text normalization as needed to improve the accuracy of the information. This processed text data is then sent to the server. Specifically, the text "I'm looking for an energy-saving air conditioner," output by speech recognition, is organized with emphasis on the topic words "energy-saving" and "air conditioner."

[0088] Step 3:

[0089] The server uses natural language processing techniques (e.g., BERT, GPT-3) to analyze the received text data. The purpose of the analysis is to identify the home appliance the user is looking for and to search for related product information from a cloud-based database. The input is pre-processed text data, and the output is a list of candidate related products. Specifically, the server extracts product information from the database using the keywords "energy saving" and "air conditioner."

[0090] Step 4:

[0091] The server uses the extracted product list to collect and analyze evaluation information and reviews for each product. Considering evaluation scores and cost-effectiveness, it selects the product that best meets user needs. This analysis utilizes evaluation data obtained from online platform APIs. The output is detailed information about the selected optimal product. Specifically, it collects product evaluations through APIs from Amazon and other online shops.

[0092] Step 5:

[0093] The server generates detailed information about the selected product and creates an information set to provide it to the user. This information set includes product features, price, review links, and purchase links. The generated information set is sent to the terminal. Specifically, it constructs information that includes links to the product page and buttons for purchase.

[0094] Step 6:

[0095] The device displays received information on a user interface, making it easy for the user to understand. After viewing product information, the user can decide whether to purchase the product via the provided link. Specifically, the device displays the product name, price, and rating on the screen, and clearly positions the purchase link.

[0096] Step 7:

[0097] The server collects user feedback and purchase history, and updates the generative AI model based on this data. This allows the system to continuously learn and improve the accuracy of future suggestions. Prompts that leverage the generative AI model include: "List the highest-rated products among the latest home appliances the user is interested in, and generate purchase links." Specific actions include training the model based on the click-through rates and ratings of the suggested products.

[0098] (Application Example 1)

[0099] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0100] In the process of purchasing home appliances, there is a challenge in that it is difficult for users to easily and efficiently select the optimal product and to smoothly complete the purchase procedure based on their selection. Traditional systems require users to manually navigate multiple platforms, which is time-consuming and laborious, and this needs to be resolved.

[0101] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0102] In this invention, the server includes means for receiving voice or text information from the user, means for analyzing the information and searching for a recording medium storing relevant product information, means for analyzing product evaluations based on the acquired information and selecting the optimal product, and means for completing the purchase through electronic payment. This enables the user to efficiently select the optimal home appliance within a single system and complete the purchase immediately.

[0103] A "user" is someone who uses the system to search for information on home appliances, select products, and make purchases.

[0104] "Voice or text information" refers to voice or text-based data that users use to input questions or requests regarding home appliances.

[0105] "Means of receiving information" refers to interfaces or modules that allow a system to receive input from users in the form of voice or text.

[0106] "Means for searching for recording media" refers to the process of searching databases and other information storage means that contain information, based on user requests, and obtaining relevant information.

[0107] "Means for analyzing product evaluations" refers to functions that analyze reviews and evaluations of acquired product data to derive the optimal choice.

[0108] "Methods for completing purchases via electronic payment" refer to systems that allow users to safely and quickly complete the purchase process for selected products through an electronic payment platform.

[0109] A "display device" is a screen or display that allows a user to view information provided by a system.

[0110] This invention provides a system for users to efficiently select and purchase the most suitable home appliances. In this system, a server acts as the central point, processing information through multiple software components. The server runs on an Amazon Web Services (AWS®) EC2 instance. Users input information about home appliances via voice or text using devices such as smartphones or personal computers. This input is received by an application built with React Native. In the case of voice input, the application converts the voice into text data.

[0111] The server uses Hugging Face's Transformers library to analyze the input text data using natural language processing technology. Based on the analyzed data, it searches a database of product information and selects a product that meets the user's requirements. For the selected product, it further analyzes its evaluation and reviews to present the user with the optimal choice.

[0112] Users can view the presented product information within the application and, if they wish to purchase, make electronic payments via the Stripe API. This creates an environment where users can purchase products safely and quickly. In particular, if a user sets specific conditions for purchasing home appliances, the generative AI model will continuously suggest the most suitable products based on those conditions, enabling more personalized selections.

[0113] For example, if a user voice-inputs, "I'm looking for a robot vacuum cleaner for my child's play area. Do you have a suitable model?", the application will recommend highly-rated robot vacuum cleaners on the market and guide the user through the purchase process. Another example of a prompt for the generating AI model is, "Please provide the best options to consider when purchasing home appliances." This system significantly improves the convenience and efficiency of users when purchasing home appliances.

[0114] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0115] Step 1:

[0116] The user inputs information about home appliances into the device via voice or text. The input information is received by an application on the device. If input is via voice, it is converted into text data by a voice recognition function. At this stage, the user's requests or questions are received as input, and text data is obtained as output.

[0117] Step 2:

[0118] The terminal sends the acquired text data to the server. The server receives this data and performs natural language processing using the Hugging Face Transformers library. Here, text data is taken as input, the data is processed to extract product-related keywords, and relevant search queries are output.

[0119] Step 3:

[0120] The server searches a database containing product information based on the search query. This database search uses the generated search query as input and outputs a list of product candidates that meet the user's requirements.

[0121] Step 4:

[0122] The server further analyzes reviews and ratings for each product based on the product candidate list. Using AI-powered data analysis technology, the evaluation data for each product is processed, and the most highly rated and optimal product is selected for recommendation to the user. In this process, the product candidate list is the input, and the highest-rated product is selected as the output.

[0123] Step 5:

[0124] The system generates detailed information about the selected product and provides it to the terminal, including a purchase link. The terminal displays this information on its user interface, allowing the user to review the suggested product. The input is the selected product information, and the output is the detailed information displayed to the user.

[0125] Step 6:

[0126] When a user indicates their intention to purchase a product, the terminal processes the electronic payment via the Stripe API. Communication with the server initiates the purchase process, ensuring secure payment processing. In this process, the input is the user's intention to purchase, and the output is a payment completion notification.

[0127] Step 7:

[0128] The server accumulates user purchase results and feedback, and uses this data as training data for the generated AI model. Based on this information, the AI ​​model is continuously trained to improve the accuracy of its recommendations. The input is purchase data and feedback, and the output is the improved AI model.

[0129] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0130] This invention is a system that utilizes user voice or text input to suggest home appliances best suited to the individual, and in particular, by incorporating an emotion engine, it understands the user's emotional state and enables highly accurate suggestions. First, the user inputs a question about the home appliance they want to search for via voice or text through a terminal such as a smartphone or personal computer. The terminal then converts the voice data into text and sends the necessary data to the server.

[0131] The server analyzes the received data and searches a database of home appliances, where the emotion engine comes into play. The emotion engine estimates the user's emotions from their voice and text input and adjusts the product selection criteria based on that emotion data. Specifically, if the user is excited, the engine will recommend the latest innovative products.

[0132] Furthermore, the server collects online reviews and ratings for the acquired product candidates and performs sentiment analysis of the reviews using natural language processing. This calculates an overall product evaluation score, which is then combined with data obtained from the sentiment engine to select the product that best suits the user's needs and emotions.

[0133] For selected products, detailed information and purchase links are generated and sent to the device. The device formats this information appropriately and displays it clearly for the user. For example, if a user shows interest in a "latest, feature-rich smart refrigerator" but also seems somewhat apprehensive, the server will simultaneously provide detailed usage instructions and support information for the selected product to help with the purchase decision.

[0134] Finally, the server continuously updates its AI model based on user feedback and purchase history to improve the accuracy of its suggestions. This optimizes future suggestions to better match the user's emotional state.

[0135] The following describes the processing flow.

[0136] Step 1:

[0137] Users input their questions using their smartphones or computers via voice or text. For example, they might type, "I'd like to know which smart refrigerators are recommended."

[0138] Step 2:

[0139] If the device uses voice input, it converts it to text using speech recognition technology. If it uses text input, it is retained as text data.

[0140] Step 3:

[0141] The terminal sends text data it has generated or stored to the server. In this case, the data is sent over a network connection.

[0142] Step 4:

[0143] The server activates an emotion engine that analyzes the user's emotional state based on the text data it receives. In the case of voice input, emotion analysis is also performed on the voice.

[0144] Step 5:

[0145] The server incorporates the analysis results from the emotion engine, taking into account the user's emotional data, and searches a database of home appliances to list related products.

[0146] Step 6:

[0147] For the listed products, the server collects online reviews and ratings, performs sentiment analysis based on these, and calculates an overall rating score for each product.

[0148] Step 7:

[0149] The server combines evaluation scores with the user's emotional state to select the most appropriate home appliance. If the user is excited, it selects innovative products; if they are feeling anxious, it selects products with abundant support information.

[0150] Step 8:

[0151] The server generates detailed information and a purchase link for the selected product and sends it to the terminal. The generated information includes product features, price, and retailer information.

[0152] Step 9:

[0153] The device displays the received information to the user, providing the information in an intuitive and easy-to-understand format. The user can then review the details and proceed with the product purchase process via the provided links.

[0154] Step 10:

[0155] The server records user feedback and purchase history, and updates the AI ​​model based on this data. This allows future suggestions to be more adaptive to the user's emotional state.

[0156] (Example 2)

[0157] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0158] Modern consumers often struggle with information overload when trying to choose the product that best suits their needs from a wide variety of options. In addition to this problem, traditional systems that select products without considering the consumer's emotional state have limitations in accuracy. Furthermore, consumer feedback is often not fully utilized, leading to a situation where suggested products gradually fail to meet user expectations.

[0159] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving voice or text input from the user, means for estimating the user's emotional state using emotion analysis means, and means for searching a relevant product database based on the emotional state and selecting a product. This makes it possible to propose products with high accuracy that take the user's emotional state into consideration.

[0160] A "user" is an information provider who provides input using voice or text.

[0161] "Voice or text input" refers to the format of information provided by the user through their device, and includes voice data and text data.

[0162] A "terminal" is a device that receives input from a user and sends it to a server, and includes smartphones and personal computers.

[0163] A "server" is a computer system that analyzes and processes data sent from users.

[0164] "Emotion analysis methods" refer to technologies used to estimate emotions from user input data.

[0165] A "database" is a collection of information in which product-related information is systematically stored and searchable.

[0166] "Natural language processing" is a technology for analyzing language data and processing information.

[0167] "Reviews and ratings" refer to data that includes opinions and evaluations of products provided by consumers on the internet.

[0168] An "AI model" is a collection of algorithms that use machine learning techniques to analyze data and perform inferences and predictions.

[0169] A "user interface" is a means of display and operation that allows a user to directly interact with a system.

[0170] To implement this invention, the user first uses a device such as a smartphone or personal computer to input a question about a home appliance in voice or text. The device then uses voice recognition software to convert the voice data into text data and sends the user's input data to a server.

[0171] The server analyzes the received text data using sentiment analysis technology to estimate the user's emotional state. This analysis uses Python sentiment analysis libraries (e.g., TextBlob or VADER). Based on the data obtained from sentiment analysis, the server searches a relevant product database and selects the most suitable home appliance candidate for the user.

[0172] Next, the server collects reviews and ratings of selected products from the internet and analyzes them using natural language processing (NLP) techniques. This process utilizes Python NLP libraries (e.g., NLTK and spaCy). An overall product evaluation score is calculated from the analysis results, and this is combined with the user's emotional state to select the most suitable product.

[0173] The server generates detailed information and purchase links for the selected product and sends them to the terminal. The terminal displays this information in a format suitable for the user interface and provides it to the user. This may include product images, key specifications, and pricing information.

[0174] Furthermore, the server collects user feedback and purchase history, and updates the generated AI model based on this data. This update improves the accuracy of future suggestions, allowing the system to provide suggestions that are more appropriate to the user's emotional state.

[0175] For example, if a user enters a prompt such as, "I want the latest smart refrigerator with lots of features, but I'm a little unsure if I really need it," the system can take this uncertainty into account and provide detailed usage instructions and support information as well.

[0176] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0177] Step 1:

[0178] Users input questions about home appliances via voice or text into the terminal. The input data might be in the format of, for example, "I want to know what the latest smart refrigerators are like." In the case of voice input, the voice data is converted into text data by voice recognition software within the terminal.

[0179] Step 2:

[0180] The terminal sends the converted text data to the server. Here, the type of data (including text converted from speech) is the input sent to the server, and the output is completed when the server successfully receives the data.

[0181] Step 3:

[0182] The server analyzes the received text data and estimates the user's emotional state using sentiment analysis tools. Specifically, it uses a Python sentiment analysis library to determine the emotion expressed by the input text (e.g., joy, anxiety, excitement). In this process, the text data is converted into sentiment data.

[0183] Step 4:

[0184] The server searches a relevant product database based on the acquired emotional data. Product selection criteria tailored to the user's emotions are applied, and potential product candidates are listed. For example, if the user indicates excitement, the server prioritizes searching for products with the latest technology.

[0185] Step 5:

[0186] The server collects online reviews and rating data for the searched product candidates. It uses a Python web scraping tool to retrieve reviews from specified websites. The collected data is in text format, and a rating score is calculated.

[0187] Step 6:

[0188] The server analyzes the collected reviews using natural language processing techniques and calculates an overall evaluation score for each product. This process utilizes a natural language processing library and includes sentiment analysis of the reviews. The output is compiled as an evaluation score for each product.

[0189] Step 7:

[0190] The server selects the optimal product based on sentiment data and evaluation scores. It generates detailed information and purchase links for the selected product. This generated information is then provided as the final output on the server side.

[0191] Step 8:

[0192] The terminal displays product information received from the server in a format suitable for the user interface. This allows users to easily view detailed information, images, prices, and purchase links. The terminal's output aims to provide information in an easily understandable format.

[0193] Step 9:

[0194] User feedback and purchase data are later sent to the server to update the AI ​​model. This improves the accuracy of future suggestions, enabling more appropriate product recommendations based on user needs and emotions.

[0195] (Application Example 2)

[0196] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0197] In modern homes, a wide variety of home appliances are available, but selecting products that suit the emotional state and needs of individual users is not easy. Furthermore, there is a need for improved accuracy in emotion-based product selection and flexible suggestions that reflect the user's emotional state in real time. However, conventional systems have limited means of effectively achieving this.

[0198] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0199] In this invention, the server includes a device for receiving voice or text input from a user, a device for analyzing the input information and searching for the data structure of related electrical equipment, a device for analyzing product reviews and evaluations based on the acquired information to select the optimal product, a device installed in a robot that uses an emotion engine to detect the user's emotional state from voice input, and a device for generating and providing detailed information and purchase routes for the product to the user. This makes it possible to suggest the most suitable electrical equipment based on the user's emotional state and to continuously improve the accuracy of the suggestion.

[0200] A "device that accepts voice or text input from a user" is an interface that receives voice or text information acquired via an external terminal such as a household robot in an initial stage and converts it into a format that can be analyzed within the system.

[0201] "A device that analyzes the input information and searches for the data structure of related electrical equipment" refers to a function within a system that analyzes the user's input information using natural language processing or the like, and queries a database to search for the relevant electrical equipment.

[0202] A "device that analyzes product reviews and evaluations based on acquired information to select the optimal product" is an algorithm that analyzes user reviews and evaluation data about products collected from the internet, etc., to select the most appropriate product.

[0203] An "emotion engine that detects emotional states from user voice input" is an analytical system that uses speech recognition and natural language processing technologies to determine the user's emotional state and utilize that information for product selection.

[0204] "A device that generates and provides to the user detailed information and purchase routes for the aforementioned product" refers to an output device that generates detailed information and purchase links related to the selected product and provides them in a format that the user can easily access.

[0205] The system for carrying out this invention consists of a user, a terminal, and a server. First, the user inputs a request via voice or text to a home robot or other smart device. This input is converted to text using speech recognition technology and sent to the server via the terminal. Google Cloud Speech-to-Text API or similar technology is used for speech recognition.

[0206] The server analyzes received text data and utilizes an internal database to search for related electrical equipment. Systems such as MySQL® are used for managing this database. The server also analyzes product reviews and ratings collected from the internet through a natural language processing engine. NLTK and other natural language processing libraries are employed for this process.

[0207] The server then runs an emotion engine that uses a generative AI model to analyze the user's emotional state. This engine leverages machine learning libraries such as PyTorch and TENSORFLOW® to estimate emotions by analyzing voice data obtained from the user. In this way, the criteria for identifying products that are suitable for the user's emotions are refined.

[0208] Subsequently, the server generates detailed information and purchase links based on the acquired product information and provides them to the user via the terminal. The user interface utilizes a web configuration or application interface to allow users to easily view the information.

[0209] For example, if a user requests a "new vacuum cleaner" from a home robot, and the voice suggests an intention to relax, the server will recommend a vacuum cleaner with superior quietness. An example of a prompt input to the generating AI model would be, "The user wants to relax, so please recommend a product that prioritizes quietness."

[0210] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0211] Step 1:

[0212] The user communicates their wishes to the home robot using voice. The input is captured as an audio signal by the robot's microphone and converted to text via a voice recognition module. As a result, the audio data is output as text data.

[0213] Step 2:

[0214] The terminal sends text data to the server. The server parses the received text data and understands the user's request through syntactic analysis. This process uses Python's natural language processing library (e.g., NLTK) to identify the user's intent. As a result of the analysis, search keywords related to the user's request are generated.

[0215] Step 3:

[0216] The server searches a database of relevant electrical equipment based on the search keywords. Here, it uses SQL queries to perform database searches and retrieve matching product information. The retrieved information is output as basic product information.

[0217] Step 4:

[0218] The server collects product reviews from the internet based on this product information. It uses web scraping techniques to obtain reviews and ratings related to the product. The collected reviews are output as raw text data.

[0219] Step 5:

[0220] The server uses natural language processing techniques to perform sentiment analysis on the acquired reviews. In this context, the text data of the reviews is input, and positive and negative sentiment scores are calculated using the NLTK library. As a result, the sentiment score for each review is output.

[0221] Step 6:

[0222] The server combines the results of sentiment analysis with the user's emotional state to select the optimal product. This process utilizes a generative AI model to determine emotions from user input and decide on recommended products. The input consists of an emotion score and user intent, and the output is information on recommended products.

[0223] Step 7:

[0224] After the server selects the most suitable product, it generates detailed information and a purchase link for that product, which are then provided to the user via the terminal. The user interface displays this information in an easy-to-understand format, making it easily accessible to the user. In this step, text formatting and the placement of visual elements are performed as concrete actions.

[0225] Step 8:

[0226] When a user provides feedback on a suggested product, the server uses this information as training data to update the AI ​​model. This improves the accuracy of future suggestions. This step is performed as part of the machine learning loop, and as a result, an improvement in the model's accuracy is expected.

[0227] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0228] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0229] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0230] [Second Embodiment]

[0231] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0232] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0233] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0234] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0235] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0236] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0237] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0238] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0239] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0240] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0241] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0242] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0243] This invention provides a system for streamlining the process by which users search for information about home appliances using voice or text. Users can input questions about home appliances using voice or text via a terminal such as a smartphone or personal computer. This input is converted into text data through voice recognition or text analysis on the terminal and transmitted to a server.

[0244] The server analyzes the received input data using natural language processing technology to search for relevant home appliances. From the resulting list of products, it further collects and analyzes reviews and ratings for each product to select the product that best suits the user's needs. This selection process considers the overall product evaluation and cost-effectiveness.

[0245] For selected products, the server generates detailed information and provides it to the user, including a purchase link. The terminal displays this received information on the user's screen, allowing the user to easily understand and make a purchase decision. For example, if the user enters "I want the latest smart refrigerator," the system evaluates several smart refrigerators on the market and displays the user detailed information on the highest-rated model along with a purchase link.

[0246] Furthermore, user behavior data and feedback are accumulated on the server, and the AI ​​model is continuously trained to improve the accuracy of suggestions. This allows users to confidently and efficiently choose home appliances that meet their needs.

[0247] The following describes the processing flow.

[0248] Step 1:

[0249] Users can ask questions about home appliances using their smartphones or computers via voice or text input. For example, they might ask, "Tell me about the latest smart refrigerators."

[0250] Step 2:

[0251] If the device receives voice input, it uses speech recognition technology to convert it into text data. If it receives text input, it is acquired as text data as is.

[0252] Step 3:

[0253] The terminal sends the generated text data to the server. In this case, the data is sent to a server in the cloud via an internet connection.

[0254] Step 4:

[0255] The server analyzes the received text data using natural language processing techniques. Specifically, it extracts keywords from the input questions and performs contextual analysis to understand the user's intent.

[0256] Step 5:

[0257] The server searches the database for a list of related home appliances based on the analysis results. Here, the list of potential home appliances is narrowed down based on the extracted keywords.

[0258] Step 6:

[0259] The server collects reviews and ratings of relevant products from online review sites for a list of home appliances. Furthermore, it uses natural language processing to analyze the sentiment of the reviews and calculate an overall rating score.

[0260] Step 7:

[0261] The server selects the most suitable product for the user based on the calculated evaluation score. Selection criteria include evaluation score, price, and features.

[0262] Step 8:

[0263] The server generates detailed information and purchase links for the selected products. This information includes product features, pricing, and retailer links.

[0264] Step 9:

[0265] The server generates information and sends it to the terminal. The terminal receives this information and displays it in a user-friendly format.

[0266] Step 10:

[0267] Users can review the information displayed on their device and, if necessary, proceed with the purchase process via the displayed link.

[0268] Step 11:

[0269] The server updates the AI ​​model based on user feedback and purchase history. This learning process will improve the accuracy of suggestions in the future.

[0270] (Example 1)

[0271] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0272] Conventional consumer electronics information retrieval systems struggle to quickly and accurately suggest the most suitable products to users. Furthermore, they lack mechanisms for efficiently collecting and utilizing feedback to improve the accuracy of suggestions based on user needs. Therefore, there is a need for the development of a comprehensive system that enhances the user experience.

[0273] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0274] In this invention, the server includes a device for receiving voice or text input data from a user, a device for analyzing the input data and searching a storage device that records information on related equipment, a device for analyzing product evaluation information and quality based on the acquired information and selecting the optimal equipment, and a device for collecting user operation information and adaptively improving search accuracy with a learning model. This makes it possible to quickly provide products suitable for the user and improve the accuracy of suggestions through continuous learning.

[0275] "Users" refer to those who obtain information through the system and select products.

[0276] A "device that accepts input data in the form of voice or text" refers to a device that has the function of receiving information in voice or text format from a user and preparing it for necessary processing.

[0277] A "device that analyzes input data and searches a storage device that records information about related equipment" refers to a device that analyzes received text data and has the function of extracting related equipment information from a database.

[0278] A "device that analyzes product evaluation information and quality to select the optimal equipment" refers to a device equipped with the function to select the equipment that best suits the user's needs based on the acquired evaluation data.

[0279] "A device that generates and provides detailed information and purchase routes to users" refers to a device that has the function of structuring detailed information and purchase methods for selected products and presenting them to users.

[0280] A "device that collects user operation information and adaptively improves search accuracy using a learning model" refers to a device that collects the user's operation history, updates an automated learning model based on that information, and has the function of improving future suggestion accuracy.

[0281] This system is an information processing device that enables users to efficiently search for home appliances and make the best selection. Users can input questions about home appliances using voice or text via devices such as smartphones or personal computers.

[0282] For voice input, the device uses speech recognition software to convert the speech into text data. Google Speech-to-Text API or similar speech recognition technologies can be used. For text input, the input text is preprocessed using natural language processing software for analysis.

[0283] The processed data is sent to the server via the Internet. The server uses natural language processing implemented with a large language model (e.g., BERT or GPT-3) to analyze the received text data. Thereby, relevant product information is quickly extracted from the database.

[0284] Regarding the information obtained in the server, the most suitable household appliances are selected based on product evaluations and reviews. Data on product evaluation scores and cost performance are collected using the APIs of Amazon and other e-commerce platforms.

[0285] Detailed information and purchasable links about the selected products are generated by the server and sent to the terminal. Based on the information displayed on the terminal screen, the user can easily compare and consider products and make purchases.

[0286] As a specific example, when the user inputs "looking for the latest eco-friendly air conditioner", the system searches for air conditioners with high energy efficiency evaluations and provides detailed information and purchase links for the most highly rated products among them.

[0287] Examples of prompt texts using the generative AI model include "List several of the highest-rated products for the latest smart refrigerators in which the user has shown interest, and enumerate the advantages most frequently mentioned in their respective reviews. In addition, generate purchasable links." This is used within the server and is devised to make more appropriate proposals to the user.

[0288] The user's operation history and feedback are used to improve the accuracy of the generative AI model through continuous learning. In this way, the accuracy of the proposals improves over time, supporting the user to make a satisfactory choice.

[0289] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0290] Step 1:

[0291] Users input questions about home appliances using a device such as a smartphone or computer, either by voice or text. If voice input is used, the input voice data is converted into text format by speech recognition software (e.g., Google Speech-to-Text API) installed on the device. This conversion ensures that the user's request is output as clear text data. For example, the user might say to the device, "I'm looking for an energy-efficient air conditioner."

[0292] Step 2:

[0293] The terminal uses natural language processing software to preprocess text data. For text input, the system performs morphological analysis and text normalization as needed to improve the accuracy of the information. This processed text data is then sent to the server. Specifically, the text "I'm looking for an energy-saving air conditioner," output by speech recognition, is organized with emphasis on the topic words "energy-saving" and "air conditioner."

[0294] Step 3:

[0295] The server uses natural language processing techniques (e.g., BERT, GPT-3) to analyze the received text data. The purpose of the analysis is to identify the home appliance the user is looking for and to search for related product information from a cloud-based database. The input is pre-processed text data, and the output is a list of candidate related products. Specifically, the server extracts product information from the database using the keywords "energy saving" and "air conditioner."

[0296] Step 4:

[0297] The server uses the extracted product list to collect and analyze evaluation information and reviews for each product. Considering evaluation scores and cost-effectiveness, it selects the product that best meets user needs. This analysis utilizes evaluation data obtained from online platform APIs. The output is detailed information about the selected optimal product. Specifically, it collects product evaluations through APIs from Amazon and other online shops.

[0298] Step 5:

[0299] The server generates detailed information about the selected product and creates an information set to provide it to the user. This information set includes product features, price, review links, and purchase links. The generated information set is sent to the terminal. Specifically, it constructs information that includes links to the product page and buttons for purchase.

[0300] Step 6:

[0301] The device displays received information on a user interface, making it easy for the user to understand. After viewing product information, the user can decide whether to purchase the product via the provided link. Specifically, the device displays the product name, price, and rating on the screen, and clearly positions the purchase link.

[0302] Step 7:

[0303] The server collects user feedback and purchase history, and updates the generative AI model based on this data. This allows the system to continuously learn and improve the accuracy of future suggestions. Prompts that leverage the generative AI model include: "List the highest-rated products among the latest home appliances the user is interested in, and generate purchase links." Specific actions include training the model based on the click-through rates and ratings of the suggested products.

[0304] (Application Example 1)

[0305] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as a "server", and the smart glasses 214 are referred to as a "terminal".

[0306] In the purchase process of household appliances, there is a problem that it is difficult for users to easily and efficiently select the optimal product and smoothly complete the purchase procedure based on the selection result. In conventional systems, users need to manually move back and forth between multiple platforms, which takes time and effort, so there is a need to solve this problem.

[0307] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0308] In this invention, the server includes means for receiving information from the user in the form of voice or text, means for analyzing the information to search for a recording medium storing related product information, means for analyzing product evaluation based on the acquired information to select the optimal product, and means for completing the purchase through electronic payment. As a result, the user can efficiently select the optimal household appliance within one system and immediately complete the purchase.

[0309] The "user" refers to a person who uses the system to search for information on household appliances, make selections, and perform purchases.

[0310] The "information in the form of voice or text" refers to voice- or character-based data used by the user to input questions or requests regarding household appliances.

[0311] The "means for receiving information" refers to an interface or module for the system to receive input from the user in the form of voice or text.

[0312] "Means for searching for recording media" refers to the process of searching databases and other information storage means that contain information, based on user requests, and obtaining relevant information.

[0313] "Means for analyzing product evaluations" refers to functions that analyze reviews and evaluations of acquired product data to derive the optimal choice.

[0314] "Methods for completing purchases via electronic payment" refer to systems that allow users to safely and quickly complete the purchase process for selected products through an electronic payment platform.

[0315] A "display device" is a screen or display that allows a user to view information provided by a system.

[0316] This invention provides a system for users to efficiently select and purchase the most suitable home appliances. In this system, a server acts as the central point, processing information through multiple software components. The server runs on an Amazon Web Services (AWS) EC2 instance. Users input information about home appliances via voice or text using devices such as smartphones or personal computers. This input is received by an application built with React Native. In the case of voice input, the application converts the voice into text data.

[0317] The server uses Hugging Face's Transformers library to analyze the input text data using natural language processing technology. Based on the analyzed data, it searches a database of product information and selects a product that meets the user's requirements. For the selected product, it further analyzes its evaluation and reviews to present the user with the optimal choice.

[0318] Users can view the presented product information within the application and, if they wish to purchase, make electronic payments via the Stripe API. This creates an environment where users can purchase products safely and quickly. In particular, if a user sets specific conditions for purchasing home appliances, the generative AI model will continuously suggest the most suitable products based on those conditions, enabling more personalized selections.

[0319] For example, if a user voice-inputs, "I'm looking for a robot vacuum cleaner for my child's play area. Do you have a suitable model?", the application will recommend highly-rated robot vacuum cleaners on the market and guide the user through the purchase process. Another example of a prompt for the generating AI model is, "Please provide the best options to consider when purchasing home appliances." This system significantly improves the convenience and efficiency of users when purchasing home appliances.

[0320] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0321] Step 1:

[0322] The user inputs information about home appliances into the device via voice or text. The input information is received by an application on the device. If input is via voice, it is converted into text data by a voice recognition function. At this stage, the user's requests or questions are received as input, and text data is obtained as output.

[0323] Step 2:

[0324] The terminal sends the acquired text data to the server. The server receives this data and performs natural language processing using the Hugging Face Transformers library. Here, text data is taken as input, the data is processed to extract product-related keywords, and relevant search queries are output.

[0325] Step 3:

[0326] The server searches a database containing product information based on the search query. This database search uses the generated search query as input and outputs a list of product candidates that meet the user's requirements.

[0327] Step 4:

[0328] The server further analyzes reviews and ratings for each product based on the product candidate list. Using AI-powered data analysis technology, the evaluation data for each product is processed, and the most highly rated and optimal product is selected for recommendation to the user. In this process, the product candidate list is the input, and the highest-rated product is selected as the output.

[0329] Step 5:

[0330] The system generates detailed information about the selected product and provides it to the terminal, including a purchase link. The terminal displays this information on its user interface, allowing the user to review the suggested product. The input is the selected product information, and the output is the detailed information displayed to the user.

[0331] Step 6:

[0332] When a user indicates their intention to purchase a product, the terminal processes the electronic payment via the Stripe API. Communication with the server initiates the purchase process, ensuring secure payment processing. In this process, the input is the user's intention to purchase, and the output is a payment completion notification.

[0333] Step 7:

[0334] The server accumulates user purchase results and feedback, and uses this data as training data for the generated AI model. Based on this information, the AI ​​model is continuously trained to improve the accuracy of its recommendations. The input is purchase data and feedback, and the output is the improved AI model.

[0335] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0336] This invention is a system that utilizes user voice or text input to suggest home appliances best suited to the individual, and in particular, by incorporating an emotion engine, it understands the user's emotional state and enables highly accurate suggestions. First, the user inputs a question about the home appliance they want to search for via voice or text through a terminal such as a smartphone or personal computer. The terminal then converts the voice data into text and sends the necessary data to the server.

[0337] The server analyzes the received data and searches a database of home appliances, where the emotion engine comes into play. The emotion engine estimates the user's emotions from their voice and text input and adjusts the product selection criteria based on that emotion data. Specifically, if the user is excited, the engine will recommend the latest innovative products.

[0338] Furthermore, the server collects online reviews and ratings for the acquired product candidates and performs sentiment analysis of the reviews using natural language processing. This calculates an overall product evaluation score, which is then combined with data obtained from the sentiment engine to select the product that best suits the user's needs and emotions.

[0339] For selected products, detailed information and purchase links are generated and sent to the device. The device formats this information appropriately and displays it clearly for the user. For example, if a user shows interest in a "latest, feature-rich smart refrigerator" but also seems somewhat apprehensive, the server will simultaneously provide detailed usage instructions and support information for the selected product to help with the purchase decision.

[0340] Finally, the server continuously updates its AI model based on user feedback and purchase history to improve the accuracy of its suggestions. This optimizes future suggestions to better match the user's emotional state.

[0341] The following describes the processing flow.

[0342] Step 1:

[0343] Users input their questions using their smartphones or computers via voice or text. For example, they might type, "I'd like to know which smart refrigerators are recommended."

[0344] Step 2:

[0345] If the device uses voice input, it converts it to text using speech recognition technology. If it uses text input, it is retained as text data.

[0346] Step 3:

[0347] The terminal sends text data it has generated or stored to the server. In this case, the data is sent over a network connection.

[0348] Step 4:

[0349] The server activates an emotion engine that analyzes the user's emotional state based on the text data it receives. In the case of voice input, emotion analysis is also performed on the voice.

[0350] Step 5:

[0351] The server incorporates the analysis results from the emotion engine, taking into account the user's emotional data, and searches a database of home appliances to list related products.

[0352] Step 6:

[0353] For the listed products, the server collects online reviews and ratings, performs sentiment analysis based on these, and calculates an overall rating score for each product.

[0354] Step 7:

[0355] The server combines evaluation scores with the user's emotional state to select the most appropriate home appliance. If the user is excited, it selects innovative products; if they are feeling anxious, it selects products with abundant support information.

[0356] Step 8:

[0357] The server generates detailed information and a purchase link for the selected product and sends it to the terminal. The generated information includes product features, price, and retailer information.

[0358] Step 9:

[0359] The device displays the received information to the user, providing the information in an intuitive and easy-to-understand format. The user can then review the details and proceed with the product purchase process via the provided links.

[0360] Step 10:

[0361] The server records user feedback and purchase history, and updates the AI ​​model based on this data. This allows future suggestions to be more adaptive to the user's emotional state.

[0362] (Example 2)

[0363] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0364] Modern consumers often struggle with information overload when trying to choose the product that best suits their needs from a wide variety of options. In addition to this problem, traditional systems that select products without considering the consumer's emotional state have limitations in accuracy. Furthermore, consumer feedback is often not fully utilized, leading to a situation where suggested products gradually fail to meet user expectations.

[0365] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving voice or text input from the user, means for estimating the user's emotional state using emotion analysis means, and means for searching a relevant product database based on the emotional state and selecting a product. This makes it possible to propose products with high accuracy that take the user's emotional state into consideration.

[0366] A "user" is an information provider who provides input using voice or text.

[0367] "Voice or text input" refers to the format of information provided by the user through their device, and includes voice data and text data.

[0368] A "terminal" is a device that receives input from a user and sends it to a server, and includes smartphones and personal computers.

[0369] A "server" is a computer system that analyzes and processes data sent from users.

[0370] "Emotion analysis methods" refer to technologies used to estimate emotions from user input data.

[0371] A "database" is a collection of information in which product-related information is systematically stored and searchable.

[0372] "Natural language processing" is a technology for analyzing language data and processing information.

[0373] "Reviews and ratings" refer to data that includes opinions and evaluations of products provided by consumers on the internet.

[0374] An "AI model" is a collection of algorithms that use machine learning techniques to analyze data and perform inferences and predictions.

[0375] A "user interface" is a means of display and operation that allows a user to directly interact with a system.

[0376] To implement this invention, the user first uses a device such as a smartphone or personal computer to input a question about a home appliance in voice or text. The device then uses voice recognition software to convert the voice data into text data and sends the user's input data to a server.

[0377] The server analyzes the received text data using sentiment analysis technology to estimate the user's emotional state. This analysis uses Python sentiment analysis libraries (e.g., TextBlob or VADER). Based on the data obtained from sentiment analysis, the server searches a relevant product database and selects the most suitable home appliance candidate for the user.

[0378] Next, the server collects reviews and ratings of selected products from the internet and analyzes them using natural language processing (NLP) techniques. This process utilizes Python NLP libraries (e.g., NLTK and spaCy). An overall product evaluation score is calculated from the analysis results, and this is combined with the user's emotional state to select the most suitable product.

[0379] The server generates detailed information and purchase links for the selected product and sends them to the terminal. The terminal displays this information in a format suitable for the user interface and provides it to the user. This may include product images, key specifications, and pricing information.

[0380] Furthermore, the server collects user feedback and purchase history, and updates the generated AI model based on this data. This update improves the accuracy of future suggestions, allowing the system to provide suggestions that are more appropriate to the user's emotional state.

[0381] For example, if a user enters a prompt such as, "I want the latest smart refrigerator with lots of features, but I'm a little unsure if I really need it," the system can take this uncertainty into account and provide detailed usage instructions and support information as well.

[0382] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0383] Step 1:

[0384] Users input questions about home appliances via voice or text into the terminal. The input data might be in the format of, for example, "I want to know what the latest smart refrigerators are like." In the case of voice input, the voice data is converted into text data by voice recognition software within the terminal.

[0385] Step 2:

[0386] The terminal sends the converted text data to the server. Here, the type of data (including text converted from speech) is the input sent to the server, and the output is completed when the server successfully receives the data.

[0387] Step 3:

[0388] The server analyzes the received text data and estimates the user's emotional state using sentiment analysis tools. Specifically, it uses a Python sentiment analysis library to determine the emotion expressed by the input text (e.g., joy, anxiety, excitement). In this process, the text data is converted into sentiment data.

[0389] Step 4:

[0390] The server searches a relevant product database based on the acquired emotional data. Product selection criteria tailored to the user's emotions are applied, and potential product candidates are listed. For example, if the user indicates excitement, the server prioritizes searching for products with the latest technology.

[0391] Step 5:

[0392] The server collects online reviews and rating data for the searched product candidates. It uses a Python web scraping tool to retrieve reviews from specified websites. The collected data is in text format, and a rating score is calculated.

[0393] Step 6:

[0394] The server analyzes the collected reviews using natural language processing techniques and calculates an overall evaluation score for each product. This process utilizes a natural language processing library and includes sentiment analysis of the reviews. The output is compiled as an evaluation score for each product.

[0395] Step 7:

[0396] The server selects the optimal product based on sentiment data and evaluation scores. It generates detailed information and purchase links for the selected product. This generated information is then provided as the final output on the server side.

[0397] Step 8:

[0398] The terminal displays product information received from the server in a format suitable for the user interface. This allows users to easily view detailed information, images, prices, and purchase links. The terminal's output aims to provide information in an easily understandable format.

[0399] Step 9:

[0400] User feedback and purchase data are later sent to the server to update the AI ​​model. This improves the accuracy of future suggestions, enabling more appropriate product recommendations based on user needs and emotions.

[0401] (Application Example 2)

[0402] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0403] In modern homes, a wide variety of home appliances are available, but selecting products that suit the emotional state and needs of individual users is not easy. Furthermore, there is a need for improved accuracy in emotion-based product selection and flexible suggestions that reflect the user's emotional state in real time. However, conventional systems have limited means of effectively achieving this.

[0404] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0405] In this invention, the server includes a device for receiving voice or text input from a user, a device for analyzing the input information and searching for the data structure of related electrical equipment, a device for analyzing product reviews and evaluations based on the acquired information to select the optimal product, a device installed in a robot that uses an emotion engine to detect the user's emotional state from voice input, and a device for generating and providing detailed information and purchase routes for the product to the user. This makes it possible to suggest the most suitable electrical equipment based on the user's emotional state and to continuously improve the accuracy of the suggestion.

[0406] A "device that accepts voice or text input from a user" is an interface that receives voice or text information acquired via an external terminal such as a household robot in an initial stage and converts it into a format that can be analyzed within the system.

[0407] "A device that analyzes the input information and searches for the data structure of related electrical equipment" refers to a function within a system that analyzes the user's input information using natural language processing or the like, and queries a database to search for the relevant electrical equipment.

[0408] A "device that analyzes product reviews and evaluations based on acquired information to select the optimal product" is an algorithm that analyzes user reviews and evaluation data about products collected from the internet, etc., to select the most appropriate product.

[0409] An "emotion engine that detects emotional states from user voice input" is an analytical system that uses speech recognition and natural language processing technologies to determine the user's emotional state and utilize that information for product selection.

[0410] "A device that generates and provides to the user detailed information and purchase routes for the aforementioned product" refers to an output device that generates detailed information and purchase links related to the selected product and provides them in a format that the user can easily access.

[0411] The system for carrying out this invention consists of a user, a terminal, and a server. First, the user inputs a request via voice or text to a home robot or other smart device. This input is converted to text using speech recognition technology and sent to the server via the terminal. Google Cloud Speech-to-Text API or similar technology is used for speech recognition.

[0412] The server analyzes received text data and utilizes an internal database to search for related electrical equipment. Systems such as MySQL are used for managing this database. The server also analyzes product reviews and ratings collected from the internet through a natural language processing engine. NLTK and other natural language processing libraries are employed for this process.

[0413] The server then runs an emotion engine that uses a generative AI model to analyze the user's emotional state. This engine leverages machine learning libraries such as PyTorch and TensorFlow to estimate emotions by analyzing audio data obtained from the user. In this way, the criteria for identifying products that are appropriate for the user's emotions are refined.

[0414] Subsequently, the server generates detailed information and purchase links based on the acquired product information and provides them to the user via the terminal. The user interface utilizes a web configuration or application interface to allow users to easily view the information.

[0415] For example, if a user requests a "new vacuum cleaner" from a home robot, and the voice suggests an intention to relax, the server will recommend a vacuum cleaner with superior quietness. An example of a prompt input to the generating AI model would be, "The user wants to relax, so please recommend a product that prioritizes quietness."

[0416] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0417] Step 1:

[0418] The user communicates their wishes to the home robot using voice. The input is captured as an audio signal by the robot's microphone and converted to text via a voice recognition module. As a result, the audio data is output as text data.

[0419] Step 2:

[0420] The terminal sends text data to the server. The server parses the received text data and understands the user's request through syntactic analysis. This process uses Python's natural language processing library (e.g., NLTK) to identify the user's intent. As a result of the analysis, search keywords related to the user's request are generated.

[0421] Step 3:

[0422] The server searches a database of relevant electrical equipment based on the search keywords. Here, it uses SQL queries to perform database searches and retrieve matching product information. The retrieved information is output as basic product information.

[0423] Step 4:

[0424] The server collects product reviews from the internet based on this product information. It uses web scraping techniques to obtain reviews and ratings related to the product. The collected reviews are output as raw text data.

[0425] Step 5:

[0426] The server uses natural language processing techniques to perform sentiment analysis on the acquired reviews. In this context, the text data of the reviews is input, and positive and negative sentiment scores are calculated using the NLTK library. As a result, the sentiment score for each review is output.

[0427] Step 6:

[0428] The server combines the results of sentiment analysis with the user's emotional state to select the optimal product. This process utilizes a generative AI model to determine emotions from user input and decide on recommended products. The input consists of an emotion score and user intent, and the output is information on recommended products.

[0429] Step 7:

[0430] After the server selects the most suitable product, it generates detailed information and a purchase link for that product, which are then provided to the user via the terminal. The user interface displays this information in an easy-to-understand format, making it easily accessible to the user. In this step, text formatting and the placement of visual elements are performed as concrete actions.

[0431] Step 8:

[0432] When a user provides feedback on a suggested product, the server uses this information as training data to update the AI ​​model. This improves the accuracy of future suggestions. This step is performed as part of the machine learning loop, and as a result, an improvement in the model's accuracy is expected.

[0433] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0434] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0435] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0436] [Third Embodiment]

[0437] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0438] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0439] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0440] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0441] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0442] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0443] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0444] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0445] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0446] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0447] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0448] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0449] This invention provides a system for streamlining the process by which users search for information about home appliances using voice or text. Users can input questions about home appliances using voice or text via a terminal such as a smartphone or personal computer. This input is converted into text data through voice recognition or text analysis on the terminal and transmitted to a server.

[0450] The server analyzes the received input data using natural language processing technology to search for relevant home appliances. From the resulting list of products, it further collects and analyzes reviews and ratings for each product to select the product that best suits the user's needs. This selection process considers the overall product evaluation and cost-effectiveness.

[0451] For selected products, the server generates detailed information and provides it to the user, including a purchase link. The terminal displays this received information on the user's screen, allowing the user to easily understand and make a purchase decision. For example, if the user enters "I want the latest smart refrigerator," the system evaluates several smart refrigerators on the market and displays the user detailed information on the highest-rated model along with a purchase link.

[0452] Furthermore, user behavior data and feedback are accumulated on the server, and the AI ​​model is continuously trained to improve the accuracy of suggestions. This allows users to confidently and efficiently choose home appliances that meet their needs.

[0453] The following describes the processing flow.

[0454] Step 1:

[0455] Users can ask questions about home appliances using their smartphones or computers via voice or text input. For example, they might ask, "Tell me about the latest smart refrigerators."

[0456] Step 2:

[0457] If the device receives voice input, it uses speech recognition technology to convert it into text data. If it receives text input, it is acquired as text data as is.

[0458] Step 3:

[0459] The terminal sends the generated text data to the server. In this case, the data is sent to a server in the cloud via an internet connection.

[0460] Step 4:

[0461] The server analyzes the received text data using natural language processing techniques. Specifically, it extracts keywords from the input questions and performs contextual analysis to understand the user's intent.

[0462] Step 5:

[0463] The server searches the database for a list of related home appliances based on the analysis results. Here, the list of potential home appliances is narrowed down based on the extracted keywords.

[0464] Step 6:

[0465] The server collects reviews and ratings of relevant products from online review sites for a list of home appliances. Furthermore, it uses natural language processing to analyze the sentiment of the reviews and calculate an overall rating score.

[0466] Step 7:

[0467] The server selects the most suitable product for the user based on the calculated evaluation score. Selection criteria include evaluation score, price, and features.

[0468] Step 8:

[0469] The server generates detailed information and purchase links for the selected products. This information includes product features, pricing, and retailer links.

[0470] Step 9:

[0471] The server generates information and sends it to the terminal. The terminal receives this information and displays it in a user-friendly format.

[0472] Step 10:

[0473] Users can review the information displayed on their device and, if necessary, proceed with the purchase process via the displayed link.

[0474] Step 11:

[0475] The server updates the AI ​​model based on user feedback and purchase history. This learning process will improve the accuracy of suggestions in the future.

[0476] (Example 1)

[0477] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0478] Conventional consumer electronics information retrieval systems struggle to quickly and accurately suggest the most suitable products to users. Furthermore, they lack mechanisms for efficiently collecting and utilizing feedback to improve the accuracy of suggestions based on user needs. Therefore, there is a need for the development of a comprehensive system that enhances the user experience.

[0479] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0480] In this invention, the server includes a device for receiving voice or text input data from a user, a device for analyzing the input data and searching a storage device that records information on related equipment, a device for analyzing product evaluation information and quality based on the acquired information and selecting the optimal equipment, and a device for collecting user operation information and adaptively improving search accuracy with a learning model. This makes it possible to quickly provide products suitable for the user and improve the accuracy of suggestions through continuous learning.

[0481] "Users" refer to those who obtain information through the system and select products.

[0482] A "device that accepts input data in the form of voice or text" refers to a device that has the function of receiving information in voice or text format from a user and preparing it for necessary processing.

[0483] A "device that analyzes input data and searches a storage device that records information about related equipment" refers to a device that analyzes received text data and has the function of extracting related equipment information from a database.

[0484] A "device that analyzes product evaluation information and quality to select the optimal equipment" refers to a device equipped with the function to select the equipment that best suits the user's needs based on the acquired evaluation data.

[0485] "A device that generates and provides detailed information and purchase routes to users" refers to a device that has the function of structuring detailed information and purchase methods for selected products and presenting them to users.

[0486] A "device that collects user operation information and adaptively improves search accuracy using a learning model" refers to a device that collects the user's operation history, updates an automated learning model based on that information, and has the function of improving future suggestion accuracy.

[0487] This system is an information processing device that enables users to efficiently search for home appliances and make the best selection. Users can input questions about home appliances using voice or text via devices such as smartphones or personal computers.

[0488] For voice input, the device uses speech recognition software to convert the speech into text data. Google Speech-to-Text API or similar speech recognition technologies can be used. For text input, the input text is preprocessed using natural language processing software for analysis.

[0489] The processed data is sent to a server via the internet. The server uses natural language processing with large-scale language models (e.g., BERT or GPT-3) to analyze the received text data. This allows for the rapid extraction of relevant product information from the database.

[0490] The server uses the acquired information to select the most suitable home appliance based on product ratings and reviews. It also utilizes APIs from Amazon and other e-commerce platforms to collect data on product ratings and cost-effectiveness.

[0491] Detailed information and purchase links for selected products are generated by the server and sent to the device. Users can easily compare and purchase products based on the information displayed on their device screen.

[0492] For example, if a user enters "I'm looking for the latest eco-friendly air conditioner," the system will search for air conditioners with high energy efficiency ratings and provide detailed information and a purchase link for the highest-rated product.

[0493] An example of a prompt using a generative AI model is: "List some of the top-rated latest smart refrigerators that the user is interested in, and enumerate the most frequently mentioned advantages from their reviews. In addition, generate purchase links." This is used on the server and is designed to provide more relevant suggestions to the user.

[0494] User interaction history and feedback are used to improve the accuracy of the generated AI model through continuous learning. In this way, the accuracy of suggestions improves over time, supporting users in making satisfactory choices.

[0495] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0496] Step 1:

[0497] Users input questions about home appliances using a device such as a smartphone or computer, either by voice or text. If voice input is used, the input voice data is converted into text format by speech recognition software (e.g., Google Speech-to-Text API) installed on the device. This conversion ensures that the user's request is output as clear text data. For example, the user might say to the device, "I'm looking for an energy-efficient air conditioner."

[0498] Step 2:

[0499] The terminal uses natural language processing software to preprocess text data. For text input, the system performs morphological analysis and text normalization as needed to improve the accuracy of the information. This processed text data is then sent to the server. Specifically, the text "I'm looking for an energy-saving air conditioner," output by speech recognition, is organized with emphasis on the topic words "energy-saving" and "air conditioner."

[0500] Step 3:

[0501] The server uses natural language processing techniques (e.g., BERT, GPT-3) to analyze the received text data. The purpose of the analysis is to identify the home appliance the user is looking for and to search for related product information from a cloud-based database. The input is pre-processed text data, and the output is a list of candidate related products. Specifically, the server extracts product information from the database using the keywords "energy saving" and "air conditioner."

[0502] Step 4:

[0503] The server uses the extracted product list to collect and analyze evaluation information and reviews for each product. Considering evaluation scores and cost-effectiveness, it selects the product that best meets user needs. This analysis utilizes evaluation data obtained from online platform APIs. The output is detailed information about the selected optimal product. Specifically, it collects product evaluations through APIs from Amazon and other online shops.

[0504] Step 5:

[0505] The server generates detailed information about the selected product and creates an information set to provide it to the user. This information set includes product features, price, review links, and purchase links. The generated information set is sent to the terminal. Specifically, it constructs information that includes links to the product page and buttons for purchase.

[0506] Step 6:

[0507] The device displays received information on a user interface, making it easy for the user to understand. After viewing product information, the user can decide whether to purchase the product via the provided link. Specifically, the device displays the product name, price, and rating on the screen, and clearly positions the purchase link.

[0508] Step 7:

[0509] The server collects user feedback and purchase history, and updates the generative AI model based on this data. This allows the system to continuously learn and improve the accuracy of future suggestions. Prompts that leverage the generative AI model include: "List the highest-rated products among the latest home appliances the user is interested in, and generate purchase links." Specific actions include training the model based on the click-through rates and ratings of the suggested products.

[0510] (Application Example 1)

[0511] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0512] In the process of purchasing home appliances, there is a challenge in that it is difficult for users to easily and efficiently select the optimal product and to smoothly complete the purchase procedure based on their selection. Traditional systems require users to manually navigate multiple platforms, which is time-consuming and laborious, and this needs to be resolved.

[0513] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0514] In this invention, the server includes means for receiving voice or text information from the user, means for analyzing the information and searching for a recording medium storing relevant product information, means for analyzing product evaluations based on the acquired information and selecting the optimal product, and means for completing the purchase through electronic payment. This enables the user to efficiently select the optimal home appliance within a single system and complete the purchase immediately.

[0515] A "user" is someone who uses the system to search for information on home appliances, select products, and make purchases.

[0516] "Voice or text information" refers to voice or text-based data that users use to input questions or requests regarding home appliances.

[0517] "Means of receiving information" refers to interfaces or modules that allow a system to receive input from users in the form of voice or text.

[0518] "Means for searching for recording media" refers to the process of searching databases and other information storage means that contain information, based on user requests, and obtaining relevant information.

[0519] "Means for analyzing product evaluations" refers to functions that analyze reviews and evaluations of acquired product data to derive the optimal choice.

[0520] "Methods for completing purchases via electronic payment" refer to systems that allow users to safely and quickly complete the purchase process for selected products through an electronic payment platform.

[0521] A "display device" is a screen or display that allows a user to view information provided by a system.

[0522] This invention provides a system for users to efficiently select and purchase the most suitable home appliances. In this system, a server acts as the central point, processing information through multiple software components. The server runs on an Amazon Web Services (AWS) EC2 instance. Users input information about home appliances via voice or text using devices such as smartphones or personal computers. This input is received by an application built with React Native. In the case of voice input, the application converts the voice into text data.

[0523] The server uses Hugging Face's Transformers library to analyze the input text data using natural language processing technology. Based on the analyzed data, it searches a database of product information and selects a product that meets the user's requirements. For the selected product, it further analyzes its evaluation and reviews to present the user with the optimal choice.

[0524] Users can view the presented product information within the application and, if they wish to purchase, make electronic payments via the Stripe API. This creates an environment where users can purchase products safely and quickly. In particular, if a user sets specific conditions for purchasing home appliances, the generative AI model will continuously suggest the most suitable products based on those conditions, enabling more personalized selections.

[0525] For example, if a user voice-inputs, "I'm looking for a robot vacuum cleaner for my child's play area. Do you have a suitable model?", the application will recommend highly-rated robot vacuum cleaners on the market and guide the user through the purchase process. Another example of a prompt for the generating AI model is, "Please provide the best options to consider when purchasing home appliances." This system significantly improves the convenience and efficiency of users when purchasing home appliances.

[0526] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0527] Step 1:

[0528] The user inputs information about home appliances into the device via voice or text. The input information is received by an application on the device. If input is via voice, it is converted into text data by a voice recognition function. At this stage, the user's requests or questions are received as input, and text data is obtained as output.

[0529] Step 2:

[0530] The terminal sends the acquired text data to the server. The server receives this data and performs natural language processing using the Hugging Face Transformers library. Here, text data is taken as input, the data is processed to extract product-related keywords, and relevant search queries are output.

[0531] Step 3:

[0532] The server searches a database containing product information based on the search query. This database search uses the generated search query as input and outputs a list of product candidates that meet the user's requirements.

[0533] Step 4:

[0534] The server further analyzes reviews and ratings for each product based on the product candidate list. Using AI-powered data analysis technology, the evaluation data for each product is processed, and the most highly rated and optimal product is selected for recommendation to the user. In this process, the product candidate list is the input, and the highest-rated product is selected as the output.

[0535] Step 5:

[0536] The system generates detailed information about the selected product and provides it to the terminal, including a purchase link. The terminal displays this information on its user interface, allowing the user to review the suggested product. The input is the selected product information, and the output is the detailed information displayed to the user.

[0537] Step 6:

[0538] When a user indicates their intention to purchase a product, the terminal processes the electronic payment via the Stripe API. Communication with the server initiates the purchase process, ensuring secure payment processing. In this process, the input is the user's intention to purchase, and the output is a payment completion notification.

[0539] Step 7:

[0540] The server accumulates user purchase results and feedback, and uses this data as training data for the generated AI model. Based on this information, the AI ​​model is continuously trained to improve the accuracy of its recommendations. The input is purchase data and feedback, and the output is the improved AI model.

[0541] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0542] This invention is a system that utilizes user voice or text input to suggest home appliances best suited to the individual, and in particular, by incorporating an emotion engine, it understands the user's emotional state and enables highly accurate suggestions. First, the user inputs a question about the home appliance they want to search for via voice or text through a terminal such as a smartphone or personal computer. The terminal then converts the voice data into text and sends the necessary data to the server.

[0543] The server analyzes the received data and searches a database of home appliances, where the emotion engine comes into play. The emotion engine estimates the user's emotions from their voice and text input and adjusts the product selection criteria based on that emotion data. Specifically, if the user is excited, the engine will recommend the latest innovative products.

[0544] Furthermore, the server collects online reviews and ratings for the acquired product candidates and performs sentiment analysis of the reviews using natural language processing. This calculates an overall product evaluation score, which is then combined with data obtained from the sentiment engine to select the product that best suits the user's needs and emotions.

[0545] For selected products, detailed information and purchase links are generated and sent to the device. The device formats this information appropriately and displays it clearly for the user. For example, if a user shows interest in a "latest, feature-rich smart refrigerator" but also seems somewhat apprehensive, the server will simultaneously provide detailed usage instructions and support information for the selected product to help with the purchase decision.

[0546] Finally, the server continuously updates its AI model based on user feedback and purchase history to improve the accuracy of its suggestions. This optimizes future suggestions to better match the user's emotional state.

[0547] The following describes the processing flow.

[0548] Step 1:

[0549] Users input their questions using their smartphones or computers via voice or text. For example, they might type, "I'd like to know which smart refrigerators are recommended."

[0550] Step 2:

[0551] If the device uses voice input, it converts it to text using speech recognition technology. If it uses text input, it is retained as text data.

[0552] Step 3:

[0553] The terminal sends text data it has generated or stored to the server. In this case, the data is sent over a network connection.

[0554] Step 4:

[0555] The server activates an emotion engine that analyzes the user's emotional state based on the text data it receives. In the case of voice input, emotion analysis is also performed on the voice.

[0556] Step 5:

[0557] The server incorporates the analysis results from the emotion engine, taking into account the user's emotional data, and searches a database of home appliances to list related products.

[0558] Step 6:

[0559] For the listed products, the server collects online reviews and ratings, performs sentiment analysis based on these, and calculates an overall rating score for each product.

[0560] Step 7:

[0561] The server combines evaluation scores with the user's emotional state to select the most appropriate home appliance. If the user is excited, it selects innovative products; if they are feeling anxious, it selects products with abundant support information.

[0562] Step 8:

[0563] The server generates detailed information and a purchase link for the selected product and sends it to the terminal. The generated information includes product features, price, and retailer information.

[0564] Step 9:

[0565] The device displays the received information to the user, providing the information in an intuitive and easy-to-understand format. The user can then review the details and proceed with the product purchase process via the provided links.

[0566] Step 10:

[0567] The server records user feedback and purchase history, and updates the AI ​​model based on this data. This allows future suggestions to be more adaptive to the user's emotional state.

[0568] (Example 2)

[0569] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0570] Modern consumers often struggle with information overload when trying to choose the product that best suits their needs from a wide variety of options. In addition to this problem, traditional systems that select products without considering the consumer's emotional state have limitations in accuracy. Furthermore, consumer feedback is often not fully utilized, leading to a situation where suggested products gradually fail to meet user expectations.

[0571] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving voice or text input from the user, means for estimating the user's emotional state using emotion analysis means, and means for searching a relevant product database based on the emotional state and selecting a product. This makes it possible to propose products with high accuracy that take the user's emotional state into consideration.

[0572] A "user" is an information provider who provides input using voice or text.

[0573] "Voice or text input" refers to the format of information provided by the user through their device, and includes voice data and text data.

[0574] A "terminal" is a device that receives input from a user and sends it to a server, and includes smartphones and personal computers.

[0575] A "server" is a computer system that analyzes and processes data sent from users.

[0576] "Emotion analysis methods" refer to technologies used to estimate emotions from user input data.

[0577] A "database" is a collection of information in which product-related information is systematically stored and searchable.

[0578] "Natural language processing" is a technology for analyzing language data and processing information.

[0579] "Reviews and ratings" refer to data that includes opinions and evaluations of products provided by consumers on the internet.

[0580] An "AI model" is a collection of algorithms that use machine learning techniques to analyze data and perform inferences and predictions.

[0581] A "user interface" is a means of display and operation that allows a user to directly interact with a system.

[0582] To implement this invention, the user first uses a device such as a smartphone or personal computer to input a question about a home appliance in voice or text. The device then uses voice recognition software to convert the voice data into text data and sends the user's input data to a server.

[0583] The server analyzes the received text data using sentiment analysis technology to estimate the user's emotional state. This analysis uses Python sentiment analysis libraries (e.g., TextBlob or VADER). Based on the data obtained from sentiment analysis, the server searches a relevant product database and selects the most suitable home appliance candidate for the user.

[0584] Next, the server collects reviews and ratings of selected products from the internet and analyzes them using natural language processing (NLP) techniques. This process utilizes Python NLP libraries (e.g., NLTK and spaCy). An overall product evaluation score is calculated from the analysis results, and this is combined with the user's emotional state to select the most suitable product.

[0585] The server generates detailed information and purchase links for the selected product and sends them to the terminal. The terminal displays this information in a format suitable for the user interface and provides it to the user. This may include product images, key specifications, and pricing information.

[0586] Furthermore, the server collects user feedback and purchase history, and updates the generated AI model based on this data. This update improves the accuracy of future suggestions, allowing the system to provide suggestions that are more appropriate to the user's emotional state.

[0587] For example, if a user enters a prompt such as, "I want the latest smart refrigerator with lots of features, but I'm a little unsure if I really need it," the system can take this uncertainty into account and provide detailed usage instructions and support information as well.

[0588] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0589] Step 1:

[0590] Users input questions about home appliances via voice or text into the terminal. The input data might be in the format of, for example, "I want to know what the latest smart refrigerators are like." In the case of voice input, the voice data is converted into text data by voice recognition software within the terminal.

[0591] Step 2:

[0592] The terminal sends the converted text data to the server. Here, the type of data (including text converted from speech) is the input sent to the server, and the output is completed when the server successfully receives the data.

[0593] Step 3:

[0594] The server analyzes the received text data and estimates the user's emotional state using sentiment analysis tools. Specifically, it uses a Python sentiment analysis library to determine the emotion expressed by the input text (e.g., joy, anxiety, excitement). In this process, the text data is converted into sentiment data.

[0595] Step 4:

[0596] The server searches a relevant product database based on the acquired emotional data. Product selection criteria tailored to the user's emotions are applied, and potential product candidates are listed. For example, if the user indicates excitement, the server prioritizes searching for products with the latest technology.

[0597] Step 5:

[0598] The server collects online reviews and rating data for the searched product candidates. It uses a Python web scraping tool to retrieve reviews from specified websites. The collected data is in text format, and a rating score is calculated.

[0599] Step 6:

[0600] The server analyzes the collected reviews using natural language processing techniques and calculates an overall evaluation score for each product. This process utilizes a natural language processing library and includes sentiment analysis of the reviews. The output is compiled as an evaluation score for each product.

[0601] Step 7:

[0602] The server selects the optimal product based on sentiment data and evaluation scores. It generates detailed information and purchase links for the selected product. This generated information is then provided as the final output on the server side.

[0603] Step 8:

[0604] The terminal displays product information received from the server in a format suitable for the user interface. This allows users to easily view detailed information, images, prices, and purchase links. The terminal's output aims to provide information in an easily understandable format.

[0605] Step 9:

[0606] User feedback and purchase data are later sent to the server to update the AI ​​model. This improves the accuracy of future suggestions, enabling more appropriate product recommendations based on user needs and emotions.

[0607] (Application Example 2)

[0608] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0609] In modern homes, a wide variety of home appliances are available, but selecting products that suit the emotional state and needs of individual users is not easy. Furthermore, there is a need for improved accuracy in emotion-based product selection and flexible suggestions that reflect the user's emotional state in real time. However, conventional systems have limited means of effectively achieving this.

[0610] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0611] In this invention, the server includes a device for receiving voice or text input from a user, a device for analyzing the input information and searching for the data structure of related electrical equipment, a device for analyzing product reviews and evaluations based on the acquired information to select the optimal product, a device installed in a robot that uses an emotion engine to detect the user's emotional state from voice input, and a device for generating and providing detailed information and purchase routes for the product to the user. This makes it possible to suggest the most suitable electrical equipment based on the user's emotional state and to continuously improve the accuracy of the suggestion.

[0612] A "device that accepts voice or text input from a user" is an interface that receives voice or text information acquired via an external terminal such as a household robot in an initial stage and converts it into a format that can be analyzed within the system.

[0613] "A device that analyzes the input information and searches for the data structure of related electrical equipment" refers to a function within a system that analyzes the user's input information using natural language processing or the like, and queries a database to search for the relevant electrical equipment.

[0614] A "device that analyzes product reviews and evaluations based on acquired information to select the optimal product" is an algorithm that analyzes user reviews and evaluation data about products collected from the internet, etc., to select the most appropriate product.

[0615] An "emotion engine that detects emotional states from user voice input" is an analytical system that uses speech recognition and natural language processing technologies to determine the user's emotional state and utilize that information for product selection.

[0616] "A device that generates and provides to the user detailed information and purchase routes for the aforementioned product" refers to an output device that generates detailed information and purchase links related to the selected product and provides them in a format that the user can easily access.

[0617] The system for carrying out this invention consists of a user, a terminal, and a server. First, the user inputs a request via voice or text to a home robot or other smart device. This input is converted to text using speech recognition technology and sent to the server via the terminal. Google Cloud Speech-to-Text API or similar technology is used for speech recognition.

[0618] The server analyzes received text data and utilizes an internal database to search for related electrical equipment. Systems such as MySQL are used for managing this database. The server also analyzes product reviews and ratings collected from the internet through a natural language processing engine. NLTK and other natural language processing libraries are employed for this process.

[0619] The server then runs an emotion engine that uses a generative AI model to analyze the user's emotional state. This engine leverages machine learning libraries such as PyTorch and TensorFlow to estimate emotions by analyzing audio data obtained from the user. In this way, the criteria for identifying products that are appropriate for the user's emotions are refined.

[0620] Subsequently, the server generates detailed information and purchase links based on the acquired product information and provides them to the user via the terminal. The user interface utilizes a web configuration or application interface to allow users to easily view the information.

[0621] For example, if a user requests a "new vacuum cleaner" from a home robot, and the voice suggests an intention to relax, the server will recommend a vacuum cleaner with superior quietness. An example of a prompt input to the generating AI model would be, "The user wants to relax, so please recommend a product that prioritizes quietness."

[0622] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0623] Step 1:

[0624] The user communicates their wishes to the home robot using voice. The input is captured as an audio signal by the robot's microphone and converted to text via a voice recognition module. As a result, the audio data is output as text data.

[0625] Step 2:

[0626] The terminal sends text data to the server. The server parses the received text data and understands the user's request through syntactic analysis. This process uses Python's natural language processing library (e.g., NLTK) to identify the user's intent. As a result of the analysis, search keywords related to the user's request are generated.

[0627] Step 3:

[0628] The server searches a database of relevant electrical equipment based on the search keywords. Here, it uses SQL queries to perform database searches and retrieve matching product information. The retrieved information is output as basic product information.

[0629] Step 4:

[0630] The server collects product reviews from the internet based on this product information. It uses web scraping techniques to obtain reviews and ratings related to the product. The collected reviews are output as raw text data.

[0631] Step 5:

[0632] The server uses natural language processing techniques to perform sentiment analysis on the acquired reviews. In this context, the text data of the reviews is input, and positive and negative sentiment scores are calculated using the NLTK library. As a result, the sentiment score for each review is output.

[0633] Step 6:

[0634] The server combines the results of sentiment analysis with the user's emotional state to select the optimal product. This process utilizes a generative AI model to determine emotions from user input and decide on recommended products. The input consists of an emotion score and user intent, and the output is information on recommended products.

[0635] Step 7:

[0636] After the server selects the most suitable product, it generates detailed information and a purchase link for that product, which are then provided to the user via the terminal. The user interface displays this information in an easy-to-understand format, making it easily accessible to the user. In this step, text formatting and the placement of visual elements are performed as concrete actions.

[0637] Step 8:

[0638] When a user provides feedback on a suggested product, the server uses this information as training data to update the AI ​​model. This improves the accuracy of future suggestions. This step is performed as part of the machine learning loop, and as a result, an improvement in the model's accuracy is expected.

[0639] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0640] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0641] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0642] [Fourth Embodiment]

[0643] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0644] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0645] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0646] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0647] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0648] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0649] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0650] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0651] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0652] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0653] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0654] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0655] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0656] This invention provides a system for streamlining the process by which users search for information about home appliances using voice or text. Users can input questions about home appliances using voice or text via a terminal such as a smartphone or personal computer. This input is converted into text data through voice recognition or text analysis on the terminal and transmitted to a server.

[0657] The server analyzes the received input data using natural language processing technology to search for relevant home appliances. From the resulting list of products, it further collects and analyzes reviews and ratings for each product to select the product that best suits the user's needs. This selection process considers the overall product evaluation and cost-effectiveness.

[0658] For selected products, the server generates detailed information and provides it to the user, including a purchase link. The terminal displays this received information on the user's screen, allowing the user to easily understand and make a purchase decision. For example, if the user enters "I want the latest smart refrigerator," the system evaluates several smart refrigerators on the market and displays the user detailed information on the highest-rated model along with a purchase link.

[0659] Furthermore, user behavior data and feedback are accumulated on the server, and the AI ​​model is continuously trained to improve the accuracy of suggestions. This allows users to confidently and efficiently choose home appliances that meet their needs.

[0660] The following describes the processing flow.

[0661] Step 1:

[0662] Users can ask questions about home appliances using their smartphones or computers via voice or text input. For example, they might ask, "Tell me about the latest smart refrigerators."

[0663] Step 2:

[0664] If the device receives voice input, it uses speech recognition technology to convert it into text data. If it receives text input, it is acquired as text data as is.

[0665] Step 3:

[0666] The terminal sends the generated text data to the server. In this case, the data is sent to a server in the cloud via an internet connection.

[0667] Step 4:

[0668] The server analyzes the received text data using natural language processing techniques. Specifically, it extracts keywords from the input questions and performs contextual analysis to understand the user's intent.

[0669] Step 5:

[0670] The server searches the database for a list of related home appliances based on the analysis results. Here, the list of potential home appliances is narrowed down based on the extracted keywords.

[0671] Step 6:

[0672] The server collects reviews and ratings of relevant products from online review sites for a list of home appliances. Furthermore, it uses natural language processing to analyze the sentiment of the reviews and calculate an overall rating score.

[0673] Step 7:

[0674] The server selects the most suitable product for the user based on the calculated evaluation score. Selection criteria include evaluation score, price, and features.

[0675] Step 8:

[0676] The server generates detailed information and purchase links for the selected products. This information includes product features, pricing, and retailer links.

[0677] Step 9:

[0678] The server generates information and sends it to the terminal. The terminal receives this information and displays it in a user-friendly format.

[0679] Step 10:

[0680] Users can review the information displayed on their device and, if necessary, proceed with the purchase process via the displayed link.

[0681] Step 11:

[0682] The server updates the AI ​​model based on user feedback and purchase history. This learning process will improve the accuracy of suggestions in the future.

[0683] (Example 1)

[0684] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0685] Conventional consumer electronics information retrieval systems struggle to quickly and accurately suggest the most suitable products to users. Furthermore, they lack mechanisms for efficiently collecting and utilizing feedback to improve the accuracy of suggestions based on user needs. Therefore, there is a need for the development of a comprehensive system that enhances the user experience.

[0686] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0687] In this invention, the server includes a device for receiving voice or text input data from a user, a device for analyzing the input data and searching a storage device that records information on related equipment, a device for analyzing product evaluation information and quality based on the acquired information and selecting the optimal equipment, and a device for collecting user operation information and adaptively improving search accuracy with a learning model. This makes it possible to quickly provide products suitable for the user and improve the accuracy of suggestions through continuous learning.

[0688] "Users" refer to those who obtain information through the system and select products.

[0689] A "device that accepts input data in the form of voice or text" refers to a device that has the function of receiving information in voice or text format from a user and preparing it for necessary processing.

[0690] A "device that analyzes input data and searches a storage device that records information about related equipment" refers to a device that analyzes received text data and has the function of extracting related equipment information from a database.

[0691] A "device that analyzes product evaluation information and quality to select the optimal equipment" refers to a device equipped with the function to select the equipment that best suits the user's needs based on the acquired evaluation data.

[0692] "A device that generates and provides detailed information and purchase routes to users" refers to a device that has the function of structuring detailed information and purchase methods for selected products and presenting them to users.

[0693] A "device that collects user operation information and adaptively improves search accuracy using a learning model" refers to a device that collects the user's operation history, updates an automated learning model based on that information, and has the function of improving future suggestion accuracy.

[0694] This system is an information processing device that enables users to efficiently search for home appliances and make the best selection. Users can input questions about home appliances using voice or text via devices such as smartphones or personal computers.

[0695] For voice input, the device uses speech recognition software to convert the speech into text data. Google Speech-to-Text API or similar speech recognition technologies can be used. For text input, the input text is preprocessed using natural language processing software for analysis.

[0696] The processed data is sent to a server via the internet. The server uses natural language processing with large-scale language models (e.g., BERT or GPT-3) to analyze the received text data. This allows for the rapid extraction of relevant product information from the database.

[0697] The server uses the acquired information to select the most suitable home appliance based on product ratings and reviews. It also utilizes APIs from Amazon and other e-commerce platforms to collect data on product ratings and cost-effectiveness.

[0698] Detailed information and purchase links for selected products are generated by the server and sent to the device. Users can easily compare and purchase products based on the information displayed on their device screen.

[0699] For example, if a user enters "I'm looking for the latest eco-friendly air conditioner," the system will search for air conditioners with high energy efficiency ratings and provide detailed information and a purchase link for the highest-rated product.

[0700] An example of a prompt using a generative AI model is: "List some of the top-rated latest smart refrigerators that the user is interested in, and enumerate the most frequently mentioned advantages from their reviews. In addition, generate purchase links." This is used on the server and is designed to provide more relevant suggestions to the user.

[0701] User interaction history and feedback are used to improve the accuracy of the generated AI model through continuous learning. In this way, the accuracy of suggestions improves over time, supporting users in making satisfactory choices.

[0702] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0703] Step 1:

[0704] Users input questions about home appliances using a device such as a smartphone or computer, either by voice or text. If voice input is used, the input voice data is converted into text format by speech recognition software (e.g., Google Speech-to-Text API) installed on the device. This conversion ensures that the user's request is output as clear text data. For example, the user might say to the device, "I'm looking for an energy-efficient air conditioner."

[0705] Step 2:

[0706] The terminal uses natural language processing software to preprocess text data. For text input, the system performs morphological analysis and text normalization as needed to improve the accuracy of the information. This processed text data is then sent to the server. Specifically, the text "I'm looking for an energy-saving air conditioner," output by speech recognition, is organized with emphasis on the topic words "energy-saving" and "air conditioner."

[0707] Step 3:

[0708] The server uses natural language processing techniques (e.g., BERT, GPT-3) to analyze the received text data. The purpose of the analysis is to identify the home appliance the user is looking for and to search for related product information from a cloud-based database. The input is pre-processed text data, and the output is a list of candidate related products. Specifically, the server extracts product information from the database using the keywords "energy saving" and "air conditioner."

[0709] Step 4:

[0710] The server uses the extracted product list to collect and analyze evaluation information and reviews for each product. Considering evaluation scores and cost-effectiveness, it selects the product that best meets user needs. This analysis utilizes evaluation data obtained from online platform APIs. The output is detailed information about the selected optimal product. Specifically, it collects product evaluations through APIs from Amazon and other online shops.

[0711] Step 5:

[0712] The server generates detailed information about the selected product and creates an information set to provide it to the user. This information set includes product features, price, review links, and purchase links. The generated information set is sent to the terminal. Specifically, it constructs information that includes links to the product page and buttons for purchase.

[0713] Step 6:

[0714] The device displays received information on a user interface, making it easy for the user to understand. After viewing product information, the user can decide whether to purchase the product via the provided link. Specifically, the device displays the product name, price, and rating on the screen, and clearly positions the purchase link.

[0715] Step 7:

[0716] The server collects user feedback and purchase history, and updates the generative AI model based on this data. This allows the system to continuously learn and improve the accuracy of future suggestions. Prompts that leverage the generative AI model include: "List the highest-rated products among the latest home appliances the user is interested in, and generate purchase links." Specific actions include training the model based on the click-through rates and ratings of the suggested products.

[0717] (Application Example 1)

[0718] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0719] In the process of purchasing home appliances, there is a challenge in that it is difficult for users to easily and efficiently select the optimal product and to smoothly complete the purchase procedure based on their selection. Traditional systems require users to manually navigate multiple platforms, which is time-consuming and laborious, and this needs to be resolved.

[0720] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0721] In this invention, the server includes means for receiving voice or text information from the user, means for analyzing the information and searching for a recording medium storing relevant product information, means for analyzing product evaluations based on the acquired information and selecting the optimal product, and means for completing the purchase through electronic payment. This enables the user to efficiently select the optimal home appliance within a single system and complete the purchase immediately.

[0722] A "user" is someone who uses the system to search for information on home appliances, select products, and make purchases.

[0723] "Voice or text information" refers to voice or text-based data that users use to input questions or requests regarding home appliances.

[0724] "Means of receiving information" refers to interfaces or modules that allow a system to receive input from users in the form of voice or text.

[0725] "Means for searching for recording media" refers to the process of searching databases and other information storage means that contain information, based on user requests, and obtaining relevant information.

[0726] "Means for analyzing product evaluations" refers to functions that analyze reviews and evaluations of acquired product data to derive the optimal choice.

[0727] "Methods for completing purchases via electronic payment" refer to systems that allow users to safely and quickly complete the purchase process for selected products through an electronic payment platform.

[0728] A "display device" is a screen or display that allows a user to view information provided by a system.

[0729] This invention provides a system for users to efficiently select and purchase the most suitable home appliances. In this system, a server acts as the central point, processing information through multiple software components. The server runs on an Amazon Web Services (AWS) EC2 instance. Users input information about home appliances via voice or text using devices such as smartphones or personal computers. This input is received by an application built with React Native. In the case of voice input, the application converts the voice into text data.

[0730] The server uses Hugging Face's Transformers library to analyze the input text data using natural language processing technology. Based on the analyzed data, it searches a database of product information and selects a product that meets the user's requirements. For the selected product, it further analyzes its evaluation and reviews to present the user with the optimal choice.

[0731] Users can view the presented product information within the application and, if they wish to purchase, make electronic payments via the Stripe API. This creates an environment where users can purchase products safely and quickly. In particular, if a user sets specific conditions for purchasing home appliances, the generative AI model will continuously suggest the most suitable products based on those conditions, enabling more personalized selections.

[0732] For example, if a user voice-inputs, "I'm looking for a robot vacuum cleaner for my child's play area. Do you have a suitable model?", the application will recommend highly-rated robot vacuum cleaners on the market and guide the user through the purchase process. Another example of a prompt for the generating AI model is, "Please provide the best options to consider when purchasing home appliances." This system significantly improves the convenience and efficiency of users when purchasing home appliances.

[0733] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0734] Step 1:

[0735] The user inputs information about home appliances into the device via voice or text. The input information is received by an application on the device. If input is via voice, it is converted into text data by a voice recognition function. At this stage, the user's requests or questions are received as input, and text data is obtained as output.

[0736] Step 2:

[0737] The terminal sends the acquired text data to the server. The server receives this data and performs natural language processing using the Hugging Face Transformers library. Here, text data is taken as input, the data is processed to extract product-related keywords, and relevant search queries are output.

[0738] Step 3:

[0739] The server searches a database containing product information based on the search query. This database search uses the generated search query as input and outputs a list of product candidates that meet the user's requirements.

[0740] Step 4:

[0741] The server further analyzes reviews and ratings for each product based on the product candidate list. Using AI-powered data analysis technology, the evaluation data for each product is processed, and the most highly rated and optimal product is selected for recommendation to the user. In this process, the product candidate list is the input, and the highest-rated product is selected as the output.

[0742] Step 5:

[0743] The system generates detailed information about the selected product and provides it to the terminal, including a purchase link. The terminal displays this information on its user interface, allowing the user to review the suggested product. The input is the selected product information, and the output is the detailed information displayed to the user.

[0744] Step 6:

[0745] When a user indicates their intention to purchase a product, the terminal processes the electronic payment via the Stripe API. Communication with the server initiates the purchase process, ensuring secure payment processing. In this process, the input is the user's intention to purchase, and the output is a payment completion notification.

[0746] Step 7:

[0747] The server accumulates user purchase results and feedback, and uses this data as training data for the generated AI model. Based on this information, the AI ​​model is continuously trained to improve the accuracy of its recommendations. The input is purchase data and feedback, and the output is the improved AI model.

[0748] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0749] This invention is a system that utilizes user voice or text input to suggest home appliances best suited to the individual, and in particular, by incorporating an emotion engine, it understands the user's emotional state and enables highly accurate suggestions. First, the user inputs a question about the home appliance they want to search for via voice or text through a terminal such as a smartphone or personal computer. The terminal then converts the voice data into text and sends the necessary data to the server.

[0750] The server analyzes the received data and searches a database of home appliances, where the emotion engine comes into play. The emotion engine estimates the user's emotions from their voice and text input and adjusts the product selection criteria based on that emotion data. Specifically, if the user is excited, the engine will recommend the latest innovative products.

[0751] Furthermore, the server collects online reviews and ratings for the acquired product candidates and performs sentiment analysis of the reviews using natural language processing. This calculates an overall product evaluation score, which is then combined with data obtained from the sentiment engine to select the product that best suits the user's needs and emotions.

[0752] For selected products, detailed information and purchase links are generated and sent to the device. The device formats this information appropriately and displays it clearly for the user. For example, if a user shows interest in a "latest, feature-rich smart refrigerator" but also seems somewhat apprehensive, the server will simultaneously provide detailed usage instructions and support information for the selected product to help with the purchase decision.

[0753] Finally, the server continuously updates its AI model based on user feedback and purchase history to improve the accuracy of its suggestions. This optimizes future suggestions to better match the user's emotional state.

[0754] The following describes the processing flow.

[0755] Step 1:

[0756] Users input their questions using their smartphones or computers via voice or text. For example, they might type, "I'd like to know which smart refrigerators are recommended."

[0757] Step 2:

[0758] If the device uses voice input, it converts it to text using speech recognition technology. If it uses text input, it is retained as text data.

[0759] Step 3:

[0760] The terminal sends text data it has generated or stored to the server. In this case, the data is sent over a network connection.

[0761] Step 4:

[0762] The server activates an emotion engine that analyzes the user's emotional state based on the text data it receives. In the case of voice input, emotion analysis is also performed on the voice.

[0763] Step 5:

[0764] The server incorporates the analysis results from the emotion engine, taking into account the user's emotional data, and searches a database of home appliances to list related products.

[0765] Step 6:

[0766] For the listed products, the server collects online reviews and ratings, performs sentiment analysis based on these, and calculates an overall rating score for each product.

[0767] Step 7:

[0768] The server combines evaluation scores with the user's emotional state to select the most appropriate home appliance. If the user is excited, it selects innovative products; if they are feeling anxious, it selects products with abundant support information.

[0769] Step 8:

[0770] The server generates detailed information and a purchase link for the selected product and sends it to the terminal. The generated information includes product features, price, and retailer information.

[0771] Step 9:

[0772] The device displays the received information to the user, providing the information in an intuitive and easy-to-understand format. The user can then review the details and proceed with the product purchase process via the provided links.

[0773] Step 10:

[0774] The server records user feedback and purchase history, and updates the AI ​​model based on this data. This allows future suggestions to be more adaptive to the user's emotional state.

[0775] (Example 2)

[0776] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0777] Modern consumers often struggle with information overload when trying to choose the product that best suits their needs from a wide variety of options. In addition to this problem, traditional systems that select products without considering the consumer's emotional state have limitations in accuracy. Furthermore, consumer feedback is often not fully utilized, leading to a situation where suggested products gradually fail to meet user expectations.

[0778] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving voice or text input from the user, means for estimating the user's emotional state using emotion analysis means, and means for searching a relevant product database based on the emotional state and selecting a product. This makes it possible to propose products with high accuracy that take the user's emotional state into consideration.

[0779] A "user" is an information provider who provides input using voice or text.

[0780] "Voice or text input" refers to the format of information provided by the user through their device, and includes voice data and text data.

[0781] A "terminal" is a device that receives input from a user and sends it to a server, and includes smartphones and personal computers.

[0782] A "server" is a computer system that analyzes and processes data sent from users.

[0783] "Emotion analysis methods" refer to technologies used to estimate emotions from user input data.

[0784] A "database" is a collection of information in which product-related information is systematically stored and searchable.

[0785] "Natural language processing" is a technology for analyzing language data and processing information.

[0786] "Reviews and ratings" refer to data that includes opinions and evaluations of products provided by consumers on the internet.

[0787] An "AI model" is a collection of algorithms that use machine learning techniques to analyze data and perform inferences and predictions.

[0788] A "user interface" is a means of display and operation that allows a user to directly interact with a system.

[0789] To implement this invention, the user first uses a device such as a smartphone or personal computer to input a question about a home appliance in voice or text. The device then uses voice recognition software to convert the voice data into text data and sends the user's input data to a server.

[0790] The server analyzes the received text data using sentiment analysis technology to estimate the user's emotional state. This analysis uses Python sentiment analysis libraries (e.g., TextBlob or VADER). Based on the data obtained from sentiment analysis, the server searches a relevant product database and selects the most suitable home appliance candidate for the user.

[0791] Next, the server collects reviews and ratings of selected products from the internet and analyzes them using natural language processing (NLP) techniques. This process utilizes Python NLP libraries (e.g., NLTK and spaCy). An overall product evaluation score is calculated from the analysis results, and this is combined with the user's emotional state to select the most suitable product.

[0792] The server generates detailed information and purchase links for the selected product and sends them to the terminal. The terminal displays this information in a format suitable for the user interface and provides it to the user. This may include product images, key specifications, and pricing information.

[0793] Furthermore, the server collects user feedback and purchase history, and updates the generated AI model based on this data. This update improves the accuracy of future suggestions, allowing the system to provide suggestions that are more appropriate to the user's emotional state.

[0794] For example, if a user enters a prompt such as, "I want the latest smart refrigerator with lots of features, but I'm a little unsure if I really need it," the system can take this uncertainty into account and provide detailed usage instructions and support information as well.

[0795] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0796] Step 1:

[0797] Users input questions about home appliances via voice or text into the terminal. The input data might be in the format of, for example, "I want to know what the latest smart refrigerators are like." In the case of voice input, the voice data is converted into text data by voice recognition software within the terminal.

[0798] Step 2:

[0799] The terminal sends the converted text data to the server. Here, the type of data (including text converted from speech) is the input sent to the server, and the output is completed when the server successfully receives the data.

[0800] Step 3:

[0801] The server analyzes the received text data and estimates the user's emotional state using sentiment analysis tools. Specifically, it uses a Python sentiment analysis library to determine the emotion expressed by the input text (e.g., joy, anxiety, excitement). In this process, the text data is converted into sentiment data.

[0802] Step 4:

[0803] The server searches a relevant product database based on the acquired emotional data. Product selection criteria tailored to the user's emotions are applied, and potential product candidates are listed. For example, if the user indicates excitement, the server prioritizes searching for products with the latest technology.

[0804] Step 5:

[0805] The server collects online reviews and rating data for the searched product candidates. It uses a Python web scraping tool to retrieve reviews from specified websites. The collected data is in text format, and a rating score is calculated.

[0806] Step 6:

[0807] The server analyzes the collected reviews using natural language processing techniques and calculates an overall evaluation score for each product. This process utilizes a natural language processing library and includes sentiment analysis of the reviews. The output is compiled as an evaluation score for each product.

[0808] Step 7:

[0809] The server selects the optimal product based on sentiment data and evaluation scores. It generates detailed information and purchase links for the selected product. This generated information is then provided as the final output on the server side.

[0810] Step 8:

[0811] The terminal displays product information received from the server in a format suitable for the user interface. This allows users to easily view detailed information, images, prices, and purchase links. The terminal's output aims to provide information in an easily understandable format.

[0812] Step 9:

[0813] User feedback and purchase data are later sent to the server to update the AI ​​model. This improves the accuracy of future suggestions, enabling more appropriate product recommendations based on user needs and emotions.

[0814] (Application Example 2)

[0815] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0816] In modern homes, a wide variety of home appliances are available, but selecting products that suit the emotional state and needs of individual users is not easy. Furthermore, there is a need for improved accuracy in emotion-based product selection and flexible suggestions that reflect the user's emotional state in real time. However, conventional systems have limited means of effectively achieving this.

[0817] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0818] In this invention, the server includes a device for receiving voice or text input from a user, a device for analyzing the input information and searching for the data structure of related electrical equipment, a device for analyzing product reviews and evaluations based on the acquired information to select the optimal product, a device installed in a robot that uses an emotion engine to detect the user's emotional state from voice input, and a device for generating and providing detailed information and purchase routes for the product to the user. This makes it possible to suggest the most suitable electrical equipment based on the user's emotional state and to continuously improve the accuracy of the suggestion.

[0819] A "device that accepts voice or text input from a user" is an interface that receives voice or text information acquired via an external terminal such as a household robot in an initial stage and converts it into a format that can be analyzed within the system.

[0820] "A device that analyzes the input information and searches for the data structure of related electrical equipment" refers to a function within a system that analyzes the user's input information using natural language processing or the like, and queries a database to search for the relevant electrical equipment.

[0821] A "device that analyzes product reviews and evaluations based on acquired information to select the optimal product" is an algorithm that analyzes user reviews and evaluation data about products collected from the internet, etc., to select the most appropriate product.

[0822] An "emotion engine that detects emotional states from user voice input" is an analytical system that uses speech recognition and natural language processing technologies to determine the user's emotional state and utilize that information for product selection.

[0823] "A device that generates and provides to the user detailed information and purchase routes for the aforementioned product" refers to an output device that generates detailed information and purchase links related to the selected product and provides them in a format that the user can easily access.

[0824] The system for carrying out this invention consists of a user, a terminal, and a server. First, the user inputs a request via voice or text to a home robot or other smart device. This input is converted to text using speech recognition technology and sent to the server via the terminal. Google Cloud Speech-to-Text API or similar technology is used for speech recognition.

[0825] The server analyzes received text data and utilizes an internal database to search for related electrical equipment. Systems such as MySQL are used for managing this database. The server also analyzes product reviews and ratings collected from the internet through a natural language processing engine. NLTK and other natural language processing libraries are employed for this process.

[0826] The server then runs an emotion engine that uses a generative AI model to analyze the user's emotional state. This engine leverages machine learning libraries such as PyTorch and TensorFlow to estimate emotions by analyzing audio data obtained from the user. In this way, the criteria for identifying products that are appropriate for the user's emotions are refined.

[0827] Subsequently, the server generates detailed information and purchase links based on the acquired product information and provides them to the user via the terminal. The user interface utilizes a web configuration or application interface to allow users to easily view the information.

[0828] For example, if a user requests a "new vacuum cleaner" from a home robot, and the voice suggests an intention to relax, the server will recommend a vacuum cleaner with superior quietness. An example of a prompt input to the generating AI model would be, "The user wants to relax, so please recommend a product that prioritizes quietness."

[0829] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0830] Step 1:

[0831] The user communicates their wishes to the home robot using voice. The input is captured as an audio signal by the robot's microphone and converted to text via a voice recognition module. As a result, the audio data is output as text data.

[0832] Step 2:

[0833] The terminal sends text data to the server. The server parses the received text data and understands the user's request through syntactic analysis. This process uses Python's natural language processing library (e.g., NLTK) to identify the user's intent. As a result of the analysis, search keywords related to the user's request are generated.

[0834] Step 3:

[0835] The server searches a database of relevant electrical equipment based on the search keywords. Here, it uses SQL queries to perform database searches and retrieve matching product information. The retrieved information is output as basic product information.

[0836] Step 4:

[0837] The server collects product reviews from the internet based on this product information. It uses web scraping techniques to obtain reviews and ratings related to the product. The collected reviews are output as raw text data.

[0838] Step 5:

[0839] The server uses natural language processing techniques to perform sentiment analysis on the acquired reviews. In this context, the text data of the reviews is input, and positive and negative sentiment scores are calculated using the NLTK library. As a result, the sentiment score for each review is output.

[0840] Step 6:

[0841] The server combines the results of sentiment analysis with the user's emotional state to select the optimal product. This process utilizes a generative AI model to determine emotions from user input and decide on recommended products. The input consists of an emotion score and user intent, and the output is information on recommended products.

[0842] Step 7:

[0843] After the server selects the most suitable product, it generates detailed information and a purchase link for that product, which are then provided to the user via the terminal. The user interface displays this information in an easy-to-understand format, making it easily accessible to the user. In this step, text formatting and the placement of visual elements are performed as concrete actions.

[0844] Step 8:

[0845] When a user provides feedback on a suggested product, the server uses this information as training data to update the AI ​​model. This improves the accuracy of future suggestions. This step is performed as part of the machine learning loop, and as a result, an improvement in the model's accuracy is expected.

[0846] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0847] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0848] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0849] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0850] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0851] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0852] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0853] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0854] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0855] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0856] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0857] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0858] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0859] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0860] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0861] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0862] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0863] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0864] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0865] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0866] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0867] The following is further disclosed regarding the embodiments described above.

[0868] (Claim 1)

[0869] A means of receiving voice or text input from the user,

[0870] A means for analyzing the aforementioned input data and searching a database of related home appliances,

[0871] A method for selecting the optimal product by analyzing product reviews and evaluations based on acquired data,

[0872] A means for generating and providing to the user detailed information and purchase links for the aforementioned product,

[0873] A system that includes this.

[0874] (Claim 2)

[0875] The system according to claim 1, further comprising means for updating the AI ​​model based on user feedback and purchase data to continuously improve the accuracy of suggestions.

[0876] (Claim 3)

[0877] The system according to claim 1, further comprising means for displaying the recommendation results received by the terminal on a user interface and presenting them as information accessible to the user.

[0878] "Example 1"

[0879] (Claim 1)

[0880] A device that accepts voice or text input data from users,

[0881] A device that analyzes the aforementioned input data and searches for a storage device that records information about related equipment,

[0882] A device that analyzes product evaluation information and quality based on acquired information to select the optimal equipment,

[0883] A device that generates and provides to users detailed information and purchase routes for the aforementioned equipment,

[0884] A device that collects user operation information and adaptively improves search accuracy using a learning model,

[0885] A mechanism that includes this.

[0886] (Claim 2)

[0887] The mechanism according to claim 1, comprising a device for updating a model generated based on user behavior information and transaction information, thereby continuously improving the accuracy of recommendations.

[0888] (Claim 3)

[0889] The mechanism according to claim 1, further comprising a device that displays the recommendation information received by the device on a user operation screen and presents it to the user as information available to the user.

[0890] "Application Example 1"

[0891] (Claim 1)

[0892] A means of receiving information from users via voice or text,

[0893] A means for analyzing the aforementioned information and searching for a recording medium that stores related product information,

[0894] A method for selecting the optimal product by analyzing product evaluations based on acquired information,

[0895] A means for generating and providing to the user detailed information and means for purchasing the said product,

[0896] A means of completing a purchase through electronic payment,

[0897] A system that includes this.

[0898] (Claim 2)

[0899] The system according to claim 1, further comprising means for improving the artificial intelligence model based on the user's evaluation and purchase history, and for continuously improving the accuracy of suggestions.

[0900] (Claim 3)

[0901] The system according to claim 1, further comprising means for displaying the recommendation results received by the terminal on a display device and presenting them as information accessible to the user.

[0902] "Example 2 of combining an emotion engine"

[0903] (Claim 1)

[0904] A means of receiving voice or text input from the user,

[0905] A means for analyzing the input data and estimating the user's emotional state using emotion analysis means,

[0906] A means for searching a relevant product database based on the aforementioned emotional state and selecting a product,

[0907] A means for collecting data from the internet and analyzing product reviews and evaluations using natural language processing,

[0908] A means for generating and providing to the user detailed information and purchase links for the aforementioned product,

[0909] A system that includes this.

[0910] (Claim 2)

[0911] The system according to claim 1, further comprising means for updating the AI ​​model based on user feedback and purchase data to continuously improve the accuracy of suggestions.

[0912] (Claim 3)

[0913] The system according to claim 1, further comprising means for displaying the recommendation results received by the terminal on a user interface and presenting them as information accessible to the user.

[0914] "Application example 2 when combining with an emotional engine"

[0915] (Claim 1)

[0916] A device that accepts voice or text input from a user,

[0917] A device that analyzes the aforementioned input information and searches for the data structure of related electrical equipment,

[0918] A device that analyzes product reviews and evaluations based on acquired information to select the optimal product,

[0919] A device that uses an emotion engine installed in a robot to detect the user's emotional state from voice input,

[0920] A device that generates and provides to the user detailed information and purchase route for the aforementioned product,

[0921] A system that includes this.

[0922] (Claim 2)

[0923] The system according to claim 1, further comprising a device that updates a learning model based on user feedback and purchase history to continuously improve the accuracy of suggestions.

[0924] (Claim 3)

[0925] The system according to claim 1, further comprising a device that displays the recommendation results received by the robot on a user interface and presents them as information accessible to the user. [Explanation of symbols]

[0926] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of receiving voice or text input from the user, A means for analyzing the aforementioned input data and searching a database of related home appliances, A method for selecting the optimal product by analyzing product reviews and evaluations based on acquired data, A means for generating and providing to the user detailed information and purchase links for the aforementioned product, A system that includes this.

2. The system according to claim 1, further comprising means for updating the AI ​​model based on user feedback and purchase data to continuously improve the accuracy of suggestions.

3. The system according to claim 1, further comprising means for displaying the recommendation results received by the terminal on a user interface and presenting them as information accessible to the user.

Images