System

Abstract

A system is provided.SOLUTION: A system comprising: means for receiving data collected from an IoT appliance; means for analyzing the data using a generative AI to recommend an optimal product to a user; and means for presenting the recommended product to the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Today's consumers lead busy lives and require technology that supports their lives efficiently. Furthermore, users are often dissatisfied with irrelevant product recommendations and the use of data provided. The e-commerce industry is also seeking new business opportunities through IoT in an effort to increase sales, but effectively integrating these new technologies is challenging. Therefore, the objective of this invention is to efficiently link IoT home appliances with e-commerce sites, realize product recommendations that meet users' needs, and improve the efficiency of their lives and the purchasing experience. [Means for solving the problem]

[0005] This invention is a system that includes a means for receiving data collected from IoT home appliances, a means for analyzing the data using a generation AI to recommend optimal products to users, and a means for presenting the recommended products to users. Specifically, the system collects data such as the operating status, usage mode, and frequency of use of IoT home appliances, analyzes it using a generation AI, and recommends products that suit the user's lifestyle and needs. Furthermore, the system analyzes and suggests products based on the stored data in response to recommendation requests from users, thereby improving user convenience and increasing sales in the e-commerce industry.

[0006] "IoT home appliances" are home appliances that can send and receive data and be operated via the Internet.

[0007] "Data" refers to information about usage and behavior patterns generated by IoT appliances.

[0008] "Generative AI" is a system that uses artificial intelligence technology to analyze data and derive results that are suitable for a specific purpose.

[0009] "Analysis" refers to the process of analyzing collected data in detail and clarifying its meaning and relationships.

[0010] "User" refers to an individual who uses IoT home appliances and is the target of product recommendations.

[0011] "Product recommendations" are a series of products suggested for purchase based on a user's usage data.

[0012] A "recommendation request" is a request from a user to a server for product suggestions.

[0013] "Storage" refers to keeping collected data in a state that it can be used in the future.

[0014] "Suggestion" means showing appropriate products and services to users based on the analysis results.

[0015] An "EC site" is a website for selling and purchasing products over the Internet. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] This embodiment of the present invention describes a system that uses data collected from IoT home appliances to implement product recommendations using generative AI. Below, the program processing flow and specific examples are explained in natural language.

[0038] Program processing

[0039] 1. Collecting and transmitting IoT home appliance data:

[0040] When a user uses an IoT home appliance, its usage data is collected. This data includes, for example, the operating status and usage mode of a washing machine. This data is acquired by the device and sent to a server.

[0041] 2. Receiving and storing data:

[0042] The server receives the IoT home appliance data sent from the device. The received data is stored in a database along with the user ID. The database stores information such as each user's device usage history and operation patterns.

[0043] 3. Receiving User Requests:

[0044] When a user attempts to purchase a product on an e-commerce site, a recommendation request is sent to the server, including the user ID.

[0045] 4. Data analysis and recommendation generation:

[0046] The server searches the stored device data and retrieves the relevant data based on the user ID. Generative AI is used to analyze the retrieved data and identify the most suitable products for the user. For example, if a user uses their washing machine frequently, it will recommend a detergent that suits their lifestyle.

[0047] 5. Presenting recommendations:

[0048] The server returns the generated recommendation results to the user. The recommendation includes a list of related products and detailed information. The user can then view the recommended products through their device and consider purchasing them.

[0049] Specific examples

[0050] A specific example is given below.

[0051] A user uses an IoT home appliance (e.g., a washing machine) on a daily basis. The usage data (e.g., using the washing machine five times a week) is collected by the device and sent to a server.

[0052] The server receives this data and stores it in a database along with the user ID.

[0053] Later, when the same user tries to purchase detergent on an e-commerce site, a recommendation request is sent to the server.

[0054] The server uses generative AI to analyze the stored data and recommend the best products for each user, such as highly concentrated detergents or environmentally friendly detergents.

[0055] The server returns the generated recommendation results to the user, and the user can check the recommended products and consider purchasing them on the EC site.

[0056] This will enable users to find products efficiently and conveniently, providing opportunities for increased sales in the e-commerce industry, and will also lead to increased user satisfaction for IoT home appliance manufacturers, strengthening the ecosystem as a whole.

[0057] The processing flow will be explained below.

[0058] Step 1:

[0059] When a user uses an IoT appliance, for example, a washing machine, the operating status, mode of use, and duration of use are recorded.

[0060] Step 2:

[0061] The device collects usage data for IoT appliances, including washing machine usage data, which includes detailed information such as the number of times it has been used, the duration of its use, and the mode of use.

[0062] Step 3:

[0063] The device sends the collected data to the server. The data sent is in the form of a user ID and device data, for example in JSON format as shown below.

[0064] json

[0065] {

[0066] "user_id": "12345",

[0067] "device_data": {

[0068] "washing_machine": {

[0069] "usage_count": 5,

[0070] "total_runtime_minutes": 300,

[0071] "modes_used": ["normal", "quick"]

[0072] }

[0073] }

[0074] }

[0075] Step 4:

[0076] The server receives the data sent from the device, stores it in a database in an appropriate format, and uses it for later analysis.

[0077] Step 5:

[0078] When a user wants to purchase a product on an e-commerce site, they send a recommendation request to the server via their device. The request includes the user ID and asks the server to suggest products.

[0079] Step 6:

[0080] When the server receives a user request, it searches the stored database for data for the corresponding user ID, for example, to retrieve washing machine usage data.

[0081] Step 7:

[0082] The server analyzes the data using generative AI. It analyzes the user's usage in detail and identifies the most suitable product for the user. For example, it identifies highly concentrated detergents and environmentally friendly detergents for users who use their washing machine frequently.

[0083] Step 8:

[0084] The server lists the products identified by the AI ​​and presents them to the user as recommendations, including the product name, description, and purchase link.

[0085] Step 9:

[0086] The user can check the recommended products through their device, consider purchasing the products based on this information, and proceed with the purchase process on the e-commerce site.

[0087] Step 10:

[0088] The server continues to store the user's purchase history in a database to help inform future recommendations. This continuous data collection and analysis allows for continuous improvement of the user experience.

[0089] Example 1

[0090] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0091] Today's consumers have diverse needs and spend a lot of time and effort selecting products. In particular, with the spread of IoT devices, there is a demand for utilizing the large amount of usage data obtained from these devices to propose individually optimized products. However, conventional systems face the challenge of effectively analyzing this data and recommending the best products for users. Furthermore, while analysis technology using generative AI is rapidly evolving, there are few systems that can properly utilize it.

[0092] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0093] In this invention, the server includes means for collecting operation and usage data from IoT devices used by users and transmitting it via the terminal, means for receiving the data transmitted from the terminal and storing it in a database together with user identification information, means for receiving a recommendation request when the user attempts to purchase a product, means for using a generative AI model to analyze the stored data and identify the product best suited to the user, and means for presenting the generated recommendation results to the user. This makes it possible to efficiently analyze large amounts of IoT device data and quickly and accurately provide the best product to each user.

[0094] "User" refers to a consumer who uses IoT devices on a daily basis and whose usage data is collected and analyzed.

[0095] "IoT devices" refers to home appliances and other electronic devices that are connected to the Internet and can automatically send and receive data.

[0096] "Operational and usage data" refers to information about how a user uses a particular device, such as the device's operating status, usage mode, and frequency of use.

[0097] "Terminal" refers to a device that transmits data collected from IoT devices to a server. Specifically, this includes smartphones and smart home hubs.

[0098] "Server" refers to a central computing unit that receives, stores, analyzes, and processes data over a network.

[0099] "Database" means a system running on a Server for structuring and storing Usage Data, including User Identification Information.

[0100] "User identification information" refers to information for identifying a specific user, and includes, for example, a user ID.

[0101] "Recommendation Request" refers to a request sent by a user to a server to request recommendations for the best products for a particular product category.

[0102] "Generative AI model" refers to an artificial intelligence model used to analyze collected data and identify the best products for users.

[0103] A "prompt" is a text-based instruction entered into a generative AI model that clearly states the data to be analyzed and the purpose.

[0104] "Recommendation Results" refers to the optimal product suggestions presented to the user as a result of analysis by the generative AI model.

[0105] This invention relates to a system that uses data collected from users' IoT devices to realize product recommendations using a generative AI model. This system can perform all processes from data collection to analysis and product proposals. The program processing flow is explained in detail below.

[0106] Collecting and transmitting IoT home appliance data

[0107] When a user uses an IoT device (e.g., a washing machine or refrigerator), data on its operation and usage is collected. This data includes, for example, its operating status, usage mode, and frequency of use. This data is acquired by a device (e.g., a smartphone or smart home hub). The device then transmits the collected data to a server via Wi-Fi or a mobile network.

[0108] Receiving and storing data

[0109] The server receives the data sent from the device. This process is performed through a secure API endpoint, ensuring safe data transmission. The received data is then stored in a database along with user identification information. A NoSQL database (e.g., MongoDB) is suitable for this database, as it is characterized by its scalability and flexibility.

[0110] Receiving a user request

[0111] When a user attempts to purchase a product on an e-commerce site, they send a recommendation request. This request is sent from the device (for example, a browser on a PC or smartphone) to the server. This request includes the user ID and the product category they are considering purchasing. The HTTPS protocol is used for communication to ensure data security.

[0112] Data analysis and recommendation generation

[0113] The server processes the received request and searches for device usage data stored in a database. It retrieves the relevant data based on the user's identity and inputs a prompt statement into the generative AI model. An example of a prompt statement would be, "Recommend the best detergent based on the usage data of user ID: 12345." The generative AI model uses text analysis and machine learning algorithms to analyze the user's behavioral patterns and identify the best product.

[0114] Presenting recommendations

[0115] The server receives the recommendation results obtained from the generative AI model and presents them to the user. These results include a list of related products and their detailed information. For example, an e-commerce site in the browser displays images, descriptions, prices, etc. of the recommended products. The user can review these and consider purchasing them.

[0116] Specific examples

[0117] Specific examples are shown below.

[0118] Suppose that usage data for an IoT device (e.g., a washing machine) that a user uses daily is collected five times a week.

[0119] This usage data includes information such as time and mode of use, and is periodically sent to a server and stored in a database.

[0120] Later, when the same user tries to purchase detergent on an e-commerce site, they send a recommendation request to the server.

[0121] Based on the stored data, the server sends a prompt to the generated AI, such as "Recommend the optimal detergent based on the usage data of user ID: 12345."

[0122] The generative AI model will return results that recommend, for example, "highly concentrated detergents and environmentally friendly detergents."

[0123] The results are presented to the user on their browser, allowing them to consider purchasing based on the recommended products.

[0124] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0125] Step 1:

[0126] When a user uses an IoT device, data on its operation and usage status is collected by the device. Specific data includes operating time, usage mode, and frequency of use. This data is acquired in real time and saved in JSON format. Sensor data and usage logs from the IoT device operated by the user are used as input. This data is temporarily saved in the device as output. Specific operations include reading data from sensors, converting it into a specific format, and saving it.

[0127] Step 2:

[0128] The device transmits the collected data to the server at regular intervals. This transmission process uses Wi-Fi or mobile networks and uses a secure communication protocol (e.g., HTTPS) to ensure data security. As input, the usage data collected in step 1 is in JSON format. As output, the data is sent to and received from the server. Specific operations include encoding, packaging, and sending the data to the server.

[0129] Step 3:

[0130] The server receives data sent from the device and stores it in a database. This data contains user identification information, so it is stored and associated with a specific user. The usage data sent from the device is used as input. The received data is stored in the database as output. Specific operations include receiving data at an API endpoint, decoding the data, and inserting it into the database.

[0131] Step 4:

[0132] When a user attempts to purchase a product on an e-commerce website, a recommendation request is sent to the server. The request includes the user ID and the desired product category. The request data from the user is used as input. The request content is received and saved on the server as output. Specific operations include collecting form data, encoding the data, and sending it to the server.

[0133] Step 5:

[0134] Based on the received request, the server retrieves relevant device usage data from the database. It also creates and sends a prompt statement to the generative AI model based on this data. The user's usage data stored in the database and the request content are used as input. The prompt statement is sent to the generative AI model as output. An example of a prompt statement is "Recommend the optimal detergent based on the usage data of user ID: 12345." Specific operations include generating and executing a database query, formatting the retrieved data, and sending it to the generative AI model.

[0135] Step 6:

[0136] The generative AI model analyzes data based on the received prompt and identifies the most suitable products. It uses machine learning algorithms to analyze user behavior patterns and product characteristics to recommend the most suitable products. The prompt is used as input. As output, a list of the most suitable products is generated as the recommendation result. Specific operations include data analysis using natural language processing and machine learning algorithms, and the generation of recommendation results.

[0137] Step 7:

[0138] The server receives the recommendation results obtained from the generative AI model and presents them to the user. These results include a list of related products and their details. The recommendation results from the generative AI model are used as input. The output provides a list of products and their details to be displayed to the user. Specific operations include receiving the result data, converting the data format, and displaying it on the user's device.

[0139] This allows users to efficiently find the product that best suits them and consider purchasing it.

[0140] (Application example 1)

[0141] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0142] In recent years, the spread of IoT home appliances has made users' daily lives increasingly convenient. However, there is still a lack of a system that can effectively utilize the vast amount of data from IoT home appliances to suggest products that are suitable for each individual user. In addition, there is a lack of means to provide timely and accurate product recommendations to users. As a result, users are faced with the problem of spending time and effort finding the products they need.

[0143] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0144] In this invention, the server includes means for receiving data collected from IoT home appliances, means for analyzing the data using a generating AI and recommending optimal products to the user, means for presenting the recommended products to the user, and means for automatically recommending products based on the data collected from the IoT home appliances and providing them as push notifications to a smartphone. This enables users to efficiently select products by utilizing the data from the IoT home appliances and receive optimal product recommendations in a timely manner.

[0145] "IoT home appliances" are electrical devices used in everyday life that are connected to the Internet and have the ability to communicate with each other.

[0146] "Means for receiving data" refers to the functions of devices or software for collecting usage information and operation information sent from IoT home appliances.

[0147] "Generative AI" is a type of artificial intelligence that refers to algorithms and models that can analyze data and generate new information and recommendations.

[0148] "Means for recommending products" refers to functions and mechanisms that analyze collected data, select the most suitable products for the user, and present them as a list.

[0149] The "means for presenting products" refers to the function of a device or software for visually displaying recommended products to a user.

[0150] "Push notification" is a function that automatically notifies communication devices such as smartphones of information in real time.

[0151] A "server" is a computer system that processes and stores data over a network and provides services to other computers and terminals.

[0152] "Data analysis" is the process of analyzing collected data to extract meaningful information and patterns.

[0153] A "user ID" is an identifier that uniquely identifies each user and is used to manage information for each user in databases and systems.

[0154] This embodiment of the present invention describes a system that uses data collected from IoT home appliances to implement product recommendations using generative AI. Below, the processing flow and specific examples of the system program are explained in natural language.

[0155] Hardware and Software Configuration

[0156] The server uses a cloud server (e.g., AWS (registered trademark) Lambda, AWS S3). The database uses AWS RDS or a NoSQL database, and the generative AI model uses GPT-4 (registered trademark) or BERT. The device is a smartphone (compatible with iOS or ANDROID (registered trademark)), and the IoT home appliance is assumed to have a Wi-Fi module or Bluetooth functionality.

[0157] Key processing steps and functions of the system

[0158] 1. IoT appliance data collection:

[0159] The server automatically receives operational information about IoT home appliances in the home (for example, the number of times the washing machine is used or how often the refrigerator is opened and closed) from the user's smartphone.

[0160] 2. Data storage:

[0161] The server then stores the received data, including the user ID, in a cloud database. This data includes the operating status, usage mode, and frequency of use of the appliances.

[0162] 3. Data analysis and recommendation generation:

[0163] The server analyzes the stored data using a generative AI model. As a result of the analysis, the best products for the user are identified and a list of those products is generated. For example, a user who frequently opens and closes their refrigerator will be recommended foods that can be stored for a long time or products that need to be replenished frequently.

[0164] 4. Product push notifications:

[0165] The server then delivers the generated recommendation results to the smartphone as a push notification, which includes products selected according to the user's interests.

[0166] Specific examples

[0167] For example, if a user opens and closes their refrigerator more than 20 times per day, this data is sent to a server via their smartphone and stored in a database on the cloud. The generative AI model then analyzes this data and generates recommendations such as "fresh, farm-fresh vegetable sets" and "long-lasting frozen foods." These recommended products are displayed on the user's smartphone as push notifications. They include automatically generated recommendations such as, "If you open and close your refrigerator more than 20 times per day, we recommend fresh, farm-fresh vegetables!"

[0168] This system will enable users to efficiently select products using data from IoT home appliances and receive timely recommendations for the best products. It is also expected to boost sales in the e-commerce industry and lead to improved user satisfaction for IoT home appliance manufacturers.

[0169] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0170] Step 1:

[0171] When a user uses an IoT home appliance such as a washing machine or refrigerator, data on its usage and frequency of operation is collected by the device. This device is typically a smartphone, and this data includes the number of times the appliance has been used, the mode of use, and the operating time. The device collects this data and sends it to a server. The input is the usage data from the IoT home appliance, and the output is the data sent to the server. Specifically, the smartphone obtains the data via Bluetooth or WiFi and uploads it to the server in a specific format.

[0172] Step 2:

[0173] The server receives the data sent from the device. The received data is stored in a database along with the user ID. The input is the data received from the device, and the output is the data stored in the database. Specifically, the server receives the data using HTTPS, associates it with the user ID, and stores it in a NoSQL database.

[0174] Step 3:

[0175] When a user attempts to purchase a product on an e-commerce site, a recommendation request is sent to the server. The input is the recommendation request from the user, and the output is the status of the request. Specifically, when the user presses the recommendation request button on the smartphone app, that information is sent to the server.

[0176] Step 4:

[0177] The server searches the stored device data and retrieves the relevant data based on the user ID. It then uses a generative AI model (e.g., GPT-4) to analyze the retrieved data and identify the best products for the user. The input is the stored device data and the recommendation request, and the output is a list of recommended products as the analysis result. Specifically, the server queries the data related to the user ID from the database and inputs that data into the generative AI model. The generative AI model performs the analysis and lists the recommended products.

[0178] Step 5:

[0179] The server returns the generated recommendation results to the user and presents them as a push notification to their smartphone. The input is the recommendation results generated by the generative AI model, and the output is the content displayed on the user's smartphone. Specifically, the server uses a push notification service (for example, Firebase Cloud Messaging) to send the generated product list to the smartphone. The user's smartphone receives the notification and displays it on the screen.

[0180] These processing steps enable the user to efficiently receive optimal product recommendations.

[0181] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0182] This embodiment of the present invention describes a system that analyzes data collected from IoT home appliances and uses a generative AI and an emotion engine to recommend optimal products to users. Below, the program processing flow and specific examples are explained in natural language.

[0183] Program processing

[0184] 1. Collecting and transmitting IoT home appliance data:

[0185] When a user uses an IoT appliance, its usage data is collected. This data includes, for example, the operation status, mode of use, and duration of use of a washing machine. This data is acquired by the device and sent to a server.

[0186] 2. Receiving and storing data:

[0187] The server receives the IoT home appliance data sent from the device. The received data is stored in a database along with the user ID. The database stores information such as each user's device usage history and operation patterns.

[0188] 3. Acquiring emotion data:

[0189] The device acquires the user's emotional data. The emotion engine recognizes the user's emotions through facial expression analysis, voice analysis, and / or text analysis. This emotional data is also sent to the server.

[0190] 4. Receiving and storing emotional data:

[0191] The server receives the user's emotion data sent from the device and stores it in a database.

[0192] 5. Receiving User Requests:

[0193] When a user attempts to purchase a product on an e-commerce site, they send a recommendation request to the server via their device. The request includes the user ID and asks the server to suggest products.

[0194] 6. Data analysis and recommendation generation:

[0195] The server searches the database for the corresponding user ID and retrieves IoT appliance data and emotion data. This data is then analyzed using generative AI to identify the most suitable product for the user. For example, if a user frequently uses their washing machine and the emotion engine recognizes that they are feeling stressed, the server will recommend a fabric softener with a relaxing effect.

[0196] 7. Presenting recommendations:

[0197] The server returns the generated recommendation results to the user. The recommendation includes a list of related products and detailed information. The user can then view the recommended products through their device and consider purchasing them.

[0198] Specific examples

[0199] A specific example is given below.

[0200] A user uses an IoT home appliance (e.g., a washing machine) on a daily basis. The usage data (e.g., using the washing machine five times a week) is collected by the device and sent to a server.

[0201] The server receives this data and stores it in a database along with the user ID.

[0202] Meanwhile, the device recognizes the user's emotions in real time and collects emotional data. For example, if the user is feeling stressed, that information can be obtained through facial expression analysis and voice analysis.

[0203] The server receives this emotion data and stores it in a database.

[0204] Later, when the same user tries to purchase detergent on an e-commerce site, a recommendation request is sent to the server.

[0205] The server uses generative AI to analyze the stored data and recommend the best product for the user, such as a fabric softener with a relaxing effect.

[0206] The server returns the generated recommendation results to the user, and the user can check the recommended products and consider purchasing them on the EC site.

[0207] This will enable users to find products efficiently and conveniently, and receive appropriate product suggestions based on their emotional state. It will also provide opportunities for increased sales in the e-commerce industry and lead to increased user satisfaction for IoT home appliance manufacturers, strengthening the ecosystem as a whole.

[0208] The processing flow will be explained below.

[0209] Step 1:

[0210] When a user uses an IoT home appliance, for example, a washing machine, the operating status, usage mode, and usage time of the washing machine are recorded.

[0211] Step 2:

[0212] The device retrieves usage data collected from IoT appliances, including details such as the number of times the washing machine was run, the duration of its run, and the mode used.

[0213] Step 3:

[0214] The device sends the collected data to the server in JSON format along with the user ID.

[0215] Step 4:

[0216] The server receives the data sent from the device and stores it in a database along with the user ID.

[0217] Step 5:

[0218] The device acquires the user's emotional data. The emotion engine recognizes the user's emotions using facial expression analysis, voice analysis, and text analysis. This emotional data is also sent to the server.

[0219] Step 6:

[0220] The server receives the emotion data sent from the device and stores it in a database, which includes the emotional state the user is feeling.

[0221] Step 7:

[0222] When a user attempts to purchase a product on an e-commerce website, they send a recommendation request to the server via their device. The request includes the user ID and requests the server to suggest products that suit the user.

[0223] Step 8:

[0224] The server retrieves the relevant data from the database based on the user ID, including usage data and emotion data of IoT appliances.

[0225] Step 9:

[0226] The server analyzes the acquired data using a generative AI, which identifies the best product for the user based on their usage and emotional state. For example, it identifies a fabric softener with a relaxing effect by taking into account how often the washing machine is used and the user's stressful situation.

[0227] Step 10:

[0228] The server lists the identified products and presents them to the user as recommendations, including the product name, description, and purchase link.

[0229] Step 11:

[0230] The user checks the product information presented on the device, and can use this information to consider purchasing the product and proceed with the purchase process on the e-commerce site.

[0231] Step 12:

[0232] The server stores the user's purchase history in a database, and uses the purchase data for analysis to improve the accuracy of future recommendations.

[0233] This allows users to receive product suggestions that take their emotional state into consideration, helping them find more suitable products more efficiently, while providing e-commerce sites and IoT home appliance manufacturers with the opportunity to improve user satisfaction and increase sales.

[0234] Example 2

[0235] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0236] Conventional recommendation systems using IoT home appliance data simply recommend products based on usage history without considering the user's emotional state, which can result in incomplete responses to user needs. Furthermore, improving the accuracy of recommendations requires diverse data sources and advanced analytical techniques, but integrating them efficiently has proven difficult.

[0237] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting and transmitting usage data when a user uses an IoT home appliance, means for a terminal to acquire and transmit emotional data of the user, means for the server to receive and store the usage data and emotional data, means for analyzing the stored data using a generative AI model based on a recommendation request from the user and identifying the most suitable product for the user, and means for presenting the recommended product to the user. This enables advanced recommendations that take into account the user's usage history as well as their real-time emotional state.

[0238] "Users" refer to consumers who use the system to operate IoT home appliances and receive product recommendations.

[0239] "IoT home appliances" refer to home appliances that can send and receive data via the Internet.

[0240] "Usage data" refers to information about the operation of IoT home appliances, such as their operating status, usage mode, and frequency of use.

[0241] "Terminal" refers to a device operated by a user that collects usage data and emotional data and transmits it to a server.

[0242] "Emotional data" refers to information about a user's emotional state, and refers to data obtained through facial expression analysis, voice analysis, and text analysis.

[0243] "Server" refers to the computer system that receives and stores usage and emotion data and analyzes such data using generative AI models.

[0244] "Database" refers to a system for systematically storing and managing usage data and emotional data.

[0245] A "recommendation request" refers to a request that a user sends to a server via a terminal indicating that they would like to receive product suggestions.

[0246] "Generative AI models" refer to advanced machine learning models used to make predictions and recommendations.

[0247] "Recommendations" refers to product suggestions that are optimal for a user, identified by a generative AI model based on the user's usage and sentiment data.

[0248] This embodiment of the present invention describes a system that analyzes data collected from IoT home appliances and recommends optimal products to users using a generative AI model and an emotion engine.

[0249] Hardware and software used

[0250] Hardware:

[0251] IoT home appliances (e.g. IoT washing machine)

[0252] Edge terminal (data collection device)

[0253] software:

[0254] Facial Expression Analysis Software

[0255] Voice Analysis Software

[0256] Text Analysis Software

[0257] Database Management Systems

[0258] Generative AI models (e.g., GPT-4)

[0259] Specific program processing flow

[0260] 1. Collecting and transmitting IoT home appliance data:

[0261] Every time a user uses an IoT home appliance, usage data is collected in real time. This data includes specific information such as usage time, usage mode, and power consumption. The device receives the data and sends it to a server via the Internet. For example, after using a washing machine, the device sends "User ID: 12345, Washing machine operation: 12:00-12:45, Mode: Standard" to the server.

[0262] 2. Receiving and storing data:

[0263] The server receives the data sent from the device and stores it in a database along with the user ID. Specifically, it stores information such as the user ID, device ID, operating time, usage mode, and power consumption in a database table.

[0264] 3. Acquiring emotion data:

[0265] The device acquires the user's emotional data. The emotion engine analyzes the user's facial expressions to determine whether they are smiling or stressed. Voice analysis is also used to analyze emotions from the tone and speed of the voice, and text analysis is also performed. The device then sends the collected emotional data back to the server. For example, the user can input what they want to say by voice, and the device generates data such as "User ID: 12345, Emotion: Stress, Voice Tone: High, Analysis Time: 14:00" and sends it to the server.

[0266] 4. Receiving and storing emotional data:

[0267] The server receives the emotion data sent from the device and stores it in a database, including information such as the user ID, emotional state, voice tone, and analysis time.

[0268] 5. Processing User Requests:

[0269] When a user attempts to purchase a product on an e-commerce site, they send a recommendation request to the server via their device. The request includes the user ID and current purchase intention. The device then sends this request to the server, along with the category and details of the product the user wants to check. For example, a request for "User ID: 12345, purchase intention: detergent, category: fabric softener" is sent.

[0270] 6. Data analysis and recommendation generation:

[0271] The server searches the database for the corresponding user ID and retrieves the collected IoT appliance data and emotion data. A generative AI model (e.g., GPT-4) analyzes this data and identifies the optimal product based on the user's emotional state and appliance usage patterns. Specifically, the server first executes an SQL query to retrieve the data and then sends a prompt to the AI ​​model. An example prompt might be, "Please suggest the optimal product based on the frequency of washing machine use and emotion data for user ID: 12345 over the past week."

[0272] 7. Presenting recommendations:

[0273] The server returns the generated recommendation results to the user, which include a list of related products and detailed information. The user can then review the recommended products through their device and consider purchasing them on the e-commerce site. For example, the information displayed might be, "Recommended product for you: Fabric softener with a relaxing effect (with link)."

[0274] Specific examples

[0275] Example: A user frequently uses a washing machine and emotion analysis indicates that they are stressed.

[0276] Example prompt:

[0277] "User ID: 12345 uses the washing machine five times a week. Recent emotional data indicates that he is feeling stressed. Please recommend the best product for this user."

[0278] This system allows users to receive optimal product recommendations based on their usage habits and emotional state, providing a better purchasing experience.

[0279] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0280] Step 1:

[0281] Collecting and transmitting IoT home appliance data

[0282] Users use IoT home appliances on a daily basis.

[0283] The device collects usage data (e.g., usage time, usage mode, power consumption) using built-in sensors and communication modules.

[0284] The usage data collected by the device is structured and sent to a server via the Internet.

[0285] Input: IoT appliance operation data (e.g., "User ID: 12345, Washing machine operation: 12:00~12:45, Mode: Standard").

[0286] Output: Structured data sent to the server.

[0287] Step 2:

[0288] Receiving and storing data

[0289] The server receives the usage data sent from the terminal.

[0290] The server saves the received data in the database, along with the user ID.

[0291] Input: Usage data sent from the device.

[0292] Output: Usage data stored in a database (e.g. "User ID: 12345, Device ID, Uptime, Usage Mode, Power Consumption").

[0293] Specific operation: When the server receives new data, it inserts the data into the database using the SQL INSERT statement.

[0294] Step 3:

[0295] Acquiring emotion data

[0296] The device acquires the user's emotional data, and the emotion engine performs facial expression analysis (e.g., using the camera), voice analysis (e.g., using the microphone), and text analysis.

[0297] The device sends the collected emotional data to the server, including the emotional state (e.g., stress, happiness), voice tone, and analysis time.

[0298] Input: Real-time user facial, voice, and text input.

[0299] Output: Structured emotion data sent to the server.

[0300] Step 4:

[0301] Receiving and storing emotional data

[0302] The server receives the emotion data sent from the terminal.

[0303] The server stores the received emotion data in a database, along with the user ID.

[0304] Input: Emotion data sent from the device.

[0305] Output: Emotion data stored in a database (e.g., "User ID: 12345, emotional state, voice tone, analysis time").

[0306] Specific operation: When the server receives new emotion data, it inserts the data into the database using the SQL INSERT statement.

[0307] Step 5:

[0308] Processing user requests

[0309] When a user attempts to purchase a product on an e-commerce site, they send a recommendation request to the server via their device, which includes the user ID and their current purchase intention.

[0310] Input: Recommendation request (e.g., "User ID: 12345, Purchase Intent: Detergent, Category: Fabric Softener").

[0311] Output: The request data sent to the server.

[0312] Step 6:

[0313] Data analysis and recommendation generation

[0314] The server searches the database for the corresponding user ID and obtains usage data and emotion data.

[0315] The server uses a generative AI model (e.g., GPT-4) to analyze the acquired data and identify the best products for the user.

[0316] Input: Usage and sentiment data associated with the user ID.

[0317] Output: The generated recommendation result (e.g., "Fabric softener with relaxing effects").

[0318] How it works: The server retrieves data using an SQL query and sends a prompt to the generative AI model, such as, "Please suggest the best product for user ID: 12345 based on their washing machine usage frequency and emotional data from the past week."

[0319] Step 7:

[0320] Presenting recommendations

[0321] The server returns the generated recommendation results to the user.

[0322] The user checks the recommended products through the device and considers purchasing them on the e-commerce site.

[0323] Input: The generated recommendation results.

[0324] Output: Product information presented to the user (e.g., "Relaxing Fabric Softener (with link)").

[0325] Specific behavior: Recommendation results are displayed on the user's device.

[0326] (Application example 2)

[0327] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0328] Conventional recommendation systems suggest products based solely on data collected from users' IoT home appliances, and therefore are unable to reflect the user's emotional state or temporary needs. This results in low recommendation accuracy and makes it difficult to suggest optimal products to users.

[0329] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0330] In this invention, the server includes means for receiving data collected from IoT home appliances, means for analyzing the data using a generative AI and recommending optimal products to the user, means for acquiring and analyzing user emotional data, means for integrating and analyzing the IoT home appliance data and the emotional data to identify optimal products, and means for presenting the recommended products to the user via a smartphone application. This enables optimal product suggestions that take into account the user's emotional state and temporary needs.

[0331] "IoT home appliances" are household electrical appliances that are connected to the Internet and can send and receive data.

[0332] "Generative AI" is a system that uses artificial intelligence technology to analyze input data and generate new data and information.

[0333] "Emotional data" is data that represents the emotional state of a user and is obtained through facial expression analysis, voice analysis, and text analysis.

[0334] "Recommendation" refers to suggesting appropriate products or services to users.

[0335] A "smartphone application" is a software program that runs on a smartphone and allows for user interaction.

[0336] A "server" is a computer system that operates on a network and stores, analyzes, and manages data.

[0337] A "database" is a system that efficiently stores, manages, and searches structured data.

[0338] A "generative AI model" is a trained artificial intelligence algorithm that is used to generate new data or information based on input data.

[0339] A "recommendation request" is when a user requests the system to suggest products based on specific criteria.

[0340] This embodiment of the present invention describes a system that analyzes data collected from IoT home appliances and recommends optimal products to users based on generative AI and emotional data.

[0341] This system includes a means for collecting data from IoT home appliances, a means for analyzing the data using generative AI, a means for acquiring and analyzing user sentiment data, and a means for identifying and recommending optimal products based on the integrated data analysis results. The recommendation results are presented to the user via a smartphone application.

[0342] Overview of the entire program

[0343] First, when a user opens the smartphone application, usage data from IoT home appliances is collected. At the same time, the user's emotional data is also acquired. This data is sent from the smartphone to a server, which analyzes the data using a generative AI model and recommends the most suitable products. Finally, the recommended products are displayed to the user on the smartphone application.

[0344] Hardware and software used

[0345] Hardware

[0346] Smartphone: Used as the primary device for data collection and display.

[0347] IoT appliances: household electrical appliances such as washing machines, refrigerators, and vacuum cleaners.

[0348] software

[0349] Emotion analysis engine: Uses facial expression analysis (OpenCV) and voice analysis (Google (registered trademark) Cloud Speech-to-Text).

[0350] Database: Data management is performed on the server side using MySQL (registered trademark).

[0351] Data analysis is performed using the generative AI model: GPT-3 (registered trademark) (OpenAI (registered trademark)).

[0352] Server application: Implement server-side processing using Python and Flask.

[0353] Specific examples of processing

[0354] When a user launches the smartphone application, data is automatically collected from IoT home appliances. For example, data on the use of a washing machine five times a week is collected. At the same time, the user's emotional data is also collected by analyzing facial expressions and voice. If the user's face shows any signs of stress, that data is also collected.

[0355] This data is then sent from the smartphone to a server and stored in a database. The server analyzes this data using GPT-3 to identify the most suitable product, such as a fabric softener with a relaxing effect. The recommendation results are then sent to the user's smartphone application, allowing the user to review the recommended product and consider purchasing it.

[0356] Examples of prompt statements

[0357] Here is an example prompt in sentence text form:

[0358] User ID: 12345

[0359] IoT data: Washing machine usage: 5 times / week

[0360] Emotion data: Stress (from facial and voice analysis)

[0361] In this way, the system provides users with the most suitable products based on usage data from IoT home appliances and user emotional data, thereby realizing a more personalized customer experience.

[0362] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0363] Step 1:

[0364] Data collection

[0365] The terminal collects usage data from IoT appliances. For example, the number of times a washing machine is used and the operating mode are acquired. The input is the IoT data acquired from the appliance, and the output is a temporary file containing the data or data stored in memory.

[0366] Step 2:

[0367] Acquiring emotion data

[0368] The device recognizes the user's face and obtains emotional data using a facial expression analysis engine (OpenCV). It also extracts emotional data from voice using a voice analysis engine (Google Cloud Speech-to-Text). The input is the user's real-time video and audio, and the output is the emotional data obtained by analyzing them.

[0369] Step 3:

[0370] Data transmission

[0371] The IoT appliance data and emotion data collected by the device are sent to the server. The input is the data obtained in Step 1 and Step 2, and the output is the data sent to the server.

[0372] Step 4:

[0373] Receiving and storing data

[0374] The server receives the transmitted data and stores it in a database. The input is the data transmitted from the terminal, and the output is the data stored in the database.

[0375] Step 5:

[0376] Data analysis

[0377] The server runs a generative AI model (GPT-3) to analyze the stored data. The input is the user's IoT appliance data and emotion data retrieved from the database, and the output is the analysis results to identify the best product for the user.

[0378] Step 6:

[0379] Recommendation Generation

[0380] The server uses the generative AI model to recommend the optimal product based on the analysis results. The input is the analysis results obtained in step 5, and the output is the recommendation result.

[0381] Step 7:

[0382] Recommendation presentation

[0383] The server sends the recommendation results to the device, which then displays them to the user. The user can check the recommended products through a smartphone application and consider purchasing them. The input is the recommendation results from the server, and the output is a list of products displayed on the device.

[0384] This will enable users to easily find the best products based on their IoT home appliance usage and emotional state.

[0385] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0386] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0387] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0388] [Second embodiment]

[0389] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0390] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0391] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0392] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0393] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0394] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0395] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0396] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0397] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0398] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0399] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0400] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0401] This embodiment of the present invention describes a system that uses data collected from IoT home appliances to implement product recommendations using generative AI. Below, the program processing flow and specific examples are explained in natural language.

[0402] Program processing

[0403] 1. Collecting and transmitting IoT home appliance data:

[0404] When a user uses an IoT home appliance, its usage data is collected. This data includes, for example, the operating status and usage mode of a washing machine. This data is acquired by the device and sent to a server.

[0405] 2. Receiving and storing data:

[0406] The server receives the IoT home appliance data sent from the device. The received data is stored in a database along with the user ID. The database stores information such as each user's device usage history and operation patterns.

[0407] 3. Receiving User Requests:

[0408] When a user attempts to purchase a product on an e-commerce site, a recommendation request is sent to the server, including the user ID.

[0409] 4. Data analysis and recommendation generation:

[0410] The server searches the stored device data and retrieves the relevant data based on the user ID. Generative AI is used to analyze the retrieved data and identify the most suitable products for the user. For example, if a user uses their washing machine frequently, it will recommend a detergent that suits their lifestyle.

[0411] 5. Presenting recommendations:

[0412] The server returns the generated recommendation results to the user. The recommendation includes a list of related products and detailed information. The user can then view the recommended products through their device and consider purchasing them.

[0413] Specific examples

[0414] A specific example is given below.

[0415] A user uses an IoT home appliance (e.g., a washing machine) on a daily basis. The usage data (e.g., using the washing machine five times a week) is collected by the device and sent to a server.

[0416] The server receives this data and stores it in a database along with the user ID.

[0417] Later, when the same user tries to purchase detergent on an e-commerce site, a recommendation request is sent to the server.

[0418] The server uses generative AI to analyze the stored data and recommend the best products for each user, such as highly concentrated detergents or environmentally friendly detergents.

[0419] The server returns the generated recommendation results to the user, and the user can check the recommended products and consider purchasing them on the EC site.

[0420] This will enable users to find products efficiently and conveniently, providing opportunities for increased sales in the e-commerce industry, and will also lead to increased user satisfaction for IoT home appliance manufacturers, strengthening the ecosystem as a whole.

[0421] The processing flow will be explained below.

[0422] Step 1:

[0423] When a user uses an IoT appliance, for example, a washing machine, the operating status, mode of use, and duration of use are recorded.

[0424] Step 2:

[0425] The device collects usage data for IoT appliances, including washing machine usage data, which includes detailed information such as the number of times it has been used, the duration of its use, and the mode of use.

[0426] Step 3:

[0427] The device sends the collected data to the server. The data sent is in the form of a user ID and device data, for example in JSON format as shown below.

[0428] json

[0429] {

[0430] "user_id": "12345",

[0431] "device_data": {

[0432] "washing_machine": {

[0433] "usage_count": 5,

[0434] "total_runtime_minutes": 300,

[0435] "modes_used": ["normal", "quick"]

[0436] }

[0437] }

[0438] }

[0439] Step 4:

[0440] The server receives the data sent from the device, stores it in a database in an appropriate format, and uses it for later analysis.

[0441] Step 5:

[0442] When a user wants to purchase a product on an e-commerce site, they send a recommendation request to the server via their device. The request includes the user ID and asks the server to suggest products.

[0443] Step 6:

[0444] When the server receives a user request, it searches the stored database for data for the corresponding user ID, for example, to retrieve washing machine usage data.

[0445] Step 7:

[0446] The server analyzes the data using generative AI. It analyzes the user's usage in detail and identifies the most suitable product for the user. For example, it identifies highly concentrated detergents and environmentally friendly detergents for users who use their washing machine frequently.

[0447] Step 8:

[0448] The server lists the products identified by the AI ​​and presents them to the user as recommendations, including the product name, description, and purchase link.

[0449] Step 9:

[0450] The user can check the recommended products through their device, consider purchasing the products based on this information, and proceed with the purchase process on the e-commerce site.

[0451] Step 10:

[0452] The server continues to store the user's purchase history in a database to help inform future recommendations. This continuous data collection and analysis allows for continuous improvement of the user experience.

[0453] Example 1

[0454] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0455] Today's consumers have diverse needs and spend a lot of time and effort selecting products. In particular, with the spread of IoT devices, there is a demand for utilizing the large amount of usage data obtained from these devices to propose individually optimized products. However, conventional systems face the challenge of effectively analyzing this data and recommending the best products for users. Furthermore, while analysis technology using generative AI is rapidly evolving, there are few systems that can properly utilize it.

[0456] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0457] In this invention, the server includes means for collecting operation and usage data from IoT devices used by users and transmitting it via the terminal, means for receiving the data transmitted from the terminal and storing it in a database together with user identification information, means for receiving a recommendation request when the user attempts to purchase a product, means for using a generative AI model to analyze the stored data and identify the product best suited to the user, and means for presenting the generated recommendation results to the user. This makes it possible to efficiently analyze large amounts of IoT device data and quickly and accurately provide the best product to each user.

[0458] "User" refers to a consumer who uses IoT devices on a daily basis and whose usage data is collected and analyzed.

[0459] "IoT devices" refers to home appliances and other electronic devices that are connected to the Internet and can automatically send and receive data.

[0460] "Operational and usage data" refers to information about how a user uses a particular device, such as the device's operating status, usage mode, and frequency of use.

[0461] "Terminal" refers to a device that transmits data collected from IoT devices to a server. Specifically, this includes smartphones and smart home hubs.

[0462] "Server" refers to a central computing unit that receives, stores, analyzes, and processes data over a network.

[0463] "Database" means a system running on a Server for structuring and storing Usage Data, including User Identification Information.

[0464] "User identification information" refers to information for identifying a specific user, and includes, for example, a user ID.

[0465] "Recommendation Request" refers to a request sent by a user to a server to request recommendations for the best products for a particular product category.

[0466] "Generative AI model" refers to an artificial intelligence model used to analyze collected data and identify the best products for users.

[0467] A "prompt" is a text-based instruction entered into a generative AI model that clearly states the data to be analyzed and the purpose.

[0468] "Recommendation Results" refers to the optimal product suggestions presented to the user as a result of analysis by the generative AI model.

[0469] This invention relates to a system that uses data collected from users' IoT devices to realize product recommendations using a generative AI model. This system can perform all processes from data collection to analysis and product proposals. The program processing flow is explained in detail below.

[0470] Collecting and transmitting IoT home appliance data

[0471] When a user uses an IoT device (e.g., a washing machine or refrigerator), data on its operation and usage is collected. This data includes, for example, its operating status, usage mode, and frequency of use. This data is acquired by a device (e.g., a smartphone or smart home hub). The device then transmits the collected data to a server via Wi-Fi or a mobile network.

[0472] Receiving and storing data

[0473] The server receives the data sent from the device. This process is performed through a secure API endpoint, ensuring safe data transmission. The received data is then stored in a database along with user identification information. A NoSQL database (e.g., MongoDB) is suitable for this database, as it is characterized by its scalability and flexibility.

[0474] Receiving a user request

[0475] When a user attempts to purchase a product on an e-commerce site, they send a recommendation request. This request is sent from the device (for example, a browser on a PC or smartphone) to the server. This request includes the user ID and the product category they are considering purchasing. The HTTPS protocol is used for communication to ensure data security.

[0476] Data analysis and recommendation generation

[0477] The server processes the received request and searches for device usage data stored in a database. It retrieves the relevant data based on the user's identity and inputs a prompt statement into the generative AI model. An example of a prompt statement would be, "Recommend the best detergent based on the usage data of user ID: 12345." The generative AI model uses text analysis and machine learning algorithms to analyze the user's behavioral patterns and identify the best product.

[0478] Presenting recommendations

[0479] The server receives the recommendation results obtained from the generative AI model and presents them to the user. These results include a list of related products and their detailed information. For example, an e-commerce site in the browser displays images, descriptions, prices, etc. of the recommended products. The user can review these and consider purchasing them.

[0480] Specific examples

[0481] Specific examples are shown below.

[0482] Suppose that usage data for an IoT device (e.g., a washing machine) that a user uses daily is collected five times a week.

[0483] This usage data includes information such as time and mode of use, and is periodically sent to a server and stored in a database.

[0484] Later, when the same user tries to purchase detergent on an e-commerce site, they send a recommendation request to the server.

[0485] Based on the stored data, the server sends a prompt to the generated AI, such as "Recommend the optimal detergent based on the usage data of user ID: 12345."

[0486] The generative AI model will return results that recommend, for example, "highly concentrated detergents and environmentally friendly detergents."

[0487] The results are presented to the user on their browser, allowing them to consider purchasing based on the recommended products.

[0488] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0489] Step 1:

[0490] When a user uses an IoT device, data on its operation and usage status is collected by the device. Specific data includes operating time, usage mode, and frequency of use. This data is acquired in real time and saved in JSON format. Sensor data and usage logs from the IoT device operated by the user are used as input. This data is temporarily saved in the device as output. Specific operations include reading data from sensors, converting it into a specific format, and saving it.

[0491] Step 2:

[0492] The device transmits the collected data to the server at regular intervals. This transmission process uses Wi-Fi or mobile networks and uses a secure communication protocol (e.g., HTTPS) to ensure data security. As input, the usage data collected in step 1 is in JSON format. As output, the data is sent to and received from the server. Specific operations include encoding, packaging, and sending the data to the server.

[0493] Step 3:

[0494] The server receives data sent from the device and stores it in a database. This data contains user identification information, so it is stored and associated with a specific user. The usage data sent from the device is used as input. The received data is stored in the database as output. Specific operations include receiving data at an API endpoint, decoding the data, and inserting it into the database.

[0495] Step 4:

[0496] When a user attempts to purchase a product on an e-commerce website, a recommendation request is sent to the server. The request includes the user ID and the desired product category. The request data from the user is used as input. The request content is received and saved on the server as output. Specific operations include collecting form data, encoding the data, and sending it to the server.

[0497] Step 5:

[0498] Based on the received request, the server retrieves relevant device usage data from the database. It also creates and sends a prompt statement to the generative AI model based on this data. The user's usage data stored in the database and the request content are used as input. The prompt statement is sent to the generative AI model as output. An example of a prompt statement is "Recommend the optimal detergent based on the usage data of user ID: 12345." Specific operations include generating and executing a database query, formatting the retrieved data, and sending it to the generative AI model.

[0499] Step 6:

[0500] The generative AI model analyzes data based on the received prompt and identifies the most suitable products. It uses machine learning algorithms to analyze user behavior patterns and product characteristics to recommend the most suitable products. The prompt is used as input. As output, a list of the most suitable products is generated as the recommendation result. Specific operations include data analysis using natural language processing and machine learning algorithms, and the generation of recommendation results.

[0501] Step 7:

[0502] The server receives the recommendation results obtained from the generative AI model and presents them to the user. These results include a list of related products and their details. The recommendation results from the generative AI model are used as input. The output provides a list of products and their details to be displayed to the user. Specific operations include receiving the result data, converting the data format, and displaying it on the user's device.

[0503] This allows users to efficiently find the product that best suits them and consider purchasing it.

[0504] (Application example 1)

[0505] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0506] In recent years, the spread of IoT home appliances has made users' daily lives increasingly convenient. However, there is still a lack of a system that can effectively utilize the vast amount of data from IoT home appliances to suggest products that are suitable for each individual user. In addition, there is a lack of means to provide timely and accurate product recommendations to users. As a result, users are faced with the problem of spending time and effort finding the products they need.

[0507] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0508] In this invention, the server includes means for receiving data collected from IoT home appliances, means for analyzing the data using a generating AI and recommending optimal products to the user, means for presenting the recommended products to the user, and means for automatically recommending products based on the data collected from the IoT home appliances and providing them as push notifications to a smartphone. This enables users to efficiently select products by utilizing the data from the IoT home appliances and receive optimal product recommendations in a timely manner.

[0509] "IoT home appliances" are electrical devices used in everyday life that are connected to the Internet and have the ability to communicate with each other.

[0510] "Means for receiving data" refers to the functions of devices or software for collecting usage information and operation information sent from IoT home appliances.

[0511] "Generative AI" is a type of artificial intelligence that refers to algorithms and models that can analyze data and generate new information and recommendations.

[0512] "Means for recommending products" refers to functions and mechanisms that analyze collected data, select the most suitable products for the user, and present them as a list.

[0513] The "means for presenting products" refers to the function of a device or software for visually displaying recommended products to a user.

[0514] "Push notification" is a function that automatically notifies communication devices such as smartphones of information in real time.

[0515] A "server" is a computer system that processes and stores data over a network and provides services to other computers and terminals.

[0516] "Data analysis" is the process of analyzing collected data to extract meaningful information and patterns.

[0517] A "user ID" is an identifier that uniquely identifies each user and is used to manage information for each user in databases and systems.

[0518] This embodiment of the present invention describes a system that uses data collected from IoT home appliances to implement product recommendations using generative AI. Below, the processing flow and specific examples of the system program are explained in natural language.

[0519] Hardware and Software Configuration

[0520] The server uses a cloud server (e.g., AWS Lambda, AWS S3). The database uses AWS RDS or a NoSQL database, and the generative AI model uses GPT-4 or BERT. The device is a smartphone (compatible with iOS or Android), and the IoT home appliance is assumed to have a Wi-Fi module or Bluetooth functionality.

[0521] Key processing steps and functions of the system

[0522] 1. IoT appliance data collection:

[0523] The server automatically receives operational information about IoT home appliances in the home (for example, the number of times the washing machine is used or how often the refrigerator is opened and closed) from the user's smartphone.

[0524] 2. Data storage:

[0525] The server then stores the received data, including the user ID, in a cloud database. This data includes the operating status, usage mode, and frequency of use of the appliances.

[0526] 3. Data analysis and recommendation generation:

[0527] The server analyzes the stored data using a generative AI model. As a result of the analysis, the best products for the user are identified and a list of those products is generated. For example, a user who frequently opens and closes their refrigerator will be recommended foods that can be stored for a long time or products that need to be replenished frequently.

[0528] 4. Product push notifications:

[0529] The server then delivers the generated recommendation results to the smartphone as a push notification, which includes products selected according to the user's interests.

[0530] Specific examples

[0531] For example, if a user opens and closes their refrigerator more than 20 times per day, this data is sent to a server via their smartphone and stored in a database on the cloud. The generative AI model then analyzes this data and generates recommendations such as "fresh, farm-fresh vegetable sets" and "long-lasting frozen foods." These recommended products are displayed on the user's smartphone as push notifications. They include automatically generated recommendations such as, "If you open and close your refrigerator more than 20 times per day, we recommend fresh, farm-fresh vegetables!"

[0532] This system will enable users to efficiently select products using data from IoT home appliances and receive timely recommendations for the best products. It is also expected to boost sales in the e-commerce industry and lead to improved user satisfaction for IoT home appliance manufacturers.

[0533] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0534] Step 1:

[0535] When a user uses an IoT home appliance such as a washing machine or refrigerator, data on its usage and frequency of operation is collected by the device. This device is typically a smartphone, and this data includes the number of times the appliance has been used, the mode of use, and the operating time. The device collects this data and sends it to a server. The input is the usage data from the IoT home appliance, and the output is the data sent to the server. Specifically, the smartphone obtains the data via Bluetooth or WiFi and uploads it to the server in a specific format.

[0536] Step 2:

[0537] The server receives the data sent from the device. The received data is stored in a database along with the user ID. The input is the data received from the device, and the output is the data stored in the database. Specifically, the server receives the data using HTTPS, associates it with the user ID, and stores it in a NoSQL database.

[0538] Step 3:

[0539] When a user attempts to purchase a product on an e-commerce site, a recommendation request is sent to the server. The input is the recommendation request from the user, and the output is the status of the request. Specifically, when the user presses the recommendation request button on the smartphone app, that information is sent to the server.

[0540] Step 4:

[0541] The server searches the stored device data and retrieves the relevant data based on the user ID. It then uses a generative AI model (e.g., GPT-4) to analyze the retrieved data and identify the best products for the user. The input is the stored device data and the recommendation request, and the output is a list of recommended products as the analysis result. Specifically, the server queries the data related to the user ID from the database and inputs that data into the generative AI model. The generative AI model performs the analysis and lists the recommended products.

[0542] Step 5:

[0543] The server returns the generated recommendation results to the user and presents them as a push notification to their smartphone. The input is the recommendation results generated by the generative AI model, and the output is the content displayed on the user's smartphone. Specifically, the server uses a push notification service (for example, Firebase Cloud Messaging) to send the generated product list to the smartphone. The user's smartphone receives the notification and displays it on the screen.

[0544] These processing steps enable the user to efficiently receive optimal product recommendations.

[0545] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0546] This embodiment of the present invention describes a system that analyzes data collected from IoT home appliances and uses a generative AI and an emotion engine to recommend optimal products to users. Below, the program processing flow and specific examples are explained in natural language.

[0547] Program processing

[0548] 1. Collecting and transmitting IoT home appliance data:

[0549] When a user uses an IoT appliance, its usage data is collected. This data includes, for example, the operation status, mode of use, and duration of use of a washing machine. This data is acquired by the device and sent to a server.

[0550] 2. Receiving and storing data:

[0551] The server receives the IoT home appliance data sent from the device. The received data is stored in a database along with the user ID. The database stores information such as each user's device usage history and operation patterns.

[0552] 3. Acquiring emotion data:

[0553] The device acquires the user's emotional data. The emotion engine recognizes the user's emotions through facial expression analysis, voice analysis, and / or text analysis. This emotional data is also sent to the server.

[0554] 4. Receiving and storing emotional data:

[0555] The server receives the user's emotion data sent from the device and stores it in a database.

[0556] 5. Receiving User Requests:

[0557] When a user attempts to purchase a product on an e-commerce site, they send a recommendation request to the server via their device. The request includes the user ID and asks the server to suggest products.

[0558] 6. Data analysis and recommendation generation:

[0559] The server searches the database for the corresponding user ID and retrieves IoT appliance data and emotion data. This data is then analyzed using generative AI to identify the most suitable product for the user. For example, if a user frequently uses their washing machine and the emotion engine recognizes that they are feeling stressed, the server will recommend a fabric softener with a relaxing effect.

[0560] 7. Presenting recommendations:

[0561] The server returns the generated recommendation results to the user. The recommendation includes a list of related products and detailed information. The user can then view the recommended products through their device and consider purchasing them.

[0562] Specific examples

[0563] A specific example is given below.

[0564] A user uses an IoT home appliance (e.g., a washing machine) on a daily basis. The usage data (e.g., using the washing machine five times a week) is collected by the device and sent to a server.

[0565] The server receives this data and stores it in a database along with the user ID.

[0566] Meanwhile, the device recognizes the user's emotions in real time and collects emotional data. For example, if the user is feeling stressed, that information can be obtained through facial expression analysis and voice analysis.

[0567] The server receives this emotion data and stores it in a database.

[0568] Later, when the same user tries to purchase detergent on an e-commerce site, a recommendation request is sent to the server.

[0569] The server uses generative AI to analyze the stored data and recommend the best product for the user, such as a fabric softener with a relaxing effect.

[0570] The server returns the generated recommendation results to the user, and the user can check the recommended products and consider purchasing them on the EC site.

[0571] This will enable users to find products efficiently and conveniently, and receive appropriate product suggestions based on their emotional state. It will also provide opportunities for increased sales in the e-commerce industry and lead to increased user satisfaction for IoT home appliance manufacturers, strengthening the ecosystem as a whole.

[0572] The processing flow will be explained below.

[0573] Step 1:

[0574] When a user uses an IoT home appliance, for example, a washing machine, the operating status, usage mode, and usage time of the washing machine are recorded.

[0575] Step 2:

[0576] The device retrieves usage data collected from IoT appliances, including details such as the number of times the washing machine was run, the duration of its run, and the mode used.

[0577] Step 3:

[0578] The device sends the collected data to the server in JSON format along with the user ID.

[0579] Step 4:

[0580] The server receives the data sent from the device and stores it in a database along with the user ID.

[0581] Step 5:

[0582] The device acquires the user's emotional data. The emotion engine recognizes the user's emotions using facial expression analysis, voice analysis, and text analysis. This emotional data is also sent to the server.

[0583] Step 6:

[0584] The server receives the emotion data sent from the device and stores it in a database, which includes the emotional state the user is feeling.

[0585] Step 7:

[0586] When a user attempts to purchase a product on an e-commerce website, they send a recommendation request to the server via their device. The request includes the user ID and requests the server to suggest products that suit the user.

[0587] Step 8:

[0588] The server retrieves the relevant data from the database based on the user ID, including usage data and emotion data of IoT appliances.

[0589] Step 9:

[0590] The server analyzes the acquired data using a generative AI, which identifies the best product for the user based on their usage and emotional state. For example, it identifies a fabric softener with a relaxing effect by taking into account how often the washing machine is used and the user's stressful situation.

[0591] Step 10:

[0592] The server lists the identified products and presents them to the user as recommendations, including the product name, description, and purchase link.

[0593] Step 11:

[0594] The user checks the product information presented on the device, and can use this information to consider purchasing the product and proceed with the purchase process on the e-commerce site.

[0595] Step 12:

[0596] The server stores the user's purchase history in a database, and uses the purchase data for analysis to improve the accuracy of future recommendations.

[0597] This allows users to receive product suggestions that take their emotional state into consideration, helping them find more suitable products more efficiently, while providing e-commerce sites and IoT home appliance manufacturers with the opportunity to improve user satisfaction and increase sales.

[0598] Example 2

[0599] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0600] Conventional recommendation systems using IoT home appliance data simply recommend products based on usage history without considering the user's emotional state, which can result in incomplete responses to user needs. Furthermore, improving the accuracy of recommendations requires diverse data sources and advanced analytical techniques, but integrating them efficiently has proven difficult.

[0601] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting and transmitting usage data when a user uses an IoT home appliance, means for a terminal to acquire and transmit emotional data of the user, means for the server to receive and store the usage data and emotional data, means for analyzing the stored data using a generative AI model based on a recommendation request from the user and identifying the most suitable product for the user, and means for presenting the recommended product to the user. This enables advanced recommendations that take into account the user's usage history as well as their real-time emotional state.

[0602] "Users" refer to consumers who use the system to operate IoT home appliances and receive product recommendations.

[0603] "IoT home appliances" refer to home appliances that can send and receive data via the Internet.

[0604] "Usage data" refers to information about the operation of IoT home appliances, such as their operating status, usage mode, and frequency of use.

[0605] "Terminal" refers to a device operated by a user that collects usage data and emotional data and transmits it to a server.

[0606] "Emotional data" refers to information about a user's emotional state, and refers to data obtained through facial expression analysis, voice analysis, and text analysis.

[0607] "Server" refers to the computer system that receives and stores usage and emotion data and analyzes such data using generative AI models.

[0608] "Database" refers to a system for systematically storing and managing usage data and emotional data.

[0609] A "recommendation request" refers to a request that a user sends to a server via a terminal indicating that they would like to receive product suggestions.

[0610] "Generative AI models" refer to advanced machine learning models used to make predictions and recommendations.

[0611] "Recommendations" refers to product suggestions that are optimal for a user, identified by a generative AI model based on the user's usage and sentiment data.

[0612] This embodiment of the present invention describes a system that analyzes data collected from IoT home appliances and recommends optimal products to users using a generative AI model and an emotion engine.

[0613] Hardware and software used

[0614] Hardware:

[0615] IoT home appliances (e.g. IoT washing machine)

[0616] Edge terminal (data collection device)

[0617] software:

[0618] Facial Expression Analysis Software

[0619] Voice Analysis Software

[0620] Text Analysis Software

[0621] Database Management Systems

[0622] Generative AI models (e.g., GPT-4)

[0623] Specific program processing flow

[0624] 1. Collecting and transmitting IoT home appliance data:

[0625] Every time a user uses an IoT home appliance, usage data is collected in real time. This data includes specific information such as usage time, usage mode, and power consumption. The device receives the data and sends it to a server via the Internet. For example, after using a washing machine, the device sends "User ID: 12345, Washing machine operation: 12:00-12:45, Mode: Standard" to the server.

[0626] 2. Receiving and storing data:

[0627] The server receives the data sent from the device and stores it in a database along with the user ID. Specifically, it stores information such as the user ID, device ID, operating time, usage mode, and power consumption in a database table.

[0628] 3. Acquiring emotion data:

[0629] The device acquires the user's emotional data. The emotion engine analyzes the user's facial expressions to determine whether they are smiling or stressed. Voice analysis is also used to analyze emotions from the tone and speed of the voice, and text analysis is also performed. The device then sends the collected emotional data back to the server. For example, the user can input what they want to say by voice, and the device generates data such as "User ID: 12345, Emotion: Stress, Voice Tone: High, Analysis Time: 14:00" and sends it to the server.

[0630] 4. Receiving and storing emotional data:

[0631] The server receives the emotion data sent from the device and stores it in a database, including information such as the user ID, emotional state, voice tone, and analysis time.

[0632] 5. Processing User Requests:

[0633] When a user attempts to purchase a product on an e-commerce site, they send a recommendation request to the server via their device. The request includes the user ID and current purchase intention. The device then sends this request to the server, along with the category and details of the product the user wants to check. For example, a request for "User ID: 12345, purchase intention: detergent, category: fabric softener" is sent.

[0634] 6. Data analysis and recommendation generation:

[0635] The server searches the database for the corresponding user ID and retrieves the collected IoT appliance data and emotion data. A generative AI model (e.g., GPT-4) analyzes this data and identifies the optimal product based on the user's emotional state and appliance usage patterns. Specifically, the server first executes an SQL query to retrieve the data and then sends a prompt to the AI ​​model. An example prompt might be, "Please suggest the optimal product based on the frequency of washing machine use and emotion data for user ID: 12345 over the past week."

[0636] 7. Presenting recommendations:

[0637] The server returns the generated recommendation results to the user, which include a list of related products and detailed information. The user can then review the recommended products through their device and consider purchasing them on the e-commerce site. For example, the information displayed might be, "Recommended product for you: Fabric softener with a relaxing effect (with link)."

[0638] Specific examples

[0639] Example: A user frequently uses a washing machine and emotion analysis indicates that they are stressed.

[0640] Example prompt:

[0641] "User ID: 12345 uses the washing machine five times a week. Recent emotional data indicates that he is feeling stressed. Please recommend the best product for this user."

[0642] This system allows users to receive optimal product recommendations based on their usage habits and emotional state, providing a better purchasing experience.

[0643] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0644] Step 1:

[0645] Collecting and transmitting IoT home appliance data

[0646] Users use IoT home appliances on a daily basis.

[0647] The device collects usage data (e.g., usage time, usage mode, power consumption) using built-in sensors and communication modules.

[0648] The usage data collected by the device is structured and sent to a server via the Internet.

[0649] Input: IoT appliance operation data (e.g., "User ID: 12345, Washing machine operation: 12:00~12:45, Mode: Standard").

[0650] Output: Structured data sent to the server.

[0651] Step 2:

[0652] Receiving and storing data

[0653] The server receives the usage data sent from the terminal.

[0654] The server saves the received data in the database, along with the user ID.

[0655] Input: Usage data sent from the device.

[0656] Output: Usage data stored in a database (e.g. "User ID: 12345, Device ID, Uptime, Usage Mode, Power Consumption").

[0657] Specific operation: When the server receives new data, it inserts the data into the database using the SQL INSERT statement.

[0658] Step 3:

[0659] Acquiring emotion data

[0660] The device acquires the user's emotional data, and the emotion engine performs facial expression analysis (e.g., using the camera), voice analysis (e.g., using the microphone), and text analysis.

[0661] The device sends the collected emotional data to the server, including the emotional state (e.g., stress, happiness), voice tone, and analysis time.

[0662] Input: Real-time user facial, voice, and text input.

[0663] Output: Structured emotion data sent to the server.

[0664] Step 4:

[0665] Receiving and storing emotional data

[0666] The server receives the emotion data sent from the terminal.

[0667] The server stores the received emotion data in a database, along with the user ID.

[0668] Input: Emotion data sent from the device.

[0669] Output: Emotion data stored in a database (e.g., "User ID: 12345, emotional state, voice tone, analysis time").

[0670] Specific operation: When the server receives new emotion data, it inserts the data into the database using the SQL INSERT statement.

[0671] Step 5:

[0672] Processing user requests

[0673] When a user attempts to purchase a product on an e-commerce site, they send a recommendation request to the server via their device, which includes the user ID and their current purchase intention.

[0674] Input: Recommendation request (e.g., "User ID: 12345, Purchase Intent: Detergent, Category: Fabric Softener").

[0675] Output: The request data sent to the server.

[0676] Step 6:

[0677] Data analysis and recommendation generation

[0678] The server searches the database for the corresponding user ID and obtains usage data and emotion data.

[0679] The server uses a generative AI model (e.g., GPT-4) to analyze the acquired data and identify the best products for the user.

[0680] Input: Usage and sentiment data associated with the user ID.

[0681] Output: The generated recommendation result (e.g., "Fabric softener with relaxing effects").

[0682] How it works: The server retrieves data using an SQL query and sends a prompt to the generative AI model, such as, "Please suggest the best product for user ID: 12345 based on their washing machine usage frequency and emotional data from the past week."

[0683] Step 7:

[0684] Presenting recommendations

[0685] The server returns the generated recommendation results to the user.

[0686] The user checks the recommended products through the device and considers purchasing them on the e-commerce site.

[0687] Input: The generated recommendation results.

[0688] Output: Product information presented to the user (e.g., "Relaxing Fabric Softener (with link)").

[0689] Specific behavior: Recommendation results are displayed on the user's device.

[0690] (Application example 2)

[0691] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0692] Conventional recommendation systems suggest products based solely on data collected from users' IoT home appliances, and therefore are unable to reflect the user's emotional state or temporary needs. This results in low recommendation accuracy and makes it difficult to suggest optimal products to users.

[0693] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0694] In this invention, the server includes means for receiving data collected from IoT home appliances, means for analyzing the data using a generative AI and recommending optimal products to the user, means for acquiring and analyzing user emotional data, means for integrating and analyzing the IoT home appliance data and the emotional data to identify optimal products, and means for presenting the recommended products to the user via a smartphone application. This enables optimal product suggestions that take into account the user's emotional state and temporary needs.

[0695] "IoT home appliances" are household electrical appliances that are connected to the Internet and can send and receive data.

[0696] "Generative AI" is a system that uses artificial intelligence technology to analyze input data and generate new data and information.

[0697] "Emotional data" is data that represents the emotional state of a user and is obtained through facial expression analysis, voice analysis, and text analysis.

[0698] "Recommendation" refers to suggesting appropriate products or services to users.

[0699] A "smartphone application" is a software program that runs on a smartphone and allows for user interaction.

[0700] A "server" is a computer system that operates on a network and stores, analyzes, and manages data.

[0701] A "database" is a system that efficiently stores, manages, and searches structured data.

[0702] A "generative AI model" is a trained artificial intelligence algorithm that is used to generate new data or information based on input data.

[0703] A "recommendation request" is when a user requests the system to suggest products based on specific criteria.

[0704] This embodiment of the present invention describes a system that analyzes data collected from IoT home appliances and recommends optimal products to users based on generative AI and emotional data.

[0705] This system includes a means for collecting data from IoT home appliances, a means for analyzing the data using generative AI, a means for acquiring and analyzing user sentiment data, and a means for identifying and recommending optimal products based on the integrated data analysis results. The recommendation results are presented to the user via a smartphone application.

[0706] Overview of the entire program

[0707] First, when a user opens the smartphone application, usage data from IoT home appliances is collected. At the same time, the user's emotional data is also acquired. This data is sent from the smartphone to a server, which analyzes the data using a generative AI model and recommends the most suitable products. Finally, the recommended products are displayed to the user on the smartphone application.

[0708] Hardware and software used

[0709] Hardware

[0710] Smartphone: Used as the primary device for data collection and display.

[0711] IoT appliances: household electrical appliances such as washing machines, refrigerators, and vacuum cleaners.

[0712] software

[0713] Emotion analysis engine: Uses facial expression analysis (OpenCV) and voice analysis (Google Cloud Speech-to-Text).

[0714] Database: Data management is performed on the server side using MySQL.

[0715] Generative AI model: Data analysis is performed using GPT-3 (OpenAI).

[0716] Server application: Implement server-side processing using Python and Flask.

[0717] Specific examples of processing

[0718] When a user launches the smartphone application, data is automatically collected from IoT home appliances. For example, data on the use of a washing machine five times a week is collected. At the same time, the user's emotional data is also collected by analyzing facial expressions and voice. If the user's face shows any signs of stress, that data is also collected.

[0719] This data is then sent from the smartphone to a server and stored in a database. The server analyzes this data using GPT-3 to identify the most suitable product, such as a fabric softener with a relaxing effect. The recommendation results are then sent to the user's smartphone application, allowing the user to review the recommended product and consider purchasing it.

[0720] Examples of prompt statements

[0721] Here is an example prompt in sentence text form:

[0722] User ID: 12345

[0723] IoT data: Washing machine usage: 5 times / week

[0724] Emotion data: Stress (from facial and voice analysis)

[0725] In this way, the system provides users with the most suitable products based on usage data from IoT home appliances and user emotional data, thereby realizing a more personalized customer experience.

[0726] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0727] Step 1:

[0728] Data collection

[0729] The terminal collects usage data from IoT appliances. For example, the number of times a washing machine is used and the operating mode are acquired. The input is the IoT data acquired from the appliance, and the output is a temporary file containing the data or data stored in memory.

[0730] Step 2:

[0731] Acquiring emotion data

[0732] The device recognizes the user's face and obtains emotional data using a facial expression analysis engine (OpenCV). It also extracts emotional data from voice using a voice analysis engine (Google Cloud Speech-to-Text). The input is the user's real-time video and audio, and the output is the emotional data obtained by analyzing them.

[0733] Step 3:

[0734] Data transmission

[0735] The IoT appliance data and emotion data collected by the device are sent to the server. The input is the data obtained in Step 1 and Step 2, and the output is the data sent to the server.

[0736] Step 4:

[0737] Receiving and storing data

[0738] The server receives the transmitted data and stores it in a database. The input is the data transmitted from the terminal, and the output is the data stored in the database.

[0739] Step 5:

[0740] Data analysis

[0741] The server runs a generative AI model (GPT-3) to analyze the stored data. The input is the user's IoT appliance data and emotion data retrieved from the database, and the output is the analysis results to identify the best product for the user.

[0742] Step 6:

[0743] Recommendation Generation

[0744] The server uses the generative AI model to recommend the optimal product based on the analysis results. The input is the analysis results obtained in step 5, and the output is the recommendation result.

[0745] Step 7:

[0746] Recommendation presentation

[0747] The server sends the recommendation results to the device, which then displays them to the user. The user can check the recommended products through a smartphone application and consider purchasing them. The input is the recommendation results from the server, and the output is a list of products displayed on the device.

[0748] This will enable users to easily find the best products based on their IoT home appliance usage and emotional state.

[0749] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0750] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0751] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0752] [Third embodiment]

[0753] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0754] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0755] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0756] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0757] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0758] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0759] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0760] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0761] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0762] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0763] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0764] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0765] This embodiment of the present invention describes a system that uses data collected from IoT home appliances to implement product recommendations using generative AI. Below, the program processing flow and specific examples are explained in natural language.

[0766] Program processing

[0767] 1. Collecting and transmitting IoT home appliance data:

[0768] When a user uses an IoT home appliance, its usage data is collected. This data includes, for example, the operating status and usage mode of a washing machine. This data is acquired by the device and sent to a server.

[0769] 2. Receiving and storing data:

[0770] The server receives the IoT home appliance data sent from the device. The received data is stored in a database along with the user ID. The database stores information such as each user's device usage history and operation patterns.

[0771] 3. Receiving User Requests:

[0772] When a user attempts to purchase a product on an e-commerce site, a recommendation request is sent to the server, including the user ID.

[0773] 4. Data analysis and recommendation generation:

[0774] The server searches the stored device data and retrieves the relevant data based on the user ID. Generative AI is used to analyze the retrieved data and identify the most suitable products for the user. For example, if a user uses their washing machine frequently, it will recommend a detergent that suits their lifestyle.

[0775] 5. Presenting recommendations:

[0776] The server returns the generated recommendation results to the user. The recommendation includes a list of related products and detailed information. The user can then view the recommended products through their device and consider purchasing them.

[0777] Specific examples

[0778] A specific example is given below.

[0779] A user uses an IoT home appliance (e.g., a washing machine) on a daily basis. The usage data (e.g., using the washing machine five times a week) is collected by the device and sent to a server.

[0780] The server receives this data and stores it in a database along with the user ID.

[0781] Later, when the same user tries to purchase detergent on an e-commerce site, a recommendation request is sent to the server.

[0782] The server uses generative AI to analyze the stored data and recommend the best products for each user, such as highly concentrated detergents or environmentally friendly detergents.

[0783] The server returns the generated recommendation results to the user, and the user can check the recommended products and consider purchasing them on the EC site.

[0784] This will enable users to find products efficiently and conveniently, providing opportunities for increased sales in the e-commerce industry, and will also lead to increased user satisfaction for IoT home appliance manufacturers, strengthening the ecosystem as a whole.

[0785] The processing flow will be explained below.

[0786] Step 1:

[0787] When a user uses an IoT appliance, for example, a washing machine, the operating status, mode of use, and duration of use are recorded.

[0788] Step 2:

[0789] The device collects usage data for IoT appliances, including washing machine usage data, which includes detailed information such as the number of times it has been used, the duration of its use, and the mode of use.

[0790] Step 3:

[0791] The device sends the collected data to the server. The data sent is in the form of a user ID and device data, for example in JSON format as shown below.

[0792] json

[0793] {

[0794] "user_id": "12345",

[0795] "device_data": {

[0796] "washing_machine": {

[0797] "usage_count": 5,

[0798] "total_runtime_minutes": 300,

[0799] "modes_used": ["normal", "quick"]

[0800] }

[0801] }

[0802] }

[0803] Step 4:

[0804] The server receives the data sent from the device, stores it in a database in an appropriate format, and uses it for later analysis.

[0805] Step 5:

[0806] When a user wants to purchase a product on an e-commerce site, they send a recommendation request to the server via their device. The request includes the user ID and asks the server to suggest products.

[0807] Step 6:

[0808] When the server receives a user request, it searches the stored database for data for the corresponding user ID, for example, to retrieve washing machine usage data.

[0809] Step 7:

[0810] The server analyzes the data using generative AI. It analyzes the user's usage in detail and identifies the most suitable product for the user. For example, it identifies highly concentrated detergents and environmentally friendly detergents for users who use their washing machine frequently.

[0811] Step 8:

[0812] The server lists the products identified by the AI ​​and presents them to the user as recommendations, including the product name, description, and purchase link.

[0813] Step 9:

[0814] The user can check the recommended products through their device, consider purchasing the products based on this information, and proceed with the purchase process on the e-commerce site.

[0815] Step 10:

[0816] The server continues to store the user's purchase history in a database to help inform future recommendations. This continuous data collection and analysis allows for continuous improvement of the user experience.

[0817] Example 1

[0818] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0819] Today's consumers have diverse needs and spend a lot of time and effort selecting products. In particular, with the spread of IoT devices, there is a demand for utilizing the large amount of usage data obtained from these devices to propose individually optimized products. However, conventional systems face the challenge of effectively analyzing this data and recommending the best products for users. Furthermore, while analysis technology using generative AI is rapidly evolving, there are few systems that can properly utilize it.

[0820] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0821] In this invention, the server includes means for collecting operation and usage data from IoT devices used by users and transmitting it via the terminal, means for receiving the data transmitted from the terminal and storing it in a database together with user identification information, means for receiving a recommendation request when the user attempts to purchase a product, means for using a generative AI model to analyze the stored data and identify the product best suited to the user, and means for presenting the generated recommendation results to the user. This makes it possible to efficiently analyze large amounts of IoT device data and quickly and accurately provide the best product to each user.

[0822] "User" refers to a consumer who uses IoT devices on a daily basis and whose usage data is collected and analyzed.

[0823] "IoT devices" refers to home appliances and other electronic devices that are connected to the Internet and can automatically send and receive data.

[0824] "Operational and usage data" refers to information about how a user uses a particular device, such as the device's operating status, usage mode, and frequency of use.

[0825] "Terminal" refers to a device that transmits data collected from IoT devices to a server. Specifically, this includes smartphones and smart home hubs.

[0826] "Server" refers to a central computing unit that receives, stores, analyzes, and processes data over a network.

[0827] "Database" means a system running on a Server for structuring and storing Usage Data, including User Identification Information.

[0828] "User identification information" refers to information for identifying a specific user, and includes, for example, a user ID.

[0829] "Recommendation Request" refers to a request sent by a user to a server to request recommendations for the best products for a particular product category.

[0830] "Generative AI model" refers to an artificial intelligence model used to analyze collected data and identify the best products for users.

[0831] A "prompt" is a text-based instruction entered into a generative AI model that clearly states the data to be analyzed and the purpose.

[0832] "Recommendation Results" refers to the optimal product suggestions presented to the user as a result of analysis by the generative AI model.

[0833] This invention relates to a system that uses data collected from users' IoT devices to realize product recommendations using a generative AI model. This system can perform all processes from data collection to analysis and product proposals. The program processing flow is explained in detail below.

[0834] Collecting and transmitting IoT home appliance data

[0835] When a user uses an IoT device (e.g., a washing machine or refrigerator), data on its operation and usage is collected. This data includes, for example, its operating status, usage mode, and frequency of use. This data is acquired by a device (e.g., a smartphone or smart home hub). The device then transmits the collected data to a server via Wi-Fi or a mobile network.

[0836] Receiving and storing data

[0837] The server receives the data sent from the device. This process is performed through a secure API endpoint, ensuring safe data transmission. The received data is then stored in a database along with user identification information. A NoSQL database (e.g., MongoDB) is suitable for this database, as it is characterized by its scalability and flexibility.

[0838] Receiving a user request

[0839] When a user attempts to purchase a product on an e-commerce site, they send a recommendation request. This request is sent from the device (for example, a browser on a PC or smartphone) to the server. This request includes the user ID and the product category they are considering purchasing. The HTTPS protocol is used for communication to ensure data security.

[0840] Data analysis and recommendation generation

[0841] The server processes the received request and searches for device usage data stored in a database. It retrieves the relevant data based on the user's identity and inputs a prompt statement into the generative AI model. An example of a prompt statement would be, "Recommend the best detergent based on the usage data of user ID: 12345." The generative AI model uses text analysis and machine learning algorithms to analyze the user's behavioral patterns and identify the best product.

[0842] Presenting recommendations

[0843] The server receives the recommendation results obtained from the generative AI model and presents them to the user. These results include a list of related products and their detailed information. For example, an e-commerce site in the browser displays images, descriptions, prices, etc. of the recommended products. The user can review these and consider purchasing them.

[0844] Specific examples

[0845] Specific examples are shown below.

[0846] Suppose that usage data for an IoT device (e.g., a washing machine) that a user uses daily is collected five times a week.

[0847] This usage data includes information such as time and mode of use, and is periodically sent to a server and stored in a database.

[0848] Later, when the same user tries to purchase detergent on an e-commerce site, they send a recommendation request to the server.

[0849] Based on the stored data, the server sends a prompt to the generated AI, such as "Recommend the optimal detergent based on the usage data of user ID: 12345."

[0850] The generative AI model will return results that recommend, for example, "highly concentrated detergents and environmentally friendly detergents."

[0851] The results are presented to the user on their browser, allowing them to consider purchasing based on the recommended products.

[0852] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0853] Step 1:

[0854] When a user uses an IoT device, data on its operation and usage status is collected by the device. Specific data includes operating time, usage mode, and frequency of use. This data is acquired in real time and saved in JSON format. Sensor data and usage logs from the IoT device operated by the user are used as input. This data is temporarily saved in the device as output. Specific operations include reading data from sensors, converting it into a specific format, and saving it.

[0855] Step 2:

[0856] The device transmits the collected data to the server at regular intervals. This transmission process uses Wi-Fi or mobile networks and uses a secure communication protocol (e.g., HTTPS) to ensure data security. As input, the usage data collected in step 1 is in JSON format. As output, the data is sent to and received from the server. Specific operations include encoding, packaging, and sending the data to the server.

[0857] Step 3:

[0858] The server receives data sent from the device and stores it in a database. This data contains user identification information, so it is stored and associated with a specific user. The usage data sent from the device is used as input. The received data is stored in the database as output. Specific operations include receiving data at an API endpoint, decoding the data, and inserting it into the database.

[0859] Step 4:

[0860] When a user attempts to purchase a product on an e-commerce website, a recommendation request is sent to the server. The request includes the user ID and the desired product category. The request data from the user is used as input. The request content is received and saved on the server as output. Specific operations include collecting form data, encoding the data, and sending it to the server.

[0861] Step 5:

[0862] Based on the received request, the server retrieves relevant device usage data from the database. It also creates and sends a prompt statement to the generative AI model based on this data. The user's usage data stored in the database and the request content are used as input. The prompt statement is sent to the generative AI model as output. An example of a prompt statement is "Recommend the optimal detergent based on the usage data of user ID: 12345." Specific operations include generating and executing a database query, formatting the retrieved data, and sending it to the generative AI model.

[0863] Step 6:

[0864] The generative AI model analyzes data based on the received prompt and identifies the most suitable products. It uses machine learning algorithms to analyze user behavior patterns and product characteristics to recommend the most suitable products. The prompt is used as input. As output, a list of the most suitable products is generated as the recommendation result. Specific operations include data analysis using natural language processing and machine learning algorithms, and the generation of recommendation results.

[0865] Step 7:

[0866] The server receives the recommendation results obtained from the generative AI model and presents them to the user. These results include a list of related products and their details. The recommendation results from the generative AI model are used as input. The output provides a list of products and their details to be displayed to the user. Specific operations include receiving the result data, converting the data format, and displaying it on the user's device.

[0867] This allows users to efficiently find the product that best suits them and consider purchasing it.

[0868] (Application example 1)

[0869] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0870] In recent years, the spread of IoT home appliances has made users' daily lives increasingly convenient. However, there is still a lack of a system that can effectively utilize the vast amount of data from IoT home appliances to suggest products that are suitable for each individual user. In addition, there is a lack of means to provide timely and accurate product recommendations to users. As a result, users are faced with the problem of spending time and effort finding the products they need.

[0871] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0872] In this invention, the server includes means for receiving data collected from IoT home appliances, means for analyzing the data using a generating AI and recommending optimal products to the user, means for presenting the recommended products to the user, and means for automatically recommending products based on the data collected from the IoT home appliances and providing them as push notifications to a smartphone. This enables users to efficiently select products by utilizing the data from the IoT home appliances and receive optimal product recommendations in a timely manner.

[0873] "IoT home appliances" are electrical devices used in everyday life that are connected to the Internet and have the ability to communicate with each other.

[0874] "Means for receiving data" refers to the functions of devices or software for collecting usage information and operation information sent from IoT home appliances.

[0875] "Generative AI" is a type of artificial intelligence that refers to algorithms and models that can analyze data and generate new information and recommendations.

[0876] "Means for recommending products" refers to functions and mechanisms that analyze collected data, select the most suitable products for the user, and present them as a list.

[0877] The "means for presenting products" refers to the function of a device or software for visually displaying recommended products to a user.

[0878] "Push notification" is a function that automatically notifies communication devices such as smartphones of information in real time.

[0879] A "server" is a computer system that processes and stores data over a network and provides services to other computers and terminals.

[0880] "Data analysis" is the process of analyzing collected data to extract meaningful information and patterns.

[0881] A "user ID" is an identifier that uniquely identifies each user and is used to manage information for each user in databases and systems.

[0882] This embodiment of the present invention describes a system that uses data collected from IoT home appliances to implement product recommendations using generative AI. Below, the processing flow and specific examples of the system program are explained in natural language.

[0883] Hardware and Software Configuration

[0884] The server uses a cloud server (e.g., AWS Lambda, AWS S3). The database uses AWS RDS or a NoSQL database, and the generative AI model uses GPT-4 or BERT. The device is a smartphone (compatible with iOS or Android), and the IoT home appliance is assumed to have a Wi-Fi module or Bluetooth functionality.

[0885] Key processing steps and functions of the system

[0886] 1. IoT appliance data collection:

[0887] The server automatically receives operational information about IoT home appliances in the home (for example, the number of times the washing machine is used or how often the refrigerator is opened and closed) from the user's smartphone.

[0888] 2. Data storage:

[0889] The server then stores the received data, including the user ID, in a cloud database. This data includes the operating status, usage mode, and frequency of use of the appliances.

[0890] 3. Data analysis and recommendation generation:

[0891] The server analyzes the stored data using a generative AI model. As a result of the analysis, the best products for the user are identified and a list of those products is generated. For example, a user who frequently opens and closes their refrigerator will be recommended foods that can be stored for a long time or products that need to be replenished frequently.

[0892] 4. Product push notifications:

[0893] The server then delivers the generated recommendation results to the smartphone as a push notification, which includes products selected according to the user's interests.

[0894] Specific examples

[0895] For example, if a user opens and closes their refrigerator more than 20 times per day, this data is sent to a server via their smartphone and stored in a database on the cloud. The generative AI model then analyzes this data and generates recommendations such as "fresh, farm-fresh vegetable sets" and "long-lasting frozen foods." These recommended products are displayed on the user's smartphone as push notifications. They include automatically generated recommendations such as, "If you open and close your refrigerator more than 20 times per day, we recommend fresh, farm-fresh vegetables!"

[0896] This system will enable users to efficiently select products using data from IoT home appliances and receive timely recommendations for the best products. It is also expected to boost sales in the e-commerce industry and lead to improved user satisfaction for IoT home appliance manufacturers.

[0897] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0898] Step 1:

[0899] When a user uses an IoT home appliance such as a washing machine or refrigerator, data on its usage and frequency of operation is collected by the device. This device is typically a smartphone, and this data includes the number of times the appliance has been used, the mode of use, and the operating time. The device collects this data and sends it to a server. The input is the usage data from the IoT home appliance, and the output is the data sent to the server. Specifically, the smartphone obtains the data via Bluetooth or WiFi and uploads it to the server in a specific format.

[0900] Step 2:

[0901] The server receives the data sent from the device. The received data is stored in a database along with the user ID. The input is the data received from the device, and the output is the data stored in the database. Specifically, the server receives the data using HTTPS, associates it with the user ID, and stores it in a NoSQL database.

[0902] Step 3:

[0903] When a user attempts to purchase a product on an e-commerce site, a recommendation request is sent to the server. The input is the recommendation request from the user, and the output is the status of the request. Specifically, when the user presses the recommendation request button on the smartphone app, that information is sent to the server.

[0904] Step 4:

[0905] The server searches the stored device data and retrieves the relevant data based on the user ID. It then uses a generative AI model (e.g., GPT-4) to analyze the retrieved data and identify the best products for the user. The input is the stored device data and the recommendation request, and the output is a list of recommended products as the analysis result. Specifically, the server queries the data related to the user ID from the database and inputs that data into the generative AI model. The generative AI model performs the analysis and lists the recommended products.

[0906] Step 5:

[0907] The server returns the generated recommendation results to the user and presents them as a push notification to their smartphone. The input is the recommendation results generated by the generative AI model, and the output is the content displayed on the user's smartphone. Specifically, the server uses a push notification service (for example, Firebase Cloud Messaging) to send the generated product list to the smartphone. The user's smartphone receives the notification and displays it on the screen.

[0908] These processing steps enable the user to efficiently receive optimal product recommendations.

[0909] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0910] This embodiment of the present invention describes a system that analyzes data collected from IoT home appliances and uses a generative AI and an emotion engine to recommend optimal products to users. Below, the program processing flow and specific examples are explained in natural language.

[0911] Program processing

[0912] 1. Collecting and transmitting IoT home appliance data:

[0913] When a user uses an IoT appliance, its usage data is collected. This data includes, for example, the operation status, mode of use, and duration of use of a washing machine. This data is acquired by the device and sent to a server.

[0914] 2. Receiving and storing data:

[0915] The server receives the IoT home appliance data sent from the device. The received data is stored in a database along with the user ID. The database stores information such as each user's device usage history and operation patterns.

[0916] 3. Acquiring emotion data:

[0917] The device acquires the user's emotional data. The emotion engine recognizes the user's emotions through facial expression analysis, voice analysis, and / or text analysis. This emotional data is also sent to the server.

[0918] 4. Receiving and storing emotional data:

[0919] The server receives the user's emotion data sent from the device and stores it in a database.

[0920] 5. Receiving User Requests:

[0921] When a user attempts to purchase a product on an e-commerce site, they send a recommendation request to the server via their device. The request includes the user ID and asks the server to suggest products.

[0922] 6. Data analysis and recommendation generation:

[0923] The server searches the database for the corresponding user ID and retrieves IoT appliance data and emotion data. This data is then analyzed using generative AI to identify the most suitable product for the user. For example, if a user frequently uses their washing machine and the emotion engine recognizes that they are feeling stressed, the server will recommend a fabric softener with a relaxing effect.

[0924] 7. Presenting recommendations:

[0925] The server returns the generated recommendation results to the user. The recommendation includes a list of related products and detailed information. The user can then view the recommended products through their device and consider purchasing them.

[0926] Specific examples

[0927] A specific example is given below.

[0928] A user uses an IoT home appliance (e.g., a washing machine) on a daily basis. The usage data (e.g., using the washing machine five times a week) is collected by the device and sent to a server.

[0929] The server receives this data and stores it in a database along with the user ID.

[0930] Meanwhile, the device recognizes the user's emotions in real time and collects emotional data. For example, if the user is feeling stressed, that information can be obtained through facial expression analysis and voice analysis.

[0931] The server receives this emotion data and stores it in a database.

[0932] Later, when the same user tries to purchase detergent on an e-commerce site, a recommendation request is sent to the server.

[0933] The server uses generative AI to analyze the stored data and recommend the best product for the user, such as a fabric softener with a relaxing effect.

[0934] The server returns the generated recommendation results to the user, and the user can check the recommended products and consider purchasing them on the EC site.

[0935] This will enable users to find products efficiently and conveniently, and receive appropriate product suggestions based on their emotional state. It will also provide opportunities for increased sales in the e-commerce industry and lead to increased user satisfaction for IoT home appliance manufacturers, strengthening the ecosystem as a whole.

[0936] The processing flow will be explained below.

[0937] Step 1:

[0938] When a user uses an IoT home appliance, for example, a washing machine, the operating status, usage mode, and usage time of the washing machine are recorded.

[0939] Step 2:

[0940] The device retrieves usage data collected from IoT appliances, including details such as the number of times the washing machine was run, the duration of its run, and the mode used.

[0941] Step 3:

[0942] The device sends the collected data to the server in JSON format along with the user ID.

[0943] Step 4:

[0944] The server receives the data sent from the device and stores it in a database along with the user ID.

[0945] Step 5:

[0946] The device acquires the user's emotional data. The emotion engine recognizes the user's emotions using facial expression analysis, voice analysis, and text analysis. This emotional data is also sent to the server.

[0947] Step 6:

[0948] The server receives the emotion data sent from the device and stores it in a database, which includes the emotional state the user is feeling.

[0949] Step 7:

[0950] When a user attempts to purchase a product on an e-commerce website, they send a recommendation request to the server via their device. The request includes the user ID and requests the server to suggest products that suit the user.

[0951] Step 8:

[0952] The server retrieves the relevant data from the database based on the user ID, including usage data and emotion data of IoT appliances.

[0953] Step 9:

[0954] The server analyzes the acquired data using a generative AI, which identifies the best product for the user based on their usage and emotional state. For example, it identifies a fabric softener with a relaxing effect by taking into account how often the washing machine is used and the user's stressful situation.

[0955] Step 10:

[0956] The server lists the identified products and presents them to the user as recommendations, including the product name, description, and purchase link.

[0957] Step 11:

[0958] The user checks the product information presented on the device, and can use this information to consider purchasing the product and proceed with the purchase process on the e-commerce site.

[0959] Step 12:

[0960] The server stores the user's purchase history in a database, and uses the purchase data for analysis to improve the accuracy of future recommendations.

[0961] This allows users to receive product suggestions that take their emotional state into consideration, helping them find more suitable products more efficiently, while providing e-commerce sites and IoT home appliance manufacturers with the opportunity to improve user satisfaction and increase sales.

[0962] Example 2

[0963] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0964] Conventional recommendation systems using IoT home appliance data simply recommend products based on usage history without considering the user's emotional state, which can result in incomplete responses to user needs. Furthermore, improving the accuracy of recommendations requires diverse data sources and advanced analytical techniques, but integrating them efficiently has proven difficult.

[0965] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting and transmitting usage data when a user uses an IoT home appliance, means for a terminal to acquire and transmit emotional data of the user, means for the server to receive and store the usage data and emotional data, means for analyzing the stored data using a generative AI model based on a recommendation request from the user and identifying the most suitable product for the user, and means for presenting the recommended product to the user. This enables advanced recommendations that take into account the user's usage history as well as their real-time emotional state.

[0966] "Users" refer to consumers who use the system to operate IoT home appliances and receive product recommendations.

[0967] "IoT home appliances" refer to home appliances that can send and receive data via the Internet.

[0968] "Usage data" refers to information about the operation of IoT home appliances, such as their operating status, usage mode, and frequency of use.

[0969] "Terminal" refers to a device operated by a user that collects usage data and emotional data and transmits it to a server.

[0970] "Emotional data" refers to information about a user's emotional state, and refers to data obtained through facial expression analysis, voice analysis, and text analysis.

[0971] "Server" refers to the computer system that receives and stores usage and emotion data and analyzes such data using generative AI models.

[0972] "Database" refers to a system for systematically storing and managing usage data and emotional data.

[0973] A "recommendation request" refers to a request that a user sends to a server via a terminal indicating that they would like to receive product suggestions.

[0974] "Generative AI models" refer to advanced machine learning models used to make predictions and recommendations.

[0975] "Recommendations" refers to product suggestions that are optimal for a user, identified by a generative AI model based on the user's usage and sentiment data.

[0976] This embodiment of the present invention describes a system that analyzes data collected from IoT home appliances and recommends optimal products to users using a generative AI model and an emotion engine.

[0977] Hardware and software used

[0978] Hardware:

[0979] IoT home appliances (e.g. IoT washing machine)

[0980] Edge terminal (data collection device)

[0981] software:

[0982] Facial Expression Analysis Software

[0983] Voice Analysis Software

[0984] Text Analysis Software

[0985] Database Management Systems

[0986] Generative AI models (e.g., GPT-4)

[0987] Specific program processing flow

[0988] 1. Collecting and transmitting IoT home appliance data:

[0989] Every time a user uses an IoT home appliance, usage data is collected in real time. This data includes specific information such as usage time, usage mode, and power consumption. The device receives the data and sends it to a server via the Internet. For example, after using a washing machine, the device sends "User ID: 12345, Washing machine operation: 12:00-12:45, Mode: Standard" to the server.

[0990] 2. Receiving and storing data:

[0991] The server receives the data sent from the device and stores it in a database along with the user ID. Specifically, it stores information such as the user ID, device ID, operating time, usage mode, and power consumption in a database table.

[0992] 3. Acquiring emotion data:

[0993] The device acquires the user's emotional data. The emotion engine analyzes the user's facial expressions to determine whether they are smiling or stressed. Voice analysis is also used to analyze emotions from the tone and speed of the voice, and text analysis is also performed. The device then sends the collected emotional data back to the server. For example, the user can input what they want to say by voice, and the device generates data such as "User ID: 12345, Emotion: Stress, Voice Tone: High, Analysis Time: 14:00" and sends it to the server.

[0994] 4. Receiving and storing emotional data:

[0995] The server receives the emotion data sent from the device and stores it in a database, including information such as the user ID, emotional state, voice tone, and analysis time.

[0996] 5. Processing User Requests:

[0997] When a user attempts to purchase a product on an e-commerce site, they send a recommendation request to the server via their device. The request includes the user ID and current purchase intention. The device then sends this request to the server, along with the category and details of the product the user wants to check. For example, a request for "User ID: 12345, purchase intention: detergent, category: fabric softener" is sent.

[0998] 6. Data analysis and recommendation generation:

[0999] The server searches the database for the corresponding user ID and retrieves the collected IoT appliance data and emotion data. A generative AI model (e.g., GPT-4) analyzes this data and identifies the optimal product based on the user's emotional state and appliance usage patterns. Specifically, the server first executes an SQL query to retrieve the data and then sends a prompt to the AI ​​model. An example prompt might be, "Please suggest the optimal product based on the frequency of washing machine use and emotion data for user ID: 12345 over the past week."

[1000] 7. Presenting recommendations:

[1001] The server returns the generated recommendation results to the user, which include a list of related products and detailed information. The user can then review the recommended products through their device and consider purchasing them on the e-commerce site. For example, the information displayed might be, "Recommended product for you: Fabric softener with a relaxing effect (with link)."

[1002] Specific examples

[1003] Example: A user frequently uses a washing machine and emotion analysis indicates that they are stressed.

[1004] Example prompt:

[1005] "User ID: 12345 uses the washing machine five times a week. Recent emotional data indicates that he is feeling stressed. Please recommend the best product for this user."

[1006] This system allows users to receive optimal product recommendations based on their usage habits and emotional state, providing a better purchasing experience.

[1007] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1008] Step 1:

[1009] Collecting and transmitting IoT home appliance data

[1010] Users use IoT home appliances on a daily basis.

[1011] The device collects usage data (e.g., usage time, usage mode, power consumption) using built-in sensors and communication modules.

[1012] The usage data collected by the device is structured and sent to a server via the Internet.

[1013] Input: IoT appliance operation data (e.g., "User ID: 12345, Washing machine operation: 12:00~12:45, Mode: Standard").

[1014] Output: Structured data sent to the server.

[1015] Step 2:

[1016] Receiving and storing data

[1017] The server receives the usage data sent from the terminal.

[1018] The server saves the received data in the database, along with the user ID.

[1019] Input: Usage data sent from the device.

[1020] Output: Usage data stored in a database (e.g. "User ID: 12345, Device ID, Uptime, Usage Mode, Power Consumption").

[1021] Specific operation: When the server receives new data, it inserts the data into the database using the SQL INSERT statement.

[1022] Step 3:

[1023] Acquiring emotion data

[1024] The device acquires the user's emotional data, and the emotion engine performs facial expression analysis (e.g., using the camera), voice analysis (e.g., using the microphone), and text analysis.

[1025] The device sends the collected emotional data to the server, including the emotional state (e.g., stress, happiness), voice tone, and analysis time.

[1026] Input: Real-time user facial, voice, and text input.

[1027] Output: Structured emotion data sent to the server.

[1028] Step 4:

[1029] Receiving and storing emotional data

[1030] The server receives the emotion data sent from the terminal.

[1031] The server stores the received emotion data in a database, along with the user ID.

[1032] Input: Emotion data sent from the device.

[1033] Output: Emotion data stored in a database (e.g., "User ID: 12345, emotional state, voice tone, analysis time").

[1034] Specific operation: When the server receives new emotion data, it inserts the data into the database using the SQL INSERT statement.

[1035] Step 5:

[1036] Processing user requests

[1037] When a user attempts to purchase a product on an e-commerce site, they send a recommendation request to the server via their device, which includes the user ID and their current purchase intention.

[1038] Input: Recommendation request (e.g., "User ID: 12345, Purchase Intent: Detergent, Category: Fabric Softener").

[1039] Output: The request data sent to the server.

[1040] Step 6:

[1041] Data analysis and recommendation generation

[1042] The server searches the database for the corresponding user ID and obtains usage data and emotion data.

[1043] The server uses a generative AI model (e.g., GPT-4) to analyze the acquired data and identify the best products for the user.

[1044] Input: Usage and sentiment data associated with the user ID.

[1045] Output: The generated recommendation result (e.g., "Fabric softener with relaxing effects").

[1046] How it works: The server retrieves data using an SQL query and sends a prompt to the generative AI model, such as, "Please suggest the best product for user ID: 12345 based on their washing machine usage frequency and emotional data from the past week."

[1047] Step 7:

[1048] Presenting recommendations

[1049] The server returns the generated recommendation results to the user.

[1050] The user checks the recommended products through the device and considers purchasing them on the e-commerce site.

[1051] Input: The generated recommendation results.

[1052] Output: Product information presented to the user (e.g., "Relaxing Fabric Softener (with link)").

[1053] Specific behavior: Recommendation results are displayed on the user's device.

[1054] (Application example 2)

[1055] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1056] Conventional recommendation systems suggest products based solely on data collected from users' IoT home appliances, and therefore are unable to reflect the user's emotional state or temporary needs. This results in low recommendation accuracy and makes it difficult to suggest optimal products to users.

[1057] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1058] In this invention, the server includes means for receiving data collected from IoT home appliances, means for analyzing the data using a generative AI and recommending optimal products to the user, means for acquiring and analyzing user emotional data, means for integrating and analyzing the IoT home appliance data and the emotional data to identify optimal products, and means for presenting the recommended products to the user via a smartphone application. This enables optimal product suggestions that take into account the user's emotional state and temporary needs.

[1059] "IoT home appliances" are household electrical appliances that are connected to the Internet and can send and receive data.

[1060] "Generative AI" is a system that uses artificial intelligence technology to analyze input data and generate new data and information.

[1061] "Emotional data" is data that represents the emotional state of a user and is obtained through facial expression analysis, voice analysis, and text analysis.

[1062] "Recommendation" refers to suggesting appropriate products or services to users.

[1063] A "smartphone application" is a software program that runs on a smartphone and allows for user interaction.

[1064] A "server" is a computer system that operates on a network and stores, analyzes, and manages data.

[1065] A "database" is a system that efficiently stores, manages, and searches structured data.

[1066] A "generative AI model" is a trained artificial intelligence algorithm that is used to generate new data or information based on input data.

[1067] A "recommendation request" is when a user requests the system to suggest products based on specific criteria.

[1068] This embodiment of the present invention describes a system that analyzes data collected from IoT home appliances and recommends optimal products to users based on generative AI and emotional data.

[1069] This system includes a means for collecting data from IoT home appliances, a means for analyzing the data using generative AI, a means for acquiring and analyzing user sentiment data, and a means for identifying and recommending optimal products based on the integrated data analysis results. The recommendation results are presented to the user via a smartphone application.

[1070] Overview of the entire program

[1071] First, when a user opens the smartphone application, usage data from IoT home appliances is collected. At the same time, the user's emotional data is also acquired. This data is sent from the smartphone to a server, which analyzes the data using a generative AI model and recommends the most suitable products. Finally, the recommended products are displayed to the user on the smartphone application.

[1072] Hardware and software used

[1073] Hardware

[1074] Smartphone: Used as the primary device for data collection and display.

[1075] IoT appliances: household electrical appliances such as washing machines, refrigerators, and vacuum cleaners.

[1076] software

[1077] Emotion analysis engine: Uses facial expression analysis (OpenCV) and voice analysis (Google Cloud Speech-to-Text).

[1078] Database: Data management is performed on the server side using MySQL.

[1079] Generative AI model: Data analysis is performed using GPT-3 (OpenAI).

[1080] Server application: Implement server-side processing using Python and Flask.

[1081] Specific examples of processing

[1082] When a user launches the smartphone application, data is automatically collected from IoT home appliances. For example, data on the use of a washing machine five times a week is collected. At the same time, the user's emotional data is also collected by analyzing facial expressions and voice. If the user's face shows any signs of stress, that data is also collected.

[1083] This data is then sent from the smartphone to a server and stored in a database. The server analyzes this data using GPT-3 to identify the most suitable product, such as a fabric softener with a relaxing effect. The recommendation results are then sent to the user's smartphone application, allowing the user to review the recommended product and consider purchasing it.

[1084] Examples of prompt statements

[1085] Here is an example prompt in sentence text form:

[1086] User ID: 12345

[1087] IoT data: Washing machine usage: 5 times / week

[1088] Emotion data: Stress (from facial and voice analysis)

[1089] In this way, the system provides users with the most suitable products based on usage data from IoT home appliances and user emotional data, thereby realizing a more personalized customer experience.

[1090] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1091] Step 1:

[1092] Data collection

[1093] The terminal collects usage data from IoT appliances. For example, the number of times a washing machine is used and the operating mode are acquired. The input is the IoT data acquired from the appliance, and the output is a temporary file containing the data or data stored in memory.

[1094] Step 2:

[1095] Acquiring emotion data

[1096] The device recognizes the user's face and obtains emotional data using a facial expression analysis engine (OpenCV). It also extracts emotional data from voice using a voice analysis engine (Google Cloud Speech-to-Text). The input is the user's real-time video and audio, and the output is the emotional data obtained by analyzing them.

[1097] Step 3:

[1098] Data transmission

[1099] The IoT appliance data and emotion data collected by the device are sent to the server. The input is the data obtained in Step 1 and Step 2, and the output is the data sent to the server.

[1100] Step 4:

[1101] Receiving and storing data

[1102] The server receives the transmitted data and stores it in a database. The input is the data transmitted from the terminal, and the output is the data stored in the database.

[1103] Step 5:

[1104] Data analysis

[1105] The server runs a generative AI model (GPT-3) to analyze the stored data. The input is the user's IoT appliance data and emotion data retrieved from the database, and the output is the analysis results to identify the best product for the user.

[1106] Step 6:

[1107] Recommendation Generation

[1108] The server uses the generative AI model to recommend the optimal product based on the analysis results. The input is the analysis results obtained in step 5, and the output is the recommendation result.

[1109] Step 7:

[1110] Recommendation presentation

[1111] The server sends the recommendation results to the device, which then displays them to the user. The user can check the recommended products through a smartphone application and consider purchasing them. The input is the recommendation results from the server, and the output is a list of products displayed on the device.

[1112] This will enable users to easily find the best products based on their IoT home appliance usage and emotional state.

[1113] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1114] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1115] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1116] [Fourth embodiment]

[1117] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1118] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1119] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1120] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1121] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1122] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1123] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1124] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1125] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1126] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1127] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1128] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1129] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1130] This embodiment of the present invention describes a system that uses data collected from IoT home appliances to implement product recommendations using generative AI. Below, the program processing flow and specific examples are explained in natural language.

[1131] Program processing

[1132] 1. Collecting and transmitting IoT home appliance data:

[1133] When a user uses an IoT home appliance, its usage data is collected. This data includes, for example, the operating status and usage mode of a washing machine. This data is acquired by the device and sent to a server.

[1134] 2. Receiving and storing data:

[1135] The server receives the IoT home appliance data sent from the device. The received data is stored in a database along with the user ID. The database stores information such as each user's device usage history and operation patterns.

[1136] 3. Receiving User Requests:

[1137] When a user attempts to purchase a product on an e-commerce site, a recommendation request is sent to the server, including the user ID.

[1138] 4. Data analysis and recommendation generation:

[1139] The server searches the stored device data and retrieves the relevant data based on the user ID. Generative AI is used to analyze the retrieved data and identify the most suitable products for the user. For example, if a user uses their washing machine frequently, it will recommend a detergent that suits their lifestyle.

[1140] 5. Presenting recommendations:

[1141] The server returns the generated recommendation results to the user. The recommendation includes a list of related products and detailed information. The user can then view the recommended products through their device and consider purchasing them.

[1142] Specific examples

[1143] A specific example is given below.

[1144] A user uses an IoT home appliance (e.g., a washing machine) on a daily basis. The usage data (e.g., using the washing machine five times a week) is collected by the device and sent to a server.

[1145] The server receives this data and stores it in a database along with the user ID.

[1146] Later, when the same user tries to purchase detergent on an e-commerce site, a recommendation request is sent to the server.

[1147] The server uses generative AI to analyze the stored data and recommend the best products for each user, such as highly concentrated detergents or environmentally friendly detergents.

[1148] The server returns the generated recommendation results to the user, and the user can check the recommended products and consider purchasing them on the EC site.

[1149] This will enable users to find products efficiently and conveniently, providing opportunities for increased sales in the e-commerce industry, and will also lead to increased user satisfaction for IoT home appliance manufacturers, strengthening the ecosystem as a whole.

[1150] The processing flow will be explained below.

[1151] Step 1:

[1152] When a user uses an IoT appliance, for example, a washing machine, the operating status, mode of use, and duration of use are recorded.

[1153] Step 2:

[1154] The device collects usage data for IoT appliances, including washing machine usage data, which includes detailed information such as the number of times it has been used, the duration of its use, and the mode of use.

[1155] Step 3:

[1156] The device sends the collected data to the server. The data sent is in the form of a user ID and device data, for example in JSON format as shown below.

[1157] json

[1158] {

[1159] "user_id": "12345",

[1160] "device_data": {

[1161] "washing_machine": {

[1162] "usage_count": 5,

[1163] "total_runtime_minutes": 300,

[1164] "modes_used": ["normal", "quick"]

[1165] }

[1166] }

[1167] }

[1168] Step 4:

[1169] The server receives the data sent from the device, stores it in a database in an appropriate format, and uses it for later analysis.

[1170] Step 5:

[1171] When a user wants to purchase a product on an e-commerce site, they send a recommendation request to the server via their device. The request includes the user ID and asks the server to suggest products.

[1172] Step 6:

[1173] When the server receives a user request, it searches the stored database for data for the corresponding user ID, for example, to retrieve washing machine usage data.

[1174] Step 7:

[1175] The server analyzes the data using generative AI. It analyzes the user's usage in detail and identifies the most suitable product for the user. For example, it identifies highly concentrated detergents and environmentally friendly detergents for users who use their washing machine frequently.

[1176] Step 8:

[1177] The server lists the products identified by the AI ​​and presents them to the user as recommendations, including the product name, description, and purchase link.

[1178] Step 9:

[1179] The user can check the recommended products through their device, consider purchasing the products based on this information, and proceed with the purchase process on the e-commerce site.

[1180] Step 10:

[1181] The server continues to store the user's purchase history in a database to help inform future recommendations. This continuous data collection and analysis allows for continuous improvement of the user experience.

[1182] Example 1

[1183] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1184] Today's consumers have diverse needs and spend a lot of time and effort selecting products. In particular, with the spread of IoT devices, there is a demand for utilizing the large amount of usage data obtained from these devices to propose individually optimized products. However, conventional systems face the challenge of effectively analyzing this data and recommending the best products for users. Furthermore, while analysis technology using generative AI is rapidly evolving, there are few systems that can properly utilize it.

[1185] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1186] In this invention, the server includes means for collecting operation and usage data from IoT devices used by users and transmitting it via the terminal, means for receiving the data transmitted from the terminal and storing it in a database together with user identification information, means for receiving a recommendation request when the user attempts to purchase a product, means for using a generative AI model to analyze the stored data and identify the product best suited to the user, and means for presenting the generated recommendation results to the user. This makes it possible to efficiently analyze large amounts of IoT device data and quickly and accurately provide the best product to each user.

[1187] "User" refers to a consumer who uses IoT devices on a daily basis and whose usage data is collected and analyzed.

[1188] "IoT devices" refers to home appliances and other electronic devices that are connected to the Internet and can automatically send and receive data.

[1189] "Operational and usage data" refers to information about how a user uses a particular device, such as the device's operating status, usage mode, and frequency of use.

[1190] "Terminal" refers to a device that transmits data collected from IoT devices to a server. Specifically, this includes smartphones and smart home hubs.

[1191] "Server" refers to a central computing unit that receives, stores, analyzes, and processes data over a network.

[1192] "Database" means a system running on a Server for structuring and storing Usage Data, including User Identification Information.

[1193] "User identification information" refers to information for identifying a specific user, and includes, for example, a user ID.

[1194] "Recommendation Request" refers to a request sent by a user to a server to request recommendations for the best products for a particular product category.

[1195] "Generative AI model" refers to an artificial intelligence model used to analyze collected data and identify the best products for users.

[1196] A "prompt" is a text-based instruction entered into a generative AI model that clearly states the data to be analyzed and the purpose.

[1197] "Recommendation Results" refers to the optimal product suggestions presented to the user as a result of analysis by the generative AI model.

[1198] This invention relates to a system that uses data collected from users' IoT devices to realize product recommendations using a generative AI model. This system can perform all processes from data collection to analysis and product proposals. The program processing flow is explained in detail below.

[1199] Collecting and transmitting IoT home appliance data

[1200] When a user uses an IoT device (e.g., a washing machine or refrigerator), data on its operation and usage is collected. This data includes, for example, its operating status, usage mode, and frequency of use. This data is acquired by a device (e.g., a smartphone or smart home hub). The device then transmits the collected data to a server via Wi-Fi or a mobile network.

[1201] Receiving and storing data

[1202] The server receives the data sent from the device. This process is performed through a secure API endpoint, ensuring safe data transmission. The received data is then stored in a database along with user identification information. A NoSQL database (e.g., MongoDB) is suitable for this database, as it is characterized by its scalability and flexibility.

[1203] Receiving a user request

[1204] When a user attempts to purchase a product on an e-commerce site, they send a recommendation request. This request is sent from the device (for example, a browser on a PC or smartphone) to the server. This request includes the user ID and the product category they are considering purchasing. The HTTPS protocol is used for communication to ensure data security.

[1205] Data analysis and recommendation generation

[1206] The server processes the received request and searches for device usage data stored in a database. It retrieves the relevant data based on the user's identity and inputs a prompt statement into the generative AI model. An example of a prompt statement would be, "Recommend the best detergent based on the usage data of user ID: 12345." The generative AI model uses text analysis and machine learning algorithms to analyze the user's behavioral patterns and identify the best product.

[1207] Presenting recommendations

[1208] The server receives the recommendation results obtained from the generative AI model and presents them to the user. These results include a list of related products and their detailed information. For example, an e-commerce site in the browser displays images, descriptions, prices, etc. of the recommended products. The user can review these and consider purchasing them.

[1209] Specific examples

[1210] Specific examples are shown below.

[1211] Suppose that usage data for an IoT device (e.g., a washing machine) that a user uses daily is collected five times a week.

[1212] This usage data includes information such as time and mode of use, and is periodically sent to a server and stored in a database.

[1213] Later, when the same user tries to purchase detergent on an e-commerce site, they send a recommendation request to the server.

[1214] Based on the stored data, the server sends a prompt to the generated AI, such as "Recommend the optimal detergent based on the usage data of user ID: 12345."

[1215] The generative AI model will return results that recommend, for example, "highly concentrated detergents and environmentally friendly detergents."

[1216] The results are presented to the user on their browser, allowing them to consider purchasing based on the recommended products.

[1217] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1218] Step 1:

[1219] When a user uses an IoT device, data on its operation and usage status is collected by the device. Specific data includes operating time, usage mode, and frequency of use. This data is acquired in real time and saved in JSON format. Sensor data and usage logs from the IoT device operated by the user are used as input. This data is temporarily saved in the device as output. Specific operations include reading data from sensors, converting it into a specific format, and saving it.

[1220] Step 2:

[1221] The device transmits the collected data to the server at regular intervals. This transmission process uses Wi-Fi or mobile networks and uses a secure communication protocol (e.g., HTTPS) to ensure data security. As input, the usage data collected in step 1 is in JSON format. As output, the data is sent to and received from the server. Specific operations include encoding, packaging, and sending the data to the server.

[1222] Step 3:

[1223] The server receives data sent from the device and stores it in a database. This data contains user identification information, so it is stored and associated with a specific user. The usage data sent from the device is used as input. The received data is stored in the database as output. Specific operations include receiving data at an API endpoint, decoding the data, and inserting it into the database.

[1224] Step 4:

[1225] When a user attempts to purchase a product on an e-commerce website, a recommendation request is sent to the server. The request includes the user ID and the desired product category. The request data from the user is used as input. The request content is received and saved on the server as output. Specific operations include collecting form data, encoding the data, and sending it to the server.

[1226] Step 5:

[1227] Based on the received request, the server retrieves relevant device usage data from the database. It also creates and sends a prompt statement to the generative AI model based on this data. The user's usage data stored in the database and the request content are used as input. The prompt statement is sent to the generative AI model as output. An example of a prompt statement is "Recommend the optimal detergent based on the usage data of user ID: 12345." Specific operations include generating and executing a database query, formatting the retrieved data, and sending it to the generative AI model.

[1228] Step 6:

[1229] The generative AI model analyzes data based on the received prompt and identifies the most suitable products. It uses machine learning algorithms to analyze user behavior patterns and product characteristics to recommend the most suitable products. The prompt is used as input. As output, a list of the most suitable products is generated as the recommendation result. Specific operations include data analysis using natural language processing and machine learning algorithms, and the generation of recommendation results.

[1230] Step 7:

[1231] The server receives the recommendation results obtained from the generative AI model and presents them to the user. These results include a list of related products and their details. The recommendation results from the generative AI model are used as input. The output provides a list of products and their details to be displayed to the user. Specific operations include receiving the result data, converting the data format, and displaying it on the user's device.

[1232] This allows users to efficiently find the product that best suits them and consider purchasing it.

[1233] (Application example 1)

[1234] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1235] In recent years, the spread of IoT home appliances has made users' daily lives increasingly convenient. However, there is still a lack of a system that can effectively utilize the vast amount of data from IoT home appliances to suggest products that are suitable for each individual user. In addition, there is a lack of means to provide timely and accurate product recommendations to users. As a result, users are faced with the problem of spending time and effort finding the products they need.

[1236] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1237] In this invention, the server includes means for receiving data collected from IoT home appliances, means for analyzing the data using a generating AI and recommending optimal products to the user, means for presenting the recommended products to the user, and means for automatically recommending products based on the data collected from the IoT home appliances and providing them as push notifications to a smartphone. This enables users to efficiently select products by utilizing the data from the IoT home appliances and receive optimal product recommendations in a timely manner.

[1238] "IoT home appliances" are electrical devices used in everyday life that are connected to the Internet and have the ability to communicate with each other.

[1239] "Means for receiving data" refers to the functions of devices or software for collecting usage information and operation information sent from IoT home appliances.

[1240] "Generative AI" is a type of artificial intelligence that refers to algorithms and models that can analyze data and generate new information and recommendations.

[1241] "Means for recommending products" refers to functions and mechanisms that analyze collected data, select the most suitable products for the user, and present them as a list.

[1242] The "means for presenting products" refers to the function of a device or software for visually displaying recommended products to a user.

[1243] "Push notification" is a function that automatically notifies communication devices such as smartphones of information in real time.

[1244] A "server" is a computer system that processes and stores data over a network and provides services to other computers and terminals.

[1245] "Data analysis" is the process of analyzing collected data to extract meaningful information and patterns.

[1246] A "user ID" is an identifier that uniquely identifies each user and is used to manage information for each user in databases and systems.

[1247] This embodiment of the present invention describes a system that uses data collected from IoT home appliances to implement product recommendations using generative AI. Below, the processing flow and specific examples of the system program are explained in natural language.

[1248] Hardware and Software Configuration

[1249] The server uses a cloud server (e.g., AWS Lambda, AWS S3). The database uses AWS RDS or a NoSQL database, and the generative AI model uses GPT-4 or BERT. The device is a smartphone (compatible with iOS or Android), and the IoT home appliance is assumed to have a Wi-Fi module or Bluetooth functionality.

[1250] Key processing steps and functions of the system

[1251] 1. IoT appliance data collection:

[1252] The server automatically receives operational information about IoT home appliances in the home (for example, the number of times the washing machine is used or how often the refrigerator is opened and closed) from the user's smartphone.

[1253] 2. Data storage:

[1254] The server then stores the received data, including the user ID, in a cloud database. This data includes the operating status, usage mode, and frequency of use of the appliances.

[1255] 3. Data analysis and recommendation generation:

[1256] The server analyzes the stored data using a generative AI model. As a result of the analysis, the best products for the user are identified and a list of those products is generated. For example, a user who frequently opens and closes their refrigerator will be recommended foods that can be stored for a long time or products that need to be replenished frequently.

[1257] 4. Product push notifications:

[1258] The server then delivers the generated recommendation results to the smartphone as a push notification, which includes products selected according to the user's interests.

[1259] Specific examples

[1260] For example, if a user opens and closes their refrigerator more than 20 times per day, this data is sent to a server via their smartphone and stored in a database on the cloud. The generative AI model then analyzes this data and generates recommendations such as "fresh, farm-fresh vegetable sets" and "long-lasting frozen foods." These recommended products are displayed on the user's smartphone as push notifications. They include automatically generated recommendations such as, "If you open and close your refrigerator more than 20 times per day, we recommend fresh, farm-fresh vegetables!"

[1261] This system will enable users to efficiently select products using data from IoT home appliances and receive timely recommendations for the best products. It is also expected to boost sales in the e-commerce industry and lead to improved user satisfaction for IoT home appliance manufacturers.

[1262] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1263] Step 1:

[1264] When a user uses an IoT home appliance such as a washing machine or refrigerator, data on its usage and frequency of operation is collected by the device. This device is typically a smartphone, and this data includes the number of times the appliance has been used, the mode of use, and the operating time. The device collects this data and sends it to a server. The input is the usage data from the IoT home appliance, and the output is the data sent to the server. Specifically, the smartphone obtains the data via Bluetooth or WiFi and uploads it to the server in a specific format.

[1265] Step 2:

[1266] The server receives the data sent from the device. The received data is stored in a database along with the user ID. The input is the data received from the device, and the output is the data stored in the database. Specifically, the server receives the data using HTTPS, associates it with the user ID, and stores it in a NoSQL database.

[1267] Step 3:

[1268] When a user attempts to purchase a product on an e-commerce site, a recommendation request is sent to the server. The input is the recommendation request from the user, and the output is the status of the request. Specifically, when the user presses the recommendation request button on the smartphone app, that information is sent to the server.

[1269] Step 4:

[1270] The server searches the stored device data and retrieves the relevant data based on the user ID. It then uses a generative AI model (e.g., GPT-4) to analyze the retrieved data and identify the best products for the user. The input is the stored device data and the recommendation request, and the output is a list of recommended products as the analysis result. Specifically, the server queries the data related to the user ID from the database and inputs that data into the generative AI model. The generative AI model performs the analysis and lists the recommended products.

[1271] Step 5:

[1272] The server returns the generated recommendation results to the user and presents them as a push notification to their smartphone. The input is the recommendation results generated by the generative AI model, and the output is the content displayed on the user's smartphone. Specifically, the server uses a push notification service (for example, Firebase Cloud Messaging) to send the generated product list to the smartphone. The user's smartphone receives the notification and displays it on the screen.

[1273] These processing steps enable the user to efficiently receive optimal product recommendations.

[1274] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1275] This embodiment of the present invention describes a system that analyzes data collected from IoT home appliances and uses a generative AI and an emotion engine to recommend optimal products to users. Below, the program processing flow and specific examples are explained in natural language.

[1276] Program processing

[1277] 1. Collecting and transmitting IoT home appliance data:

[1278] When a user uses an IoT appliance, its usage data is collected. This data includes, for example, the operation status, mode of use, and duration of use of a washing machine. This data is acquired by the device and sent to a server.

[1279] 2. Receiving and storing data:

[1280] The server receives the IoT home appliance data sent from the device. The received data is stored in a database along with the user ID. The database stores information such as each user's device usage history and operation patterns.

[1281] 3. Acquiring emotion data:

[1282] The device acquires the user's emotional data. The emotion engine recognizes the user's emotions through facial expression analysis, voice analysis, and / or text analysis. This emotional data is also sent to the server.

[1283] 4. Receiving and storing emotional data:

[1284] The server receives the user's emotion data sent from the device and stores it in a database.

[1285] 5. Receiving User Requests:

[1286] When a user attempts to purchase a product on an e-commerce site, they send a recommendation request to the server via their device. The request includes the user ID and asks the server to suggest products.

[1287] 6. Data analysis and recommendation generation:

[1288] The server searches the database for the corresponding user ID and retrieves IoT appliance data and emotion data. This data is then analyzed using generative AI to identify the most suitable product for the user. For example, if a user frequently uses their washing machine and the emotion engine recognizes that they are feeling stressed, the server will recommend a fabric softener with a relaxing effect.

[1289] 7. Presenting recommendations:

[1290] The server returns the generated recommendation results to the user. The recommendation includes a list of related products and detailed information. The user can then view the recommended products through their device and consider purchasing them.

[1291] Specific examples

[1292] A specific example is given below.

[1293] A user uses an IoT home appliance (e.g., a washing machine) on a daily basis. The usage data (e.g., using the washing machine five times a week) is collected by the device and sent to a server.

[1294] The server receives this data and stores it in a database along with the user ID.

[1295] Meanwhile, the device recognizes the user's emotions in real time and collects emotional data. For example, if the user is feeling stressed, that information can be obtained through facial expression analysis and voice analysis.

[1296] The server receives this emotion data and stores it in a database.

[1297] Later, when the same user tries to purchase detergent on an e-commerce site, a recommendation request is sent to the server.

[1298] The server uses generative AI to analyze the stored data and recommend the best product for the user, such as a fabric softener with a relaxing effect.

[1299] The server returns the generated recommendation results to the user, and the user can check the recommended products and consider purchasing them on the EC site.

[1300] This will enable users to find products efficiently and conveniently, and receive appropriate product suggestions based on their emotional state. It will also provide opportunities for increased sales in the e-commerce industry and lead to increased user satisfaction for IoT home appliance manufacturers, strengthening the ecosystem as a whole.

[1301] The processing flow will be explained below.

[1302] Step 1:

[1303] When a user uses an IoT home appliance, for example, a washing machine, the operating status, usage mode, and usage time of the washing machine are recorded.

[1304] Step 2:

[1305] The device retrieves usage data collected from IoT appliances, including details such as the number of times the washing machine was run, the duration of its run, and the mode used.

[1306] Step 3:

[1307] The device sends the collected data to the server in JSON format along with the user ID.

[1308] Step 4:

[1309] The server receives the data sent from the device and stores it in a database along with the user ID.

[1310] Step 5:

[1311] The device acquires the user's emotional data. The emotion engine recognizes the user's emotions using facial expression analysis, voice analysis, and text analysis. This emotional data is also sent to the server.

[1312] Step 6:

[1313] The server receives the emotion data sent from the device and stores it in a database, which includes the emotional state the user is feeling.

[1314] Step 7:

[1315] When a user attempts to purchase a product on an e-commerce website, they send a recommendation request to the server via their device. The request includes the user ID and requests the server to suggest products that suit the user.

[1316] Step 8:

[1317] The server retrieves the relevant data from the database based on the user ID, including usage data and emotion data of IoT appliances.

[1318] Step 9:

[1319] The server analyzes the acquired data using a generative AI, which identifies the best product for the user based on their usage and emotional state. For example, it identifies a fabric softener with a relaxing effect by taking into account how often the washing machine is used and the user's stressful situation.

[1320] Step 10:

[1321] The server lists the identified products and presents them to the user as recommendations, including the product name, description, and purchase link.

[1322] Step 11:

[1323] The user checks the product information presented on the device, and can use this information to consider purchasing the product and proceed with the purchase process on the e-commerce site.

[1324] Step 12:

[1325] The server stores the user's purchase history in a database, and uses the purchase data for analysis to improve the accuracy of future recommendations.

[1326] This allows users to receive product suggestions that take their emotional state into consideration, helping them find more suitable products more efficiently, while providing e-commerce sites and IoT home appliance manufacturers with the opportunity to improve user satisfaction and increase sales.

[1327] Example 2

[1328] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1329] Conventional recommendation systems using IoT home appliance data simply recommend products based on usage history without considering the user's emotional state, which can result in incomplete responses to user needs. Furthermore, improving the accuracy of recommendations requires diverse data sources and advanced analytical techniques, but integrating them efficiently has proven difficult.

[1330] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting and transmitting usage data when a user uses an IoT home appliance, means for a terminal to acquire and transmit emotional data of the user, means for the server to receive and store the usage data and emotional data, means for analyzing the stored data using a generative AI model based on a recommendation request from the user and identifying the most suitable product for the user, and means for presenting the recommended product to the user. This enables advanced recommendations that take into account the user's usage history as well as their real-time emotional state.

[1331] "Users" refer to consumers who use the system to operate IoT home appliances and receive product recommendations.

[1332] "IoT home appliances" refer to home appliances that can send and receive data via the Internet.

[1333] "Usage data" refers to information about the operation of IoT home appliances, such as their operating status, usage mode, and frequency of use.

[1334] "Terminal" refers to a device operated by a user that collects usage data and emotional data and transmits it to a server.

[1335] "Emotional data" refers to information about a user's emotional state, and refers to data obtained through facial expression analysis, voice analysis, and text analysis.

[1336] "Server" refers to the computer system that receives and stores usage and emotion data and analyzes such data using generative AI models.

[1337] "Database" refers to a system for systematically storing and managing usage data and emotional data.

[1338] A "recommendation request" refers to a request that a user sends to a server via a terminal indicating that they would like to receive product suggestions.

[1339] "Generative AI models" refer to advanced machine learning models used to make predictions and recommendations.

[1340] "Recommendations" refers to product suggestions that are optimal for a user, identified by a generative AI model based on the user's usage and sentiment data.

[1341] This embodiment of the present invention describes a system that analyzes data collected from IoT home appliances and recommends optimal products to users using a generative AI model and an emotion engine.

[1342] Hardware and software used

[1343] Hardware:

[1344] IoT home appliances (e.g. IoT washing machine)

[1345] Edge terminal (data collection device)

[1346] software:

[1347] Facial Expression Analysis Software

[1348] Voice Analysis Software

[1349] Text Analysis Software

[1350] Database Management Systems

[1351] Generative AI models (e.g., GPT-4)

[1352] Specific program processing flow

[1353] 1. Collecting and transmitting IoT home appliance data:

[1354] Every time a user uses an IoT home appliance, usage data is collected in real time. This data includes specific information such as usage time, usage mode, and power consumption. The device receives the data and sends it to a server via the Internet. For example, after using a washing machine, the device sends "User ID: 12345, Washing machine operation: 12:00-12:45, Mode: Standard" to the server.

[1355] 2. Receiving and storing data:

[1356] The server receives the data sent from the device and stores it in a database along with the user ID. Specifically, it stores information such as the user ID, device ID, operating time, usage mode, and power consumption in a database table.

[1357] 3. Acquiring emotion data:

[1358] The device acquires the user's emotional data. The emotion engine analyzes the user's facial expressions to determine whether they are smiling or stressed. Voice analysis is also used to analyze emotions from the tone and speed of the voice, and text analysis is also performed. The device then sends the collected emotional data back to the server. For example, the user can input what they want to say by voice, and the device generates data such as "User ID: 12345, Emotion: Stress, Voice Tone: High, Analysis Time: 14:00" and sends it to the server.

[1359] 4. Receiving and storing emotional data:

[1360] The server receives the emotion data sent from the device and stores it in a database, including information such as the user ID, emotional state, voice tone, and analysis time.

[1361] 5. Processing User Requests:

[1362] When a user attempts to purchase a product on an e-commerce site, they send a recommendation request to the server via their device. The request includes the user ID and current purchase intention. The device then sends this request to the server, along with the category and details of the product the user wants to check. For example, a request for "User ID: 12345, purchase intention: detergent, category: fabric softener" is sent.

[1363] 6. Data analysis and recommendation generation:

[1364] The server searches the database for the corresponding user ID and retrieves the collected IoT appliance data and emotion data. A generative AI model (e.g., GPT-4) analyzes this data and identifies the optimal product based on the user's emotional state and appliance usage patterns. Specifically, the server first executes an SQL query to retrieve the data and then sends a prompt to the AI ​​model. An example prompt might be, "Please suggest the optimal product based on the frequency of washing machine use and emotion data for user ID: 12345 over the past week."

[1365] 7. Presenting recommendations:

[1366] The server returns the generated recommendation results to the user, which include a list of related products and detailed information. The user can then review the recommended products through their device and consider purchasing them on the e-commerce site. For example, the information displayed might be, "Recommended product for you: Fabric softener with a relaxing effect (with link)."

[1367] Specific examples

[1368] Example: A user frequently uses a washing machine and emotion analysis indicates that they are stressed.

[1369] Example prompt:

[1370] "User ID: 12345 uses the washing machine five times a week. Recent emotional data indicates that he is feeling stressed. Please recommend the best product for this user."

[1371] This system allows users to receive optimal product recommendations based on their usage habits and emotional state, providing a better purchasing experience.

[1372] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1373] Step 1:

[1374] Collecting and transmitting IoT home appliance data

[1375] Users use IoT home appliances on a daily basis.

[1376] The device collects usage data (e.g., usage time, usage mode, power consumption) using built-in sensors and communication modules.

[1377] The usage data collected by the device is structured and sent to a server via the Internet.

[1378] Input: IoT appliance operation data (e.g., "User ID: 12345, Washing machine operation: 12:00~12:45, Mode: Standard").

[1379] Output: Structured data sent to the server.

[1380] Step 2:

[1381] Receiving and storing data

[1382] The server receives the usage data sent from the terminal.

[1383] The server saves the received data in the database, along with the user ID.

[1384] Input: Usage data sent from the device.

[1385] Output: Usage data stored in a database (e.g. "User ID: 12345, Device ID, Uptime, Usage Mode, Power Consumption").

[1386] Specific operation: When the server receives new data, it inserts the data into the database using the SQL INSERT statement.

[1387] Step 3:

[1388] Acquiring emotion data

[1389] The device acquires the user's emotional data, and the emotion engine performs facial expression analysis (e.g., using the camera), voice analysis (e.g., using the microphone), and text analysis.

[1390] The device sends the collected emotional data to the server, including the emotional state (e.g., stress, happiness), voice tone, and analysis time.

[1391] Input: Real-time user facial, voice, and text input.

[1392] Output: Structured emotion data sent to the server.

[1393] Step 4:

[1394] Receiving and storing emotional data

[1395] The server receives the emotion data sent from the terminal.

[1396] The server stores the received emotion data in a database, along with the user ID.

[1397] Input: Emotion data sent from the device.

[1398] Output: Emotion data stored in a database (e.g., "User ID: 12345, emotional state, voice tone, analysis time").

[1399] Specific operation: When the server receives new emotion data, it inserts the data into the database using the SQL INSERT statement.

[1400] Step 5:

[1401] Processing user requests

[1402] When a user attempts to purchase a product on an e-commerce site, they send a recommendation request to the server via their device, which includes the user ID and their current purchase intention.

[1403] Input: Recommendation request (e.g., "User ID: 12345, Purchase Intent: Detergent, Category: Fabric Softener").

[1404] Output: The request data sent to the server.

[1405] Step 6:

[1406] Data analysis and recommendation generation

[1407] The server searches the database for the corresponding user ID and obtains usage data and emotion data.

[1408] The server uses a generative AI model (e.g., GPT-4) to analyze the acquired data and identify the best products for the user.

[1409] Input: Usage and sentiment data associated with the user ID.

[1410] Output: The generated recommendation result (e.g., "Fabric softener with relaxing effects").

[1411] How it works: The server retrieves data using an SQL query and sends a prompt to the generative AI model, such as, "Please suggest the best product for user ID: 12345 based on their washing machine usage frequency and emotional data from the past week."

[1412] Step 7:

[1413] Presenting recommendations

[1414] The server returns the generated recommendation results to the user.

[1415] The user checks the recommended products through the device and considers purchasing them on the e-commerce site.

[1416] Input: The generated recommendation results.

[1417] Output: Product information presented to the user (e.g., "Relaxing Fabric Softener (with link)").

[1418] Specific behavior: Recommendation results are displayed on the user's device.

[1419] (Application example 2)

[1420] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1421] Conventional recommendation systems suggest products based solely on data collected from users' IoT home appliances, and therefore are unable to reflect the user's emotional state or temporary needs. This results in low recommendation accuracy and makes it difficult to suggest optimal products to users.

[1422] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1423] In this invention, the server includes means for receiving data collected from IoT home appliances, means for analyzing the data using a generative AI and recommending optimal products to the user, means for acquiring and analyzing user emotional data, means for integrating and analyzing the IoT home appliance data and the emotional data to identify optimal products, and means for presenting the recommended products to the user via a smartphone application. This enables optimal product suggestions that take into account the user's emotional state and temporary needs.

[1424] "IoT home appliances" are household electrical appliances that are connected to the Internet and can send and receive data.

[1425] "Generative AI" is a system that uses artificial intelligence technology to analyze input data and generate new data and information.

[1426] "Emotional data" is data that represents the emotional state of a user and is obtained through facial expression analysis, voice analysis, and text analysis.

[1427] "Recommendation" refers to suggesting appropriate products or services to users.

[1428] A "smartphone application" is a software program that runs on a smartphone and allows for user interaction.

[1429] A "server" is a computer system that operates on a network and stores, analyzes, and manages data.

[1430] A "database" is a system that efficiently stores, manages, and searches structured data.

[1431] A "generative AI model" is a trained artificial intelligence algorithm that is used to generate new data or information based on input data.

[1432] A "recommendation request" is when a user requests the system to suggest products based on specific criteria.

[1433] This embodiment of the present invention describes a system that analyzes data collected from IoT home appliances and recommends optimal products to users based on generative AI and emotional data.

[1434] This system includes a means for collecting data from IoT home appliances, a means for analyzing the data using generative AI, a means for acquiring and analyzing user sentiment data, and a means for identifying and recommending optimal products based on the integrated data analysis results. The recommendation results are presented to the user via a smartphone application.

[1435] Overview of the entire program

[1436] First, when a user opens the smartphone application, usage data from IoT home appliances is collected. At the same time, the user's emotional data is also acquired. This data is sent from the smartphone to a server, which analyzes the data using a generative AI model and recommends the most suitable products. Finally, the recommended products are displayed to the user on the smartphone application.

[1437] Hardware and software used

[1438] Hardware

[1439] Smartphone: Used as the primary device for data collection and display.

[1440] IoT appliances: household electrical appliances such as washing machines, refrigerators, and vacuum cleaners.

[1441] software

[1442] Emotion analysis engine: Uses facial expression analysis (OpenCV) and voice analysis (Google Cloud Speech-to-Text).

[1443] Database: Data management is performed on the server side using MySQL.

[1444] Generative AI model: Data analysis is performed using GPT-3 (OpenAI).

[1445] Server application: Implement server-side processing using Python and Flask.

[1446] Specific examples of processing

[1447] When a user launches the smartphone application, data is automatically collected from IoT home appliances. For example, data on the use of a washing machine five times a week is collected. At the same time, the user's emotional data is also collected by analyzing facial expressions and voice. If the user's face shows any signs of stress, that data is also collected.

[1448] This data is then sent from the smartphone to a server and stored in a database. The server analyzes this data using GPT-3 to identify the most suitable product, such as a fabric softener with a relaxing effect. The recommendation results are then sent to the user's smartphone application, allowing the user to review the recommended product and consider purchasing it.

[1449] Examples of prompt statements

[1450] Here is an example prompt in sentence text form:

[1451] User ID: 12345

[1452] IoT data: Washing machine usage: 5 times / week

[1453] Emotion data: Stress (from facial and voice analysis)

[1454] In this way, the system provides users with the most suitable products based on usage data from IoT home appliances and user emotional data, thereby realizing a more personalized customer experience.

[1455] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1456] Step 1:

[1457] Data collection

[1458] The terminal collects usage data from IoT appliances. For example, the number of times a washing machine is used and the operating mode are acquired. The input is the IoT data acquired from the appliance, and the output is a temporary file containing the data or data stored in memory.

[1459] Step 2:

[1460] Acquiring emotion data

[1461] The device recognizes the user's face and obtains emotional data using a facial expression analysis engine (OpenCV). It also extracts emotional data from voice using a voice analysis engine (Google Cloud Speech-to-Text). The input is the user's real-time video and audio, and the output is the emotional data obtained by analyzing them.

[1462] Step 3:

[1463] Data transmission

[1464] The IoT appliance data and emotion data collected by the device are sent to the server. The input is the data obtained in Step 1 and Step 2, and the output is the data sent to the server.

[1465] Step 4:

[1466] Receiving and storing data

[1467] The server receives the transmitted data and stores it in a database. The input is the data transmitted from the terminal, and the output is the data stored in the database.

[1468] Step 5:

[1469] Data analysis

[1470] The server runs a generative AI model (GPT-3) to analyze the stored data. The input is the user's IoT appliance data and emotion data retrieved from the database, and the output is the analysis results to identify the best product for the user.

[1471] Step 6:

[1472] Recommendation Generation

[1473] The server uses the generative AI model to recommend the optimal product based on the analysis results. The input is the analysis results obtained in step 5, and the output is the recommendation result.

[1474] Step 7:

[1475] Recommendation presentation

[1476] The server sends the recommendation results to the device, which then displays them to the user. The user can check the recommended products through a smartphone application and consider purchasing them. The input is the recommendation results from the server, and the output is a list of products displayed on the device.

[1477] This will enable users to easily find the best products based on their IoT home appliance usage and emotional state.

[1478] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1479] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1480] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1481] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1482] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1483] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1484] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1485] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1486] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1487] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1488] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1489] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1490] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1491] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1492] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1493] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1494] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1495] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1496] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1497] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1498] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1499] The following is further disclosed regarding the above embodiment.

[1500] (Claim 1)

[1501] A means for receiving data collected from IoT home appliances;

[1502] A means for analyzing the data using generative AI to recommend optimal products to the user;

[1503] A means for presenting recommended products to users;

[1504] A system including:

[1505] (Claim 2)

[1506] The system of claim 1 , wherein the data of the IoT home appliance includes the operating status, usage mode, and usage frequency of the home appliance.

[1507] (Claim 3)

[1508] The system of claim 1 , further comprising: means for analyzing the stored data in response to the user's recommendation request.

[1509] "Example 1"

[1510] (Claim 1)

[1511] A means to collect operation and usage data from the IoT devices used by users and transmit it via the terminal,

[1512] means for receiving data transmitted from the terminal and storing the data in a database together with user identification information;

[1513] a means for receiving recommendation requests when a user is attempting to purchase a product;

[1514] A means of using generative AI models to analyze stored data and identify the best products for users;

[1515] A means for presenting the generated recommendation results to a user;

[1516] A system including:

[1517] (Claim 2)

[1518] 2. The system of claim 1, wherein the IoT device data includes device operating status, usage mode, and usage frequency.

[1519] (Claim 3)

[1520] The system of claim 1, further comprising means for inputting a prompt sentence to the generative AI model and identifying the most suitable product for the user based on the analysis result.

[1521] "Application Example 1"

[1522] (Claim 1)

[1523] A means for receiving data collected from IoT home appliances;

[1524] A means for analyzing the data using generative AI to recommend optimal products to the user;

[1525] A means for presenting recommended products to users;

[1526] A means for automatically recommending products based on data collected from the IoT home appliances and providing the recommendations as push notifications to a smartphone;

[1527] A system including:

[1528] (Claim 2)

[1529] The system of claim 1 , wherein the data of the IoT home appliance includes the operating status, usage mode, and usage frequency of the home appliance.

[1530] (Claim 3)

[1531] The system of claim 1 , further comprising: means for analyzing the stored data in response to the user's recommendation request.

[1532] "Example 2: Combining Emotion Engines"

[1533] (Claim 1)

[1534] A means to collect and transmit usage data when a user uses IoT home appliances, and

[1535] A means for the device to acquire and transmit user emotion data;

[1536] a server receiving and storing the usage data and emotion data;

[1537] A means for analyzing the stored data using a generative AI model to identify the best products for a user based on the user's recommendation request;

[1538] A means for presenting recommended products to users;

[1539] A system including:

[1540] (Claim 2)

[1541] The system of claim 1 , wherein the data of the IoT home appliance includes the operating status, usage mode, and usage frequency of the home appliance.

[1542] (Claim 3)

[1543] The system of claim 1 , wherein the terminal includes means for acquiring user emotion data using facial expression analysis, voice analysis, and text analysis.

[1544] "Application example 2 when combining emotion engines"

[1545] (Claim 1)

[1546] A means for receiving data collected from IoT home appliances;

[1547] A means for analyzing the data using generative AI to recommend optimal products to the user;

[1548] a means for acquiring and analyzing user emotion data;

[1549] A means for integrating and analyzing the data of the IoT home appliances and the emotional data to identify the most suitable product;

[1550] A method for presenting recommended products to users through a smartphone application;

[1551] A system including:

[1552] (Claim 2)

[1553] The system of claim 1 , wherein the data of the IoT home appliance includes the operating status, usage mode, and usage frequency of the home appliance.

[1554] (Claim 3)

[1555] 10. The system of claim 1, further comprising: means for analyzing the stored data and generating a recommendation result using a generative AI model in response to the user's recommendation request. [Explanation of symbols]

[1556] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for receiving data collected from IoT home appliances; A means for analyzing the data using generative AI to recommend optimal products to the user; A means for presenting recommended products to users; A system including:

2. The system of claim 1 , wherein the data of the IoT home appliance includes the operating status, usage mode, and usage frequency of the home appliance.

3. The system of claim 1 , further comprising means for analyzing the stored data in response to the user's recommendation request.

Images