System

Abstract

A system is provided.SOLUTION: A system comprising: means for collecting data of an article consumed by a home appliance; means for analyzing consumption data of the article and learning a consumption pattern; means for estimating an inventory status based on the consumption pattern; means for automatically ordering the article when the estimated inventory status falls below a set threshold; and means for notifying a user of an order content and an estimated arrival date of the article.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Managing daily necessities is a time-consuming and labor-intensive task in many households. In particular, the process of regularly checking consumable inventory, taking notes, and then going out to buy them can be a burden in busy lives. Furthermore, if items are not replenished at the appropriate time, it can disrupt daily life. This invention aims to solve these problems and free users from the hassle of inventory management and shopping. [Means for solving the problem]

[0005] The system includes: means for collecting data on items consumed by household electrical appliances; means for analyzing the item consumption data and learning consumption patterns; means for estimating inventory status based on the consumption patterns; means for automatically ordering items when the estimated inventory status falls below a set threshold; and means for notifying a user of the order details and expected arrival date of the items. The system also includes means for a user to verbally report inventory status and special requests, and means for analyzing the user's report and adjusting inventory estimation and ordering frequency. The system further includes means for ordering items using an online shop API and means for confirming the order details and expected arrival date based on a response from the API.

[0006] "Consumer appliances" are devices that run on electricity and power consumer goods and everyday items used in the home.

[0007] "Items" are consumables and necessary objects used in daily life.

[0008] The "means for collecting data" is an interface for recording usage information of the goods consumed by the household appliance and transmitting it to the server.

[0009] "Consumption data" is information that indicates specific consumption conditions such as the amount and frequency of use of an item.

[0010] The "means for analyzing and learning consumption patterns" refers to an algorithm that uses artificial intelligence technology to predict users' consumption behavior based on collected consumption data and learns their trends.

[0011] A "means for estimating inventory status" is a process for calculating current inventory levels based on consumption patterns and predicting whether there will be shortages for future consumption.

[0012] "Means for automatically ordering goods" refers to a system for automatically purchasing necessary goods via an online shop, etc., when inventory falls below a set threshold.

[0013] The "means for notifying the user of the order contents and expected arrival date" is an interface for providing the user with information on the order contents and expected arrival date of the item after the order is completed.

[0014] "Means for verbally reporting stock status or special requests" refers to a device or function that allows a user to input stock status or additional requests via voice.

[0015] "Means for analyzing user reports and adjusting inventory estimates and ordering frequency" refers to the process of analyzing user reports using artificial intelligence and optimizing inventory management and ordering strategies based on the results.

[0016] "Means for ordering goods using online shop APIs" refers to a system for automatically ordering goods using an application programming interface provided by an online shop.

[0017] "Means for confirming the order details and expected arrival date based on the response from the API" refers to the process of analyzing the response data from the online shop API and confirming the details of the ordered items and their expected arrival date. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram illustrating a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0019] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0022] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0023] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0024] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0026] [First embodiment]

[0027] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0028] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0031] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0034] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0038] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0039] The present invention relates to a system for managing inventory and automatically ordering daily necessities in the home. The following describes an embodiment of the invention. This system is composed of various elements, including home appliances, a server, terminals (smart speakers and apps), and users.

[0040] System Overview

[0041] In this system, data on the items consumed by household appliances is collected and sent to a server. The server analyzes this data and learns consumption patterns. Based on the learned consumption patterns, the server estimates the inventory status of each item and automatically orders items when inventory falls below a set threshold. The order details and expected arrival date are notified to the user via their terminal, and the user can also verbally report inventory status or special requests. The user's reports are also analyzed by the server and reflected in inventory forecasts and adjustments to ordering frequency.

[0042] Program processing overview

[0043] 1. Data Collection:

[0044] The server receives consumption data sent from the home appliances, including the type of item used, the amount used, and the date and time of use.

[0045] Example: Data is sent to the server that a detergent dispenser has used 10ml of detergent.

[0046] 2. Data analysis and learning:

[0047] The server analyzes the received consumption data and learns the user's consumption patterns, which allows it to predict future consumption based on past consumption data.

[0048] Example: Learning a pattern that the average amount of detergent used per day over a week is 45ml.

[0049] 3. Inventory Estimation:

[0050] The server estimates current inventory levels and predicts when inventory will fall below a set threshold based on consumption patterns.

[0051] Example: It is estimated that a 500ml stock will fall below the planned consumption level in 20 days.

[0052] 4. Automated ordering:

[0053] When the amount falls below the set threshold, the server automatically orders items from the online shop, using the online shop's API to place the order.

[0054] Example: When you run out of detergent below 50ml, order a 500ml bottle of detergent from an online shop.

[0055] 5. User Notice:

[0056] The terminal will notify the user of the order details and estimated arrival date, and the user can receive notifications via their smartphone or smart speaker.

[0057] For example, the device will announce, "We're low on detergent, so we've placed an order for more. It's expected to arrive in two days."

[0058] 6. User Reports:

[0059] Users can verbally report stock availability and special requests, and this information is sent to the server via the smart speaker and used to analyze consumption patterns and estimate inventory.

[0060] Example: A user reports, "I have a party and need some extra detergent."

[0061] Specific examples

[0062] As an example, consider a household where users do laundry every day and consume 30ml of laundry detergent per day. In this case, the washing machine (a household appliance) sends the amount of detergent used per load to a server every day. The server analyzes past data and learns that the user consumes approximately 210ml of detergent per week. If the user has 500ml in stock, the server predicts that the stock will run out in approximately 11.67 days and automatically orders more detergent before it falls below a threshold (e.g., 100ml). The user is notified of the order and the expected arrival date, and can report any special requests via a smart speaker. Based on this information, the server adjusts inventory management and ordering frequency.

[0063] In this way, users can avoid the hassle of inventory management and shopping and receive the items they need at the right time.

[0064] The processing flow will be explained below.

[0065] Step 1: Data collection

[0066] The server receives consumption data sent from the home appliances, including the type of item used, the amount used, and the date and time of use.

[0067] Example: Data that a detergent dispenser used 10ml of detergent is sent to the server, and the server records it as "Detergent 10ml Used Date and Time: 2023-10-05 18:45".

[0068] Step 2: Save data

[0069] The server stores the received consumption data in a database in chronological order.

[0070] Example: Store information in a database in the format "Product: Detergent, Amount Used: 10ml, Date and Time: 2023-10-05 18:45".

[0071] Step 3: Data analysis

[0072] The server executes data analysis tasks at regular intervals, learning and analyzing users' consumption patterns based on past consumption data.

[0073] Example: Data for one month is collected and the average consumption is calculated. The analysis results show that an average of 45ml of detergent is consumed per day.

[0074] Step 4: Inventory estimation

[0075] The server estimates the current inventory level based on the analysis results and predicts when the inventory will fall below a set threshold based on consumption patterns.

[0076] Example: If the initial inventory is 500ml and 45ml is consumed per day, it is predicted that the inventory will run out in 500ml ÷ 45ml / day ≈ 11.11 days.

[0077] Step 5: Automatic ordering decision

[0078] The server checks whether the estimated inventory level falls below a set threshold, and if so, automatically places an order.

[0079] Example: If the threshold is set to 100ml, an auto-order flag will be set when the estimated stock level falls below 100ml.

[0080] Step 6: Automatic Order Execution

[0081] The server calls the online shop's API to order the necessary items.

[0082] Example: The server sends a request to the "Online Shop API" to "Order one 500ml bottle of detergent."

[0083] Step 7: Confirm order and prepare notification

[0084] The server receives the response from the online shop API and confirms the order details and expected delivery date.

[0085] Example: Receive a response from the API saying "500ml detergent, order completed, expected arrival date: 2023-10-07" and save the data.

[0086] Step 8: User Notification

[0087] The device will notify the user of the order details and estimated arrival date, and provide information via a smart speaker or app.

[0088] Example: A smart speaker notifies the user by voice, "We're running low on detergent, so we've placed an order for more. The expected delivery date is in two days."

[0089] Step 9: User Reporting

[0090] Users report stock availability and special requests through their smart speakers.

[0091] Example: A user reports, "I have a party and need some extra detergent."

[0092] Step 10: Report analysis and inventory adjustments

[0093] The server analyzes user reports and adjusts consumption forecasts, inventory estimates, and order frequency.

[0094] Example: The server receives a report of a user's "party" and predicts that consumption will increase next week, so it orders more to cover that increase.

[0095] This series of steps creates a system that allows users to conveniently manage inventory and automatically order daily necessities.

[0096] Example 1

[0097] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0098] Managing the inventory of everyday items in the home is a complicated and tedious task for many people. It is difficult to predict when items will run out and to purchase them appropriately, and there is a high possibility of inconvenience when they run out. Furthermore, flexible and accurate inventory management is required to respond to users' individual consumption patterns and special requests. However, existing systems are unable to adequately resolve these issues.

[0099] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0100] In this invention, the server includes: means for collecting data on items consumed by household electrical appliances; means for analyzing consumption data of the items and learning consumption patterns; means for estimating inventory status based on the consumption patterns; means for automatically ordering items when the estimated inventory status falls below a set threshold; means for notifying the user of the order details and expected arrival date of the items; and means for the user to verbally report the inventory status and special requests, and for analyzing the report to adjust the order frequency and inventory management. This allows the user to automatically manage the inventory status of daily necessities and automatically order necessary items at the appropriate time, significantly reducing the effort required for inventory management.

[0101] "Household appliances" refers to all electrically powered devices used in the home, including those equipped with sensors that collect consumption data.

[0102] "Goods" refers to consumables and daily necessities consumed by household electrical appliances, specifically including detergents, paper products, food ingredients, etc.

[0103] "Data collection means" refers to the functions and methods for recording consumption data generated when a household electrical appliance is used and transmitting it to a server.

[0104] "Data analysis means" refers to software or algorithms for processing received consumption data and analyzing user consumption patterns.

[0105] "Consumption pattern" is information that indicates the tendency of how a user consumes items within a specific period of time based on past consumption data.

[0106] "Inventory estimation means" refers to a function that uses consumption data and estimation algorithms to calculate current inventory levels and predict when inventory will run out in the future.

[0107] A "threshold" is a reference value set during inventory management, which becomes a trigger point for automatically ordering goods when the value falls below this value.

[0108] "Automatic ordering method" refers to a function or method for automatically ordering necessary items through the online shop's API when inventory falls below a threshold.

[0109] The "notification means" is a means for informing users of information such as order details and the expected arrival date of the item, and is implemented using a smartphone or smart speaker.

[0110] "User reporting means" refers to a device such as a smart speaker that allows a user to verbally report stock status or special requests, and the server receives the report.

[0111] An "online shop API" is a program interface provided by an online shopping platform that is used to order and confirm goods.

[0112] A "machine learning algorithm" refers to a mathematical model or computational procedure that learns patterns and knowledge from data and makes predictions and inferences.

[0113] The present invention is a system for efficiently managing inventory and automatically ordering consumer goods in the home. Specific embodiments of the system will be described in detail below.

[0114] This system consists of various elements, including household appliances, servers, terminals (smart speakers and apps), and users.

[0115] System configuration

[0116] 1. Household appliances:

[0117] Each consumer electronic device is fitted with sensors that collect data whenever the consumer item is used, and this data is transmitted to a server via Wi-Fi or other communication means.

[0118] Examples include washing machines and detergent dispensers, which send data such as the type and amount of items used and the date and time of use to a server.

[0119] 2. Server:

[0120] The server analyzes the received data and uses machine learning algorithms, including ridge regression and other statistical models, to learn the user's consumption patterns.

[0121] The server uses programming languages ​​such as Python to preprocess and analyze the data, specifically performing tasks such as filling in missing values, detecting outliers, and normalizing the data.

[0122] 3. Devices (smart speakers and apps):

[0123] The device has the function of notifying the user of the order details and expected arrival date, and conveys the information to the user via push notifications on their smartphone or voice notifications on their smart speaker.

[0124] These devices allow users to report stock availability and special requests by voice, which is converted to text by the device and sent to the server.

[0125] Specific processing examples

[0126] 1. Data Collection:

[0127] Example: If the washing machine uses 30ml of detergent in one wash, it sends that information to the server.

[0128] 2. Data analysis and learning:

[0129] Example: Based on data from the past week, the server learns that an average of 45ml of detergent is consumed per day.

[0130] 3. Inventory Estimation:

[0131] Example: There is currently 500ml in stock, and the server estimates that in 20 days the stock will fall below the threshold.

[0132] 4. Automated ordering:

[0133] Example: When stock falls below a threshold, the server orders 500ml of detergent through an online shop API.

[0134] 5. User Notice:

[0135] Example: "We're low on detergent so we've placed an order. It's expected to arrive in two days," your smart speaker will announce.

[0136] 6. User Reports:

[0137] Example: A user reports, "I'm having a party and need some extra detergent," and the smart speaker sends that information to the server.

[0138] This system allows users to eliminate the hassle of daily inventory management and automatically receive the items they need at the right time. Specific examples of prompts include "Tell me what's in stock today," "Tell me when the next shipment of detergent is expected to arrive," and "Order more detergent for a party."

[0139] The system provided by this invention can thoroughly automate the management of household items and improve user convenience.

[0140] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0141] Step 1: Data collection

[0142] 1. Input:

[0143] Consumption data generated by household appliances (e.g., type of item used, amount used, date and time of use).

[0144] 2. Processing:

[0145] Household appliances use sensors to measure and collect consumption data, for example, if a washing machine uses 30ml of detergent, it will measure that data.

[0146] 3. Data Processing:

[0147] The household appliances format the measured data and send it to a server via Wi-Fi.

[0148] 4. Output:

[0149] Formatted consumption data sent to the server.

[0150] 5. Specific operation:

[0151] When the washing machine starts washing and 30ml of detergent is used, the sensor detects the amount used, compiles the data and sends it to the server.

[0152] Step 2: Data analysis and training

[0153] 1. Input:

[0154] Consumption data of household appliances sent to a server.

[0155] 2. Processing:

[0156] The server preprocesses the received data, performing missing value imputation, outlier detection, and data normalization.

[0157] 3. Data Processing:

[0158] The server inputs the organized data into a machine learning algorithm (e.g., ridge regression) to learn the user's consumption patterns.

[0159] 4. Output:

[0160] A model that learns consumption patterns.

[0161] 5. Specific operation:

[0162] The server analyzes consumption data from the past week and learns that "users consume an average of 45ml of detergent per day."

[0163] Step 3: Inventory estimation

[0164] 1. Input:

[0165] Learned consumption pattern model and current inventory levels.

[0166] 2. Processing:

[0167] The server accumulates the initial inventory and consumption data to estimate the current inventory amount.

[0168] 3. Data Calculation:

[0169] Based on consumption patterns, the server predicts when inventory will fall below a set threshold.

[0170] 4. Output:

[0171] The point at which it is estimated to fall below the threshold.

[0172] 5. Specific operation:

[0173] The server calculates that the current detergent inventory of 500ml will fall below consumption in 20 days.

[0174] Step 4: Automated ordering

[0175] 1. Input:

[0176] Prediction of when inventory will fall below a threshold.

[0177] 2. Processing:

[0178] When the amount falls below the set threshold, the server calls the online shop API to order the item.

[0179] 3. Data Calculation:

[0180] The server sends the order details to the online shop using an HTTP POST request.

[0181] 4. Output:

[0182] Order requests sent to the online shop.

[0183] 5. Specific operation:

[0184] The server uses the API to send a request to the online shop to "order one 500ml bottle of detergent."

[0185] Step 5: User Notification

[0186] 1. Input:

[0187] Notification data that the order has been completed.

[0188] 2. Processing:

[0189] The terminal receives the notification data from the server and notifies the user by display or voice.

[0190] 3. Data Calculation:

[0191] The terminal converts the notification content into an appropriate format and transmits it to the user.

[0192] 4. Output:

[0193] The order details and expected delivery date notified to the user.

[0194] 5. Specific operation:

[0195] The smart speaker will announce, "We're running low on detergent, so we've placed an order for more. The expected delivery date is two days from now."

[0196] Step 6: User Reporting

[0197] 1. Input:

[0198] User voice reporting.

[0199] 2. Processing:

[0200] The device records the audio data and converts it into text.

[0201] 3. Data Processing:

[0202] The terminal transmits the converted text data to the server.

[0203] 4. Output:

[0204] The user's reported text data sent to the server.

[0205] 5. Specific operation:

[0206] The user reports, "I'm having a party so I need some extra detergent," and the smart speaker sends that information to the server.

[0207] The above is the specific processing flow of this system. The specific input, processing, and output are clearly defined at each step, and it is designed to enable users to quickly and efficiently manage their home inventory.

[0208] (Application example 1)

[0209] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0210] In modern households, inventory management of daily consumed goods and food is time-consuming and difficult to replenish at the appropriate time. In particular, busy households and households with elderly people are prone to stock shortages, which increases the effort required for shopping to address these shortages. Furthermore, there is a need for an inventory management method that can flexibly respond to unexpected events and changes in plans. To solve these problems, a system is needed that automatically manages inventory and replenishes necessary goods and food at the appropriate time.

[0211] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0212] In this invention, the server includes: means for collecting food consumption data using sensors in household appliances; means for analyzing the collected food consumption data and learning a user's consumption patterns; means for estimating inventory status based on the consumption patterns; means for automatically ordering food from a food delivery service when the estimated inventory status falls below a set threshold; means for notifying the user of the food order details and expected arrival date via the food delivery service's API; means for the user to verbally report inventory status and special requests; means for analyzing the user's report and adjusting inventory estimation and order frequency; means for collecting food data using sensors installed in household refrigerators and smart microwaves; means for learning consumption patterns based on the collected food data and estimating food inventory status; means for ordering items using an online shop API; means for confirming the order details and expected arrival date based on a response from the API; means for placing an order using the food delivery service API; and means for notifying the user of the order details and expected arrival date. This reduces the user's effort and enables automatic replenishment of necessary items and foods at the appropriate time.

[0213] "Household electrical appliances" is a general term for products that operate using electricity and are used in the home.

[0214] "Consumption data" refers to information such as the type of items used by a household electrical appliance, the amount used, and the date and time of use.

[0215] "Consumption patterns" are data that indicate trends in the frequency and amount of goods consumed based on past consumption data.

[0216] "Inventory status" refers to the current stock of items in the home.

[0217] The "threshold" is a reference value below which inventory is automatically deemed to need replenishment.

[0218] "Food delivery service" refers to a service that delivers ingredients and food to a designated location.

[0219] An "API" is an interface for exchanging data with other programs or services.

[0220] "User notification" means informing the user of information such as the order contents and expected arrival date.

[0221] "Verbal reporting" refers to the act of a user reporting stock availability or special requests using voice.

[0222] "Analysis" refers to processing collected data to extract useful information.

[0223] "Ordering" refers to the act of ordering necessary goods or food.

[0224] "Estimated Delivery Date" means the date on which the ordered items are expected to arrive at the specified location.

[0225] The present invention provides a system for managing inventory and automatically ordering daily necessities in the home. This system is composed of various elements, including home appliances, a server, terminals (e.g., smart speakers and smartphone apps), and users.

[0226] The program for realizing this system consists of the following steps:

[0227] Data collection

[0228] The server collects food consumption data using sensors built into household appliances (e.g., refrigerators and smart microwaves). The collected data includes the type of food consumed, the amount consumed, and the date and time of consumption. This data is transmitted to the server via Wi-Fi.

[0229] Data Analysis and Learning

[0230] The server analyzes the received food consumption data and learns the user's consumption patterns. This analysis is performed using data analysis libraries such as pandas and numpy. Based on past data, the server predicts how often and how much food the user will consume.

[0231] Inventory Estimation

[0232] The server estimates the household food inventory based on consumption patterns. When the estimated inventory falls below a set threshold, it automatically orders food from a food delivery service. This order uses the food delivery service's API to confirm the order details and expected arrival date.

[0233] Automatic ordering

[0234] The server automatically orders food from a food delivery service when the ingredients fall below a threshold. This ordering process uses an API interface, allowing the order details to be confirmed. The required food is ordered via the food delivery service's API.

[0235] User Notifications

[0236] The terminal notifies the user of the order details and expected arrival date. The notification is made via voice using a smart speaker API. Users can receive the information via their smartphone or smart speaker.

[0237] User Reports

[0238] Users can verbally report stock availability and special requests, and this information is sent to the server via the smart speaker, which then uses it to manage inventory and adjust ordering frequency.

[0239] Specific examples

[0240] For example, suppose a home refrigerator contains a variety of vegetables, and a sensor measures daily consumption. The server analyzes this data and learns that an average person consumes 300g of vegetables per week. If inventory falls below a 100g threshold, it automatically orders 500g of vegetables from a food delivery service. At this time, the user is notified that "We're running low on vegetables, so we've ordered an additional 500g. The estimated delivery date is tomorrow." If the user also tells the smart speaker, "A friend is coming over this weekend, so please order some more fruit," this information will be reflected in the next order.

[0241] Example prompts to input to the generative AI model

[0242] Calculate the average vegetable consumption for a week from the vegetable inventory management data in the refrigerator, and determine whether the inventory is below the set threshold (100g). If it falls below 100g, automatically order 500g of vegetables via a food delivery service and notify the user.

[0243] This system allows users to significantly reduce the hassle of daily inventory management and shopping, and allows them to receive the items and food they need at the right time.

[0244] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0245] Step 1:

[0246] The server collects food consumption data through sensors built into household appliances (e.g., refrigerators and smart microwaves). The data measured by the sensors includes the type of food consumed, the amount consumed, and the date and time of consumption. This data is transmitted to the server via Wi-Fi. The input is raw data from the sensors, and the output is data stored on the server for analysis.

[0247] Step 2:

[0248] The server analyzes the collected food consumption data and learns the user's consumption patterns. It uses data analysis libraries such as pandas and numpy to predict how often and how much food the user will consume based on past data. The input is the consumption data collected in step 1, and the output is the consumption pattern as a result of the analysis.

[0249] Step 3:

[0250] The server estimates the household food inventory status based on the learned consumption patterns. Based on the current inventory and consumption patterns, it predicts when the inventory will fall below a threshold. The inputs are the consumption patterns and the current inventory, and the output is the estimated inventory status.

[0251] Step 4:

[0252] The server automatically orders food from a food delivery service when the estimated inventory falls below a set threshold. The required food is ordered through the food delivery service's API. The input is the condition that inventory has fallen below the threshold and the order details, and the output is an order confirmation sent to the food delivery service.

[0253] Step 5:

[0254] The terminal notifies the user of the order details and expected arrival date. A smart speaker API is used to notify the user by voice or via a smartphone app. The input is the order details and expected arrival date, and the output is the notification information sent to the user.

[0255] Step 6:

[0256] Users verbally report stock status and special requests. This information is sent to the server via a smart speaker and used to estimate the next inventory and adjust ordering frequency. The input is the user's voice report, and the output is the report data stored on the server.

[0257] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0258] This paper describes an inventory management and automatic ordering system for daily necessities using household electrical appliances, and in particular a system that realizes optimal inventory management according to the user's emotional state by combining an emotion engine. This system is composed of electrical appliances installed in the home, a server, a terminal (smart speaker or app), an emotion engine, and the user.

[0259] System Overview

[0260] In this system, data on the goods consumed by household electrical appliances is collected on a daily basis and sent to a server. The server analyzes the received data and learns consumption patterns. Based on the learned consumption patterns, inventory status is estimated, and when inventory falls below a set threshold, goods are automatically ordered from an online shop. The order details and expected arrival date are notified to the user via their terminal. In addition, users can verbally report inventory status or special requests, which are also processed on the server side. In addition, an emotion engine is used to recognize user emotions and optimize inventory management and ordering processes based on emotional information.

[0261] Program processing overview

[0262] 1. Data Collection and Storage:

[0263] The server receives consumption data sent from the home appliances and stores it in a database in chronological order.

[0264] Example: A detergent dispenser receives data that 10ml of detergent has been used and records it as "Detergent 10ml Used Date: 2023-10-05 18:45".

[0265] 2. Data analysis and pattern learning:

[0266] The server analyzes the received consumption data and learns the user's daily consumption patterns, and then predicts future consumption based on the results of this learning.

[0267] Example: After analyzing data for one month, we learn that the average daily consumption of detergent is 45ml.

[0268] 3. Inventory Estimation:

[0269] Based on the learned consumption patterns, the server estimates the current inventory level and predicts when the inventory level will fall below a set threshold.

[0270] Example: If the inventory is 500ml and 45ml is consumed per day, it is estimated that the inventory will run out in approximately 11.11 days.

[0271] 4. Automated ordering:

[0272] If the server determines that the inventory level is below a set threshold, it calls the online shop API to automatically order the necessary items.

[0273] Example: When the warehouse stock drops below 50ml, a request is sent to the online shop API to order one 500ml bottle of detergent.

[0274] 5. Notification Processing:

[0275] The terminal receives notification of the order contents and expected arrival date from the server and notifies the user of the information.

[0276] For example, "We're low on detergent so we've placed an order for more. It's expected to arrive in two days."

[0277] 6. User Reporting Process:

[0278] Users verbally report stock availability and special requests through their smart speakers, and the information is sent to the server.

[0279] Example: A user reports, "I have a party and need some extra detergent."

[0280] 7. Emotion recognition:

[0281] The emotion engine recognizes emotions from the user's voice and facial expressions and transmits the emotion information to the server.

[0282] Example: Determine if the user is under stress based on their tone of voice and facial expression analysis.

[0283] 8. Emotion-based processing:

[0284] The server adjusts inventory management and ordering strategies based on the emotional information from the emotion engine.

[0285] Example: If a user is stressed, order needed items sooner or in larger quantities.

[0286] Specific examples

[0287] Here is a specific example of managing a user's daily detergent consumption. Assume that a user consumes 45 ml of household detergent per day, with an initial inventory of 500 ml. In this case, the washing machine (a household appliance) sends the amount of detergent consumed per use to the server. The server analyzes past data and learns that the user consumes approximately 315 ml of detergent per week. If the user's inventory is 500 ml, the server predicts that inventory will fall below 100 ml in approximately 11.11 days and places an order for additional detergent from an online shop at the appropriate time. The user is notified of the order details and expected arrival date. Furthermore, if the user verbally reports, "I need more detergent because I have a party next week," the server adjusts the order amount based on that information. If the emotion engine recognizes the user's stress level, the server can respond even more quickly and place an order immediately to reassure the user.

[0288] The processing flow will be explained below.

[0289] Step 1: Data collection

[0290] The server receives consumption data of items from the home appliances, including the type of item, the amount used, and the date and time of use.

[0291] Example: A washing machine sends data that 30ml of detergent has been used to the server, and the server records it as "30ml of detergent used: 2023-10-05 07:30".

[0292] Step 2: Save data

[0293] The server stores the received consumption data in a database in chronological order.

[0294] Example: Store information in a database in the format "Product: Detergent, Amount Used: 30ml, Date and Time: 2023-10-05 07:30".

[0295] Step 3: Data analysis and pattern learning

[0296] The server periodically executes a data analysis task to analyze consumption data and learn user consumption patterns.

[0297] Example: Based on one month's data, we calculate that the average daily detergent consumption is 45ml.

[0298] Step 4: Inventory estimation

[0299] The server estimates the current inventory level based on the learning results and predicts when the inventory will fall below a set threshold.

[0300] Example: Initial inventory is 500ml, consumption is 45ml per day, and it is predicted that inventory will be 50ml after approximately 11 days.

[0301] Step 5: Automatic ordering decision

[0302] The server checks whether the inventory level has fallen below the set threshold. If it has, it automatically places an order.

[0303] Example: When the stock reaches 50ml, the server determines that an automatic order is required.

[0304] Step 6: Automatic Order Execution

[0305] The server calls the online shop's API to order the necessary items.

[0306] Example: The server sends a request to the "Online Shop API" to "Order one 500ml bottle of detergent."

[0307] Step 7: Confirm order and prepare notification

[0308] The server receives the response from the online shop API and confirms the order details and expected delivery date.

[0309] Example: Receive a response from the API saying "500ml of detergent, order completed, expected arrival date: 2023-10-07" and save it in the database.

[0310] Step 8: User Notification

[0311] The terminal provides the user with notification of the order details and expected arrival date from the server.

[0312] Example: A smart speaker announces, "We're running low on detergent, so we've placed an order for more. The estimated delivery date is two days later."

[0313] Step 9: User Reporting

[0314] Users verbally report stock availability and special requests to the smart speaker.

[0315] Example: A user reports, "I have a party and need more detergent."

[0316] Step 10: Report analysis and inventory adjustments

[0317] The server analyzes the user's reports and adjusts inventory forecasts and order frequency.

[0318] Example: The server changes the settings to "Order more detergent than usual due to increased detergent consumption."

[0319] Step 11: Emotion Recognition

[0320] The emotion engine recognizes emotions from the user's tone of voice and facial expressions and sends that information to the server.

[0321] Example: When a user feels stressed, the engine sends the analysis results to the server.

[0322] Step 12: Emotion-Based Processing

[0323] The server adjusts inventory management and ordering strategies based on the emotional data from the emotion engine.

[0324] Example: If users are stressed, order more detergent early to avoid running out.

[0325] This series of steps will create a system that allows users to manage inventory of daily necessities, predict consumption, and receive emotion-based assistance.

[0326] Example 2

[0327] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0328] While conventional inventory management systems for daily necessities using home appliances are capable of managing inventory levels and automatically ordering based on consumption patterns, they have the problem of being unable to flexibly respond to user emotions or special circumstances. This has resulted in insufficient improvements in user convenience and satisfaction, and has led to waste due to inventory shortages and over-ordering under certain circumstances.

[0329] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0330] In this invention, the server includes means for collecting data on items consumed by the household electrical appliances, means for analyzing the consumption data of the items and learning consumption patterns, means for estimating inventory status based on the consumption patterns, means for automatically ordering items when the estimated inventory status falls below a set threshold, means for notifying a user of the order contents and expected arrival date of the items, means for recognizing the user's emotions, and means for adjusting inventory management and ordering strategies based on the emotion information, thereby enabling effective inventory management and automatic ordering according to the user's emotions and special requests.

[0331] "Household electrical appliances" refers to home appliances used in the home, including washing machines, refrigerators, microwave ovens, etc.

[0332] "Goods" refers to consumable items such as detergents and food consumed within the home.

[0333] "Means for collecting data" refers to a device or system capable of collecting consumption data generated by a household appliance.

[0334] "Means for analyzing data and learning consumption patterns" refers to a device or system that has the function of analyzing collected consumption data and learning users' consumption patterns using machine learning algorithms or the like.

[0335] "Means for estimating inventory status" refers to a device or system capable of estimating the current inventory of an item based on consumption patterns.

[0336] "Means for automatically ordering goods" means a device or system that has the capability to automatically order goods using an online system when inventory levels fall below a set threshold.

[0337] "Means of notifying the user" refers to telecommunications means for informing the user of the order details and expected arrival date of the item, including smart speakers and smartphones.

[0338] "Means for recognizing emotions" refers to a device or system that has the function of analyzing and recognizing emotions from the user's voice and facial expressions.

[0339] "Means for adjusting inventory management and ordering strategies based on emotional information" refers to a device or system that has the function of appropriately adjusting inventory management and ordering processes based on recognized user emotions.

[0340] "Online Shop API" refers to an application programming interface that connects with online shops via the Internet to order goods and check their status.

[0341] "Means for analyzing reports and adjusting inventory estimates and ordering frequencies" refers to a device or system that has the function of analyzing reports from users and appropriately adjusting inventory estimates and ordering frequencies based on that information.

[0342] This invention relates to an inventory management and automatic ordering system for daily necessities using household electrical appliances, and in particular to a system that realizes optimal inventory management according to the user's emotional state by combining an emotion engine. This system is composed of electrical appliances installed in the home, a server, a terminal (smart speaker or app), an emotion engine, and the user.

[0343] The server receives consumption data sent from household appliances and stores it in a database. For example, if a washing machine consumes 10ml of detergent in one use, the data is recorded as "Detergent 10ml Use Date: 2023-10-05 18:45." Data is collected in real time and sent to the server via the Internet. This data is organized chronologically and stored in a database.

[0344] The server then analyzes the collected data and uses machine learning algorithms (e.g., Python's pandas or scikit-learn) to learn the user's daily consumption patterns. For example, based on one month's consumption data, the server may learn that the average daily consumption of detergent is 45ml. This allows the server to make future consumption predictions.

[0345] Furthermore, the server estimates the current inventory level based on this learning result and predicts when the inventory will fall below a set threshold. For example, if the initial inventory is 500 ml and 45 ml is consumed per day, the server estimates that the inventory will fall below 50 ml in approximately 11.11 days.

[0346] When it is determined that the inventory level is below the threshold, the server calls an online store API (such as Amazon's API or Rakuten API) and automatically places an order for the necessary items. Specifically, when inventory falls below 50ml, the server sends a request to the online store API to order one 500ml bottle of detergent. This is done using an HTTP request that includes the API key and order information.

[0347] Information about the order contents and expected arrival date from the server is notified to the user via a device (smart speaker or app). For example, a notification message such as "We've placed an additional order for detergent because we're running low on it. The expected arrival date is two days later" is displayed. This is achieved by using the smart speaker's text-to-speech (TTS) function or the smartphone's push notification function.

[0348] Users can also use smart speakers to verbally report inventory status or special requests. For example, if a user says, "I'm going to a party and need more detergent," the voice data is converted into text data and sent to the server. The server analyzes this information and adjusts inventory estimates and ordering frequency.

[0349] Furthermore, the emotion engine analyzes emotional information from the user's voice and facial expressions and sends the results to the server. For example, if a user says "I feel very stressed" in a strong tone, the emotion engine will analyze the voice tone and facial expression and determine that the user is in a stressed state. This is done using libraries and services specialized in emotion recognition (e.g., AWS Rekognition, Google Cloud Vision).

[0350] The server adjusts inventory management and ordering strategies based on the emotion information from the emotion engine. For example, if the emotion engine determines that the user is feeling stressed, the server will order necessary items such as detergent more quickly or in larger quantities. Also, if it determines that special measures are needed, the server will adjust the order by raising the priority.

[0351] Example prompts to input to the generative AI model

[0352] Describe the specific process for an inventory management and automated ordering system for everyday items using consumer electronics, from data collection to processing based on emotion recognition. Include the following keywords: inventory management, consumption patterns, online shop, notification, emotion recognition.

[0353] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0354] Step 1: Data collection and storage

[0355] The server receives consumption data sent from home appliances in real time and stores it in a database. Specifically, the server receives data sent from washing machines, dispensers, etc. and stores it in a chronological order in the database.

[0356] Input: Consumption data from a household appliance (e.g., 10 ml of detergent used).

[0357] Data processing: Organize received data with timestamps.

[0358] Output: Organized consumption data is stored in a database.

[0359] Specific operation: The server receives data such as "Detergent 10ml, date and time of use: 2023-10-05 18:45" and records it in the database.

[0360] Step 2: Data analysis and pattern learning

[0361] The server analyzes the consumption data stored in the database and learns the user's daily consumption patterns using machine learning algorithms (e.g., Python's pandas and scikit-learn).

[0362] Input: Consumption data stored in a database.

[0363] Data processing: Machine learning algorithms are used to analyze daily consumption patterns.

[0364] Output: Learning results about users' consumption patterns.

[0365] Specific operation: The server analyzes, for example, one month's worth of data and learns that "users consume an average of 45 ml of detergent per day."

[0366] Step 3: Inventory estimation

[0367] The server estimates the current inventory level based on the learned consumption patterns and predicts when it will fall below the threshold.

[0368] Input: Learning results of consumption patterns and current inventory.

[0369] Data processing: Uses a mathematical formula to estimate based on inventory volume and consumption rate.

[0370] Output: Prediction of when inventory will fall below threshold.

[0371] Specific operation: If the inventory is 500 ml and daily consumption is 45 ml / day, the server estimates that the inventory will fall below 50 ml in approximately 11.11 days.

[0372] Step 4: Automated ordering

[0373] If the server determines that the inventory level is below the threshold, it calls the online shop API to automatically order the item.

[0374] Input: Inventory quantity estimation result.

[0375] Data processing: Generation of order requests using the online shop API.

[0376] Output: Order request and response to online shop.

[0377] Specific operation: When the stock falls below 50ml, the server sends a request to the online shop API to order one 500ml bottle of detergent.

[0378] Step 5: Notification Processing

[0379] The terminal receives notification of the order contents and expected arrival date from the server and conveys the information to the user.

[0380] Input: Order details and expected arrival date information from the server.

[0381] Data processing: generating text messages and voice notifications.

[0382] Output: Notification to the user.

[0383] Specific operation: The terminal notifies the user, "We are running low on detergent, so we have placed an additional order. The expected delivery date is two days later."

[0384] Step 6: User Report Processing

[0385] Users use smart speakers to verbally report stock availability and special requests, which are then sent to the server.

[0386] Input: User's voice report.

[0387] Data processing: Text conversion and analysis of voice data.

[0388] Output: Report to the server.

[0389] Specific operation: When a user verbally reports, "I'm having a party and need some extra detergent," the speech is converted into text and sent to the server.

[0390] Step 7: Emotion Recognition

[0391] The emotion engine analyzes emotional information from the user's voice and facial expressions and sends the results to the server.

[0392] Input: User's voice and facial expression data.

[0393] Data processing: Applying emotion recognition algorithms.

[0394] Output: User's emotional information.

[0395] Specific behavior: The emotion engine analyzes the user's tone of voice and facial expressions to recognize stress levels and other emotions.

[0396] Step 8: Emotion-Based Processing

[0397] The server adjusts inventory management and ordering strategies based on the emotional information from the emotion engine.

[0398] Input: Emotion information from the emotion engine.

[0399] Data processing: Adjusting inventory management and ordering strategies.

[0400] Output: Coordinated inventory management and ordering instructions.

[0401] Specific behavior: If the server recognizes that the user is stressed, it will order the necessary items more quickly or in larger quantities.

[0402] (Application example 2)

[0403] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0404] In modern store operations, inventory management and the quality of customer service are important issues. In particular, there is a need to prevent lost sales opportunities due to inventory shortages and to respond to customers' emotional states in a detailed manner. However, conventional systems have difficulty simultaneously solving these problems, making it difficult to achieve efficient inventory management and customer service. Therefore, there is a need for a system that combines an emotion engine to provide optimal inventory management and customer service that responds to the user's emotional state.

[0405] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting data on items consumed by household electrical appliances, means for analyzing the consumption data of the items and learning consumption patterns, means for estimating inventory status based on the consumption patterns, means for automatically ordering items when the estimated inventory status falls below a set threshold, means for notifying the user of the order details and expected arrival date of the items, and means for optimizing inventory management and ordering processes based on emotions using an emotion engine that recognizes the user's emotional state. This makes it possible to improve the efficiency of inventory management and user satisfaction.

[0406] Below are definitions of important words:

[0407] "Household appliances" are electrical devices used in ordinary households that can be used as a means of collecting consumption data.

[0408] "Goods" refers to the items that are consumed or controlled by a consumer electronics device.

[0409] "Data Collection Measure" means a device or software for periodically collecting data on goods consumed from a consumer electronics device.

[0410] "Data analysis means" refers to a device or software that analyzes and learns consumption patterns based on collected consumption data.

[0411] The "consumption pattern" indicates the user's tendency to consume items on a daily basis, and is learned by the data analysis means.

[0412] An "inventory estimator" is a device or software for estimating current inventory levels based on consumption patterns and assessing inventory status.

[0413] An "automatic ordering means" is a device or software for automatically ordering items through an online shop when estimated inventory falls below a set threshold.

[0414] The "notification means" is a device or software for notifying the user of the ordered item and the expected arrival date.

[0415] An "emotion engine" is a device or software that recognizes a user's emotional state from their voice and facial expressions and performs processing based on the emotional information.

[0416] A "server" is a computer system or service that coordinates a series of processes such as data collection, analysis, inventory estimation, automatic ordering, and notification.

[0417] A "smartphone" is a mobile communication terminal that has advanced processing capabilities in addition to the functions of a mobile phone and can run applications.

[0418] "Smart glasses" are glasses-shaped devices that can display information and collect data.

[0419] "Emotion information" is data indicating the emotional state recognized by the emotion engine from the user's voice and facial expression.

[0420] "Customer service actions" refer to the service content and response policy provided depending on the user's emotional state.

[0421] This invention relates to an inventory management and automatic ordering system for daily necessities using household electrical appliances, and in particular to a system that realizes optimal inventory management according to the user's emotional state by combining an emotion engine. This system is composed of electrical appliances installed in the home, a server, terminals (smartphones or smart glasses), the emotion engine, and the user.

[0422] System Overview

[0423] 1. Data Collection and Storage:

[0424] The server receives consumption data sent from the home appliances and stores it in a database in chronological order. For example, a detergent dispenser sends the amount of detergent consumed to the server and records it in the database.

[0425] 2. Data analysis and pattern learning:

[0426] The server analyzes the received consumption data and learns the user's daily consumption patterns. For example, it analyzes consumption amounts on a monthly basis to understand daily consumption trends.

[0427] 3. Inventory Estimation:

[0428] The server estimates current inventory levels based on learned consumption patterns and predicts when inventory levels will fall below a set threshold, allowing for timely replenishment.

[0429] 4. Automated ordering:

[0430] When the server determines that the inventory level is below the set threshold, it calls the online shop API to automatically order the necessary items, thereby preventing stockouts.

[0431] 5. Notification Processing:

[0432] The terminal notifies the user of the order details and expected arrival date from the server, and the user can receive this information via their smartphone or smart glasses.

[0433] 6. User Reporting Process:

[0434] Users verbally report stock availability and special requests via their smartphones or smart glasses, which are then sent to the server and reflected in inventory estimates and order frequency.

[0435] 7. Emotion recognition:

[0436] The emotion engine recognizes emotions from the user's voice and facial expressions and sends that emotional information to the server. For example, if the user is under stress, the emotion engine analyzes that information in real time and notifies the server.

[0437] 8. Emotion-based processing:

[0438] The server adjusts inventory management and ordering strategies based on the emotion information from the emotion engine. For example, if a user is feeling stressed, it can improve user satisfaction by quickly replenishing necessary items.

[0439] Hardware and Software Use Cases

[0440] Hardware:

[0441] Smartphone: A device that allows users to check inventory status and send emotional information.

[0442] Smart glasses: A device that allows store staff to check inventory information and how to respond based on customer sentiment.

[0443] Consumer electronics: Devices for collecting consumption data and sending it to a server.

[0444] software:

[0445] Server: A platform that oversees data collection, analysis, inventory estimation, automatic ordering, and notifications.

[0446] Emotion engine: Software for recognizing the user's emotional state.

[0447] Consumption data analysis software: Tools for learning consumption patterns and estimating inventory status.

[0448] Specific examples

[0449] Here is a specific example of managing a user's daily detergent consumption. For example, if 45 ml of household detergent is consumed daily and the initial inventory is 500 ml, the washing machine sends the amount of detergent consumed each time to the server. The server analyzes past data and learns that the user consumes approximately 315 ml of detergent per week. If the user's inventory is 500 ml, the server predicts that the inventory will fall below 100 ml in approximately 11.11 days and orders additional detergent from the online shop at the appropriate time. Furthermore, if the user verbally reports that they need more detergent because they have a party next week, the server adjusts the order amount based on that information. If the emotion engine recognizes the user's stress level, the server responds more quickly and places an order immediately to reassure the user.

[0450] Prompt Sentence Examples

[0451] "Please tell us about the structure and process of an app for inventory management and improving customer service in a physical store. Please include specific examples, such as a case where you recognized customer emotions and linked them to an inventory management system."

[0452] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0453] Step 1:

[0454] Data collection

[0455] The server collects data on the items consumed by household electrical appliances (e.g., washing machines and detergent dispensers) in real time. The input at this time is consumption data sent from each household electrical appliance, specifically including the amount consumed (e.g., 10 ml of detergent) and the date and time of consumption. The server receives this data and stores it in a database in chronological order. The information inserted into the database consists of "item name, amount consumed, and date and time of consumption."

[0456] Step 2:

[0457] Data analysis

[0458] The server analyzes the collected consumption data and learns the user's daily consumption patterns. The input is past consumption data stored in a database, and the output is consumption patterns (e.g., average daily consumption). The server uses data analysis software to statistically analyze the past consumption data and calculate the consumption patterns of the user's goods. Specifically, it calculates consumption trends by day, week, and month.

[0459] Step 3:

[0460] Inventory Estimation

[0461] The server estimates the current inventory level based on the learned consumption patterns. The input is the current inventory level and consumption pattern data, and the output is a prediction of when the inventory will fall below a set threshold. The server uses an inventory estimation algorithm to calculate how much inventory is currently remaining and predict when the inventory will run out. Specifically, the server estimates when the inventory will run out by subtracting the consumption pattern from the current inventory level.

[0462] Step 4:

[0463] Automatic ordering

[0464] If the server determines that the inventory level is below a set threshold, it calls the online shop API to automatically order the required items. The input is the inventory estimation result (when inventory will run out), and the output is the result of the order request sent. The server sends an order request through the online shop API to instruct the replenishment of the required items. Specifically, it sends a request to the API including the product ID and order quantity, and checks the response.

[0465] Step 5:

[0466] Notification Processing

[0467] The server notifies the user of the order details and expected arrival date. The input is the response data from the online shop API (order details and expected arrival date), and the output is a notification message. The server uses notification generation software to create a notification message for the user and sends it to the user via their smartphone or smart glasses. Specifically, a notification including the product name, order quantity, and expected arrival date is pushed to the user's device.

[0468] Step 6:

[0469] User Report Processing

[0470] Users verbally report inventory status or special requests via smartphones or smart glasses. The input is voice input from the user, and the output is a request converted into text. The device uses voice recognition software to analyze the voice input, convert it into text data, and send it to the server. Specifically, a voice request such as "We need to order more for next week's party" can be converted into text and sent to the server.

[0471] Step 7:

[0472] emotion recognition

[0473] The emotion engine recognizes emotions from the user's voice and facial expressions and sends that emotional information to the server. The input is the user's voice and facial expression data, and the output is analyzed emotional information. The device uses the emotion engine to analyze emotions from the voice and facial expressions and sends the emotional state (e.g., stress, joy, etc.) to the server. Specifically, changes in voice tone and facial expressions are detected in real time and recorded as the emotional state.

[0474] Step 8:

[0475] Emotion-Based Processing

[0476] The server adjusts inventory management and ordering strategies based on the emotion information from the emotion engine. The input is the emotion recognition results, and the output is adjusted ordering instructions and inventory management strategies. The server analyzes the emotion recognition results, and if the user is feeling stressed, for example, it can improve user satisfaction by quickly ordering the necessary items. Specifically, it can take measures such as setting an urgent order flag and placing an order immediately.

[0477] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0478] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0479] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0480] [Second embodiment]

[0481] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0482] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0483] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0484] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0485] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0486] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0487] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0488] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0489] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0490] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0491] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0492] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0493] The present invention relates to a system for managing inventory and automatically ordering daily necessities in the home. The following describes an embodiment of the invention. This system is composed of various elements, including home appliances, a server, terminals (smart speakers and apps), and users.

[0494] System Overview

[0495] In this system, data on the items consumed by household appliances is collected and sent to a server. The server analyzes this data and learns consumption patterns. Based on the learned consumption patterns, the server estimates the inventory status of each item and automatically orders items when inventory falls below a set threshold. The order details and expected arrival date are notified to the user via their terminal, and the user can also verbally report inventory status or special requests. The user's reports are also analyzed by the server and reflected in inventory forecasts and adjustments to ordering frequency.

[0496] Program processing overview

[0497] 1. Data Collection:

[0498] The server receives consumption data sent from the home appliances, including the type of item used, the amount used, and the date and time of use.

[0499] Example: Data is sent to the server that a detergent dispenser has used 10ml of detergent.

[0500] 2. Data analysis and learning:

[0501] The server analyzes the received consumption data and learns the user's consumption patterns, which allows it to predict future consumption based on past consumption data.

[0502] Example: Learning a pattern that the average amount of detergent used per day over a week is 45ml.

[0503] 3. Inventory Estimation:

[0504] The server estimates current inventory levels and predicts when inventory will fall below a set threshold based on consumption patterns.

[0505] Example: It is estimated that a 500ml stock will fall below the planned consumption level in 20 days.

[0506] 4. Automated ordering:

[0507] When the amount falls below the set threshold, the server automatically orders items from the online shop, using the online shop's API to place the order.

[0508] Example: When you run out of detergent below 50ml, order a 500ml bottle of detergent from an online shop.

[0509] 5. User Notice:

[0510] The terminal will notify the user of the order details and estimated arrival date, and the user can receive notifications via their smartphone or smart speaker.

[0511] For example, the device will announce, "We're low on detergent, so we've placed an order for more. It's expected to arrive in two days."

[0512] 6. User Reports:

[0513] Users can verbally report stock availability and special requests, and this information is sent to the server via the smart speaker and used to analyze consumption patterns and estimate inventory.

[0514] Example: A user reports, "I have a party and need some extra detergent."

[0515] Specific examples

[0516] As an example, consider a household where users do laundry every day and consume 30ml of laundry detergent per day. In this case, the washing machine (a household appliance) sends the amount of detergent used per load to a server every day. The server analyzes past data and learns that the user consumes approximately 210ml of detergent per week. If the user has 500ml in stock, the server predicts that the stock will run out in approximately 11.67 days and automatically orders more detergent before it falls below a threshold (e.g., 100ml). The user is notified of the order and the expected arrival date, and can report any special requests via a smart speaker. Based on this information, the server adjusts inventory management and ordering frequency.

[0517] In this way, users can save time and effort in inventory management and shopping, and receive the items they need at the right time.

[0518] The processing flow will be explained below.

[0519] Step 1: Data collection

[0520] The server receives consumption data sent from the home appliances, including the type of item used, the amount used, and the date and time of use.

[0521] Example: Data that a detergent dispenser used 10ml of detergent is sent to the server, and the server records it as "Detergent 10ml Used Date and Time: 2023-10-05 18:45".

[0522] Step 2: Save data

[0523] The server stores the received consumption data in a database in chronological order.

[0524] Example: Store information in a database in the format "Product: Detergent, Amount Used: 10ml, Date and Time: 2023-10-05 18:45".

[0525] Step 3: Data analysis

[0526] The server executes data analysis tasks at regular intervals, learning and analyzing users' consumption patterns based on past consumption data.

[0527] Example: Data for one month is collected and the average consumption is calculated. The analysis results show that an average of 45ml of detergent is consumed per day.

[0528] Step 4: Inventory estimation

[0529] The server estimates the current inventory level based on the analysis results and predicts when the inventory will fall below a set threshold based on consumption patterns.

[0530] Example: If the initial inventory is 500ml and 45ml is consumed per day, it is predicted that the inventory will run out in 500ml ÷ 45ml / day ≈ 11.11 days.

[0531] Step 5: Automatic ordering decision

[0532] The server checks whether the estimated inventory level falls below a set threshold, and if so, automatically places an order.

[0533] Example: If the threshold is set to 100ml, an auto-order flag will be set when the estimated stock level falls below 100ml.

[0534] Step 6: Automatic Order Execution

[0535] The server calls the online shop's API to order the necessary items.

[0536] Example: The server sends a request to the "Online Shop API" to "Order one 500ml bottle of detergent."

[0537] Step 7: Confirm order and prepare notification

[0538] The server receives the response from the online shop API and confirms the order details and expected delivery date.

[0539] Example: Receive a response from the API saying "500ml detergent, order completed, expected arrival date: 2023-10-07" and save the data.

[0540] Step 8: User Notification

[0541] The device will notify the user of the order details and estimated arrival date, and provide information via a smart speaker or app.

[0542] Example: A smart speaker notifies the user by voice, "We're running low on detergent, so we've placed an order for more. The expected delivery date is in two days."

[0543] Step 9: User Reporting

[0544] Users report stock availability and special requests through their smart speakers.

[0545] Example: A user reports, "I have a party and need some extra detergent."

[0546] Step 10: Report analysis and inventory adjustments

[0547] The server analyzes user reports and adjusts consumption forecasts, inventory estimates, and order frequency.

[0548] Example: The server receives a report of a user's "party" and predicts that consumption will increase next week, so it orders more to cover that increase.

[0549] This series of steps creates a system that allows users to conveniently manage inventory and automatically order daily necessities.

[0550] Example 1

[0551] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0552] Managing the inventory of everyday items in the home is a complicated and tedious task for many people. It is difficult to predict when items will run out and to purchase them appropriately, and there is a high possibility of inconvenience when they run out. Furthermore, flexible and accurate inventory management is required to respond to users' individual consumption patterns and special requests. However, existing systems are unable to adequately resolve these issues.

[0553] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0554] In this invention, the server includes: means for collecting data on items consumed by household electrical appliances; means for analyzing consumption data of the items and learning consumption patterns; means for estimating inventory status based on the consumption patterns; means for automatically ordering items when the estimated inventory status falls below a set threshold; means for notifying the user of the order details and expected arrival date of the items; and means for the user to verbally report the inventory status and special requests, and for analyzing the report to adjust the order frequency and inventory management. This allows the user to automatically manage the inventory status of daily necessities and automatically order necessary items at the appropriate time, significantly reducing the effort required for inventory management.

[0555] "Household appliances" refers to all electrically powered devices used in the home, including those equipped with sensors that collect consumption data.

[0556] "Goods" refers to consumables and daily necessities consumed by household electrical appliances, specifically including detergents, paper products, food ingredients, etc.

[0557] "Data collection means" refers to the functions and methods for recording consumption data generated when a household electrical appliance is used and transmitting it to a server.

[0558] "Data analysis means" refers to software or algorithms for processing received consumption data and analyzing user consumption patterns.

[0559] "Consumption pattern" is information that indicates the tendency of how a user consumes items within a specific period of time based on past consumption data.

[0560] "Inventory estimation means" refers to a function that uses consumption data and estimation algorithms to calculate current inventory levels and predict when inventory will run out in the future.

[0561] A "threshold" is a reference value set during inventory management, which becomes a trigger point for automatically ordering goods when the value falls below this value.

[0562] "Automatic ordering method" refers to a function or method for automatically ordering necessary items through the online shop's API when inventory falls below a threshold.

[0563] The "notification means" is a means for informing users of information such as order details and the expected arrival date of the item, and is implemented using a smartphone or smart speaker.

[0564] "User reporting means" refers to a device such as a smart speaker that allows a user to verbally report stock status or special requests, and the server receives the report.

[0565] An "online shop API" is a program interface provided by an online shopping platform that is used to order and confirm goods.

[0566] A "machine learning algorithm" refers to a mathematical model or computational procedure that learns patterns and knowledge from data and makes predictions and inferences.

[0567] The present invention is a system for efficiently managing inventory and automatically ordering consumer goods in the home. Specific embodiments of the system will be described in detail below.

[0568] This system consists of various elements, including household appliances, servers, terminals (smart speakers and apps), and users.

[0569] System configuration

[0570] 1. Household appliances:

[0571] Each consumer electronic device is fitted with a sensor that collects data whenever the consumer item is used, and this data is sent to a server via Wi-Fi or other communication means.

[0572] Examples include washing machines and detergent dispensers, which send data such as the type and amount of items used and the date and time of use to a server.

[0573] 2. Server:

[0574] The server analyzes the received data and uses machine learning algorithms, including ridge regression and other statistical models, to learn the user's consumption patterns.

[0575] The server uses programming languages ​​such as Python to preprocess and analyze the data, specifically performing tasks such as filling in missing values, detecting outliers, and normalizing the data.

[0576] 3. Devices (smart speakers and apps):

[0577] The device has the ability to notify users of the order details and expected arrival date, and conveys this information to them via push notifications on their smartphones or voice notifications on their smart speakers.

[0578] These devices allow users to report stock availability and special requests by voice, which is converted to text and sent to the server.

[0579] Specific processing examples

[0580] 1. Data Collection:

[0581] Example: If the washing machine uses 30ml of detergent in one wash, it sends that information to the server.

[0582] 2. Data analysis and learning:

[0583] Example: Based on data from the past week, the server learns that an average of 45ml of detergent is consumed per day.

[0584] 3. Inventory Estimation:

[0585] Example: There is currently 500ml in stock, and the server estimates that in 20 days the stock will fall below the threshold.

[0586] 4. Automated ordering:

[0587] Example: When stock falls below a threshold, the server orders 500ml of detergent through an online shop API.

[0588] 5. User Notice:

[0589] Example: "We're low on detergent so we've placed an order. It's expected to arrive in two days," your smart speaker will announce.

[0590] 6. User Reports:

[0591] Example: A user reports, "I'm having a party and need some extra detergent," and the smart speaker sends that information to the server.

[0592] This system allows users to eliminate the hassle of daily inventory management and automatically receive the items they need at the right time. Specific examples of prompts include "Tell me what's in stock today," "Tell me when the next shipment of detergent is expected to arrive," and "Order more detergent for a party."

[0593] The system provided by this invention can thoroughly automate the management of household items and improve user convenience.

[0594] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0595] Step 1: Data collection

[0596] 1. Input:

[0597] Consumption data generated by household appliances (e.g., type of item used, amount used, date and time of use).

[0598] 2. Processing:

[0599] Household appliances use sensors to measure and collect consumption data, for example, if a washing machine uses 30ml of detergent, it will measure that data.

[0600] 3. Data Processing:

[0601] The household appliances format the measured data and send it to a server via Wi-Fi.

[0602] 4. Output:

[0603] Formatted consumption data sent to the server.

[0604] 5. Specific operation:

[0605] When the washing machine starts washing and 30ml of detergent is used, the sensor detects the amount used, compiles the data and sends it to the server.

[0606] Step 2: Data analysis and training

[0607] 1. Input:

[0608] Consumption data of household appliances sent to a server.

[0609] 2. Processing:

[0610] The server preprocesses the received data, performing missing value imputation, outlier detection, and data normalization.

[0611] 3. Data Processing:

[0612] The server inputs the organized data into a machine learning algorithm (e.g., ridge regression) to learn the user's consumption patterns.

[0613] 4. Output:

[0614] A model that learns consumption patterns.

[0615] 5. Specific operation:

[0616] The server analyzes consumption data from the past week and learns that "users consume an average of 45ml of detergent per day."

[0617] Step 3: Inventory estimation

[0618] 1. Input:

[0619] Learned consumption pattern model and current inventory levels.

[0620] 2. Processing:

[0621] The server accumulates the initial inventory and consumption data to estimate the current inventory amount.

[0622] 3. Data Calculation:

[0623] Based on consumption patterns, the server predicts when inventory will fall below a set threshold.

[0624] 4. Output:

[0625] The point at which it is estimated to fall below the threshold.

[0626] 5. Specific operation:

[0627] The server calculates that the current detergent inventory of 500ml will fall below consumption in 20 days.

[0628] Step 4: Automated ordering

[0629] 1. Input:

[0630] Prediction of when inventory will fall below a threshold.

[0631] 2. Processing:

[0632] When the amount falls below the set threshold, the server calls the online shop API to order the item.

[0633] 3. Data Calculation:

[0634] The server sends the order details to the online shop using an HTTP POST request.

[0635] 4. Output:

[0636] Order requests sent to the online shop.

[0637] 5. Specific operation:

[0638] The server uses the API to send a request to the online shop to "order one 500ml bottle of detergent."

[0639] Step 5: User Notification

[0640] 1. Input:

[0641] Notification data that the order has been completed.

[0642] 2. Processing:

[0643] The terminal receives the notification data from the server and notifies the user by display or voice.

[0644] 3. Data Calculation:

[0645] The terminal converts the notification content into an appropriate format and transmits it to the user.

[0646] 4. Output:

[0647] The order details and expected delivery date notified to the user.

[0648] 5. Specific operation:

[0649] The smart speaker will announce, "We're running low on detergent, so we've placed an order for more. The expected delivery date is two days from now."

[0650] Step 6: User Reporting

[0651] 1. Input:

[0652] User voice reporting.

[0653] 2. Processing:

[0654] The device records the audio data and converts it into text.

[0655] 3. Data Processing:

[0656] The terminal transmits the converted text data to the server.

[0657] 4. Output:

[0658] The user's reported text data sent to the server.

[0659] 5. Specific operation:

[0660] The user reports, "I'm having a party so I need some extra detergent," and the smart speaker sends that information to the server.

[0661] The above is the specific processing flow of this system. The specific input, processing, and output are clearly defined at each step, and it is designed to enable users to quickly and efficiently manage their home inventory.

[0662] (Application example 1)

[0663] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0664] In modern households, inventory management of daily consumed goods and food is time-consuming and difficult to replenish at the appropriate time. In particular, busy households and households with elderly people are prone to stock shortages, which increases the effort required for shopping to address these shortages. Furthermore, there is a need for an inventory management method that can flexibly respond to unexpected events and changes in plans. To solve these problems, a system is needed that automatically manages inventory and replenishes necessary goods and food at the appropriate time.

[0665] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0666] In this invention, the server includes: means for collecting food consumption data using sensors in household appliances; means for analyzing the collected food consumption data and learning a user's consumption patterns; means for estimating inventory status based on the consumption patterns; means for automatically ordering food from a food delivery service when the estimated inventory status falls below a set threshold; means for notifying the user of the food order details and expected arrival date via the food delivery service's API; means for the user to verbally report inventory status and special requests; means for analyzing the user's report and adjusting inventory estimation and order frequency; means for collecting food data using sensors installed in household refrigerators and smart microwaves; means for learning consumption patterns based on the collected food data and estimating food inventory status; means for ordering items using an online shop API; means for confirming the order details and expected arrival date based on a response from the API; means for placing an order using the food delivery service API; and means for notifying the user of the order details and expected arrival date. This reduces the user's effort and enables automatic replenishment of necessary items and foods at the appropriate time.

[0667] "Household electrical appliances" is a general term for products that operate using electricity and are used in the home.

[0668] "Consumption data" refers to information such as the type of items used by a household electrical appliance, the amount used, and the date and time of use.

[0669] "Consumption patterns" are data that indicate trends in the frequency and amount of goods consumed based on past consumption data.

[0670] "Inventory status" refers to the current stock of items in the home.

[0671] The "threshold" is a reference value below which inventory is automatically deemed to need replenishment.

[0672] "Food delivery service" refers to a service that delivers ingredients and food to a designated location.

[0673] An "API" is an interface for exchanging data with other programs or services.

[0674] "User notification" means informing the user of information such as the order contents and expected arrival date.

[0675] "Verbal reporting" refers to the act of a user reporting stock availability or special requests using voice.

[0676] "Analysis" refers to processing collected data to extract useful information.

[0677] "Ordering" refers to the act of ordering necessary goods or food.

[0678] "Estimated Delivery Date" means the date on which the ordered items are expected to arrive at the specified location.

[0679] The present invention provides a system for managing inventory and automatically ordering daily necessities in the home. This system is composed of various elements, including home appliances, a server, terminals (e.g., smart speakers and smartphone apps), and users.

[0680] The program for realizing this system consists of the following steps:

[0681] Data collection

[0682] The server collects food consumption data using sensors built into household appliances (e.g., refrigerators and smart microwaves). The collected data includes the type of food consumed, the amount consumed, and the date and time of consumption. This data is transmitted to the server via Wi-Fi.

[0683] Data Analysis and Learning

[0684] The server analyzes the received food consumption data and learns the user's consumption patterns. This analysis is performed using data analysis libraries such as pandas and numpy. Based on past data, the server predicts how often and how much food the user will consume.

[0685] Inventory Estimation

[0686] The server estimates the household food inventory based on consumption patterns. When the estimated inventory falls below a set threshold, it automatically orders food from a food delivery service. This order uses the food delivery service's API to confirm the order details and expected arrival date.

[0687] Automatic ordering

[0688] The server automatically orders food from a food delivery service when the ingredients fall below a threshold. This ordering process uses an API interface, allowing the order details to be confirmed. The required food is ordered via the food delivery service's API.

[0689] User Notifications

[0690] The terminal notifies the user of the order details and expected arrival date. The notification is made via voice using a smart speaker API. Users can receive the information via their smartphone or smart speaker.

[0691] User Reports

[0692] Users can verbally report stock availability and special requests, and this information is sent to the server via the smart speaker, which then uses it to manage inventory and adjust ordering frequency.

[0693] Specific examples

[0694] For example, suppose a home refrigerator contains a variety of vegetables, and a sensor measures daily consumption. The server analyzes this data and learns that an average person consumes 300g of vegetables per week. If inventory falls below a 100g threshold, it automatically orders 500g of vegetables from a food delivery service. At this time, the user is notified that "We're running low on vegetables, so we've ordered an additional 500g. The estimated delivery date is tomorrow." If the user also tells the smart speaker, "A friend is coming over this weekend, so please order some more fruit," this information will be reflected in the next order.

[0695] Example prompts to input to the generative AI model

[0696] Calculate the average vegetable consumption for a week from the vegetable inventory management data in the refrigerator, and determine whether the inventory is below the set threshold (100g). If it falls below 100g, automatically order 500g of vegetables via a food delivery service and notify the user.

[0697] This system allows users to significantly reduce the hassle of daily inventory management and shopping, and allows them to receive the items and food they need at the right time.

[0698] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0699] Step 1:

[0700] The server collects food consumption data through sensors built into household appliances (e.g., refrigerators and smart microwaves). The data measured by the sensors includes the type of food consumed, the amount consumed, and the date and time of consumption. This data is transmitted to the server via Wi-Fi. The input is raw data from the sensors, and the output is data stored on the server for analysis.

[0701] Step 2:

[0702] The server analyzes the collected food consumption data and learns the user's consumption patterns. It uses data analysis libraries such as pandas and numpy to predict how often and how much food the user will consume based on past data. The input is the consumption data collected in step 1, and the output is the consumption pattern as a result of the analysis.

[0703] Step 3:

[0704] The server estimates the household food inventory status based on the learned consumption patterns. Based on the current inventory and consumption patterns, it predicts when the inventory will fall below a threshold. The inputs are the consumption patterns and the current inventory, and the output is the estimated inventory status.

[0705] Step 4:

[0706] The server automatically orders food from a food delivery service when the estimated inventory falls below a set threshold. The required food is ordered through the food delivery service's API. The input is the condition that inventory has fallen below the threshold and the order details, and the output is an order confirmation sent to the food delivery service.

[0707] Step 5:

[0708] The terminal notifies the user of the order details and expected arrival date. A smart speaker API is used to notify the user by voice or via a smartphone app. The input is the order details and expected arrival date, and the output is the notification information sent to the user.

[0709] Step 6:

[0710] Users verbally report stock status and special requests. This information is sent to the server via a smart speaker and used to estimate the next inventory and adjust ordering frequency. The input is the user's voice report, and the output is the report data stored on the server.

[0711] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0712] This paper describes an inventory management and automatic ordering system for daily necessities using household electrical appliances, and in particular a system that realizes optimal inventory management according to the user's emotional state by combining an emotion engine. This system is composed of electrical appliances installed in the home, a server, a terminal (smart speaker or app), an emotion engine, and the user.

[0713] System Overview

[0714] In this system, data on the goods consumed by household electrical appliances is collected on a daily basis and sent to a server. The server analyzes the received data and learns consumption patterns. Based on the learned consumption patterns, inventory status is estimated, and when inventory falls below a set threshold, goods are automatically ordered from an online shop. The order details and expected arrival date are notified to the user via their terminal. In addition, users can verbally report inventory status or special requests, which are also processed on the server side. In addition, an emotion engine is used to recognize user emotions and optimize inventory management and ordering processes based on emotional information.

[0715] Program processing overview

[0716] 1. Data Collection and Storage:

[0717] The server receives consumption data sent from the home appliances and stores it in a database in chronological order.

[0718] Example: A detergent dispenser receives data that 10ml of detergent has been used and records it as "Detergent 10ml Used Date: 2023-10-05 18:45".

[0719] 2. Data analysis and pattern learning:

[0720] The server analyzes the received consumption data and learns the user's daily consumption patterns, and then predicts future consumption based on the results of this learning.

[0721] Example: After analyzing data for one month, we learn that the average daily consumption of detergent is 45ml.

[0722] 3. Inventory Estimation:

[0723] Based on the learned consumption patterns, the server estimates the current inventory level and predicts when the inventory level will fall below a set threshold.

[0724] Example: If the inventory is 500ml and 45ml is consumed per day, it is estimated that the inventory will run out in approximately 11.11 days.

[0725] 4. Automated ordering:

[0726] If the server determines that the inventory level is below a set threshold, it calls the online shop API to automatically order the necessary items.

[0727] Example: When the warehouse stock drops below 50ml, a request is sent to the online shop API to order one 500ml bottle of detergent.

[0728] 5. Notification Processing:

[0729] The terminal receives notification of the order contents and expected arrival date from the server and notifies the user of the information.

[0730] For example, "We're low on detergent so we've placed an order for more. It's expected to arrive in two days."

[0731] 6. User Reporting Process:

[0732] Users verbally report stock availability and special requests through their smart speakers, and the information is sent to the server.

[0733] Example: A user reports, "I have a party and need some extra detergent."

[0734] 7. Emotion recognition:

[0735] The emotion engine recognizes emotions from the user's voice and facial expressions and transmits the emotion information to the server.

[0736] Example: Determine if the user is under stress based on their tone of voice and facial expression analysis.

[0737] 8. Emotion-based processing:

[0738] The server adjusts inventory management and ordering strategies based on the emotional information from the emotion engine.

[0739] Example: If a user is stressed, order needed items sooner or in larger quantities.

[0740] Specific examples

[0741] Here is a specific example of managing a user's daily detergent consumption. Assume that a user consumes 45 ml of household detergent per day, with an initial inventory of 500 ml. In this case, the washing machine (a household appliance) sends the amount of detergent consumed per use to the server. The server analyzes past data and learns that the user consumes approximately 315 ml of detergent per week. If the user's inventory is 500 ml, the server predicts that inventory will fall below 100 ml in approximately 11.11 days and places an order for additional detergent from an online shop at the appropriate time. The user is notified of the order details and expected arrival date. Furthermore, if the user verbally reports, "I need more detergent because I have a party next week," the server adjusts the order amount based on that information. If the emotion engine recognizes the user's stress level, the server can respond even more quickly and place an order immediately to reassure the user.

[0742] The processing flow will be explained below.

[0743] Step 1: Data collection

[0744] The server receives consumption data of items from the home appliances, including the type of item, the amount used, and the date and time of use.

[0745] Example: A washing machine sends data that 30ml of detergent has been used to the server, and the server records it as "30ml of detergent used: 2023-10-05 07:30".

[0746] Step 2: Save data

[0747] The server stores the received consumption data in a database in chronological order.

[0748] Example: Store information in a database in the format "Product: Detergent, Amount Used: 30ml, Date and Time: 2023-10-05 07:30".

[0749] Step 3: Data analysis and pattern learning

[0750] The server periodically executes a data analysis task to analyze consumption data and learn user consumption patterns.

[0751] Example: Based on one month's data, we calculate that the average daily detergent consumption is 45ml.

[0752] Step 4: Inventory estimation

[0753] The server estimates the current inventory level based on the learning results and predicts when the inventory will fall below a set threshold.

[0754] Example: Initial inventory is 500ml, consumption is 45ml per day, and it is predicted that inventory will be 50ml after approximately 11 days.

[0755] Step 5: Automatic ordering decision

[0756] The server checks whether the inventory level has fallen below the set threshold. If it has, it automatically places an order.

[0757] Example: When the stock reaches 50ml, the server determines that an automatic order is required.

[0758] Step 6: Automatic Order Execution

[0759] The server calls the online shop's API to order the necessary items.

[0760] Example: The server sends a request to the "Online Shop API" to "Order one 500ml bottle of detergent."

[0761] Step 7: Confirm order and prepare notification

[0762] The server receives the response from the online shop API and confirms the order details and expected delivery date.

[0763] Example: Receive a response from the API saying "500ml of detergent, order completed, expected arrival date: 2023-10-07" and save it in the database.

[0764] Step 8: User Notification

[0765] The terminal provides the user with notification of the order details and expected arrival date from the server.

[0766] Example: A smart speaker announces, "We're running low on detergent, so we've placed an order for more. The estimated delivery date is two days later."

[0767] Step 9: User Reporting

[0768] Users verbally report stock availability and special requests to the smart speaker.

[0769] Example: A user reports, "I have a party and need more detergent."

[0770] Step 10: Report analysis and inventory adjustments

[0771] The server analyzes the user's reports and adjusts inventory forecasts and order frequency.

[0772] Example: The server changes the settings to "Order more detergent than usual due to increased detergent consumption."

[0773] Step 11: Emotion Recognition

[0774] The emotion engine recognizes emotions from the user's tone of voice and facial expressions and sends that information to the server.

[0775] Example: When a user feels stressed, the engine sends the analysis results to the server.

[0776] Step 12: Emotion-Based Processing

[0777] The server adjusts inventory management and ordering strategies based on the emotional data from the emotion engine.

[0778] Example: If users are stressed, order more detergent early to avoid running out.

[0779] This series of steps will create a system that allows users to manage inventory of daily necessities, predict consumption, and receive emotion-based assistance.

[0780] Example 2

[0781] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0782] While conventional inventory management systems for daily necessities using home appliances are capable of managing inventory levels and automatically ordering based on consumption patterns, they have the problem of being unable to flexibly respond to user emotions or special circumstances. This has resulted in insufficient improvements in user convenience and satisfaction, and has led to waste due to inventory shortages and over-ordering under certain circumstances.

[0783] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0784] In this invention, the server includes means for collecting data on items consumed by the household electrical appliances, means for analyzing the consumption data of the items and learning consumption patterns, means for estimating inventory status based on the consumption patterns, means for automatically ordering items when the estimated inventory status falls below a set threshold, means for notifying a user of the order contents and expected arrival date of the items, means for recognizing the user's emotions, and means for adjusting inventory management and ordering strategies based on the emotion information, thereby enabling effective inventory management and automatic ordering according to the user's emotions and special requests.

[0785] "Household electrical appliances" refers to home appliances used in the home, including washing machines, refrigerators, microwave ovens, etc.

[0786] "Goods" refers to consumable items such as detergents and food consumed within the home.

[0787] "Means for collecting data" refers to a device or system capable of collecting consumption data generated by a household appliance.

[0788] "Means for analyzing data and learning consumption patterns" refers to a device or system that has the function of analyzing collected consumption data and learning users' consumption patterns using machine learning algorithms or the like.

[0789] "Means for estimating inventory status" refers to a device or system capable of estimating the current inventory of an item based on consumption patterns.

[0790] "Means for automatically ordering goods" means a device or system that has the capability to automatically order goods using an online system when inventory levels fall below a set threshold.

[0791] "Means of notifying the user" refers to telecommunications means for informing the user of the order details and expected arrival date of the item, including smart speakers and smartphones.

[0792] "Means for recognizing emotions" refers to a device or system that has the function of analyzing and recognizing emotions from the user's voice and facial expressions.

[0793] "Means for adjusting inventory management and ordering strategies based on emotional information" refers to a device or system that has the function of appropriately adjusting inventory management and ordering processes based on recognized user emotions.

[0794] "Online Shop API" refers to an application programming interface that connects with online shops via the Internet to order goods and check their status.

[0795] "Means for analyzing reports and adjusting inventory estimates and ordering frequencies" refers to a device or system that has the function of analyzing reports from users and appropriately adjusting inventory estimates and ordering frequencies based on that information.

[0796] This invention relates to an inventory management and automatic ordering system for daily necessities using household electrical appliances, and in particular to a system that realizes optimal inventory management according to the user's emotional state by combining an emotion engine. This system is composed of electrical appliances installed in the home, a server, a terminal (smart speaker or app), an emotion engine, and the user.

[0797] The server receives consumption data sent from household appliances and stores it in a database. For example, if a washing machine consumes 10ml of detergent in one use, the data is recorded as "Detergent 10ml Use Date: 2023-10-05 18:45." Data is collected in real time and sent to the server via the Internet. This data is organized chronologically and stored in a database.

[0798] The server then analyzes the collected data and uses machine learning algorithms (e.g., Python's pandas or scikit-learn) to learn the user's daily consumption patterns. For example, based on one month's consumption data, the server may learn that the average daily consumption of detergent is 45ml. This allows the server to make future consumption predictions.

[0799] Furthermore, the server estimates the current inventory level based on this learning result and predicts when the inventory will fall below a set threshold. For example, if the initial inventory is 500 ml and 45 ml is consumed per day, the server estimates that the inventory will fall below 50 ml in approximately 11.11 days.

[0800] When it is determined that the inventory level is below the threshold, the server calls an online store API (such as Amazon's API or Rakuten API) and automatically places an order for the necessary items. Specifically, when inventory falls below 50ml, the server sends a request to the online store API to order one 500ml bottle of detergent. This is done using an HTTP request that includes the API key and order information.

[0801] Information about the order contents and expected arrival date from the server is notified to the user via a device (smart speaker or app). For example, a notification message such as "We've placed an additional order for detergent because we're running low on it. The expected arrival date is two days later" is displayed. This is achieved by using the smart speaker's text-to-speech (TTS) function or the smartphone's push notification function.

[0802] Users can also use smart speakers to verbally report inventory status or special requests. For example, if a user says, "I'm going to a party and need more detergent," the voice data is converted into text data and sent to the server. The server analyzes this information and adjusts inventory estimates and ordering frequency.

[0803] Furthermore, the emotion engine analyzes emotional information from the user's voice and facial expressions and sends the results to the server. For example, if a user says "I feel very stressed" in a strong tone, the emotion engine will analyze the voice tone and facial expression and determine that the user is in a stressed state. This is done using libraries and services specialized in emotion recognition (e.g., AWS Rekognition, Google Cloud Vision).

[0804] The server adjusts inventory management and ordering strategies based on the emotion information from the emotion engine. For example, if the emotion engine determines that the user is feeling stressed, the server will order necessary items such as detergent more quickly or in larger quantities. Also, if it determines that special measures are needed, the server will adjust the order by raising the priority.

[0805] Example prompts to input to the generative AI model

[0806] Describe the specific process for an inventory management and automated ordering system for everyday items using consumer electronics, from data collection to processing based on emotion recognition. Include the following keywords: inventory management, consumption patterns, online shop, notification, emotion recognition.

[0807] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0808] Step 1: Data collection and storage

[0809] The server receives consumption data sent from home appliances in real time and stores it in a database. Specifically, the server receives data sent from washing machines, dispensers, etc. and stores it in a chronological order in the database.

[0810] Input: Consumption data from a household appliance (e.g., 10 ml of detergent used).

[0811] Data processing: Organize received data with timestamps.

[0812] Output: Organized consumption data is stored in a database.

[0813] Specific operation: The server receives data such as "Detergent 10ml, date and time of use: 2023-10-05 18:45" and records it in the database.

[0814] Step 2: Data analysis and pattern learning

[0815] The server analyzes the consumption data stored in the database and learns the user's daily consumption patterns using machine learning algorithms (e.g., Python's pandas and scikit-learn).

[0816] Input: Consumption data stored in a database.

[0817] Data processing: Machine learning algorithms are used to analyze daily consumption patterns.

[0818] Output: Learning results about users' consumption patterns.

[0819] Specific operation: The server analyzes, for example, one month's worth of data and learns that "users consume an average of 45 ml of detergent per day."

[0820] Step 3: Inventory estimation

[0821] The server estimates the current inventory level based on the learned consumption patterns and predicts when it will fall below the threshold.

[0822] Input: Learning results of consumption patterns and current inventory.

[0823] Data processing: Uses a mathematical formula to estimate based on inventory volume and consumption rate.

[0824] Output: Prediction of when inventory will fall below threshold.

[0825] Specific operation: If the inventory is 500 ml and daily consumption is 45 ml / day, the server estimates that the inventory will fall below 50 ml in approximately 11.11 days.

[0826] Step 4: Automated ordering

[0827] If the server determines that the inventory level is below the threshold, it calls the online shop API to automatically order the item.

[0828] Input: Inventory quantity estimation result.

[0829] Data processing: Generation of order requests using the online shop API.

[0830] Output: Order request and response to online shop.

[0831] Specific operation: When the stock falls below 50ml, the server sends a request to the online shop API to order one 500ml bottle of detergent.

[0832] Step 5: Notification Processing

[0833] The terminal receives notification of the order contents and expected arrival date from the server and conveys the information to the user.

[0834] Input: Order details and expected arrival date information from the server.

[0835] Data processing: generating text messages and voice notifications.

[0836] Output: Notification to the user.

[0837] Specific operation: The terminal notifies the user, "We are running low on detergent, so we have placed an additional order. The expected delivery date is two days later."

[0838] Step 6: User Report Processing

[0839] Users use smart speakers to verbally report stock availability and special requests, which are then sent to the server.

[0840] Input: User's voice report.

[0841] Data processing: Text conversion and analysis of voice data.

[0842] Output: Report to the server.

[0843] Specific operation: When a user verbally reports, "I'm having a party and need some extra detergent," the speech is converted into text and sent to the server.

[0844] Step 7: Emotion Recognition

[0845] The emotion engine analyzes emotional information from the user's voice and facial expressions and sends the results to the server.

[0846] Input: User's voice and facial expression data.

[0847] Data processing: Applying emotion recognition algorithms.

[0848] Output: User's emotional information.

[0849] Specific behavior: The emotion engine analyzes the user's tone of voice and facial expressions to recognize stress levels and other emotions.

[0850] Step 8: Emotion-Based Processing

[0851] The server adjusts inventory management and ordering strategies based on the emotional information from the emotion engine.

[0852] Input: Emotion information from the emotion engine.

[0853] Data processing: Adjusting inventory management and ordering strategies.

[0854] Output: Coordinated inventory management and ordering instructions.

[0855] Specific behavior: If the server recognizes that the user is stressed, it will order the necessary items more quickly or in larger quantities.

[0856] (Application example 2)

[0857] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0858] In modern store operations, inventory management and the quality of customer service are important issues. In particular, there is a need to prevent lost sales opportunities due to inventory shortages and to respond to customers' emotional states in a detailed manner. However, conventional systems have difficulty simultaneously solving these problems, making it difficult to achieve efficient inventory management and customer service. Therefore, there is a need for a system that combines an emotion engine to provide optimal inventory management and customer service that responds to the user's emotional state.

[0859] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting data on items consumed by household electrical appliances, means for analyzing the consumption data of the items and learning consumption patterns, means for estimating inventory status based on the consumption patterns, means for automatically ordering items when the estimated inventory status falls below a set threshold, means for notifying the user of the order details and expected arrival date of the items, and means for optimizing inventory management and ordering processes based on emotions using an emotion engine that recognizes the user's emotional state. This makes it possible to improve the efficiency of inventory management and user satisfaction.

[0860] Below are definitions of important words:

[0861] "Household appliances" are electrical devices used in ordinary households that can be used as a means of collecting consumption data.

[0862] "Goods" refers to the items that are consumed or controlled by a consumer electronics device.

[0863] "Data Collection Measure" means a device or software for periodically collecting data on goods consumed from a consumer electronics device.

[0864] "Data analysis means" refers to a device or software that analyzes and learns consumption patterns based on collected consumption data.

[0865] The "consumption pattern" indicates the user's tendency to consume items on a daily basis, and is learned by the data analysis means.

[0866] An "inventory estimator" is a device or software for estimating current inventory levels based on consumption patterns and assessing inventory status.

[0867] An "automatic ordering means" is a device or software for automatically ordering items through an online shop when estimated inventory falls below a set threshold.

[0868] The "notification means" is a device or software for notifying the user of the ordered item and the expected arrival date.

[0869] An "emotion engine" is a device or software that recognizes a user's emotional state from their voice and facial expressions and performs processing based on the emotional information.

[0870] A "server" is a computer system or service that coordinates a series of processes such as data collection, analysis, inventory estimation, automatic ordering, and notification.

[0871] A "smartphone" is a mobile communication terminal that has advanced processing capabilities in addition to the functions of a mobile phone and can run applications.

[0872] "Smart glasses" are glasses-shaped devices that can display information and collect data.

[0873] "Emotion information" is data indicating the emotional state recognized by the emotion engine from the user's voice and facial expression.

[0874] "Customer service actions" refer to the service content and response policy provided depending on the user's emotional state.

[0875] This invention relates to an inventory management and automatic ordering system for daily necessities using household electrical appliances, and in particular to a system that realizes optimal inventory management according to the user's emotional state by combining an emotion engine. This system is composed of electrical appliances installed in the home, a server, terminals (smartphones or smart glasses), the emotion engine, and the user.

[0876] System Overview

[0877] 1. Data Collection and Storage:

[0878] The server receives consumption data sent from the home appliances and stores it in a database in chronological order. For example, a detergent dispenser sends the amount of detergent consumed to the server and records it in the database.

[0879] 2. Data analysis and pattern learning:

[0880] The server analyzes the received consumption data and learns the user's daily consumption patterns. For example, it analyzes consumption amounts on a monthly basis to understand daily consumption trends.

[0881] 3. Inventory Estimation:

[0882] The server estimates current inventory levels based on learned consumption patterns and predicts when inventory levels will fall below a set threshold, allowing for timely replenishment.

[0883] 4. Automated ordering:

[0884] When the server determines that the inventory level is below the set threshold, it calls the online shop API to automatically order the necessary items, thereby preventing stockouts.

[0885] 5. Notification Processing:

[0886] The terminal notifies the user of the order details and expected arrival date from the server, and the user can receive this information via their smartphone or smart glasses.

[0887] 6. User Reporting Process:

[0888] Users verbally report stock availability and special requests via their smartphones or smart glasses, which are then sent to the server and reflected in inventory estimates and order frequency.

[0889] 7. Emotion recognition:

[0890] The emotion engine recognizes emotions from the user's voice and facial expressions and sends that emotional information to the server. For example, if the user is under stress, the emotion engine analyzes that information in real time and notifies the server.

[0891] 8. Emotion-based processing:

[0892] The server adjusts inventory management and ordering strategies based on the emotion information from the emotion engine. For example, if a user is feeling stressed, it can improve user satisfaction by quickly replenishing necessary items.

[0893] Hardware and Software Use Cases

[0894] Hardware:

[0895] Smartphone: A device that allows users to check inventory status and send emotional information.

[0896] Smart glasses: A device that allows store staff to check inventory information and how to respond based on customer sentiment.

[0897] Consumer electronics: Devices for collecting consumption data and sending it to a server.

[0898] software:

[0899] Server: A platform that oversees data collection, analysis, inventory estimation, automatic ordering, and notifications.

[0900] Emotion engine: Software for recognizing the user's emotional state.

[0901] Consumption data analysis software: Tools for learning consumption patterns and estimating inventory status.

[0902] Specific examples

[0903] Here is a specific example of managing a user's daily detergent consumption. For example, if 45 ml of household detergent is consumed daily and the initial inventory is 500 ml, the washing machine sends the amount of detergent consumed each time to the server. The server analyzes past data and learns that the user consumes approximately 315 ml of detergent per week. If the user's inventory is 500 ml, the server predicts that the inventory will fall below 100 ml in approximately 11.11 days and orders additional detergent from the online shop at the appropriate time. Furthermore, if the user verbally reports that they need more detergent because they have a party next week, the server adjusts the order amount based on that information. If the emotion engine recognizes the user's stress level, the server responds more quickly and places an order immediately to reassure the user.

[0904] Prompt Sentence Examples

[0905] "Please tell us about the structure and process of an app for inventory management and improving customer service in a physical store. Please include specific examples, such as a case where you recognized customer emotions and linked them to an inventory management system."

[0906] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0907] Step 1:

[0908] Data collection

[0909] The server collects data on the items consumed by household electrical appliances (e.g., washing machines and detergent dispensers) in real time. The input at this time is consumption data sent from each household electrical appliance, specifically including the amount consumed (e.g., 10 ml of detergent) and the date and time of consumption. The server receives this data and stores it in a database in chronological order. The information inserted into the database consists of "item name, amount consumed, and date and time of consumption."

[0910] Step 2:

[0911] Data analysis

[0912] The server analyzes the collected consumption data and learns the user's daily consumption patterns. The input is past consumption data stored in a database, and the output is consumption patterns (e.g., average daily consumption). The server uses data analysis software to statistically analyze the past consumption data and calculate the consumption patterns of the user's goods. Specifically, it calculates consumption trends by day, week, and month.

[0913] Step 3:

[0914] Inventory Estimation

[0915] The server estimates the current inventory level based on the learned consumption patterns. The input is the current inventory level and consumption pattern data, and the output is a prediction of when the inventory will fall below a set threshold. The server uses an inventory estimation algorithm to calculate how much inventory is currently remaining and predict when the inventory will run out. Specifically, the server estimates when the inventory will run out by subtracting the consumption pattern from the current inventory level.

[0916] Step 4:

[0917] Automatic ordering

[0918] If the server determines that the inventory level is below a set threshold, it calls the online shop API to automatically order the required items. The input is the inventory estimation result (when inventory will run out), and the output is the result of the order request sent. The server sends an order request through the online shop API to instruct the replenishment of the required items. Specifically, it sends a request to the API including the product ID and order quantity, and checks the response.

[0919] Step 5:

[0920] Notification Processing

[0921] The server notifies the user of the order details and expected arrival date. The input is the response data from the online shop API (order details and expected arrival date), and the output is a notification message. The server uses notification generation software to create a notification message for the user and sends it to the user via their smartphone or smart glasses. Specifically, a notification including the product name, order quantity, and expected arrival date is pushed to the user's device.

[0922] Step 6:

[0923] User Report Processing

[0924] Users verbally report inventory status or special requests via smartphones or smart glasses. The input is voice input from the user, and the output is a request converted into text. The device uses voice recognition software to analyze the voice input, convert it into text data, and send it to the server. Specifically, a voice request such as "We need to order more for next week's party" can be converted into text and sent to the server.

[0925] Step 7:

[0926] emotion recognition

[0927] The emotion engine recognizes emotions from the user's voice and facial expressions and sends that emotional information to the server. The input is the user's voice and facial expression data, and the output is analyzed emotional information. The device uses the emotion engine to analyze emotions from the voice and facial expressions and sends the emotional state (e.g., stress, joy, etc.) to the server. Specifically, changes in voice tone and facial expressions are detected in real time and recorded as the emotional state.

[0928] Step 8:

[0929] Emotion-Based Processing

[0930] The server adjusts inventory management and ordering strategies based on the emotion information from the emotion engine. The input is the emotion recognition results, and the output is adjusted ordering instructions and inventory management strategies. The server analyzes the emotion recognition results, and if the user is feeling stressed, for example, it can improve user satisfaction by quickly ordering the necessary items. Specifically, it can take measures such as setting an urgent order flag and placing an order immediately.

[0931] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0932] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0933] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0934] [Third embodiment]

[0935] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0936] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0937] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0938] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0939] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0940] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0941] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0942] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0943] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0944] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0945] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0946] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0947] The present invention relates to a system for managing inventory and automatically ordering daily necessities in the home. The following describes an embodiment of the invention. This system is composed of various elements, including home appliances, a server, terminals (smart speakers and apps), and users.

[0948] System Overview

[0949] In this system, data on the items consumed by household appliances is collected and sent to a server. The server analyzes this data and learns consumption patterns. Based on the learned consumption patterns, the server estimates the inventory status of each item and automatically orders items when inventory falls below a set threshold. The order details and expected arrival date are notified to the user via their terminal, and the user can also verbally report inventory status or special requests. The user's reports are also analyzed by the server and reflected in inventory forecasts and adjustments to ordering frequency.

[0950] Program processing overview

[0951] 1. Data Collection:

[0952] The server receives consumption data sent from the home appliances, including the type of item used, the amount used, and the date and time of use.

[0953] Example: Data is sent to the server that a detergent dispenser has used 10ml of detergent.

[0954] 2. Data analysis and learning:

[0955] The server analyzes the received consumption data and learns the user's consumption patterns, which allows it to predict future consumption based on past consumption data.

[0956] Example: Learning a pattern that the average amount of detergent used per day over a week is 45ml.

[0957] 3. Inventory Estimation:

[0958] The server estimates current inventory levels and predicts when inventory will fall below a set threshold based on consumption patterns.

[0959] Example: It is estimated that a 500ml stock will fall below the planned consumption level in 20 days.

[0960] 4. Automated ordering:

[0961] When the amount falls below the set threshold, the server automatically orders items from the online shop, using the online shop's API to place the order.

[0962] Example: When you run out of detergent below 50ml, order a 500ml bottle of detergent from an online shop.

[0963] 5. User Notice:

[0964] The terminal will notify the user of the order details and estimated arrival date, and the user can receive notifications via their smartphone or smart speaker.

[0965] For example, the device will announce, "We're low on detergent, so we've placed an order for more. It's expected to arrive in two days."

[0966] 6. User Reports:

[0967] Users can verbally report stock availability and special requests, and this information is sent to the server via the smart speaker and used to analyze consumption patterns and estimate inventory.

[0968] Example: A user reports, "I have a party and need some extra detergent."

[0969] Specific examples

[0970] As an example, consider a household where users do laundry every day and consume 30ml of laundry detergent per day. In this case, the washing machine (a household appliance) sends the amount of detergent used per load to a server every day. The server analyzes past data and learns that the user consumes approximately 210ml of detergent per week. If the user has 500ml in stock, the server predicts that the stock will run out in approximately 11.67 days and automatically orders more detergent before it falls below a threshold (e.g., 100ml). The user is notified of the order and the expected arrival date, and can report any special requests via a smart speaker. Based on this information, the server adjusts inventory management and ordering frequency.

[0971] In this way, users can save time and effort in inventory management and shopping, and receive the items they need at the right time.

[0972] The processing flow will be explained below.

[0973] Step 1: Data collection

[0974] The server receives consumption data sent from the home appliances, including the type of item used, the amount used, and the date and time of use.

[0975] Example: Data that a detergent dispenser used 10ml of detergent is sent to the server, and the server records it as "Detergent 10ml Used Date and Time: 2023-10-05 18:45".

[0976] Step 2: Save data

[0977] The server stores the received consumption data in a database in chronological order.

[0978] Example: Store information in a database in the format "Product: Detergent, Amount Used: 10ml, Date and Time: 2023-10-05 18:45".

[0979] Step 3: Data analysis

[0980] The server executes data analysis tasks at regular intervals, learning and analyzing users' consumption patterns based on past consumption data.

[0981] Example: Data for one month is collected and the average consumption is calculated. The analysis results show that an average of 45ml of detergent is consumed per day.

[0982] Step 4: Inventory estimation

[0983] The server estimates the current inventory level based on the analysis results and predicts when the inventory will fall below a set threshold based on consumption patterns.

[0984] Example: If the initial inventory is 500ml and 45ml is consumed per day, it is predicted that the inventory will run out in 500ml ÷ 45ml / day ≈ 11.11 days.

[0985] Step 5: Automatic ordering decision

[0986] The server checks whether the estimated inventory level falls below a set threshold, and if so, automatically places an order.

[0987] Example: If the threshold is set to 100ml, an auto-order flag will be set when the estimated stock level falls below 100ml.

[0988] Step 6: Automatic Order Execution

[0989] The server calls the online shop's API to order the necessary items.

[0990] Example: The server sends a request to the "Online Shop API" to "Order one 500ml bottle of detergent."

[0991] Step 7: Confirm order and prepare notification

[0992] The server receives the response from the online shop API and confirms the order details and expected delivery date.

[0993] Example: Receive a response from the API saying "500ml detergent, order completed, expected arrival date: 2023-10-07" and save the data.

[0994] Step 8: User Notification

[0995] The device will notify the user of the order details and estimated arrival date, and provide information via a smart speaker or app.

[0996] Example: A smart speaker notifies the user by voice, "We're running low on detergent, so we've placed an order for more. The expected delivery date is in two days."

[0997] Step 9: User Reporting

[0998] Users report stock availability and special requests through their smart speakers.

[0999] Example: A user reports, "I have a party and need some extra detergent."

[1000] Step 10: Report analysis and inventory adjustments

[1001] The server analyzes user reports and adjusts consumption forecasts, inventory estimates, and order frequency.

[1002] Example: The server receives a report of a user's "party" and predicts that consumption will increase next week, so it orders more to cover that increase.

[1003] This series of steps creates a system that allows users to conveniently manage inventory and automatically order daily necessities.

[1004] Example 1

[1005] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1006] Managing the inventory of everyday items in the home is a complicated and tedious task for many people. It is difficult to predict when items will run out and to purchase them appropriately, and there is a high possibility of inconvenience when they run out. Furthermore, flexible and accurate inventory management is required to respond to users' individual consumption patterns and special requests. However, existing systems are unable to adequately resolve these issues.

[1007] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1008] In this invention, the server includes: means for collecting data on items consumed by household electrical appliances; means for analyzing consumption data of the items and learning consumption patterns; means for estimating inventory status based on the consumption patterns; means for automatically ordering items when the estimated inventory status falls below a set threshold; means for notifying the user of the order details and expected arrival date of the items; and means for the user to verbally report the inventory status and special requests, and for analyzing the report to adjust the order frequency and inventory management. This allows the user to automatically manage the inventory status of daily necessities and automatically order necessary items at the appropriate time, significantly reducing the effort required for inventory management.

[1009] "Household appliances" refers to all electrically powered devices used in the home, including those equipped with sensors that collect consumption data.

[1010] "Goods" refers to consumables and daily necessities consumed by household electrical appliances, specifically including detergents, paper products, food ingredients, etc.

[1011] "Data collection means" refers to the functions and methods for recording consumption data generated when a household electrical appliance is used and transmitting it to a server.

[1012] "Data analysis means" refers to software or algorithms for processing received consumption data and analyzing user consumption patterns.

[1013] "Consumption pattern" is information that indicates the tendency of how a user consumes items within a specific period of time based on past consumption data.

[1014] "Inventory estimation means" refers to a function that uses consumption data and estimation algorithms to calculate current inventory levels and predict when inventory will run out in the future.

[1015] A "threshold" is a reference value set during inventory management, which becomes a trigger point for automatically ordering goods when the value falls below this value.

[1016] "Automatic ordering method" refers to a function or method for automatically ordering necessary items through the online shop's API when inventory falls below a threshold.

[1017] The "notification means" is a means for informing users of information such as order details and the expected arrival date of the item, and is implemented using a smartphone or smart speaker.

[1018] "User reporting means" refers to a device such as a smart speaker that allows a user to verbally report stock status or special requests, and the server receives the report.

[1019] An "online shop API" is a program interface provided by an online shopping platform that is used to order and confirm goods.

[1020] A "machine learning algorithm" refers to a mathematical model or computational procedure that learns patterns and knowledge from data and makes predictions and inferences.

[1021] The present invention is a system for efficiently managing inventory and automatically ordering consumer goods in the home. Specific embodiments of the system will be described in detail below.

[1022] This system consists of various elements, including household appliances, servers, terminals (smart speakers and apps), and users.

[1023] System configuration

[1024] 1. Household appliances:

[1025] Each consumer electronic device is fitted with a sensor that collects data whenever the consumer item is used, and this data is sent to a server via Wi-Fi or other communication means.

[1026] Examples include washing machines and detergent dispensers, which send data such as the type and amount of items used and the date and time of use to a server.

[1027] 2. Server:

[1028] The server analyzes the received data and uses machine learning algorithms, including ridge regression and other statistical models, to learn the user's consumption patterns.

[1029] The server uses programming languages ​​such as Python to preprocess and analyze the data, specifically performing tasks such as filling in missing values, detecting outliers, and normalizing the data.

[1030] 3. Devices (smart speakers and apps):

[1031] The device has the ability to notify users of the order details and expected arrival date, and conveys this information to them via push notifications on their smartphones or voice notifications on their smart speakers.

[1032] These devices allow users to report stock availability and special requests by voice, which is converted to text and sent to the server.

[1033] Specific processing examples

[1034] 1. Data Collection:

[1035] Example: If the washing machine uses 30ml of detergent in one wash, it sends that information to the server.

[1036] 2. Data analysis and learning:

[1037] Example: Based on data from the past week, the server learns that an average of 45ml of detergent is consumed per day.

[1038] 3. Inventory Estimation:

[1039] Example: There is currently 500ml in stock, and the server estimates that in 20 days the stock will fall below the threshold.

[1040] 4. Automated ordering:

[1041] Example: When stock falls below a threshold, the server orders 500ml of detergent through an online shop API.

[1042] 5. User Notice:

[1043] Example: "We're low on detergent so we've placed an order. It's expected to arrive in two days," your smart speaker will announce.

[1044] 6. User Reports:

[1045] Example: A user reports, "I'm having a party and need some extra detergent," and the smart speaker sends that information to the server.

[1046] This system allows users to eliminate the hassle of daily inventory management and automatically receive the items they need at the right time. Specific examples of prompts include "Tell me what's in stock today," "Tell me when the next shipment of detergent is expected to arrive," and "Order more detergent for a party."

[1047] The system provided by this invention can thoroughly automate the management of household items and improve user convenience.

[1048] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1049] Step 1: Data collection

[1050] 1. Input:

[1051] Consumption data generated by household appliances (e.g., type of item used, amount used, date and time of use).

[1052] 2. Processing:

[1053] Household appliances use sensors to measure and collect consumption data, for example, if a washing machine uses 30ml of detergent, it will measure that data.

[1054] 3. Data Processing:

[1055] The household appliances format the measured data and send it to a server via Wi-Fi.

[1056] 4. Output:

[1057] Formatted consumption data sent to the server.

[1058] 5. Specific operation:

[1059] When the washing machine starts washing and 30ml of detergent is used, the sensor detects the amount used, compiles the data and sends it to the server.

[1060] Step 2: Data analysis and training

[1061] 1. Input:

[1062] Consumption data of household appliances sent to a server.

[1063] 2. Processing:

[1064] The server preprocesses the received data, performing missing value imputation, outlier detection, and data normalization.

[1065] 3. Data Processing:

[1066] The server inputs the organized data into a machine learning algorithm (e.g., ridge regression) to learn the user's consumption patterns.

[1067] 4. Output:

[1068] A model that learns consumption patterns.

[1069] 5. Specific operation:

[1070] The server analyzes consumption data from the past week and learns that "users consume an average of 45ml of detergent per day."

[1071] Step 3: Estimate inventory

[1072] 1. Input:

[1073] Learned consumption pattern model and current inventory levels.

[1074] 2. Processing:

[1075] The server accumulates the initial inventory and consumption data to estimate the current inventory amount.

[1076] 3. Data Calculation:

[1077] Based on consumption patterns, the server predicts when inventory will fall below a set threshold.

[1078] 4. Output:

[1079] The point at which it is estimated to fall below the threshold.

[1080] 5. Specific operation:

[1081] The server calculates that the current detergent inventory of 500ml will fall below consumption in 20 days.

[1082] Step 4: Automated ordering

[1083] 1. Input:

[1084] Prediction of when inventory will fall below a threshold.

[1085] 2. Processing:

[1086] When the amount falls below the set threshold, the server calls the online shop API to order the item.

[1087] 3. Data Calculation:

[1088] The server sends the order details to the online shop using an HTTP POST request.

[1089] 4. Output:

[1090] Order requests sent to the online shop.

[1091] 5. Specific operation:

[1092] The server uses the API to send a request to the online shop to "order one 500ml bottle of detergent."

[1093] Step 5: User Notification

[1094] 1. Input:

[1095] Notification data that the order has been completed.

[1096] 2. Processing:

[1097] The terminal receives the notification data from the server and notifies the user by display or voice.

[1098] 3. Data Calculation:

[1099] The terminal converts the notification content into an appropriate format and transmits it to the user.

[1100] 4. Output:

[1101] The order details and expected delivery date notified to the user.

[1102] 5. Specific operation:

[1103] The smart speaker will announce, "We're running low on detergent, so we've placed an order for more. The expected delivery date is two days from now."

[1104] Step 6: User Reporting

[1105] 1. Input:

[1106] User voice reporting.

[1107] 2. Processing:

[1108] The device records the audio data and converts it into text.

[1109] 3. Data Processing:

[1110] The terminal transmits the converted text data to the server.

[1111] 4. Output:

[1112] The user's reported text data sent to the server.

[1113] 5. Specific operation:

[1114] The user reports, "I'm having a party so I need some extra detergent," and the smart speaker sends that information to the server.

[1115] The above is the specific processing flow of this system. The specific input, processing, and output are clearly defined at each step, and it is designed to enable users to quickly and efficiently manage their home inventory.

[1116] (Application example 1)

[1117] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1118] In modern households, inventory management of daily consumed goods and food is time-consuming and difficult to replenish at the appropriate time. In particular, busy households and households with elderly people are prone to stock shortages, which increases the effort required for shopping to address these shortages. Furthermore, there is a need for an inventory management method that can flexibly respond to unexpected events and changes in plans. To solve these problems, a system is needed that automatically manages inventory and replenishes necessary goods and food at the appropriate time.

[1119] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1120] In this invention, the server includes: means for collecting food consumption data using sensors in household appliances; means for analyzing the collected food consumption data and learning a user's consumption patterns; means for estimating inventory status based on the consumption patterns; means for automatically ordering food from a food delivery service when the estimated inventory status falls below a preset threshold; means for notifying the user of the food order details and expected arrival date via the food delivery service's API; means for the user to verbally report the inventory status and special requests; means for analyzing the user's report and adjusting the inventory estimate and ordering frequency; means for collecting food data using sensors installed in a household refrigerator or smart microwave; means for learning the consumption pattern based on the collected food data and estimating the food inventory status; means for ordering items using an online shop API; means for confirming the order details and expected arrival date based on a response from the API; means for placing an order using the food delivery service API; and means for notifying the user of the order details and expected arrival date. This reduces the user's effort and enables automatic replenishment of necessary items and foods at the appropriate time.

[1121] "Household electrical appliances" is a general term for products that operate using electricity and are used in the home.

[1122] "Consumption data" refers to information such as the type of items used by a household electrical appliance, the amount used, and the date and time of use.

[1123] "Consumption patterns" are data that indicate trends in the frequency and amount of goods consumed based on past consumption data.

[1124] "Inventory status" refers to the current stock of items in the home.

[1125] The "threshold" is a reference value below which inventory is automatically deemed to need replenishment.

[1126] "Food delivery service" refers to a service that delivers ingredients and food to a designated location.

[1127] An "API" is an interface for exchanging data with other programs or services.

[1128] "User notification" refers to informing the user of information such as the order contents and expected arrival date.

[1129] "Verbal reporting" refers to the act of a user reporting stock availability or special requests using voice.

[1130] "Analysis" refers to processing collected data to extract useful information.

[1131] "Ordering" refers to the act of ordering necessary goods or food.

[1132] "Estimated Delivery Date" means the date on which the ordered items are expected to arrive at the specified location.

[1133] The present invention provides a system for managing inventory and automatically ordering daily necessities in the home. This system is composed of various elements, including home appliances, a server, terminals (e.g., smart speakers and smartphone apps), and users.

[1134] The program for realizing this system consists of the following steps:

[1135] Data collection

[1136] The server collects food consumption data using sensors built into household appliances (e.g., refrigerators and smart microwaves). The collected data includes the type of food consumed, the amount consumed, and the date and time of consumption. This data is transmitted to the server via Wi-Fi.

[1137] Data Analysis and Learning

[1138] The server analyzes the received food consumption data and learns the user's consumption patterns. This analysis is performed using data analysis libraries such as pandas and numpy. Based on past data, the server predicts how often and how much food the user will consume.

[1139] Inventory Estimation

[1140] The server estimates the household food inventory based on consumption patterns. When the estimated inventory falls below a set threshold, it automatically orders food from a food delivery service. This order uses the food delivery service's API to confirm the order details and expected arrival date.

[1141] Automatic ordering

[1142] The server automatically orders food from a food delivery service when the ingredients fall below a threshold. This ordering process uses an API interface, allowing the order details to be confirmed. The required food is ordered via the food delivery service's API.

[1143] User Notifications

[1144] The terminal notifies the user of the order details and expected arrival date. The notification is made via voice using a smart speaker API. The user can receive the information via their smartphone or smart speaker.

[1145] User Reports

[1146] Users can verbally report stock availability and special requests, and this information is sent to the server via the smart speaker, which then uses it to manage inventory and adjust ordering frequency.

[1147] Specific examples

[1148] For example, suppose a home refrigerator contains a variety of vegetables, and a sensor measures daily consumption. The server analyzes this data and learns that an average person consumes 300g of vegetables per week. If inventory falls below a 100g threshold, it automatically orders 500g of vegetables from a food delivery service. At this time, the user is notified that "We're running low on vegetables, so we've ordered an additional 500g. The estimated delivery date is tomorrow." If the user also tells the smart speaker, "A friend is coming over this weekend, so please order some more fruit," this information will be reflected in the next order.

[1149] Example prompts to be input to the generative AI model

[1150] Calculate the average vegetable consumption for a week from the vegetable inventory management data in the refrigerator, and determine whether the inventory is below the set threshold (100g). If it falls below 100g, automatically order 500g of vegetables via a food delivery service and notify the user.

[1151] This system allows users to significantly reduce the hassle of daily inventory management and shopping, and allows them to receive the items and food they need at the right time.

[1152] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1153] Step 1:

[1154] The server collects food consumption data through sensors built into household appliances (e.g., refrigerators and smart microwaves). The data measured by the sensors includes the type of food consumed, the amount consumed, and the date and time of consumption. This data is transmitted to the server via Wi-Fi. The input is raw data from the sensors, and the output is data stored on the server for analysis.

[1155] Step 2:

[1156] The server analyzes the collected food consumption data and learns the user's consumption patterns. It uses data analysis libraries such as pandas and numpy to predict how often and how much food the user will consume based on past data. The input is the consumption data collected in step 1, and the output is the consumption pattern as a result of the analysis.

[1157] Step 3:

[1158] The server estimates the household food inventory status based on the learned consumption patterns. Based on the current inventory and consumption patterns, it predicts when the inventory will fall below a threshold. The inputs are the consumption patterns and the current inventory, and the output is the estimated inventory status.

[1159] Step 4:

[1160] The server automatically orders food from a food delivery service when the estimated inventory falls below a set threshold. The required food is ordered through the food delivery service's API. The input is the condition that inventory has fallen below the threshold and the order details, and the output is an order confirmation sent to the food delivery service.

[1161] Step 5:

[1162] The terminal notifies the user of the order details and expected arrival date. A smart speaker API is used to notify the user by voice or via a smartphone app. The input is the order details and expected arrival date, and the output is the notification information sent to the user.

[1163] Step 6:

[1164] Users verbally report stock status and special requests. This information is sent to the server via a smart speaker and used to estimate the next inventory and adjust ordering frequency. The input is the user's voice report, and the output is the report data stored on the server.

[1165] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1166] This paper describes an inventory management and automatic ordering system for daily necessities using household electrical appliances, and in particular a system that realizes optimal inventory management according to the user's emotional state by combining an emotion engine. This system is composed of electrical appliances installed in the home, a server, a terminal (smart speaker or app), an emotion engine, and the user.

[1167] System Overview

[1168] In this system, data on the goods consumed by household electrical appliances is collected on a daily basis and sent to a server. The server analyzes the received data and learns consumption patterns. Based on the learned consumption patterns, inventory status is estimated, and when inventory falls below a set threshold, goods are automatically ordered from an online shop. The order details and expected arrival date are notified to the user via their terminal. In addition, users can verbally report inventory status or special requests, which are also processed on the server side. In addition, an emotion engine is used to recognize user emotions and optimize inventory management and ordering processes based on emotional information.

[1169] Program processing overview

[1170] 1. Data Collection and Storage:

[1171] The server receives consumption data sent from the home appliances and stores it in a database in chronological order.

[1172] Example: A detergent dispenser receives data that 10ml of detergent has been used and records it as "Detergent 10ml Used Date: 2023-10-05 18:45".

[1173] 2. Data analysis and pattern learning:

[1174] The server analyzes the received consumption data and learns the user's daily consumption patterns, and then predicts future consumption based on the results of this learning.

[1175] Example: After analyzing data for one month, we learn that the average daily consumption of detergent is 45ml.

[1176] 3. Inventory Estimation:

[1177] Based on the learned consumption patterns, the server estimates the current inventory level and predicts when the inventory level will fall below a set threshold.

[1178] Example: If the inventory is 500ml and 45ml is consumed per day, it is estimated that the inventory will run out in approximately 11.11 days.

[1179] 4. Automated ordering:

[1180] If the server determines that the inventory level is below the set threshold, it calls the online shop API to automatically order the necessary items.

[1181] Example: When the warehouse stock drops below 50ml, a request is sent to the online shop API to order one 500ml bottle of detergent.

[1182] 5. Notification Processing:

[1183] The terminal receives notification of the order details and expected arrival date from the server and notifies the user of that information.

[1184] For example, "We're low on detergent so we've placed an order for more. It's expected to arrive in two days."

[1185] 6. User Reporting Process:

[1186] Users verbally report stock availability and special requests through their smart speakers, and the information is sent to the server.

[1187] Example: A user reports, "I have a party and need some extra detergent."

[1188] 7. Emotion recognition:

[1189] The emotion engine recognizes emotions from the user's voice and facial expressions and transmits the emotion information to the server.

[1190] Example: Determine if the user is under stress based on their tone of voice and facial expression analysis.

[1191] 8. Emotion-based processing:

[1192] The server adjusts inventory management and ordering strategies based on the emotional information from the emotion engine.

[1193] Example: If a user is stressed, order needed items sooner or in larger quantities.

[1194] Specific examples

[1195] Here is a specific example of managing a user's daily detergent consumption. Assume that a user consumes 45 ml of household detergent per day, with an initial inventory of 500 ml. In this case, the washing machine (a household appliance) sends the amount of detergent consumed per use to the server. The server analyzes past data and learns that the user consumes approximately 315 ml of detergent per week. If the user's inventory is 500 ml, the server predicts that inventory will fall below 100 ml in approximately 11.11 days and places an order for additional detergent from an online shop at the appropriate time. The user is notified of the order details and expected arrival date. Furthermore, if the user verbally reports, "I need more detergent because I have a party next week," the server adjusts the order amount based on that information. If the emotion engine recognizes the user's stress level, the server can respond even more quickly and place an order immediately to reassure the user.

[1196] The processing flow will be explained below.

[1197] Step 1: Data collection

[1198] The server receives consumption data of items from the home appliances, including the type of item, the amount used, and the date and time of use.

[1199] Example: A washing machine sends data that 30ml of detergent has been used to the server, and the server records it as "30ml of detergent used: 2023-10-05 07:30".

[1200] Step 2: Save data

[1201] The server stores the received consumption data in a database in chronological order.

[1202] Example: Store information in a database in the format "Product: Detergent, Amount Used: 30ml, Date and Time: 2023-10-05 07:30".

[1203] Step 3: Data analysis and pattern learning

[1204] The server periodically executes a data analysis task to analyze consumption data and learn user consumption patterns.

[1205] Example: Based on one month's data, we calculate that the average daily detergent consumption is 45ml.

[1206] Step 4: Inventory estimation

[1207] The server estimates the current inventory level based on the learning results and predicts when the inventory will fall below a set threshold.

[1208] Example: Initial inventory is 500ml, consumption is 45ml per day, and it is predicted that inventory will be 50ml after approximately 11 days.

[1209] Step 5: Automatic ordering decision

[1210] The server checks whether the inventory level has fallen below the set threshold. If it has, it automatically places an order.

[1211] Example: When the stock reaches 50ml, the server determines that an automatic order is required.

[1212] Step 6: Automatic Order Execution

[1213] The server calls the online shop's API to order the necessary items.

[1214] Example: The server sends a request to the "Online Shop API" to "Order one 500ml bottle of detergent."

[1215] Step 7: Confirm order and prepare notification

[1216] The server receives the response from the online shop API and confirms the order details and expected delivery date.

[1217] Example: Receive a response from the API saying "500ml of detergent, order completed, expected arrival date: 2023-10-07" and save it in the database.

[1218] Step 8: User Notification

[1219] The terminal provides the user with notification of the order details and expected arrival date from the server.

[1220] Example: A smart speaker announces, "We're running low on detergent, so we've placed an order for more. The estimated delivery date is two days later."

[1221] Step 9: User Reporting

[1222] Users verbally report stock availability and special requests to the smart speaker.

[1223] Example: A user reports, "I have a party and need more detergent."

[1224] Step 10: Report analysis and inventory adjustments

[1225] The server analyzes the user's reports and adjusts inventory forecasts and order frequency.

[1226] Example: The server changes the settings to "Order more detergent than usual due to increased detergent consumption."

[1227] Step 11: Emotion Recognition

[1228] The emotion engine recognizes emotions from the user's tone of voice and facial expressions and sends that information to the server.

[1229] Example: When a user feels stressed, the engine sends the analysis results to the server.

[1230] Step 12: Emotion-Based Processing

[1231] The server adjusts inventory management and ordering strategies based on the emotional data from the emotion engine.

[1232] Example: If users are stressed, order more detergent early to avoid running out.

[1233] This series of steps will create a system that allows users to manage inventory of daily necessities, predict consumption, and receive emotion-based assistance.

[1234] Example 2

[1235] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1236] While conventional inventory management systems for daily necessities using home appliances are capable of managing inventory levels and automatically ordering based on consumption patterns, they have the problem of being unable to flexibly respond to user emotions or special circumstances. This has resulted in insufficient improvements in user convenience and satisfaction, and has led to waste due to inventory shortages and over-ordering under certain circumstances.

[1237] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1238] In this invention, the server includes means for collecting data on items consumed by the household electrical appliances, means for analyzing the consumption data of the items and learning consumption patterns, means for estimating inventory status based on the consumption patterns, means for automatically ordering items when the estimated inventory status falls below a set threshold, means for notifying a user of the order contents and expected arrival date of the items, means for recognizing the user's emotions, and means for adjusting inventory management and ordering strategies based on the emotion information, thereby enabling effective inventory management and automatic ordering according to the user's emotions and special requests.

[1239] "Household electrical appliances" refers to home appliances used in the home, including washing machines, refrigerators, microwave ovens, etc.

[1240] "Goods" refers to consumable items such as detergents and food consumed within the home.

[1241] "Means for collecting data" refers to a device or system capable of collecting consumption data generated by a household appliance.

[1242] "Means for analyzing data and learning consumption patterns" refers to a device or system that has the function of analyzing collected consumption data and learning users' consumption patterns using machine learning algorithms or the like.

[1243] "Means for estimating inventory status" refers to a device or system capable of estimating the current inventory of an item based on consumption patterns.

[1244] "Means for automatically ordering goods" means a device or system that has the capability to automatically order goods using an online system when inventory levels fall below a set threshold.

[1245] "Means of notifying the user" refers to telecommunications means for informing the user of the order details and expected arrival date of the item, including smart speakers and smartphones.

[1246] "Means for recognizing emotions" refers to a device or system that has the function of analyzing and recognizing emotions from the user's voice and facial expressions.

[1247] "Means for adjusting inventory management and ordering strategies based on emotional information" refers to a device or system that has the function of appropriately adjusting inventory management and ordering processes based on recognized user emotions.

[1248] "Online Shop API" refers to an application programming interface that connects with online shops via the Internet to order goods and check their status.

[1249] "Means for analyzing reports and adjusting inventory estimates and ordering frequencies" refers to a device or system that has the function of analyzing reports from users and appropriately adjusting inventory estimates and ordering frequencies based on that information.

[1250] This invention relates to an inventory management and automatic ordering system for daily necessities using household electrical appliances, and in particular to a system that realizes optimal inventory management according to the user's emotional state by combining an emotion engine. This system is composed of electrical appliances installed in the home, a server, a terminal (smart speaker or app), an emotion engine, and the user.

[1251] The server receives consumption data sent from household appliances and stores it in a database. For example, if a washing machine consumes 10ml of detergent in one use, the data is recorded as "Detergent 10ml Use Date: 2023-10-05 18:45." Data is collected in real time and sent to the server via the Internet. This data is organized chronologically and stored in a database.

[1252] The server then analyzes the collected data and uses machine learning algorithms (e.g., Python's pandas or scikit-learn) to learn the user's daily consumption patterns. For example, based on one month's consumption data, the server may learn that the average daily consumption of detergent is 45ml. This allows the server to make future consumption predictions.

[1253] Furthermore, the server estimates the current inventory level based on this learning result and predicts when the inventory will fall below a set threshold. For example, if the initial inventory is 500 ml and 45 ml is consumed per day, the server estimates that the inventory will fall below 50 ml in approximately 11.11 days.

[1254] When it is determined that the inventory level is below the threshold, the server calls an online store API (such as Amazon's API or Rakuten API) and automatically places an order for the necessary items. Specifically, when inventory falls below 50ml, the server sends a request to the online store API to order one 500ml bottle of detergent. This is done using an HTTP request that includes the API key and order information.

[1255] Information about the order contents and expected arrival date from the server is notified to the user via a device (smart speaker or app). For example, a notification message such as "We've placed an additional order for detergent because we're running low on it. The expected arrival date is two days later" is displayed. This is achieved by using the smart speaker's text-to-speech (TTS) function or the smartphone's push notification function.

[1256] Users can also use smart speakers to verbally report inventory status or special requests. For example, if a user says, "I'm going to a party and need more detergent," the voice data is converted into text data and sent to the server. The server analyzes this information and adjusts inventory estimates and ordering frequency.

[1257] Furthermore, the emotion engine analyzes emotional information from the user's voice and facial expressions and sends the results to the server. For example, if a user says "I feel very stressed" in a strong tone, the emotion engine will analyze the voice tone and facial expression and determine that the user is in a stressed state. This is done using libraries and services specialized in emotion recognition (e.g., AWS Rekognition, Google Cloud Vision).

[1258] The server adjusts inventory management and ordering strategies based on the emotion information from the emotion engine. For example, if the emotion engine determines that the user is feeling stressed, the server will order necessary items such as detergent more quickly or in larger quantities. Also, if it determines that special measures are needed, the server will adjust the order by raising the priority.

[1259] Example prompts to be input to the generative AI model

[1260] Describe the specific process for an inventory management and automated ordering system for everyday items using consumer electronics, from data collection to processing based on emotion recognition. Include the following keywords: inventory management, consumption patterns, online shop, notification, emotion recognition.

[1261] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1262] Step 1: Data collection and storage

[1263] The server receives consumption data sent from home appliances in real time and stores it in a database. Specifically, the server receives data sent from washing machines, dispensers, etc. and stores it in a chronological order in the database.

[1264] Input: Consumption data from a household appliance (e.g., 10 ml of detergent used).

[1265] Data processing: Organize received data with timestamps.

[1266] Output: Organized consumption data is stored in a database.

[1267] Specific operation: The server receives data such as "Detergent 10ml, date and time of use: 2023-10-05 18:45" and records it in the database.

[1268] Step 2: Data analysis and pattern learning

[1269] The server analyzes the consumption data stored in the database and learns the user's daily consumption patterns using machine learning algorithms (e.g., Python's pandas and scikit-learn).

[1270] Input: Consumption data stored in a database.

[1271] Data processing: Machine learning algorithms are used to analyze daily consumption patterns.

[1272] Output: Learning results about users' consumption patterns.

[1273] Specific operation: The server analyzes, for example, one month's worth of data and learns that "users consume an average of 45 ml of detergent per day."

[1274] Step 3: Estimate inventory

[1275] The server estimates the current inventory level based on the learned consumption patterns and predicts when it will fall below the threshold.

[1276] Input: Learning results of consumption patterns and current inventory.

[1277] Data processing: Uses a mathematical formula to estimate based on inventory volume and consumption rate.

[1278] Output: Prediction of when inventory will fall below threshold.

[1279] Specific operation: If the inventory is 500 ml and daily consumption is 45 ml / day, the server estimates that the inventory will fall below 50 ml in approximately 11.11 days.

[1280] Step 4: Automated ordering

[1281] If the server determines that the inventory level is below the threshold, it calls the online shop API to automatically order the item.

[1282] Input: Inventory quantity estimation result.

[1283] Data processing: Generation of order requests using online shop API.

[1284] Output: Order request and response to online shop.

[1285] Specific operation: When the stock falls below 50ml, the server sends a request to the online shop API to order one 500ml bottle of detergent.

[1286] Step 5: Notification Processing

[1287] The terminal receives notification of the order contents and expected arrival date from the server and conveys the information to the user.

[1288] Input: Order details and expected arrival date information from the server.

[1289] Data processing: generating text messages and voice notifications.

[1290] Output: Notification to the user.

[1291] Specific operation: The terminal notifies the user, "We are running low on detergent, so we have placed an additional order. The expected delivery date is two days later."

[1292] Step 6: User Report Processing

[1293] Users use smart speakers to verbally report stock availability and special requests, which are then sent to the server.

[1294] Input: User's voice report.

[1295] Data processing: Text conversion and analysis of voice data.

[1296] Output: Report to the server.

[1297] Specific operation: When a user verbally reports, "I'm having a party and need some extra detergent," the speech is converted into text and sent to the server.

[1298] Step 7: Emotion Recognition

[1299] The emotion engine analyzes emotional information from the user's voice and facial expressions and sends the results to the server.

[1300] Input: User's voice and facial expression data.

[1301] Data processing: Applying emotion recognition algorithms.

[1302] Output: User's emotional information.

[1303] Specific behavior: The emotion engine analyzes the user's tone of voice and facial expressions to recognize stress levels and other emotions.

[1304] Step 8: Emotion-Based Processing

[1305] The server adjusts inventory management and ordering strategies based on the emotional information from the emotion engine.

[1306] Input: Emotion information from the emotion engine.

[1307] Data processing: Adjusting inventory management and ordering strategies.

[1308] Output: Coordinated inventory management and ordering instructions.

[1309] Specific behavior: If the server recognizes that the user is stressed, it will order the necessary items more quickly or in larger quantities.

[1310] (Application example 2)

[1311] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1312] In modern store operations, inventory management and the quality of customer service are important issues. In particular, there is a need to prevent lost sales opportunities due to inventory shortages and to respond to customers' emotional states in a detailed manner. However, conventional systems have difficulty simultaneously solving these problems, making it difficult to achieve efficient inventory management and customer service. Therefore, there is a need for a system that combines an emotion engine to provide optimal inventory management and customer service that responds to the user's emotional state.

[1313] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting data on items consumed by household electrical appliances, means for analyzing the consumption data of the items and learning consumption patterns, means for estimating inventory status based on the consumption patterns, means for automatically ordering items when the estimated inventory status falls below a set threshold, means for notifying the user of the order details and expected arrival date of the items, and means for optimizing inventory management and ordering processes based on emotions using an emotion engine that recognizes the user's emotional state. This makes it possible to improve the efficiency of inventory management and user satisfaction.

[1314] Below are definitions of important words:

[1315] "Household appliances" are electrical devices used in ordinary households that can be used as a means of collecting consumption data.

[1316] "Goods" refers to the items that are consumed or controlled by a consumer electronics device.

[1317] "Data Collection Measure" means a device or software for periodically collecting data on goods consumed from a consumer electronics device.

[1318] "Data analysis means" refers to a device or software that analyzes and learns consumption patterns based on collected consumption data.

[1319] The "consumption pattern" indicates the user's tendency to consume items on a daily basis, and is learned by the data analysis means.

[1320] An "inventory estimating means" is a device or software for estimating current inventory levels based on consumption patterns and assessing inventory status.

[1321] An "automatic ordering means" is a device or software for automatically ordering items through an online shop when estimated inventory falls below a set threshold.

[1322] The "notification means" is a device or software for notifying the user of the ordered item and the expected arrival date.

[1323] An "emotion engine" is a device or software that recognizes a user's emotional state from their voice and facial expressions and performs processing based on the emotional information.

[1324] A "server" is a computer system or service that coordinates a series of processes such as data collection, analysis, inventory estimation, automatic ordering, and notification.

[1325] A "smartphone" is a mobile communication terminal that has advanced processing capabilities in addition to the functions of a mobile phone and can run applications.

[1326] "Smart glasses" are glasses-shaped devices that can display information and collect data.

[1327] "Emotion information" is data indicating the emotional state recognized by the emotion engine from the user's voice and facial expression.

[1328] "Customer service actions" refer to the service content and response policy provided depending on the user's emotional state.

[1329] This invention relates to an inventory management and automatic ordering system for daily necessities using household electrical appliances, and in particular to a system that realizes optimal inventory management according to the user's emotional state by combining an emotion engine. This system is composed of electrical appliances installed in the home, a server, terminals (smartphones or smart glasses), the emotion engine, and the user.

[1330] System Overview

[1331] 1. Data Collection and Storage:

[1332] The server receives consumption data sent from the home appliances and stores it in a database in chronological order. For example, a detergent dispenser sends the amount of detergent consumed to the server and records it in the database.

[1333] 2. Data analysis and pattern learning:

[1334] The server analyzes the received consumption data and learns the user's daily consumption patterns. For example, it analyzes consumption amounts on a monthly basis to understand daily consumption trends.

[1335] 3. Inventory Estimation:

[1336] The server estimates current inventory levels based on learned consumption patterns and predicts when inventory levels will fall below a set threshold, allowing for timely replenishment.

[1337] 4. Automated ordering:

[1338] When the server determines that the inventory level is below the set threshold, it calls the online shop API to automatically order the necessary items, thereby preventing stockouts.

[1339] 5. Notification Processing:

[1340] The terminal notifies the user of the order details and expected arrival date from the server, and the user can receive this information via their smartphone or smart glasses.

[1341] 6. User Reporting Process:

[1342] Users verbally report stock availability and special requests via their smartphones or smart glasses, which are then sent to the server and reflected in inventory estimates and order frequency.

[1343] 7. Emotion recognition:

[1344] The emotion engine recognizes emotions from the user's voice and facial expressions and sends that emotional information to the server. For example, if the user is under stress, the emotion engine analyzes that information in real time and notifies the server.

[1345] 8. Emotion-based processing:

[1346] The server adjusts inventory management and ordering strategies based on the emotion information from the emotion engine. For example, if a user is feeling stressed, it can improve user satisfaction by quickly replenishing necessary items.

[1347] Hardware and Software Use Cases

[1348] Hardware:

[1349] Smartphone: A device that allows users to check inventory status and send emotional information.

[1350] Smart glasses: A device that allows store staff to check inventory information and how to respond based on customer sentiment.

[1351] Consumer electronics: Devices for collecting consumption data and sending it to a server.

[1352] software:

[1353] Server: A platform that oversees data collection, analysis, inventory estimation, automatic ordering, and notifications.

[1354] Emotion engine: Software for recognizing the user's emotional state.

[1355] Consumption data analysis software: Tools for learning consumption patterns and estimating inventory status.

[1356] Specific examples

[1357] Here is a specific example of managing a user's daily detergent consumption. For example, if 45 ml of household detergent is consumed daily and the initial inventory is 500 ml, the washing machine sends the amount of detergent consumed each time to the server. The server analyzes past data and learns that the user consumes approximately 315 ml of detergent per week. If the user's inventory is 500 ml, the server predicts that the inventory will fall below 100 ml in approximately 11.11 days and orders additional detergent from the online shop at the appropriate time. Furthermore, if the user verbally reports that they need more detergent because they have a party next week, the server adjusts the order amount based on that information. If the emotion engine recognizes the user's stress level, the server responds more quickly and places an order immediately to reassure the user.

[1358] Prompt Sentence Examples

[1359] "Please tell us about the structure and process of an app for inventory management and improving customer service in a physical store. Please include specific examples, such as a case where you recognized customer emotions and linked them to an inventory management system."

[1360] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1361] Step 1:

[1362] Data collection

[1363] The server collects data on the items consumed by household electrical appliances (e.g., washing machines and detergent dispensers) in real time. The input at this time is consumption data sent from each household electrical appliance, specifically including the amount consumed (e.g., 10 ml of detergent) and the date and time of consumption. The server receives this data and stores it in a database in chronological order. The information inserted into the database consists of "item name, amount consumed, and date and time of consumption."

[1364] Step 2:

[1365] Data analysis

[1366] The server analyzes the collected consumption data and learns the user's daily consumption patterns. The input is past consumption data stored in a database, and the output is consumption patterns (e.g., average daily consumption). The server uses data analysis software to statistically analyze the past consumption data and calculate the consumption patterns of the user's goods. Specifically, it calculates consumption trends by day, week, and month.

[1367] Step 3:

[1368] Inventory Estimation

[1369] The server estimates the current inventory level based on the learned consumption patterns. The input is the current inventory level and consumption pattern data, and the output is a prediction of when the inventory will fall below a set threshold. The server uses an inventory estimation algorithm to calculate how much inventory is currently remaining and predict when the inventory will run out. Specifically, the server estimates when the inventory will run out by subtracting the consumption pattern from the current inventory level.

[1370] Step 4:

[1371] Automatic ordering

[1372] If the server determines that the inventory level is below a set threshold, it calls the online shop API to automatically order the required items. The input is the inventory estimation result (when inventory will run out), and the output is the result of the order request sent. The server sends an order request through the online shop API to instruct the replenishment of the required items. Specifically, it sends a request to the API including the product ID and order quantity, and checks the response.

[1373] Step 5:

[1374] Notification Processing

[1375] The server notifies the user of the order details and expected arrival date. The input is the response data from the online shop API (order details and expected arrival date), and the output is a notification message. The server uses notification generation software to create a notification message for the user and sends it to the user via their smartphone or smart glasses. Specifically, a notification including the product name, order quantity, and expected arrival date is pushed to the user's device.

[1376] Step 6:

[1377] User Report Processing

[1378] Users verbally report inventory status or special requests via smartphones or smart glasses. The input is voice input from the user, and the output is a request converted into text. The device uses voice recognition software to analyze the voice input, convert it into text data, and send it to the server. Specifically, a voice request such as "We need to order more for next week's party" can be converted into text and sent to the server.

[1379] Step 7:

[1380] emotion recognition

[1381] The emotion engine recognizes emotions from the user's voice and facial expressions and sends that emotional information to the server. The input is the user's voice and facial expression data, and the output is analyzed emotional information. The device uses the emotion engine to analyze emotions from the voice and facial expressions and sends the emotional state (e.g., stress, joy, etc.) to the server. Specifically, changes in voice tone and facial expressions are detected in real time and recorded as the emotional state.

[1382] Step 8:

[1383] Emotion-Based Processing

[1384] The server adjusts inventory management and ordering strategies based on the emotion information from the emotion engine. The input is the emotion recognition results, and the output is adjusted ordering instructions and inventory management strategies. The server analyzes the emotion recognition results, and if the user is feeling stressed, for example, it can improve user satisfaction by quickly ordering the necessary items. Specifically, it can take measures such as setting an urgent order flag and placing an order immediately.

[1385] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1386] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1387] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1388] [Fourth embodiment]

[1389] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1390] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1391] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1392] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1393] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1394] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1395] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1396] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1397] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1398] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1399] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1400] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1401] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1402] The present invention relates to a system for managing inventory and automatically ordering daily necessities in the home. The following describes an embodiment of the invention. This system is composed of various elements, including home appliances, a server, terminals (smart speakers and apps), and users.

[1403] System Overview

[1404] In this system, data on the items consumed by household appliances is collected and sent to a server. The server analyzes this data and learns consumption patterns. Based on the learned consumption patterns, the server estimates the inventory status of each item and automatically orders items when inventory falls below a set threshold. The order details and expected arrival date are notified to the user via their terminal, and the user can also verbally report inventory status or special requests. The user's reports are also analyzed by the server and reflected in inventory forecasts and adjustments to ordering frequency.

[1405] Program processing overview

[1406] 1. Data Collection:

[1407] The server receives consumption data sent from the home appliances, including the type of item used, the amount used, and the date and time of use.

[1408] Example: Data is sent to the server that a detergent dispenser has used 10ml of detergent.

[1409] 2. Data analysis and learning:

[1410] The server analyzes the received consumption data and learns the user's consumption patterns, which allows it to predict future consumption based on past consumption data.

[1411] Example: Learning a pattern that the average amount of detergent used per day over a week is 45ml.

[1412] 3. Inventory Estimation:

[1413] The server estimates current inventory levels and predicts when inventory will fall below a set threshold based on consumption patterns.

[1414] Example: It is estimated that a 500ml stock will fall below the planned consumption level in 20 days.

[1415] 4. Automated ordering:

[1416] When the amount falls below the set threshold, the server automatically orders items from the online shop, using the online shop's API to place the order.

[1417] Example: When you run out of detergent below 50ml, order a 500ml bottle of detergent from an online shop.

[1418] 5. User Notice:

[1419] The terminal will notify the user of the order details and estimated arrival date, and the user can receive notifications via their smartphone or smart speaker.

[1420] For example, the device will announce, "We're low on detergent, so we've placed an order for more. It's expected to arrive in two days."

[1421] 6. User Reports:

[1422] Users can verbally report stock availability and special requests, and this information is sent to the server via the smart speaker and used to analyze consumption patterns and estimate inventory.

[1423] Example: A user reports, "I have a party and need some extra detergent."

[1424] Specific examples

[1425] As an example, consider a household where users do laundry every day and consume 30ml of laundry detergent per day. In this case, the washing machine (a household appliance) sends the amount of detergent used per load to a server every day. The server analyzes past data and learns that the user consumes approximately 210ml of detergent per week. If the user has 500ml in stock, the server predicts that the stock will run out in approximately 11.67 days and automatically orders more detergent before it falls below a threshold (e.g., 100ml). The user is notified of the order and the expected arrival date, and can report any special requests via a smart speaker. Based on this information, the server adjusts inventory management and ordering frequency.

[1426] In this way, users can save time and effort in inventory management and shopping, and receive the items they need at the right time.

[1427] The processing flow will be explained below.

[1428] Step 1: Data collection

[1429] The server receives consumption data sent from the home appliances, including the type of item used, the amount used, and the date and time of use.

[1430] Example: Data that a detergent dispenser used 10ml of detergent is sent to the server, and the server records it as "Detergent 10ml Used Date and Time: 2023-10-05 18:45".

[1431] Step 2: Save data

[1432] The server stores the received consumption data in a database in chronological order.

[1433] Example: Store information in a database in the format "Product: Detergent, Amount Used: 10ml, Date and Time: 2023-10-05 18:45".

[1434] Step 3: Data analysis

[1435] The server executes data analysis tasks at regular intervals, learning and analyzing users' consumption patterns based on past consumption data.

[1436] Example: Data for one month is collected and the average consumption is calculated. The analysis results show that an average of 45ml of detergent is consumed per day.

[1437] Step 4: Inventory estimation

[1438] The server estimates the current inventory level based on the analysis results and predicts when the inventory will fall below a set threshold based on consumption patterns.

[1439] Example: If the initial inventory is 500ml and 45ml is consumed per day, it is predicted that the inventory will run out in 500ml ÷ 45ml / day ≈ 11.11 days.

[1440] Step 5: Automatic ordering decision

[1441] The server checks whether the estimated inventory level falls below a set threshold, and if so, automatically places an order.

[1442] Example: If the threshold is set to 100ml, an auto-order flag will be set when the estimated stock level falls below 100ml.

[1443] Step 6: Automatic Order Execution

[1444] The server calls the online shop's API to order the necessary items.

[1445] Example: The server sends a request to the "Online Shop API" to "Order one 500ml bottle of detergent."

[1446] Step 7: Confirm order and prepare notification

[1447] The server receives the response from the online shop API and confirms the order details and expected delivery date.

[1448] Example: Receive a response from the API saying "500ml detergent, order completed, expected arrival date: 2023-10-07" and save the data.

[1449] Step 8: User Notification

[1450] The device will notify the user of the order details and estimated arrival date, and provide information via a smart speaker or app.

[1451] Example: A smart speaker notifies the user by voice, "We're running low on detergent, so we've placed an order for more. The expected delivery date is in two days."

[1452] Step 9: User Reporting

[1453] Users report stock availability and special requests through their smart speakers.

[1454] Example: A user reports, "I have a party and need some extra detergent."

[1455] Step 10: Report analysis and inventory adjustments

[1456] The server analyzes user reports and adjusts consumption forecasts, inventory estimates, and order frequency.

[1457] Example: The server receives a report of a user's "party" and predicts that consumption will increase next week, so it orders more to cover that increase.

[1458] This series of steps creates a system that allows users to conveniently manage inventory and automatically order daily necessities.

[1459] Example 1

[1460] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1461] Managing the inventory of everyday items in the home is a complicated and tedious task for many people. It is difficult to predict when items will run out and to purchase them appropriately, and there is a high possibility of inconvenience when they run out. Furthermore, flexible and accurate inventory management is required to respond to users' individual consumption patterns and special requests. However, existing systems are unable to adequately resolve these issues.

[1462] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1463] In this invention, the server includes: means for collecting data on items consumed by household electrical appliances; means for analyzing consumption data of the items and learning consumption patterns; means for estimating inventory status based on the consumption patterns; means for automatically ordering items when the estimated inventory status falls below a set threshold; means for notifying the user of the order details and expected arrival date of the items; and means for the user to verbally report the inventory status and special requests, and for analyzing the report to adjust the order frequency and inventory management. This allows the user to automatically manage the inventory status of daily necessities and automatically order necessary items at the appropriate time, significantly reducing the effort required for inventory management.

[1464] "Household appliances" refers to all electrically powered devices used in the home, including those equipped with sensors that collect consumption data.

[1465] "Goods" refers to consumables and daily necessities consumed by household electrical appliances, specifically including detergents, paper products, food ingredients, etc.

[1466] "Data collection means" refers to the functions and methods for recording consumption data generated when a household electrical appliance is used and transmitting it to a server.

[1467] "Data analysis means" refers to software or algorithms for processing received consumption data and analyzing user consumption patterns.

[1468] "Consumption pattern" is information that indicates the tendency of how a user consumes items within a specific period of time based on past consumption data.

[1469] "Inventory estimation means" refers to a function that uses consumption data and estimation algorithms to calculate current inventory levels and predict when inventory will run out in the future.

[1470] A "threshold" is a reference value set during inventory management, which becomes a trigger point for automatically ordering goods when the value falls below this value.

[1471] "Automatic ordering method" refers to a function or method for automatically ordering necessary items through the online shop's API when inventory falls below a threshold.

[1472] The "notification means" is a means for informing users of information such as order details and the expected arrival date of the item, and is implemented using a smartphone or smart speaker.

[1473] "User reporting means" refers to a device such as a smart speaker that allows a user to verbally report stock status or special requests, and the server receives the report.

[1474] An "online shop API" is a program interface provided by an online shopping platform that is used to order and confirm goods.

[1475] A "machine learning algorithm" refers to a mathematical model or computational procedure that learns patterns and knowledge from data and makes predictions and inferences.

[1476] The present invention is a system for efficiently managing inventory and automatically ordering consumer goods in the home. Specific embodiments of the system will be described in detail below.

[1477] This system consists of various elements, including household appliances, servers, terminals (smart speakers and apps), and users.

[1478] System configuration

[1479] 1. Household appliances:

[1480] Each consumer electronic device is fitted with a sensor that collects data whenever the consumer item is used, and this data is sent to a server via Wi-Fi or other communication means.

[1481] Examples include washing machines and detergent dispensers, which send data such as the type and amount of items used and the date and time of use to a server.

[1482] 2. Server:

[1483] The server analyzes the received data and uses machine learning algorithms, including ridge regression and other statistical models, to learn the user's consumption patterns.

[1484] The server uses programming languages ​​such as Python to preprocess and analyze the data, specifically performing tasks such as filling in missing values, detecting outliers, and normalizing the data.

[1485] 3. Devices (smart speakers and apps):

[1486] The device has the ability to notify users of the order details and expected arrival date, and conveys this information to them via push notifications on their smartphones or voice notifications on their smart speakers.

[1487] These devices allow users to report stock availability and special requests by voice, which is converted to text and sent to the server.

[1488] Specific processing examples

[1489] 1. Data Collection:

[1490] Example: If the washing machine uses 30ml of detergent in one wash, it sends that information to the server.

[1491] 2. Data analysis and learning:

[1492] Example: Based on data from the past week, the server learns that an average of 45ml of detergent is consumed per day.

[1493] 3. Inventory Estimation:

[1494] Example: There is currently 500ml in stock, and the server estimates that in 20 days the stock will fall below the threshold.

[1495] 4. Automated ordering:

[1496] Example: When stock falls below a threshold, the server orders 500ml of detergent through an online shop API.

[1497] 5. User Notice:

[1498] Example: "We're low on detergent so we've placed an order. It's expected to arrive in two days," your smart speaker will announce.

[1499] 6. User Reports:

[1500] Example: A user reports, "I'm having a party and need some extra detergent," and the smart speaker sends that information to the server.

[1501] This system allows users to eliminate the hassle of daily inventory management and automatically receive the items they need at the right time. Specific examples of prompts include "Tell me what's in stock today," "Tell me when the next shipment of detergent is expected to arrive," and "Order more detergent for a party."

[1502] The system provided by this invention can thoroughly automate the management of household items and improve user convenience.

[1503] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1504] Step 1: Data collection

[1505] 1. Input:

[1506] Consumption data generated by household appliances (e.g., type of item used, amount used, date and time of use).

[1507] 2. Processing:

[1508] Household appliances use sensors to measure and collect consumption data, for example, if a washing machine uses 30ml of detergent, it will measure that data.

[1509] 3. Data Processing:

[1510] The household appliances format the measured data and send it to a server via Wi-Fi.

[1511] 4. Output:

[1512] Formatted consumption data sent to the server.

[1513] 5. Specific operation:

[1514] When the washing machine starts washing and 30ml of detergent is used, the sensor detects the amount used, compiles the data and sends it to the server.

[1515] Step 2: Data analysis and training

[1516] 1. Input:

[1517] Consumption data of household appliances sent to a server.

[1518] 2. Processing:

[1519] The server preprocesses the received data, performing missing value imputation, outlier detection, and data normalization.

[1520] 3. Data Processing:

[1521] The server inputs the organized data into a machine learning algorithm (e.g., ridge regression) to learn the user's consumption patterns.

[1522] 4. Output:

[1523] A model that learns consumption patterns.

[1524] 5. Specific operation:

[1525] The server analyzes consumption data from the past week and learns that "users consume an average of 45ml of detergent per day."

[1526] Step 3: Estimate inventory

[1527] 1. Input:

[1528] Learned consumption pattern model and current inventory levels.

[1529] 2. Processing:

[1530] The server accumulates the initial inventory and consumption data to estimate the current inventory amount.

[1531] 3. Data Calculation:

[1532] Based on consumption patterns, the server predicts when inventory will fall below a set threshold.

[1533] 4. Output:

[1534] The point at which it is estimated to fall below the threshold.

[1535] 5. Specific operation:

[1536] The server calculates that the current detergent inventory of 500ml will fall below consumption in 20 days.

[1537] Step 4: Automated ordering

[1538] 1. Input:

[1539] Prediction of when inventory will fall below a threshold.

[1540] 2. Processing:

[1541] When the amount falls below the set threshold, the server calls the online shop API to order the item.

[1542] 3. Data Calculation:

[1543] The server sends the order details to the online shop using an HTTP POST request.

[1544] 4. Output:

[1545] Order requests sent to the online shop.

[1546] 5. Specific operation:

[1547] The server uses the API to send a request to the online shop to "order one 500ml bottle of detergent."

[1548] Step 5: User Notification

[1549] 1. Input:

[1550] Notification data that the order has been completed.

[1551] 2. Processing:

[1552] The terminal receives the notification data from the server and notifies the user by display or voice.

[1553] 3. Data Calculation:

[1554] The terminal converts the notification content into an appropriate format and transmits it to the user.

[1555] 4. Output:

[1556] The order details and expected delivery date notified to the user.

[1557] 5. Specific operation:

[1558] The smart speaker will announce, "We're running low on detergent, so we've placed an order for more. The expected delivery date is two days from now."

[1559] Step 6: User Reporting

[1560] 1. Input:

[1561] User voice reporting.

[1562] 2. Processing:

[1563] The device records the audio data and converts it into text.

[1564] 3. Data Processing:

[1565] The terminal transmits the converted text data to the server.

[1566] 4. Output:

[1567] The user's reported text data sent to the server.

[1568] 5. Specific operation:

[1569] The user reports, "I'm having a party so I need some extra detergent," and the smart speaker sends that information to the server.

[1570] The above is the specific processing flow of this system. The specific input, processing, and output are clearly defined at each step, and it is designed to enable users to quickly and efficiently manage their home inventory.

[1571] (Application example 1)

[1572] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1573] In modern households, inventory management of daily consumed goods and food is time-consuming and difficult to replenish at the appropriate time. In particular, busy households and households with elderly people are prone to stock shortages, which increases the effort required for shopping to address these shortages. Furthermore, there is a need for an inventory management method that can flexibly respond to unexpected events and changes in plans. To solve these problems, a system is needed that automatically manages inventory and replenishes necessary goods and food at the appropriate time.

[1574] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1575] In this invention, the server includes: means for collecting food consumption data using sensors in household appliances; means for analyzing the collected food consumption data and learning a user's consumption patterns; means for estimating inventory status based on the consumption patterns; means for automatically ordering food from a food delivery service when the estimated inventory status falls below a preset threshold; means for notifying the user of the food order details and expected arrival date via the food delivery service's API; means for the user to verbally report the inventory status and special requests; means for analyzing the user's report and adjusting the inventory estimate and ordering frequency; means for collecting food data using sensors installed in a household refrigerator or smart microwave; means for learning the consumption pattern based on the collected food data and estimating the food inventory status; means for ordering items using an online shop API; means for confirming the order details and expected arrival date based on a response from the API; means for placing an order using the food delivery service API; and means for notifying the user of the order details and expected arrival date. This reduces the user's effort and enables automatic replenishment of necessary items and foods at the appropriate time.

[1576] "Household electrical appliances" is a general term for products that operate using electricity and are used in the home.

[1577] "Consumption data" refers to information such as the type of items used by a household electrical appliance, the amount used, and the date and time of use.

[1578] "Consumption patterns" are data that indicate trends in the frequency and amount of goods consumed based on past consumption data.

[1579] "Inventory status" refers to the current stock of items in the home.

[1580] The "threshold" is a reference value below which inventory is automatically deemed to need replenishment.

[1581] "Food delivery service" refers to a service that delivers ingredients and food to a designated location.

[1582] An "API" is an interface for exchanging data with other programs or services.

[1583] "User notification" refers to informing the user of information such as the order contents and expected arrival date.

[1584] "Verbal reporting" refers to the act of a user reporting stock availability or special requests using voice.

[1585] "Analysis" refers to processing collected data to extract useful information.

[1586] "Ordering" refers to the act of ordering necessary goods or food.

[1587] "Estimated Delivery Date" means the date on which the ordered items are expected to arrive at the specified location.

[1588] The present invention provides a system for managing inventory and automatically ordering daily necessities in the home. This system is composed of various elements, including home appliances, a server, terminals (e.g., smart speakers and smartphone apps), and users.

[1589] The program for realizing this system consists of the following steps:

[1590] Data collection

[1591] The server collects food consumption data using sensors built into household appliances (e.g., refrigerators and smart microwaves). The collected data includes the type of food consumed, the amount consumed, and the date and time of consumption. This data is transmitted to the server via Wi-Fi.

[1592] Data Analysis and Learning

[1593] The server analyzes the received food consumption data and learns the user's consumption patterns. This analysis is performed using data analysis libraries such as pandas and numpy. Based on past data, the server predicts how often and how much food the user will consume.

[1594] Inventory Estimation

[1595] The server estimates the household food inventory based on consumption patterns. When the estimated inventory falls below a set threshold, it automatically orders food from a food delivery service. This order uses the food delivery service's API to confirm the order details and expected arrival date.

[1596] Automatic ordering

[1597] The server automatically orders food from a food delivery service when the ingredients fall below a threshold. This ordering process uses an API interface, allowing the order details to be confirmed. The required food is ordered via the food delivery service's API.

[1598] User Notifications

[1599] The terminal notifies the user of the order details and expected arrival date. The notification is made via voice using a smart speaker API. The user can receive the information via their smartphone or smart speaker.

[1600] User Reports

[1601] Users can verbally report stock availability and special requests, and this information is sent to the server via the smart speaker, which then uses it to manage inventory and adjust ordering frequency.

[1602] Specific examples

[1603] For example, suppose a home refrigerator contains a variety of vegetables, and a sensor measures daily consumption. The server analyzes this data and learns that an average person consumes 300g of vegetables per week. If inventory falls below a 100g threshold, it automatically orders 500g of vegetables from a food delivery service. At this time, the user is notified that "We're running low on vegetables, so we've ordered an additional 500g. The estimated delivery date is tomorrow." If the user also tells the smart speaker, "A friend is coming over this weekend, so please order some more fruit," this information will be reflected in the next order.

[1604] Example prompts to be input to the generative AI model

[1605] Calculate the average vegetable consumption for a week from the vegetable inventory management data in the refrigerator, and determine whether the inventory is below the set threshold (100g). If it falls below 100g, automatically order 500g of vegetables via a food delivery service and notify the user.

[1606] This system allows users to significantly reduce the hassle of daily inventory management and shopping, and allows them to receive the items and food they need at the right time.

[1607] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1608] Step 1:

[1609] The server collects food consumption data through sensors built into household appliances (e.g., refrigerators and smart microwaves). The data measured by the sensors includes the type of food consumed, the amount consumed, and the date and time of consumption. This data is transmitted to the server via Wi-Fi. The input is raw data from the sensors, and the output is data stored on the server for analysis.

[1610] Step 2:

[1611] The server analyzes the collected food consumption data and learns the user's consumption patterns. It uses data analysis libraries such as pandas and numpy to predict how often and how much food the user will consume based on past data. The input is the consumption data collected in step 1, and the output is the consumption pattern as a result of the analysis.

[1612] Step 3:

[1613] The server estimates the household food inventory status based on the learned consumption patterns. Based on the current inventory and consumption patterns, it predicts when the inventory will fall below a threshold. The inputs are the consumption patterns and the current inventory, and the output is the estimated inventory status.

[1614] Step 4:

[1615] The server automatically orders food from a food delivery service when the estimated inventory falls below a set threshold. The required food is ordered through the food delivery service's API. The input is the condition that inventory has fallen below the threshold and the order details, and the output is an order confirmation sent to the food delivery service.

[1616] Step 5:

[1617] The terminal notifies the user of the order details and expected arrival date. A smart speaker API is used to notify the user by voice or via a smartphone app. The input is the order details and expected arrival date, and the output is the notification information sent to the user.

[1618] Step 6:

[1619] Users verbally report stock status and special requests. This information is sent to the server via a smart speaker and used to estimate the next inventory and adjust ordering frequency. The input is the user's voice report, and the output is the report data stored on the server.

[1620] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1621] This paper describes an inventory management and automatic ordering system for daily necessities using household electrical appliances, and in particular a system that realizes optimal inventory management according to the user's emotional state by combining an emotion engine. This system is composed of electrical appliances installed in the home, a server, a terminal (smart speaker or app), an emotion engine, and the user.

[1622] System Overview

[1623] In this system, data on the goods consumed by household electrical appliances is collected on a daily basis and sent to a server. The server analyzes the received data and learns consumption patterns. Based on the learned consumption patterns, inventory status is estimated, and when inventory falls below a set threshold, goods are automatically ordered from an online shop. The order details and expected arrival date are notified to the user via their terminal. In addition, users can verbally report inventory status or special requests, which are also processed on the server side. In addition, an emotion engine is used to recognize user emotions and optimize inventory management and ordering processes based on emotional information.

[1624] Program processing overview

[1625] 1. Data Collection and Storage:

[1626] The server receives consumption data sent from the home appliances and stores it in a database in chronological order.

[1627] Example: A detergent dispenser receives data that 10ml of detergent has been used and records it as "Detergent 10ml Used Date: 2023-10-05 18:45".

[1628] 2. Data analysis and pattern learning:

[1629] The server analyzes the received consumption data and learns the user's daily consumption patterns, and then predicts future consumption based on the results of this learning.

[1630] Example: After analyzing data for one month, we learn that the average daily consumption of detergent is 45ml.

[1631] 3. Inventory Estimation:

[1632] Based on the learned consumption patterns, the server estimates the current inventory level and predicts when the inventory level will fall below a set threshold.

[1633] Example: If the inventory is 500ml and 45ml is consumed per day, it is estimated that the inventory will run out in approximately 11.11 days.

[1634] 4. Automated ordering:

[1635] If the server determines that the inventory level is below the set threshold, it calls the online shop API to automatically order the necessary items.

[1636] Example: When the warehouse stock drops below 50ml, a request is sent to the online shop API to order one 500ml bottle of detergent.

[1637] 5. Notification Processing:

[1638] The terminal receives notification of the order details and expected arrival date from the server and notifies the user of that information.

[1639] For example, "We're low on detergent so we've placed an order for more. It's expected to arrive in two days."

[1640] 6. User Reporting Process:

[1641] Users verbally report stock availability and special requests through their smart speakers, and the information is sent to the server.

[1642] Example: A user reports, "I have a party and need some extra detergent."

[1643] 7. Emotion recognition:

[1644] The emotion engine recognizes emotions from the user's voice and facial expressions and transmits the emotion information to the server.

[1645] Example: Determine if the user is under stress based on their tone of voice and facial expression analysis.

[1646] 8. Emotion-based processing:

[1647] The server adjusts inventory management and ordering strategies based on the emotional information from the emotion engine.

[1648] Example: If a user is stressed, order needed items sooner or in larger quantities.

[1649] Specific examples

[1650] Here is a specific example of managing a user's daily detergent consumption. Assume that a user consumes 45 ml of household detergent per day, with an initial inventory of 500 ml. In this case, the washing machine (a household appliance) sends the amount of detergent consumed per use to the server. The server analyzes past data and learns that the user consumes approximately 315 ml of detergent per week. If the user's inventory is 500 ml, the server predicts that inventory will fall below 100 ml in approximately 11.11 days and places an order for additional detergent from an online shop at the appropriate time. The user is notified of the order details and expected arrival date. Furthermore, if the user verbally reports, "I need more detergent because I have a party next week," the server adjusts the order amount based on that information. If the emotion engine recognizes the user's stress level, the server can respond even more quickly and place an order immediately to reassure the user.

[1651] The processing flow will be explained below.

[1652] Step 1: Data collection

[1653] The server receives consumption data of items from the home appliances, including the type of item, the amount used, and the date and time of use.

[1654] Example: A washing machine sends data that 30ml of detergent has been used to the server, and the server records it as "30ml of detergent used: 2023-10-05 07:30".

[1655] Step 2: Save data

[1656] The server stores the received consumption data in a database in chronological order.

[1657] Example: Store information in a database in the format "Product: Detergent, Amount Used: 30ml, Date and Time: 2023-10-05 07:30".

[1658] Step 3: Data analysis and pattern learning

[1659] The server periodically executes a data analysis task to analyze consumption data and learn user consumption patterns.

[1660] Example: Based on one month's data, we calculate that the average daily detergent consumption is 45ml.

[1661] Step 4: Inventory estimation

[1662] The server estimates the current inventory level based on the learning results and predicts when the inventory will fall below a set threshold.

[1663] Example: Initial inventory is 500ml, consumption is 45ml per day, and it is predicted that inventory will be 50ml after approximately 11 days.

[1664] Step 5: Automatic ordering decision

[1665] The server checks whether the inventory level has fallen below the set threshold. If it has, it automatically places an order.

[1666] Example: When the stock reaches 50ml, the server determines that an automatic order is required.

[1667] Step 6: Automatic Order Execution

[1668] The server calls the online shop's API to order the necessary items.

[1669] Example: The server sends a request to the "Online Shop API" to "Order one 500ml bottle of detergent."

[1670] Step 7: Confirm order and prepare notification

[1671] The server receives the response from the online shop API and confirms the order details and expected delivery date.

[1672] Example: Receive a response from the API saying "500ml of detergent, order completed, expected arrival date: 2023-10-07" and save it in the database.

[1673] Step 8: User Notification

[1674] The terminal provides the user with notification of the order details and expected arrival date from the server.

[1675] Example: A smart speaker announces, "We're running low on detergent, so we've placed an order for more. The estimated delivery date is two days later."

[1676] Step 9: User Reporting

[1677] Users verbally report stock availability and special requests to the smart speaker.

[1678] Example: A user reports, "I have a party and need more detergent."

[1679] Step 10: Report analysis and inventory adjustments

[1680] The server analyzes the user's reports and adjusts inventory forecasts and order frequency.

[1681] Example: The server changes the settings to "Order more detergent than usual due to increased detergent consumption."

[1682] Step 11: Emotion Recognition

[1683] The emotion engine recognizes emotions from the user's tone of voice and facial expressions and sends that information to the server.

[1684] Example: When a user feels stressed, the engine sends the analysis results to the server.

[1685] Step 12: Emotion-Based Processing

[1686] The server adjusts inventory management and ordering strategies based on the emotional data from the emotion engine.

[1687] Example: If users are stressed, order more detergent early to avoid running out.

[1688] This series of steps will create a system that allows users to manage inventory of daily necessities, predict consumption, and receive emotion-based assistance.

[1689] Example 2

[1690] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1691] While conventional inventory management systems for daily necessities using home appliances are capable of managing inventory levels and automatically ordering based on consumption patterns, they have the problem of being unable to flexibly respond to user emotions or special circumstances. This has resulted in insufficient improvements in user convenience and satisfaction, and has led to waste due to inventory shortages and over-ordering under certain circumstances.

[1692] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1693] In this invention, the server includes means for collecting data on items consumed by the household electrical appliances, means for analyzing the consumption data of the items and learning consumption patterns, means for estimating inventory status based on the consumption patterns, means for automatically ordering items when the estimated inventory status falls below a set threshold, means for notifying a user of the order contents and expected arrival date of the items, means for recognizing the user's emotions, and means for adjusting inventory management and ordering strategies based on the emotion information, thereby enabling effective inventory management and automatic ordering according to the user's emotions and special requests.

[1694] "Household electrical appliances" refers to home appliances used in the home, including washing machines, refrigerators, microwave ovens, etc.

[1695] "Goods" refers to consumable items such as detergents and food consumed within the home.

[1696] "Means for collecting data" refers to a device or system capable of collecting consumption data generated by a household appliance.

[1697] "Means for analyzing data and learning consumption patterns" refers to a device or system that has the function of analyzing collected consumption data and learning users' consumption patterns using machine learning algorithms or the like.

[1698] "Means for estimating inventory status" refers to a device or system capable of estimating the current inventory of an item based on consumption patterns.

[1699] "Means for automatically ordering goods" means a device or system that has the capability to automatically order goods using an online system when inventory levels fall below a set threshold.

[1700] "Means of notifying the user" refers to telecommunications means for informing the user of the order details and expected arrival date of the item, including smart speakers and smartphones.

[1701] "Means for recognizing emotions" refers to a device or system that has the function of analyzing and recognizing emotions from the user's voice and facial expressions.

[1702] "Means for adjusting inventory management and ordering strategies based on emotional information" refers to a device or system that has the function of appropriately adjusting inventory management and ordering processes based on recognized user emotions.

[1703] "Online Shop API" refers to an application programming interface that connects with online shops via the Internet to order goods and check their status.

[1704] "Means for analyzing reports and adjusting inventory estimates and ordering frequencies" refers to a device or system that has the function of analyzing reports from users and appropriately adjusting inventory estimates and ordering frequencies based on that information.

[1705] This invention relates to an inventory management and automatic ordering system for daily necessities using household electrical appliances, and in particular to a system that realizes optimal inventory management according to the user's emotional state by combining an emotion engine. This system is composed of electrical appliances installed in the home, a server, a terminal (smart speaker or app), an emotion engine, and the user.

[1706] The server receives consumption data sent from household appliances and stores it in a database. For example, if a washing machine consumes 10ml of detergent in one use, the data is recorded as "Detergent 10ml Use Date: 2023-10-05 18:45." Data is collected in real time and sent to the server via the Internet. This data is organized chronologically and stored in a database.

[1707] The server then analyzes the collected data and uses machine learning algorithms (e.g., Python's pandas or scikit-learn) to learn the user's daily consumption patterns. For example, based on one month's consumption data, the server may learn that the average daily consumption of detergent is 45ml. This allows the server to make future consumption predictions.

[1708] Furthermore, the server estimates the current inventory level based on this learning result and predicts when the inventory will fall below a set threshold. For example, if the initial inventory is 500 ml and 45 ml is consumed per day, the server estimates that the inventory will fall below 50 ml in approximately 11.11 days.

[1709] When it is determined that the inventory level is below the threshold, the server calls an online store API (such as Amazon's API or Rakuten API) and automatically places an order for the necessary items. Specifically, when inventory falls below 50ml, the server sends a request to the online store API to order one 500ml bottle of detergent. This is done using an HTTP request that includes the API key and order information.

[1710] Information about the order contents and expected arrival date from the server is notified to the user via a device (smart speaker or app). For example, a notification message such as "We've placed an additional order for detergent because we're running low on it. The expected arrival date is two days later" is displayed. This is achieved by using the smart speaker's text-to-speech (TTS) function or the smartphone's push notification function.

[1711] Users can also use smart speakers to verbally report inventory status or special requests. For example, if a user says, "I'm going to a party and need more detergent," the voice data is converted into text data and sent to the server. The server analyzes this information and adjusts inventory estimates and ordering frequency.

[1712] Furthermore, the emotion engine analyzes emotional information from the user's voice and facial expressions and sends the results to the server. For example, if a user says "I feel very stressed" in a strong tone, the emotion engine will analyze the voice tone and facial expression and determine that the user is in a stressed state. This is done using libraries and services specialized in emotion recognition (e.g., AWS Rekognition, Google Cloud Vision).

[1713] The server adjusts inventory management and ordering strategies based on the emotion information from the emotion engine. For example, if the emotion engine determines that the user is feeling stressed, the server will order necessary items such as detergent more quickly or in larger quantities. Also, if it determines that special measures are needed, the server will adjust the order by raising the priority.

[1714] Example prompts to be input to the generative AI model

[1715] Describe the specific process for an inventory management and automated ordering system for everyday items using consumer electronics, from data collection to processing based on emotion recognition. Include the following keywords: inventory management, consumption patterns, online shop, notification, emotion recognition.

[1716] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1717] Step 1: Data collection and storage

[1718] The server receives consumption data sent from home appliances in real time and stores it in a database. Specifically, the server receives data sent from washing machines, dispensers, etc. and stores it in a chronological order in the database.

[1719] Input: Consumption data from a household appliance (e.g., 10 ml of detergent used).

[1720] Data processing: Organize received data with timestamps.

[1721] Output: Organized consumption data is stored in a database.

[1722] Specific operation: The server receives data such as "Detergent 10ml, date and time of use: 2023-10-05 18:45" and records it in the database.

[1723] Step 2: Data analysis and pattern learning

[1724] The server analyzes the consumption data stored in the database and learns the user's daily consumption patterns using machine learning algorithms (e.g., Python's pandas and scikit-learn).

[1725] Input: Consumption data stored in a database.

[1726] Data processing: Machine learning algorithms are used to analyze daily consumption patterns.

[1727] Output: Learning results about users' consumption patterns.

[1728] Specific operation: The server analyzes, for example, one month's worth of data and learns that "users consume an average of 45 ml of detergent per day."

[1729] Step 3: Estimate inventory

[1730] The server estimates the current inventory level based on the learned consumption patterns and predicts when it will fall below the threshold.

[1731] Input: Learning results of consumption patterns and current inventory.

[1732] Data processing: Uses a mathematical formula to estimate based on inventory volume and consumption rate.

[1733] Output: Prediction of when inventory will fall below threshold.

[1734] Specific operation: If the inventory is 500 ml and daily consumption is 45 ml / day, the server estimates that the inventory will fall below 50 ml in approximately 11.11 days.

[1735] Step 4: Automated ordering

[1736] If the server determines that the inventory level is below the threshold, it calls the online shop API to automatically order the item.

[1737] Input: Inventory quantity estimation result.

[1738] Data processing: Generation of order requests using online shop API.

[1739] Output: Order request and response to online shop.

[1740] Specific operation: When the stock falls below 50ml, the server sends a request to the online shop API to order one 500ml bottle of detergent.

[1741] Step 5: Notification Processing

[1742] The terminal receives notification of the order contents and expected arrival date from the server and conveys the information to the user.

[1743] Input: Order details and expected arrival date information from the server.

[1744] Data processing: generating text messages and voice notifications.

[1745] Output: Notification to the user.

[1746] Specific operation: The terminal notifies the user, "We are running low on detergent, so we have placed an additional order. The expected delivery date is two days later."

[1747] Step 6: User Report Processing

[1748] Users use smart speakers to verbally report stock availability and special requests, which are then sent to the server.

[1749] Input: User's voice report.

[1750] Data processing: Text conversion and analysis of voice data.

[1751] Output: Report to the server.

[1752] Specific operation: When a user verbally reports, "I'm having a party and need some extra detergent," the speech is converted into text and sent to the server.

[1753] Step 7: Emotion Recognition

[1754] The emotion engine analyzes emotional information from the user's voice and facial expressions and sends the results to the server.

[1755] Input: User's voice and facial expression data.

[1756] Data processing: Applying emotion recognition algorithms.

[1757] Output: User's emotional information.

[1758] Specific behavior: The emotion engine analyzes the user's tone of voice and facial expressions to recognize stress levels and other emotions.

[1759] Step 8: Emotion-Based Processing

[1760] The server adjusts inventory management and ordering strategies based on the emotional information from the emotion engine.

[1761] Input: Emotion information from the emotion engine.

[1762] Data processing: Adjusting inventory management and ordering strategies.

[1763] Output: Coordinated inventory management and ordering instructions.

[1764] Specific behavior: If the server recognizes that the user is stressed, it will order the necessary items more quickly or in larger quantities.

[1765] (Application example 2)

[1766] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1767] In modern store operations, inventory management and the quality of customer service are important issues. In particular, there is a need to prevent lost sales opportunities due to inventory shortages and to respond to customers' emotional states in a detailed manner. However, conventional systems have difficulty simultaneously solving these problems, making it difficult to achieve efficient inventory management and customer service. Therefore, there is a need for a system that combines an emotion engine to provide optimal inventory management and customer service that responds to the user's emotional state.

[1768] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting data on items consumed by household electrical appliances, means for analyzing the consumption data of the items and learning consumption patterns, means for estimating inventory status based on the consumption patterns, means for automatically ordering items when the estimated inventory status falls below a set threshold, means for notifying the user of the order details and expected arrival date of the items, and means for optimizing inventory management and ordering processes based on emotions using an emotion engine that recognizes the user's emotional state. This makes it possible to improve the efficiency of inventory management and user satisfaction.

[1769] Below are definitions of important words:

[1770] "Household appliances" are electrical devices used in ordinary households that can be used as a means of collecting consumption data.

[1771] "Goods" refers to the items that are consumed or controlled by a consumer electronics device.

[1772] "Data Collection Measure" means a device or software for periodically collecting data on goods consumed from a consumer electronics device.

[1773] "Data analysis means" refers to a device or software that analyzes and learns consumption patterns based on collected consumption data.

[1774] The "consumption pattern" indicates the user's tendency to consume items on a daily basis, and is learned by the data analysis means.

[1775] An "inventory estimating means" is a device or software for estimating current inventory levels based on consumption patterns and assessing inventory status.

[1776] An "automatic ordering means" is a device or software for automatically ordering items through an online shop when estimated inventory falls below a set threshold.

[1777] The "notification means" is a device or software for notifying the user of the ordered item and the expected arrival date.

[1778] An "emotion engine" is a device or software that recognizes a user's emotional state from their voice and facial expressions and performs processing based on the emotional information.

[1779] A "server" is a computer system or service that coordinates a series of processes such as data collection, analysis, inventory estimation, automatic ordering, and notification.

[1780] A "smartphone" is a mobile communication terminal that has advanced processing capabilities in addition to the functions of a mobile phone and can run applications.

[1781] "Smart glasses" are glasses-shaped devices that can display information and collect data.

[1782] "Emotion information" is data indicating the emotional state recognized by the emotion engine from the user's voice and facial expression.

[1783] "Customer service actions" refer to the service content and response policy provided depending on the user's emotional state.

[1784] This invention relates to an inventory management and automatic ordering system for daily necessities using household electrical appliances, and in particular to a system that realizes optimal inventory management according to the user's emotional state by combining an emotion engine. This system is composed of electrical appliances installed in the home, a server, terminals (smartphones or smart glasses), the emotion engine, and the user.

[1785] System Overview

[1786] 1. Data Collection and Storage:

[1787] The server receives consumption data sent from the home appliances and stores it in a database in chronological order. For example, a detergent dispenser sends the amount of detergent consumed to the server and records it in the database.

[1788] 2. Data analysis and pattern learning:

[1789] The server analyzes the received consumption data and learns the user's daily consumption patterns. For example, it analyzes consumption amounts on a monthly basis to understand daily consumption trends.

[1790] 3. Inventory Estimation:

[1791] The server estimates current inventory levels based on learned consumption patterns and predicts when inventory levels will fall below a set threshold, allowing for timely replenishment.

[1792] 4. Automated ordering:

[1793] When the server determines that the inventory level is below the set threshold, it calls the online shop API to automatically order the necessary items, thereby preventing stockouts.

[1794] 5. Notification Processing:

[1795] The terminal notifies the user of the order details and expected arrival date from the server, and the user can receive this information via their smartphone or smart glasses.

[1796] 6. User Reporting Process:

[1797] Users verbally report stock availability and special requests via their smartphones or smart glasses, which are then sent to the server and reflected in inventory estimates and order frequency.

[1798] 7. Emotion recognition:

[1799] The emotion engine recognizes emotions from the user's voice and facial expressions and sends that emotional information to the server. For example, if the user is under stress, the emotion engine analyzes that information in real time and notifies the server.

[1800] 8. Emotion-based processing:

[1801] The server adjusts inventory management and ordering strategies based on the emotion information from the emotion engine. For example, if a user is feeling stressed, it can improve user satisfaction by quickly replenishing necessary items.

[1802] Hardware and Software Use Cases

[1803] Hardware:

[1804] Smartphone: A device that allows users to check inventory status and send emotional information.

[1805] Smart glasses: A device that allows store staff to check inventory information and how to respond based on customer sentiment.

[1806] Consumer electronics: Devices for collecting consumption data and sending it to a server.

[1807] software:

[1808] Server: A platform that oversees data collection, analysis, inventory estimation, automatic ordering, and notifications.

[1809] Emotion engine: Software for recognizing the user's emotional state.

[1810] Consumption data analysis software: Tools for learning consumption patterns and estimating inventory status.

[1811] Specific examples

[1812] Here is a specific example of managing a user's daily detergent consumption. For example, if 45 ml of household detergent is consumed daily and the initial inventory is 500 ml, the washing machine sends the amount of detergent consumed each time to the server. The server analyzes past data and learns that the user consumes approximately 315 ml of detergent per week. If the user's inventory is 500 ml, the server predicts that the inventory will fall below 100 ml in approximately 11.11 days and orders additional detergent from the online shop at the appropriate time. Furthermore, if the user verbally reports that they need more detergent because they have a party next week, the server adjusts the order amount based on that information. If the emotion engine recognizes the user's stress level, the server responds more quickly and places an order immediately to reassure the user.

[1813] Prompt Sentence Examples

[1814] "Please tell us about the structure and process of an app for inventory management and improving customer service in a physical store. Please include specific examples, such as a case where you recognized customer emotions and linked them to an inventory management system."

[1815] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1816] Step 1:

[1817] Data collection

[1818] The server collects data on the items consumed by household electrical appliances (e.g., washing machines and detergent dispensers) in real time. The input at this time is consumption data sent from each household electrical appliance, specifically including the amount consumed (e.g., 10 ml of detergent) and the date and time of consumption. The server receives this data and stores it in a database in chronological order. The information inserted into the database consists of "item name, amount consumed, and date and time of consumption."

[1819] Step 2:

[1820] Data analysis

[1821] The server analyzes the collected consumption data and learns the user's daily consumption patterns. The input is past consumption data stored in a database, and the output is consumption patterns (e.g., average daily consumption). The server uses data analysis software to statistically analyze the past consumption data and calculate the consumption patterns of the user's goods. Specifically, it calculates consumption trends by day, week, and month.

[1822] Step 3:

[1823] Inventory Estimation

[1824] The server estimates the current inventory level based on the learned consumption patterns. The input is the current inventory level and consumption pattern data, and the output is a prediction of when the inventory will fall below a set threshold. The server uses an inventory estimation algorithm to calculate how much inventory is currently remaining and predict when the inventory will run out. Specifically, the server estimates when the inventory will run out by subtracting the consumption pattern from the current inventory level.

[1825] Step 4:

[1826] Automatic ordering

[1827] If the server determines that the inventory level is below a set threshold, it calls the online shop API to automatically order the required items. The input is the inventory estimation result (when inventory will run out), and the output is the result of the order request sent. The server sends an order request through the online shop API to instruct the replenishment of the required items. Specifically, it sends a request to the API including the product ID and order quantity, and checks the response.

[1828] Step 5:

[1829] Notification Processing

[1830] The server notifies the user of the order details and expected arrival date. The input is the response data from the online shop API (order details and expected arrival date), and the output is a notification message. The server uses notification generation software to create a notification message for the user and sends it to the user via their smartphone or smart glasses. Specifically, a notification including the product name, order quantity, and expected arrival date is pushed to the user's device.

[1831] Step 6:

[1832] User Report Processing

[1833] Users verbally report inventory status or special requests via smartphones or smart glasses. The input is voice input from the user, and the output is a request converted into text. The device uses voice recognition software to analyze the voice input, convert it into text data, and send it to the server. Specifically, a voice request such as "We need to order more for next week's party" can be converted into text and sent to the server.

[1834] Step 7:

[1835] emotion recognition

[1836] The emotion engine recognizes emotions from the user's voice and facial expressions and sends that emotional information to the server. The input is the user's voice and facial expression data, and the output is analyzed emotional information. The device uses the emotion engine to analyze emotions from the voice and facial expressions and sends the emotional state (e.g., stress, joy, etc.) to the server. Specifically, changes in voice tone and facial expressions are detected in real time and recorded as the emotional state.

[1837] Step 8:

[1838] Emotion-Based Processing

[1839] The server adjusts inventory management and ordering strategies based on the emotion information from the emotion engine. The input is the emotion recognition results, and the output is adjusted ordering instructions and inventory management strategies. The server analyzes the emotion recognition results, and if the user is feeling stressed, for example, it can improve user satisfaction by quickly ordering the necessary items. Specifically, it can take measures such as setting an urgent order flag and placing an order immediately.

[1840] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1841] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1842] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1843] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1844] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1845] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1846] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1847] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1848] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1849] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1850] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1851] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1852] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1853] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1854] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1855] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1856] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1857] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1858] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit ele...

Claims

1. A means for collecting data on goods consumed by the consumer electronics device; means for analyzing consumption data of the items and learning consumption patterns; means for estimating an inventory status based on said consumption pattern; means for automatically ordering items when the estimated inventory status falls below a set threshold; means for notifying the user of the order details and expected arrival date of the item; A system including:

2. a means for the user to verbally report stock availability or special requests; means for analyzing the user reports and adjusting inventory estimates and ordering frequency; The system of claim 1 further comprising:

3. A means for ordering goods using the online store API; A means for confirming the order details and expected arrival date based on the response from the API; The system of claim 1 , comprising:

Images