System

Abstract

A system is provided.SOLUTION: This system is provided with a means for allowing a user to manually register daily necessities, a means for automatically registering the daily necessities from merchandise photographs or receipts, a means for analyzing the use conditions of the registered daily necessities by generated AI, a means for predicting and reminding replenishing timing on the basis of the analyzed result, and a means for providing the user with personalized discount information or recommended merchandise information.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Modern consumers spend a lot of time and effort managing and purchasing daily necessities. Therefore, it is necessary to properly manage the consumption of daily necessities and replenish them at the right time before they run out. In order to live an economical life, it is also important not to miss out on the latest discount information and great deals. However, doing these things manually is cumbersome, and an efficient method is needed. [Means for solving the problem]

[0005] The present invention provides a system that includes a means for users to manually register daily necessities, a means for automatically registering daily necessities from product photos and receipts, a means for analyzing the usage of registered daily necessities using a generation AI, a means for predicting and reminding users when to replenish based on the analysis results, and a means for providing users with personalized discount information and recommended product information. This system efficiently supports users' daily necessities management and purchasing, significantly reducing the amount of work required. Furthermore, by utilizing the generation AI, personalized suggestions based on the user's purchasing trends can be made to support economical lifestyles.

[0006] "User" refers to an individual who uses the system to register, manage, check usage status, and receive replenishment reminders for everyday items.

[0007] "Daily necessities" refers to consumer goods that are frequently used in the daily lives of households and individuals, and specifically includes cleaning supplies such as shampoo and detergent, as well as consumables.

[0008] "Manual registration means" refers to the function that allows users to manually enter information about everyday items through an input form and register them in the system.

[0009] "Means for automatic registration" refers to the function of automatically extracting product information from product photos and receipt images and registering it in the system.

[0010] "Generative AI" refers to artificial intelligence technology that analyzes registered data to predict and analyze how users use everyday items.

[0011] "Means of analyzing usage status using generation AI" refers to the function of using generation AI to analyze the frequency and quantity of use of registered daily necessities and understand the usage patterns of each user.

[0012] "Means to predict and remind users when replenishment is necessary" refers to a function that predicts when daily necessities need to be replenished based on the analysis results of the generative AI and notifies the user.

[0013] "Means of providing personalized discount information and recommended product information" refers to a function that suggests optimal discount information and recommended product information based on the user's usage and purchase history.

[0014] "Image recognition algorithm" refers to technology that extracts information about target everyday items from photographs or receipts.

[0015] "Database" refers to a storage system that allows the system to store and manage information on everyday items, usage status, purchase history, etc. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] The present invention is a system that allows users to efficiently manage their daily necessities and provides optimal replenishment timing and personalized discount information. This system includes a means for users to manually register daily necessities, a means for automatically registering items from product photos and receipts, a means for analyzing usage status using generation AI, a means for predicting and reminding replenishment timing, and a means for providing personalized discount information and recommended product information.

[0038] Program processing

[0039] 1. How users register everyday items

[0040] Users launch the application using a device such as a smartphone or tablet. When manually registering daily necessities, users enter the product name, purchase date, price, store information, etc. into an input form. The device sends this information in JSON format to the server, which then stores the received data in a database.

[0041] Users can also take photos of receipts and products and register them. The images are sent from the device to a server, where they are analyzed using an image recognition algorithm (e.g., OCR technology). The server then extracts product information from the images and registers it in a database.

[0042] Examples:

[0043] When a user purchases shampoo, they take a photo of the receipt. The image is sent to a server, and OCR technology extracts the purchase date, product name, price, etc., and registers them in a database. This allows users to register product information in the system without any hassle.

[0044] 2. Usage analysis

[0045] The server periodically analyzes the information about daily necessities stored in the database. Utilizing generative AI, it analyzes each user's usage and consumption rate of daily necessities and predicts when replenishment is necessary. Based on this information, the server reminds the user of the optimal time to replenish.

[0046] Examples:

[0047] The server analyzes the shampoo usage history in the database and finds that, on average, users use up a bottle every three weeks. One week before the next bottle of shampoo runs out, the server sends a reminder to the device, informing the user that the shampoo needs to be refilled.

[0048] 3. Providing discount information and recommended products

[0049] The server uses AI to generate personalized product information based on each user's usage and purchase history, and also retrieves the latest discount information and related product information from external data sources to make optimal suggestions for the user.

[0050] Examples:

[0051] The server analyzes the user's shampoo purchase history and retrieves information about new and discounted products from the same brand from external data sources. Based on the information retrieved, a notification is sent to the device stating, "Recommended shampoo is now 20% off," and the user can check the details within the app.

[0052] Implementation environment

[0053] The system operates in a network environment that includes user devices (smartphones, tablets, etc.), a server, and a database. Users use an application to register their daily necessities and send the necessary information to the server. The server stores the received information in a database, periodically analyzes the data, and uses generative AI to provide optimal replenishment timing and product information.

[0054] This configuration allows users to manage their daily necessities more efficiently, supporting them in replenishing their supplies when needed and making purchases at a good price.

[0055] The processing flow will be explained below.

[0056] Program processing (specific steps)

[0057] Manually inputting daily necessities

[0058] Step 1:

[0059] A user launches an application on their smartphone or tablet.

[0060] Step 2:

[0061] The user taps the "Register a new product" button and enters information such as the product name, purchase date, price, and store.

[0062] Step 3:

[0063] Check the information entered by the user and tap the "Register" button.

[0064] Step 4:

[0065] The terminal sends the entered data to the server in JSON format.

[0066] Step 5:

[0067] The server analyzes the received JSON data and generates an SQL statement to save it in the database.

[0068] Step 6:

[0069] The server executes the generated SQL statement and saves the new product information in the database.

[0070] Automatically register daily necessities from product photos and receipts

[0071] Step 1:

[0072] A user launches an application on their smartphone or tablet.

[0073] Step 2:

[0074] The user taps the "Scan Receipt" button and takes a photo of the receipt or item.

[0075] Step 3:

[0076] The device temporarily stores the captured image and sends the image data to the server.

[0077] Step 4:

[0078] The server receives the image and runs an image recognition algorithm (such as OCR) to extract product information.

[0079] Step 5:

[0080] The server analyzes the extracted product information and obtains information such as product name, purchase date, price, and store.

[0081] Step 6:

[0082] The server generates an SQL statement to save the product information to the database.

[0083] Step 7:

[0084] The server executes the generated SQL statement and saves the new product information in the database.

[0085] Usage analysis and replenishment forecast

[0086] Step 1:

[0087] The server runs a job that is triggered periodically to retrieve user commodity usage data from the database.

[0088] Step 2:

[0089] The server uses generated AI to analyze the acquired data and calculate each user's daily necessities usage rate.

[0090] Step 3:

[0091] The server predicts the next replenishment timing based on the calculation results.

[0092] Step 4:

[0093] The server sets a reminder based on the predicted replenishment timing.

[0094] Step 5:

[0095] The device receives the set reminder and displays a notification to the user.

[0096] Providing discount information and recommended products

[0097] Step 1:

[0098] The server retrieves the user's purchase history and usage information from the database.

[0099] Step 2:

[0100] The server uses generated AI to analyze the acquired data and generate a list of recommended products for each user.

[0101] Step 3:

[0102] The server retrieves the latest discount information from an external data source.

[0103] Step 4:

[0104] The server combines the recommended product list and discount information to generate information to be provided to the user.

[0105] Step 5:

[0106] The device displays the information received from the server and sends notifications to the user.

[0107] Example 1

[0108] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0109] Today's busy lives make it difficult for users to properly manage their daily necessities and determine when to replenish them. Furthermore, obtaining optimal discount information and recommended product information takes time and effort. To address these issues, a system is needed that can efficiently manage daily necessities, provide optimal replenishment timing, and provide users with personalized, useful information.

[0110] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0111] In this invention, the server includes: a means for a user to launch an application using a smart device and manually register information about daily necessities; a means for converting the manually entered data into JSON format and sending it to the server; a means for the server to store the received data in a database; a means for sending receipts and product photos to the server and analyzing the images using OCR technology; a means for the server to register product information extracted from the images in the database; a means for periodically analyzing the database data using a generative AI model to analyze consumption rates; a means for predicting replenishment timing and sending reminders to the terminal; a means for obtaining discount information and related product information from external data sources and generating personalized suggestions; and a means for sending discount information and recommended product information to the user's terminal. This enables users to efficiently manage and replenish daily necessities and receive personalized, useful product information and discount information.

[0112] "User" means an individual or corporation that uses the system to register and manage everyday items.

[0113] A "smart device" is a device, such as a smartphone or tablet, that can connect to the Internet and run applications.

[0114] "Application" means a software program that a user uses to register and manage everyday item information on a smart device.

[0115] "Daily necessities" are products that users use on a daily basis and consume, such as shampoo, detergent, and toilet paper.

[0116] "Manual input" refers to the act of a user directly entering information into an application's input form using a smart device.

[0117] "JSON format" stands for JavaScript Object Notation and is a lightweight data format for structuring and exchanging data.

[0118] "Server" means a computer system that receives and processes data sent by users and stores it in a database.

[0119] A "database" is a collection of data managed by a server, a system that allows for efficient information storage and retrieval.

[0120] A "receipt" is a paper or electronic record that contains details of a purchased item (e.g., item name, price, purchase date, etc.).

[0121] "OCR technology" is an abbreviation for Optical Character Recognition, and is a technology that automatically recognizes character information in an image and converts it into text data.

[0122] A "generative AI model" is an artificial intelligence model that uses technologies such as deep learning to learn patterns from data and make predictions and classifications.

[0123] A "reminder" is a notification sent by the server to a user's device, and is a message used to encourage a specific action.

[0124] "External data sources" are information sources from outside the system, including web services and databases that provide up-to-date discount and product information.

[0125] "Personalized suggestions" are product and service suggestions that are individually generated based on a user's usage and purchase history.

[0126] This invention is a system that allows users to efficiently manage daily necessities and provides optimal replenishment timing and personalized discount information. The system includes a means for users to manually register daily necessities, a means for automatically registering daily necessities from product photos and receipts, a means for analyzing usage status using a generative AI model, a means for predicting and reminding replenishment timing, and a means for providing personalized discount information and recommended product information.

[0127] System Configuration

[0128] Hardware

[0129] User device: A device that can connect to the internet, such as a smartphone or tablet.

[0130] Server: A computer system that receives and processes data and accesses a database.

[0131] Database: A collection of data managed by a server that stores information about everyday items.

[0132] software

[0133] Application: Software that users use on their devices to register and manage everyday items.

[0134] OCR technology: Software that analyzes photos of receipts or products and extracts text information. Example: Tesseract OCR.

[0135] Generative AI model: An artificial intelligence model that analyzes data and predicts replenishment timing. Example: TensorFlow.

[0136] Overview of operation procedure

[0137] Daily necessities registration

[0138] 1. Manual registration:

[0139] Users launch the application using their smartphone or tablet and manually enter information about the daily necessities they purchased (product name, purchase date, price, store information, etc.). After completing the entry, they press the "Register" button.

[0140] The terminal converts the input data into JSON format and sends it to the server.

[0141] The server stores the received data in a database.

[0142] Examples:

[0143] When a user enters shampoo purchase information into the input form and clicks the "Register" button, the device converts the input data into JSON format and sends it to the server, which then analyzes the received data and stores it in a database.

[0144] Example prompt sentence:

[0145] Please manually enter the shampoo information you purchased (product name, purchase date, price, store information) using your smartphone.

[0146] 2. Registration from an image:

[0147] The user takes a photo of the product or receipt of the purchased item, and then sends the image data to the server via the application.

[0148] The terminal transmits the captured image data to the server.

[0149] The server analyzes the received image data using OCR technology and registers the product information extracted from the image in a database.

[0150] Examples:

[0151] When a user takes a photo of a receipt using their smartphone camera and sends the image from the app to the server, the server activates the OCR engine to extract text data from the image, analyzes the extracted text data, and stores it in a database.

[0152] Example prompt sentence:

[0153] "Take a photo of the receipt or item you purchased and submit it through the application."

[0154] Usage analytics and reminders

[0155] The server periodically analyzes the information on daily items stored in the database, utilizing a generative AI model to analyze each user's usage and consumption rate of daily items.

[0156] Based on the analysis results, the server predicts when replenishment is needed and sends a reminder to the user's device.

[0157] Examples:

[0158] The server analyzes the shampoo usage history in the database and finds that, on average, users use up a bottle every three weeks. One week before the next bottle runs out, the server sends a reminder to the device saying, "You need to refill your shampoo."

[0159] Providing discount information and recommended products

[0160] The server generates personalized product information using a generative AI model based on each user's usage and purchase history, and also retrieves the latest discount information and related product information from external data sources to make optimal suggestions for the user.

[0161] The server transmits the generated discount information and recommended product information to the user's terminal.

[0162] Examples:

[0163] The server analyzes the user's shampoo purchase history and retrieves information about new and discounted products from the same brand from external data sources. Based on the information retrieved, a notification is sent to the device stating, "Recommended shampoo is now 20% off," and the user can check the details within the app.

[0164] This system helps users manage their daily necessities more efficiently, supporting them in replenishing their supplies when needed and making purchases at a good price.

[0165] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0166] Daily necessities registration phase

[0167] Step 1:

[0168] The user launches the application using a smart device and manually inputs information about the purchased daily necessities (product name, purchase date, price, store information, etc.). Once the input is complete, the user presses the "Register" button. The input data includes the product name, purchase date, price, and store information.

[0169] Specific behavior:

[0170] The user enters shampoo purchase information into the input form and clicks the "Register" button.

[0171] input:

[0172] Product information entered by the user (product name, purchase date, price, store information)

[0173] output:

[0174] JSON format data on the device

[0175] Step 2:

[0176] The terminal converts the manually entered data into JSON format, which includes the product name, purchase date, price, and store information.

[0177] Specific behavior:

[0178] The terminal converts the input data into JSON format.

[0179] input:

[0180] Product information entered by the user

[0181] output:

[0182] JSON format data

[0183] Step 3:

[0184] The terminal sends the converted JSON format data to the server.

[0185] Specific behavior:

[0186] The device starts the process of sending JSON data to the server.

[0187] input:

[0188] Structured product data in JSON format

[0189] output:

[0190] JSON data sent to the server

[0191] Step 4:

[0192] The server receives the JSON data sent from the device, analyzes it, and stores it in a database.

[0193] Specific behavior:

[0194] The server stores the received data in a database.

[0195] input:

[0196] JSON format data

[0197] output:

[0198] Information on everyday items in the database

[0199] Image information registration phase

[0200] Step 1:

[0201] The user takes a photo of the product or receipt of the purchased item, and then sends the image data to the server via the application.

[0202] Specific behavior:

[0203] The user takes a photo of the receipt using their smartphone camera and the app sends the image to the server.

[0204] input:

[0205] Receipts and product photos

[0206] output:

[0207] Image data stored on the device

[0208] Step 2:

[0209] The terminal transmits the captured image data to the server. The transmitted data is an image that may contain product information.

[0210] Specific behavior:

[0211] The device starts the process of sending the captured image to the server.

[0212] input:

[0213] Image data

[0214] output:

[0215] Image data sent to the server

[0216] Step 3:

[0217] The server analyzes the received image data using OCR technology and registers the product information extracted from the image (product name, purchase date, price, etc.) in a database.

[0218] Specific behavior:

[0219] The server starts the OCR engine, extracts text data from the image, analyzes the extracted text data, and stores it in a database.

[0220] input:

[0221] Image data

[0222] output:

[0223] Product information stored in the database

[0224] Usage analysis phase

[0225] Step 1:

[0226] The server periodically analyzes the information stored in the database and uses a generative AI model to analyze the usage and consumption rate of each item for each user.

[0227] Specific behavior:

[0228] The server runs scheduled jobs and performs data analysis using generative AI models.

[0229] input:

[0230] Information on everyday items in the database

[0231] output:

[0232] Usage and consumption rate per user

[0233] Step 2:

[0234] Based on the analysis results, the server predicts when each user's daily necessities will need to be replenished.

[0235] Specific behavior:

[0236] The server calculates the timing of replenishment based on the analysis results.

[0237] input:

[0238] AI model analysis results

[0239] output:

[0240] Predicted replenishment timing

[0241] Step 3:

[0242] The server sends a reminder to the user's device when the predicted replenishment time approaches.

[0243] Specific behavior:

[0244] The server generates a reminder notification and sends it to the device.

[0245] input:

[0246] Predicted replenishment timing

[0247] output:

[0248] Reminder messages sent to your device

[0249] Discount information and recommended products phase

[0250] Step 1:

[0251] The server generates personalized product information using a generative AI model based on each user's usage and purchase history, and also retrieves the latest discount information and related product information from external data sources.

[0252] Specific behavior:

[0253] The server periodically calls an external API to retrieve the latest discount information.

[0254] input:

[0255] User usage, purchase history, external data

[0256] output:

[0257] Personalized product information

[0258] Step 2:

[0259] Based on the collected data, the server generates and stores discount information and recommended product information that is optimal for the user.

[0260] Specific behavior:

[0261] The server applies an algorithm based on the information it obtains to generate personalized suggestions.

[0262] input:

[0263] Data analysis results, external data

[0264] output:

[0265] Saved discounts and recommended products

[0266] Step 3:

[0267] The server transmits the generated discount information and recommended product information to the user's terminal.

[0268] Specific behavior:

[0269] The server generates the proposal as a notification and sends it to the device.

[0270] input:

[0271] Generated proposals

[0272] output:

[0273] Notification messages sent to the device

[0274] (Application example 1)

[0275] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0276] Conventional daily necessities management systems have the drawback of making it difficult to properly determine when to replenish items because they require users to register their purchases and do not adequately analyze usage.Furthermore, they do not provide users with personalized discount information or recommended products, making it difficult to efficiently manage daily necessities and purchase the products they need at the optimal time.

[0277] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0278] In this invention, the server includes a means for users to manually register daily necessities, a means for automatically registering daily necessities from product photos and receipts, and a means for analyzing the usage of registered daily necessities using a generating AI. This allows users to easily register daily necessities, and the analysis of usage using the generating AI makes it possible to predict the optimal timing for replenishment and send reminders. Furthermore, the server includes a means for predicting the usage rate and replenishment timing using a generating AI model, a means for extracting and automatically registering product information using image recognition, and a means for obtaining and providing discount information and recommended product information from external data sources. This allows users to efficiently manage their daily necessities and purchase products at great prices based on personalized discount information.

[0279] "Means for manually registering daily necessities" is a function that allows users to manually enter product name, purchase date, price, store information, etc. to register daily necessities in the system.

[0280] "Means for automatically registering daily necessities from product photos and receipts" is a function that analyzes product photos and receipt images taken by users, automatically extracts product information, and registers it in the system.

[0281] "Means for analyzing usage status using generative AI" refers to a function that uses a generative AI model to analyze information on stored daily necessities, and analyzes the pace of use and consumption of daily necessities for each user.

[0282] "Means to predict and remind the timing of replenishment" is a function that predicts the next time replenishment is required based on the results of analysis by the generation AI and notifies the user.

[0283] "Means for providing personalized discount information and recommended product information" refers to a function that uses generative AI to provide optimal discount information and related product information based on each user's usage and purchase history.

[0284] "Means for predicting usage rate and replenishment timing using a generative AI model" is a function that uses a generative AI model to predict the usage rate and optimal replenishment timing of registered daily necessities.

[0285] "Means for extracting and automatically registering product information using image recognition" refers to a function that uses OCR technology or image recognition algorithms to extract product information from receipt or product images and automatically registers it in the system.

[0286] "Means for obtaining and providing discount information and recommended product information from external data sources" refers to a function that analyzes the latest discount information and recommended product information obtained from external data sources and provides it to users in a personalized form.

[0287] MODE FOR CARRYING OUT THE INVENTION

[0288] The present invention is a system that allows users to efficiently manage their daily necessities, and provides optimal replenishment timing and personalized discount information. This system operates using a smartphone application, a server, and a database.

[0289] System configuration

[0290] User Device

[0291] Users launch the application using a device such as a smartphone or tablet. There are two ways to register daily necessities: manual entry or automatic registration.

[0292] 1. Manual registration

[0293] The user enters the product name, purchase date, price, store information, etc. into an input form and sends this information in JSON format to the server.

[0294] 2. Automatic Registration

[0295] Users take photos of receipts or products and send the images to the server, which uses OCR technology to extract product information from the images and register it in a database.

[0296] Servers and Databases

[0297] 1. Usage analysis

[0298] The server uses a generative AI model to analyze information about daily necessities stored in a database, and analyzes each user's pace of use and consumption of daily necessities.

[0299] 2. Predicting replenishment timing

[0300] Based on the analysis results, the server predicts when the next replenishment is required and sends a reminder to the user.

[0301] 3. Providing discount information and recommended products

[0302] The server retrieves the latest discount information and related product information from external data sources and generates personalized product information using a generative AI model.

[0303] Based on this information, the server provides users with optimal discount information and related product information.

[0304] Specific examples of implementation methods

[0305] Manual registration example

[0306] When a user purchases shampoo, they manually enter product information using the application, including details such as the purchase date, product name, and price, and send the information to the server, which then registers the received information in the database.

[0307] Auto-registration Example

[0308] The user takes a photo of the receipt and sends it to the server via the application. The server uses OCR technology to extract the purchase date, product name, price, etc. from the image and registers them in the database. This allows the user to register product information without any hassle.

[0309] Usage analysis example

[0310] The server analyzes the shampoo usage history stored in the database and finds that the average user uses up a bottle every three weeks. One week before the next bottle of shampoo runs out, the server sends a reminder to the user's smartphone, informing them that they need to refill their shampoo.

[0311] Examples of discounts and product recommendations

[0312] The server analyzes the user's shampoo purchase history and retrieves information about new and discounted products from the same brand from external data sources. Based on the information retrieved, the server sends the user a notification saying, "Recommended shampoo is now 20% off," and allows them to check the details within the app.

[0313] Examples of using generative AI models and prompts

[0314] An example of a prompt that the server uses to perform analysis using a generative AI model:

[0315] This system is designed to help you efficiently manage your daily necessities, provide optimal replenishment timing, and provide personalized discount information. Simply take a photo of your receipt or product and register it in the system, and you will receive timely replenishment notifications and discount information.

[0316] This allows users to efficiently manage their daily necessities and purchase the products they need at the best possible time.

[0317] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0318] Step 1:

[0319] Users manually register daily items

[0320] The user launches the smartphone application and enters the product name, purchase date, price, store information, etc. into the input form. The entered information is sent to the server in JSON format. The server parses the received JSON data and registers it in the database.

[0321] Input: Product name, purchase date, price, store information (user input)

[0322] Output: Product information is saved in the database

[0323] Step 2:

[0324] Users take photos of receipts and products and register them

[0325] A user uses a smartphone application to take a photo of a receipt or product. The image data is then uploaded to a server. The server uses OCR technology to extract text information from the image and convert it into data such as product name, purchase date, price, and store information. This data is then registered in a database.

[0326] Input: Image of receipt or product (taken by user)

[0327] Output: The extracted product information is saved in the database.

[0328] Step 3:

[0329] The server analyzes the usage of everyday items

[0330] The server periodically analyzes the information on daily items stored in the database using a generative AI model, which calculates each user's usage rate and consumption rate of daily items.

[0331] Input: Information about everyday items stored in a database

[0332] Output: Analysis results of usage pace and wear rate (analysis results from generative AI model)

[0333] Step 4:

[0334] Refill forecast and reminder sending

[0335] The server predicts when the next refill is needed based on the analysis results of the generative AI model. Based on the prediction, a reminder notification is sent to the user's smartphone. For example, if shampoo runs out within one week of the predicted time, the user will be notified that "shampoo needs to be refilled."

[0336] Input: Analysis results of the generative AI model

[0337] Output: Reminder notification for refilling (sent to user's smartphone)

[0338] Step 5:

[0339] Providing discount information and recommended products

[0340] The server uses a generative AI model based on each user's usage and purchase history to generate personalized product information. It also obtains discount information and recommended product information from external data sources and stores it in a database. The generated information is then combined to notify users of appropriate discount information and recommended products.

[0341] Inputs: User usage, purchase history, discount information from external data sources

[0342] Output: Notification of recommended products and discount information (sent to the user's smartphone)

[0343] This allows users to efficiently manage their daily necessities and replenish them at the optimal time and at a good price.

[0344] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0345] This invention combines an emotion engine with a system that allows users to efficiently manage their daily necessities and provides optimal replenishment timing and personalized discount information, thereby realizing personalized suggestions based on user emotion recognition. This system includes a means for users to manually register daily necessities, a means for automatically registering items from product photos and receipts, a means for analyzing usage status using generative AI, a means for predicting and reminding users when replenishment is necessary, a means for providing personalized discount information and recommended product information, and a means for recognizing user emotions using the emotion engine.

[0346] Program processing

[0347] 1. Manual and automatic commodity registration

[0348] Users launch the application using a device such as a smartphone or tablet. When registering manually, users enter the product name, purchase date, price, store information, etc. into an input form. The device sends this information in JSON format to the server, which then stores the received data in a database.

[0349] When automatically registering using product photos or receipts, users take a photo and send it to the server from their device. The server then uses an image recognition algorithm (such as OCR) to extract product information from the image and register it in the database.

[0350] Examples:

[0351] When a user purchases shampoo, they take a photo of the receipt. The image is sent to a server, and OCR technology extracts the purchase date, product name, price, etc., and registers them in a database. This allows users to register product information in the system without any hassle.

[0352] 2. Analyzing usage and predicting replenishment timing

[0353] The server periodically analyzes the information about daily necessities stored in the database. Utilizing generative AI, it analyzes each user's usage and consumption rate of daily necessities and predicts when replenishment is necessary. Based on this information, the server reminds the user of the optimal time to replenish.

[0354] Examples:

[0355] The server analyzes the shampoo usage history in the database and finds that, on average, users use up a bottle every three weeks. One week before the next bottle of shampoo runs out, the server sends a reminder to the device, informing the user that the shampoo needs to be refilled.

[0356] 3. Personalized discounts and product recommendations

[0357] The server uses AI to generate personalized product information based on each user's usage and purchase history, and also retrieves the latest discount information and related product information from external data sources to make optimal suggestions for the user.

[0358] Examples:

[0359] The server analyzes the user's shampoo purchase history and retrieves information about new and discounted products from the same brand from external data sources. Based on the information retrieved, a notification is sent to the device stating, "Recommended shampoo is now 20% off," and the user can check the details within the app.

[0360] 4. Emotion Recognition and Personalization with Emotion Engine

[0361] The emotion engine recognizes emotions based on user input and usage, and tailors personalized product recommendations and reminders accordingly.

[0362] Examples:

[0363] If a user types "I'm tired" into the app, the emotion engine will recognize that emotion and suggest products to reduce stress (e.g., relaxing bath salts). If the user's emotion is fatigued, the app will also express the reminder in gentler terms, such as "Thank you for your hard work. Your next shampoo is needed."

[0364] 5. Storage and utilization of emotional data

[0365] The server periodically stores the user's emotional data recognized by the emotion engine in a database, which will be used for future personalized suggestions, such as more personalized product suggestions and tailoring of reminder content.

[0366] Examples:

[0367] Based on the user's emotional history, the system analyzes times when stress is likely to increase and provides focused recommendations on relaxation products and other products that will help relieve stress.

[0368] Implementation environment

[0369] The system operates in a network environment that includes user devices (smartphones, tablets, etc.), a server, and a database. Users use an application to register their daily necessities and send the necessary information to the server. The server stores the received information in a database and analyzes it using generative AI and an emotion engine to provide optimal replenishment timing and personalized product information.

[0370] This configuration allows for more efficient management of daily necessities for users, and makes it possible to provide optimal product suggestions and replenishment notifications tailored to each individual's emotional state.

[0371] The processing flow will be explained below.

[0372] Manually inputting daily necessities

[0373] Step 1:

[0374] A user launches an application on their smartphone or tablet.

[0375] Step 2:

[0376] The user taps the "Register a new product" button and enters information such as the product name, purchase date, price, and store.

[0377] Step 3:

[0378] Check the information entered by the user and tap the "Register" button.

[0379] Step 4:

[0380] The terminal sends the entered data to the server in JSON format.

[0381] Step 5:

[0382] The server analyzes the received JSON data and generates an SQL statement to save it in the database.

[0383] Step 6:

[0384] The server executes the generated SQL statement and saves the new product information in the database.

[0385] Automatically register daily necessities from product photos and receipts

[0386] Step 1:

[0387] A user launches an application on their smartphone or tablet.

[0388] Step 2:

[0389] The user taps the "Scan Receipt" button and takes a photo of the receipt or item.

[0390] Step 3:

[0391] The device temporarily stores the captured image and sends the image data to the server.

[0392] Step 4:

[0393] The server receives the image and runs an image recognition algorithm (such as OCR) to analyze the extracted product information.

[0394] Step 5:

[0395] The server analyzes the extracted product information and obtains information such as the product name, purchase date, price, and store.

[0396] Step 6:

[0397] The server generates an SQL statement to save the product information to the database.

[0398] Step 7:

[0399] The server executes the generated SQL statement and saves the new product information in the database.

[0400] Usage analysis and replenishment forecast

[0401] Step 1:

[0402] The server runs a job that is triggered periodically to retrieve user commodity usage data from the database.

[0403] Step 2:

[0404] The server uses generated AI to analyze the collected data and calculate each user's daily necessities usage rate.

[0405] Step 3:

[0406] The server predicts the next replenishment timing based on the calculation results.

[0407] Step 4:

[0408] The server sets a reminder based on the predicted replenishment timing.

[0409] Step 5:

[0410] The device receives the set reminder and displays a notification to the user.

[0411] Providing discount information and recommended products

[0412] Step 1:

[0413] The server retrieves the user's purchase history and usage information from the database.

[0414] Step 2:

[0415] The server uses a generation AI to analyze the acquired data and generate a list of recommended products for each user.

[0416] Step 3:

[0417] The server retrieves the latest discount information from an external data source.

[0418] Step 4:

[0419] The server combines the recommended product list and discount information to generate information to be provided to the user.

[0420] Step 5:

[0421] The device displays the information received from the server and sends notifications to the user.

[0422] Emotion recognition and personalization with emotion engine

[0423] Step 1:

[0424] Providing information about emotions that users input into the application (e.g., tired, stressed, etc.).

[0425] Step 2:

[0426] The emotion engine analyzes the user's input data and recognizes the user's emotions.

[0427] Step 3:

[0428] The server generates a personalized product suggestion list based on the analysis results of the emotion engine.

[0429] Step 4:

[0430] The server adjusts the content of the reminder to use gentle language and appropriate expressions to match the user's emotions.

[0431] Step 5:

[0432] The device displays personalized suggestions and reminders received from the server and sends notifications to the user.

[0433] Storing and utilizing emotional data

[0434] Step 1:

[0435] The server stores the user's emotion data recognized using the emotion engine in a database.

[0436] Step 2:

[0437] The server periodically analyzes the stored emotional data to understand the user's emotional patterns.

[0438] Step 3:

[0439] The server uses the emotion data to tailor future personalized suggestions and reminders.

[0440] Step 4:

[0441] The server provides product information for stress reduction and relaxation in a timely manner according to the user's emotional state.

[0442] Step 5:

[0443] The device displays the suggestion information based on the emotion received from the server and sends a notification to the user.

[0444] Example 2

[0445] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0446] Conventional daily necessities management systems make it difficult for users to efficiently obtain information on replenishment timing and discounts for daily necessities, and do not provide personalized suggestions based on the user's emotional state. As a result, they have not been able to fully improve the user experience or stimulate purchasing motivation. To solve this, a system that takes into account not only the user's usage status but also their emotional state is needed.

[0447] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0448] In this invention, the server includes a means for a user to manually register daily necessities, a means for automatically registering daily necessities from product photos and receipts, a means for analyzing the usage of the registered daily necessities using a generative AI model, a means for predicting and reminding the user to replenish based on the analysis results, a means for providing personalized discount information and recommended product information to the user, and an emotion engine for recognizing the user's emotions and making personalized suggestions based on those emotions. This makes it possible to streamline the user's daily necessities management and make optimal product suggestions and replenishment notifications based on the user's emotions.

[0449] "User" means an individual or organization that uses the system to manage and register everyday items.

[0450] "Daily commodities" are consumables and products that users use on a daily basis.

[0451] "Manual registration" refers to an operation in which a user directly inputs information about a daily necessities using a terminal.

[0452] "Automatic registration" is a method of automatically extracting and registering information about everyday items from product photos and receipts.

[0453] A "generative AI model" is an algorithm that analyzes the usage of everyday items based on data and generates personalized information.

[0454] The "emotion engine" is a system component that recognizes emotions based on user input and usage and makes corresponding suggestions.

[0455] "Replenishment timing" refers to the next time a user should purchase or replenish daily necessities.

[0456] "Remind" refers to sending notifications or information to users to encourage them to take necessary action.

[0457] "Personalization" refers to customizing information and offers based on a user's individual usage and emotional state.

[0458] "Discount Information" means information about a price reduction applied to a product or service.

[0459] "Recommended Products" are products suggested to users based on their usage and emotional state.

[0460] The "database" is a system that stores information about everyday items registered by users and analysis results.

[0461] "Analysis" is the process of deriving specific information or results from collected data.

[0462] A "Notification" is a message or alert that conveys a reminder or offer to the User.

[0463] This invention combines an emotion engine with a system that allows users to efficiently manage their daily necessities and provides optimal replenishment timing and personalized discount information, thereby realizing personalized suggestions based on user emotion recognition. A detailed description of specific embodiments of this system is provided below.

[0464] 1. Manual and automatic commodity registration

[0465] Users launch the application using a device such as a smartphone or tablet. When registering manually, users enter the product name, purchase date, price, store information, etc. into an input form. The device sends this information in JSON format to the server, which then stores the received data in a database.

[0466] When automatically registering using product photos or receipts, users take a photo and send it to the server from their device. The server then uses an image recognition algorithm (such as OCR) to extract product information from the image and register it in the database.

[0467] Examples:

[0468] When a user purchases shampoo, they take a photo of the receipt. The image is sent to a server, and OCR technology extracts the purchase date, product name, price, etc., and registers them in a database. This allows users to register product information in the system without any hassle.

[0469] 2. Analyzing usage and predicting replenishment timing

[0470] The server periodically analyzes the information about daily necessities stored in the database. Utilizing generative AI, it analyzes each user's usage and consumption rate of daily necessities and predicts when replenishment is necessary. Based on this information, the server reminds the user of the optimal time to replenish.

[0471] Examples:

[0472] The server analyzes the shampoo usage history in the database and finds that, on average, users use up a bottle every three weeks. One week before the next bottle of shampoo runs out, the server sends a reminder to the device, informing the user that the shampoo needs to be refilled.

[0473] 3. Personalized discounts and product recommendations

[0474] The server uses AI to generate personalized product information based on each user's usage and purchase history, and also retrieves the latest discount information and related product information from external data sources to make optimal suggestions for the user.

[0475] Examples:

[0476] The server analyzes the user's shampoo purchase history and retrieves information about new and discounted products from the same brand from external data sources. Based on the information retrieved, a notification is sent to the device stating, "Recommended shampoo is now 20% off," and the user can check the details within the app.

[0477] 4. Emotion Recognition and Personalization with Emotion Engine

[0478] The emotion engine recognizes emotions based on user input and usage, and tailors personalized product recommendations and reminders accordingly.

[0479] Examples:

[0480] If a user types "I'm tired" into the app, the emotion engine will recognize that emotion and suggest products to reduce stress (e.g., relaxing bath salts). If the user's emotion is fatigued, the app will also express the reminder in gentler terms, such as "Thank you for your hard work. Your next shampoo is needed."

[0481] 5. Storage and utilization of emotional data

[0482] The server periodically stores the user's emotional data recognized by the emotion engine in a database, which will be used for future personalized suggestions, such as more personalized product suggestions and tailoring of reminder content.

[0483] Examples:

[0484] Based on the user's emotional history, the system analyzes times when stress is likely to increase and provides focused recommendations on relaxation products and other products that will help relieve stress.

[0485] This system allows users to efficiently manage their daily necessities and receive optimal product suggestions and replenishment notifications tailored to their individual emotional state.

[0486] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0487] Step 1:

[0488] A user launches the application using a device such as a smartphone or tablet. When manually registering a daily necessities item, the user enters the product name, purchase date, price, store information, etc. into an input form. The device sends the entered information in JSON format to the server. The server analyzes the received data and stores it in a database.

[0489] Input: Product name, purchase date, price, store information

[0490] Output: Product information registered in the database

[0491] Step 2:

[0492] The user takes a photo of a product or receipt and sends the image data from their device to the server. The server analyzes the received image data using an image recognition algorithm (such as OCR) and extracts product information. The extracted information is registered in a database.

[0493] Input: Product photo or receipt image data

[0494] Output: Product information registered in the database

[0495] Step 3:

[0496] The server periodically analyzes the information on daily items stored in the database and uses a generative AI model to calculate each user's usage and consumption rate of daily items.

[0497] Input: Daily necessities information in the database

[0498] Output: Analysis of usage pace and wear rate

[0499] Step 4:

[0500] Based on the analysis results, the server predicts when replenishment is necessary, calculates the next replenishment time, and sends a reminder to the user.

[0501] Input: Analysis results of usage pace and wear rate

[0502] Output: Reminder to user

[0503] Step 5:

[0504] The server uses AI to generate personalized product information based on each user's usage and purchase history, and retrieves the latest discount information and related product information from external data sources to make optimal suggestions to users.

[0505] Input: Information from usage, purchase history, and external data sources

[0506] Output: Personalized product and discount information

[0507] Step 6:

[0508] When users input their emotions into the app, the emotion engine analyzes them and tailors personalized suggestions and reminders.

[0509] Input: User emotion input

[0510] Output: Emotion-based personalized suggestions and reminders

[0511] Step 7:

[0512] The server periodically stores the user's emotion data recognized by the emotion engine in a database, which will be used for future personalized suggestions.

[0513] Input: Analysis results by emotion engine

[0514] Output: Emotion data stored in a database

[0515] Step 8:

[0516] Based on the stored emotional data, the server analyzes the periods when stress is likely to increase for each user and adjusts product suggestions and reminder content to suit that period.

[0517] Input: Emotion data stored in a database

[0518] Output: Personalized suggestions and reminders based on specific times

[0519] (Application example 2)

[0520] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0521] In conventional daily necessities management systems, registering users' daily necessities and managing replenishment timings are often done manually, making efficient management difficult. Furthermore, product suggestions based on users' emotions and personalized information provision are not provided, making it difficult to fully address individual user needs. The present invention aims to solve these problems by streamlining daily necessities management and product suggestions for users, and realizing personalized responses based on emotions.

[0522] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for a user to manually register daily necessities, a means for automatically registering daily necessities from product images and receipts, a means for analyzing the usage status of the registered daily necessities using a generation AI, a means for predicting replenishment timing based on the analysis results and reminding the user, a means for providing personalized discount information and recommended product information to the user, a means for recognizing the user's emotions and making personalized product suggestions based on the emotions, and a means for scanning products using smart glasses or a smartphone and automatically registering and suggesting products. This improves the efficiency of the user's daily necessities management and enables product suggestions that meet individual needs based on emotions.

[0523] "Daily necessities" refer to consumables and household items that are frequently used in daily life.

[0524] "Manual registration" means that the user manually enters the information.

[0525] "Product images" refer to photographs or image data of products that are the subject of daily necessities.

[0526] "Receipt" refers to a proof of purchase that details the daily necessities purchased.

[0527] "Automatic registration" refers to a method of automatically extracting and registering product information from product images and receipts using image recognition technology, OCR, etc.

[0528] "Generative AI" refers to artificial intelligence that uses algorithms to generate and analyze data.

[0529] "Analysis" is the process of analyzing data, extracting information, and understanding it.

[0530] "Predicting replenishment timing" refers to analyzing the usage rate of consumables and predicting when the next replenishment will be required.

[0531] "Reminding" means sending a notification to the user prompting them to take necessary action.

[0532] "Personalized Discounts" refers to special discounts provided to you based on your individual needs and preferences.

[0533] "Recommended product information" refers to product information recommended based on a user's past purchase history and usage status.

[0534] "Emotion recognition" refers to technology that determines emotions from a user's facial expressions and input.

[0535] "Personalized product suggestions" refers to suggesting products that are individually suited to the user's emotional state and usage situation.

[0536] "Smart glasses" refers to eyeglass-like devices with built-in computer functions.

[0537] A "smartphone" refers to a highly functional mobile phone that has calling and internet connection capabilities.

[0538] "Scanning a product" refers to reading product information using smart glasses or a smartphone.

[0539] A system for implementing this invention includes a user terminal (smartphone, smart glasses), a server, a network, and a database.

[0540] First, users manually register everyday items using their smartphones or smart glasses. They launch the application and enter information such as the product name, purchase date, and price. This information is then sent to the server in JSON format and stored in a database.

[0541] Next, a method is provided to automatically register everyday items from product images or receipts. Users use their devices to take photos of product images or receipts and send them to the server. The server then uses OCR technology (e.g., Tesseract OCR) to extract product information from the images and register it in a database.

[0542] The server periodically analyzes the usage of registered daily items using AI. For example, it analyzes the pace at which a user consumes a particular daily item and predicts when it will need to be replenished. Based on this information, the server sends reminders to the user to inform them when it is time to replenish.

[0543] In addition, the server provides users with personalized discount information and recommended product information. Based on the user's usage and purchase history, the AI ​​generator selects the most suitable discount information and recommended products, allowing users to obtain the product information that best suits them.

[0544] An emotion engine is a way to recognize a user's emotions and make personalized product recommendations. For example, a user can input "I'm tired" into smart glasses or a smartphone, or read emotions from a facial image. As a result, it can recommend products that reduce stress (e.g., relaxing bath salts or aroma candles).

[0545] It also includes a way for users to scan products in physical stores using smart glasses or smartphones, which will automatically register and suggest products. For example, when a user scans a shampoo in the store, detailed information about the product will be displayed and product suggestions will be made based on emotions.

[0546] This system uses smartphones, smart glasses, and servers as hardware, and OCR technology (Tesseract OCR), generative AI (Keras, etc.), and an emotion engine as software. Data is sent and received over a network, and information is managed in a database.

[0547] Specific examples

[0548] When a user is shopping in a physical store, they use smart glasses to scan the barcode of a shampoo. The system automatically retrieves product information and displays it to the user. If the system recognizes that the user is feeling fatigued, it will suggest relaxing bath salts. Below is an example of a prompt for this process:

[0549] "Recommend the best daily essentials and add-ons based on the user's emotions and purchase history. If the user is stressed, suggest products that will help them relax."

[0550] As described above, this system will improve the efficiency of users' daily necessities management and enable product suggestions that meet individual emotional needs.

[0551] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0552] Step 1:

[0553] Users register everyday items.

[0554] Input: The user manually enters the product name, purchase date, and price, or takes a photo of the product or receipt.

[0555] Processing: In the case of manual entry, the device converts the information entered by the user into JSON format and sends it to the server. In the case of automatic registration, the device takes a photo and sends it to the server.

[0556] Output: In case of manual input, the server stores the received JSON data in the database. In case of automatic registration, the server uses OCR technology to extract product information and stores it in the database.

[0557] Step 2:

[0558] Usage is analyzed using generative AI.

[0559] Input: Registration data of everyday items stored in the database.

[0560] Processing: The server periodically retrieves the registration data from the database and analyzes it using a generative AI model (e.g., using Keras). It analyzes consumption patterns and usage history.

[0561] Output: Analytics showing usage and consumption pace.

[0562] Step 3:

[0563] Predicts replenishment timing and sends reminders.

[0564] Input: The analysis results from step 2.

[0565] Processing: The server predicts when the next replenishment is required based on the consumption pace provided by the generation AI. A reminder notification is generated based on the prediction result.

[0566] Output: A reminder notification is sent to the user's device, with a message such as "Your shampoo needs refilling."

[0567] Step 4:

[0568] Providing personalized discounts and product recommendations.

[0569] Input: User usage and purchase history.

[0570] Processing: The server uses a generative AI model to select optimal discounts and product recommendations based on usage and purchase history, and also retrieves the latest discounts and related product information from external data sources.

[0571] Output: Personalized discount information and product recommendations are sent to the user's device.

[0572] Step 5:

[0573] Recognize emotions and make personalized product recommendations.

[0574] Input: User's face image and text input.

[0575] Processing: The device takes a picture of the user's face and sends it to the server. The server uses an emotion recognition model to determine the emotion from the facial image. The server also recognizes emotions when the user enters their emotion in text. The server then makes appropriate product suggestions based on the emotion.

[0576] Output: Personalized product suggestions based on emotions are displayed on the device.

[0577] Step 6:

[0578] Products are scanned using smart glasses or a smartphone, and are automatically registered and suggested.

[0579] Input: Barcode or QR code of the product in the physical store.

[0580] Processing: The device (smart glasses or smartphone) scans the product and sends the product information to the server. The server analyzes the product information, registers it in a database, and suggests appropriate products based on the user's emotional data.

[0581] Output: Product details and recommended product information are displayed on the device.

[0582] In this way, the system can streamline users' daily necessities management and provide personalized, emotion-based suggestions.

[0583] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0584] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0585] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0586] [Second embodiment]

[0587] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0588] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0589] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0590] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0591] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0592] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0593] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0594] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0595] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0596] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0597] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0598] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0599] The present invention is a system that allows users to efficiently manage their daily necessities and provides optimal replenishment timing and personalized discount information. This system includes a means for users to manually register daily necessities, a means for automatically registering items from product photos and receipts, a means for analyzing usage status using generation AI, a means for predicting and reminding replenishment timing, and a means for providing personalized discount information and recommended product information.

[0600] Program processing

[0601] 1. How users register everyday items

[0602] Users launch the application using a device such as a smartphone or tablet. When manually registering daily necessities, users enter the product name, purchase date, price, store information, etc. into an input form. The device sends this information in JSON format to the server, which then stores the received data in a database.

[0603] Users can also take photos of receipts and products and register them. The images are sent from the device to a server, where they are analyzed using an image recognition algorithm (e.g., OCR technology). The server then extracts product information from the images and registers it in a database.

[0604] Examples:

[0605] When a user purchases shampoo, they take a photo of the receipt. The image is sent to a server, and OCR technology extracts the purchase date, product name, price, etc., and registers them in a database. This allows users to register product information in the system without any hassle.

[0606] 2. Usage analysis

[0607] The server periodically analyzes the information about daily necessities stored in the database. Utilizing generative AI, it analyzes each user's usage and consumption rate of daily necessities and predicts when replenishment is necessary. Based on this information, the server reminds the user of the optimal time to replenish.

[0608] Examples:

[0609] The server analyzes the shampoo usage history in the database and finds that, on average, users use up a bottle every three weeks. One week before the next bottle of shampoo runs out, the server sends a reminder to the device, informing the user that the shampoo needs to be refilled.

[0610] 3. Providing discount information and recommended products

[0611] The server uses AI to generate personalized product information based on each user's usage and purchase history, and also retrieves the latest discount information and related product information from external data sources to make optimal suggestions for the user.

[0612] Examples:

[0613] The server analyzes the user's shampoo purchase history and retrieves information about new and discounted products from the same brand from external data sources. Based on the information retrieved, a notification is sent to the device stating, "Recommended shampoo is now 20% off," and the user can check the details within the app.

[0614] Implementation environment

[0615] The system operates in a network environment that includes user devices (smartphones, tablets, etc.), a server, and a database. Users use an application to register their daily necessities and send the necessary information to the server. The server stores the received information in a database, periodically analyzes the data, and uses generative AI to provide optimal replenishment timing and product information.

[0616] This configuration allows users to manage their daily necessities more efficiently, supporting them in replenishing their supplies when needed and making purchases at a good price.

[0617] The processing flow will be explained below.

[0618] Program processing (specific steps)

[0619] Manually inputting daily necessities

[0620] Step 1:

[0621] A user launches an application on their smartphone or tablet.

[0622] Step 2:

[0623] The user taps the "Register a new product" button and enters information such as the product name, purchase date, price, and store.

[0624] Step 3:

[0625] Check the information entered by the user and tap the "Register" button.

[0626] Step 4:

[0627] The terminal sends the entered data to the server in JSON format.

[0628] Step 5:

[0629] The server analyzes the received JSON data and generates an SQL statement to save it in the database.

[0630] Step 6:

[0631] The server executes the generated SQL statement and saves the new product information in the database.

[0632] Automatically register daily necessities from product photos and receipts

[0633] Step 1:

[0634] A user launches an application on their smartphone or tablet.

[0635] Step 2:

[0636] The user taps the "Scan Receipt" button and takes a photo of the receipt or item.

[0637] Step 3:

[0638] The device temporarily stores the captured image and sends the image data to the server.

[0639] Step 4:

[0640] The server receives the image and runs an image recognition algorithm (such as OCR) to extract product information.

[0641] Step 5:

[0642] The server analyzes the extracted product information and obtains information such as product name, purchase date, price, and store.

[0643] Step 6:

[0644] The server generates an SQL statement to save the product information to the database.

[0645] Step 7:

[0646] The server executes the generated SQL statement and saves the new product information in the database.

[0647] Usage analysis and replenishment forecast

[0648] Step 1:

[0649] The server runs a job that is triggered periodically to retrieve user commodity usage data from the database.

[0650] Step 2:

[0651] The server uses generated AI to analyze the acquired data and calculate each user's daily necessities usage rate.

[0652] Step 3:

[0653] The server predicts the next replenishment timing based on the calculation results.

[0654] Step 4:

[0655] The server sets a reminder based on the predicted replenishment timing.

[0656] Step 5:

[0657] The device receives the set reminder and displays a notification to the user.

[0658] Providing discount information and recommended products

[0659] Step 1:

[0660] The server retrieves the user's purchase history and usage information from the database.

[0661] Step 2:

[0662] The server uses generated AI to analyze the acquired data and generate a list of recommended products for each user.

[0663] Step 3:

[0664] The server retrieves the latest discount information from an external data source.

[0665] Step 4:

[0666] The server combines the recommended product list and discount information to generate information to be provided to the user.

[0667] Step 5:

[0668] The device displays the information received from the server and sends notifications to the user.

[0669] Example 1

[0670] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0671] Today's busy lives make it difficult for users to properly manage their daily necessities and determine when to replenish them. Furthermore, obtaining optimal discount information and recommended product information takes time and effort. To address these issues, a system is needed that can efficiently manage daily necessities, provide optimal replenishment timing, and provide users with personalized, useful information.

[0672] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0673] In this invention, the server includes: a means for a user to launch an application using a smart device and manually register information about daily necessities; a means for converting the manually entered data into JSON format and sending it to the server; a means for the server to store the received data in a database; a means for sending receipts and product photos to the server and analyzing the images using OCR technology; a means for the server to register product information extracted from the images in the database; a means for periodically analyzing the database data using a generative AI model to analyze consumption rates; a means for predicting replenishment timing and sending reminders to the terminal; a means for obtaining discount information and related product information from external data sources and generating personalized suggestions; and a means for sending discount information and recommended product information to the user's terminal. This enables users to efficiently manage and replenish daily necessities and receive personalized, useful product information and discount information.

[0674] "User" means an individual or corporation that uses the system to register and manage everyday items.

[0675] A "smart device" is a device, such as a smartphone or tablet, that can connect to the Internet and run applications.

[0676] "Application" means a software program that a user uses to register and manage everyday item information on a smart device.

[0677] "Daily necessities" are products that users use on a daily basis and consume, such as shampoo, detergent, and toilet paper.

[0678] "Manual input" refers to the act of a user directly entering information into an application's input form using a smart device.

[0679] "JSON format" stands for JavaScript Object Notation and is a lightweight data format for structuring and exchanging data.

[0680] "Server" means a computer system that receives and processes data sent by users and stores it in a database.

[0681] A "database" is a collection of data managed by a server, a system that allows for efficient information storage and retrieval.

[0682] A "receipt" is a paper or electronic record that contains details of a purchased item (e.g., item name, price, purchase date, etc.).

[0683] "OCR technology" is an abbreviation for Optical Character Recognition, and is a technology that automatically recognizes character information in an image and converts it into text data.

[0684] A "generative AI model" is an artificial intelligence model that uses technologies such as deep learning to learn patterns from data and make predictions and classifications.

[0685] A "reminder" is a notification sent by the server to a user's device, and is a message used to encourage a specific action.

[0686] "External data sources" are information sources from outside the system, including web services and databases that provide up-to-date discount and product information.

[0687] "Personalized suggestions" are product and service suggestions that are individually generated based on a user's usage and purchase history.

[0688] This invention is a system that allows users to efficiently manage daily necessities and provides optimal replenishment timing and personalized discount information. The system includes a means for users to manually register daily necessities, a means for automatically registering daily necessities from product photos and receipts, a means for analyzing usage status using a generative AI model, a means for predicting and reminding replenishment timing, and a means for providing personalized discount information and recommended product information.

[0689] System Configuration

[0690] Hardware

[0691] User device: A device that can connect to the internet, such as a smartphone or tablet.

[0692] Server: A computer system that receives and processes data and accesses a database.

[0693] Database: A collection of data managed by a server that stores information about everyday items.

[0694] software

[0695] Application: Software that users use on their devices to register and manage everyday items.

[0696] OCR technology: Software that analyzes photos of receipts or products and extracts text information. Example: Tesseract OCR.

[0697] Generative AI model: An artificial intelligence model that analyzes data and predicts replenishment timing. Example: TensorFlow.

[0698] Overview of operation procedure

[0699] Daily necessities registration

[0700] 1. Manual registration:

[0701] Users launch the application using their smartphone or tablet and manually enter information about the daily necessities they purchased (product name, purchase date, price, store information, etc.). After completing the entry, they press the "Register" button.

[0702] The terminal converts the input data into JSON format and sends it to the server.

[0703] The server stores the received data in a database.

[0704] Examples:

[0705] When a user enters shampoo purchase information into the input form and clicks the "Register" button, the device converts the input data into JSON format and sends it to the server, which then analyzes the received data and stores it in a database.

[0706] Example prompt sentence:

[0707] Please manually enter the shampoo information you purchased (product name, purchase date, price, store information) using your smartphone.

[0708] 2. Registration from an image:

[0709] The user takes a photo of the product or receipt of the purchased item, and then sends the image data to the server via the application.

[0710] The terminal transmits the captured image data to the server.

[0711] The server analyzes the received image data using OCR technology and registers the product information extracted from the image in a database.

[0712] Examples:

[0713] When a user takes a photo of a receipt using their smartphone camera and sends the image from the app to the server, the server activates the OCR engine to extract text data from the image, analyzes the extracted text data, and stores it in a database.

[0714] Example prompt sentence:

[0715] "Take a photo of the receipt or item you purchased and submit it through the application."

[0716] Usage analytics and reminders

[0717] The server periodically analyzes the information on daily items stored in the database, utilizing a generative AI model to analyze each user's usage and consumption rate of daily items.

[0718] Based on the analysis results, the server predicts when replenishment is needed and sends a reminder to the user's device.

[0719] Examples:

[0720] The server analyzes the shampoo usage history in the database and finds that, on average, users use up a bottle every three weeks. One week before the next bottle runs out, the server sends a reminder to the device saying, "You need to refill your shampoo."

[0721] Providing discount information and recommended products

[0722] The server generates personalized product information using a generative AI model based on each user's usage and purchase history, and also retrieves the latest discount information and related product information from external data sources to make optimal suggestions for the user.

[0723] The server transmits the generated discount information and recommended product information to the user's terminal.

[0724] Examples:

[0725] The server analyzes the user's shampoo purchase history and retrieves information about new and discounted products from the same brand from external data sources. Based on the information retrieved, a notification is sent to the device stating, "Recommended shampoo is now 20% off," and the user can check the details within the app.

[0726] This system helps users manage their daily necessities more efficiently, supporting them in replenishing their supplies when needed and making purchases at a good price.

[0727] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0728] Daily necessities registration phase

[0729] Step 1:

[0730] The user launches the application using a smart device and manually inputs information about the purchased daily necessities (product name, purchase date, price, store information, etc.). Once the input is complete, the user presses the "Register" button. The input data includes the product name, purchase date, price, and store information.

[0731] Specific behavior:

[0732] The user enters shampoo purchase information into the input form and clicks the "Register" button.

[0733] input:

[0734] Product information entered by the user (product name, purchase date, price, store information)

[0735] output:

[0736] JSON format data on the device

[0737] Step 2:

[0738] The terminal converts the manually entered data into JSON format, which includes the product name, purchase date, price, and store information.

[0739] Specific behavior:

[0740] The terminal converts the input data into JSON format.

[0741] input:

[0742] Product information entered by the user

[0743] output:

[0744] JSON format data

[0745] Step 3:

[0746] The terminal sends the converted JSON format data to the server.

[0747] Specific behavior:

[0748] The device starts the process of sending JSON data to the server.

[0749] input:

[0750] Structured product data in JSON format

[0751] output:

[0752] JSON data sent to the server

[0753] Step 4:

[0754] The server receives the JSON data sent from the device, analyzes it, and stores it in a database.

[0755] Specific behavior:

[0756] The server stores the received data in a database.

[0757] input:

[0758] JSON format data

[0759] output:

[0760] Information on everyday items in the database

[0761] Image information registration phase

[0762] Step 1:

[0763] The user takes a photo of the product or receipt of the purchased item, and then sends the image data to the server via the application.

[0764] Specific behavior:

[0765] The user takes a photo of the receipt using their smartphone camera and the app sends the image to the server.

[0766] input:

[0767] Receipts and product photos

[0768] output:

[0769] Image data stored on the device

[0770] Step 2:

[0771] The terminal transmits the captured image data to the server. The transmitted data is an image that may contain product information.

[0772] Specific behavior:

[0773] The device starts the process of sending the captured image to the server.

[0774] input:

[0775] Image data

[0776] output:

[0777] Image data sent to the server

[0778] Step 3:

[0779] The server analyzes the received image data using OCR technology and registers the product information extracted from the image (product name, purchase date, price, etc.) in a database.

[0780] Specific behavior:

[0781] The server starts the OCR engine, extracts text data from the image, analyzes the extracted text data, and stores it in a database.

[0782] input:

[0783] Image data

[0784] output:

[0785] Product information stored in the database

[0786] Usage analysis phase

[0787] Step 1:

[0788] The server periodically analyzes the information stored in the database and uses a generative AI model to analyze the usage and consumption rate of each item for each user.

[0789] Specific behavior:

[0790] The server runs scheduled jobs and performs data analysis using generative AI models.

[0791] input:

[0792] Information on everyday items in the database

[0793] output:

[0794] Usage and consumption rate per user

[0795] Step 2:

[0796] Based on the analysis results, the server predicts when each user's daily necessities will need to be replenished.

[0797] Specific behavior:

[0798] The server calculates the timing of replenishment based on the analysis results.

[0799] input:

[0800] AI model analysis results

[0801] output:

[0802] Predicted replenishment timing

[0803] Step 3:

[0804] The server sends a reminder to the user's device when the predicted replenishment time approaches.

[0805] Specific behavior:

[0806] The server generates a reminder notification and sends it to the device.

[0807] input:

[0808] Predicted replenishment timing

[0809] output:

[0810] Reminder messages sent to your device

[0811] Discount information and recommended products phase

[0812] Step 1:

[0813] The server generates personalized product information using a generative AI model based on each user's usage and purchase history, and also retrieves the latest discount information and related product information from external data sources.

[0814] Specific behavior:

[0815] The server periodically calls an external API to retrieve the latest discount information.

[0816] input:

[0817] User usage, purchase history, external data

[0818] output:

[0819] Personalized product information

[0820] Step 2:

[0821] Based on the collected data, the server generates and stores discount information and recommended product information that is optimal for the user.

[0822] Specific behavior:

[0823] The server applies an algorithm based on the information it obtains to generate personalized suggestions.

[0824] input:

[0825] Data analysis results, external data

[0826] output:

[0827] Saved discounts and recommended products

[0828] Step 3:

[0829] The server transmits the generated discount information and recommended product information to the user's terminal.

[0830] Specific behavior:

[0831] The server generates the proposal as a notification and sends it to the device.

[0832] input:

[0833] Generated proposals

[0834] output:

[0835] Notification messages sent to the device

[0836] (Application example 1)

[0837] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0838] Conventional daily necessities management systems have the drawback of making it difficult to properly determine when to replenish items because they require users to register their purchases and do not adequately analyze usage.Furthermore, they do not provide users with personalized discount information or recommended products, making it difficult to efficiently manage daily necessities and purchase the products they need at the optimal time.

[0839] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0840] In this invention, the server includes a means for users to manually register daily necessities, a means for automatically registering daily necessities from product photos and receipts, and a means for analyzing the usage of registered daily necessities using a generating AI. This allows users to easily register daily necessities, and the analysis of usage using the generating AI makes it possible to predict the optimal timing for replenishment and send reminders. Furthermore, the server includes a means for predicting the usage rate and replenishment timing using a generating AI model, a means for extracting and automatically registering product information using image recognition, and a means for obtaining and providing discount information and recommended product information from external data sources. This allows users to efficiently manage their daily necessities and purchase products at great prices based on personalized discount information.

[0841] "Means for manually registering daily necessities" is a function that allows users to manually enter product name, purchase date, price, store information, etc. to register daily necessities in the system.

[0842] "Means for automatically registering daily necessities from product photos and receipts" is a function that analyzes product photos and receipt images taken by users, automatically extracts product information, and registers it in the system.

[0843] "Means for analyzing usage status using generative AI" refers to a function that uses a generative AI model to analyze information on stored daily necessities, and analyzes the pace of use and consumption of daily necessities for each user.

[0844] "Means to predict and remind the timing of replenishment" is a function that predicts the next time replenishment is required based on the results of analysis by the generation AI and notifies the user.

[0845] "Means for providing personalized discount information and recommended product information" refers to a function that uses generative AI to provide optimal discount information and related product information based on each user's usage and purchase history.

[0846] "Means for predicting usage rate and replenishment timing using a generative AI model" is a function that uses a generative AI model to predict the usage rate and optimal replenishment timing of registered daily necessities.

[0847] "Means for extracting and automatically registering product information using image recognition" refers to a function that uses OCR technology or image recognition algorithms to extract product information from receipt or product images and automatically registers it in the system.

[0848] "Means for obtaining and providing discount information and recommended product information from external data sources" refers to a function that analyzes the latest discount information and recommended product information obtained from external data sources and provides it to users in a personalized form.

[0849] MODE FOR CARRYING OUT THE INVENTION

[0850] The present invention is a system that allows users to efficiently manage their daily necessities, and provides optimal replenishment timing and personalized discount information. This system operates using a smartphone application, a server, and a database.

[0851] System configuration

[0852] User Device

[0853] Users launch the application using a device such as a smartphone or tablet. There are two ways to register daily necessities: manual entry or automatic registration.

[0854] 1. Manual registration

[0855] The user enters the product name, purchase date, price, store information, etc. into an input form and sends this information in JSON format to the server.

[0856] 2. Automatic Registration

[0857] Users take photos of receipts or products and send the images to the server, which uses OCR technology to extract product information from the images and register it in a database.

[0858] Servers and Databases

[0859] 1. Usage analysis

[0860] The server uses a generative AI model to analyze information about daily necessities stored in a database, and analyzes each user's pace of use and consumption of daily necessities.

[0861] 2. Predicting replenishment timing

[0862] Based on the analysis results, the server predicts when the next replenishment is required and sends a reminder to the user.

[0863] 3. Providing discount information and recommended products

[0864] The server retrieves the latest discount information and related product information from external data sources and generates personalized product information using a generative AI model.

[0865] Based on this information, the server provides users with optimal discount information and related product information.

[0866] Specific examples of implementation methods

[0867] Manual registration example

[0868] When a user purchases shampoo, they manually enter product information using the application, including details such as the purchase date, product name, and price, and send the information to the server, which then registers the received information in the database.

[0869] Auto-registration Example

[0870] The user takes a photo of the receipt and sends it to the server via the application. The server uses OCR technology to extract the purchase date, product name, price, etc. from the image and registers them in the database. This allows the user to register product information without any hassle.

[0871] Usage analysis example

[0872] The server analyzes the shampoo usage history stored in the database and finds that the average user uses up a bottle every three weeks. One week before the next bottle of shampoo runs out, the server sends a reminder to the user's smartphone, informing them that they need to refill their shampoo.

[0873] Examples of discounts and product recommendations

[0874] The server analyzes the user's shampoo purchase history and retrieves information about new and discounted products from the same brand from external data sources. Based on the information retrieved, the server sends the user a notification saying, "Recommended shampoo is now 20% off," and allows them to check the details within the app.

[0875] Examples of using generative AI models and prompts

[0876] An example of a prompt that the server uses to perform analysis using a generative AI model:

[0877] This system is designed to help you efficiently manage your daily necessities, provide optimal replenishment timing, and provide personalized discount information. Simply take a photo of your receipt or product and register it in the system, and you will receive timely replenishment notifications and discount information.

[0878] This allows users to efficiently manage their daily necessities and purchase the products they need at the best possible time.

[0879] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0880] Step 1:

[0881] Users manually register daily items

[0882] The user launches the smartphone application and enters the product name, purchase date, price, store information, etc. into the input form. The entered information is sent to the server in JSON format. The server parses the received JSON data and registers it in the database.

[0883] Input: Product name, purchase date, price, store information (user input)

[0884] Output: Product information is saved in the database

[0885] Step 2:

[0886] Users take photos of receipts and products and register them

[0887] A user uses a smartphone application to take a photo of a receipt or product. The image data is then uploaded to a server. The server uses OCR technology to extract text information from the image and convert it into data such as product name, purchase date, price, and store information. This data is then registered in a database.

[0888] Input: Image of receipt or product (taken by user)

[0889] Output: The extracted product information is saved in the database.

[0890] Step 3:

[0891] The server analyzes the usage of everyday items

[0892] The server periodically analyzes the information on daily items stored in the database using a generative AI model, which calculates each user's usage rate and consumption rate of daily items.

[0893] Input: Information about everyday items stored in a database

[0894] Output: Analysis results of usage pace and wear rate (analysis results from generative AI model)

[0895] Step 4:

[0896] Refill forecast and reminder sending

[0897] The server predicts when the next refill is needed based on the analysis results of the generative AI model. Based on the prediction, a reminder notification is sent to the user's smartphone. For example, if shampoo runs out within one week of the predicted time, the user will be notified that "shampoo needs to be refilled."

[0898] Input: Analysis results of the generative AI model

[0899] Output: Reminder notification for refilling (sent to user's smartphone)

[0900] Step 5:

[0901] Providing discount information and recommended products

[0902] The server uses a generative AI model based on each user's usage and purchase history to generate personalized product information. It also obtains discount information and recommended product information from external data sources and stores it in a database. The generated information is then combined to notify users of appropriate discount information and recommended products.

[0903] Inputs: User usage, purchase history, discount information from external data sources

[0904] Output: Notification of recommended products and discount information (sent to the user's smartphone)

[0905] This allows users to efficiently manage their daily necessities and replenish them at the optimal time and at a good price.

[0906] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0907] This invention combines an emotion engine with a system that allows users to efficiently manage their daily necessities and provides optimal replenishment timing and personalized discount information, thereby realizing personalized suggestions based on user emotion recognition. This system includes a means for users to manually register daily necessities, a means for automatically registering items from product photos and receipts, a means for analyzing usage status using generative AI, a means for predicting and reminding users when replenishment is necessary, a means for providing personalized discount information and recommended product information, and a means for recognizing user emotions using the emotion engine.

[0908] Program processing

[0909] 1. Manual and automatic commodity registration

[0910] Users launch the application using a device such as a smartphone or tablet. When registering manually, users enter the product name, purchase date, price, store information, etc. into an input form. The device sends this information in JSON format to the server, which then stores the received data in a database.

[0911] When automatically registering using product photos or receipts, users take a photo and send it to the server from their device. The server then uses an image recognition algorithm (such as OCR) to extract product information from the image and register it in the database.

[0912] Examples:

[0913] When a user purchases shampoo, they take a photo of the receipt. The image is sent to a server, and OCR technology extracts the purchase date, product name, price, etc., and registers them in a database. This allows users to register product information in the system without any hassle.

[0914] 2. Analyzing usage and predicting replenishment timing

[0915] The server periodically analyzes the information about daily necessities stored in the database. Utilizing generative AI, it analyzes each user's usage and consumption rate of daily necessities and predicts when replenishment is necessary. Based on this information, the server reminds the user of the optimal time to replenish.

[0916] Examples:

[0917] The server analyzes the shampoo usage history in the database and finds that, on average, users use up a bottle every three weeks. One week before the next bottle of shampoo runs out, the server sends a reminder to the device, informing the user that the shampoo needs to be refilled.

[0918] 3. Personalized discounts and product recommendations

[0919] The server uses AI to generate personalized product information based on each user's usage and purchase history, and also retrieves the latest discount information and related product information from external data sources to make optimal suggestions for the user.

[0920] Examples:

[0921] The server analyzes the user's shampoo purchase history and retrieves information about new and discounted products from the same brand from external data sources. Based on the information retrieved, a notification is sent to the device stating, "Recommended shampoo is now 20% off," and the user can check the details within the app.

[0922] 4. Emotion Recognition and Personalization with Emotion Engine

[0923] The emotion engine recognizes emotions based on user input and usage, and tailors personalized product recommendations and reminders accordingly.

[0924] Examples:

[0925] If a user types "I'm tired" into the app, the emotion engine will recognize that emotion and suggest products to reduce stress (e.g., relaxing bath salts). If the user's emotion is fatigued, the app will also express the reminder in gentler terms, such as "Thank you for your hard work. Your next shampoo is needed."

[0926] 5. Storage and utilization of emotional data

[0927] The server periodically stores the user's emotional data recognized by the emotion engine in a database, which will be used for future personalized suggestions, such as more personalized product suggestions and tailoring of reminder content.

[0928] Examples:

[0929] Based on the user's emotional history, the system analyzes times when stress is likely to increase and provides focused recommendations on relaxation products and other products that will help relieve stress.

[0930] Implementation environment

[0931] The system operates in a network environment that includes user devices (smartphones, tablets, etc.), a server, and a database. Users use an application to register their daily necessities and send the necessary information to the server. The server stores the received information in a database and analyzes it using generative AI and an emotion engine to provide optimal replenishment timing and personalized product information.

[0932] This configuration allows for more efficient management of daily necessities for users, and makes it possible to provide optimal product suggestions and replenishment notifications tailored to each individual's emotional state.

[0933] The processing flow will be explained below.

[0934] Manually inputting daily necessities

[0935] Step 1:

[0936] A user launches an application on their smartphone or tablet.

[0937] Step 2:

[0938] The user taps the "Register a new product" button and enters information such as the product name, purchase date, price, and store.

[0939] Step 3:

[0940] Check the information entered by the user and tap the "Register" button.

[0941] Step 4:

[0942] The terminal sends the entered data to the server in JSON format.

[0943] Step 5:

[0944] The server analyzes the received JSON data and generates an SQL statement to save it in the database.

[0945] Step 6:

[0946] The server executes the generated SQL statement and saves the new product information in the database.

[0947] Automatically register daily necessities from product photos and receipts

[0948] Step 1:

[0949] A user launches an application on their smartphone or tablet.

[0950] Step 2:

[0951] The user taps the "Scan Receipt" button and takes a photo of the receipt or item.

[0952] Step 3:

[0953] The device temporarily stores the captured image and sends the image data to the server.

[0954] Step 4:

[0955] The server receives the image and runs an image recognition algorithm (such as OCR) to analyze the extracted product information.

[0956] Step 5:

[0957] The server analyzes the extracted product information and obtains information such as the product name, purchase date, price, and store.

[0958] Step 6:

[0959] The server generates an SQL statement to save the product information to the database.

[0960] Step 7:

[0961] The server executes the generated SQL statement and saves the new product information in the database.

[0962] Usage analysis and replenishment forecast

[0963] Step 1:

[0964] The server runs a job that is triggered periodically to retrieve user commodity usage data from the database.

[0965] Step 2:

[0966] The server uses generated AI to analyze the collected data and calculate each user's daily necessities usage rate.

[0967] Step 3:

[0968] The server predicts the next replenishment timing based on the calculation results.

[0969] Step 4:

[0970] The server sets a reminder based on the predicted replenishment timing.

[0971] Step 5:

[0972] The device receives the set reminder and displays a notification to the user.

[0973] Providing discount information and recommended products

[0974] Step 1:

[0975] The server retrieves the user's purchase history and usage information from the database.

[0976] Step 2:

[0977] The server uses a generation AI to analyze the acquired data and generate a list of recommended products for each user.

[0978] Step 3:

[0979] The server retrieves the latest discount information from an external data source.

[0980] Step 4:

[0981] The server combines the recommended product list and discount information to generate information to be provided to the user.

[0982] Step 5:

[0983] The device displays the information received from the server and sends notifications to the user.

[0984] Emotion recognition and personalization with emotion engine

[0985] Step 1:

[0986] Providing information about emotions that users input into the application (e.g., tired, stressed, etc.).

[0987] Step 2:

[0988] The emotion engine analyzes the user's input data and recognizes the user's emotions.

[0989] Step 3:

[0990] The server generates a personalized product suggestion list based on the analysis results of the emotion engine.

[0991] Step 4:

[0992] The server adjusts the content of the reminder to use gentle language and appropriate expressions to match the user's emotions.

[0993] Step 5:

[0994] The device displays personalized suggestions and reminders received from the server and sends notifications to the user.

[0995] Storing and utilizing emotional data

[0996] Step 1:

[0997] The server stores the user's emotion data recognized using the emotion engine in a database.

[0998] Step 2:

[0999] The server periodically analyzes the stored emotional data to understand the user's emotional patterns.

[1000] Step 3:

[1001] The server uses the emotion data to tailor future personalized suggestions and reminders.

[1002] Step 4:

[1003] The server provides product information for stress reduction and relaxation in a timely manner according to the user's emotional state.

[1004] Step 5:

[1005] The device displays the suggestion information based on the emotion received from the server and sends a notification to the user.

[1006] Example 2

[1007] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1008] Conventional daily necessities management systems make it difficult for users to efficiently obtain information on replenishment timing and discounts for daily necessities, and do not provide personalized suggestions based on the user's emotional state. As a result, they have not been able to fully improve the user experience or stimulate purchasing motivation. To solve this, a system that takes into account not only the user's usage status but also their emotional state is needed.

[1009] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1010] In this invention, the server includes a means for a user to manually register daily necessities, a means for automatically registering daily necessities from product photos and receipts, a means for analyzing the usage of the registered daily necessities using a generative AI model, a means for predicting and reminding the user to replenish based on the analysis results, a means for providing personalized discount information and recommended product information to the user, and an emotion engine for recognizing the user's emotions and making personalized suggestions based on those emotions. This makes it possible to streamline the user's daily necessities management and make optimal product suggestions and replenishment notifications based on the user's emotions.

[1011] "User" means an individual or organization that uses the system to manage and register everyday items.

[1012] "Daily commodities" are consumables and products that users use on a daily basis.

[1013] "Manual registration" refers to an operation in which a user directly inputs information about a daily necessities using a terminal.

[1014] "Automatic registration" is a method of automatically extracting and registering information about everyday items from product photos and receipts.

[1015] A "generative AI model" is an algorithm that analyzes the usage of everyday items based on data and generates personalized information.

[1016] The "emotion engine" is a system component that recognizes emotions based on user input and usage and makes corresponding suggestions.

[1017] "Replenishment timing" refers to the next time a user should purchase or replenish daily necessities.

[1018] "Remind" refers to sending notifications or information to users to encourage them to take necessary action.

[1019] "Personalization" refers to customizing information and offers based on a user's individual usage and emotional state.

[1020] "Discount Information" means information about a price reduction applied to a product or service.

[1021] "Recommended Products" are products suggested to users based on their usage and emotional state.

[1022] The "database" is a system that stores information about everyday items registered by users and analysis results.

[1023] "Analysis" is the process of deriving specific information or results from collected data.

[1024] A "Notification" is a message or alert that conveys a reminder or offer to the User.

[1025] This invention combines an emotion engine with a system that allows users to efficiently manage their daily necessities and provides optimal replenishment timing and personalized discount information, thereby realizing personalized suggestions based on user emotion recognition. A detailed description of specific embodiments of this system is provided below.

[1026] 1. Manual and automatic commodity registration

[1027] Users launch the application using a device such as a smartphone or tablet. When registering manually, users enter the product name, purchase date, price, store information, etc. into an input form. The device sends this information in JSON format to the server, which then stores the received data in a database.

[1028] When automatically registering using product photos or receipts, users take a photo and send it to the server from their device. The server then uses an image recognition algorithm (such as OCR) to extract product information from the image and register it in the database.

[1029] Examples:

[1030] When a user purchases shampoo, they take a photo of the receipt. The image is sent to a server, and OCR technology extracts the purchase date, product name, price, etc., and registers them in a database. This allows users to register product information in the system without any hassle.

[1031] 2. Analyzing usage and predicting replenishment timing

[1032] The server periodically analyzes the information about daily necessities stored in the database. Utilizing generative AI, it analyzes each user's usage and consumption rate of daily necessities and predicts when replenishment is necessary. Based on this information, the server reminds the user of the optimal time to replenish.

[1033] Examples:

[1034] The server analyzes the shampoo usage history in the database and finds that, on average, users use up a bottle every three weeks. One week before the next bottle of shampoo runs out, the server sends a reminder to the device, informing the user that the shampoo needs to be refilled.

[1035] 3. Personalized discounts and product recommendations

[1036] The server uses AI to generate personalized product information based on each user's usage and purchase history, and also retrieves the latest discount information and related product information from external data sources to make optimal suggestions for the user.

[1037] Examples:

[1038] The server analyzes the user's shampoo purchase history and retrieves information about new and discounted products from the same brand from external data sources. Based on the information retrieved, a notification is sent to the device stating, "Recommended shampoo is now 20% off," and the user can check the details within the app.

[1039] 4. Emotion Recognition and Personalization with Emotion Engine

[1040] The emotion engine recognizes emotions based on user input and usage, and tailors personalized product recommendations and reminders accordingly.

[1041] Examples:

[1042] If a user types "I'm tired" into the app, the emotion engine will recognize that emotion and suggest products to reduce stress (e.g., relaxing bath salts). If the user's emotion is fatigued, the app will also express the reminder in gentler terms, such as "Thank you for your hard work. Your next shampoo is needed."

[1043] 5. Storage and utilization of emotional data

[1044] The server periodically stores the user's emotional data recognized by the emotion engine in a database, which will be used for future personalized suggestions, such as more personalized product suggestions and tailoring of reminder content.

[1045] Examples:

[1046] Based on the user's emotional history, the system analyzes times when stress is likely to increase and provides focused recommendations on relaxation products and other products that will help relieve stress.

[1047] This system allows users to efficiently manage their daily necessities and receive optimal product suggestions and replenishment notifications tailored to their individual emotional state.

[1048] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1049] Step 1:

[1050] A user launches the application using a device such as a smartphone or tablet. When manually registering a daily necessities item, the user enters the product name, purchase date, price, store information, etc. into an input form. The device sends the entered information in JSON format to the server. The server analyzes the received data and stores it in a database.

[1051] Input: Product name, purchase date, price, store information

[1052] Output: Product information registered in the database

[1053] Step 2:

[1054] The user takes a photo of a product or receipt and sends the image data from their device to the server. The server analyzes the received image data using an image recognition algorithm (such as OCR) and extracts product information. The extracted information is registered in a database.

[1055] Input: Product photo or receipt image data

[1056] Output: Product information registered in the database

[1057] Step 3:

[1058] The server periodically analyzes the information on daily items stored in the database and uses a generative AI model to calculate each user's usage and consumption rate of daily items.

[1059] Input: Daily necessities information in the database

[1060] Output: Analysis of usage pace and wear rate

[1061] Step 4:

[1062] Based on the analysis results, the server predicts when replenishment is necessary, calculates the next replenishment time, and sends a reminder to the user.

[1063] Input: Analysis results of usage pace and wear rate

[1064] Output: Reminder to user

[1065] Step 5:

[1066] The server uses AI to generate personalized product information based on each user's usage and purchase history, and retrieves the latest discount information and related product information from external data sources to make optimal suggestions to users.

[1067] Input: Information from usage, purchase history, and external data sources

[1068] Output: Personalized product and discount information

[1069] Step 6:

[1070] When users input their emotions into the app, the emotion engine analyzes them and tailors personalized suggestions and reminders.

[1071] Input: User emotion input

[1072] Output: Emotion-based personalized suggestions and reminders

[1073] Step 7:

[1074] The server periodically stores the user's emotion data recognized by the emotion engine in a database, which will be used for future personalized suggestions.

[1075] Input: Analysis results by emotion engine

[1076] Output: Emotion data stored in a database

[1077] Step 8:

[1078] Based on the stored emotional data, the server analyzes the periods when stress is likely to increase for each user and adjusts product suggestions and reminder content to suit that period.

[1079] Input: Emotion data stored in a database

[1080] Output: Personalized suggestions and reminders based on specific times

[1081] (Application example 2)

[1082] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1083] In conventional daily necessities management systems, registering users' daily necessities and managing replenishment timings are often done manually, making efficient management difficult. Furthermore, product suggestions based on users' emotions and personalized information provision are not provided, making it difficult to fully address individual user needs. The present invention aims to solve these problems by streamlining daily necessities management and product suggestions for users, and realizing personalized responses based on emotions.

[1084] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for a user to manually register daily necessities, a means for automatically registering daily necessities from product images and receipts, a means for analyzing the usage status of the registered daily necessities using a generation AI, a means for predicting replenishment timing based on the analysis results and reminding the user, a means for providing personalized discount information and recommended product information to the user, a means for recognizing the user's emotions and making personalized product suggestions based on the emotions, and a means for scanning products using smart glasses or a smartphone and automatically registering and suggesting products. This improves the efficiency of the user's daily necessities management and enables product suggestions that meet individual needs based on emotions.

[1085] "Daily necessities" refer to consumables and household items that are frequently used in daily life.

[1086] "Manual registration" means that the user manually enters the information.

[1087] "Product images" refer to photographs or image data of products that are the subject of daily necessities.

[1088] "Receipt" refers to a proof of purchase that details the daily necessities purchased.

[1089] "Automatic registration" refers to a method of automatically extracting and registering product information from product images and receipts using image recognition technology, OCR, etc.

[1090] "Generative AI" refers to artificial intelligence that uses algorithms to generate and analyze data.

[1091] "Analysis" is the process of analyzing data, extracting information, and understanding it.

[1092] "Predicting replenishment timing" refers to analyzing the usage rate of consumables and predicting when the next replenishment will be required.

[1093] "Reminding" means sending a notification to the user prompting them to take necessary action.

[1094] "Personalized Discounts" refers to special discounts provided to you based on your individual needs and preferences.

[1095] "Recommended product information" refers to product information recommended based on a user's past purchase history and usage status.

[1096] "Emotion recognition" refers to technology that determines emotions from a user's facial expressions and input.

[1097] "Personalized product suggestions" refers to suggesting products that are individually suited to the user's emotional state and usage situation.

[1098] "Smart glasses" refers to eyeglass-like devices with built-in computer functions.

[1099] A "smartphone" refers to a highly functional mobile phone that has calling and internet connection capabilities.

[1100] "Scanning a product" refers to reading product information using smart glasses or a smartphone.

[1101] A system for implementing this invention includes a user terminal (smartphone, smart glasses), a server, a network, and a database.

[1102] First, users manually register everyday items using their smartphones or smart glasses. They launch the application and enter information such as the product name, purchase date, and price. This information is then sent to the server in JSON format and stored in a database.

[1103] Next, a method is provided to automatically register everyday items from product images or receipts. Users use their devices to take photos of product images or receipts and send them to the server. The server then uses OCR technology (e.g., Tesseract OCR) to extract product information from the images and register it in a database.

[1104] The server periodically analyzes the usage of registered daily items using AI. For example, it analyzes the pace at which a user consumes a particular daily item and predicts when it will need to be replenished. Based on this information, the server sends reminders to the user to inform them when it is time to replenish.

[1105] In addition, the server provides users with personalized discount information and recommended product information. Based on the user's usage and purchase history, the AI ​​generator selects the most suitable discount information and recommended products, allowing users to obtain the product information that best suits them.

[1106] An emotion engine is a way to recognize a user's emotions and make personalized product recommendations. For example, a user can input "I'm tired" into smart glasses or a smartphone, or read emotions from a facial image. As a result, it can recommend products that reduce stress (e.g., relaxing bath salts or aroma candles).

[1107] It also includes a way for users to scan products in physical stores using smart glasses or smartphones, which will automatically register and suggest products. For example, when a user scans a shampoo in the store, detailed information about the product will be displayed and product suggestions will be made based on emotions.

[1108] This system uses smartphones, smart glasses, and servers as hardware, and OCR technology (Tesseract OCR), generative AI (Keras, etc.), and an emotion engine as software. Data is sent and received over a network, and information is managed in a database.

[1109] Specific examples

[1110] When a user is shopping in a physical store, they use smart glasses to scan the barcode of a shampoo. The system automatically retrieves product information and displays it to the user. If the system recognizes that the user is feeling fatigued, it will suggest relaxing bath salts. Below is an example of a prompt for this process:

[1111] "Recommend the best daily essentials and add-ons based on the user's emotions and purchase history. If the user is stressed, suggest products that will help them relax."

[1112] As described above, this system will improve the efficiency of users' daily necessities management and enable product suggestions that meet individual emotional needs.

[1113] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1114] Step 1:

[1115] Users register everyday items.

[1116] Input: The user manually enters the product name, purchase date, and price, or takes a photo of the product or receipt.

[1117] Processing: In the case of manual entry, the device converts the information entered by the user into JSON format and sends it to the server. In the case of automatic registration, the device takes a photo and sends it to the server.

[1118] Output: In case of manual input, the server stores the received JSON data in the database. In case of automatic registration, the server uses OCR technology to extract product information and stores it in the database.

[1119] Step 2:

[1120] Usage is analyzed using generative AI.

[1121] Input: Registration data of everyday items stored in the database.

[1122] Processing: The server periodically retrieves the registration data from the database and analyzes it using a generative AI model (e.g., using Keras). It analyzes consumption patterns and usage history.

[1123] Output: Analytics showing usage and consumption pace.

[1124] Step 3:

[1125] Predicts replenishment timing and sends reminders.

[1126] Input: The analysis results from step 2.

[1127] Processing: The server predicts when the next replenishment is required based on the consumption pace provided by the generation AI. A reminder notification is generated based on the prediction result.

[1128] Output: A reminder notification is sent to the user's device, with a message such as "Your shampoo needs refilling."

[1129] Step 4:

[1130] Providing personalized discounts and product recommendations.

[1131] Input: User usage and purchase history.

[1132] Processing: The server uses a generative AI model to select optimal discounts and product recommendations based on usage and purchase history, and also retrieves the latest discounts and related product information from external data sources.

[1133] Output: Personalized discount information and product recommendations are sent to the user's device.

[1134] Step 5:

[1135] Recognize emotions and make personalized product recommendations.

[1136] Input: User's face image and text input.

[1137] Processing: The device takes a picture of the user's face and sends it to the server. The server uses an emotion recognition model to determine the emotion from the facial image. The server also recognizes emotions when the user enters their emotion in text. The server then makes appropriate product suggestions based on the emotion.

[1138] Output: Personalized product suggestions based on emotions are displayed on the device.

[1139] Step 6:

[1140] Products are scanned using smart glasses or a smartphone, and are automatically registered and suggested.

[1141] Input: Barcode or QR code of the product in the physical store.

[1142] Processing: The device (smart glasses or smartphone) scans the product and sends the product information to the server. The server analyzes the product information, registers it in a database, and suggests appropriate products based on the user's emotional data.

[1143] Output: Product details and recommended product information are displayed on the device.

[1144] In this way, the system can streamline users' daily necessities management and provide personalized, emotion-based suggestions.

[1145] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1146] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1147] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1148] [Third embodiment]

[1149] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1150] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1151] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1152] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1153] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1154] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1155] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1156] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1157] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1158] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1159] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1160] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1161] The present invention is a system that allows users to efficiently manage their daily necessities and provides optimal replenishment timing and personalized discount information. This system includes a means for users to manually register daily necessities, a means for automatically registering items from product photos and receipts, a means for analyzing usage status using generation AI, a means for predicting and reminding replenishment timing, and a means for providing personalized discount information and recommended product information.

[1162] Program processing

[1163] 1. How users register everyday items

[1164] Users launch the application using a device such as a smartphone or tablet. When manually registering daily necessities, users enter the product name, purchase date, price, store information, etc. into an input form. The device sends this information in JSON format to the server, which then stores the received data in a database.

[1165] Users can also take photos of receipts and products and register them. The images are sent from the device to a server, where they are analyzed using an image recognition algorithm (e.g., OCR technology). The server then extracts product information from the images and registers it in a database.

[1166] Examples:

[1167] When a user purchases shampoo, they take a photo of the receipt. The image is sent to a server, and OCR technology extracts the purchase date, product name, price, etc., and registers them in a database. This allows users to register product information in the system without any hassle.

[1168] 2. Usage analysis

[1169] The server periodically analyzes the information about daily necessities stored in the database. Utilizing generative AI, it analyzes each user's usage and consumption rate of daily necessities and predicts when replenishment is necessary. Based on this information, the server reminds the user of the optimal time to replenish.

[1170] Examples:

[1171] The server analyzes the shampoo usage history in the database and finds that, on average, users use up a bottle every three weeks. One week before the next bottle of shampoo runs out, the server sends a reminder to the device, informing the user that the shampoo needs to be refilled.

[1172] 3. Providing discount information and recommended products

[1173] The server uses AI to generate personalized product information based on each user's usage and purchase history, and also retrieves the latest discount information and related product information from external data sources to make optimal suggestions for the user.

[1174] Examples:

[1175] The server analyzes the user's shampoo purchase history and retrieves information about new and discounted products from the same brand from external data sources. Based on the information retrieved, a notification is sent to the device stating, "Recommended shampoo is now 20% off," and the user can check the details within the app.

[1176] Implementation environment

[1177] The system operates in a network environment that includes user devices (smartphones, tablets, etc.), a server, and a database. Users use an application to register their daily necessities and send the necessary information to the server. The server stores the received information in a database, periodically analyzes the data, and uses generative AI to provide optimal replenishment timing and product information.

[1178] This configuration allows users to manage their daily necessities more efficiently, supporting them in replenishing their supplies when needed and making purchases at a good price.

[1179] The processing flow will be explained below.

[1180] Program processing (specific steps)

[1181] Manually inputting daily necessities

[1182] Step 1:

[1183] A user launches an application on their smartphone or tablet.

[1184] Step 2:

[1185] The user taps the "Register a new product" button and enters information such as the product name, purchase date, price, and store.

[1186] Step 3:

[1187] Check the information entered by the user and tap the "Register" button.

[1188] Step 4:

[1189] The terminal sends the entered data to the server in JSON format.

[1190] Step 5:

[1191] The server analyzes the received JSON data and generates an SQL statement to save it in the database.

[1192] Step 6:

[1193] The server executes the generated SQL statement and saves the new product information in the database.

[1194] Automatically register daily necessities from product photos and receipts

[1195] Step 1:

[1196] A user launches an application on their smartphone or tablet.

[1197] Step 2:

[1198] The user taps the "Scan Receipt" button and takes a photo of the receipt or item.

[1199] Step 3:

[1200] The device temporarily stores the captured image and sends the image data to the server.

[1201] Step 4:

[1202] The server receives the image and runs an image recognition algorithm (such as OCR) to extract product information.

[1203] Step 5:

[1204] The server analyzes the extracted product information and obtains information such as product name, purchase date, price, and store.

[1205] Step 6:

[1206] The server generates an SQL statement to save the product information to the database.

[1207] Step 7:

[1208] The server executes the generated SQL statement and saves the new product information in the database.

[1209] Usage analysis and replenishment forecast

[1210] Step 1:

[1211] The server runs a job that is triggered periodically to retrieve user commodity usage data from the database.

[1212] Step 2:

[1213] The server uses generated AI to analyze the acquired data and calculate each user's daily necessities usage rate.

[1214] Step 3:

[1215] The server predicts the next replenishment timing based on the calculation results.

[1216] Step 4:

[1217] The server sets a reminder based on the predicted replenishment timing.

[1218] Step 5:

[1219] The device receives the set reminder and displays a notification to the user.

[1220] Providing discount information and recommended products

[1221] Step 1:

[1222] The server retrieves the user's purchase history and usage information from the database.

[1223] Step 2:

[1224] The server uses generated AI to analyze the acquired data and generate a list of recommended products for each user.

[1225] Step 3:

[1226] The server retrieves the latest discount information from an external data source.

[1227] Step 4:

[1228] The server combines the recommended product list and discount information to generate information to be provided to the user.

[1229] Step 5:

[1230] The device displays the information received from the server and sends notifications to the user.

[1231] Example 1

[1232] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1233] Today's busy lives make it difficult for users to properly manage their daily necessities and determine when to replenish them. Furthermore, obtaining optimal discount information and recommended product information takes time and effort. To address these issues, a system is needed that can efficiently manage daily necessities, provide optimal replenishment timing, and provide users with personalized, useful information.

[1234] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1235] In this invention, the server includes: a means for a user to launch an application using a smart device and manually register information about daily necessities; a means for converting the manually entered data into JSON format and sending it to the server; a means for the server to store the received data in a database; a means for sending receipts and product photos to the server and analyzing the images using OCR technology; a means for the server to register product information extracted from the images in the database; a means for periodically analyzing the database data using a generative AI model to analyze consumption rates; a means for predicting replenishment timing and sending reminders to the terminal; a means for obtaining discount information and related product information from external data sources and generating personalized suggestions; and a means for sending discount information and recommended product information to the user's terminal. This enables users to efficiently manage and replenish daily necessities and receive personalized, useful product information and discount information.

[1236] "User" means an individual or corporation that uses the system to register and manage everyday items.

[1237] A "smart device" is a device, such as a smartphone or tablet, that can connect to the Internet and run applications.

[1238] "Application" means a software program that a user uses to register and manage everyday item information on a smart device.

[1239] "Daily necessities" are products that users use on a daily basis and consume, such as shampoo, detergent, and toilet paper.

[1240] "Manual input" refers to the act of a user directly entering information into an application's input form using a smart device.

[1241] "JSON format" stands for JavaScript Object Notation and is a lightweight data format for structuring and exchanging data.

[1242] "Server" means a computer system that receives and processes data sent by users and stores it in a database.

[1243] A "database" is a collection of data managed by a server, a system that allows for efficient information storage and retrieval.

[1244] A "receipt" is a paper or electronic record that contains details of a purchased item (e.g., item name, price, purchase date, etc.).

[1245] "OCR technology" is an abbreviation for Optical Character Recognition, and is a technology that automatically recognizes character information in an image and converts it into text data.

[1246] A "generative AI model" is an artificial intelligence model that uses technologies such as deep learning to learn patterns from data and make predictions and classifications.

[1247] A "reminder" is a notification sent by the server to a user's device, and is a message used to encourage a specific action.

[1248] "External data sources" are information sources from outside the system, including web services and databases that provide up-to-date discount and product information.

[1249] "Personalized suggestions" are product and service suggestions that are individually generated based on a user's usage and purchase history.

[1250] This invention is a system that allows users to efficiently manage daily necessities and provides optimal replenishment timing and personalized discount information. The system includes a means for users to manually register daily necessities, a means for automatically registering daily necessities from product photos and receipts, a means for analyzing usage status using a generative AI model, a means for predicting and reminding replenishment timing, and a means for providing personalized discount information and recommended product information.

[1251] System Configuration

[1252] Hardware

[1253] User device: A device that can connect to the internet, such as a smartphone or tablet.

[1254] Server: A computer system that receives and processes data and accesses a database.

[1255] Database: A collection of data managed by a server that stores information about everyday items.

[1256] software

[1257] Application: Software that users use on their devices to register and manage everyday items.

[1258] OCR technology: Software that analyzes photos of receipts or products and extracts text information. Example: Tesseract OCR.

[1259] Generative AI model: An artificial intelligence model that analyzes data and predicts replenishment timing. Example: TensorFlow.

[1260] Overview of operation procedure

[1261] Daily necessities registration

[1262] 1. Manual registration:

[1263] Users launch the application using their smartphone or tablet and manually enter information about the daily necessities they purchased (product name, purchase date, price, store information, etc.). After completing the entry, they press the "Register" button.

[1264] The terminal converts the input data into JSON format and sends it to the server.

[1265] The server stores the received data in a database.

[1266] Examples:

[1267] When a user enters shampoo purchase information into the input form and clicks the "Register" button, the device converts the input data into JSON format and sends it to the server, which then analyzes the received data and stores it in a database.

[1268] Example prompt sentence:

[1269] Please manually enter the shampoo information you purchased (product name, purchase date, price, store information) using your smartphone.

[1270] 2. Registration from an image:

[1271] The user takes a photo of the product or receipt of the purchased item, and then sends the image data to the server via the application.

[1272] The terminal transmits the captured image data to the server.

[1273] The server analyzes the received image data using OCR technology and registers the product information extracted from the image in a database.

[1274] Examples:

[1275] When a user takes a photo of a receipt using their smartphone camera and sends the image from the app to the server, the server activates the OCR engine to extract text data from the image, analyzes the extracted text data, and stores it in a database.

[1276] Example prompt sentence:

[1277] "Take a photo of the receipt or item you purchased and submit it through the application."

[1278] Usage analytics and reminders

[1279] The server periodically analyzes the information on daily items stored in the database, utilizing a generative AI model to analyze each user's usage and consumption rate of daily items.

[1280] Based on the analysis results, the server predicts when replenishment is needed and sends a reminder to the user's device.

[1281] Examples:

[1282] The server analyzes the shampoo usage history in the database and finds that, on average, users use up a bottle every three weeks. One week before the next bottle runs out, the server sends a reminder to the device saying, "You need to refill your shampoo."

[1283] Providing discount information and recommended products

[1284] The server generates personalized product information using a generative AI model based on each user's usage and purchase history, and also retrieves the latest discount information and related product information from external data sources to make optimal suggestions for the user.

[1285] The server transmits the generated discount information and recommended product information to the user's terminal.

[1286] Examples:

[1287] The server analyzes the user's shampoo purchase history and retrieves information about new and discounted products from the same brand from external data sources. Based on the information retrieved, a notification is sent to the device stating, "Recommended shampoo is now 20% off," and the user can check the details within the app.

[1288] This system helps users manage their daily necessities more efficiently, supporting them in replenishing their supplies when needed and making purchases at a good price.

[1289] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1290] Daily necessities registration phase

[1291] Step 1:

[1292] The user launches the application using a smart device and manually inputs information about the purchased daily necessities (product name, purchase date, price, store information, etc.). Once the input is complete, the user presses the "Register" button. The input data includes the product name, purchase date, price, and store information.

[1293] Specific behavior:

[1294] The user enters shampoo purchase information into the input form and clicks the "Register" button.

[1295] input:

[1296] Product information entered by the user (product name, purchase date, price, store information)

[1297] output:

[1298] JSON format data on the device

[1299] Step 2:

[1300] The terminal converts the manually entered data into JSON format, which includes the product name, purchase date, price, and store information.

[1301] Specific behavior:

[1302] The terminal converts the input data into JSON format.

[1303] input:

[1304] Product information entered by the user

[1305] output:

[1306] JSON format data

[1307] Step 3:

[1308] The terminal sends the converted JSON format data to the server.

[1309] Specific behavior:

[1310] The device starts the process of sending JSON data to the server.

[1311] input:

[1312] Structured product data in JSON format

[1313] output:

[1314] JSON data sent to the server

[1315] Step 4:

[1316] The server receives the JSON data sent from the device, analyzes it, and stores it in a database.

[1317] Specific behavior:

[1318] The server stores the received data in a database.

[1319] input:

[1320] JSON format data

[1321] output:

[1322] Information on everyday items in the database

[1323] Image information registration phase

[1324] Step 1:

[1325] The user takes a photo of the product or receipt of the purchased item, and then sends the image data to the server via the application.

[1326] Specific behavior:

[1327] The user takes a photo of the receipt using their smartphone camera and the app sends the image to the server.

[1328] input:

[1329] Receipts and product photos

[1330] output:

[1331] Image data stored on the device

[1332] Step 2:

[1333] The terminal transmits the captured image data to the server. The transmitted data is an image that may contain product information.

[1334] Specific behavior:

[1335] The device starts the process of sending the captured image to the server.

[1336] input:

[1337] Image data

[1338] output:

[1339] Image data sent to the server

[1340] Step 3:

[1341] The server analyzes the received image data using OCR technology and registers the product information extracted from the image (product name, purchase date, price, etc.) in a database.

[1342] Specific behavior:

[1343] The server starts the OCR engine, extracts text data from the image, analyzes the extracted text data, and stores it in a database.

[1344] input:

[1345] Image data

[1346] output:

[1347] Product information stored in the database

[1348] Usage analysis phase

[1349] Step 1:

[1350] The server periodically analyzes the information stored in the database and uses a generative AI model to analyze the usage and consumption rate of each item for each user.

[1351] Specific behavior:

[1352] The server runs scheduled jobs and performs data analysis using generative AI models.

[1353] input:

[1354] Information on everyday items in the database

[1355] output:

[1356] Usage and consumption rate per user

[1357] Step 2:

[1358] Based on the analysis results, the server predicts when each user's daily necessities will need to be replenished.

[1359] Specific behavior:

[1360] The server calculates the timing of replenishment based on the analysis results.

[1361] input:

[1362] AI model analysis results

[1363] output:

[1364] Predicted replenishment timing

[1365] Step 3:

[1366] The server sends a reminder to the user's device when the predicted replenishment time approaches.

[1367] Specific behavior:

[1368] The server generates a reminder notification and sends it to the device.

[1369] input:

[1370] Predicted replenishment timing

[1371] output:

[1372] Reminder messages sent to your device

[1373] Discount information and recommended products phase

[1374] Step 1:

[1375] The server generates personalized product information using a generative AI model based on each user's usage and purchase history, and also retrieves the latest discount information and related product information from external data sources.

[1376] Specific behavior:

[1377] The server periodically calls an external API to retrieve the latest discount information.

[1378] input:

[1379] User usage, purchase history, external data

[1380] output:

[1381] Personalized product information

[1382] Step 2:

[1383] Based on the collected data, the server generates and stores discount information and recommended product information that is optimal for the user.

[1384] Specific behavior:

[1385] The server applies an algorithm based on the information it obtains to generate personalized suggestions.

[1386] input:

[1387] Data analysis results, external data

[1388] output:

[1389] Saved discounts and recommended products

[1390] Step 3:

[1391] The server transmits the generated discount information and recommended product information to the user's terminal.

[1392] Specific behavior:

[1393] The server generates the proposal as a notification and sends it to the device.

[1394] input:

[1395] Generated proposals

[1396] output:

[1397] Notification messages sent to the device

[1398] (Application example 1)

[1399] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1400] Conventional daily necessities management systems have the drawback of making it difficult to properly determine when to replenish items because they require users to register their purchases and do not adequately analyze usage.Furthermore, they do not provide users with personalized discount information or recommended products, making it difficult to efficiently manage daily necessities and purchase the products they need at the optimal time.

[1401] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1402] In this invention, the server includes a means for users to manually register daily necessities, a means for automatically registering daily necessities from product photos and receipts, and a means for analyzing the usage of registered daily necessities using a generating AI. This allows users to easily register daily necessities, and the analysis of usage using the generating AI makes it possible to predict the optimal timing for replenishment and send reminders. Furthermore, the server includes a means for predicting the usage rate and replenishment timing using a generating AI model, a means for extracting and automatically registering product information using image recognition, and a means for obtaining and providing discount information and recommended product information from external data sources. This allows users to efficiently manage their daily necessities and purchase products at great prices based on personalized discount information.

[1403] "Means for manually registering daily necessities" is a function that allows users to manually enter product name, purchase date, price, store information, etc. to register daily necessities in the system.

[1404] "Means for automatically registering daily necessities from product photos and receipts" is a function that analyzes product photos and receipt images taken by users, automatically extracts product information, and registers it in the system.

[1405] "Means for analyzing usage status using generative AI" refers to a function that uses a generative AI model to analyze information on stored daily necessities, and analyzes the pace of use and consumption of daily necessities for each user.

[1406] "Means to predict and remind the timing of replenishment" is a function that predicts the next time replenishment is required based on the results of analysis by the generation AI and notifies the user.

[1407] "Means for providing personalized discount information and recommended product information" refers to a function that uses generative AI to provide optimal discount information and related product information based on each user's usage and purchase history.

[1408] "Means for predicting usage rate and replenishment timing using a generative AI model" is a function that uses a generative AI model to predict the usage rate and optimal replenishment timing of registered daily necessities.

[1409] "Means for extracting and automatically registering product information using image recognition" refers to a function that uses OCR technology or image recognition algorithms to extract product information from receipt or product images and automatically registers it in the system.

[1410] "Means for obtaining and providing discount information and recommended product information from external data sources" refers to a function that analyzes the latest discount information and recommended product information obtained from external data sources and provides it to users in a personalized form.

[1411] MODE FOR CARRYING OUT THE INVENTION

[1412] The present invention is a system that allows users to efficiently manage their daily necessities, and provides optimal replenishment timing and personalized discount information. This system operates using a smartphone application, a server, and a database.

[1413] System configuration

[1414] User Device

[1415] Users launch the application using a device such as a smartphone or tablet. There are two ways to register daily necessities: manual entry or automatic registration.

[1416] 1. Manual registration

[1417] The user enters the product name, purchase date, price, store information, etc. into an input form and sends this information in JSON format to the server.

[1418] 2. Automatic Registration

[1419] Users take photos of receipts or products and send the images to the server, which uses OCR technology to extract product information from the images and register it in a database.

[1420] Servers and Databases

[1421] 1. Usage analysis

[1422] The server uses a generative AI model to analyze information about daily necessities stored in a database, and analyzes each user's pace of use and consumption of daily necessities.

[1423] 2. Predicting replenishment timing

[1424] Based on the analysis results, the server predicts when the next replenishment is required and sends a reminder to the user.

[1425] 3. Providing discount information and recommended products

[1426] The server retrieves the latest discount information and related product information from external data sources and generates personalized product information using a generative AI model.

[1427] Based on this information, the server provides users with optimal discount information and related product information.

[1428] Specific examples of implementation methods

[1429] Manual registration example

[1430] When a user purchases shampoo, they manually enter product information using the application, including details such as the purchase date, product name, and price, and send the information to the server, which then registers the received information in the database.

[1431] Auto-registration Example

[1432] The user takes a photo of the receipt and sends it to the server via the application. The server uses OCR technology to extract the purchase date, product name, price, etc. from the image and registers them in the database. This allows the user to register product information without any hassle.

[1433] Usage analysis example

[1434] The server analyzes the shampoo usage history stored in the database and finds that the average user uses up a bottle every three weeks. One week before the next bottle of shampoo runs out, the server sends a reminder to the user's smartphone, informing them that they need to refill their shampoo.

[1435] Examples of discounts and product recommendations

[1436] The server analyzes the user's shampoo purchase history and retrieves information about new and discounted products from the same brand from external data sources. Based on the information retrieved, the server sends the user a notification saying, "Recommended shampoo is now 20% off," and allows them to check the details within the app.

[1437] Examples of using generative AI models and prompts

[1438] An example of a prompt that the server uses to perform analysis using a generative AI model:

[1439] This system is designed to help you efficiently manage your daily necessities, provide optimal replenishment timing, and provide personalized discount information. Simply take a photo of your receipt or product and register it in the system, and you will receive timely replenishment notifications and discount information.

[1440] This allows users to efficiently manage their daily necessities and purchase the products they need at the best possible time.

[1441] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1442] Step 1:

[1443] Users manually register daily items

[1444] The user launches the smartphone application and enters the product name, purchase date, price, store information, etc. into the input form. The entered information is sent to the server in JSON format. The server parses the received JSON data and registers it in the database.

[1445] Input: Product name, purchase date, price, store information (user input)

[1446] Output: Product information is saved in the database

[1447] Step 2:

[1448] Users take photos of receipts and products and register them

[1449] A user uses a smartphone application to take a photo of a receipt or product. The image data is then uploaded to a server. The server uses OCR technology to extract text information from the image and convert it into data such as product name, purchase date, price, and store information. This data is then registered in a database.

[1450] Input: Image of receipt or product (taken by user)

[1451] Output: The extracted product information is saved in the database.

[1452] Step 3:

[1453] The server analyzes the usage of everyday items

[1454] The server periodically analyzes the information on daily items stored in the database using a generative AI model, which calculates each user's usage rate and consumption rate of daily items.

[1455] Input: Information about everyday items stored in a database

[1456] Output: Analysis results of usage pace and wear rate (analysis results from generative AI model)

[1457] Step 4:

[1458] Refill forecast and reminder sending

[1459] The server predicts when the next refill is needed based on the analysis results of the generative AI model. Based on the prediction, a reminder notification is sent to the user's smartphone. For example, if shampoo runs out within one week of the predicted time, the user will be notified that "shampoo needs to be refilled."

[1460] Input: Analysis results of the generative AI model

[1461] Output: Reminder notification for refilling (sent to user's smartphone)

[1462] Step 5:

[1463] Providing discount information and recommended products

[1464] The server uses a generative AI model based on each user's usage and purchase history to generate personalized product information. It also obtains discount information and recommended product information from external data sources and stores it in a database. The generated information is then combined to notify users of appropriate discount information and recommended products.

[1465] Inputs: User usage, purchase history, discount information from external data sources

[1466] Output: Notification of recommended products and discount information (sent to the user's smartphone)

[1467] This allows users to efficiently manage their daily necessities and replenish them at the optimal time and at a good price.

[1468] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1469] This invention combines an emotion engine with a system that allows users to efficiently manage their daily necessities and provides optimal replenishment timing and personalized discount information, thereby realizing personalized suggestions based on user emotion recognition. This system includes a means for users to manually register daily necessities, a means for automatically registering items from product photos and receipts, a means for analyzing usage status using generative AI, a means for predicting and reminding users when replenishment is necessary, a means for providing personalized discount information and recommended product information, and a means for recognizing user emotions using the emotion engine.

[1470] Program processing

[1471] 1. Manual and automatic commodity registration

[1472] Users launch the application using a device such as a smartphone or tablet. When registering manually, users enter the product name, purchase date, price, store information, etc. into an input form. The device sends this information in JSON format to the server, which then stores the received data in a database.

[1473] When automatically registering using product photos or receipts, users take a photo and send it to the server from their device. The server then uses an image recognition algorithm (such as OCR) to extract product information from the image and register it in the database.

[1474] Examples:

[1475] When a user purchases shampoo, they take a photo of the receipt. The image is sent to a server, and OCR technology extracts the purchase date, product name, price, etc., and registers them in a database. This allows users to register product information in the system without any hassle.

[1476] 2. Analyzing usage and predicting replenishment timing

[1477] The server periodically analyzes the information about daily necessities stored in the database. Utilizing generative AI, it analyzes each user's usage and consumption rate of daily necessities and predicts when replenishment is necessary. Based on this information, the server reminds the user of the optimal time to replenish.

[1478] Examples:

[1479] The server analyzes the shampoo usage history in the database and finds that, on average, users use up a bottle every three weeks. One week before the next bottle of shampoo runs out, the server sends a reminder to the device, informing the user that the shampoo needs to be refilled.

[1480] 3. Personalized discounts and product recommendations

[1481] The server uses AI to generate personalized product information based on each user's usage and purchase history, and also retrieves the latest discount information and related product information from external data sources to make optimal suggestions for the user.

[1482] Examples:

[1483] The server analyzes the user's shampoo purchase history and retrieves information about new and discounted products from the same brand from external data sources. Based on the information retrieved, a notification is sent to the device stating, "Recommended shampoo is now 20% off," and the user can check the details within the app.

[1484] 4. Emotion Recognition and Personalization with Emotion Engine

[1485] The emotion engine recognizes emotions based on user input and usage, and tailors personalized product recommendations and reminders accordingly.

[1486] Examples:

[1487] If a user types "I'm tired" into the app, the emotion engine will recognize that emotion and suggest products to reduce stress (e.g., relaxing bath salts). If the user's emotion is fatigued, the app will also express the reminder in gentler terms, such as "Thank you for your hard work. Your next shampoo is needed."

[1488] 5. Storage and utilization of emotional data

[1489] The server periodically stores the user's emotional data recognized by the emotion engine in a database, which will be used for future personalized suggestions, such as more personalized product suggestions and tailoring of reminder content.

[1490] Examples:

[1491] Based on the user's emotional history, the system analyzes times when stress is likely to increase and provides focused recommendations on relaxation products and other products that will help relieve stress.

[1492] Implementation environment

[1493] The system operates in a network environment that includes user devices (smartphones, tablets, etc.), a server, and a database. Users use an application to register their daily necessities and send the necessary information to the server. The server stores the received information in a database and analyzes it using generative AI and an emotion engine to provide optimal replenishment timing and personalized product information.

[1494] This configuration allows for more efficient management of daily necessities for users, and makes it possible to provide optimal product suggestions and replenishment notifications tailored to each individual's emotional state.

[1495] The processing flow will be explained below.

[1496] Manually inputting daily necessities

[1497] Step 1:

[1498] A user launches an application on their smartphone or tablet.

[1499] Step 2:

[1500] The user taps the "Register a new product" button and enters information such as the product name, purchase date, price, and store.

[1501] Step 3:

[1502] Check the information entered by the user and tap the "Register" button.

[1503] Step 4:

[1504] The terminal sends the entered data to the server in JSON format.

[1505] Step 5:

[1506] The server analyzes the received JSON data and generates an SQL statement to save it in the database.

[1507] Step 6:

[1508] The server executes the generated SQL statement and saves the new product information in the database.

[1509] Automatically register daily necessities from product photos and receipts

[1510] Step 1:

[1511] A user launches an application on their smartphone or tablet.

[1512] Step 2:

[1513] The user taps the "Scan Receipt" button and takes a photo of the receipt or item.

[1514] Step 3:

[1515] The device temporarily stores the captured image and sends the image data to the server.

[1516] Step 4:

[1517] The server receives the image and runs an image recognition algorithm (such as OCR) to analyze the extracted product information.

[1518] Step 5:

[1519] The server analyzes the extracted product information and obtains information such as the product name, purchase date, price, and store.

[1520] Step 6:

[1521] The server generates an SQL statement to save the product information to the database.

[1522] Step 7:

[1523] The server executes the generated SQL statement and saves the new product information in the database.

[1524] Usage analysis and replenishment forecast

[1525] Step 1:

[1526] The server runs a job that is triggered periodically to retrieve user commodity usage data from the database.

[1527] Step 2:

[1528] The server uses generated AI to analyze the collected data and calculate each user's daily necessities usage rate.

[1529] Step 3:

[1530] The server predicts the next replenishment timing based on the calculation results.

[1531] Step 4:

[1532] The server sets a reminder based on the predicted replenishment timing.

[1533] Step 5:

[1534] The device receives the set reminder and displays a notification to the user.

[1535] Providing discount information and recommended products

[1536] Step 1:

[1537] The server retrieves the user's purchase history and usage information from the database.

[1538] Step 2:

[1539] The server uses a generation AI to analyze the acquired data and generate a list of recommended products for each user.

[1540] Step 3:

[1541] The server retrieves the latest discount information from an external data source.

[1542] Step 4:

[1543] The server combines the recommended product list and discount information to generate information to be provided to the user.

[1544] Step 5:

[1545] The device displays the information received from the server and sends notifications to the user.

[1546] Emotion recognition and personalization with emotion engine

[1547] Step 1:

[1548] Providing information about emotions that users input into the application (e.g., tired, stressed, etc.).

[1549] Step 2:

[1550] The emotion engine analyzes the user's input data and recognizes the user's emotions.

[1551] Step 3:

[1552] The server generates a personalized product suggestion list based on the analysis results of the emotion engine.

[1553] Step 4:

[1554] The server adjusts the content of the reminder to use gentle language and appropriate expressions to match the user's emotions.

[1555] Step 5:

[1556] The device displays personalized suggestions and reminders received from the server and sends notifications to the user.

[1557] Storing and utilizing emotional data

[1558] Step 1:

[1559] The server stores the user's emotion data recognized using the emotion engine in a database.

[1560] Step 2:

[1561] The server periodically analyzes the stored emotional data to understand the user's emotional patterns.

[1562] Step 3:

[1563] The server uses the emotion data to tailor future personalized suggestions and reminders.

[1564] Step 4:

[1565] The server provides product information for stress reduction and relaxation in a timely manner according to the user's emotional state.

[1566] Step 5:

[1567] The device displays the suggestion information based on the emotion received from the server and sends a notification to the user.

[1568] Example 2

[1569] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1570] Conventional daily necessities management systems make it difficult for users to efficiently obtain information on replenishment timing and discounts for daily necessities, and do not provide personalized suggestions based on the user's emotional state. As a result, they have not been able to fully improve the user experience or stimulate purchasing motivation. To solve this, a system that takes into account not only the user's usage status but also their emotional state is needed.

[1571] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1572] In this invention, the server includes a means for a user to manually register daily necessities, a means for automatically registering daily necessities from product photos and receipts, a means for analyzing the usage of the registered daily necessities using a generative AI model, a means for predicting and reminding the user to replenish based on the analysis results, a means for providing personalized discount information and recommended product information to the user, and an emotion engine for recognizing the user's emotions and making personalized suggestions based on those emotions. This makes it possible to streamline the user's daily necessities management and make optimal product suggestions and replenishment notifications based on the user's emotions.

[1573] "User" means an individual or organization that uses the system to manage and register everyday items.

[1574] "Daily commodities" are consumables and products that users use on a daily basis.

[1575] "Manual registration" refers to an operation in which a user directly inputs information about a daily necessities using a terminal.

[1576] "Automatic registration" is a method of automatically extracting and registering information about everyday items from product photos and receipts.

[1577] A "generative AI model" is an algorithm that analyzes the usage of everyday items based on data and generates personalized information.

[1578] The "emotion engine" is a system component that recognizes emotions based on user input and usage and makes corresponding suggestions.

[1579] "Replenishment timing" refers to the next time a user should purchase or replenish daily necessities.

[1580] "Remind" refers to sending notifications or information to users to encourage them to take necessary action.

[1581] "Personalization" refers to customizing information and offers based on a user's individual usage and emotional state.

[1582] "Discount Information" means information about a price reduction applied to a product or service.

[1583] "Recommended Products" are products suggested to users based on their usage and emotional state.

[1584] The "database" is a system that stores information about everyday items registered by users and analysis results.

[1585] "Analysis" is the process of deriving specific information or results from collected data.

[1586] A "Notification" is a message or alert that conveys a reminder or offer to the User.

[1587] This invention combines an emotion engine with a system that allows users to efficiently manage their daily necessities and provides optimal replenishment timing and personalized discount information, thereby realizing personalized suggestions based on user emotion recognition. A detailed description of specific embodiments of this system is provided below.

[1588] 1. Manual and automatic commodity registration

[1589] Users launch the application using a device such as a smartphone or tablet. When registering manually, users enter the product name, purchase date, price, store information, etc. into an input form. The device sends this information in JSON format to the server, which then stores the received data in a database.

[1590] When automatically registering using product photos or receipts, users take a photo and send it to the server from their device. The server then uses an image recognition algorithm (such as OCR) to extract product information from the image and register it in the database.

[1591] Examples:

[1592] When a user purchases shampoo, they take a photo of the receipt. The image is sent to a server, and OCR technology extracts the purchase date, product name, price, etc., and registers them in a database. This allows users to register product information in the system without any hassle.

[1593] 2. Analyzing usage and predicting replenishment timing

[1594] The server periodically analyzes the information about daily necessities stored in the database. Utilizing generative AI, it analyzes each user's usage and consumption rate of daily necessities and predicts when replenishment is necessary. Based on this information, the server reminds the user of the optimal time to replenish.

[1595] Examples:

[1596] The server analyzes the shampoo usage history in the database and finds that, on average, users use up a bottle every three weeks. One week before the next bottle of shampoo runs out, the server sends a reminder to the device, informing the user that the shampoo needs to be refilled.

[1597] 3. Personalized discounts and product recommendations

[1598] The server uses AI to generate personalized product information based on each user's usage and purchase history, and also retrieves the latest discount information and related product information from external data sources to make optimal suggestions for the user.

[1599] Examples:

[1600] The server analyzes the user's shampoo purchase history and retrieves information about new and discounted products from the same brand from external data sources. Based on the information retrieved, a notification is sent to the device stating, "Recommended shampoo is now 20% off," and the user can check the details within the app.

[1601] 4. Emotion Recognition and Personalization with Emotion Engine

[1602] The emotion engine recognizes emotions based on user input and usage, and tailors personalized product recommendations and reminders accordingly.

[1603] Examples:

[1604] If a user types "I'm tired" into the app, the emotion engine will recognize that emotion and suggest products to reduce stress (e.g., relaxing bath salts). If the user's emotion is fatigued, the app will also express the reminder in gentler terms, such as "Thank you for your hard work. Your next shampoo is needed."

[1605] 5. Storage and utilization of emotional data

[1606] The server periodically stores the user's emotional data recognized by the emotion engine in a database, which will be used for future personalized suggestions, such as more personalized product suggestions and tailoring of reminder content.

[1607] Examples:

[1608] Based on the user's emotional history, the system analyzes times when stress is likely to increase and provides focused recommendations on relaxation products and other products that will help relieve stress.

[1609] This system allows users to efficiently manage their daily necessities and receive optimal product suggestions and replenishment notifications tailored to their individual emotional state.

[1610] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1611] Step 1:

[1612] A user launches the application using a device such as a smartphone or tablet. When manually registering a daily necessities item, the user enters the product name, purchase date, price, store information, etc. into an input form. The device sends the entered information in JSON format to the server. The server analyzes the received data and stores it in a database.

[1613] Input: Product name, purchase date, price, store information

[1614] Output: Product information registered in the database

[1615] Step 2:

[1616] The user takes a photo of a product or receipt and sends the image data from their device to the server. The server analyzes the received image data using an image recognition algorithm (such as OCR) and extracts product information. The extracted information is registered in a database.

[1617] Input: Product photo or receipt image data

[1618] Output: Product information registered in the database

[1619] Step 3:

[1620] The server periodically analyzes the information on daily items stored in the database and uses a generative AI model to calculate each user's usage and consumption rate of daily items.

[1621] Input: Daily necessities information in the database

[1622] Output: Analysis of usage pace and wear rate

[1623] Step 4:

[1624] Based on the analysis results, the server predicts when replenishment is necessary, calculates the next replenishment time, and sends a reminder to the user.

[1625] Input: Analysis results of usage pace and wear rate

[1626] Output: Reminder to user

[1627] Step 5:

[1628] The server uses AI to generate personalized product information based on each user's usage and purchase history, and retrieves the latest discount information and related product information from external data sources to make optimal suggestions to users.

[1629] Input: Information from usage, purchase history, and external data sources

[1630] Output: Personalized product and discount information

[1631] Step 6:

[1632] When users input their emotions into the app, the emotion engine analyzes them and tailors personalized suggestions and reminders.

[1633] Input: User emotion input

[1634] Output: Emotion-based personalized suggestions and reminders

[1635] Step 7:

[1636] The server periodically stores the user's emotion data recognized by the emotion engine in a database, which will be used for future personalized suggestions.

[1637] Input: Analysis results by emotion engine

[1638] Output: Emotion data stored in a database

[1639] Step 8:

[1640] Based on the stored emotional data, the server analyzes the periods when stress is likely to increase for each user and adjusts product suggestions and reminder content to suit that period.

[1641] Input: Emotion data stored in a database

[1642] Output: Personalized suggestions and reminders based on specific times

[1643] (Application example 2)

[1644] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1645] In conventional daily necessities management systems, registering users' daily necessities and managing replenishment timings are often done manually, making efficient management difficult. Furthermore, product suggestions based on users' emotions and personalized information provision are not provided, making it difficult to fully address individual user needs. The present invention aims to solve these problems by streamlining daily necessities management and product suggestions for users, and realizing personalized responses based on emotions.

[1646] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for a user to manually register daily necessities, a means for automatically registering daily necessities from product images and receipts, a means for analyzing the usage status of the registered daily necessities using a generation AI, a means for predicting replenishment timing based on the analysis results and reminding the user, a means for providing personalized discount information and recommended product information to the user, a means for recognizing the user's emotions and making personalized product suggestions based on the emotions, and a means for scanning products using smart glasses or a smartphone and automatically registering and suggesting products. This improves the efficiency of the user's daily necessities management and enables product suggestions that meet individual needs based on emotions.

[1647] "Daily necessities" refer to consumables and household items that are frequently used in daily life.

[1648] "Manual registration" means that the user manually enters the information.

[1649] "Product images" refer to photographs or image data of products that are the subject of daily necessities.

[1650] "Receipt" refers to a proof of purchase that details the daily necessities purchased.

[1651] "Automatic registration" refers to a method of automatically extracting and registering product information from product images and receipts using image recognition technology, OCR, etc.

[1652] "Generative AI" refers to artificial intelligence that uses algorithms to generate and analyze data.

[1653] "Analysis" is the process of analyzing data, extracting information, and understanding it.

[1654] "Predicting replenishment timing" refers to analyzing the usage rate of consumables and predicting when the next replenishment will be required.

[1655] "Reminding" means sending a notification to the user prompting them to take necessary action.

[1656] "Personalized Discounts" refers to special discounts provided to you based on your individual needs and preferences.

[1657] "Recommended product information" refers to product information recommended based on a user's past purchase history and usage status.

[1658] "Emotion recognition" refers to technology that determines emotions from a user's facial expressions and input.

[1659] "Personalized product suggestions" refers to suggesting products that are individually suited to the user's emotional state and usage situation.

[1660] "Smart glasses" refers to eyeglass-like devices with built-in computer functions.

[1661] A "smartphone" refers to a highly functional mobile phone that has calling and internet connection capabilities.

[1662] "Scanning a product" refers to reading product information using smart glasses or a smartphone.

[1663] A system for implementing this invention includes a user terminal (smartphone, smart glasses), a server, a network, and a database.

[1664] First, users manually register everyday items using their smartphones or smart glasses. They launch the application and enter information such as the product name, purchase date, and price. This information is then sent to the server in JSON format and stored in a database.

[1665] Next, a method is provided to automatically register everyday items from product images or receipts. Users use their devices to take photos of product images or receipts and send them to the server. The server then uses OCR technology (e.g., Tesseract OCR) to extract product information from the images and register it in a database.

[1666] The server periodically analyzes the usage of registered daily items using AI. For example, it analyzes the pace at which a user consumes a particular daily item and predicts when it will need to be replenished. Based on this information, the server sends reminders to the user to inform them when it is time to replenish.

[1667] In addition, the server provides users with personalized discount information and recommended product information. Based on the user's usage and purchase history, the AI ​​generator selects the most suitable discount information and recommended products, allowing users to obtain the product information that best suits them.

[1668] An emotion engine is a way to recognize a user's emotions and make personalized product recommendations. For example, a user can input "I'm tired" into smart glasses or a smartphone, or read emotions from a facial image. As a result, it can recommend products that reduce stress (e.g., relaxing bath salts or aroma candles).

[1669] It also includes a way for users to scan products in physical stores using smart glasses or smartphones, which will automatically register and suggest products. For example, when a user scans a shampoo in the store, detailed information about the product will be displayed and product suggestions will be made based on emotions.

[1670] This system uses smartphones, smart glasses, and servers as hardware, and OCR technology (Tesseract OCR), generative AI (Keras, etc.), and an emotion engine as software. Data is sent and received over a network, and information is managed in a database.

[1671] Specific examples

[1672] When a user is shopping in a physical store, they use smart glasses to scan the barcode of a shampoo. The system automatically retrieves product information and displays it to the user. If the system recognizes that the user is feeling fatigued, it will suggest relaxing bath salts. Below is an example of a prompt for this process:

[1673] "Recommend the best daily essentials and add-ons based on the user's emotions and purchase history. If the user is stressed, suggest products that will help them relax."

[1674] As described above, this system will improve the efficiency of users' daily necessities management and enable product suggestions that meet individual emotional needs.

[1675] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1676] Step 1:

[1677] Users register everyday items.

[1678] Input: The user manually enters the product name, purchase date, and price, or takes a photo of the product or receipt.

[1679] Processing: In the case of manual entry, the device converts the information entered by the user into JSON format and sends it to the server. In the case of automatic registration, the device takes a photo and sends it to the server.

[1680] Output: In case of manual input, the server stores the received JSON data in the database. In case of automatic registration, the server uses OCR technology to extract product information and stores it in the database.

[1681] Step 2:

[1682] Usage is analyzed using generative AI.

[1683] Input: Registration data of everyday items stored in the database.

[1684] Processing: The server periodically retrieves the registration data from the database and analyzes it using a generative AI model (e.g., using Keras). It analyzes consumption patterns and usage history.

[1685] Output: Analytics showing usage and consumption pace.

[1686] Step 3:

[1687] Predicts replenishment timing and sends reminders.

[1688] Input: The analysis results from step 2.

[1689] Processing: The server predicts when the next replenishment is required based on the consumption pace provided by the generation AI. A reminder notification is generated based on the prediction result.

[1690] Output: A reminder notification is sent to the user's device, with a message such as "Your shampoo needs refilling."

[1691] Step 4:

[1692] Providing personalized discounts and product recommendations.

[1693] Input: User usage and purchase history.

[1694] Processing: The server uses a generative AI model to select optimal discounts and product recommendations based on usage and purchase history, and also retrieves the latest discounts and related product information from external data sources.

[1695] Output: Personalized discount information and product recommendations are sent to the user's device.

[1696] Step 5:

[1697] Recognize emotions and make personalized product recommendations.

[1698] Input: User's face image and text input.

[1699] Processing: The device takes a picture of the user's face and sends it to the server. The server uses an emotion recognition model to determine the emotion from the facial image. The server also recognizes emotions when the user enters their emotion in text. The server then makes appropriate product suggestions based on the emotion.

[1700] Output: Personalized product suggestions based on emotions are displayed on the device.

[1701] Step 6:

[1702] Products are scanned using smart glasses or a smartphone, and are automatically registered and suggested.

[1703] Input: Barcode or QR code of the product in the physical store.

[1704] Processing: The device (smart glasses or smartphone) scans the product and sends the product information to the server. The server analyzes the product information, registers it in a database, and suggests appropriate products based on the user's emotional data.

[1705] Output: Product details and recommended product information are displayed on the device.

[1706] In this way, the system can streamline users' daily necessities management and provide personalized, emotion-based suggestions.

[1707] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1708] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1709] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1710] [Fourth embodiment]

[1711] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1712] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1713] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1714] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1715] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1716] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1717] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1718] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1719] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1720] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1721] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1722] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1723] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1724] The present invention is a system that allows users to efficiently manage their daily necessities and provides optimal replenishment timing and personalized discount information. This system includes a means for users to manually register daily necessities, a means for automatically registering items from product photos and receipts, a means for analyzing usage status using generation AI, a means for predicting and reminding replenishment timing, and a means for providing personalized discount information and recommended product information.

[1725] Program processing

[1726] 1. How users register everyday items

[1727] Users launch the application using a device such as a smartphone or tablet. When manually registering daily necessities, users enter the product name, purchase date, price, store information, etc. into an input form. The device sends this information in JSON format to the server, which then stores the received data in a database.

[1728] Users can also take photos of receipts and products and register them. The images are sent from the device to a server, where they are analyzed using an image recognition algorithm (e.g., OCR technology). The server then extracts product information from the images and registers it in a database.

[1729] Examples:

[1730] When a user purchases shampoo, they take a photo of the receipt. The image is sent to a server, and OCR technology extracts the purchase date, product name, price, etc., and registers them in a database. This allows users to register product information in the system without any hassle.

[1731] 2. Usage analysis

[1732] The server periodically analyzes the information about daily necessities stored in the database. Utilizing generative AI, it analyzes each user's usage and consumption rate of daily necessities and predicts when replenishment is necessary. Based on this information, the server reminds the user of the optimal time to replenish.

[1733] Examples:

[1734] The server analyzes the shampoo usage history in the database and finds that, on average, users use up a bottle every three weeks. One week before the next bottle of shampoo runs out, the server sends a reminder to the device, informing the user that the shampoo needs to be refilled.

[1735] 3. Providing discount information and recommended products

[1736] The server uses AI to generate personalized product information based on each user's usage and purchase history, and also retrieves the latest discount information and related product information from external data sources to make optimal suggestions for the user.

[1737] Examples:

[1738] The server analyzes the user's shampoo purchase history and retrieves information about new and discounted products from the same brand from external data sources. Based on the information retrieved, a notification is sent to the device stating, "Recommended shampoo is now 20% off," and the user can check the details within the app.

[1739] Implementation environment

[1740] The system operates in a network environment that includes user devices (smartphones, tablets, etc.), a server, and a database. Users use an application to register their daily necessities and send the necessary information to the server. The server stores the received information in a database, periodically analyzes the data, and uses generative AI to provide optimal replenishment timing and product information.

[1741] This configuration allows users to manage their daily necessities more efficiently, supporting them in replenishing their supplies when needed and making purchases at a good price.

[1742] The processing flow will be explained below.

[1743] Program processing (specific steps)

[1744] Manually inputting daily necessities

[1745] Step 1:

[1746] A user launches an application on their smartphone or tablet.

[1747] Step 2:

[1748] The user taps the "Register a new product" button and enters information such as the product name, purchase date, price, and store.

[1749] Step 3:

[1750] Check the information entered by the user and tap the "Register" button.

[1751] Step 4:

[1752] The terminal sends the entered data to the server in JSON format.

[1753] Step 5:

[1754] The server analyzes the received JSON data and generates an SQL statement to save it in the database.

[1755] Step 6:

[1756] The server executes the generated SQL statement and saves the new product information in the database.

[1757] Automatically register daily necessities from product photos and receipts

[1758] Step 1:

[1759] A user launches an application on their smartphone or tablet.

[1760] Step 2:

[1761] The user taps the "Scan Receipt" button and takes a photo of the receipt or item.

[1762] Step 3:

[1763] The device temporarily stores the captured image and sends the image data to the server.

[1764] Step 4:

[1765] The server receives the image and runs an image recognition algorithm (such as OCR) to extract product information.

[1766] Step 5:

[1767] The server analyzes the extracted product information and obtains information such as product name, purchase date, price, and store.

[1768] Step 6:

[1769] The server generates an SQL statement to save the product information to the database.

[1770] Step 7:

[1771] The server executes the generated SQL statement and saves the new product information in the database.

[1772] Usage analysis and replenishment forecast

[1773] Step 1:

[1774] The server runs a job that is triggered periodically to retrieve user commodity usage data from the database.

[1775] Step 2:

[1776] The server uses generated AI to analyze the acquired data and calculate each user's daily necessities usage rate.

[1777] Step 3:

[1778] The server predicts the next replenishment timing based on the calculation results.

[1779] Step 4:

[1780] The server sets a reminder based on the predicted replenishment timing.

[1781] Step 5:

[1782] The device receives the set reminder and displays a notification to the user.

[1783] Providing discount information and recommended products

[1784] Step 1:

[1785] The server retrieves the user's purchase history and usage information from the database.

[1786] Step 2:

[1787] The server uses generated AI to analyze the acquired data and generate a list of recommended products for each user.

[1788] Step 3:

[1789] The server retrieves the latest discount information from an external data source.

[1790] Step 4:

[1791] The server combines the recommended product list and discount information to generate information to be provided to the user.

[1792] Step 5:

[1793] The device displays the information received from the server and sends notifications to the user.

[1794] Example 1

[1795] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1796] Today's busy lives make it difficult for users to properly manage their daily necessities and determine when to replenish them. Furthermore, obtaining optimal discount information and recommended product information takes time and effort. To address these issues, a system is needed that can efficiently manage daily necessities, provide optimal replenishment timing, and provide users with personalized, useful information.

[1797] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1798] In this invention, the server includes: a means for a user to launch an application using a smart device and manually register information about daily necessities; a means for converting the manually entered data into JSON format and sending it to the server; a means for the server to store the received data in a database; a means for sending receipts and product photos to the server and analyzing the images using OCR technology; a means for the server to register product information extracted from the images in the database; a means for periodically analyzing the database data using a generative AI model to analyze consumption rates; a means for predicting replenishment timing and sending reminders to the terminal; a means for obtaining discount information and related product information from external data sources and generating personalized suggestions; and a means for sending discount information and recommended product information to the user's terminal. This enables users to efficiently manage and replenish daily necessities and receive personalized, useful product information and discount information.

[1799] "User" means an individual or corporation that uses the system to register and manage everyday items.

[1800] A "smart device" is a device, such as a smartphone or tablet, that can connect to the Internet and run applications.

[1801] "Application" means a software program that a user uses to register and manage everyday item information on a smart device.

[1802] "Daily necessities" are products that users use on a daily basis and consume, such as shampoo, detergent, and toilet paper.

[1803] "Manual input" refers to the act of a user directly entering information into an application's input form using a smart device.

[1804] "JSON format" stands for JavaScript Object Notation and is a lightweight data format for structuring and exchanging data.

[1805] "Server" means a computer system that receives and processes data sent by users and stores it in a database.

[1806] A "database" is a collection of data managed by a server, a system that allows for efficient information storage and retrieval.

[1807] A "receipt" is a paper or electronic record that contains details of a purchased item (e.g., item name, price, purchase date, etc.).

[1808] "OCR technology" is an abbreviation for Optical Character Recognition, and is a technology that automatically recognizes character information in an image and converts it into text data.

[1809] A "generative AI model" is an artificial intelligence model that uses technologies such as deep learning to learn patterns from data and make predictions and classifications.

[1810] A "reminder" is a notification sent by the server to a user's device, and is a message used to encourage a specific action.

[1811] "External data sources" are information sources from outside the system, including web services and databases that provide up-to-date discount and product information.

[1812] "Personalized suggestions" are product and service suggestions that are individually generated based on a user's usage and purchase history.

[1813] This invention is a system that allows users to efficiently manage daily necessities and provides optimal replenishment timing and personalized discount information. The system includes a means for users to manually register daily necessities, a means for automatically registering daily necessities from product photos and receipts, a means for analyzing usage status using a generative AI model, a means for predicting and reminding replenishment timing, and a means for providing personalized discount information and recommended product information.

[1814] System Configuration

[1815] Hardware

[1816] User device: A device that can connect to the internet, such as a smartphone or tablet.

[1817] Server: A computer system that receives and processes data and accesses a database.

[1818] Database: A collection of data managed by a server that stores information about everyday items.

[1819] software

[1820] Application: Software that users use on their devices to register and manage everyday items.

[1821] OCR technology: Software that analyzes photos of receipts or products and extracts text information. Example: Tesseract OCR.

[1822] Generative AI model: An artificial intelligence model that analyzes data and predicts replenishment timing. Example: TensorFlow.

[1823] Overview of operation procedure

[1824] Daily necessities registration

[1825] 1. Manual registration:

[1826] Users launch the application using their smartphone or tablet and manually enter information about the daily necessities they purchased (product name, purchase date, price, store information, etc.). After completing the entry, they press the "Register" button.

[1827] The terminal converts the input data into JSON format and sends it to the server.

[1828] The server stores the received data in a database.

[1829] Examples:

[1830] When a user enters shampoo purchase information into the input form and clicks the "Register" button, the device converts the input data into JSON format and sends it to the server, which then analyzes the received data and stores it in a database.

[1831] Example prompt sentence:

[1832] Please manually enter the shampoo information you purchased (product name, purchase date, price, store information) using your smartphone.

[1833] 2. Registration from an image:

[1834] The user takes a photo of the product or receipt of the purchased item, and then sends the image data to the server via the application.

[1835] The terminal transmits the captured image data to the server.

[1836] The server analyzes the received image data using OCR technology and registers the product information extracted from the image in a database.

[1837] Examples:

[1838] When a user takes a photo of a receipt using their smartphone camera and sends the image from the app to the server, the server activates the OCR engine to extract text data from the image, analyzes the extracted text data, and stores it in a database.

[1839] Example prompt sentence:

[1840] "Take a photo of the receipt or item you purchased and submit it through the application."

[1841] Usage analytics and reminders

[1842] The server periodically analyzes the information on daily items stored in the database, utilizing a generative AI model to analyze each user's usage and consumption rate of daily items.

[1843] Based on the analysis results, the server predicts when replenishment is needed and sends a reminder to the user's device.

[1844] Examples:

[1845] The server analyzes the shampoo usage history in the database and finds that, on average, users use up a bottle every three weeks. One week before the next bottle runs out, the server sends a reminder to the device saying, "You need to refill your shampoo."

[1846] Providing discount information and recommended products

[1847] The server generates personalized product information using a generative AI model based on each user's usage and purchase history, and also retrieves the latest discount information and related product information from external data sources to make optimal suggestions for the user.

[1848] The server transmits the generated discount information and recommended product information to the user's terminal.

[1849] Examples:

[1850] The server analyzes the user's shampoo purchase history and retrieves information about new and discounted products from the same brand from external data sources. Based on the information retrieved, a notification is sent to the device stating, "Recommended shampoo is now 20% off," and the user can check the details within the app.

[1851] This system helps users manage their daily necessities more efficiently, supporting them in replenishing their supplies when needed and making purchases at a good price.

[1852] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1853] Daily necessities registration phase

[1854] Step 1:

[1855] The user launches the application using a smart device and manually inputs information about the purchased daily necessities (product name, purchase date, price, store information, etc.). Once the input is complete, the user presses the "Register" button. The input data includes the product name, purchase date, price, and store information.

[1856] Specific behavior:

[1857] The user enters shampoo purchase information into the input form and clicks the "Register" button.

[1858] input:

[1859] Product information entered by the user (product name, purchase date, price, store information)

[1860] output:

[1861] JSON format data on the device

[1862] Step 2:

[1863] The terminal converts the manually entered data into JSON format, which includes the product name, purchase date, price, and store information.

[1864] Specific behavior:

[1865] The terminal converts the input data into JSON format.

[1866] input:

[1867] Product information entered by the user

[1868] output:

[1869] JSON format data

[1870] Step 3:

[1871] The terminal sends the converted JSON format data to the server.

[1872] Specific behavior:

[1873] The device starts the process of sending JSON data to the server.

[1874] input:

[1875] Structured product data in JSON format

[1876] output:

[1877] JSON data sent to the server

[1878] Step 4:

[1879] The server receives the JSON data sent from the device, analyzes it, and stores it in a database.

[1880] Specific behavior:

[1881] The server stores the received data in a database.

[1882] input:

[1883] JSON format data

[1884] output:

[1885] Information on everyday items in the database

[1886] Image information registration phase

[1887] Step 1:

[1888] The user takes a photo of the product or receipt of the purchased item, and then sends the image data to the server via the application.

[1889] Specific behavior:

[1890] The user takes a photo of the receipt using their smartphone camera and the app sends the image to the server.

[1891] input:

[1892] Receipts and product photos

[1893] output:

[1894] Image data stored on the device

[1895] Step 2:

[1896] The terminal transmits the captured image data to the server. The transmitted data is an image that may contain product information.

[1897] Specific behavior:

[1898] The device starts the process of sending the captured image to the server.

[1899] input:

[1900] Image data

[1901] output:

[1902] Image data sent to the server

[1903] Step 3:

[1904] The server analyzes the received image data using OCR technology and registers the product information extracted from the image (product name, purchase date, price, etc.) in a database.

[1905] Specific behavior:

[1906] The server starts the OCR engine, extracts text data from the image, analyzes the extracted text data, and stores it in a database.

[1907] input:

[1908] Image data

[1909] output:

[1910] Product information stored in the database

[1911] Usage analysis phase

[1912] Step 1:

[1913] The server periodically analyzes the information stored in the database and uses a generative AI model to analyze the usage and consumption rate of each item for each user.

[1914] Specific behavior:

[1915] The server runs scheduled jobs and performs data analysis using generative AI models.

[1916] input:

[1917] Information on everyday items in the database

[1918] output:

[1919] Usage and consumption rate per user

[1920] Step 2:

[1921] Based on the analysis results, the server predicts when each user's daily necessities will need to be replenished.

[1922] Specific behavior:

[1923] The server calculates the timing of replenishment based on the analysis results.

[1924] input:

[1925] AI model analysis results

[1926] output:

[1927] Predicted replenishment timing

[1928] Step 3:

[1929] The server sends a reminder to the user's device when the predicted replenishment time approaches.

[1930] Specific behavior:

[1931] The server generates a reminder notification and sends it to the device.

[1932] input:

[1933] Predicted replenishment timing

[1934] output:

[1935] Reminder messages sent to your device

[1936] Discount information and recommended products phase

[1937] Step 1:

[1938] The server generates personalized product information using a generative AI model based on each user's usage and purchase history, and also retrieves the latest discount information and related product information from external data sources.

[1939] Specific behavior:

[1940] The server periodically calls an external API to retrieve the latest discount information.

[1941] input:

[1942] User usage, purchase history, external data

[1943] output:

[1944] Personalized product information

[1945] Step 2:

[1946] Based on the collected data, the server generates and stores discount information and recommended product information that is optimal for the user.

[1947] Specific behavior:

[1948] The server applies an algorithm based on the information it obtains to generate personalized suggestions.

[1949] input:

[1950] Data analysis results, external data

[1951] output:

[1952] Saved discounts and recommended products

[1953] Step 3:

[1954] The server transmits the generated discount information and recommended product information to the user's terminal.

[1955] Specific behavior:

[1956] The server generates the proposal as a notification and sends it to the device.

[1957] input:

[1958] Generated proposals

[1959] output:

[1960] Notification messages sent to the device

[1961] (Application example 1)

[1962] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1963] Conventional daily necessities management systems have the drawback of making it difficult to properly determine when to replenish items because they require users to register their purchases and do not adequately analyze usage.Furthermore, they do not provide users with personalized discount information or recommended products, making it difficult to efficiently manage daily necessities and purchase the products they need at the optimal time.

[1964] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1965] In this invention, the server includes a means for users to manually register daily necessities, a means for automatically registering daily necessities from product photos and receipts, and a means for analyzing the usage of registered daily necessities using a generating AI. This allows users to easily register daily necessities, and the analysis of usage using the generating AI makes it possible to predict the optimal timing for replenishment and send reminders. Furthermore, the server includes a means for predicting the usage rate and replenishment timing using a generating AI model, a means for extracting and automatically registering product information using image recognition, and a means for obtaining and providing discount information and recommended product information from external data sources. This allows users to efficiently manage their daily necessities and purchase products at great prices based on personalized discount information.

[1966] "Means for manually registering daily necessities" is a function that allows users to manually enter product name, purchase date, price, store information, etc. to register daily necessities in the system.

[1967] "Means for automatically registering daily necessities from product photos and receipts" is a function that analyzes product photos and receipt images taken by users, automatically extracts product information, and registers it in the system.

[1968] "Means for analyzing usage status using generative AI" refers to a function that uses a generative AI model to analyze information on stored daily necessities, and analyzes the pace of use and consumption of daily necessities for each user.

[1969] "Means to predict and remind the timing of replenishment" is a function that predicts the next time replenishment is required based on the results of analysis by the generation AI and notifies the user.

[1970] "Means for providing personalized discount information and recommended product information" refers to a function that uses generative AI to provide optimal discount information and related product information based on each user's usage and purchase history.

[1971] "Means for predicting usage rate and replenishment timing using a generative AI model" is a function that uses a generative AI model to predict the usage rate and optimal replenishment timing of registered daily necessities.

[1972] "Means for extracting and automatically registering product information using image recognition" refers to a function that uses OCR technology or image recognition algorithms to extract product information from receipt or product images and automatically registers it in the system.

[1973] "Means for obtaining and providing discount information and recommended product information from external data sources" refers to a function that analyzes the latest discount information and recommended product information obtained from external data sources and provides it to users in a personalized form.

[1974] MODE FOR CARRYING OUT THE INVENTION

[1975] The present invention is a system that allows users to efficiently manage their daily necessities, and provides optimal replenishment timing and personalized discount information. This system operates using a smartphone application, a server, and a database.

[1976] System configuration

[1977] User Device

[1978] Users launch the application using a device such as a smartphone or tablet. There are two ways to register daily necessities: manual entry or automatic registration.

[1979] 1. Manual registration

[1980] The user enters the product name, purchase date, price, store information, etc. into an input form and sends this information in JSON format to the server.

[1981] 2. Automatic Registration

[1982] Users take photos of receipts or products and send the images to the server, which uses OCR technology to extract product information from the images and register it in a database.

[1983] Servers and Databases

[1984] 1. Usage analysis

[1985] The server uses a generative AI model to analyze information about daily necessities stored in a database, and analyzes each user's pace of use and consumption of daily necessities.

[1986] 2. Predicting replenishment timing

[1987] Based on the analysis results, the server predicts when the next replenishment is required and sends a reminder to the user.

[1988] 3. Providing discount information and recommended products

[1989] The server retrieves the latest discount information and related product information from external data sources and generates personalized product information using a generative AI model.

[1990] Based on this information, the server provides users with optimal discount information and related product information.

[1991] Specific examples of implementation methods

[1992] Manual registration example

[1993] When a user purchases shampoo, they manually enter product information using the application, including details such as the purchase date, product name, and price, and send the information to the server, which then registers the received information in the database.

[1994] Auto-registration Example

[1995] The user takes a photo of the receipt and sends it to the server via the application. The server uses OCR technology to extract the purchase date, product name, price, etc. from the image and registers them in the database. This allows the user to register product information without any hassle.

[1996] Usage analysis example

[1997] The server analyzes the shampoo usage history stored in the database and finds that the average user uses up a bottle every three weeks. One week before the next bottle of shampoo runs out, the server sends a reminder to the user's smartphone, informing them that they need to refill their shampoo.

[1998] Examples of discounts and product recommendations

[1999] The server analyzes the user's shampoo purchase history and retrieves information about new and discounted products from the same brand from external data sources. Based on the information retrieved, the server sends the user a notification saying, "Recommended shampoo is now 20% off," and allows them to check the details within the app.

[2000] Examples of using generative AI models and prompts

[2001] An example of a prompt that the server uses to perform analysis using a generative AI model:

[2002] This system is designed to help you efficiently manage your daily necessities, provide optimal replenishment timing, and provide personalized discount information. Simply take a photo of your receipt or product and register it in the system, and you will receive timely replenishment notifications and discount information.

[2003] This allows users to efficiently manage their daily necessities and purchase the products they need at the best possible time.

[2004] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2005] Step 1:

[2006] Users manually register daily items

[2007] The user launches the smartphone application and enters the product name, purchase date, price, store information, etc. into the input form. The entered information is sent to the server in JSON format. The server parses the received JSON data and registers it in the database.

[2008] Input: Product name, purchase date, price, store information (user input)

[2009] Output: Product information is saved in the database

[2010] Step 2:

[2011] Users take photos of receipts and products and register them

[2012] A user uses a smartphone application to take a photo of a receipt or product. The image data is then uploaded to a server. The server uses OCR technology to extract text information from the image and convert it into data such as product name, purchase date, price, and store information. This data is then registered in a database.

[2013] Input: Image of receipt or product (taken by user)

[2014] Output: The extracted product information is saved in the database.

[2015] Step 3:

[2016] The server analyzes the usage of everyday items

[2017] The server periodically analyzes the information on daily items stored in the database using a generative AI model, which calculates each user's usage rate and consumption rate of daily items.

[2018] Input: Information about everyday items stored in a database

[2019] Output: Analysis results of usage pace and wear rate (analysis results from generative AI model)

[2020] Step 4:

[2021] Refill forecast and reminder sending

[2022] The server predicts when the next refill is needed based on the analysis results of the generative AI model. Based on the prediction, a reminder notification is sent to the user's smartphone. For example, if shampoo runs out within one week of the predicted time, the user will be notified that "shampoo needs to be refilled."

[2023] Input: Analysis results of the generative AI model

[2024] Output: Reminder notification for refilling (sent to user's smartphone)

[2025] Step 5:

[2026] Providing discount information and recommended products

[2027] The server uses a generative AI model based on each user's usage and purchase history to generate personalized product information. It also obtains discount information and recommended product information from external data sources and stores it in a database. The generated information is then combined to notify users of appropriate discount information and recommended products.

[2028] Inputs: User usage, purchase history, discount information from external data sources

[2029] Output: Notification of recommended products and discount information (sent to the user's smartphone)

[2030] This allows users to efficiently manage their daily necessities and replenish them at the optimal time and at a good price.

[2031] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2032] This invention combines an emotion engine with a system that allows users to efficiently manage their daily necessities and provides optimal replenishment timing and personalized discount information, thereby realizing personalized suggestions based on user emotion recognition. This system includes a means for users to manually register daily necessities, a means for automatically registering items from product photos and receipts, a means for analyzing usage status using generative AI, a means for predicting and reminding users when replenishment is necessary, a means for providing personalized discount information and recommended product information, and a means for recognizing user emotions using the emotion engine.

[2033] Program processing

[2034] 1. Manual and automatic commodity registration

[2035] Users launch the application using a device such as a smartphone or tablet. When registering manually, users enter the product name, purchase date, price, store information, etc. into an input form. The device sends this information in JSON format to the server, which then stores the received data in a database.

[2036] When automatically registering using product photos or receipts, users take a photo and send it to the server from their device. The server then uses an image recognition algorithm (such as OCR) to extract product information from the image and register it in the database.

[2037] Examples:

[2038] When a user purchases shampoo, they take a photo of the receipt. The image is sent to a server, and OCR technology extracts the purchase date, product name, price, etc., and registers them in a database. This allows users to register product information in the system without any hassle.

[2039] 2. Analyzing usage and predicting replenishment timing

[2040] The server periodically analyzes the information about daily necessities stored in the database. Utilizing generative AI, it analyzes each user's usage and consumption rate of daily necessities and predicts when replenishment is necessary. Based on this information, the server reminds the user of the optimal time to replenish.

[2041] Examples:

[2042] The server analyzes the shampoo usage history in the database and finds that, on average, users use up a bottle every three weeks. One week before the next bottle of shampoo runs out, the server sends a reminder to the device, informing the user that the shampoo needs to be refilled.

[2043] 3. Personalized discounts and product recommendations

[2044] The server uses AI to generate personalized product information based on each user's usage and purchase history, and also retrieves the latest discount information and related product information from external data sources to make optimal suggestions for the user.

[2045] Examples:

[2046] The server analyzes the user's shampoo purchase history and retrieves information about new and discounted products from the same brand from external data sources. Based on the information retrieved, a notification is sent to the device stating, "Recommended shampoo is now 20% off," and the user can check the details within the app.

[2047] 4. Emotion Recognition and Personalization with Emotion Engine

[2048] The emotion engine recognizes emotions based on user input and usage, and tailors personalized product recommendations and reminders accordingly.

[2049] Examples:

[2050] If a user types "I'm tired" into the app, the emotion engine will recognize that emotion and suggest products to reduce stress (e.g., relaxing bath salts). If the user's emotion is fatigued, the app will also express the reminder in gentler terms, such as "Thank you for your hard work. Your next shampoo is needed."

[2051] 5. Storage and utilization of emotional data

[2052] The server periodically stores the user's emotional data recognized by the emotion engine in a database, which will be used for future personalized suggestions, such as more personalized product suggestions and tailoring of reminder content.

[2053] Examples:

[2054] Based on the user's emotional history, the system analyzes times when stress is likely to increase and provides focused recommendations on relaxation products and other products that will help relieve stress.

[2055] Implementation environment

[2056] The system operates in a network environment that includes user devices (smartphones, tablets, etc.), a server, and a database. Users use an application to register their daily necessities and send the necessary information to the server. The server stores the received information in a database and analyzes it using generative AI and an emotion engine to provide optimal replenishment timing and personalized product information.

[2057] This configuration allows for more efficient management of daily necessities for users, and makes it possible to provide optimal product suggestions and replenishment notifications tailored to each individual's emotional state.

[2058] The processing flow will be explained below.

[2059] Manually inputting daily necessities

[2060] Step 1:

[2061] A user launches an application on their smartphone or tablet.

[2062] Step 2:

[2063] The user taps the "Register a new product" button and enters information such as the product name, purchase date, price, and store.

[2064] Step 3:

[2065] Check the information entered by the user and tap the "Register" button.

[2066] Step 4:

[2067] The terminal sends the entered data to the server in JSON format.

[2068] Step 5:

[2069] The server analyzes the received JSON data and generates an SQL statement to save it in the database.

[2070] Step 6:

[2071] The server executes the generated SQL statement and saves the new product information in the database.

[2072] Automatically register daily necessities from product photos and receipts

[2073] Step 1:

[2074] A user launches an application on their smartphone or tablet.

[2075] Step 2:

[2076] The user taps the "Scan Receipt" button and takes a photo of the receipt or item.

[2077] Step 3:

[2078] The device temporarily stores the captured image and sends the image data to the server.

[2079] Step 4:

[2080] The server receives the image and runs an image recognition algorithm (such as OCR) to analyze the extracted product information.

[2081] Step 5:

[2082] The server analyzes the extracted product information and obtains information such as the product name, purchase date, price, and store.

[2083] Step 6:

[2084] The server generates an SQL statement to save the product information to the database.

[2085] Step 7:

[2086] The server executes the generated SQL statement and saves the new product information in the database.

[2087] Usage analysis and replenishment forecast

[2088] Step 1:

[2089] The server runs a job that is triggered periodically to retrieve user commodity usage data from the database.

[2090] Step 2:

[2091] The server uses generated AI to analyze the collected data and calculate each user's daily necessities usage rate.

[2092] Step 3:

[2093] The server predicts the next replenishment timing based on the calculation results.

[2094] Step 4:

[2095] The server sets a reminder based on the predicted replenishment timing.

[2096] Step 5:

[2097] The device receives the set reminder and displays a notification to the user.

[2098] Providing discount information and recommended products

[2099] Step 1:

[2100] The server retrieves the user's purchase history and usage information from the database.

[2101] Step 2:

[2102] The server uses a generation AI to analyze the acquired data and generate a list of recommended products for each user.

[2103] Step 3:

[2104] The server retrieves the latest discount information from an external data source.

[2105] Step 4:

[2106] The server combines the recommended product list and discount information to generate information to be provided to the user.

[2107] Step 5:

[2108] The device displays the information received from the server and sends notifications to the user.

[2109] Emotion recognition and personalization with emotion engine

[2110] Step 1:

[2111] Providing information about emotions that users input into the application (e.g., tired, stressed, etc.).

[2112] Step 2:

[2113] The emotion engine analyzes the user's input data and recognizes the user's emotions.

[2114] Step 3:

[2115] The server generates a personalized product suggestion list based on the analysis results of the emotion engine.

[2116] Step 4:

[2117] The server adjusts the content of the reminder to use gentle language and appropriate expressions to match the user's emotions.

[2118] Step 5:

[2119] The device displays personalized suggestions and reminders received from the server and sends notifications to the user.

[2120] Storing and utilizing emotional data

[2121] Step 1:

[2122] The server stores the user's emotion data recognized using the emotion engine in a database.

[2123] Step 2:

[2124] The server periodically analyzes the stored emotional data to understand the user's emotional patterns.

[2125] Step 3:

[2126] The server uses the emotion data to tailor future personalized suggestions and reminders.

[2127] Step 4:

[2128] The server provides product information for stress reduction and relaxation in a timely manner according to the user's emotional state.

[2129] Step 5:

[2130] The device displays the suggestion information based on the emotion received from the server and sends a notification to the user.

[2131] Example 2

[2132] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2133] Conventional daily necessities management systems make it difficult for users to efficiently obtain information on replenishment timing and discounts for daily necessities, and do not provide personalized suggestions based on the user's emotional state. As a result, they have not been able to fully improve the user experience or stimulate purchasing motivation. To solve this, a system that takes into account not only the user's usage status but also their emotional state is needed.

[2134] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2135] In this invention, the server includes a means for a user to manually register daily necessities, a means for automatically registering daily necessities from product photos and receipts, a means for analyzing the usage of the registered daily necessities using a generative AI model, a means for predicting and reminding the user to replenish based on the analysis results, a means for providing personalized discount information and recommended product information to the user, and an emotion engine for recognizing the user's emotions and making personalized suggestions based on those emotions. This makes it possible to streamline the user's daily necessities management and make optimal product suggestions and replenishment notifications based on the user's emotions.

[2136] "User" means an individual or organization that uses the system to manage and register everyday items.

[2137] "Daily commodities" are consumables and products that users use on a daily basis.

[2138] "Manual registration" refers to an operation in which a user directly inputs information about a daily necessities using a terminal.

[2139] "Automatic registration" is a method of automatically extracting and registering information about everyday items from product photos and receipts.

[2140] A "generative AI model" is an algorithm that analyzes the usage of everyday items based on data and generates personalized information.

[2141] The "emotion engine" is a system component that recognizes emotions based on user input and usage and makes corresponding suggestions.

[2142] "Replenishment timing" refers to the next time a user should purchase or replenish daily necessities.

[2143] "Remind" refers to sending notifications or information to users to encourage them to take necessary action.

[2144] "Personalization" refers to customizing information and offers based on a user's individual usage and emotional state.

[2145] "Discount Information" means information about a price reduction applied to a product or service.

[2146] "Recommended Products" are products suggested to users based on their usage and emotional state.

[2147] The "database" is a system that stores information about everyday items registered by users and analysis results.

[2148] "Analysis" is the process of deriving specific information or results from collected data.

[2149] A "Notification" is a message or alert that conveys a reminder or offer to the User.

[2150] This invention combines an emotion engine with a system that allows users to efficiently manage their daily necessities and provides optimal replenishment timing and personalized discount information, thereby realizing personalized suggestions based on user emotion recognition. A detailed description of specific embodiments of this system is provided below.

[2151] 1. Manual and automatic commodity registration

[2152] Users launch the application using a device such as a smartphone or tablet. When registering manually, users enter the product name, purchase date, price, store information, etc. into an input form. The device sends this information in JSON format to the server, which then stores the received data in a database.

[2153] When automatically registering using product photos or receipts, users take a photo and send it to the server from their device. The server then uses an image recognition algorithm (such as OCR) to extract product information from the image and register it in the database.

[2154] Examples:

[2155] When a user purchases shampoo, they take a photo of the receipt. The image is sent to a server, and OCR technology extracts the purchase date, product name, price, etc., and registers them in a database. This allows users to register product information in the system without any hassle.

[2156] 2. Analyzing usage and predicting replenishment timing

[2157] The server periodically analyzes the information about daily necessities stored in the database. Utilizing generative AI, it analyzes each user's usage and consumption rate of daily necessities and predicts when replenishment is necessary. Based on this information, the server reminds the user of the optimal time to replenish.

[2158] Examples:

[2159] The server analyzes the shampoo usage history in the database and finds that, on average, users use up a bottle every three weeks. One week before the next bottle of shampoo runs out, the server sends a reminder to the device, informing the user that the shampoo needs to be refilled.

[2160] 3. Personalized discounts and product recommendations

[2161] The server uses AI to generate personalized product information based on each user's usage and purchase history, and also retrieves the latest discount information and related product information from external data sources to make optimal suggestions for the user.

[2162] Examples:

[2163] The server analyzes the user's shampoo purchase history and retrieves information about new and discounted products from the same brand from external data sources. Based on the information retrieved, a notification is sent to the device stating, "Recommended shampoo is now 20% off," and the user can check the details within the app.

[2164] 4. Emotion Recognition and Personalization with Emotion Engine

[2165] The emotion engine recognizes emotions based on user input and usage, and tailors personalized product recommendations and reminders accordingly.

[2166] Examples:

[2167] If a user types "I'm tired" into the app, the emotion engine will recognize that emotion and suggest products to reduce stress (e.g., relaxing bath salts). If the user's emotion is fatigued, the app will also express the reminder in gentler terms, such as "Thank you for your hard work. Your next shampoo is needed."

[2168] 5. Storage and utilization of emotional data

[2169] The server periodically stores the user's emotional data recognized by the emotion engine in a database, which will be used for future personalized suggestions, such as more personalized product suggestions and tailoring of reminder content.

[2170] Examples:

[2171] Based on the user's emotional history, the system analyzes times when stress is likely to increase and provides focused recommendations on relaxation products and other products that will help relieve stress.

[2172] This system allows users to efficiently manage their daily necessities and receive optimal product suggestions and replenishment notifications tailored to their individual emotional state.

[2173] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2174] Step 1:

[2175] A user launches the application using a device such as a smartphone or tablet. When manually registering a daily necessities item, the user enters the product name, purchase date, price, store information, etc. into an input form. The device sends the entered information in JSON format to the server. The server analyzes the received data and stores it in a database.

[2176] Input: Product name, purchase date, price, store information

[2177] Output: Product information registered in the database

[2178] Step 2:

[2179] The user takes a photo of a product or receipt and sends the image data from their device to the server. The server analyzes the received image data using an image recognition algorithm (such as OCR) and extracts product information. The extracted information is registered in a database.

[2180] Input: Product photo or receipt image data

[2181] Output: Product information registered in the database

[2182] Step 3:

[2183] The server periodically analyzes the information on daily items stored in the database and uses a generative AI model to calculate each user's usage and consumption rate of daily items.

[2184] Input: Daily necessities information in the database

[2185] Output: Analysis of usage pace and wear rate

[2186] Step 4:

[2187] Based on the analysis results, the server predicts when replenishment is necessary, calculates the next replenishment time, and sends a reminder to the user.

[2188] Input: Analysis results of usage pace and wear rate

[2189] Output: Reminder to user

[2190] Step 5:

[2191] The server uses AI to generate personalized product information based on each user's usage and purchase history, and retrieves the latest discount information and related product information from external data sources to make optimal suggestions to users.

[2192] Input: Information from usage, purchase history, and external data sources

[2193] Output: Personalized product and discount information

[2194] Step 6:

[2195] When users input their emotions into the app, the emotion engine analyzes them and tailors personalized suggestions and reminders.

[2196] Input: User emotion input

[2197] Output: Emotion-based personalized suggestions and reminders

[2198] Step 7:

[2199] The server periodically stores the user's emotion data recognized by the emotion engine in a database, which will be used for future personalized suggestions.

[2200] Input: Analysis results by emotion engine

[2201] Output: Emotion data stored in a database

[2202] Step 8:

[2203] Based on the stored emotional data, the server analyzes the periods when stress is likely to increase for each user and adjusts product suggestions and reminder content to suit that period.

[2204] Input: Emotion data stored in a database

[2205] Output: Personalized suggestions and reminders based on specific times

[2206] (Application example 2)

[2207] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2208] In conventional daily necessities management systems, registering users' daily necessities and managing replenishment timings are often done manually, making efficient management difficult. Furthermore, product suggestions based on users' emotions and personalized information provision are not provided, making it difficult to fully address individual user needs. The present invention aims to solve these problems by streamlining daily necessities management and product suggestions for users, and realizing personalized responses based on emotions.

[2209] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for a user to manually register daily necessities, a means for automatically registering daily necessities from product images and receipts, a means for analyzing the usage status of the registered daily necessities using a generation AI, a means for predicting replenishment timing based on the analysis results and reminding the user, a means for providing personalized discount information and recommended product information to the user, a means for recognizing the user's emotions and making personalized product suggestions based on the emotions, and a means for scanning products using smart glasses or a smartphone and automatically registering and suggesting products. This improves the efficiency of the user's daily necessities management and enables product suggestions that meet individual needs based on emotions.

[2210] "Daily necessities" refer to consumables and household items that are frequently used in daily life.

[2211] "Manual registration" means that the user manually enters the information.

[2212] "Product images" refer to photographs or image data of products that are the subject of daily necessities.

[2213] "Receipt" refers to a proof of purchase that details the daily necessities purchased.

[2214] "Automatic registration" refers to a method of automatically extracting and registering product information from product images and receipts using image recognition technology, OCR, etc.

[2215] "Generative AI" refers to artificial intelligence that uses algorithms to generate and analyze data.

[2216] "Analysis" is the process of analyzing data, extracting information, and understanding it.

[2217] "Predicting replenishment timing" refers to analyzing the usage rate of consumables and predicting when the next replenishment will be required.

[2218] "Reminding" means sending a notification to the user prompting them to take necessary action.

[2219] "Personalized Discounts" refers to special discounts provided to you based on your individual needs and preferences.

[2220] "Recommended product information" refers to product information recommended based on a user's past purchase history and usage status.

[2221] "Emotion recognition" refers to technology that determines emotions from a user's facial expressions and input.

[2222] "Personalized product suggestions" refers to suggesting products that are individually suited to the user's emotional state and usage situation.

[2223] "Smart glasses" refers to eyeglass-like devices with built-in computer functions.

[2224] A "smartphone" refers to a highly functional mobile phone that has calling and internet connection capabilities.

[2225] "Scanning a product" refers to reading product information using smart glasses or a smartphone.

[2226] A system for implementing this invention includes a user terminal (smartphone, smart glasses), a server, a network, and a database.

[2227] First, users manually register everyday items using their smartphones or smart glasses. They launch the application and enter information such as the product name, purchase date, and price. This information is then sent to the server in JSON format and stored in a database.

[2228] Next, a method is provided to automatically register everyday items from product images or receipts. Users use their devices to take photos of product images or receipts and send them to the server. The server then uses OCR technology (e.g., Tesseract OCR) to extract product information from the images and register it in a database.

[2229] The server periodically analyzes the usage of registered daily items using AI. For example, it analyzes the pace at which a user consumes a particular daily item and predicts when it will need to be replenished. Based on this information, the server sends reminders to the user to inform them when it is time to replenish.

[2230] In addition, the server provides users with personalized discount information and recommended product information. Based on the user's usage and purchase history, the AI ​​generator selects the most suitable discount information and recommended products, allowing users to obtain the product information that best suits them.

[2231] An emotion engine is a way to recognize a user's emotions and make personalized product recommendations. For example, a user can input "I'm tired" into smart glasses or a smartphone, or read emotions from a facial image. As a result, it can recommend products that reduce stress (e.g., relaxing bath salts or aroma candles).

[2232] It also includes a way for users to scan products in physical stores using smart glasses or smartphones, which will automatically register and suggest products. For example, when a user scans a shampoo in the store, detailed information about the product will be displayed and product suggestions will be made based on emotions.

[2233] This system uses smartphones, smart glasses, and servers as hardware, and OCR technology (Tesseract OCR), generative AI (Keras, etc.), and an emotion engine as software. Data is sent and received over a network, and information is managed in a database.

[2234] Specific examples

[2235] When a user is shopping in a physical store, they use smart glasses to scan the barcode of a shampoo. The system automatically retrieves product information and displays it to the user. If the system recognizes that the user is feeling fatigued, it will suggest relaxing bath salts. Below is an example of a prompt for this process:

[2236] "Recommend the best daily essentials and add-ons based on the user's emotions and purchase history. If the user is stressed, suggest products that will help them relax."

[2237] As described above, this system will improve the efficiency of users' daily necessities management and enable product suggestions that meet individual emotional needs.

[2238] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2239] Step 1:

[2240] Users register everyday items.

[2241] Input: The user manually enters the product name, purchase date, and price, or takes a photo of the product or receipt.

[2242] Processing: In the case of manual entry, the device converts the information entered by the user into JSON format and sends it to the server. In the case of automatic registration, the device takes a photo and sends it to the server.

[2243] Output: In case of manual input, the server stores the received JSON data in the database. In case of automatic registration, the server uses OCR technology to extract product information and stores it in the database.

[2244] Step 2:

[2245] Usage is analyzed using generative AI.

[2246] Input: Registration data of everyday items stored in the database.

[2247] Processing: The server periodically retrieves the registration data from the database and analyzes it using a generative AI model (e.g., using Keras). It analyzes consumption patterns and usage history.

[2248] Output: Analytics showing usage and consumption pace.

[2249] Step 3:

[2250] Predicts replenishment timing and sends reminders.

[2251] Input: The analysis results from step 2.

[2252] Processing: The server predicts when the next replenishment is required based on the consumption pace provided by the generation AI. A reminder notification is generated based on the prediction result.

[2253] Output: A reminder notification is sent to the user's device, with a message such as "Your shampoo needs refilling."

[2254] Step 4:

[2255] Providing personalized discounts and product recommendations.

[2256] Input: User usage and purchase history.

[2257] Processing: The server uses a generative AI model to select optimal discounts and product recommendations based on usage and purchase history, and also retrieves the latest discounts and related product information from external data sources.

[2258] Output: Personalized discount information and product recommendations are sent to the user's device.

[2259] Step 5:

[2260] Recognize emotions and make personalized product recommendations.

[2261] Input: User's face image and text input.

[2262] Processing: The device takes a picture of the user's face and sends it to the server. The server uses an emotion recognition model to determine the emotion from the facial image. The server also recognizes emotions when the user enters their emotion in text. The server then makes appropriate product suggestions based on the emotion.

[2263] Output: Personalized product suggestions based on emotions are displayed on the device.

[2264] Step 6:

[2265] Products are scanned using smart glasses or a smartphone, and are automatically registered and suggested.

[2266] Input: Barcode or QR code of the product in the physical store.

[2267] Processing: The device (smart glasses or smartphone) scans the product and sends the product information to the server. The server analyzes the product information, registers it in a database, and suggests appropriate products based on the user's emotional data.

[2268] Output: Product details and recommended product information are displayed on the device.

[2269] In this way, the system can streamline users' daily necessities management and provide personalized, emotion-based suggestions.

[2270] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2271] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2272] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2273] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2274] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2275] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2276] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2277] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2278] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2279] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2280] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general in...

Claims

1. A way for users to manually register everyday items, A way to automatically register daily necessities from product photos and receipts, A means of analyzing the usage of registered daily necessities using generation AI, A means of predicting and reminding replenishment timing based on the analysis results; A system including a means for providing users with personalized discount information and product recommendations.

2. A means for analyzing image data received from a user and automatically extracting product information; 2. The system according to claim 1, further comprising means for registering the extracted product information in a database.

3. A method to periodically analyze the usage rate of registered daily items and calculate the next replenishment timing, The system of claim 1 further comprising means for sending a reminder to the user based on the calculation result.

4. A means to obtain the user's purchase history from the database and generate a personalized list of recommended products using generative AI. A means of retrieving up-to-date discount information from external data sources; and 10. The system according to claim 1, further comprising means for integrating recommended product lists and discount information and providing the same to the user.

5. A means to provide premium features based on users' daily product usage and purchase history, Partner with retailers or manufacturers to provide new product information and discounts. The system according to claim 1, further comprising means for receiving affiliate income based on the provided product information.

Images