System

Abstract

A system is provided.SOLUTION: A system comprising: input means for a user to scan product information; processing means for transmitting the input product information to a database and acquiring corresponding product information; management means for creating and updating a list of users based on the acquired product information; analysis means for analyzing purchase data and consumption of the user and calculating an optimal replacement time; notification means for notifying the user of a reminder based on the calculated replacement time; and monetization means for providing advertisement display and a premium function.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] The challenge is to eliminate the stress and inefficiency caused by forgetting to buy daily necessities and consumables or by purchasing duplicates. There is also a need for a system that allows users to purchase the products they need in a timely manner. Furthermore, it is important that such a system ensures profitability. [Means for solving the problem]

[0005] The present invention utilizes a barcode scanner as a means for users to input product information. It includes a processing means for sending the input product information to a database and retrieving the corresponding product information. It also provides a management means for creating and updating the user's list based on the retrieved product information. It also includes an analysis means for analyzing the user's purchase data and consumption amount using AI to calculate the optimal replacement timing. It also includes a notification means for notifying the user of reminders based on the calculated replacement timing. Finally, it incorporates a monetization means for displaying advertisements and providing premium features. This system prevents users from forgetting to buy or making duplicate purchases, provides an efficient shopping experience, and ensures revenue.

[0006] "Input means" refers to a device or interface that allows a user to provide product information to the system.

[0007] "Processing means" refers to hardware and software for transmitting input product information to a database and retrieving the corresponding product information.

[0008] "Management means" refers to a system component that has the function of creating and updating a user's list based on acquired product information.

[0009] "Analysis methods" refers to algorithms and technologies that analyze user purchasing data and consumption amounts to calculate the optimal time to replace items.

[0010] "Notification means" refers to a mechanism for notifying the user of reminders based on the calculated replacement period.

[0011] "Monetization methods" refers to the methods and processes for generating revenue through displaying advertisements or providing premium features.

[0012] "Barcode scanner" refers to a device that reads barcodes attached to products. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0014] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0017] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0018] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0019] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0021] [First embodiment]

[0022] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0023] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0026] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0029] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0033] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0034] This invention is a system that helps users avoid forgetting to buy things and supports efficient shopping by allowing them to create a list by scanning product information and receiving reminders at the appropriate time.The operation of each part of the system is described in detail below.

[0035] List creation function

[0036] 1. The user launches the app and logs in

[0037] The user launches the smartphone app and logs in.

[0038] The terminal checks the user's authentication information and displays the main screen if the login is successful.

[0039] 2. Access the list creation screen

[0040] Users access the "Create a List" section from the app's menu.

[0041] The device displays a list creation screen.

[0042] 3. Enter product information

[0043] The user scans the product using the barcode scanning function.

[0044] The terminal reads the barcode and transmits the information to the server.

[0045] 4. Obtain product information

[0046] The server searches the database for the corresponding product information based on the received barcode information.

[0047] The server returns the product information to the terminal.

[0048] The terminal displays the product information and updates the user's list.

[0049] For example, if a user wants to add milk to the list, they scan the barcode of the milk. The server retrieves the product data from the barcode information and displays it on the terminal. The user confirms and adds it to the list.

[0050] Reminder function

[0051] 1. Data collection and analysis

[0052] The server collects user purchase data and consumption data for each product.

[0053] The server uses AI algorithms to analyze the data and predict the optimal time to replace each product.

[0054] 2. Create and send reminders

[0055] The server creates reminders based on the analysis results.

[0056] The server sends the reminder to the device at the appropriate time.

[0057] The device displays the reminder to the user as a push notification.

[0058] For example, if a user regularly buys milk, the server will analyze their past consumption patterns and predict when they will need to buy it again, and a reminder will be sent to the user's device at that time.

[0059] Monetization methods

[0060] 1. Advertising

[0061] The server manages in-app advertising display settings.

[0062] The device displays advertisements at appropriate times while the user is using the app.

[0063] 2. Premium Features

[0064] The user reviews the details of the premium features and selects a subscription.

[0065] The terminal transmits the user's payment information to the server.

[0066] The server processes the payment and, if successful, notifies the device to unlock the premium features.

[0067] For example, if a user wants to purchase a premium feature to remove ads, they select the subscription and make the payment. The server verifies the payment and the premium feature is activated.

[0068] As described above, this system helps users to shop efficiently and stress-free, and is realized by the cooperation of each function.

[0069] The processing flow will be explained below.

[0070] List creation function

[0071] Step 1:

[0072] The user launches the smartphone app and logs in.

[0073] Step 2:

[0074] The device checks the user's authentication information and displays the main screen if the login is successful.

[0075] Step 3:

[0076] The user accesses the "Create a List" section from the app's menu.

[0077] Step 4:

[0078] The device displays the list creation screen.

[0079] Step 5:

[0080] The user uses the barcode scanning feature to add everyday items to the list.

[0081] Step 6:

[0082] The device activates the smartphone camera and reads the barcode.

[0083] Step 7:

[0084] The terminal transmits the barcode information to the server.

[0085] Step 8:

[0086] The server searches the database for and retrieves the relevant product information.

[0087] Step 9:

[0088] The server returns product information (product name, category, price) to the terminal.

[0089] Step 10:

[0090] The terminal adds the acquired product information to the user's list and updates the screen.

[0091] Step 11:

[0092] The user confirms the product addition, manually enters the product name if necessary, and adds the item to the list.

[0093] Step 12:

[0094] The user finishes updating the list and taps the save button.

[0095] Step 13:

[0096] The terminal transmits the list information to the server.

[0097] Step 14:

[0098] The server saves the list in a database and sends a message to the terminal indicating that the list has been saved.

[0099] Step 15:

[0100] The device displays a notification to the user that the save is complete.

[0101] Reminder function

[0102] Step 1:

[0103] The server periodically collects user purchase data and consumption data for each product.

[0104] Step 2:

[0105] The server uses AI algorithms to analyze the data and predict the optimal time to replace each product.

[0106] Step 3:

[0107] The server creates reminders based on the analysis results.

[0108] Step 4:

[0109] The server sends the created reminder to the device at the appropriate time.

[0110] Step 5:

[0111] The device displays the reminder to the user as a push notification.

[0112] Step 6:

[0113] The user checks the reminder and checks the shopping list.

[0114] Monetization methods

[0115] Step 1:

[0116] The server manages in-app ad display settings.

[0117] Step 2:

[0118] The device displays ads at appropriate times while the user is using the app.

[0119] Step 3:

[0120] The user reviews the premium features and selects a subscription service.

[0121] Step 4:

[0122] The user enters payment information to purchase a subscription and taps the "Purchase" button.

[0123] Step 5:

[0124] The terminal sends the payment information to the server.

[0125] Step 6:

[0126] The server processes the payment and, if successful, notifies the device to unlock the premium features.

[0127] Step 7:

[0128] The device will confirm the release of the premium features and notify the user.

[0129] Example 1

[0130] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0131] In conventional shopping support systems, users had to manually input product information, which was time-consuming, and it was difficult to receive timely reminders based on purchase history and consumption amounts.In addition, advertising and premium features were not provided sufficiently, which led to issues with the convenience and profitability of the entire system.

[0132] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0133] In this invention, the server includes an input means for a user to scan product information, a processing means for transmitting the input product information to a database and retrieving the corresponding product information, a management means for creating and updating the user's list based on the retrieved product information, a data analysis means for analyzing the user's purchase data and consumption amount and calculating the appropriate replacement timing, a notification means for notifying the user of a reminder based on the calculated replacement timing, a monetization means for displaying advertisements and providing premium features, an authentication means for performing user authentication and displaying a main screen upon successful authentication, a notification display means for sending the displayed reminder as a push notification, and a reading means for using a camera to read barcode information for product scanning. This reduces the user's effort, efficiently lists product information, realizes timely reminder notifications, and further strengthens the monetization means.

[0134] "Input means" refers to a device or method by which a user inputs product information into the system.

[0135] The "processing means" refers to a device or method that has the function of transmitting input product information to a database and searching for and retrieving the corresponding product information.

[0136] The "management means" is a device or method that has the function of creating and updating a user list based on the acquired product information.

[0137] The "data analysis means" is a device or method for analyzing a user's purchasing data and consumption amount, and calculating the appropriate time to replace the product.

[0138] The "notification means" is a device or method for notifying the user of a reminder based on the calculated replacement timing.

[0139] A "monetization means" is a device or method that has the function of displaying advertisements and providing premium features.

[0140] "Authentication means" refers to a device or method for authenticating a user and displaying the main screen when authentication is successful.

[0141] A "notification display means" is a device or method for sending a displayed reminder as a push notification.

[0142] "Reading means" refers to a device or method that has the function of reading barcode information using a camera for scanning products.

[0143] This invention is a system that supports efficient shopping by allowing users to scan product information, create a list, and receive reminders at appropriate times to prevent forgetting to buy something. The following components and specific operations are included in the embodiment of the invention.

[0144] System Configuration

[0145] The system allows users to input product information through a smartphone application, and the server processes the information to create shopping lists, send reminder notifications, and generate revenue. The details of each component are shown below.

[0146] Hardware and Software

[0147] Smartphone: A device used by a user that runs applications and uses a camera to scan barcodes.

[0148] Server: Accesses the database and processes product information for management, analysis, notification, and monetization.

[0149] Database: Stores product information and user purchasing data.

[0150] Main features

[0151] 1. Input method:

[0152] The user scans the product information using the barcode scanner function of their smartphone.

[0153] The scanned barcode information is sent to the server via the terminal.

[0154] 2. Processing Method:

[0155] The server searches the database based on the received barcode information and obtains the corresponding product information.

[0156] The acquired product information is returned to the terminal and displayed.

[0157] 3. Control measures:

[0158] A list is created and updated on the user's smartphone based on the returned product information.

[0159] 4. Data analysis methods:

[0160] The server analyzes the user's purchasing data and consumption data, and uses an AI model to calculate the optimal time to replace the device.

[0161] 5. Means of notification:

[0162] Based on the calculated replacement time, the server generates a reminder and sends it to the device at the appropriate time.

[0163] The device displays the reminder to the user as a push notification.

[0164] 6. Monetization methods:

[0165] The server manages in-app advertisements and sets them to be displayed on the device at the appropriate time.

[0166] If the user selects the premium feature, the server processes the payment and unlocks the premium feature.

[0167] 7. Authentication Methods:

[0168] When a user logs in to the application, the server authenticates the user and displays the main screen if the authentication is successful.

[0169] 8. Notification display means:

[0170] The device displays the reminder sent from the server as a push notification.

[0171] 9. Reading Method:

[0172] Product information is read using the smartphone camera and barcode information is obtained.

[0173] Specific examples

[0174] For example, if a user wants to add milk to the list, the system works as follows:

[0175] A user scans the barcode on the milk with their smartphone (User).

[0176] The terminal sends the barcode information to the server (terminal).

[0177] The server retrieves the relevant milk information from the database and returns it to the terminal (server).

[0178] The terminal displays the acquired product information, and the user adds it to the list (terminal).

[0179] Furthermore, if a user regularly purchases milk, the server analyzes the purchase data and predicts when the next replacement purchase is due, then sends a reminder to the user's smartphone as a push notification at the appropriate time.

[0180] Prompt Sentence Examples

[0181] Here are some examples of prompts that specifically use generative AI models:

[0182] "What are the steps to add milk to the list using the system?"

[0183] "Please explain how you can create reminders for items you regularly purchase."

[0184] "How do I upgrade to premium features?"

[0185] In this way, the system of the present invention can support users in shopping efficiently and stress-free.

[0186] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0187] Step 1: Launch the app and log in

[0188] The user launches the smartphone app and the login screen is displayed (input: smartphone app, output: login screen).

[0189] The user enters login information (username and password) and presses the "Login" button (input: username and password, output: login information).

[0190] The terminal sends the entered login information to the server (input: login information, output: request data).

[0191] The server checks the received authentication information, and if it is correct, returns a successful authentication to the terminal (input: request data, output: authentication result).

[0192] The terminal receives the authentication success message and displays the main screen (input: authentication result, output: main screen).

[0193] Step 2: Access the list creation screen

[0194] The user selects "Create a list" from the menu within the app (input: menu selection, output: request to the list creation screen).

[0195] The terminal displays the "Create List" section (Input: Request to create list screen, Output: Create List screen).

[0196] Step 3: Scan the product

[0197] The user presses the "Add product" button to activate the barcode scan function (input: button operation, output: barcode scan screen).

[0198] The device will start the camera and display a view for barcode scanning (input: barcode scanning screen, output: camera view).

[0199] The user scans the product's barcode with a camera (input: barcode, output: barcode data).

[0200] The terminal reads the barcode information and sends it to the server (input: barcode data, output: request to server).

[0201] Step 4: Get product information

[0202] The server searches the database for the corresponding product information based on the received barcode information (input: barcode information, output: product information).

[0203] The server returns the searched product information to the terminal (input: product information, output: response data to the request).

[0204] The terminal displays the product information and updates the user's list (input: response data, output: updated list).

[0205] Step 5: How the Reminders feature works

[0206] The server collects user purchasing data and consumption data for each product (input: purchasing data, output: database dataset).

[0207] The server uses an AI algorithm to analyze the data and predict the optimal replacement time for each product (input: dataset, output: predicted replacement time).

[0208] The server creates a reminder based on the analysis results (input: predicted replacement time, output: reminder).

[0209] The server sends the reminder to the device at the appropriate time (input: reminder, output: request data).

[0210] The device displays the reminder to the user as a push notification (Input: request data, Output: push notification).

[0211] Step 6: Monetization methods in action

[0212] The server manages in-app ad display settings (input: ad data, output: ad settings).

[0213] The device displays ads at appropriate times while the user is using the app (input: ad settings, output: ad display).

[0214] The user reviews the details of the premium features and selects a subscription (Input: Subscription selection, Output: Selection data).

[0215] The terminal sends the user's payment information to the server (input: selection data and payment information, output: payment request).

[0216] The server processes the payment and, if successful, notifies the terminal to unlock the premium features (input: payment request, output: payment result).

[0217] The terminal notifies the user that the premium feature has been enabled (input: payment result, output: notification).

[0218] In this way, the system as a whole allows users to efficiently manage their shopping.

[0219] (Application example 1)

[0220] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0221] Conventional shopping support systems suffer from the cumbersome process of obtaining product information and creating lists, as well as the problem of missing the right timing to purchase. This makes it difficult for users to avoid forgetting to buy something or making unnecessary purchases. Furthermore, they do not adequately provide reminder functions that take individual purchasing patterns into account. Furthermore, they do not adequately provide efficient advertising displays or premium features, which prevents users from increasing satisfaction. To address these issues, a system that innovates and streamlines the user shopping experience is needed.

[0222] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0223] In this invention, the server includes an input means for users to scan product information, a processing means for sending the input product information to a database and retrieving corresponding product information, a management means for creating and updating the user's list based on the retrieved product information, an analysis means for analyzing the user's purchase data and consumption amount and calculating the optimal replacement timing, a notification means for notifying the user of a reminder based on the calculated replacement timing, a monetization means for displaying advertisements and providing premium features, a means for using an application installed on a smartphone, smart glasses, a head-mounted display, or a robot, a means for scanning barcode information with a camera and retrieving product information from the database, a means for accumulating past consumption data and using an AI algorithm to analyze consumption patterns, a means for generating prompt sentences for reminder notifications using a generative AI model, and a means for using the server, Flask, and scikit-learn libraries to create lists and send reminders. This allows users to easily retrieve product information and efficiently create and update lists. Furthermore, by predicting the optimal purchase timing and providing reminders, users can prevent forgetting to buy items and reduce wasteful purchases. Additionally, personalized notifications based on individual purchasing patterns can increase user satisfaction.

[0224] "Input means" refers to a device or interface that provides a function for a user to scan product information.

[0225] The "processing means" is a component that sends input product information to a database and acquires the corresponding product information.

[0226] "Management means" refers to a part of the system that has the function of creating and updating a user's list based on the acquired product information.

[0227] "Analysis means" refers to algorithms or software that analyzes user purchasing data and consumption amounts and calculates the optimal time to replace items.

[0228] The "notification means" is a mechanism for notifying the user of a reminder based on the calculated replacement timing.

[0229] "Monetization Method" means a system or method for displaying advertisements and providing premium features.

[0230] "Means for using an application" refers to the functions and interfaces for using an application installed on a smartphone, smart glasses, head-mounted display, or robot.

[0231] "Means for scanning with a camera" is a function for scanning barcode information with a camera and obtaining product information from a database.

[0232] "Means using AI algorithms" refers to methods that use artificial intelligence algorithms to accumulate past consumption data and analyze consumption patterns.

[0233] "Means for utilizing a generative AI model" refers to a function that uses a generative AI model to generate prompt text for reminder notifications.

[0234] "Methods using a server, Flask, and scikit-learn library" refers to a method of processing using a cloud server and specific program libraries to create lists and send reminders.

[0235] This invention is a system that supports users in shopping efficiently in physical stores. The operation of the system and a specific implementation method are described below.

[0236] System Overview

[0237] The system consists of the following main components:

[0238] Input method: A device such as a smartphone or smart glasses that allows the user to scan product information.

[0239] Processing method: A function that sends scanned barcode information to a database and retrieves the corresponding product information.

[0240] Management method: A system that creates and updates user lists based on acquired product information.

[0241] Analysis method: An artificial intelligence (AI) algorithm that analyzes user purchasing data and consumption to calculate the optimal time to replace items.

[0242] Notifications: A feature that notifies users with reminders based on the calculated replacement period.

[0243] Monetization methods: A system for displaying advertisements and providing premium features.

[0244] Means of using the application: The application is installed on a smartphone, smart glasses, head-mounted display or robot.

[0245] Scan with camera: A function that scans the barcode information of products with a camera and retrieves product information from the database.

[0246] Using AI algorithms: Ability to accumulate past consumption data and use AI algorithms to analyze consumption patterns.

[0247] How to use generative AI models: The ability to use generative AI models to generate prompts for reminder notifications.

[0248] Using a server, Flask, or scikit-learn library: This method uses a cloud server and specific program libraries to create lists and send reminders.

[0249] Program processing

[0250] The server supports an input method that allows users to scan product information. The product information is scanned as barcode information using a smartphone camera and sent to the server. The server then queries a database for the received barcode information and retrieves the corresponding product information. The server then returns the retrieved product information to the terminal, where the user can check the displayed product information to update the list.

[0251] Past consumption data is stored in a cloud environment, and the server analyzes the data using an AI algorithm (e.g., using the scikit-learn library). The server analyzes purchasing patterns and consumption amounts to predict the next purchase date. Based on the calculated replacement date, a generative AI model is used to generate a prompt message.

[0252] The device will then send a reminder to the user based on the generated prompt text, which will be displayed as a push notification to prevent the user from forgetting to make their next purchase.

[0253] The application features advertising and offers the option to disable advertising as a premium feature, which allows users to enjoy more personalized notifications and additional customization features.

[0254] Specific examples

[0255] For example, when a user scans a barcode on a bottle of milk with their smartphone camera, the server receives the barcode information and retrieves the milk's details from the database. The information is then displayed on the device, allowing the user to add the milk to their shopping list. Based on past data, an AI algorithm predicts when the next purchase will be made, and a reminder notification is sent to the user at the appropriate time.

[0256] Prompt Sentence Examples

[0257] User: I scanned the barcode for milk. Add it to the list.

[0258] System: Milk has been added to your list. We'll let you know when it's time for your next purchase.

[0259] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0260] Step 1:

[0261] The user scans the product information.

[0262] Input: Scan the product barcode with your smartphone camera.

[0263] Output: The barcode information read.

[0264] Specific operation: The user launches the smartphone app and uses the barcode scanner function to scan the product barcode with the camera.

[0265] Step 2:

[0266] The terminal transmits the barcode information to the server.

[0267] Input: The barcode information read.

[0268] Output: Barcode information sent to the server.

[0269] Specific operation: The terminal sends the acquired barcode information to the server using an HTTP request, etc.

[0270] Step 3:

[0271] The server retrieves product information from the database.

[0272] Input: The barcode information sent to the server.

[0273] Output: Product information retrieved from the database.

[0274] Specific operation: The server queries the database based on the received barcode information and retrieves related product information.

[0275] Step 4:

[0276] The server returns the acquired product information to the terminal.

[0277] Input: Product information retrieved from the database.

[0278] Output: Product information sent back to the device.

[0279] Specific operation: The server sends the acquired product information to the terminal using an HTTP response, etc.

[0280] Step 5:

[0281] The device displays product information and updates the list.

[0282] Input: Product information received from the server.

[0283] Output: Display of the updated shopping list.

[0284] Specific operation: The terminal displays the received product information on the user interface, and the user confirms and adds the product information to the list.

[0285] Step 6:

[0286] The server analyzes the user's purchasing data.

[0287] Input: User's past purchase and consumption data.

[0288] Output: Prediction of optimal replacement time.

[0289] Specific operation: The server uses an AI algorithm (e.g., the scikit-learn library) to analyze past purchase data and consumption data and calculate the timing of the next purchase.

[0290] Step 7:

[0291] The server generates the reminder prompt.

[0292] Input: Optimal replacement time prediction results.

[0293] Output: The reminder prompt.

[0294] Specific behavior: The server uses the generative AI model to generate prompt text for reminder notifications based on the prediction results.

[0295] Step 8:

[0296] The server sends the reminder to the device.

[0297] Input: The generated prompt statement.

[0298] Output: The reminder notification sent to the device.

[0299] Specific operation: The server sends the generated prompt text to the device in the form of a push notification.

[0300] Step 9:

[0301] The device notifies the user of the reminder.

[0302] Input: Reminder notification sent from the server.

[0303] Output: The reminder notification that is displayed to the user.

[0304] Specific behavior: The device uses the push notification function to display a reminder to the user.

[0305] Example prompt sentence:

[0306] User: I scanned the barcode for milk. Add it to the list.

[0307] System: Milk has been added to your list. We'll let you know when it's time for your next purchase.

[0308] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0309] This invention is a system that allows users to scan product information to create a list, receive reminders at the optimal time, and recognize the user's emotions and display appropriate notifications and advertisements based on those emotions, thereby preventing users from forgetting to buy things and providing efficient shopping. The components of this system and their operation are described in detail below.

[0310] List creation function

[0311] 1. The user launches the app and logs in

[0312] The user launches the smartphone app and logs in.

[0313] The device checks the user's authentication information and displays the main screen if the login is successful.

[0314] 2. Access the list creation screen

[0315] The user accesses the "Create a List" section from the app's menu.

[0316] The device displays the list creation screen.

[0317] 3. Enter product information

[0318] The user scans the item using the barcode scanning feature.

[0319] The terminal reads the barcode and sends the information to the server.

[0320] 4. Obtain product information

[0321] The server searches the database for the corresponding product information based on the received barcode information.

[0322] The server returns the product information to the terminal.

[0323] The terminal displays the product information and updates the user's list.

[0324] For example, if a user wants to add milk to the list, they scan the barcode of the milk. The server retrieves the product data from the barcode information and displays it on the terminal. The user confirms and adds it to the list.

[0325] Reminder function

[0326] 1. Data collection and analysis

[0327] The server periodically collects user purchase data and consumption data for each product.

[0328] The server uses AI algorithms to analyze the data and predict the optimal time to replace each product.

[0329] 2. Create and send reminders

[0330] The server creates reminders based on the analysis results.

[0331] The server sends the created reminder to the device at the appropriate time.

[0332] The device displays the reminder to the user as a push notification.

[0333] For example, if a user regularly buys milk, the server will analyze their past consumption patterns and predict when they will need to buy it again, and a reminder will be sent to the user's device at that time.

[0334] Emotion recognition function

[0335] 1. Emotional awareness

[0336] The device uses a camera and microphone to collect emotional information from the user's facial expressions and voice.

[0337] The device uses an emotion engine to analyze the collected information and recognize the user's emotions.

[0338] 2. Emotion-Based Adjustment

[0339] The server uses data from the emotion engine to adjust the timing and content of reminders based on the user's emotions.

[0340] The server also optimizes the content and timing of advertisements.

[0341] For example, if a user is feeling stressed, the server will suppress reminder notifications or change them to more stressful phrases. Advertisements will also be prioritized to display items related to relaxation and activities.

[0342] Monetization methods

[0343] 1. Advertising

[0344] The server manages in-app ad display settings.

[0345] The device displays ads at appropriate times while the user is using the app.

[0346] 2. Premium Features

[0347] The user reviews the premium features and selects a subscription service.

[0348] The terminal transmits the user's payment information to the server.

[0349] The server processes the payment and, if successful, notifies the device to unlock the premium features.

[0350] For example, if a user wants to purchase a premium feature to remove ads, they select the subscription and make the payment. The server verifies the payment and the premium feature is activated.

[0351] As described above, by incorporating an emotion engine, it is possible to provide flexible reminders and advertisement displays that respond to the user's emotions, providing a more personalized shopping experience.

[0352] The processing flow will be explained below.

[0353] List creation function

[0354] Step 1:

[0355] The user launches the smartphone app and logs in.

[0356] Step 2:

[0357] The device checks the user's authentication information and displays the main screen if the login is successful.

[0358] Step 3:

[0359] The user accesses the "Create a List" section from the app's menu.

[0360] Step 4:

[0361] The device displays the list creation screen.

[0362] Step 5:

[0363] The user scans the item using the barcode scanning feature.

[0364] Step 6:

[0365] The device activates the smartphone camera and reads the barcode.

[0366] Step 7:

[0367] The terminal transmits the barcode information to the server.

[0368] Step 8:

[0369] The server searches the database for and retrieves the relevant product information.

[0370] Step 9:

[0371] The server returns product information (product name, category, price) to the terminal.

[0372] Step 10:

[0373] The terminal adds the acquired product information to the user's list and updates the screen.

[0374] Step 11:

[0375] The user confirms the product addition, manually enters the product name if necessary, and adds the item to the list.

[0376] Step 12:

[0377] The user finishes updating the list and taps the save button.

[0378] Step 13:

[0379] The terminal transmits the list information to the server.

[0380] Step 14:

[0381] The server saves the list in a database and sends a message to the terminal indicating that the list has been saved.

[0382] Step 15:

[0383] The device displays a notification to the user that the save is complete.

[0384] Reminder function

[0385] Step 1:

[0386] The server periodically collects user purchase data and consumption data for each product.

[0387] Step 2:

[0388] The server uses AI algorithms to analyze the data and predict the optimal time to replace each product.

[0389] Step 3:

[0390] The server creates reminders based on the analysis results.

[0391] Step 4:

[0392] The server sends the created reminder to the device at the appropriate time.

[0393] Step 5:

[0394] The device displays the reminder to the user as a push notification.

[0395] Step 6:

[0396] The user checks the reminder and checks the shopping list.

[0397] Emotion recognition function

[0398] Step 1:

[0399] The device uses a camera and microphone to collect emotional information from the user's facial expressions and voice.

[0400] Step 2:

[0401] The device uses an emotion engine to analyze the collected information and recognize the user's emotions.

[0402] Step 3:

[0403] The server uses data from the emotion engine to adjust the timing and content of reminders based on the user's emotions.

[0404] Step 4:

[0405] The server also optimizes the content and timing of advertisements.

[0406] Step 5:

[0407] The device displays tailored reminders and advertisements to the user.

[0408] For example, if a user is feeling stressed, the server will suppress reminder notifications or change them to more "menos estresante" language. It also prioritizes advertisements related to relaxation items and activities.

[0409] Monetization methods

[0410] Step 1:

[0411] The server manages in-app ad display settings.

[0412] Step 2:

[0413] The device displays ads at appropriate times while the user is using the app.

[0414] Step 3:

[0415] The user reviews the premium features and selects a subscription service.

[0416] Step 4:

[0417] The user enters payment information to purchase a subscription and taps the "Purchase" button.

[0418] Step 5:

[0419] The terminal sends the payment information to the server.

[0420] Step 6:

[0421] The server processes the payment and, if successful, notifies the device to unlock the premium features.

[0422] Step 7:

[0423] The device will confirm the release of the premium features and notify the user.

[0424] Example 2

[0425] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0426] In today's highly information-driven society, users need tools that allow them to shop efficiently and without missing anything. However, conventional systems are limited to managing product information and reminder functions, and are unable to respond flexibly to user emotions. As a result, users may feel more stressed, and reminders and advertisements may not be as effective as they should be.

[0427] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0428] In this invention, the server includes input means for a user to scan product information, processing means for transmitting the input product information to a database and retrieving corresponding product information, management means for creating and updating the user's list based on the retrieved product information, analysis means for analyzing the user's purchase data and consumption amount and calculating the optimal replacement timing, notification means for notifying the user of a reminder based on the calculated replacement timing, emotion recognition means for recognizing emotions from the user's facial expressions and voice, adaptation means for adjusting the content of reminders and advertisements based on the recognized emotions, and monetization means for displaying advertisements and providing premium features, thereby enabling a flexible and stress-free shopping experience that responds to the user's emotions.

[0429] "Input means" refers to an interface that allows a user to input product information into the system, and specifically includes a barcode scanner, a camera, a touch screen, etc.

[0430] "Processing means" refers to a computer system for receiving input product information and transmitting it to a database to retrieve corresponding product information.

[0431] "Management means" refers to tools and software for creating and updating user lists based on acquired product information.

[0432] "Analysis method" refers to a system that collects user purchasing data and consumption data and uses AI technology to calculate the optimal time to replace items.

[0433] The "notification means" refers to a method for notifying the user of a reminder based on the calculated replacement timing, and mainly includes push notification and email.

[0434] "Emotion recognition means" refers to technology or systems that analyze a user's facial expressions, voice, etc. to recognize the user's emotions.

[0435] "Adaptation" refers to algorithms or systems that tailor reminder or ad content based on perceived emotions.

[0436] "Monetization Method" refers to a method or system for generating revenue through displaying advertisements and providing premium features.

[0437] This invention is a system that allows users to scan product information to create a list, receive reminders at optimal times, and recognize the user's emotions and display optimal notifications and advertisements based on those emotions, thereby preventing users from forgetting to buy items and providing efficient shopping. Specific embodiments of this system are described below.

[0438] First, the user launches the smartphone app and logs in. The device displays the login screen, and the user logs in by entering their email address and password. The device sends the entered authentication information to the server, which then compares it with a database for authentication. If authentication is successful, the server sends a command to the device to display the main screen, and the device displays the main screen.

[0439] Next, the user accesses the "Create a List" section from the app's menu. The device displays the list creation screen, and the user taps the "Add New Item" button. The device then activates its camera and displays the message "Please scan the barcode." When the user holds the product's barcode up to the camera to scan it, the device reads the barcode information and sends it to the server.

[0440] Based on the received barcode information, the server searches a database (for example, a MySQL (registered trademark) database) for the corresponding product information. The server returns the searched product information (product name, price, manufacturer, etc.) to the terminal, which then displays the product information on the screen and asks the user for confirmation. Once the user confirms, they can tap the "Add to list" button to add the product to the list.

[0441] After the list is created, the server periodically collects the user's purchase data and consumption data for each product. The collected data is analyzed using Python's Pandas and Scikit-learn to train an AI model to predict the optimal replacement timing for each product. Reminders are created based on the prediction results, and the server sends the reminders to the device at the appropriate time. The device then displays the reminders to the user as push notifications.

[0442] The device also uses a camera and microphone to collect the user's facial expressions and voice. The collected data is analyzed using Google® Cloud Vision API or Microsoft® Azure® Cognitive Services to recognize the user's emotions. The server receives the emotion data and understands the user's emotional state.

[0443] Based on the emotion data, the server can adjust the timing and content of reminders. For example, if the user is feeling stressed, the server might tone down reminders or change them to less stressful phrases. Similarly, the server might prioritize ads for products and services related to relaxation.

[0444] Finally, as a means of monetization, the server manages in-app advertising using Google AdMob or Facebook Audience Network. The device displays ads at appropriate times while the user is using the app. Furthermore, the user checks detailed information in the "Premium Features" section of the app, selects a subscription service, and enters payment information. The device sends the payment information to the server, which processes the payment using the Stripe API. If the payment is successful, the server sends a command to unlock the premium features to the device, and the device activates the premium features.

[0445] For example, if a user wants to add milk to a list, they scan the milk's barcode. The server retrieves the product data from the barcode and displays it on the device. The user confirms and adds it to the list. If the user regularly purchases milk, the server analyzes past consumption patterns and predicts when the next purchase will be necessary. A reminder will be sent to the user's device at that time.

[0446] This allows users to enjoy an efficient and personalized shopping experience.

[0447] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0448] List creation function

[0449] Step 1:

[0450] A user launches a smartphone app and accesses the login screen. As input, the user enters an email address and password. Based on this, the device sends the entered authentication information to the server. The server compares it with a database and performs authentication. As output, if authentication is successful, the server sends a command to display the main screen to the device, and the device displays the main screen.

[0451] Step 2:

[0452] The user accesses the "Create a List" section from the app's menu. As input, the user taps the menu icon. Based on this, the device displays the list creation screen.

[0453] Step 3:

[0454] The user taps the "Add New Product" button to activate the camera function. As input, the user holds the product's barcode up to the camera to scan it. Based on this, the device reads the barcode information and sends it to the server.

[0455] Step 4:

[0456] Based on the barcode information received by the server, the server searches for the corresponding product information in a database. A database query is made, receiving the barcode information as input. Based on this, the server retrieves product information (product name, price, manufacturer, etc.) and returns it to the terminal. As output, the server returns the product information to the terminal, which then displays the product information on the screen.

[0457] Step 5:

[0458] The user checks the product information and taps the "Add to List" button to add the product to the list. As input, there is the user's action of tapping the "Add to List" button. Based on this, the device adds the product to the user's list and updates the list. As output, the list is updated.

[0459] Reminder function

[0460] Step 1:

[0461] The server periodically collects user purchase data and consumption data for each product. The inputs are purchase history and consumption data. Based on this, the server uses Python's Pandas library to format the data and create a dataset for the AI ​​model. The formatted dataset is generated as the output.

[0462] Step 2:

[0463] The server uses Scikit-learn to train an AI model to predict the optimal replacement time for each product. The input is a formatted dataset. Based on this, the AI ​​algorithm analyzes the data and generates a prediction. The output is a prediction of the optimal replacement time.

[0464] Step 3:

[0465] The server creates a reminder to notify the next purchase timing based on the prediction results. The input is the predicted optimal replacement time. Based on this, the server generates the reminder text and designs the appropriate timing. The output is a reminder notification.

[0466] Step 4:

[0467] The device displays the reminder to the user as a push notification. As input, there is a reminder notification sent from the server. Based on this, the device displays a push notification to the user. As output, the user receives the reminder.

[0468] Emotion recognition function

[0469] Step 1:

[0470] The device uses a camera and microphone to collect the user's facial expressions and voice. The input is the user's facial expression and voice data. Based on this, the device sends the data to Google Cloud Vision API or Microsoft Azure Cognitive Services for analysis. The output is emotion data.

[0471] Step 2:

[0472] The server receives the emotional data and understands the user's emotional state. The input is the analyzed emotional data. Based on this, the server evaluates the user's emotional state and generates emotional state information. The output is the emotional state information.

[0473] Step 3:

[0474] The server adjusts the timing and content of reminders based on emotional data. The input is emotional state information. Based on this, the server generates and adjusts the content of reminders and advertisements appropriately. The output is the adjusted reminders and advertisements.

[0475] Monetization methods

[0476] Step 1:

[0477] The server manages the ad network settings and sets the timing and content of ad display. The input is the ad network configuration information. Based on this, the server manages ads using Google AdMob or Facebook Audience Network. The output is the ad display settings.

[0478] Step 2:

[0479] The device displays ads at appropriate times while the user is using the app. As input, there are ad display settings received from the server. Based on this, the device displays ads while the user is using the app. As output, the ads are displayed to the user.

[0480] Step 3:

[0481] The user checks the details in the "Premium Features" section and selects a subscription service. As an input, there is an action where the user selects a subscription service. Based on this, the terminal sends payment information to the server. As an output, payment information is sent to the server.

[0482] Step 4:

[0483] The server receives the payment information and processes the payment. As input, it has the user's payment information. Based on this, the server processes the payment using Stripe API and receives the result. As output, it gets the payment success or failure status.

[0484] Step 5:

[0485] If the payment is successful, the server sends a command to unlock the premium feature to the terminal. As input, there is a payment success status. Based on this, the server generates a command to unlock the premium feature and sends it to the terminal. As output, the premium feature becomes active.

[0486] As a result, this system can provide a flexible and personalized shopping experience that responds to the user's emotions.

[0487] (Application example 2)

[0488] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0489] Conventional shopping support systems often have problems such as users forgetting to buy items or not being able to purchase items at the optimal time. Furthermore, because the user experience is not personalized through standard notifications and advertisements, they are unable to provide an efficient and comfortable shopping experience. Furthermore, because they do not take into account the user's emotional state, notifications and advertisements are displayed at inappropriate times, which can cause stress to the user.

[0490] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes an input means for the user to scan product information, a processing means for transmitting the input product information to a database and acquiring corresponding product information, and a management means for creating and updating the user's list based on the acquired product information. This system also includes an analysis means for analyzing the user's purchase data and consumption amount to calculate the optimal replacement timing, a notification means for notifying the user of a reminder based on the calculated replacement timing, an emotion recognition means for recognizing the user's emotions and displaying notifications and advertisements based on the emotions, and a monetization means for displaying advertisements and providing premium features. This not only prevents the user from forgetting to buy something and enables them to purchase products at the optimal time, but also provides personalized notifications and advertisements based on the user's emotions.

[0491] "Input means" is a function that allows the user to scan and input product information.

[0492] The "processing means" is a function for transmitting input product information to a database and retrieving the corresponding product information.

[0493] The "management means" is a function for creating and updating a user list based on the acquired product information.

[0494] The "analysis means" is a function for analyzing the user's purchasing data and consumption amount, and calculating the optimal replacement time.

[0495] The "notification means" is a function for notifying the user of a reminder based on the calculated replacement timing.

[0496] The "emotion recognition means" is a function for recognizing the user's emotions and displaying notifications and advertisements based on those emotions.

[0497] "Monetization means" refers to functions for displaying advertisements and providing premium features.

[0498] This invention is a system that allows users to scan product information and create and update lists. It also recognizes the user's emotions and displays reminders, notifications, and advertisements at optimal times based on those emotions, preventing users from forgetting to buy something and providing an efficient shopping experience.

[0499] System configuration

[0500] The system mainly consists of the following components:

[0501] 1. Input Method

[0502] This is a means for users to scan products using a barcode scanner application installed on a device such as a smartphone.

[0503] 2. Processing Methods

[0504] The entered product information is sent to the database, and the corresponding product information is retrieved. At this time, the server searches the database for the product information and returns it to the user's terminal.

[0505] 3. Control measures

[0506] This function creates and updates the user's list based on the acquired product information. Products scanned by the user are automatically added to the list.

[0507] 4. Analysis tools

[0508] The AI ​​algorithm analyzes user purchasing data and consumption data to calculate the optimal time to replace the product. Here, the AI ​​analysis model is implemented using programming languages ​​such as Python.

[0509] 5. Means of notification

[0510] Based on the calculated replacement timing, a reminder is created and the user is notified at the appropriate time via the smartphone's push notification function.

[0511] 6. Emotion recognition means

[0512] It recognizes users' emotions and adjusts notifications and ads accordingly. Emotion recognition uses the smartphone's camera and microphone to collect emotional data from the user's facial expressions and voice, which is then analyzed by an AI model that includes a pre-trained emotion recognition algorithm.

[0513] 7. Monetization Methods

[0514] It displays advertisements and provides premium features. Users can choose to subscribe to the service to receive premium features such as the ability to hide advertisements.

[0515] Explanation of program processing

[0516] The system uses the following hardware and software:

[0517] Smartphone: Barcode scanning, emotion recognition (camera / microphone), push notification display.

[0518] Server: Database management, data analysis using AI algorithms, and reminder generation.

[0519] AI model: A trained model for recognizing user emotions (e.g., an emotion recognition model using Keras or TENSORFLOW (registered trademark)).

[0520] Database: Stores product information and user purchase history data.

[0521] The specific process is as follows:

[0522] 1. The smartphone application scans the product barcode and sends it to the server.

[0523] 2. The server retrieves product information from the database and sends it back to the smartphone.

[0524] 3. Product information will be added and updated to the list.

[0525] 4. An AI algorithm analyzes purchasing history and calculates the optimal time to replace the product.

[0526] 5. Reminders are sent as push notifications at the right time.

[0527] 6. The smartphone's camera and microphone capture the user's facial expressions and voice, which are then analyzed by an emotion recognition model.

[0528] 7. Reminders and ads are tailored based on user emotions.

[0529] Specific examples

[0530] For example, if a user scans a bottle of milk at the supermarket, a product information API is called and detailed information about the product is displayed on the smartphone. Also, if the user is feeling stressed, a push notification will be sent discreetly or a product suggestion will be made to help them relax.

[0531] Prompt Sentence Examples

[0532] Design an application that allows users to scan items in a physical store using a smartphone app and automatically create a shopping list. Use emotion recognition technology to display notifications and ads based on the user's emotions and send reminders at optimal times based on the user's purchasing history. Specific features should include real-time item scanning, emotion recognition, list creation, reminders, and personalized ads.

[0533] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0534] Step 1:

[0535] The user scans the item.

[0536] Input: Scan the product barcode with your smartphone's barcode scanner.

[0537] How it works: A user holds an item up to their smartphone camera to scan the barcode.

[0538] Output: The barcode information is entered into the terminal.

[0539] Step 2:

[0540] The terminal transmits the barcode information to the server.

[0541] Input: Barcode information.

[0542] Operation: The terminal generates a request to send the barcode information to the database API.

[0543] Output: The barcode information sent to the server.

[0544] Step 3:

[0545] The server searches the database for the relevant product information.

[0546] Input: The barcode information sent.

[0547] Operation: The server generates a database query to retrieve product information corresponding to the barcode.

[0548] Output: Product information.

[0549] Step 4:

[0550] The server returns the product information to the terminal.

[0551] Input: Product information retrieved from the database.

[0552] Operation: The server generates a response to return product information to the terminal.

[0553] Output: Product information sent to the device.

[0554] Step 5:

[0555] The device displays product information and updates the list.

[0556] Input: Product information received from the server.

[0557] Action: The device displays product information to the user and adds the product to an existing shopping list.

[0558] Output: The updated shopping list.

[0559] Step 6:

[0560] The server analyzes the purchase data and consumption amounts.

[0561] Input: User purchase history data and consumption data.

[0562] How it works: The server uses AI algorithms to analyze the data and predict the best time to make your next purchase.

[0563] Output: Best time to replace.

[0564] Step 7:

[0565] The server generates reminders and sends them to the device at the appropriate time.

[0566] Input: Optimal replacement time.

[0567] How it works: The server generates a reminder and sends it to the device as a push notification.

[0568] Output: The reminder displayed on the user's device.

[0569] Step 8:

[0570] The device recognizes the user's emotions.

[0571] Input: User facial and voice data collected by the smartphone camera and microphone.

[0572] How it works: The device inputs the collected data into an emotion recognition model to analyze the user's emotions.

[0573] Output: The user's emotional state.

[0574] Step 9:

[0575] The server tailors reminders and advertisements based on the emotional state.

[0576] Input: User emotional state data.

[0577] How it works: The server adjusts the content and timing of notifications based on the user's emotional state, generating personalized ads.

[0578] Output: Tailored reminders and ads.

[0579] Step 10:

[0580] Your device will display tailored reminders and ads.

[0581] Enter: tailored reminders and ads.

[0582] How it works: The device displays reminders and advertisements to the user at appropriate times.

[0583] Output: Personalized reminders and ads shown to the user.

[0584] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0585] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0586] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0587] [Second embodiment]

[0588] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0589] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0590] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0591] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0592] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0593] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0594] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0595] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0596] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0597] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0598] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0599] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0600] This invention is a system that helps users avoid forgetting to buy things and supports efficient shopping by allowing them to create a list by scanning product information and receiving reminders at the appropriate time.The operation of each part of the system is described in detail below.

[0601] List creation function

[0602] 1. The user launches the app and logs in

[0603] The user launches the smartphone app and logs in.

[0604] The terminal checks the user's authentication information and displays the main screen if the login is successful.

[0605] 2. Access the list creation screen

[0606] Users access the "Create a List" section from the app's menu.

[0607] The device displays a list creation screen.

[0608] 3. Enter product information

[0609] The user scans the product using the barcode scanning function.

[0610] The terminal reads the barcode and transmits the information to the server.

[0611] 4. Obtain product information

[0612] The server searches the database for the corresponding product information based on the received barcode information.

[0613] The server returns the product information to the terminal.

[0614] The terminal displays the product information and updates the user's list.

[0615] For example, if a user wants to add milk to the list, they scan the barcode of the milk. The server retrieves the product data from the barcode information and displays it on the terminal. The user confirms and adds it to the list.

[0616] Reminder function

[0617] 1. Data collection and analysis

[0618] The server collects user purchase data and consumption data for each product.

[0619] The server uses AI algorithms to analyze the data and predict the optimal time to replace each product.

[0620] 2. Create and send reminders

[0621] The server creates reminders based on the analysis results.

[0622] The server sends the reminder to the device at the appropriate time.

[0623] The device displays the reminder to the user as a push notification.

[0624] For example, if a user regularly buys milk, the server will analyze their past consumption patterns and predict when they will need to buy it again, and a reminder will be sent to the user's device at that time.

[0625] Monetization methods

[0626] 1. Advertising

[0627] The server manages in-app advertising display settings.

[0628] The device displays advertisements at appropriate times while the user is using the app.

[0629] 2. Premium Features

[0630] The user reviews the details of the premium features and selects a subscription.

[0631] The terminal transmits the user's payment information to the server.

[0632] The server processes the payment and, if successful, notifies the device to unlock the premium features.

[0633] For example, if a user wants to purchase a premium feature to remove ads, they select the subscription and make the payment. The server verifies the payment and the premium feature is activated.

[0634] As described above, this system helps users to shop efficiently and stress-free, and is realized by the cooperation of each function.

[0635] The processing flow will be explained below.

[0636] List creation function

[0637] Step 1:

[0638] The user launches the smartphone app and logs in.

[0639] Step 2:

[0640] The device checks the user's authentication information and displays the main screen if the login is successful.

[0641] Step 3:

[0642] The user accesses the "Create a List" section from the app's menu.

[0643] Step 4:

[0644] The device displays the list creation screen.

[0645] Step 5:

[0646] The user uses the barcode scanning feature to add everyday items to the list.

[0647] Step 6:

[0648] The device activates the smartphone camera and reads the barcode.

[0649] Step 7:

[0650] The terminal transmits the barcode information to the server.

[0651] Step 8:

[0652] The server searches the database for and retrieves the relevant product information.

[0653] Step 9:

[0654] The server returns product information (product name, category, price) to the terminal.

[0655] Step 10:

[0656] The terminal adds the acquired product information to the user's list and updates the screen.

[0657] Step 11:

[0658] The user confirms the product addition, manually enters the product name if necessary, and adds the item to the list.

[0659] Step 12:

[0660] The user finishes updating the list and taps the save button.

[0661] Step 13:

[0662] The terminal transmits the list information to the server.

[0663] Step 14:

[0664] The server saves the list in a database and sends a message to the terminal indicating that the list has been saved.

[0665] Step 15:

[0666] The device displays a notification to the user that the save is complete.

[0667] Reminder function

[0668] Step 1:

[0669] The server periodically collects user purchase data and consumption data for each product.

[0670] Step 2:

[0671] The server uses AI algorithms to analyze the data and predict the optimal time to replace each product.

[0672] Step 3:

[0673] The server creates reminders based on the analysis results.

[0674] Step 4:

[0675] The server sends the created reminder to the device at the appropriate time.

[0676] Step 5:

[0677] The device displays the reminder to the user as a push notification.

[0678] Step 6:

[0679] The user checks the reminder and checks the shopping list.

[0680] Monetization methods

[0681] Step 1:

[0682] The server manages in-app ad display settings.

[0683] Step 2:

[0684] The device displays ads at appropriate times while the user is using the app.

[0685] Step 3:

[0686] The user reviews the premium features and selects a subscription service.

[0687] Step 4:

[0688] The user enters payment information to purchase a subscription and taps the "Purchase" button.

[0689] Step 5:

[0690] The terminal sends the payment information to the server.

[0691] Step 6:

[0692] The server processes the payment and, if successful, notifies the device to unlock the premium features.

[0693] Step 7:

[0694] The device will confirm the release of the premium features and notify the user.

[0695] Example 1

[0696] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0697] In conventional shopping support systems, users had to manually input product information, which was time-consuming, and it was difficult to receive timely reminders based on purchase history and consumption amounts.In addition, advertising and premium features were not provided sufficiently, which led to issues with the convenience and profitability of the entire system.

[0698] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0699] In this invention, the server includes an input means for a user to scan product information, a processing means for transmitting the input product information to a database and retrieving the corresponding product information, a management means for creating and updating the user's list based on the retrieved product information, a data analysis means for analyzing the user's purchase data and consumption amount and calculating the appropriate replacement timing, a notification means for notifying the user of a reminder based on the calculated replacement timing, a monetization means for displaying advertisements and providing premium features, an authentication means for performing user authentication and displaying a main screen upon successful authentication, a notification display means for sending the displayed reminder as a push notification, and a reading means for using a camera to read barcode information for product scanning. This reduces the user's effort, efficiently lists product information, realizes timely reminder notifications, and further strengthens the monetization means.

[0700] "Input means" refers to a device or method by which a user inputs product information into the system.

[0701] The "processing means" refers to a device or method that has the function of transmitting input product information to a database and searching for and retrieving the corresponding product information.

[0702] The "management means" is a device or method that has the function of creating and updating a user list based on the acquired product information.

[0703] The "data analysis means" is a device or method for analyzing a user's purchasing data and consumption amount, and calculating the appropriate time to replace the product.

[0704] The "notification means" is a device or method for notifying the user of a reminder based on the calculated replacement timing.

[0705] A "monetization means" is a device or method that has the function of displaying advertisements and providing premium features.

[0706] "Authentication means" refers to a device or method for authenticating a user and displaying the main screen when authentication is successful.

[0707] A "notification display means" is a device or method for sending a displayed reminder as a push notification.

[0708] "Reading means" refers to a device or method that has the function of reading barcode information using a camera for scanning products.

[0709] This invention is a system that supports efficient shopping by allowing users to scan product information, create a list, and receive reminders at appropriate times to prevent forgetting to buy something. The following components and specific operations are included in the embodiment of the invention.

[0710] System Configuration

[0711] The system allows users to input product information through a smartphone application, and the server processes the information to create shopping lists, send reminder notifications, and generate revenue. The details of each component are shown below.

[0712] Hardware and Software

[0713] Smartphone: A device used by a user that runs applications and uses a camera to scan barcodes.

[0714] Server: Accesses the database and processes product information for management, analysis, notification, and monetization.

[0715] Database: Stores product information and user purchasing data.

[0716] Main features

[0717] 1. Input method:

[0718] The user scans the product information using the barcode scanner function of their smartphone.

[0719] The scanned barcode information is sent to the server via the terminal.

[0720] 2. Processing Method:

[0721] The server searches the database based on the received barcode information and obtains the corresponding product information.

[0722] The acquired product information is returned to the terminal and displayed.

[0723] 3. Control measures:

[0724] A list is created and updated on the user's smartphone based on the returned product information.

[0725] 4. Data analysis methods:

[0726] The server analyzes the user's purchasing data and consumption data, and uses an AI model to calculate the optimal time to replace the device.

[0727] 5. Means of notification:

[0728] Based on the calculated replacement time, the server generates a reminder and sends it to the device at the appropriate time.

[0729] The device displays the reminder to the user as a push notification.

[0730] 6. Monetization methods:

[0731] The server manages in-app advertisements and sets them to be displayed on the device at the appropriate time.

[0732] If the user selects the premium feature, the server processes the payment and unlocks the premium feature.

[0733] 7. Authentication Methods:

[0734] When a user logs in to the application, the server authenticates the user and displays the main screen if the authentication is successful.

[0735] 8. Notification display means:

[0736] The device displays the reminder sent from the server as a push notification.

[0737] 9. Reading Method:

[0738] Product information is read using the smartphone camera and barcode information is obtained.

[0739] Specific examples

[0740] For example, if a user wants to add milk to the list, the system works as follows:

[0741] A user scans the barcode on the milk with their smartphone (User).

[0742] The terminal sends the barcode information to the server (terminal).

[0743] The server retrieves the relevant milk information from the database and returns it to the terminal (server).

[0744] The terminal displays the acquired product information, and the user adds it to the list (terminal).

[0745] Furthermore, if a user regularly purchases milk, the server analyzes the purchase data and predicts when the next replacement purchase is due, then sends a reminder to the user's smartphone as a push notification at the appropriate time.

[0746] Prompt Sentence Examples

[0747] Here are some examples of prompts that specifically use generative AI models:

[0748] "What are the steps to add milk to the list using the system?"

[0749] "Please explain how you can create reminders for items you regularly purchase."

[0750] "How do I upgrade to premium features?"

[0751] In this way, the system of the present invention can support users in shopping efficiently and stress-free.

[0752] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0753] Step 1: Launch the app and log in

[0754] The user launches the smartphone app and the login screen is displayed (input: smartphone app, output: login screen).

[0755] The user enters login information (username and password) and presses the "Login" button (input: username and password, output: login information).

[0756] The terminal sends the entered login information to the server (input: login information, output: request data).

[0757] The server checks the received authentication information, and if it is correct, returns a successful authentication to the terminal (input: request data, output: authentication result).

[0758] The terminal receives the authentication success message and displays the main screen (input: authentication result, output: main screen).

[0759] Step 2: Access the list creation screen

[0760] The user selects "Create a list" from the menu within the app (input: menu selection, output: request to the list creation screen).

[0761] The terminal displays the "Create List" section (Input: Request to create list screen, Output: Create List screen).

[0762] Step 3: Scan the product

[0763] The user presses the "Add product" button to activate the barcode scan function (input: button operation, output: barcode scan screen).

[0764] The device will start the camera and display a view for barcode scanning (input: barcode scanning screen, output: camera view).

[0765] The user scans the product's barcode with a camera (input: barcode, output: barcode data).

[0766] The terminal reads the barcode information and sends it to the server (input: barcode data, output: request to server).

[0767] Step 4: Get product information

[0768] The server searches the database for the corresponding product information based on the received barcode information (input: barcode information, output: product information).

[0769] The server returns the searched product information to the terminal (input: product information, output: response data to the request).

[0770] The terminal displays the product information and updates the user's list (input: response data, output: updated list).

[0771] Step 5: How the Reminders feature works

[0772] The server collects user purchasing data and consumption data for each product (input: purchasing data, output: database dataset).

[0773] The server uses an AI algorithm to analyze the data and predict the optimal replacement time for each product (input: dataset, output: predicted replacement time).

[0774] The server creates a reminder based on the analysis results (input: predicted replacement time, output: reminder).

[0775] The server sends the reminder to the device at the appropriate time (input: reminder, output: request data).

[0776] The device displays the reminder to the user as a push notification (Input: request data, Output: push notification).

[0777] Step 6: Monetization methods in action

[0778] The server manages in-app ad display settings (input: ad data, output: ad settings).

[0779] The device displays ads at appropriate times while the user is using the app (input: ad settings, output: ad display).

[0780] The user reviews the details of the premium features and selects a subscription (Input: Subscription selection, Output: Selection data).

[0781] The terminal sends the user's payment information to the server (input: selection data and payment information, output: payment request).

[0782] The server processes the payment and, if successful, notifies the terminal to unlock the premium features (input: payment request, output: payment result).

[0783] The terminal notifies the user that the premium feature has been enabled (input: payment result, output: notification).

[0784] In this way, the system as a whole allows users to efficiently manage their shopping.

[0785] (Application example 1)

[0786] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0787] Conventional shopping support systems suffer from the cumbersome process of obtaining product information and creating lists, as well as the problem of missing the right timing to purchase. This makes it difficult for users to avoid forgetting to buy something or making unnecessary purchases. Furthermore, they do not adequately provide reminder functions that take individual purchasing patterns into account. Furthermore, they do not adequately provide efficient advertising displays or premium features, which prevents users from increasing satisfaction. To address these issues, a system that innovates and streamlines the user shopping experience is needed.

[0788] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0789] In this invention, the server includes an input means for users to scan product information, a processing means for sending the input product information to a database and retrieving corresponding product information, a management means for creating and updating the user's list based on the retrieved product information, an analysis means for analyzing the user's purchase data and consumption amount and calculating the optimal replacement timing, a notification means for notifying the user of a reminder based on the calculated replacement timing, a monetization means for displaying advertisements and providing premium features, a means for using an application installed on a smartphone, smart glasses, a head-mounted display, or a robot, a means for scanning barcode information with a camera and retrieving product information from the database, a means for accumulating past consumption data and using an AI algorithm to analyze consumption patterns, a means for generating prompt sentences for reminder notifications using a generative AI model, and a means for using the server, Flask, and scikit-learn libraries to create lists and send reminders. This allows users to easily retrieve product information and efficiently create and update lists. Furthermore, by predicting the optimal purchase timing and providing reminders, users can prevent forgetting to buy items and reduce wasteful purchases. Additionally, personalized notifications based on individual purchasing patterns can increase user satisfaction.

[0790] "Input means" refers to a device or interface that provides a function for a user to scan product information.

[0791] The "processing means" is a component that sends input product information to a database and acquires the corresponding product information.

[0792] "Management means" refers to a part of the system that has the function of creating and updating a user's list based on the acquired product information.

[0793] "Analysis means" refers to algorithms or software that analyzes user purchasing data and consumption amounts and calculates the optimal time to replace items.

[0794] The "notification means" is a mechanism for notifying the user of a reminder based on the calculated replacement timing.

[0795] "Monetization Method" means a system or method for displaying advertisements and providing premium features.

[0796] "Means for using an application" refers to the functions and interfaces for using an application installed on a smartphone, smart glasses, head-mounted display, or robot.

[0797] "Means for scanning with a camera" is a function for scanning barcode information with a camera and obtaining product information from a database.

[0798] "Means using AI algorithms" refers to methods that use artificial intelligence algorithms to accumulate past consumption data and analyze consumption patterns.

[0799] "Means for utilizing a generative AI model" refers to a function that uses a generative AI model to generate prompt text for reminder notifications.

[0800] "Methods using a server, Flask, and scikit-learn library" refers to a method of processing using a cloud server and specific program libraries to create lists and send reminders.

[0801] This invention is a system that supports users in shopping efficiently in physical stores. The operation of the system and a specific implementation method are described below.

[0802] System Overview

[0803] The system consists of the following main components:

[0804] Input method: A device such as a smartphone or smart glasses that allows the user to scan product information.

[0805] Processing method: A function that sends scanned barcode information to a database and retrieves the corresponding product information.

[0806] Management method: A system that creates and updates user lists based on acquired product information.

[0807] Analysis method: An artificial intelligence (AI) algorithm that analyzes user purchasing data and consumption to calculate the optimal time to replace items.

[0808] Notifications: A feature that notifies users with reminders based on the calculated replacement period.

[0809] Monetization methods: A system for displaying advertisements and providing premium features.

[0810] Means of using the application: The application is installed on a smartphone, smart glasses, head-mounted display or robot.

[0811] Scan with camera: A function that scans the barcode information of products with a camera and retrieves product information from the database.

[0812] Using AI algorithms: Ability to accumulate past consumption data and use AI algorithms to analyze consumption patterns.

[0813] How to use generative AI models: The ability to use generative AI models to generate prompts for reminder notifications.

[0814] Using a server, Flask, or scikit-learn library: This method uses a cloud server and specific program libraries to create lists and send reminders.

[0815] Program processing

[0816] The server supports an input method that allows users to scan product information. The product information is scanned as barcode information using a smartphone camera and sent to the server. The server then queries a database for the received barcode information and retrieves the corresponding product information. The server then returns the retrieved product information to the terminal, where the user can check the displayed product information to update the list.

[0817] Past consumption data is stored in a cloud environment, and the server analyzes the data using an AI algorithm (e.g., using the scikit-learn library). The server analyzes purchasing patterns and consumption amounts to predict the next purchase date. Based on the calculated replacement date, a generative AI model is used to generate a prompt message.

[0818] The device will then send a reminder to the user based on the generated prompt text, which will be displayed as a push notification to prevent the user from forgetting to make their next purchase.

[0819] The application features advertising and offers the option to disable advertising as a premium feature, which allows users to enjoy more personalized notifications and additional customization features.

[0820] Specific examples

[0821] For example, when a user scans a barcode on a bottle of milk with their smartphone camera, the server receives the barcode information and retrieves the milk's details from the database. The information is then displayed on the device, allowing the user to add the milk to their shopping list. Based on past data, an AI algorithm predicts when the next purchase will be made, and a reminder notification is sent to the user at the appropriate time.

[0822] Prompt Sentence Examples

[0823] User: I scanned the barcode for milk. Add it to the list.

[0824] System: Milk has been added to your list. We'll let you know when it's time for your next purchase.

[0825] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0826] Step 1:

[0827] The user scans the product information.

[0828] Input: Scan the product barcode with your smartphone camera.

[0829] Output: The barcode information read.

[0830] Specific operation: The user launches the smartphone app and uses the barcode scanner function to scan the product barcode with the camera.

[0831] Step 2:

[0832] The terminal transmits the barcode information to the server.

[0833] Input: The barcode information read.

[0834] Output: Barcode information sent to the server.

[0835] Specific operation: The terminal sends the acquired barcode information to the server using an HTTP request, etc.

[0836] Step 3:

[0837] The server retrieves product information from the database.

[0838] Input: The barcode information sent to the server.

[0839] Output: Product information retrieved from the database.

[0840] Specific operation: The server queries the database based on the received barcode information and retrieves related product information.

[0841] Step 4:

[0842] The server returns the acquired product information to the terminal.

[0843] Input: Product information retrieved from the database.

[0844] Output: Product information sent back to the device.

[0845] Specific operation: The server sends the acquired product information to the terminal using an HTTP response, etc.

[0846] Step 5:

[0847] The device displays product information and updates the list.

[0848] Input: Product information received from the server.

[0849] Output: Display of the updated shopping list.

[0850] Specific operation: The terminal displays the received product information on the user interface, and the user confirms and adds the product information to the list.

[0851] Step 6:

[0852] The server analyzes the user's purchasing data.

[0853] Input: User's past purchase and consumption data.

[0854] Output: Prediction of optimal replacement time.

[0855] Specific operation: The server uses an AI algorithm (e.g., the scikit-learn library) to analyze past purchase data and consumption data and calculate the timing of the next purchase.

[0856] Step 7:

[0857] The server generates the reminder prompt.

[0858] Input: Optimal replacement time prediction results.

[0859] Output: The reminder prompt.

[0860] Specific behavior: The server uses the generative AI model to generate prompt text for reminder notifications based on the prediction results.

[0861] Step 8:

[0862] The server sends the reminder to the device.

[0863] Input: The generated prompt statement.

[0864] Output: The reminder notification sent to the device.

[0865] Specific operation: The server sends the generated prompt text to the device in the form of a push notification.

[0866] Step 9:

[0867] The device notifies the user of the reminder.

[0868] Input: Reminder notification sent from the server.

[0869] Output: The reminder notification that is displayed to the user.

[0870] Specific behavior: The device uses the push notification function to display a reminder to the user.

[0871] Example prompt sentence:

[0872] User: I scanned the barcode for milk. Add it to the list.

[0873] System: Milk has been added to your list. We'll let you know when it's time for your next purchase.

[0874] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0875] This invention is a system that allows users to scan product information to create a list, receive reminders at the optimal time, and recognize the user's emotions and display appropriate notifications and advertisements based on those emotions, thereby preventing users from forgetting to buy things and providing efficient shopping. The components of this system and their operation are described in detail below.

[0876] List creation function

[0877] 1. The user launches the app and logs in

[0878] The user launches the smartphone app and logs in.

[0879] The device checks the user's authentication information and displays the main screen if the login is successful.

[0880] 2. Access the list creation screen

[0881] The user accesses the "Create a List" section from the app's menu.

[0882] The device displays the list creation screen.

[0883] 3. Enter product information

[0884] The user scans the item using the barcode scanning feature.

[0885] The terminal reads the barcode and sends the information to the server.

[0886] 4. Obtain product information

[0887] The server searches the database for the corresponding product information based on the received barcode information.

[0888] The server returns the product information to the terminal.

[0889] The terminal displays the product information and updates the user's list.

[0890] For example, if a user wants to add milk to the list, they scan the barcode of the milk. The server retrieves the product data from the barcode information and displays it on the terminal. The user confirms and adds it to the list.

[0891] Reminder function

[0892] 1. Data collection and analysis

[0893] The server periodically collects user purchase data and consumption data for each product.

[0894] The server uses AI algorithms to analyze the data and predict the optimal time to replace each product.

[0895] 2. Create and send reminders

[0896] The server creates reminders based on the analysis results.

[0897] The server sends the created reminder to the device at the appropriate time.

[0898] The device displays the reminder to the user as a push notification.

[0899] For example, if a user regularly buys milk, the server will analyze their past consumption patterns and predict when they will need to buy it again, and a reminder will be sent to the user's device at that time.

[0900] Emotion recognition function

[0901] 1. Emotional awareness

[0902] The device uses a camera and microphone to collect emotional information from the user's facial expressions and voice.

[0903] The device uses an emotion engine to analyze the collected information and recognize the user's emotions.

[0904] 2. Emotion-Based Adjustment

[0905] The server uses data from the emotion engine to adjust the timing and content of reminders based on the user's emotions.

[0906] The server also optimizes the content and timing of advertisements.

[0907] For example, if a user is feeling stressed, the server will suppress reminder notifications or change them to more stressful phrases. Advertisements will also be prioritized to display items related to relaxation and activities.

[0908] Monetization methods

[0909] 1. Advertising

[0910] The server manages in-app ad display settings.

[0911] The device displays ads at appropriate times while the user is using the app.

[0912] 2. Premium Features

[0913] The user reviews the premium features and selects a subscription service.

[0914] The terminal transmits the user's payment information to the server.

[0915] The server processes the payment and, if successful, notifies the device to unlock the premium features.

[0916] For example, if a user wants to purchase a premium feature to remove ads, they select the subscription and make the payment. The server verifies the payment and the premium feature is activated.

[0917] As described above, by incorporating an emotion engine, it is possible to provide flexible reminders and advertisement displays that respond to the user's emotions, providing a more personalized shopping experience.

[0918] The processing flow will be explained below.

[0919] List creation function

[0920] Step 1:

[0921] The user launches the smartphone app and logs in.

[0922] Step 2:

[0923] The device checks the user's authentication information and displays the main screen if the login is successful.

[0924] Step 3:

[0925] The user accesses the "Create a List" section from the app's menu.

[0926] Step 4:

[0927] The device displays the list creation screen.

[0928] Step 5:

[0929] The user scans the item using the barcode scanning feature.

[0930] Step 6:

[0931] The device activates the smartphone camera and reads the barcode.

[0932] Step 7:

[0933] The terminal transmits the barcode information to the server.

[0934] Step 8:

[0935] The server searches the database for and retrieves the relevant product information.

[0936] Step 9:

[0937] The server returns product information (product name, category, price) to the terminal.

[0938] Step 10:

[0939] The terminal adds the acquired product information to the user's list and updates the screen.

[0940] Step 11:

[0941] The user confirms the product addition, manually enters the product name if necessary, and adds the item to the list.

[0942] Step 12:

[0943] The user finishes updating the list and taps the save button.

[0944] Step 13:

[0945] The terminal transmits the list information to the server.

[0946] Step 14:

[0947] The server saves the list in a database and sends a message to the terminal indicating that the list has been saved.

[0948] Step 15:

[0949] The device displays a notification to the user that the save is complete.

[0950] Reminder function

[0951] Step 1:

[0952] The server periodically collects user purchase data and consumption data for each product.

[0953] Step 2:

[0954] The server uses AI algorithms to analyze the data and predict the optimal time to replace each product.

[0955] Step 3:

[0956] The server creates reminders based on the analysis results.

[0957] Step 4:

[0958] The server sends the created reminder to the device at the appropriate time.

[0959] Step 5:

[0960] The device displays the reminder to the user as a push notification.

[0961] Step 6:

[0962] The user checks the reminder and checks the shopping list.

[0963] Emotion recognition function

[0964] Step 1:

[0965] The device uses a camera and microphone to collect emotional information from the user's facial expressions and voice.

[0966] Step 2:

[0967] The device uses an emotion engine to analyze the collected information and recognize the user's emotions.

[0968] Step 3:

[0969] The server uses data from the emotion engine to adjust the timing and content of reminders based on the user's emotions.

[0970] Step 4:

[0971] The server also optimizes the content and timing of advertisements.

[0972] Step 5:

[0973] The device displays tailored reminders and advertisements to the user.

[0974] For example, if a user is feeling stressed, the server will suppress reminder notifications or change them to more "menos estresante" language. It also prioritizes advertisements related to relaxation items and activities.

[0975] Monetization methods

[0976] Step 1:

[0977] The server manages in-app ad display settings.

[0978] Step 2:

[0979] The device displays ads at appropriate times while the user is using the app.

[0980] Step 3:

[0981] The user reviews the premium features and selects a subscription service.

[0982] Step 4:

[0983] The user enters payment information to purchase a subscription and taps the "Purchase" button.

[0984] Step 5:

[0985] The terminal sends the payment information to the server.

[0986] Step 6:

[0987] The server processes the payment and, if successful, notifies the device to unlock the premium features.

[0988] Step 7:

[0989] The device will confirm the release of the premium features and notify the user.

[0990] Example 2

[0991] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0992] In today's highly information-driven society, users need tools that allow them to shop efficiently and without missing anything. However, conventional systems are limited to managing product information and reminder functions, and are unable to respond flexibly to user emotions. As a result, users may feel more stressed, and reminders and advertisements may not be as effective as they should be.

[0993] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0994] In this invention, the server includes input means for a user to scan product information, processing means for transmitting the input product information to a database and retrieving corresponding product information, management means for creating and updating the user's list based on the retrieved product information, analysis means for analyzing the user's purchase data and consumption amount and calculating the optimal replacement timing, notification means for notifying the user of a reminder based on the calculated replacement timing, emotion recognition means for recognizing emotions from the user's facial expressions and voice, adaptation means for adjusting the content of reminders and advertisements based on the recognized emotions, and monetization means for displaying advertisements and providing premium features, thereby enabling a flexible and stress-free shopping experience that responds to the user's emotions.

[0995] "Input means" refers to an interface that allows a user to input product information into the system, and specifically includes a barcode scanner, a camera, a touch screen, etc.

[0996] "Processing means" refers to a computer system for receiving input product information and transmitting it to a database to retrieve corresponding product information.

[0997] "Management means" refers to tools and software for creating and updating user lists based on acquired product information.

[0998] "Analysis method" refers to a system that collects user purchasing data and consumption data and uses AI technology to calculate the optimal time to replace items.

[0999] The "notification means" refers to a method for notifying the user of a reminder based on the calculated replacement timing, and mainly includes push notification and email.

[1000] "Emotion recognition means" refers to technology or systems that analyze a user's facial expressions, voice, etc. to recognize the user's emotions.

[1001] "Adaptation" refers to algorithms or systems that tailor reminder or ad content based on perceived emotions.

[1002] "Monetization Method" refers to a method or system for generating revenue through displaying advertisements and providing premium features.

[1003] This invention is a system that allows users to scan product information to create a list, receive reminders at optimal times, and recognize the user's emotions and display optimal notifications and advertisements based on those emotions, thereby preventing users from forgetting to buy items and providing efficient shopping. Specific embodiments of this system are described below.

[1004] First, the user launches the smartphone app and logs in. The device displays the login screen, and the user logs in by entering their email address and password. The device sends the entered authentication information to the server, which then compares it with a database for authentication. If authentication is successful, the server sends a command to the device to display the main screen, and the device displays the main screen.

[1005] Next, the user accesses the "Create a List" section from the app's menu. The device displays the list creation screen, and the user taps the "Add New Item" button. The device then activates its camera and displays the message "Please scan the barcode." When the user holds the product's barcode up to the camera to scan it, the device reads the barcode information and sends it to the server.

[1006] The server searches for the corresponding product information in a database (e.g., a MySQL database) based on the received barcode information. The server returns the searched product information (product name, price, manufacturer, etc.) to the terminal, which displays the product information on the screen and asks the user for confirmation. Once the user confirms, they can tap the "Add to List" button to add the product to the list.

[1007] After the list is created, the server periodically collects the user's purchase data and consumption data for each product. The collected data is analyzed using Python's Pandas and Scikit-learn to train an AI model to predict the optimal replacement timing for each product. Reminders are created based on the prediction results, and the server sends the reminders to the device at the appropriate time. The device then displays the reminders to the user as push notifications.

[1008] The device also uses a camera and microphone to collect the user's facial expressions and voice. The collected data is analyzed using Google Cloud Vision API and Microsoft Azure Cognitive Services to recognize the user's emotions. The server receives the emotion data and understands the user's emotional state.

[1009] Based on the emotion data, the server can adjust the timing and content of reminders. For example, if the user is feeling stressed, the server might tone down reminders or change them to less stressful phrases. Similarly, the server might prioritize ads for products and services related to relaxation.

[1010] Finally, as a means of monetization, the server manages in-app advertising using Google AdMob or Facebook Audience Network. The device displays ads at appropriate times while the user is using the app. Furthermore, the user checks detailed information in the "Premium Features" section of the app, selects a subscription service, and enters payment information. The device sends the payment information to the server, which processes the payment using the Stripe API. If the payment is successful, the server sends a command to unlock the premium features to the device, and the device activates the premium features.

[1011] For example, if a user wants to add milk to a list, they scan the milk's barcode. The server retrieves the product data from the barcode and displays it on the device. The user confirms and adds it to the list. If the user regularly purchases milk, the server analyzes past consumption patterns and predicts when the next purchase will be necessary. A reminder will be sent to the user's device at that time.

[1012] This allows users to enjoy an efficient and personalized shopping experience.

[1013] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1014] List creation function

[1015] Step 1:

[1016] A user launches a smartphone app and accesses the login screen. As input, the user enters an email address and password. Based on this, the device sends the entered authentication information to the server. The server compares it with a database and performs authentication. As output, if authentication is successful, the server sends a command to display the main screen to the device, and the device displays the main screen.

[1017] Step 2:

[1018] The user accesses the "Create a List" section from the app's menu. As input, the user taps the menu icon. Based on this, the device displays the list creation screen.

[1019] Step 3:

[1020] The user taps the "Add New Product" button to activate the camera function. As input, the user holds the product's barcode up to the camera to scan it. Based on this, the device reads the barcode information and sends it to the server.

[1021] Step 4:

[1022] Based on the barcode information received by the server, the server searches for the corresponding product information in a database. A database query is made, receiving the barcode information as input. Based on this, the server retrieves product information (product name, price, manufacturer, etc.) and returns it to the terminal. As output, the server returns the product information to the terminal, which then displays the product information on the screen.

[1023] Step 5:

[1024] The user checks the product information and taps the "Add to List" button to add the product to the list. As input, there is the user's action of tapping the "Add to List" button. Based on this, the device adds the product to the user's list and updates the list. As output, the list is updated.

[1025] Reminder function

[1026] Step 1:

[1027] The server periodically collects user purchase data and consumption data for each product. The inputs are purchase history and consumption data. Based on this, the server uses Python's Pandas library to format the data and create a dataset for the AI ​​model. The formatted dataset is generated as the output.

[1028] Step 2:

[1029] The server uses Scikit-learn to train an AI model to predict the optimal replacement time for each product. The input is a formatted dataset. Based on this, the AI ​​algorithm analyzes the data and generates a prediction. The output is a prediction of the optimal replacement time.

[1030] Step 3:

[1031] The server creates a reminder to notify the next purchase timing based on the prediction results. The input is the predicted optimal replacement time. Based on this, the server generates the reminder text and designs the appropriate timing. The output is a reminder notification.

[1032] Step 4:

[1033] The device displays the reminder to the user as a push notification. As input, there is a reminder notification sent from the server. Based on this, the device displays a push notification to the user. As output, the user receives the reminder.

[1034] Emotion recognition function

[1035] Step 1:

[1036] The device uses a camera and microphone to collect the user's facial expressions and voice. The input is the user's facial expression and voice data. Based on this, the device sends the data to Google Cloud Vision API or Microsoft Azure Cognitive Services for analysis. The output is emotion data.

[1037] Step 2:

[1038] The server receives the emotional data and understands the user's emotional state. The input is the analyzed emotional data. Based on this, the server evaluates the user's emotional state and generates emotional state information. The output is the emotional state information.

[1039] Step 3:

[1040] The server adjusts the timing and content of reminders based on emotional data. The input is emotional state information. Based on this, the server generates and adjusts the content of reminders and advertisements appropriately. The output is the adjusted reminders and advertisements.

[1041] Monetization methods

[1042] Step 1:

[1043] The server manages the ad network settings and sets the timing and content of ad display. The input is the ad network configuration information. Based on this, the server manages ads using Google AdMob or Facebook Audience Network. The output is the ad display settings.

[1044] Step 2:

[1045] The device displays ads at appropriate times while the user is using the app. As input, there are ad display settings received from the server. Based on this, the device displays ads while the user is using the app. As output, the ads are displayed to the user.

[1046] Step 3:

[1047] The user checks the details in the "Premium Features" section and selects a subscription service. As an input, there is an action where the user selects a subscription service. Based on this, the terminal sends payment information to the server. As an output, payment information is sent to the server.

[1048] Step 4:

[1049] The server receives the payment information and processes the payment. As input, it has the user's payment information. Based on this, the server processes the payment using Stripe API and receives the result. As output, it gets the payment success or failure status.

[1050] Step 5:

[1051] If the payment is successful, the server sends a command to unlock the premium feature to the terminal. As input, there is a payment success status. Based on this, the server generates a command to unlock the premium feature and sends it to the terminal. As output, the premium feature becomes active.

[1052] As a result, this system can provide a flexible and personalized shopping experience that responds to the user's emotions.

[1053] (Application example 2)

[1054] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1055] Conventional shopping support systems often have problems such as users forgetting to buy items or not being able to purchase items at the optimal time. Furthermore, because the user experience is not personalized through standard notifications and advertisements, they are unable to provide an efficient and comfortable shopping experience. Furthermore, because they do not take into account the user's emotional state, notifications and advertisements are displayed at inappropriate times, which can cause stress to the user.

[1056] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes an input means for the user to scan product information, a processing means for transmitting the input product information to a database and acquiring corresponding product information, and a management means for creating and updating the user's list based on the acquired product information. This system also includes an analysis means for analyzing the user's purchase data and consumption amount to calculate the optimal replacement timing, a notification means for notifying the user of a reminder based on the calculated replacement timing, an emotion recognition means for recognizing the user's emotions and displaying notifications and advertisements based on the emotions, and a monetization means for displaying advertisements and providing premium features. This not only prevents the user from forgetting to buy something and enables them to purchase products at the optimal time, but also provides personalized notifications and advertisements based on the user's emotions.

[1057] "Input means" is a function that allows the user to scan and input product information.

[1058] The "processing means" is a function for transmitting input product information to a database and retrieving the corresponding product information.

[1059] The "management means" is a function for creating and updating a user list based on the acquired product information.

[1060] The "analysis means" is a function for analyzing the user's purchasing data and consumption amount, and calculating the optimal replacement time.

[1061] The "notification means" is a function for notifying the user of a reminder based on the calculated replacement timing.

[1062] The "emotion recognition means" is a function for recognizing the user's emotions and displaying notifications and advertisements based on those emotions.

[1063] "Monetization means" refers to functions for displaying advertisements and providing premium features.

[1064] This invention is a system that allows users to scan product information and create and update lists. It also recognizes the user's emotions and displays reminders, notifications, and advertisements at optimal times based on those emotions, preventing users from forgetting to buy something and providing an efficient shopping experience.

[1065] System configuration

[1066] The system mainly consists of the following components:

[1067] 1. Input Method

[1068] This is a means for users to scan products using a barcode scanner application installed on a device such as a smartphone.

[1069] 2. Processing Methods

[1070] The entered product information is sent to the database, and the corresponding product information is retrieved. At this time, the server searches the database for the product information and returns it to the user's terminal.

[1071] 3. Control measures

[1072] This function creates and updates the user's list based on the acquired product information. Products scanned by the user are automatically added to the list.

[1073] 4. Analysis tools

[1074] The AI ​​algorithm analyzes user purchasing data and consumption data to calculate the optimal time to replace the product. Here, the AI ​​analysis model is implemented using programming languages ​​such as Python.

[1075] 5. Means of notification

[1076] Based on the calculated replacement timing, a reminder is created and the user is notified at the appropriate time via the smartphone's push notification function.

[1077] 6. Emotion recognition means

[1078] It recognizes users' emotions and adjusts notifications and ads accordingly. Emotion recognition uses the smartphone's camera and microphone to collect emotional data from the user's facial expressions and voice, which is then analyzed by an AI model that includes a pre-trained emotion recognition algorithm.

[1079] 7. Monetization Methods

[1080] It displays advertisements and provides premium features. Users can choose to subscribe to the service to receive premium features such as the ability to hide advertisements.

[1081] Explanation of program processing

[1082] The system uses the following hardware and software:

[1083] Smartphone: Barcode scanning, emotion recognition (camera / microphone), push notification display.

[1084] Server: Database management, data analysis using AI algorithms, and reminder generation.

[1085] AI model: A pre-trained model for recognizing user emotions (e.g., emotion recognition models using Keras or TensorFlow).

[1086] Database: Stores product information and user purchase history data.

[1087] The specific process is as follows:

[1088] 1. The smartphone application scans the product barcode and sends it to the server.

[1089] 2. The server retrieves product information from the database and sends it back to the smartphone.

[1090] 3. Product information will be added and updated to the list.

[1091] 4. An AI algorithm analyzes purchasing history and calculates the optimal time to replace the product.

[1092] 5. Reminders are sent as push notifications at the right time.

[1093] 6. The smartphone's camera and microphone capture the user's facial expressions and voice, which are then analyzed by an emotion recognition model.

[1094] 7. Reminders and ads are tailored based on user emotions.

[1095] Specific examples

[1096] For example, if a user scans a bottle of milk at the supermarket, a product information API is called and detailed information about the product is displayed on the smartphone. Also, if the user is feeling stressed, a push notification will be sent discreetly or a product suggestion will be made to help them relax.

[1097] Prompt Sentence Examples

[1098] Design an application that allows users to scan items in a physical store using a smartphone app and automatically create a shopping list. Use emotion recognition technology to display notifications and ads based on the user's emotions and send reminders at optimal times based on the user's purchasing history. Specific features should include real-time item scanning, emotion recognition, list creation, reminders, and personalized ads.

[1099] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1100] Step 1:

[1101] The user scans the item.

[1102] Input: Scan the product barcode with your smartphone's barcode scanner.

[1103] How it works: A user holds an item up to their smartphone camera to scan the barcode.

[1104] Output: The barcode information is entered into the terminal.

[1105] Step 2:

[1106] The terminal transmits the barcode information to the server.

[1107] Input: Barcode information.

[1108] Operation: The terminal generates a request to send the barcode information to the database API.

[1109] Output: The barcode information sent to the server.

[1110] Step 3:

[1111] The server searches the database for the relevant product information.

[1112] Input: The barcode information sent.

[1113] Operation: The server generates a database query to retrieve product information corresponding to the barcode.

[1114] Output: Product information.

[1115] Step 4:

[1116] The server returns the product information to the terminal.

[1117] Input: Product information retrieved from the database.

[1118] Operation: The server generates a response to return product information to the terminal.

[1119] Output: Product information sent to the device.

[1120] Step 5:

[1121] The device displays product information and updates the list.

[1122] Input: Product information received from the server.

[1123] Action: The device displays product information to the user and adds the product to an existing shopping list.

[1124] Output: The updated shopping list.

[1125] Step 6:

[1126] The server analyzes the purchase data and consumption amounts.

[1127] Input: User purchase history data and consumption data.

[1128] How it works: The server uses AI algorithms to analyze the data and predict the best time to make your next purchase.

[1129] Output: Best time to replace.

[1130] Step 7:

[1131] The server generates reminders and sends them to the device at the appropriate time.

[1132] Input: Optimal replacement time.

[1133] How it works: The server generates a reminder and sends it to the device as a push notification.

[1134] Output: The reminder displayed on the user's device.

[1135] Step 8:

[1136] The device recognizes the user's emotions.

[1137] Input: User facial and voice data collected by the smartphone camera and microphone.

[1138] How it works: The device inputs the collected data into an emotion recognition model to analyze the user's emotions.

[1139] Output: The user's emotional state.

[1140] Step 9:

[1141] The server tailors reminders and advertisements based on the emotional state.

[1142] Input: User emotional state data.

[1143] How it works: The server adjusts the content and timing of notifications based on the user's emotional state, generating personalized ads.

[1144] Output: Tailored reminders and ads.

[1145] Step 10:

[1146] Your device will display tailored reminders and ads.

[1147] Enter: tailored reminders and ads.

[1148] How it works: The device displays reminders and advertisements to the user at appropriate times.

[1149] Output: Personalized reminders and ads shown to the user.

[1150] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1151] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1152] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1153] [Third embodiment]

[1154] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1155] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1156] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1157] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1158] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1159] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1160] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1161] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1162] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1163] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1164] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1165] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1166] This invention is a system that helps users avoid forgetting to buy things and supports efficient shopping by allowing them to create a list by scanning product information and receiving reminders at the appropriate time.The operation of each part of the system is described in detail below.

[1167] List creation function

[1168] 1. The user launches the app and logs in

[1169] The user launches the smartphone app and logs in.

[1170] The terminal checks the user's authentication information and displays the main screen if the login is successful.

[1171] 2. Access the list creation screen

[1172] Users access the "Create a List" section from the app's menu.

[1173] The device displays a list creation screen.

[1174] 3. Enter product information

[1175] The user scans the product using the barcode scanning function.

[1176] The terminal reads the barcode and transmits the information to the server.

[1177] 4. Obtain product information

[1178] The server searches the database for the corresponding product information based on the received barcode information.

[1179] The server returns the product information to the terminal.

[1180] The terminal displays the product information and updates the user's list.

[1181] For example, if a user wants to add milk to the list, they scan the barcode of the milk. The server retrieves the product data from the barcode information and displays it on the terminal. The user confirms and adds it to the list.

[1182] Reminder function

[1183] 1. Data collection and analysis

[1184] The server collects user purchase data and consumption data for each product.

[1185] The server uses AI algorithms to analyze the data and predict the optimal time to replace each product.

[1186] 2. Create and send reminders

[1187] The server creates reminders based on the analysis results.

[1188] The server sends the reminder to the device at the appropriate time.

[1189] The device displays the reminder to the user as a push notification.

[1190] For example, if a user regularly buys milk, the server will analyze their past consumption patterns and predict when they will need to buy it again, and a reminder will be sent to the user's device at that time.

[1191] Monetization methods

[1192] 1. Advertising

[1193] The server manages in-app advertising display settings.

[1194] The device displays advertisements at appropriate times while the user is using the app.

[1195] 2. Premium Features

[1196] The user reviews the details of the premium features and selects a subscription.

[1197] The terminal transmits the user's payment information to the server.

[1198] The server processes the payment and, if successful, notifies the device to unlock the premium features.

[1199] For example, if a user wants to purchase a premium feature to remove ads, they select the subscription and make the payment. The server verifies the payment and the premium feature is activated.

[1200] As described above, this system helps users to shop efficiently and stress-free, and is realized by the cooperation of each function.

[1201] The processing flow will be explained below.

[1202] List creation function

[1203] Step 1:

[1204] The user launches the smartphone app and logs in.

[1205] Step 2:

[1206] The device checks the user's authentication information and displays the main screen if the login is successful.

[1207] Step 3:

[1208] The user accesses the "Create a List" section from the app's menu.

[1209] Step 4:

[1210] The device displays the list creation screen.

[1211] Step 5:

[1212] The user uses the barcode scanning feature to add everyday items to the list.

[1213] Step 6:

[1214] The device activates the smartphone camera and reads the barcode.

[1215] Step 7:

[1216] The terminal transmits the barcode information to the server.

[1217] Step 8:

[1218] The server searches the database for and retrieves the relevant product information.

[1219] Step 9:

[1220] The server returns product information (product name, category, price) to the terminal.

[1221] Step 10:

[1222] The terminal adds the acquired product information to the user's list and updates the screen.

[1223] Step 11:

[1224] The user confirms the product addition, manually enters the product name if necessary, and adds the item to the list.

[1225] Step 12:

[1226] The user finishes updating the list and taps the save button.

[1227] Step 13:

[1228] The terminal transmits the list information to the server.

[1229] Step 14:

[1230] The server saves the list in a database and sends a message to the terminal indicating that the list has been saved.

[1231] Step 15:

[1232] The device displays a notification to the user that the save is complete.

[1233] Reminder function

[1234] Step 1:

[1235] The server periodically collects user purchase data and consumption data for each product.

[1236] Step 2:

[1237] The server uses AI algorithms to analyze the data and predict the optimal time to replace each product.

[1238] Step 3:

[1239] The server creates reminders based on the analysis results.

[1240] Step 4:

[1241] The server sends the created reminder to the device at the appropriate time.

[1242] Step 5:

[1243] The device displays the reminder to the user as a push notification.

[1244] Step 6:

[1245] The user checks the reminder and checks the shopping list.

[1246] Monetization methods

[1247] Step 1:

[1248] The server manages in-app ad display settings.

[1249] Step 2:

[1250] The device displays ads at appropriate times while the user is using the app.

[1251] Step 3:

[1252] The user reviews the premium features and selects a subscription service.

[1253] Step 4:

[1254] The user enters payment information to purchase a subscription and taps the "Purchase" button.

[1255] Step 5:

[1256] The terminal sends the payment information to the server.

[1257] Step 6:

[1258] The server processes the payment and, if successful, notifies the device to unlock the premium features.

[1259] Step 7:

[1260] The device will confirm the release of the premium features and notify the user.

[1261] Example 1

[1262] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1263] In conventional shopping support systems, users had to manually input product information, which was time-consuming, and it was difficult to receive timely reminders based on purchase history and consumption amounts.In addition, advertising and premium features were not provided sufficiently, which led to issues with the convenience and profitability of the entire system.

[1264] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1265] In this invention, the server includes an input means for a user to scan product information, a processing means for transmitting the input product information to a database and retrieving the corresponding product information, a management means for creating and updating the user's list based on the retrieved product information, a data analysis means for analyzing the user's purchase data and consumption amount and calculating the appropriate replacement timing, a notification means for notifying the user of a reminder based on the calculated replacement timing, a monetization means for displaying advertisements and providing premium features, an authentication means for performing user authentication and displaying a main screen upon successful authentication, a notification display means for sending the displayed reminder as a push notification, and a reading means for using a camera to read barcode information for product scanning. This reduces the user's effort, efficiently lists product information, realizes timely reminder notifications, and further strengthens the monetization means.

[1266] "Input means" refers to a device or method by which a user inputs product information into the system.

[1267] The "processing means" refers to a device or method that has the function of transmitting input product information to a database and searching for and retrieving the corresponding product information.

[1268] The "management means" is a device or method that has the function of creating and updating a user list based on the acquired product information.

[1269] The "data analysis means" is a device or method for analyzing a user's purchasing data and consumption amount, and calculating the appropriate time to replace the product.

[1270] The "notification means" is a device or method for notifying the user of a reminder based on the calculated replacement timing.

[1271] A "monetization means" is a device or method that has the function of displaying advertisements and providing premium features.

[1272] "Authentication means" refers to a device or method for authenticating a user and displaying the main screen when authentication is successful.

[1273] A "notification display means" is a device or method for sending a displayed reminder as a push notification.

[1274] "Reading means" refers to a device or method that has the function of reading barcode information using a camera for scanning products.

[1275] This invention is a system that supports efficient shopping by allowing users to scan product information, create a list, and receive reminders at appropriate times to prevent forgetting to buy something. The following components and specific operations are included in the embodiment of the invention.

[1276] System Configuration

[1277] The system allows users to input product information through a smartphone application, and the server processes the information to create shopping lists, send reminder notifications, and generate revenue. The details of each component are shown below.

[1278] Hardware and Software

[1279] Smartphone: A device used by a user that runs applications and uses a camera to scan barcodes.

[1280] Server: Accesses the database and processes product information for management, analysis, notification, and monetization.

[1281] Database: Stores product information and user purchasing data.

[1282] Main features

[1283] 1. Input method:

[1284] The user scans the product information using the barcode scanner function of their smartphone.

[1285] The scanned barcode information is sent to the server via the terminal.

[1286] 2. Processing Method:

[1287] The server searches the database based on the received barcode information and obtains the corresponding product information.

[1288] The acquired product information is returned to the terminal and displayed.

[1289] 3. Control measures:

[1290] A list is created and updated on the user's smartphone based on the returned product information.

[1291] 4. Data analysis methods:

[1292] The server analyzes the user's purchasing data and consumption data, and uses an AI model to calculate the optimal time to replace the device.

[1293] 5. Means of notification:

[1294] Based on the calculated replacement time, the server generates a reminder and sends it to the device at the appropriate time.

[1295] The device displays the reminder to the user as a push notification.

[1296] 6. Monetization methods:

[1297] The server manages in-app advertisements and sets them to be displayed on the device at the appropriate time.

[1298] If the user selects the premium feature, the server processes the payment and unlocks the premium feature.

[1299] 7. Authentication Methods:

[1300] When a user logs in to the application, the server authenticates the user and displays the main screen if the authentication is successful.

[1301] 8. Notification display means:

[1302] The device displays the reminder sent from the server as a push notification.

[1303] 9. Reading Method:

[1304] Product information is read using the smartphone camera and barcode information is obtained.

[1305] Specific examples

[1306] For example, if a user wants to add milk to the list, the system works as follows:

[1307] A user scans the barcode on the milk with their smartphone (User).

[1308] The terminal sends the barcode information to the server (terminal).

[1309] The server retrieves the relevant milk information from the database and returns it to the terminal (server).

[1310] The terminal displays the acquired product information, and the user adds it to the list (terminal).

[1311] Furthermore, if a user regularly purchases milk, the server analyzes the purchase data and predicts when the next replacement purchase is due, then sends a reminder to the user's smartphone as a push notification at the appropriate time.

[1312] Prompt Sentence Examples

[1313] Here are some examples of prompts that specifically use generative AI models:

[1314] "What are the steps to add milk to the list using the system?"

[1315] "Please explain how you can create reminders for items you regularly purchase."

[1316] "How do I upgrade to premium features?"

[1317] In this way, the system of the present invention can support users in shopping efficiently and stress-free.

[1318] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1319] Step 1: Launch the app and log in

[1320] The user launches the smartphone app and the login screen is displayed (input: smartphone app, output: login screen).

[1321] The user enters login information (username and password) and presses the "Login" button (input: username and password, output: login information).

[1322] The terminal sends the entered login information to the server (input: login information, output: request data).

[1323] The server checks the received authentication information, and if it is correct, returns a successful authentication to the terminal (input: request data, output: authentication result).

[1324] The terminal receives the authentication success message and displays the main screen (input: authentication result, output: main screen).

[1325] Step 2: Access the list creation screen

[1326] The user selects "Create a list" from the menu within the app (input: menu selection, output: request to the list creation screen).

[1327] The terminal displays the "Create List" section (Input: Request to create list screen, Output: Create List screen).

[1328] Step 3: Scan the product

[1329] The user presses the "Add product" button to activate the barcode scan function (input: button operation, output: barcode scan screen).

[1330] The device will start the camera and display a view for barcode scanning (input: barcode scanning screen, output: camera view).

[1331] The user scans the product's barcode with a camera (input: barcode, output: barcode data).

[1332] The terminal reads the barcode information and sends it to the server (input: barcode data, output: request to server).

[1333] Step 4: Get product information

[1334] The server searches the database for the corresponding product information based on the received barcode information (input: barcode information, output: product information).

[1335] The server returns the searched product information to the terminal (input: product information, output: response data to the request).

[1336] The terminal displays the product information and updates the user's list (input: response data, output: updated list).

[1337] Step 5: How the Reminders feature works

[1338] The server collects user purchasing data and consumption data for each product (input: purchasing data, output: database dataset).

[1339] The server uses an AI algorithm to analyze the data and predict the optimal replacement time for each product (input: dataset, output: predicted replacement time).

[1340] The server creates a reminder based on the analysis results (input: predicted replacement time, output: reminder).

[1341] The server sends the reminder to the device at the appropriate time (input: reminder, output: request data).

[1342] The device displays the reminder to the user as a push notification (Input: request data, Output: push notification).

[1343] Step 6: Monetization methods in action

[1344] The server manages in-app ad display settings (input: ad data, output: ad settings).

[1345] The device displays ads at appropriate times while the user is using the app (input: ad settings, output: ad display).

[1346] The user reviews the details of the premium features and selects a subscription (Input: Subscription selection, Output: Selection data).

[1347] The terminal sends the user's payment information to the server (input: selection data and payment information, output: payment request).

[1348] The server processes the payment and, if successful, notifies the terminal to unlock the premium features (input: payment request, output: payment result).

[1349] The terminal notifies the user that the premium feature has been enabled (input: payment result, output: notification).

[1350] In this way, the system as a whole allows users to efficiently manage their shopping.

[1351] (Application example 1)

[1352] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1353] Conventional shopping support systems suffer from the cumbersome process of obtaining product information and creating lists, as well as the problem of missing the right timing to purchase. This makes it difficult for users to avoid forgetting to buy something or making unnecessary purchases. Furthermore, they do not adequately provide reminder functions that take individual purchasing patterns into account. Furthermore, they do not adequately provide efficient advertising displays or premium features, which prevents users from increasing satisfaction. To address these issues, a system that innovates and streamlines the user shopping experience is needed.

[1354] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1355] In this invention, the server includes an input means for users to scan product information, a processing means for sending the input product information to a database and retrieving corresponding product information, a management means for creating and updating the user's list based on the retrieved product information, an analysis means for analyzing the user's purchase data and consumption amount and calculating the optimal replacement timing, a notification means for notifying the user of a reminder based on the calculated replacement timing, a monetization means for displaying advertisements and providing premium features, a means for using an application installed on a smartphone, smart glasses, a head-mounted display, or a robot, a means for scanning barcode information with a camera and retrieving product information from the database, a means for accumulating past consumption data and using an AI algorithm to analyze consumption patterns, a means for generating prompt sentences for reminder notifications using a generative AI model, and a means for using the server, Flask, and scikit-learn libraries to create lists and send reminders. This allows users to easily retrieve product information and efficiently create and update lists. Furthermore, by predicting the optimal purchase timing and providing reminders, users can prevent forgetting to buy items and reduce wasteful purchases. Additionally, personalized notifications based on individual purchasing patterns can increase user satisfaction.

[1356] "Input means" refers to a device or interface that provides a function for a user to scan product information.

[1357] The "processing means" is a component that sends input product information to a database and acquires the corresponding product information.

[1358] "Management means" refers to a part of the system that has the function of creating and updating a user's list based on the acquired product information.

[1359] "Analysis means" refers to algorithms or software that analyzes user purchasing data and consumption amounts and calculates the optimal time to replace items.

[1360] The "notification means" is a mechanism for notifying the user of a reminder based on the calculated replacement timing.

[1361] "Monetization Method" means a system or method for displaying advertisements and providing premium features.

[1362] "Means for using an application" refers to the functions and interfaces for using an application installed on a smartphone, smart glasses, head-mounted display, or robot.

[1363] "Means for scanning with a camera" is a function for scanning barcode information with a camera and obtaining product information from a database.

[1364] "Means using AI algorithms" refers to methods that use artificial intelligence algorithms to accumulate past consumption data and analyze consumption patterns.

[1365] "Means for utilizing a generative AI model" refers to a function that uses a generative AI model to generate prompt text for reminder notifications.

[1366] "Methods using a server, Flask, and scikit-learn library" refers to a method of processing using a cloud server and specific program libraries to create lists and send reminders.

[1367] This invention is a system that supports users in shopping efficiently in physical stores. The operation of the system and a specific implementation method are described below.

[1368] System Overview

[1369] The system consists of the following main components:

[1370] Input method: A device such as a smartphone or smart glasses that allows the user to scan product information.

[1371] Processing method: A function that sends scanned barcode information to a database and retrieves the corresponding product information.

[1372] Management method: A system that creates and updates user lists based on acquired product information.

[1373] Analysis method: An artificial intelligence (AI) algorithm that analyzes user purchasing data and consumption to calculate the optimal time to replace items.

[1374] Notifications: A feature that notifies users with reminders based on the calculated replacement period.

[1375] Monetization methods: A system for displaying advertisements and providing premium features.

[1376] Means of using the application: The application is installed on a smartphone, smart glasses, head-mounted display or robot.

[1377] Scan with camera: A function that scans the barcode information of products with a camera and retrieves product information from the database.

[1378] Using AI algorithms: Ability to accumulate past consumption data and use AI algorithms to analyze consumption patterns.

[1379] How to use generative AI models: The ability to use generative AI models to generate prompts for reminder notifications.

[1380] Using a server, Flask, or scikit-learn library: This method uses a cloud server and specific program libraries to create lists and send reminders.

[1381] Program processing

[1382] The server supports an input method that allows users to scan product information. The product information is scanned as barcode information using a smartphone camera and sent to the server. The server then queries a database for the received barcode information and retrieves the corresponding product information. The server then returns the retrieved product information to the terminal, where the user can check the displayed product information to update the list.

[1383] Past consumption data is stored in a cloud environment, and the server analyzes the data using an AI algorithm (e.g., using the scikit-learn library). The server analyzes purchasing patterns and consumption amounts to predict the next purchase date. Based on the calculated replacement date, a generative AI model is used to generate a prompt message.

[1384] The device will then send a reminder to the user based on the generated prompt text, which will be displayed as a push notification to prevent the user from forgetting to make their next purchase.

[1385] The application features advertising and offers the option to disable advertising as a premium feature, which allows users to enjoy more personalized notifications and additional customization features.

[1386] Specific examples

[1387] For example, when a user scans a barcode on a bottle of milk with their smartphone camera, the server receives the barcode information and retrieves the milk's details from the database. The information is then displayed on the device, allowing the user to add the milk to their shopping list. Based on past data, an AI algorithm predicts when the next purchase will be made, and a reminder notification is sent to the user at the appropriate time.

[1388] Prompt Sentence Examples

[1389] User: I scanned the barcode for milk. Add it to the list.

[1390] System: Milk has been added to your list. We'll let you know when it's time for your next purchase.

[1391] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1392] Step 1:

[1393] The user scans the product information.

[1394] Input: Scan the product barcode with your smartphone camera.

[1395] Output: The barcode information read.

[1396] Specific operation: The user launches the smartphone app and uses the barcode scanner function to scan the product barcode with the camera.

[1397] Step 2:

[1398] The terminal transmits the barcode information to the server.

[1399] Input: The barcode information read.

[1400] Output: Barcode information sent to the server.

[1401] Specific operation: The terminal sends the acquired barcode information to the server using an HTTP request, etc.

[1402] Step 3:

[1403] The server retrieves product information from the database.

[1404] Input: The barcode information sent to the server.

[1405] Output: Product information retrieved from the database.

[1406] Specific operation: The server queries the database based on the received barcode information and retrieves related product information.

[1407] Step 4:

[1408] The server returns the acquired product information to the terminal.

[1409] Input: Product information retrieved from the database.

[1410] Output: Product information sent back to the device.

[1411] Specific operation: The server sends the acquired product information to the terminal using an HTTP response, etc.

[1412] Step 5:

[1413] The device displays product information and updates the list.

[1414] Input: Product information received from the server.

[1415] Output: Display of the updated shopping list.

[1416] Specific operation: The terminal displays the received product information on the user interface, and the user confirms and adds the product information to the list.

[1417] Step 6:

[1418] The server analyzes the user's purchasing data.

[1419] Input: User's past purchase and consumption data.

[1420] Output: Prediction of optimal replacement time.

[1421] Specific operation: The server uses an AI algorithm (e.g., the scikit-learn library) to analyze past purchase data and consumption data and calculate the timing of the next purchase.

[1422] Step 7:

[1423] The server generates the reminder prompt.

[1424] Input: Optimal replacement time prediction results.

[1425] Output: The reminder prompt.

[1426] Specific behavior: The server uses the generative AI model to generate prompt text for reminder notifications based on the prediction results.

[1427] Step 8:

[1428] The server sends the reminder to the device.

[1429] Input: The generated prompt statement.

[1430] Output: The reminder notification sent to the device.

[1431] Specific operation: The server sends the generated prompt text to the device in the form of a push notification.

[1432] Step 9:

[1433] The device notifies the user of the reminder.

[1434] Input: Reminder notification sent from the server.

[1435] Output: The reminder notification that is displayed to the user.

[1436] Specific behavior: The device uses the push notification function to display a reminder to the user.

[1437] Example prompt sentence:

[1438] User: I scanned the barcode for milk. Add it to the list.

[1439] System: Milk has been added to your list. We'll let you know when it's time for your next purchase.

[1440] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1441] This invention is a system that allows users to scan product information to create a list, receive reminders at the optimal time, and recognize the user's emotions and display appropriate notifications and advertisements based on those emotions, thereby preventing users from forgetting to buy things and providing efficient shopping. The components of this system and their operation are described in detail below.

[1442] List creation function

[1443] 1. The user launches the app and logs in

[1444] The user launches the smartphone app and logs in.

[1445] The device checks the user's authentication information and displays the main screen if the login is successful.

[1446] 2. Access the list creation screen

[1447] The user accesses the "Create a List" section from the app's menu.

[1448] The device displays the list creation screen.

[1449] 3. Enter product information

[1450] The user scans the item using the barcode scanning feature.

[1451] The terminal reads the barcode and sends the information to the server.

[1452] 4. Obtain product information

[1453] The server searches the database for the corresponding product information based on the received barcode information.

[1454] The server returns the product information to the terminal.

[1455] The terminal displays the product information and updates the user's list.

[1456] For example, if a user wants to add milk to the list, they scan the barcode of the milk. The server retrieves the product data from the barcode information and displays it on the terminal. The user confirms and adds it to the list.

[1457] Reminder function

[1458] 1. Data collection and analysis

[1459] The server periodically collects user purchase data and consumption data for each product.

[1460] The server uses AI algorithms to analyze the data and predict the optimal time to replace each product.

[1461] 2. Create and send reminders

[1462] The server creates reminders based on the analysis results.

[1463] The server sends the created reminder to the device at the appropriate time.

[1464] The device displays the reminder to the user as a push notification.

[1465] For example, if a user regularly buys milk, the server will analyze their past consumption patterns and predict when they will need to buy it again, and a reminder will be sent to the user's device at that time.

[1466] Emotion recognition function

[1467] 1. Emotional awareness

[1468] The device uses a camera and microphone to collect emotional information from the user's facial expressions and voice.

[1469] The device uses an emotion engine to analyze the collected information and recognize the user's emotions.

[1470] 2. Emotion-Based Adjustment

[1471] The server uses data from the emotion engine to adjust the timing and content of reminders based on the user's emotions.

[1472] The server also optimizes the content and timing of advertisements.

[1473] For example, if a user is feeling stressed, the server will suppress reminder notifications or change them to more stressful phrases. Advertisements will also be prioritized to display items related to relaxation and activities.

[1474] Monetization methods

[1475] 1. Advertising

[1476] The server manages in-app ad display settings.

[1477] The device displays ads at appropriate times while the user is using the app.

[1478] 2. Premium Features

[1479] The user reviews the premium features and selects a subscription service.

[1480] The terminal transmits the user's payment information to the server.

[1481] The server processes the payment and, if successful, notifies the device to unlock the premium features.

[1482] For example, if a user wants to purchase a premium feature to remove ads, they select the subscription and make the payment. The server verifies the payment and the premium feature is activated.

[1483] As described above, by incorporating an emotion engine, it is possible to provide flexible reminders and advertisement displays that respond to the user's emotions, providing a more personalized shopping experience.

[1484] The processing flow will be explained below.

[1485] List creation function

[1486] Step 1:

[1487] The user launches the smartphone app and logs in.

[1488] Step 2:

[1489] The device checks the user's authentication information and displays the main screen if the login is successful.

[1490] Step 3:

[1491] The user accesses the "Create a List" section from the app's menu.

[1492] Step 4:

[1493] The device displays the list creation screen.

[1494] Step 5:

[1495] The user scans the item using the barcode scanning feature.

[1496] Step 6:

[1497] The device activates the smartphone camera and reads the barcode.

[1498] Step 7:

[1499] The terminal transmits the barcode information to the server.

[1500] Step 8:

[1501] The server searches the database for and retrieves the relevant product information.

[1502] Step 9:

[1503] The server returns product information (product name, category, price) to the terminal.

[1504] Step 10:

[1505] The terminal adds the acquired product information to the user's list and updates the screen.

[1506] Step 11:

[1507] The user confirms the product addition, manually enters the product name if necessary, and adds the item to the list.

[1508] Step 12:

[1509] The user finishes updating the list and taps the save button.

[1510] Step 13:

[1511] The terminal transmits the list information to the server.

[1512] Step 14:

[1513] The server saves the list in a database and sends a message to the terminal indicating that the list has been saved.

[1514] Step 15:

[1515] The device displays a notification to the user that the save is complete.

[1516] Reminder function

[1517] Step 1:

[1518] The server periodically collects user purchase data and consumption data for each product.

[1519] Step 2:

[1520] The server uses AI algorithms to analyze the data and predict the optimal time to replace each product.

[1521] Step 3:

[1522] The server creates reminders based on the analysis results.

[1523] Step 4:

[1524] The server sends the created reminder to the device at the appropriate time.

[1525] Step 5:

[1526] The device displays the reminder to the user as a push notification.

[1527] Step 6:

[1528] The user checks the reminder and checks the shopping list.

[1529] Emotion recognition function

[1530] Step 1:

[1531] The device uses a camera and microphone to collect emotional information from the user's facial expressions and voice.

[1532] Step 2:

[1533] The device uses an emotion engine to analyze the collected information and recognize the user's emotions.

[1534] Step 3:

[1535] The server uses data from the emotion engine to adjust the timing and content of reminders based on the user's emotions.

[1536] Step 4:

[1537] The server also optimizes the content and timing of advertisements.

[1538] Step 5:

[1539] The device displays tailored reminders and advertisements to the user.

[1540] For example, if a user is feeling stressed, the server will suppress reminder notifications or change them to more "menos estresante" language. It also prioritizes advertisements related to relaxation items and activities.

[1541] Monetization methods

[1542] Step 1:

[1543] The server manages in-app ad display settings.

[1544] Step 2:

[1545] The device displays ads at appropriate times while the user is using the app.

[1546] Step 3:

[1547] The user reviews the premium features and selects a subscription service.

[1548] Step 4:

[1549] The user enters payment information to purchase a subscription and taps the "Purchase" button.

[1550] Step 5:

[1551] The terminal sends the payment information to the server.

[1552] Step 6:

[1553] The server processes the payment and, if successful, notifies the device to unlock the premium features.

[1554] Step 7:

[1555] The device will confirm the release of the premium features and notify the user.

[1556] Example 2

[1557] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1558] In today's highly information-driven society, users need tools that allow them to shop efficiently and without missing anything. However, conventional systems are limited to managing product information and reminder functions, and are unable to respond flexibly to user emotions. As a result, users may feel more stressed, and reminders and advertisements may not be as effective as they should be.

[1559] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1560] In this invention, the server includes input means for a user to scan product information, processing means for transmitting the input product information to a database and retrieving corresponding product information, management means for creating and updating the user's list based on the retrieved product information, analysis means for analyzing the user's purchase data and consumption amount and calculating the optimal replacement timing, notification means for notifying the user of a reminder based on the calculated replacement timing, emotion recognition means for recognizing emotions from the user's facial expressions and voice, adaptation means for adjusting the content of reminders and advertisements based on the recognized emotions, and monetization means for displaying advertisements and providing premium features, thereby enabling a flexible and stress-free shopping experience that responds to the user's emotions.

[1561] "Input means" refers to an interface that allows a user to input product information into the system, and specifically includes a barcode scanner, a camera, a touch screen, etc.

[1562] "Processing means" refers to a computer system for receiving input product information and transmitting it to a database to retrieve corresponding product information.

[1563] "Management means" refers to tools and software for creating and updating user lists based on acquired product information.

[1564] "Analysis method" refers to a system that collects user purchasing data and consumption data and uses AI technology to calculate the optimal time to replace items.

[1565] The "notification means" refers to a method for notifying the user of a reminder based on the calculated replacement timing, and mainly includes push notification and email.

[1566] "Emotion recognition means" refers to technology or systems that analyze a user's facial expressions, voice, etc. to recognize the user's emotions.

[1567] "Adaptation" refers to algorithms or systems that tailor reminder or ad content based on perceived emotions.

[1568] "Monetization Method" refers to a method or system for generating revenue through displaying advertisements and providing premium features.

[1569] This invention is a system that allows users to scan product information to create a list, receive reminders at optimal times, and recognize the user's emotions and display optimal notifications and advertisements based on those emotions, thereby preventing users from forgetting to buy items and providing efficient shopping. Specific embodiments of this system are described below.

[1570] First, the user launches the smartphone app and logs in. The device displays the login screen, and the user logs in by entering their email address and password. The device sends the entered authentication information to the server, which then compares it with a database for authentication. If authentication is successful, the server sends a command to the device to display the main screen, and the device displays the main screen.

[1571] Next, the user accesses the "Create a List" section from the app's menu. The device displays the list creation screen, and the user taps the "Add New Item" button. The device then activates its camera and displays the message "Please scan the barcode." When the user holds the product's barcode up to the camera to scan it, the device reads the barcode information and sends it to the server.

[1572] The server searches for the corresponding product information in a database (e.g., a MySQL database) based on the received barcode information. The server returns the searched product information (product name, price, manufacturer, etc.) to the terminal, which displays the product information on the screen and asks the user for confirmation. Once the user confirms, they can tap the "Add to List" button to add the product to the list.

[1573] After the list is created, the server periodically collects the user's purchase data and consumption data for each product. The collected data is analyzed using Python's Pandas and Scikit-learn to train an AI model to predict the optimal replacement timing for each product. Reminders are created based on the prediction results, and the server sends the reminders to the device at the appropriate time. The device then displays the reminders to the user as push notifications.

[1574] The device also uses a camera and microphone to collect the user's facial expressions and voice. The collected data is analyzed using Google Cloud Vision API and Microsoft Azure Cognitive Services to recognize the user's emotions. The server receives the emotion data and understands the user's emotional state.

[1575] Based on the emotion data, the server can adjust the timing and content of reminders. For example, if the user is feeling stressed, the server might tone down reminders or change them to less stressful phrases. Similarly, the server might prioritize ads for products and services related to relaxation.

[1576] Finally, as a means of monetization, the server manages in-app advertising using Google AdMob or Facebook Audience Network. The device displays ads at appropriate times while the user is using the app. Furthermore, the user checks detailed information in the "Premium Features" section of the app, selects a subscription service, and enters payment information. The device sends the payment information to the server, which processes the payment using the Stripe API. If the payment is successful, the server sends a command to unlock the premium features to the device, and the device activates the premium features.

[1577] For example, if a user wants to add milk to a list, they scan the milk's barcode. The server retrieves the product data from the barcode and displays it on the device. The user confirms and adds it to the list. If the user regularly purchases milk, the server analyzes past consumption patterns and predicts when the next purchase will be necessary. A reminder will be sent to the user's device at that time.

[1578] This allows users to enjoy an efficient and personalized shopping experience.

[1579] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1580] List creation function

[1581] Step 1:

[1582] A user launches a smartphone app and accesses the login screen. As input, the user enters an email address and password. Based on this, the device sends the entered authentication information to the server. The server compares it with a database and performs authentication. As output, if authentication is successful, the server sends a command to display the main screen to the device, and the device displays the main screen.

[1583] Step 2:

[1584] The user accesses the "Create a List" section from the app's menu. As input, the user taps the menu icon. Based on this, the device displays the list creation screen.

[1585] Step 3:

[1586] The user taps the "Add New Product" button to activate the camera function. As input, the user holds the product's barcode up to the camera to scan it. Based on this, the device reads the barcode information and sends it to the server.

[1587] Step 4:

[1588] Based on the barcode information received by the server, the server searches for the corresponding product information in a database. A database query is made, receiving the barcode information as input. Based on this, the server retrieves product information (product name, price, manufacturer, etc.) and returns it to the terminal. As output, the server returns the product information to the terminal, which then displays the product information on the screen.

[1589] Step 5:

[1590] The user checks the product information and taps the "Add to List" button to add the product to the list. As input, there is the user's action of tapping the "Add to List" button. Based on this, the device adds the product to the user's list and updates the list. As output, the list is updated.

[1591] Reminder function

[1592] Step 1:

[1593] The server periodically collects user purchase data and consumption data for each product. The inputs are purchase history and consumption data. Based on this, the server uses Python's Pandas library to format the data and create a dataset for the AI ​​model. The formatted dataset is generated as the output.

[1594] Step 2:

[1595] The server uses Scikit-learn to train an AI model to predict the optimal replacement time for each product. The input is a formatted dataset. Based on this, the AI ​​algorithm analyzes the data and generates a prediction. The output is a prediction of the optimal replacement time.

[1596] Step 3:

[1597] The server creates a reminder to notify the next purchase timing based on the prediction results. The input is the predicted optimal replacement time. Based on this, the server generates the reminder text and designs the appropriate timing. The output is a reminder notification.

[1598] Step 4:

[1599] The device displays the reminder to the user as a push notification. As input, there is a reminder notification sent from the server. Based on this, the device displays a push notification to the user. As output, the user receives the reminder.

[1600] Emotion recognition function

[1601] Step 1:

[1602] The device uses a camera and microphone to collect the user's facial expressions and voice. The input is the user's facial expression and voice data. Based on this, the device sends the data to Google Cloud Vision API or Microsoft Azure Cognitive Services for analysis. The output is emotion data.

[1603] Step 2:

[1604] The server receives the emotional data and understands the user's emotional state. The input is the analyzed emotional data. Based on this, the server evaluates the user's emotional state and generates emotional state information. The output is the emotional state information.

[1605] Step 3:

[1606] The server adjusts the timing and content of reminders based on emotional data. The input is emotional state information. Based on this, the server generates and adjusts the content of reminders and advertisements appropriately. The output is the adjusted reminders and advertisements.

[1607] Monetization methods

[1608] Step 1:

[1609] The server manages the ad network settings and sets the timing and content of ad display. The input is the ad network configuration information. Based on this, the server manages ads using Google AdMob or Facebook Audience Network. The output is the ad display settings.

[1610] Step 2:

[1611] The device displays ads at appropriate times while the user is using the app. As input, there are ad display settings received from the server. Based on this, the device displays ads while the user is using the app. As output, the ads are displayed to the user.

[1612] Step 3:

[1613] The user checks the details in the "Premium Features" section and selects a subscription service. As an input, there is an action where the user selects a subscription service. Based on this, the terminal sends payment information to the server. As an output, payment information is sent to the server.

[1614] Step 4:

[1615] The server receives the payment information and processes the payment. As input, it has the user's payment information. Based on this, the server processes the payment using Stripe API and receives the result. As output, it gets the payment success or failure status.

[1616] Step 5:

[1617] If the payment is successful, the server sends a command to unlock the premium feature to the terminal. As input, there is a payment success status. Based on this, the server generates a command to unlock the premium feature and sends it to the terminal. As output, the premium feature becomes active.

[1618] As a result, this system can provide a flexible and personalized shopping experience that responds to the user's emotions.

[1619] (Application example 2)

[1620] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1621] Conventional shopping support systems often have problems such as users forgetting to buy items or not being able to purchase items at the optimal time. Furthermore, because the user experience is not personalized through standard notifications and advertisements, they are unable to provide an efficient and comfortable shopping experience. Furthermore, because they do not take into account the user's emotional state, notifications and advertisements are displayed at inappropriate times, which can cause stress to the user.

[1622] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes an input means for the user to scan product information, a processing means for transmitting the input product information to a database and acquiring corresponding product information, and a management means for creating and updating the user's list based on the acquired product information. This system also includes an analysis means for analyzing the user's purchase data and consumption amount to calculate the optimal replacement timing, a notification means for notifying the user of a reminder based on the calculated replacement timing, an emotion recognition means for recognizing the user's emotions and displaying notifications and advertisements based on the emotions, and a monetization means for displaying advertisements and providing premium features. This not only prevents the user from forgetting to buy something and enables them to purchase products at the optimal time, but also provides personalized notifications and advertisements based on the user's emotions.

[1623] "Input means" is a function that allows the user to scan and input product information.

[1624] The "processing means" is a function for transmitting input product information to a database and retrieving the corresponding product information.

[1625] The "management means" is a function for creating and updating a user list based on the acquired product information.

[1626] The "analysis means" is a function for analyzing the user's purchasing data and consumption amount, and calculating the optimal replacement time.

[1627] The "notification means" is a function for notifying the user of a reminder based on the calculated replacement timing.

[1628] The "emotion recognition means" is a function for recognizing the user's emotions and displaying notifications and advertisements based on those emotions.

[1629] "Monetization means" refers to functions for displaying advertisements and providing premium features.

[1630] This invention is a system that allows users to scan product information and create and update lists. It also recognizes the user's emotions and displays reminders, notifications, and advertisements at optimal times based on those emotions, preventing users from forgetting to buy something and providing an efficient shopping experience.

[1631] System configuration

[1632] The system mainly consists of the following components:

[1633] 1. Input Method

[1634] This is a means for users to scan products using a barcode scanner application installed on a device such as a smartphone.

[1635] 2. Processing Methods

[1636] The entered product information is sent to the database, and the corresponding product information is retrieved. At this time, the server searches the database for the product information and returns it to the user's terminal.

[1637] 3. Control measures

[1638] This function creates and updates the user's list based on the acquired product information. Products scanned by the user are automatically added to the list.

[1639] 4. Analysis tools

[1640] The AI ​​algorithm analyzes user purchasing data and consumption data to calculate the optimal time to replace the product. Here, the AI ​​analysis model is implemented using programming languages ​​such as Python.

[1641] 5. Means of notification

[1642] Based on the calculated replacement timing, a reminder is created and the user is notified at the appropriate time via the smartphone's push notification function.

[1643] 6. Emotion recognition means

[1644] It recognizes users' emotions and adjusts notifications and ads accordingly. Emotion recognition uses the smartphone's camera and microphone to collect emotional data from the user's facial expressions and voice, which is then analyzed by an AI model that includes a pre-trained emotion recognition algorithm.

[1645] 7. Monetization Methods

[1646] It displays advertisements and provides premium features. Users can choose to subscribe to the service to receive premium features such as the ability to hide advertisements.

[1647] Explanation of program processing

[1648] The system uses the following hardware and software:

[1649] Smartphone: Barcode scanning, emotion recognition (camera / microphone), push notification display.

[1650] Server: Database management, data analysis using AI algorithms, and reminder generation.

[1651] AI model: A pre-trained model for recognizing user emotions (e.g., emotion recognition models using Keras or TensorFlow).

[1652] Database: Stores product information and user purchase history data.

[1653] The specific process is as follows:

[1654] 1. The smartphone application scans the product barcode and sends it to the server.

[1655] 2. The server retrieves product information from the database and sends it back to the smartphone.

[1656] 3. Product information will be added and updated to the list.

[1657] 4. An AI algorithm analyzes purchasing history and calculates the optimal time to replace the product.

[1658] 5. Reminders are sent as push notifications at the right time.

[1659] 6. The smartphone's camera and microphone capture the user's facial expressions and voice, which are then analyzed by an emotion recognition model.

[1660] 7. Reminders and ads are tailored based on user emotions.

[1661] Specific examples

[1662] For example, if a user scans a bottle of milk at the supermarket, a product information API is called and detailed information about the product is displayed on the smartphone. Also, if the user is feeling stressed, a push notification will be sent discreetly or a product suggestion will be made to help them relax.

[1663] Prompt Sentence Examples

[1664] Design an application that allows users to scan items in a physical store using a smartphone app and automatically create a shopping list. Use emotion recognition technology to display notifications and ads based on the user's emotions and send reminders at optimal times based on the user's purchasing history. Specific features should include real-time item scanning, emotion recognition, list creation, reminders, and personalized ads.

[1665] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1666] Step 1:

[1667] The user scans the item.

[1668] Input: Scan the product barcode with your smartphone's barcode scanner.

[1669] How it works: A user holds an item up to their smartphone camera to scan the barcode.

[1670] Output: The barcode information is entered into the terminal.

[1671] Step 2:

[1672] The terminal transmits the barcode information to the server.

[1673] Input: Barcode information.

[1674] Operation: The terminal generates a request to send the barcode information to the database API.

[1675] Output: The barcode information sent to the server.

[1676] Step 3:

[1677] The server searches the database for the relevant product information.

[1678] Input: The barcode information sent.

[1679] Operation: The server generates a database query to retrieve product information corresponding to the barcode.

[1680] Output: Product information.

[1681] Step 4:

[1682] The server returns the product information to the terminal.

[1683] Input: Product information retrieved from the database.

[1684] Operation: The server generates a response to return product information to the terminal.

[1685] Output: Product information sent to the device.

[1686] Step 5:

[1687] The device displays product information and updates the list.

[1688] Input: Product information received from the server.

[1689] Action: The device displays product information to the user and adds the product to an existing shopping list.

[1690] Output: The updated shopping list.

[1691] Step 6:

[1692] The server analyzes the purchase data and consumption amounts.

[1693] Input: User purchase history data and consumption data.

[1694] How it works: The server uses AI algorithms to analyze the data and predict the best time to make your next purchase.

[1695] Output: Best time to replace.

[1696] Step 7:

[1697] The server generates reminders and sends them to the device at the appropriate time.

[1698] Input: Optimal replacement time.

[1699] How it works: The server generates a reminder and sends it to the device as a push notification.

[1700] Output: The reminder displayed on the user's device.

[1701] Step 8:

[1702] The device recognizes the user's emotions.

[1703] Input: User facial and voice data collected by the smartphone camera and microphone.

[1704] How it works: The device inputs the collected data into an emotion recognition model to analyze the user's emotions.

[1705] Output: The user's emotional state.

[1706] Step 9:

[1707] The server tailors reminders and advertisements based on the emotional state.

[1708] Input: User emotional state data.

[1709] How it works: The server adjusts the content and timing of notifications based on the user's emotional state, generating personalized ads.

[1710] Output: Tailored reminders and ads.

[1711] Step 10:

[1712] Your device will display tailored reminders and ads.

[1713] Enter: tailored reminders and ads.

[1714] How it works: The device displays reminders and advertisements to the user at appropriate times.

[1715] Output: Personalized reminders and ads shown to the user.

[1716] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1717] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1718] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1719] [Fourth embodiment]

[1720] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1721] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1722] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1723] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1724] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1725] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1726] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1727] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1728] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1729] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1730] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1731] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1732] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1733] This invention is a system that helps users avoid forgetting to buy things and supports efficient shopping by allowing them to create a list by scanning product information and receiving reminders at the appropriate time.The operation of each part of the system is described in detail below.

[1734] List creation function

[1735] 1. The user launches the app and logs in

[1736] The user launches the smartphone app and logs in.

[1737] The terminal checks the user's authentication information and displays the main screen if the login is successful.

[1738] 2. Access the list creation screen

[1739] Users access the "Create a List" section from the app's menu.

[1740] The device displays a list creation screen.

[1741] 3. Enter product information

[1742] The user scans the product using the barcode scanning function.

[1743] The terminal reads the barcode and transmits the information to the server.

[1744] 4. Obtain product information

[1745] The server searches the database for the corresponding product information based on the received barcode information.

[1746] The server returns the product information to the terminal.

[1747] The terminal displays the product information and updates the user's list.

[1748] For example, if a user wants to add milk to the list, they scan the barcode of the milk. The server retrieves the product data from the barcode information and displays it on the terminal. The user confirms and adds it to the list.

[1749] Reminder function

[1750] 1. Data collection and analysis

[1751] The server collects user purchase data and consumption data for each product.

[1752] The server uses AI algorithms to analyze the data and predict the optimal time to replace each product.

[1753] 2. Create and send reminders

[1754] The server creates reminders based on the analysis results.

[1755] The server sends the reminder to the device at the appropriate time.

[1756] The device displays the reminder to the user as a push notification.

[1757] For example, if a user regularly buys milk, the server will analyze their past consumption patterns and predict when they will need to buy it again, and a reminder will be sent to the user's device at that time.

[1758] Monetization methods

[1759] 1. Advertising

[1760] The server manages in-app advertising display settings.

[1761] The device displays advertisements at appropriate times while the user is using the app.

[1762] 2. Premium Features

[1763] The user reviews the details of the premium features and selects a subscription.

[1764] The terminal transmits the user's payment information to the server.

[1765] The server processes the payment and, if successful, notifies the device to unlock the premium features.

[1766] For example, if a user wants to purchase a premium feature to remove ads, they select the subscription and make the payment. The server verifies the payment and the premium feature is activated.

[1767] As described above, this system helps users to shop efficiently and stress-free, and is realized by the cooperation of each function.

[1768] The processing flow will be explained below.

[1769] List creation function

[1770] Step 1:

[1771] The user launches the smartphone app and logs in.

[1772] Step 2:

[1773] The device checks the user's authentication information and displays the main screen if the login is successful.

[1774] Step 3:

[1775] The user accesses the "Create a List" section from the app's menu.

[1776] Step 4:

[1777] The device displays the list creation screen.

[1778] Step 5:

[1779] The user uses the barcode scanning feature to add everyday items to the list.

[1780] Step 6:

[1781] The device activates the smartphone camera and reads the barcode.

[1782] Step 7:

[1783] The terminal transmits the barcode information to the server.

[1784] Step 8:

[1785] The server searches the database for and retrieves the relevant product information.

[1786] Step 9:

[1787] The server returns product information (product name, category, price) to the terminal.

[1788] Step 10:

[1789] The terminal adds the acquired product information to the user's list and updates the screen.

[1790] Step 11:

[1791] The user confirms the product addition, manually enters the product name if necessary, and adds the item to the list.

[1792] Step 12:

[1793] The user finishes updating the list and taps the save button.

[1794] Step 13:

[1795] The terminal transmits the list information to the server.

[1796] Step 14:

[1797] The server saves the list in a database and sends a message to the terminal indicating that the list has been saved.

[1798] Step 15:

[1799] The device displays a notification to the user that the save is complete.

[1800] Reminder function

[1801] Step 1:

[1802] The server periodically collects user purchase data and consumption data for each product.

[1803] Step 2:

[1804] The server uses AI algorithms to analyze the data and predict the optimal time to replace each product.

[1805] Step 3:

[1806] The server creates reminders based on the analysis results.

[1807] Step 4:

[1808] The server sends the created reminder to the device at the appropriate time.

[1809] Step 5:

[1810] The device displays the reminder to the user as a push notification.

[1811] Step 6:

[1812] The user checks the reminder and checks the shopping list.

[1813] Monetization methods

[1814] Step 1:

[1815] The server manages in-app ad display settings.

[1816] Step 2:

[1817] The device displays ads at appropriate times while the user is using the app.

[1818] Step 3:

[1819] The user reviews the premium features and selects a subscription service.

[1820] Step 4:

[1821] The user enters payment information to purchase a subscription and taps the "Purchase" button.

[1822] Step 5:

[1823] The terminal sends the payment information to the server.

[1824] Step 6:

[1825] The server processes the payment and, if successful, notifies the device to unlock the premium features.

[1826] Step 7:

[1827] The device will confirm the release of the premium features and notify the user.

[1828] Example 1

[1829] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1830] In conventional shopping support systems, users had to manually input product information, which was time-consuming, and it was difficult to receive timely reminders based on purchase history and consumption amounts.In addition, advertising and premium features were not provided sufficiently, which led to issues with the convenience and profitability of the entire system.

[1831] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1832] In this invention, the server includes an input means for a user to scan product information, a processing means for transmitting the input product information to a database and retrieving the corresponding product information, a management means for creating and updating the user's list based on the retrieved product information, a data analysis means for analyzing the user's purchase data and consumption amount and calculating the appropriate replacement timing, a notification means for notifying the user of a reminder based on the calculated replacement timing, a monetization means for displaying advertisements and providing premium features, an authentication means for performing user authentication and displaying a main screen upon successful authentication, a notification display means for sending the displayed reminder as a push notification, and a reading means for using a camera to read barcode information for product scanning. This reduces the user's effort, efficiently lists product information, realizes timely reminder notifications, and further strengthens the monetization means.

[1833] "Input means" refers to a device or method by which a user inputs product information into the system.

[1834] The "processing means" refers to a device or method that has the function of transmitting input product information to a database and searching for and retrieving the corresponding product information.

[1835] The "management means" is a device or method that has the function of creating and updating a user list based on the acquired product information.

[1836] The "data analysis means" is a device or method for analyzing a user's purchasing data and consumption amount, and calculating the appropriate time to replace the product.

[1837] The "notification means" is a device or method for notifying the user of a reminder based on the calculated replacement timing.

[1838] A "monetization means" is a device or method that has the function of displaying advertisements and providing premium features.

[1839] "Authentication means" refers to a device or method for authenticating a user and displaying the main screen when authentication is successful.

[1840] A "notification display means" is a device or method for sending a displayed reminder as a push notification.

[1841] "Reading means" refers to a device or method that has the function of reading barcode information using a camera for scanning products.

[1842] This invention is a system that supports efficient shopping by allowing users to scan product information, create a list, and receive reminders at appropriate times to prevent forgetting to buy something. The following components and specific operations are included in the embodiment of the invention.

[1843] System Configuration

[1844] The system allows users to input product information through a smartphone application, and the server processes the information to create shopping lists, send reminder notifications, and generate revenue. The details of each component are shown below.

[1845] Hardware and Software

[1846] Smartphone: A device used by a user that runs applications and uses a camera to scan barcodes.

[1847] Server: Accesses the database and processes product information for management, analysis, notification, and monetization.

[1848] Database: Stores product information and user purchasing data.

[1849] Main features

[1850] 1. Input method:

[1851] The user scans the product information using the barcode scanner function of their smartphone.

[1852] The scanned barcode information is sent to the server via the terminal.

[1853] 2. Processing Method:

[1854] The server searches the database based on the received barcode information and obtains the corresponding product information.

[1855] The acquired product information is returned to the terminal and displayed.

[1856] 3. Control measures:

[1857] A list is created and updated on the user's smartphone based on the returned product information.

[1858] 4. Data analysis methods:

[1859] The server analyzes the user's purchasing data and consumption data, and uses an AI model to calculate the optimal time to replace the device.

[1860] 5. Means of notification:

[1861] Based on the calculated replacement time, the server generates a reminder and sends it to the device at the appropriate time.

[1862] The device displays the reminder to the user as a push notification.

[1863] 6. Monetization methods:

[1864] The server manages in-app advertisements and sets them to be displayed on the device at the appropriate time.

[1865] If the user selects the premium feature, the server processes the payment and unlocks the premium feature.

[1866] 7. Authentication Methods:

[1867] When a user logs in to the application, the server authenticates the user and displays the main screen if the authentication is successful.

[1868] 8. Notification display means:

[1869] The device displays the reminder sent from the server as a push notification.

[1870] 9. Reading Method:

[1871] Product information is read using the smartphone camera and barcode information is obtained.

[1872] Specific examples

[1873] For example, if a user wants to add milk to the list, the system works as follows:

[1874] A user scans the barcode on the milk with their smartphone (User).

[1875] The terminal sends the barcode information to the server (terminal).

[1876] The server retrieves the relevant milk information from the database and returns it to the terminal (server).

[1877] The terminal displays the acquired product information, and the user adds it to the list (terminal).

[1878] Furthermore, if a user regularly purchases milk, the server analyzes the purchase data and predicts when the next replacement purchase is due, then sends a reminder to the user's smartphone as a push notification at the appropriate time.

[1879] Prompt Sentence Examples

[1880] Here are some examples of prompts that specifically use generative AI models:

[1881] "What are the steps to add milk to the list using the system?"

[1882] "Please explain how you can create reminders for items you regularly purchase."

[1883] "How do I upgrade to premium features?"

[1884] In this way, the system of the present invention can support users in shopping efficiently and stress-free.

[1885] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1886] Step 1: Launch the app and log in

[1887] The user launches the smartphone app and the login screen is displayed (input: smartphone app, output: login screen).

[1888] The user enters login information (username and password) and presses the "Login" button (input: username and password, output: login information).

[1889] The terminal sends the entered login information to the server (input: login information, output: request data).

[1890] The server checks the received authentication information, and if it is correct, returns a successful authentication to the terminal (input: request data, output: authentication result).

[1891] The terminal receives the authentication success message and displays the main screen (input: authentication result, output: main screen).

[1892] Step 2: Access the list creation screen

[1893] The user selects "Create a list" from the menu within the app (input: menu selection, output: request to the list creation screen).

[1894] The terminal displays the "Create List" section (Input: Request to create list screen, Output: Create List screen).

[1895] Step 3: Scan the product

[1896] The user presses the "Add product" button to activate the barcode scan function (input: button operation, output: barcode scan screen).

[1897] The device will start the camera and display a view for barcode scanning (input: barcode scanning screen, output: camera view).

[1898] The user scans the product's barcode with a camera (input: barcode, output: barcode data).

[1899] The terminal reads the barcode information and sends it to the server (input: barcode data, output: request to server).

[1900] Step 4: Get product information

[1901] The server searches the database for the corresponding product information based on the received barcode information (input: barcode information, output: product information).

[1902] The server returns the searched product information to the terminal (input: product information, output: response data to the request).

[1903] The terminal displays the product information and updates the user's list (input: response data, output: updated list).

[1904] Step 5: How the Reminders feature works

[1905] The server collects user purchasing data and consumption data for each product (input: purchasing data, output: database dataset).

[1906] The server uses an AI algorithm to analyze the data and predict the optimal replacement time for each product (input: dataset, output: predicted replacement time).

[1907] The server creates a reminder based on the analysis results (input: predicted replacement time, output: reminder).

[1908] The server sends the reminder to the device at the appropriate time (input: reminder, output: request data).

[1909] The device displays the reminder to the user as a push notification (Input: request data, Output: push notification).

[1910] Step 6: Monetization methods in action

[1911] The server manages in-app ad display settings (input: ad data, output: ad settings).

[1912] The device displays ads at appropriate times while the user is using the app (input: ad settings, output: ad display).

[1913] The user reviews the details of the premium features and selects a subscription (Input: Subscription selection, Output: Selection data).

[1914] The terminal sends the user's payment information to the server (input: selection data and payment information, output: payment request).

[1915] The server processes the payment and, if successful, notifies the terminal to unlock the premium features (input: payment request, output: payment result).

[1916] The terminal notifies the user that the premium feature has been enabled (input: payment result, output: notification).

[1917] In this way, the system as a whole allows users to efficiently manage their shopping.

[1918] (Application example 1)

[1919] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1920] Conventional shopping support systems suffer from the cumbersome process of obtaining product information and creating lists, as well as the problem of missing the right timing to purchase. This makes it difficult for users to avoid forgetting to buy something or making unnecessary purchases. Furthermore, they do not adequately provide reminder functions that take individual purchasing patterns into account. Furthermore, they do not adequately provide efficient advertising displays or premium features, which prevents users from increasing satisfaction. To address these issues, a system that innovates and streamlines the user shopping experience is needed.

[1921] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1922] In this invention, the server includes an input means for users to scan product information, a processing means for sending the input product information to a database and retrieving corresponding product information, a management means for creating and updating the user's list based on the retrieved product information, an analysis means for analyzing the user's purchase data and consumption amount and calculating the optimal replacement timing, a notification means for notifying the user of a reminder based on the calculated replacement timing, a monetization means for displaying advertisements and providing premium features, a means for using an application installed on a smartphone, smart glasses, a head-mounted display, or a robot, a means for scanning barcode information with a camera and retrieving product information from the database, a means for accumulating past consumption data and using an AI algorithm to analyze consumption patterns, a means for generating prompt sentences for reminder notifications using a generative AI model, and a means for using the server, Flask, and scikit-learn libraries to create lists and send reminders. This allows users to easily retrieve product information and efficiently create and update lists. Furthermore, by predicting the optimal purchase timing and providing reminders, users can prevent forgetting to buy items and reduce wasteful purchases. Additionally, personalized notifications based on individual purchasing patterns can increase user satisfaction.

[1923] "Input means" refers to a device or interface that provides a function for a user to scan product information.

[1924] The "processing means" is a component that sends input product information to a database and acquires the corresponding product information.

[1925] "Management means" refers to a part of the system that has the function of creating and updating a user's list based on the acquired product information.

[1926] "Analysis means" refers to algorithms or software that analyzes user purchasing data and consumption amounts and calculates the optimal time to replace items.

[1927] The "notification means" is a mechanism for notifying the user of a reminder based on the calculated replacement timing.

[1928] "Monetization Method" means a system or method for displaying advertisements and providing premium features.

[1929] "Means for using an application" refers to the functions and interfaces for using an application installed on a smartphone, smart glasses, head-mounted display, or robot.

[1930] "Means for scanning with a camera" is a function for scanning barcode information with a camera and obtaining product information from a database.

[1931] "Means using AI algorithms" refers to methods that use artificial intelligence algorithms to accumulate past consumption data and analyze consumption patterns.

[1932] "Means for utilizing a generative AI model" refers to a function that uses a generative AI model to generate prompt text for reminder notifications.

[1933] "Methods using a server, Flask, and scikit-learn library" refers to a method of processing using a cloud server and specific program libraries to create lists and send reminders.

[1934] This invention is a system that supports users in shopping efficiently in physical stores. The operation of the system and a specific implementation method are described below.

[1935] System Overview

[1936] The system consists of the following main components:

[1937] Input method: A device such as a smartphone or smart glasses that allows the user to scan product information.

[1938] Processing method: A function that sends scanned barcode information to a database and retrieves the corresponding product information.

[1939] Management method: A system that creates and updates user lists based on acquired product information.

[1940] Analysis method: An artificial intelligence (AI) algorithm that analyzes user purchasing data and consumption to calculate the optimal time to replace items.

[1941] Notifications: A feature that notifies users with reminders based on the calculated replacement period.

[1942] Monetization methods: A system for displaying advertisements and providing premium features.

[1943] Means of using the application: The application is installed on a smartphone, smart glasses, head-mounted display or robot.

[1944] Scan with camera: A function that scans the barcode information of products with a camera and retrieves product information from the database.

[1945] Using AI algorithms: Ability to accumulate past consumption data and use AI algorithms to analyze consumption patterns.

[1946] How to use generative AI models: The ability to use generative AI models to generate prompts for reminder notifications.

[1947] Using a server, Flask, or scikit-learn library: This method uses a cloud server and specific program libraries to create lists and send reminders.

[1948] Program processing

[1949] The server supports an input method that allows users to scan product information. The product information is scanned as barcode information using a smartphone camera and sent to the server. The server then queries a database for the received barcode information and retrieves the corresponding product information. The server then returns the retrieved product information to the terminal, where the user can check the displayed product information to update the list.

[1950] Past consumption data is stored in a cloud environment, and the server analyzes the data using an AI algorithm (e.g., using the scikit-learn library). The server analyzes purchasing patterns and consumption amounts to predict the next purchase date. Based on the calculated replacement date, a generative AI model is used to generate a prompt message.

[1951] The device will then send a reminder to the user based on the generated prompt text, which will be displayed as a push notification to prevent the user from forgetting to make their next purchase.

[1952] The application features advertising and offers the option to disable advertising as a premium feature, which allows users to enjoy more personalized notifications and additional customization features.

[1953] Specific examples

[1954] For example, when a user scans a barcode on a bottle of milk with their smartphone camera, the server receives the barcode information and retrieves the milk's details from the database. The information is then displayed on the device, allowing the user to add the milk to their shopping list. Based on past data, an AI algorithm predicts when the next purchase will be made, and a reminder notification is sent to the user at the appropriate time.

[1955] Prompt Sentence Examples

[1956] User: I scanned the barcode for milk. Add it to the list.

[1957] System: Milk has been added to your list. We'll let you know when it's time for your next purchase.

[1958] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1959] Step 1:

[1960] The user scans the product information.

[1961] Input: Scan the product barcode with your smartphone camera.

[1962] Output: The barcode information read.

[1963] Specific operation: The user launches the smartphone app and uses the barcode scanner function to scan the product barcode with the camera.

[1964] Step 2:

[1965] The terminal transmits the barcode information to the server.

[1966] Input: The barcode information read.

[1967] Output: Barcode information sent to the server.

[1968] Specific operation: The terminal sends the acquired barcode information to the server using an HTTP request, etc.

[1969] Step 3:

[1970] The server retrieves product information from the database.

[1971] Input: The barcode information sent to the server.

[1972] Output: Product information retrieved from the database.

[1973] Specific operation: The server queries the database based on the received barcode information and retrieves related product information.

[1974] Step 4:

[1975] The server returns the acquired product information to the terminal.

[1976] Input: Product information retrieved from the database.

[1977] Output: Product information sent back to the device.

[1978] Specific operation: The server sends the acquired product information to the terminal using an HTTP response, etc.

[1979] Step 5:

[1980] The device displays product information and updates the list.

[1981] Input: Product information received from the server.

[1982] Output: Display of the updated shopping list.

[1983] Specific operation: The terminal displays the received product information on the user interface, and the user confirms and adds the product information to the list.

[1984] Step 6:

[1985] The server analyzes the user's purchasing data.

[1986] Input: User's past purchase and consumption data.

[1987] Output: Prediction of optimal replacement time.

[1988] Specific operation: The server uses an AI algorithm (e.g., the scikit-learn library) to analyze past purchase data and consumption data and calculate the timing of the next purchase.

[1989] Step 7:

[1990] The server generates the reminder prompt.

[1991] Input: Optimal replacement time prediction results.

[1992] Output: The reminder prompt.

[1993] Specific behavior: The server uses the generative AI model to generate prompt text for reminder notifications based on the prediction results.

[1994] Step 8:

[1995] The server sends the reminder to the device.

[1996] Input: The generated prompt statement.

[1997] Output: The reminder notification sent to the device.

[1998] Specific operation: The server sends the generated prompt text to the device in the form of a push notification.

[1999] Step 9:

[2000] The device notifies the user of the reminder.

[2001] Input: Reminder notification sent from the server.

[2002] Output: The reminder notification that is displayed to the user.

[2003] Specific behavior: The device uses the push notification function to display a reminder to the user.

[2004] Example prompt sentence:

[2005] User: I scanned the barcode for milk. Add it to the list.

[2006] System: Milk has been added to your list. We'll let you know when it's time for your next purchase.

[2007] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2008] This invention is a system that allows users to scan product information to create a list, receive reminders at the optimal time, and recognize the user's emotions and display appropriate notifications and advertisements based on those emotions, thereby preventing users from forgetting to buy things and providing efficient shopping. The components of this system and their operation are described in detail below.

[2009] List creation function

[2010] 1. The user launches the app and logs in

[2011] The user launches the smartphone app and logs in.

[2012] The device checks the user's authentication information and displays the main screen if the login is successful.

[2013] 2. Access the list creation screen

[2014] The user accesses the "Create a List" section from the app's menu.

[2015] The device displays the list creation screen.

[2016] 3. Enter product information

[2017] The user scans the item using the barcode scanning feature.

[2018] The terminal reads the barcode and sends the information to the server.

[2019] 4. Obtain product information

[2020] The server searches the database for the corresponding product information based on the received barcode information.

[2021] The server returns the product information to the terminal.

[2022] The terminal displays the product information and updates the user's list.

[2023] For example, if a user wants to add milk to the list, they scan the barcode of the milk. The server retrieves the product data from the barcode information and displays it on the terminal. The user confirms and adds it to the list.

[2024] Reminder function

[2025] 1. Data collection and analysis

[2026] The server periodically collects user purchase data and consumption data for each product.

[2027] The server uses AI algorithms to analyze the data and predict the optimal time to replace each product.

[2028] 2. Create and send reminders

[2029] The server creates reminders based on the analysis results.

[2030] The server sends the created reminder to the device at the appropriate time.

[2031] The device displays the reminder to the user as a push notification.

[2032] For example, if a user regularly buys milk, the server will analyze their past consumption patterns and predict when they will need to buy it again, and a reminder will be sent to the user's device at that time.

[2033] Emotion recognition function

[2034] 1. Emotional awareness

[2035] The device uses a camera and microphone to collect emotional information from the user's facial expressions and voice.

[2036] The device uses an emotion engine to analyze the collected information and recognize the user's emotions.

[2037] 2. Emotion-Based Adjustment

[2038] The server uses data from the emotion engine to adjust the timing and content of reminders based on the user's emotions.

[2039] The server also optimizes the content and timing of advertisements.

[2040] For example, if a user is feeling stressed, the server will suppress reminder notifications or change them to more stressful phrases. Advertisements will also be prioritized to display items related to relaxation and activities.

[2041] Monetization methods

[2042] 1. Advertising

[2043] The server manages in-app ad display settings.

[2044] The device displays ads at appropriate times while the user is using the app.

[2045] 2. Premium Features

[2046] The user reviews the premium features and selects a subscription service.

[2047] The terminal transmits the user's payment information to the server.

[2048] The server processes the payment and, if successful, notifies the device to unlock the premium features.

[2049] For example, if a user wants to purchase a premium feature to remove ads, they select the subscription and make the payment. The server verifies the payment and the premium feature is activated.

[2050] As described above, by incorporating an emotion engine, it is possible to provide flexible reminders and advertisement displays that respond to the user's emotions, providing a more personalized shopping experience.

[2051] The processing flow will be explained below.

[2052] List creation function

[2053] Step 1:

[2054] The user launches the smartphone app and logs in.

[2055] Step 2:

[2056] The device checks the user's authentication information and displays the main screen if the login is successful.

[2057] Step 3:

[2058] The user accesses the "Create a List" section from the app's menu.

[2059] Step 4:

[2060] The device displays the list creation screen.

[2061] Step 5:

[2062] The user scans the item using the barcode scanning feature.

[2063] Step 6:

[2064] The device activates the smartphone camera and reads the barcode.

[2065] Step 7:

[2066] The terminal transmits the barcode information to the server.

[2067] Step 8:

[2068] The server searches the database for and retrieves the relevant product information.

[2069] Step 9:

[2070] The server returns product information (product name, category, price) to the terminal.

[2071] Step 10:

[2072] The terminal adds the acquired product information to the user's list and updates the screen.

[2073] Step 11:

[2074] The user confirms the product addition, manually enters the product name if necessary, and adds the item to the list.

[2075] Step 12:

[2076] The user finishes updating the list and taps the save button.

[2077] Step 13:

[2078] The terminal transmits the list information to the server.

[2079] Step 14:

[2080] The server saves the list in a database and sends a message to the terminal indicating that the list has been saved.

[2081] Step 15:

[2082] The device displays a notification to the user that the save is complete.

[2083] Reminder function

[2084] Step 1:

[2085] The server periodically collects user purchase data and consumption data for each product.

[2086] Step 2:

[2087] The server uses AI algorithms to analyze the data and predict the optimal time to replace each product.

[2088] Step 3:

[2089] The server creates reminders based on the analysis results.

[2090] Step 4:

[2091] The server sends the created reminder to the device at the appropriate time.

[2092] Step 5:

[2093] The device displays the reminder to the user as a push notification.

[2094] Step 6:

[2095] The user checks the reminder and checks the shopping list.

[2096] Emotion recognition function

[2097] Step 1:

[2098] The device uses a camera and microphone to collect emotional information from the user's facial expressions and voice.

[2099] Step 2:

[2100] The device uses an emotion engine to analyze the collected information and recognize the user's emotions.

[2101] Step 3:

[2102] The server uses data from the emotion engine to adjust the timing and content of reminders based on the user's emotions.

[2103] Step 4:

[2104] The server also optimizes the content and timing of advertisements.

[2105] Step 5:

[2106] The device displays tailored reminders and advertisements to the user.

[2107] For example, if a user is feeling stressed, the server will suppress reminder notifications or change them to more "menos estresante" language. It also prioritizes advertisements related to relaxation items and activities.

[2108] Monetization methods

[2109] Step 1:

[2110] The server manages in-app ad display settings.

[2111] Step 2:

[2112] The device displays ads at appropriate times while the user is using the app.

[2113] Step 3:

[2114] The user reviews the premium features and selects a subscription service.

[2115] Step 4:

[2116] The user enters payment information to purchase a subscription and taps the "Purchase" button.

[2117] Step 5:

[2118] The terminal sends the payment information to the server.

[2119] Step 6:

[2120] The server processes the payment and, if successful, notifies the device to unlock the premium features.

[2121] Step 7:

[2122] The device will confirm the release of the premium features and notify the user.

[2123] Example 2

[2124] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2125] In today's highly information-driven society, users need tools that allow them to shop efficiently and without missing anything. However, conventional systems are limited to managing product information and reminder functions, and are unable to respond flexibly to user emotions. As a result, users may feel more stressed, and reminders and advertisements may not be as effective as they should be.

[2126] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2127] In this invention, the server includes input means for a user to scan product information, processing means for transmitting the input product information to a database and retrieving corresponding product information, management means for creating and updating the user's list based on the retrieved product information, analysis means for analyzing the user's purchase data and consumption amount and calculating the optimal replacement timing, notification means for notifying the user of a reminder based on the calculated replacement timing, emotion recognition means for recognizing emotions from the user's facial expressions and voice, adaptation means for adjusting the content of reminders and advertisements based on the recognized emotions, and monetization means for displaying advertisements and providing premium features, thereby enabling a flexible and stress-free shopping experience that responds to the user's emotions.

[2128] "Input means" refers to an interface that allows a user to input product information into the system, and specifically includes a barcode scanner, a camera, a touch screen, etc.

[2129] "Processing means" refers to a computer system for receiving input product information and transmitting it to a database to retrieve corresponding product information.

[2130] "Management means" refers to tools and software for creating and updating user lists based on acquired product information.

[2131] "Analysis method" refers to a system that collects user purchasing data and consumption data and uses AI technology to calculate the optimal time to replace items.

[2132] The "notification means" refers to a method for notifying the user of a reminder based on the calculated replacement timing, and mainly includes push notification and email.

[2133] "Emotion recognition means" refers to technology or systems that analyze a user's facial expressions, voice, etc. to recognize the user's emotions.

[2134] "Adaptation" refers to algorithms or systems that tailor reminder or ad content based on perceived emotions.

[2135] "Monetization Method" refers to a method or system for generating revenue through displaying advertisements and providing premium features.

[2136] This invention is a system that allows users to scan product information to create a list, receive reminders at optimal times, and recognize the user's emotions and display optimal notifications and advertisements based on those emotions, thereby preventing users from forgetting to buy items and providing efficient shopping. Specific embodiments of this system are described below.

[2137] First, the user launches the smartphone app and logs in. The device displays the login screen, and the user logs in by entering their email address and password. The device sends the entered authentication information to the server, which then compares it with a database for authentication. If authentication is successful, the server sends a command to the device to display the main screen, and the device displays the main screen.

[2138] Next, the user accesses the "Create a List" section from the app's menu. The device displays the list creation screen, and the user taps the "Add New Item" button. The device then activates its camera and displays the message "Please scan the barcode." When the user holds the product's barcode up to the camera to scan it, the device reads the barcode information and sends it to the server.

[2139] The server searches for the corresponding product information in a database (e.g., a MySQL database) based on the received barcode information. The server returns the searched product information (product name, price, manufacturer, etc.) to the terminal, which displays the product information on the screen and asks the user for confirmation. Once the user confirms, they can tap the "Add to List" button to add the product to the list.

[2140] After the list is created, the server periodically collects the user's purchase data and consumption data for each product. The collected data is analyzed using Python's Pandas and Scikit-learn to train an AI model to predict the optimal replacement timing for each product. Reminders are created based on the prediction results, and the server sends the reminders to the device at the appropriate time. The device then displays the reminders to the user as push notifications.

[2141] The device also uses a camera and microphone to collect the user's facial expressions and voice. The collected data is analyzed using Google Cloud Vision API and Microsoft Azure Cognitive Services to recognize the user's emotions. The server receives the emotion data and understands the user's emotional state.

[2142] Based on the emotion data, the server can adjust the timing and content of reminders. For example, if the user is feeling stressed, the server might tone down reminders or change them to less stressful phrases. Similarly, the server might prioritize ads for products and services related to relaxation.

[2143] Finally, as a means of monetization, the server manages in-app advertising using Google AdMob or Facebook Audience Network. The device displays ads at appropriate times while the user is using the app. Furthermore, the user checks detailed information in the "Premium Features" section of the app, selects a subscription service, and enters payment information. The device sends the payment information to the server, which processes the payment using the Stripe API. If the payment is successful, the server sends a command to unlock the premium features to the device, and the device activates the premium features.

[2144] For example, if a user wants to add milk to a list, they scan the milk's barcode. The server retrieves the product data from the barcode and displays it on the device. The user confirms and adds it to the list. If the user regularly purchases milk, the server analyzes past consumption patterns and predicts when the next purchase will be necessary. A reminder will be sent to the user's device at that time.

[2145] This allows users to enjoy an efficient and personalized shopping experience.

[2146] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2147] List creation function

[2148] Step 1:

[2149] A user launches a smartphone app and accesses the login screen. As input, the user enters an email address and password. Based on this, the device sends the entered authentication information to the server. The server compares it with a database and performs authentication. As output, if authentication is successful, the server sends a command to display the main screen to the device, and the device displays the main screen.

[2150] Step 2:

[2151] The user accesses the "Create a List" section from the app's menu. As input, the user taps the menu icon. Based on this, the device displays the list creation screen.

[2152] Step 3:

[2153] The user taps the "Add New Product" button to activate the camera function. As input, the user holds the product's barcode up to the camera to scan it. Based on this, the device reads the barcode information and sends it to the server.

[2154] Step 4:

[2155] Based on the barcode information received by the server, the server searches for the corresponding product information in a database. A database query is made, receiving the barcode information as input. Based on this, the server retrieves product information (product name, price, manufacturer, etc.) and returns it to the terminal. As output, the server returns the product information to the terminal, which then displays the product information on the screen.

[2156] Step 5:

[2157] The user checks the product information and taps the "Add to List" button to add the product to the list. As input, there is the user's action of tapping the "Add to List" button. Based on this, the device adds the product to the user's list and updates the list. As output, the list is updated.

[2158] Reminder function

[2159] Step 1:

[2160] The server periodically collects user purchase data and consumption data for each product. The inputs are purchase history and consumption data. Based on this, the server uses Python's Pandas library to format the data and create a dataset for the AI ​​model. The formatted dataset is generated as the output.

[2161] Step 2:

[2162] The server uses Scikit-learn to train an AI model to predict the optimal replacement time for each product. The input is a formatted dataset. Based on this, the AI ​​algorithm analyzes the data and generates a prediction. The output is a prediction of the optimal replacement time.

[2163] Step 3:

[2164] The server creates a reminder to notify the next purchase timing based on the prediction results. The input is the predicted optimal replacement time. Based on this, the server generates the reminder text and designs the appropriate timing. The output is a reminder notification.

[2165] Step 4:

[2166] The device displays the reminder to the user as a push notification. As input, there is a reminder notification sent from the server. Based on this, the device displays a push notification to the user. As output, the user receives the reminder.

[2167] Emotion recognition function

[2168] Step 1:

[2169] The device uses a camera and microphone to collect the user's facial expressions and voice. The input is the user's facial expression and voice data. Based on this, the device sends the data to Google Cloud Vision API or Microsoft Azure Cognitive Services for analysis. The output is emotion data.

[2170] Step 2:

[2171] The server receives the emotional data and understands the user's emotional state. The input is the analyzed emotional data. Based on this, the server evaluates the user's emotional state and generates emotional state information. The output is the emotional state information.

[2172] Step 3:

[2173] The server adjusts the timing and content of reminders based on emotional data. The input is emotional state information. Based on this, the server generates and adjusts the content of reminders and advertisements appropriately. The output is the adjusted reminders and advertisements.

[2174] Monetization methods

[2175] Step 1:

[2176] The server manages the ad network settings and sets the timing and content of ad display. The input is the ad network configuration information. Based on this, the server manages ads using Google AdMob or Facebook Audience Network. The output is the ad display settings.

[2177] Step 2:

[2178] The device displays ads at appropriate times while the user is using the app. As input, there are ad display settings received from the server. Based on this, the device displays ads while the user is using the app. As output, the ads are displayed to the user.

[2179] Step 3:

[2180] The user checks the details in the "Premium Features" section and selects a subscription service. As an input, there is an action where the user selects a subscription service. Based on this, the terminal sends payment information to the server. As an output, payment information is sent to the server.

[2181] Step 4:

[2182] The server receives the payment information and processes the payment. As input, it has the user's payment information. Based on this, the server processes the payment using Stripe API and receives the result. As output, it gets the payment success or failure status.

[2183] Step 5:

[2184] If the payment is successful, the server sends a command to unlock the premium feature to the terminal. As input, there is a payment success status. Based on this, the server generates a command to unlock the premium feature and sends it to the terminal. As output, the premium feature becomes active.

[2185] As a result, this system can provide a flexible and personalized shopping experience that responds to the user's emotions.

[2186] (Application example 2)

[2187] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2188] Conventional shopping support systems often have problems such as users forgetting to buy items or not being able to purchase items at the optimal time. Furthermore, because the user experience is not personalized through standard notifications and advertisements, they are unable to provide an efficient and comfortable shopping experience. Furthermore, because they do not take into account the user's emotional state, notifications and advertisements are displayed at inappropriate times, which can cause stress to the user.

[2189] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes an input means for the user to scan product information, a processing means for transmitting the input product information to a database and acquiring corresponding product information, and a management means for creating and updating the user's list based on the acquired product information. This system also includes an analysis means for analyzing the user's purchase data and consumption amount to calculate the optimal replacement timing, a notification means for notifying the user of a reminder based on the calculated replacement timing, an emotion recognition means for recognizing the user's emotions and displaying notifications and advertisements based on the emotions, and a monetization means for displaying advertisements and providing premium features. This not only prevents the user from forgetting to buy something and enables them to purchase products at the optimal time, but also provides personalized notifications and advertisements based on the user's emotions.

[2190] "Input means" is a function that allows the user to scan and input product information.

[2191] The "processing means" is a function for transmitting input product information to a database and retrieving the corresponding product information.

[2192] The "management means" is a function for creating and updating a user list based on the acquired product information.

[2193] The "analysis means" is a function for analyzing the user's purchasing data and consumption amount, and calculating the optimal replacement time.

[2194] The "notification means" is a function for notifying the user of a reminder based on the calculated replacement timing.

[2195] The "emotion recognition means" is a function for recognizing the user's emotions and displaying notifications and advertisements based on those emotions.

[2196] "Monetization means" refers to functions for displaying advertisements and providing premium features.

[2197] This invention is a system that allows users to scan product information and create and update lists. It also recognizes the user's emotions and displays reminders, notifications, and advertisements at optimal times based on those emotions, preventing users from forgetting to buy something and providing an efficient shopping experience.

[2198] System configuration

[2199] The system mainly consists of the following components:

[2200] 1. Input Method

[2201] This is a means for users to scan products using a barcode scanner application installed on a device such as a smartphone.

[2202] 2. Processing Methods

[2203] The entered product information is sent to the database, and the corresponding product information is retrieved. At this time, the server searches the database for the product information and returns it to the user's terminal.

[2204] 3. Control measures

[2205] This function creates and updates the user's list based on the acquired product information. Products scanned by the user are automatically added to the list.

[2206] 4. Analysis tools

[2207] The AI ​​algorithm analyzes user purchasing data and consumption data to calculate the optimal time to replace the product. Here, the AI ​​analysis model is implemented using programming languages ​​such as Python.

[2208] 5. Means of notification

[2209] Based on the calculated replacement timing, a reminder is created and the user is notified at the appropriate time via the smartphone's push notification function.

[2210] 6. Emotion recognition means

[2211] It recognizes users' emotions and adjusts notifications and ads accordingly. Emotion recognition uses the smartphone's camera and microphone to collect emotional data from the user's facial expressions and voice, which is then analyzed by an AI model that includes a pre-trained emotion recognition algorithm.

[2212] 7. Monetization Methods

[2213] It displays advertisements and provides premium features. Users can choose to subscribe to the service to receive premium features such as the ability to hide advertisements.

[2214] Explanation of program processing

[2215] The system uses the following hardware and software:

[2216] Smartphone: Barcode scanning, emotion recognition (camera / microphone), push notification display.

[2217] Server: Database management, data analysis using AI algorithms, and reminder generation.

[2218] AI model: A pre-trained model for recognizing user emotions (e.g., emotion recognition models using Keras or TensorFlow).

[2219] Database: Stores product information and user purchase history data.

[2220] The specific process is as follows:

[2221] 1. The smartphone application scans the product barcode and sends it to the server.

[2222] 2. The server retrieves product information from the database and sends it back to the smartphone.

[2223] 3. Product information will be added and updated to the list.

[2224] 4. An AI algorithm analyzes purchasing history and calculates the optimal time to replace the product.

[2225] 5. Reminders are sent as push notifications at the right time.

[2226] 6. The smartphone's camera and microphone capture the user's facial expressions and voice, which are then analyzed by an emotion recognition model.

[2227] 7. Reminders and ads are tailored based on user emotions.

[2228] Specific examples

[2229] For example, if a user scans a bottle of milk at the supermarket, a product information API is called and detailed information about the product is displayed on the smartphone. Also, if the user is feeling stressed, a push notification will be sent discreetly or a product suggestion will be made to help them relax.

[2230] Prompt Sentence Examples

[2231] Design an application that allows users to scan items in a physical store using a smartphone app and automatically create a shopping list. Use emotion recognition technology to display notifications and ads based on the user's emotions and send reminders at optimal times based on the user's purchasing history. Specific features should include real-time item scanning, emotion recognition, list creation, reminders, and personalized ads.

[2232] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2233] Step 1:

[2234] The user scans the item.

[2235] Input: Scan the product barcode with your smartphone's barcode scanner.

[2236] How it works: A user holds an item up to their smartphone camera to scan the barcode.

[2237] Output: The barcode information is entered into the terminal.

[2238] Step 2:

[2239] The terminal transmits the barcode information to the server.

[2240] Input: Barcode information.

[2241] Operation: The terminal generates a request to send the barcode information to the database API.

[2242] Output: The barcode information sent to the server.

[2243] Step 3:

[2244] The server searches the database for the relevant product information.

[2245] Input: The barcode information sent.

[2246] Operation: The server generates a database query to retrieve product information corresponding to the barcode.

[2247] Output: Product information.

[2248] Step 4:

[2249] The server returns the product information to the terminal.

[2250] Input: Product information retrieved from the database.

[2251] Operation: The server generates a response to return product information to the terminal.

[2252] Output: Product information sent to the device.

[2253] Step 5:

[2254] The device displays product information and updates the list.

[2255] Input: Product information received from the server.

[2256] Action: The device displays product information to the user and adds the product to an existing shopping list.

[2257] Output: The updated shopping list.

[2258] Step 6:

[2259] The server analyzes the purchase data and consumption amounts.

[2260] Input: User purchase history data and consumption data.

[2261] How it works: The server uses AI algorithms to analyze the data and predict the best time to make your next purchase.

[2262] Output: Best time to replace.

[2263] Step 7:

[2264] The server generates reminders and sends them to the device at the appropriate time.

[2265] Input: Optimal replacement time.

[2266] How it works: The server generates a reminder and sends it to the device as a push notification.

[2267] Output: The reminder displayed on the user's device.

[2268] Step 8:

[2269] The device recognizes the user's emotions.

[2270] Input: User facial and voice data collected by the smartphone camera and microphone.

[2271] How it works: The device inputs the collected data into an emotion recognition model to analyze the user's emotions.

[2272] Output: The user's emotional state.

[2273] Step 9:

[2274] The server tailors reminders and advertisements based on the emotional state.

[2275] Input: User emotional state data.

[2276] How it works: The server adjusts the content and timing of notifications based on the user's emotional state, generating personalized ads.

[2277] Output: Tailored reminders and ads.

[2278] Step 10:

[2279] Your device will display tailored reminders and ads.

[2280] Enter: tailored reminders and ads.

[2281] How it works: The device displays reminders and advertisements to the user at appropriate times.

[2282] Output: Personalized reminders and ads shown to the user.

[2283] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2284] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2285] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2286] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2287] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2288] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2289] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2290] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2291] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2292] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2293] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2294] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2295] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2296] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2297] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2298] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2299] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2300] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2301] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2302] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2303] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2304] The following is further disclosed regarding the above embodiment.

[2305] (Claim 1)

[2306] an input means for a user to scan product information;

[2307] a processing means for transmitting the input product information to a database and acquiring the corresponding product information;

[2308] A management means for creating and updating user lists based on the acquired product information;

[2309] An analytical means for analyzing user purchasing data and consumption to calculate the optimal replacement timing;

[2310] a notification means for notifying a user of a reminder based on the calculated replacement time;

[2311] Monetization methods for displaying advertisements and providing premium features

[2312] Including system.

[2313] (Claim 2)

[2314] 2. The system according to claim 1, further comprising means for utilizing a barcode scanner as means for inputting product information.

[2315] (Claim 3)

[2316] The system according to claim 1, further comprising means for analyzing user purchasing data and consumption amounts using AI.

[2317] "Example 1"

[2318] (Claim 1)

[2319] an input means for a user to scan product information;

[2320] a processing means for transmitting the input product information to a database and acquiring the corresponding product information;

[2321] A management means for creating and updating user lists based on the acquired product information;

[2322] a data analysis means for analyzing user purchase data and consumption amounts and calculating the appropriate time to replace the product;

[2323] a notification means for notifying a user of a reminder based on the calculated replacement time;

[2324] Monetization methods for displaying advertisements and providing premium features;

[2325] an authentication means for performing user authentication and displaying a main screen when authentication is successful;

[2326] a notification display means for sending the displayed reminder as a push notification;

[2327] A reading means for reading barcode information using a camera for scanning products;

[2328] A system including:

[2329] (Claim 2)

[2330] 2. The system according to claim 1, further comprising means for utilizing a barcode scanner as means for inputting product information.

[2331] (Claim 3)

[2332] The system according to claim 1, further comprising means for analyzing user purchasing data and consumption amounts using AI.

[2333] "Application Example 1"

[2334] (Claim 1)

[2335] an input means for a user to scan product information;

[2336] a processing means for transmitting the input product information to a database and acquiring the corresponding product information;

[2337] A management means for creating and updating user lists based on the acquired product information;

[2338] An analytical means for analyzing user purchasing data and consumption to calculate the optimal replacement timing;

[2339] a notification means for notifying a user of a reminder based on the calculated replacement time;

[2340] Monetization methods for displaying advertisements and providing premium features;

[2341] A means for using an application installed on a smartphone, smart glasses, a head-mounted display, or a robot;

[2342] A means for scanning barcode information with a camera and retrieving product information from a database;

[2343] A method for accumulating historical consumption data and using AI algorithms to analyze consumption patterns;

[2344] A means for generating a prompt for a reminder notification using a generative AI model;

[2345] We will introduce a method to use a server, Flask, and scikit-learn library to create lists and send reminders.

[2346] Including system.

[2347] (Claim 2)

[2348] 2. The system according to claim 1, further comprising means for utilizing a barcode scanner as means for inputting product information.

[2349] (Claim 3)

[2350] The system according to claim 1, further comprising means for analyzing user purchasing data and consumption amounts using AI.

[2351] "Example 2: Combining Emotion Engines"

[2352] (Claim 1)

[2353] an input means for a user to scan product information;

[2354] a processing means for transmitting the input product information to a database and acquiring the corresponding product information;

[2355] A management means for creating and updating user lists based on the acquired product information;

[2356] An analytical means for analyzing user purchasing data and consumption to calculate the optimal replacement timing;

[2357] a notification means for notifying a user of a reminder based on the calculated replacement time;

[2358] emotion recognition means for recognizing emotions from facial expressions and voice of a user;

[2359] adaptive means for tailoring reminder and advertising content based on the perceived emotions;

[2360] Monetization methods for displaying advertisements and providing premium features 【236...

Claims

1. an input means for a user to scan product information; a processing means for transmitting the input product information to a database and acquiring the corresponding product information; A management means for creating and updating a user's list based on the acquired product information; An analytical means for analyzing user purchasing data and consumption to calculate the optimal replacement timing; a notification means for notifying a user of a reminder based on the calculated replacement time; Monetization methods for displaying advertisements and providing premium features Including system.

2. 2. The system according to claim 1, further comprising means for utilizing a barcode scanner as the means for inputting product information.

3. The system according to claim 1, further comprising means for analyzing user purchasing data and consumption amounts using AI.

Images