System

Abstract

A system is provided.SOLUTION: A system comprising: means for inputting a place of residence by a consumer; means for inputting a list of products that the consumer wishes to purchase; means for collecting handbill information from a plurality of stores in an area based on the place of residence; means for searching for the cheapest product from the collected handbill information based on the list of products that the consumer wishes to purchase; and means for displaying the search results on a consumer terminal.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Currently, when consumers make their daily food purchases, they face the challenge of having difficulty obtaining a consolidated view of price information from multiple local stores. This prevents them from purchasing products at the best price, increasing the stress of shopping. Additionally, each store lacks the means to set appropriate prices or forecast sales based on consumer purchasing behavior, making it difficult to efficiently manage inventory and improve sales strategies. A solution to these issues is needed. [Means for solving the problem]

[0005] a means for the consumer to input their place of residence;

[0006] a means for consumers to input a list of products they wish to purchase;

[0007] A means for collecting flyer information from a plurality of stores in the area based on the residence;

[0008] A means for searching for the cheapest product from the collected flyer information based on the list of products desired to be purchased;

[0009] means for displaying the search results on a consumer terminal;

[0010] This system allows consumers to efficiently purchase products at the best price. Furthermore, by adding a means for integrating collected flyer information and store price information and storing them in a database, and for analyzing flyer images provided by consumers, extracting price information, and integrating it into the database, it is possible to provide the most up-to-date and accurate price information. Furthermore, by adding a means for searching for related recipes from a list of products desired to be purchased and information on the cheapest products, and displaying the recipes on the consumer's terminal, it is possible to improve consumer convenience.

[0011] "Place of residence" refers to the area or place where a consumer lives on a daily basis, including postal code and city / town level areas.

[0012] A "product list" refers to a list of specific product names or categories that a consumer wishes to purchase.

[0013] "Store" refers to a physical or online point of sale where consumers can actually purchase products, including supermarkets, greengrocers, butchers, fishmongers, drugstores, etc.

[0014] "Flyer information" refers to promotional materials in print or digital media that contain price information and sale information for products offered by a store.

[0015] "Aggregating" refers to the means of gathering data from multiple sources and compiling it into a single data set.

[0016] "Searching" refers to the process of locating relevant information from a database or information list based on specific conditions or keywords.

[0017] "Consumer devices" refers to digital devices used by consumers, such as computers, smartphones, and tablets.

[0018] "Database" refers to a system for efficiently managing and storing multiple data and making them accessible as needed.

[0019] "Analyze" refers to the process of examining and analyzing input information or data using specific algorithms or techniques to extract useful information.

[0020] A "recipe" is a set of instructions for making a dish or product, specifying specific ingredients and steps. [Brief explanation of the drawings]

[0021] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0022] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0023] First, the terms used in the following description will be explained.

[0024] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0025] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0026] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0027] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0028] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0029] [First embodiment]

[0030] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0031] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0032] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0033] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0034] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0035] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0036] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0037] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0038] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0039] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0040] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0041] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0042] This invention is a system that allows consumers to efficiently purchase products at optimal prices in their daily lives. The system allows consumers to input their location and the product they wish to purchase, and uses price information collected from multiple stores in the area to search for and recommend the cheapest products. The system can also analyze the collected data to suggest recipes, forecast sales, and manage inventory.

[0043] Consumer Features

[0044] Residence registration

[0045] The user registers their local area (e.g., Shinjuku-ku, Tokyo). The user enters their area using the smartphone app interface.

[0046] Enter a list of products

[0047] Users register a list of items they want to buy, such as "tomatoes, milk, eggs," through the app.

[0048] Collecting flyer information

[0049] The server collects the latest flyer information from multiple stores in the area based on the registered residential address. This includes web scraping and social media information gathering. The server also analyzes images of paper flyers provided by consumers using OCR technology and integrates price information into the database.

[0050] Search for the lowest price

[0051] The server searches for the cheapest product entered by the user from the flyer information collected by the server. The server sends the search results to the user's terminal and displays them. For example, if the cheapest tomato price is "100 yen per unit at Ito-Yokado," that information is provided to the user.

[0052] Recipe suggestions

[0053] The server searches for relevant recipes based on the shopping list entered by the user and displays them on the user's device. For example, if "tomato, milk, egg" is entered, the server will suggest a recipe for "tomato omelette" and display its ingredients list.

[0054] Retail Features

[0055] Sales forecast

[0056] The server collects and analyzes consumer shopping list data and makes sales forecasts. For example, if many users plan to buy tomatoes, the server predicts that demand for tomatoes will be high and notifies stores.

[0057] Coupon distribution and sale announcements

[0058] The server obtains sales information for nearby stores, generates coupons based on predicted store visits, and provides them to consumers. For example, based on sales information for a 2-liter pack of milk, a 10% off coupon for purchasing milk is generated and notified to the user.

[0059] Inventory management

[0060] The server calculates the appropriate inventory quantity based on the sales forecast and connects to the inventory management system to suggest the appropriate amount to purchase. For example, if the sales forecast for tomatoes is "300 units next Monday," the server will suggest to the store to "purchase an additional 200 tomatoes."

[0061] Specific examples

[0062] As a concrete example, suppose a user uses a smartphone app to register the area where they live and enter "tomatoes, milk, eggs" as part of their weekend shopping list. The server collects the latest flyer information from stores in Shinjuku Ward, searches for the lowest prices, and generates a list such as "tomatoes for 100 yen at Ito-Yokado, milk for 150 yen at Aeon, and eggs for 180 yen at a nearby greengrocer." It also suggests a recipe for "tomato omelette." Based on this data, retailers can distribute sale coupons and manage inventory appropriately.

[0063] In this way, the system provides optimal support for both consumers and retailers.

[0064] The processing flow will be explained below.

[0065] Consumer Features

[0066] Residence registration

[0067] Step 1:

[0068] The user accesses the smartphone app and enters the postal code or city where they live.

[0069] Step 2:

[0070] The user terminal transmits the input information to the server.

[0071] Step 3:

[0072] The server stores the user's location information in a database.

[0073] Enter a list of products

[0074] Step 4:

[0075] A user enters a list of items they want to buy into the app (e.g., "tomatoes, milk, eggs").

[0076] Step 5:

[0077] The user terminal transmits the input product list to the server.

[0078] Collecting flyer information

[0079] Step 6:

[0080] The server collects flyer information from each store's website and social media based on a pre-registered list of local stores.

[0081] Step 7:

[0082] The server uses web scraping technology to extract flyer information and store it in a database.

[0083] Step 8:

[0084] The user takes a photo of a paper flyer and uploads it to the app.

[0085] Step 9:

[0086] The user terminal transmits the uploaded flyer image to the server.

[0087] Step 10:

[0088] The server uses OCR (Optical Character Recognition) technology to extract price information from the image.

[0089] Step 11:

[0090] The server integrates the extracted price information into a database.

[0091] Search for the lowest price

[0092] Step 12:

[0093] The server searches the database for the cheapest products based on the product list entered by the user.

[0094] Step 13:

[0095] The server sends the search results to the user terminal.

[0096] Step 14:

[0097] The user's device displays the lowest price information (e.g., "Tomatoes: Ito-Yokado 100 yen / each").

[0098] Recipe suggestions

[0099] Step 15:

[0100] The server searches a recipe database for relevant recipes based on the user's shopping list.

[0101] Step 16:

[0102] The server sends the found recipe and its ingredient list to the user terminal.

[0103] Step 17:

[0104] The user device displays the recipe information (e.g., "Tomato Omelette: Tomato, Egg, Salt, Pepper").

[0105] Retail Features

[0106] Sales forecast

[0107] Step 18:

[0108] The server collects past consumer shopping list data and begins analyzing it.

[0109] Step 19:

[0110] The server generates a sales forecast based on the analysis results (e.g., "Tomato sales are predicted to increase next Monday").

[0111] Step 20:

[0112] The server transmits the generated sales forecast to the store's management terminal and notifies it.

[0113] Coupon distribution and sale announcements

[0114] Step 21:

[0115] The server periodically collects and analyzes sales information from nearby stores.

[0116] Step 22:

[0117] The server generates appropriate coupons based on predicted store visits (e.g., "10% off coupon when purchasing 2 liters of milk").

[0118] Step 23:

[0119] The server transmits the generated coupon information to the user terminal and performs a push notification.

[0120] Inventory management

[0121] Step 24:

[0122] The server calculates the appropriate inventory quantity based on sales forecasts.

[0123] Step 25:

[0124] The server connects with the inventory management system and suggests appropriate purchase quantities to the store (e.g., "Purchase an additional 200 tomatoes").

[0125] The above processing steps enable consumers to purchase products at the best price and enable retailers to efficiently manage inventory and forecast sales.

[0126] Example 1

[0127] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0128] Today's consumers are required to browse numerous stores and online platforms to find the cheapest prices for products. However, manually collecting and comparing advertising information from individual stores is time-consuming and inefficient. Furthermore, there is a lack of ways to integrate and analyze multiple advertising information, making it difficult for consumers to find the cheapest products. Furthermore, there is a lack of related cooking suggestions, discount information to promote purchases, and inventory optimization in retail stores.

[0129] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0130] In this invention, the server includes a means for inputting a consumer's place of residence, a means for inputting a list of products the consumer wishes to buy, a means for collecting advertising information from multiple stores in the area based on the consumer's place of residence, a means for searching for the cheapest products from the collected advertising information based on the list of products the consumer wishes to buy, a means for displaying the search results on the consumer's terminal, and a means for analyzing the advertising information and integrating the price information into a database. This allows consumers to efficiently find the cheapest products and receive related cooking suggestions and discount information to promote purchases based on the results. It also enables retail stores to optimize inventory based on sales forecasts.

[0131] A "means for inputting a place of residence" is a means for providing an interface for a consumer to input the area in which they live.

[0132] The "means for inputting a list of products to be purchased" is a means for providing an interface for a consumer to input products that he or she intends to purchase.

[0133] The "means for collecting advertising information" is a means for automatically acquiring the latest advertising information from a plurality of local stores based on residential location information.

[0134] The "means for analyzing advertising information" is a means for deciphering the collected advertising information and converting it into usable price data.

[0135] "Means for integrating price information into a database" refers to a means for organizing and storing analyzed price data in a single database.

[0136] The "means for searching for the cheapest product" is a means for searching for the cheapest product among the products input by the user from the collected and analyzed advertising information.

[0137] The "means for displaying search results on a consumer terminal" refers to a means for displaying the searched lowest price product information on the consumer's electronic device.

[0138] The "means for searching for a recipe" is a means for searching for a related recipe based on the product list entered by the user and the searched lowest price product information.

[0139] The "means for providing discount coupons or promotional information for promoting purchases" is a means for the server to notify consumers of discount coupons or special sale information based on predicted data.

[0140] "Means for optimizing inventory" refers to means for calculating and proposing the appropriate inventory amount for a store based on predicted sales data.

[0141] This invention is a system that enables consumers to efficiently purchase products at optimal prices in their daily lives. The system allows consumers to input their location and the product they wish to purchase, and uses advertising information collected from multiple stores in the area to search for and recommend the cheapest products. The system can also analyze the collected data to suggest related recipes, forecast sales, and manage inventory.

[0142] Consumer Features

[0143] Residence registration

[0144] The user registers their local area (e.g., Shinjuku-ku, Tokyo). The user enters their area using the smartphone app interface. This information is sent to the server and stored.

[0145] Enter a list of products

[0146] The user registers a list of items they want to buy. The user inputs items such as "tomatoes, milk, eggs" through the app. This information is also sent to the server and stored.

[0147] Collecting advertising information

[0148] The server collects the latest advertising information from multiple stores in the area based on the registered residential address. This includes web scraping technology, APIs, and social media information collection. The server also analyzes images of print advertisements provided by consumers using OCR (optical character recognition) technology and integrates the price information into a database. This database centrally manages the price information collected from multiple stores.

[0149] Search for the lowest price

[0150] The server searches for the cheapest product entered by the user from the advertising information collected. For example, if the cheapest price for tomatoes is "100 yen per unit at a specific supermarket," that information is sent to the user's device and displayed.

[0151] Recipe suggestions

[0152] The server searches for related recipes based on the shopping list entered by the user and displays them on the user's terminal. For example, if "tomato, milk, egg" is entered, the server will suggest a recipe for "tomato omelette" and display its ingredients list.

[0153] Retail Features

[0154] Sales forecast

[0155] The server collects and analyzes consumer shopping list data and makes sales forecasts. For example, if many users plan to buy tomatoes, the server predicts that demand for tomatoes will be high and notifies stores.

[0156] Coupon distribution and sale announcements

[0157] The server obtains sales information for nearby stores, generates discount coupons based on predicted store visits, and provides them to consumers. For example, based on sales information for a 2-liter pack of milk, a 10% discount coupon for purchasing milk is generated and notified to the user.

[0158] Inventory management

[0159] The server calculates the appropriate inventory quantity based on the sales forecast and connects to the inventory management system to suggest the appropriate amount to purchase. For example, if the sales forecast for tomatoes is "300 units next Monday," the server will suggest to the store to "purchase an additional 200 tomatoes."

[0160] Specific examples

[0161] As a concrete example, suppose a user uses a smartphone app to register the area where they live and enter "tomatoes, milk, eggs" as part of their weekend shopping list. The server collects the latest advertising information from multiple stores in the registered area and searches for the lowest prices. In this example, a list is generated such as "tomatoes for 100 yen at a specific supermarket, milk for 150 yen at another supermarket, and eggs for 180 yen at a local store." In addition, a recipe for "tomato omelette" is suggested. Retailers can use this data to distribute discount coupons and manage inventory appropriately.

[0162] Example prompts for generative AI models

[0163] "I live in a particular area. I want to buy tomatoes, milk, and eggs on the weekend. Which store is the cheapest?"

[0164] "Please tell me some recipes that use tomatoes, milk, and eggs."

[0165] In this way, the system provides optimal support for both consumers and retailers.

[0166] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0167] Step 1: Register your residence

[0168] Input: The user uses a smartphone app to enter the area where they live (e.g., Shinjuku-ku, Tokyo).

[0169] Specific operation: The user enters their place of residence into the form within the app and presses the registration button. The device captures this information and sends it to the server.

[0170] Data processing or data calculation: The server stores the received residence information in a database and associates it with the user profile.

[0171] Output: The server stores the residence information and prepares it for use in the next processing step.

[0172] Step 2: Enter your shopping list

[0173] Input: The user enters a list of items they want to buy (e.g., tomatoes, milk, eggs).

[0174] Specific operation: The user enters a list of items into the shopping list entry screen within the app and presses the submit button. The device acquires this information and sends it to the server.

[0175] Data processing or data calculation: The server stores the received product list in a database and associates it with the user profile.

[0176] Output: The server stores the product listing information and prepares it for use in the next processing step.

[0177] Step 3: Collect advertising information

[0178] Input: Server gets location and product listing information.

[0179] How it works: The server uses web scraping, APIs, and social media information gathering technology to collect the latest advertising information from multiple stores in the area. Images of print advertisements provided by users are analyzed using OCR technology.

[0180] Data processing or data calculation: Analyzing the collected advertising data, extracting the necessary price information and integrating it into the database.

[0181] Output: The server stores the consolidated pricing information data in a database.

[0182] Step 4: Search for the best price

[0183] Input: The server retrieves the consolidated pricing data and the user's shopping list.

[0184] What happens: The server searches the price information in the database to find the cheapest price for the product entered by the user.

[0185] Data processing or data crunching: Using search algorithms to identify the cheapest products and organize their details.

[0186] Output: The server lists the cheapest products and sends them to the user's device.

[0187] Step 5: Recipe suggestions

[0188] Input: The server retrieves the user's shopping list information.

[0189] What happens: The server searches its recipe database to find recipes related to the user's shopping list.

[0190] Data processing or data calculation: Selecting the best recipe from the recipe database and preparing its details.

[0191] Output: The server sends the relevant recipe information to the user's device.

[0192] Step 6: View results and redeem coupons

[0193] Input: Receives the lowest price product information and cooking method information sent to the user's terminal.

[0194] Specific operation: The user browses the lowest price product information and cooking method information on the smartphone app. In addition, the user checks and uses the provided coupon information.

[0195] Data processing or data calculation: verifying the validity of coupon codes and recording their usage in a database.

[0196] Output: The user can efficiently purchase the cheapest products and use coupons.

[0197] As a result, the present invention provides optimal support to both consumers and retailers, and creates an environment in which consumers can shop efficiently.

[0198] (Application example 1)

[0199] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0200] To enable consumers to purchase products efficiently, it is necessary not only to collect product price information and find the cheapest products, but also to suggest recipes, provide navigation to the cheapest stores, and present related coupon information. However, existing systems have difficulty providing all of these in a centralized manner, making it difficult for consumers to make optimal purchases without hassle. Therefore, an objective of the present invention is to provide a comprehensive system that enables consumers to efficiently and optimally conduct purchasing activities.

[0201] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0202] In this invention, the server includes means for a consumer to input their place of residence, means for inputting a list of products the consumer wants to buy, means for collecting price information from a plurality of stores in the area based on the place of residence, means for searching for the cheapest products from the collected price information based on the list of products the consumer wants to buy, means for displaying the search results on the consumer terminal, means for suggesting recipes related to the consumer, and means for providing route guidance to the store with the cheapest products. This enables the consumer to efficiently purchase products at the best price with a single operation, and to obtain related recipe information, route guidance to the store, and even coupon information.

[0203] "Consumer" refers to an individual or household who purchases goods or services.

[0204] "Place of residence" refers to the area where a consumer lives and shops on a daily basis.

[0205] "List of Products" refers to a specific list of products that a consumer wishes to purchase.

[0206] "Store" means a physical location or online sales site where products are sold.

[0207] "Price information" refers to the sales price set by each store for the product.

[0208] "Collect" refers to gathering specified information and integrating it into a single piece of data.

[0209] "Searching" refers to finding the desired information based on input data and conditions.

[0210] "Displaying" refers to outputting information on a terminal in a form that can be viewed by a consumer.

[0211] A "recipe" refers to a document or data that explains how to prepare a dish using specific ingredients.

[0212] "Navigation" refers to providing directions or routes to reach a specified destination.

[0213] "Coupon information" refers to information that allows you to receive discounts on products or services under certain conditions.

[0214] A "generative AI model" refers to an artificial intelligence model that automatically generates new information or suggestions based on given data.

[0215] This system follows the steps below to enable consumers to purchase products efficiently at the best price. First, the user uses a smartphone application to input their address and a list of products they wish to purchase. This address and product list information is then sent to the server.

[0216] The server uses web scraping and OCR (Optical Character Recognition) technologies to collect price information from multiple local stores based on the collected information. Specifically, the requests and BeautifulSoup libraries are used for web scraping, and an appropriate OCR library is used for OCR analysis.

[0217] The collected price information is stored and integrated in a database that includes paper flyer images provided by consumers and data obtained through web scraping, allowing price information from a variety of sources to be managed in a unified manner.

[0218] The server searches the collected price information for the cheapest products based on the product list entered by the consumer and displays the search results on the consumer's smartphone, allowing the consumer to easily decide which store to purchase which products.

[0219] The server then suggests related recipes based on the input product list and the cheapest product information. For example, if a user inputs tomatoes, milk, and eggs, the server will search for related "tomato omelette" recipes and display them on the consumer's smartphone. This recipe suggestion uses a generative AI model to automatically generate the optimal recipe.

[0220] Additionally, the server utilizes a Geographic Information System (GIS) to provide route guidance to the store offering the cheapest product. Specifically, it uses the geopy library and the Nominatim service to generate and present to the consumer a navigation link to the store offering the cheapest product.

[0221] The server also provides coupon information related to the cheapest products, which is retrieved using the coupon service API and notified to the consumer.

[0222] As a concrete example, suppose a user launches a smartphone application and inputs their address ("Shinjuku Ward, Tokyo") and a list of items they wish to purchase ("Tomatoes, Milk, Eggs"). The server collects price information from stores in Shinjuku Ward, searches for the cheapest prices, and generates and presents a list such as "Tomatoes for 100 yen at Store A, Milk for 150 yen at Store B, and Eggs for 180 yen at Store C." It also suggests a recipe for "Tomato Omelette" and provides route guidance links to Stores A and B. The user is then notified of a "10% off coupon that can be used at Store A."

[0223] Example prompts to input to a generative AI model:

[0224] "Suppose a user living in Shinjuku Ward, Tokyo wants to buy tomatoes, milk, and eggs, and we want to collect information on the cheapest prices from local stores. As a result, we want to show the cheapest store and price. We also want to suggest recipes using these ingredients. We also want to provide a navigation link to the cheapest store and coupon information."

[0225] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0226] Step 1:

[0227] The user inputs their address and a list of products they wish to purchase. Using a smartphone application, the user inputs their address as "Shinjuku-ku, Tokyo" and then adds "tomatoes, milk, and eggs" to the list of products they wish to purchase. This information is sent from the device to the server. The input data includes the user's address and product list. The output is the user's address and product list information stored on the server.

[0228] Step 2:

[0229] The server collects price information from multiple stores in the area. Specifically, it uses web scraping technology (using requests and the BeautifulSoup library) to collect price information from store websites. It then uses OCR technology to analyze paper flyer images provided by consumers and extract price information. The input is the URL of the store website and the paper flyer image. The output is a dataset containing the collected price information.

[0230] Step 3:

[0231] The server saves and integrates the collected price information in a database. The collected price information is stored in a database and integrated based on the product list entered by the consumer. This database centralizes store information and its price information. The input is the price information collected in step 2. The output is the integrated database.

[0232] Step 4:

[0233] The server searches the integrated database for the cheapest products. Based on the product list entered by the consumer, the server runs an algorithm to find the cheapest price for each product based on the price information in the database. The input is the consumer's product list and the price information in the database. The output is a list of the cheapest products.

[0234] Step 5:

[0235] The server displays the search results on the consumer's smartphone. The server sends the list of cheapest products obtained in step 4 to the consumer's smartphone and displays it on the application. The input is the list of cheapest products. The output is the displayed information on the cheapest products.

[0236] Step 6:

[0237] The server suggests related recipes based on the input product list and information on the cheapest products. Using a generative AI model, it analyzes the consumer's product list and the ingredient information in the database to generate the optimal recipe. For example, it suggests a recipe for "tomato omelette" for a list of "tomatoes, milk, eggs." The input is the list of cheapest products and the generative AI model. The output is the suggested recipe.

[0238] Step 7:

[0239] The server provides route guidance to the store with the cheapest product. The server uses the geopy library and Nominatim service to calculate the route from the consumer's current location to the store offering the cheapest product and generate a navigation link. The input is the consumer's location and the store's address. The output is the generated navigation link.

[0240] Step 8:

[0241] The server provides coupon information related to the cheapest product. It calls the coupon service API to obtain the coupon related to the cheapest product and notifies the smartphone. The input is the cheapest product and store information. The output is the obtained coupon information.

[0242] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0243] This invention is a system that combines an emotion engine that recognizes the user's emotions to enable consumers to efficiently purchase products at optimal prices in their daily lives. This system allows consumers to input their place of residence and the product they wish to purchase, and uses price information collected from multiple stores in the area to search for and recommend the cheapest products. Furthermore, the emotion engine can analyze the user's emotions and provide personalized recommended products and services based on the results.

[0244] Consumer Features

[0245] Residence registration

[0246] A user accesses the smartphone app and enters the postal code or city where they live. The user's device sends this information to the server, which then stores the residential address information in a database.

[0247] Enter a list of products

[0248] The user enters a list of items they wish to purchase into the app (e.g., "tomatoes, milk, eggs"), and the user's device sends this list to the server.

[0249] Collecting flyer information

[0250] The server collects the latest flyer information from multiple stores in the area based on the registered residential address, including through web scraping and social media. The server also analyzes images of paper flyers provided by consumers using OCR technology and integrates price information into the database.

[0251] Search for the lowest price

[0252] The server searches the database for the lowest priced items based on the product list entered by the user, and sends the search results to the user's terminal, where the lowest price information is displayed.

[0253] Recipe suggestions

[0254] The server searches for relevant recipes from a recipe database based on the shopping list entered by the user and displays them on the user's terminal. For example, if the user enters "tomato, milk, egg," the server will suggest a "tomato omelette" recipe and display its ingredient list.

[0255] Retail Features

[0256] Sales forecast

[0257] The server collects past shopping list data from consumers and begins analysis. Based on the analysis results, it generates sales forecasts and notifies the store's management terminal.

[0258] Coupon distribution and sale announcements

[0259] The server periodically collects and analyzes sales information from nearby stores. It then generates appropriate coupons based on store visit predictions and provides them to consumers. For example, based on sales information for a 2-liter carton of milk, a "10% off coupon when purchasing milk" is generated and notified to the user.

[0260] Inventory management

[0261] The server calculates the appropriate inventory level based on sales forecasts and, in cooperation with the inventory management system, suggests the appropriate amount of stock to the store. For example, if the sales forecast for tomatoes is "300 units next Monday," the server will suggest to the store that they "purchase an additional 200 tomatoes."

[0262] Emotion Engine Functions

[0263] User sentiment analysis

[0264] The emotion engine analyzes the user's emotions using data collected while the user is using the app (e.g., facial recognition data, typing speed and patterns, etc.). The analysis results reflect the user's state in real time and are reflected in the system.

[0265] Providing personalized product and service recommendations

[0266] The server then provides personalized product recommendations and services based on the user's emotions analyzed by the emotion engine. For example, if the user is feeling stressed, the server suggests products and services that have a relaxing effect.

[0267] Providing coupon and sale information at the optimal time

[0268] The server then provides coupon and sale information at the optimal time based on the analysis results of the emotion engine. For example, it executes marketing strategies according to the user's emotional state, such as providing special discount coupons only when the user is in a good mood.

[0269] Specific examples

[0270] As a concrete example, suppose a user uses a smartphone app to register their place of residence and enter "tomatoes, milk, eggs" as part of their weekend shopping list, and the emotion engine detects the user's stress during this process. The server collects the latest flyer information from stores in Shinjuku Ward, searches for the lowest prices, and provides them to the user. At the same time, recipes and products with a relaxing effect are suggested, and special coupons based on sale information are provided to the user. Based on this data, the store can implement appropriate inventory management and sales strategies.

[0271] In this way, the system, combined with the emotion engine, provides more advanced support to both consumers and retailers, improving personalization and efficiency.

[0272] The processing flow will be explained below.

[0273] Consumer Features

[0274] Residence registration

[0275] Step 1:

[0276] The user accesses the smartphone app and enters the postal code or city where they live.

[0277] Step 2:

[0278] The user terminal transmits the entered residence information to the server.

[0279] Step 3:

[0280] The server stores the user's location information in a database.

[0281] Enter a list of products

[0282] Step 4:

[0283] A user enters a list of items they want to buy into the app (e.g., "tomatoes, milk, eggs").

[0284] Step 5:

[0285] The user terminal transmits the input product list to the server.

[0286] Collecting flyer information

[0287] Step 6:

[0288] The server collects flyer information from each store's website and social media based on a pre-registered list of local stores.

[0289] Step 7:

[0290] The server uses web scraping technology to extract flyer information and store it in a database.

[0291] Step 8:

[0292] The user takes a photo of a paper flyer and uploads it to the app.

[0293] Step 9:

[0294] The user terminal transmits the uploaded flyer image to the server.

[0295] Step 10:

[0296] The server uses OCR (Optical Character Recognition) technology to extract price information from the image.

[0297] Step 11:

[0298] The server integrates the extracted price information into a database.

[0299] Search for the lowest price

[0300] Step 12:

[0301] The server searches the database for the cheapest products based on the product list entered by the user.

[0302] Step 13:

[0303] The server sends the search results to the user terminal.

[0304] Step 14:

[0305] The user's device displays the lowest price information (e.g., "Tomatoes: Ito-Yokado 100 yen / each").

[0306] Recipe suggestions

[0307] Step 15:

[0308] The server searches the recipe database for relevant recipes based on the user's shopping list.

[0309] Step 16:

[0310] The server sends the found recipe and its ingredient list to the user terminal.

[0311] Step 17:

[0312] The user device displays the recipe information (e.g., "Tomato Omelette: Tomato, Egg, Salt, Pepper").

[0313] Retail Features

[0314] Sales forecast

[0315] Step 18:

[0316] The server collects past consumer shopping list data and begins analyzing it.

[0317] Step 19:

[0318] The server generates a sales forecast based on the analysis results (e.g., "Tomato sales are predicted to increase next Monday").

[0319] Step 20:

[0320] The server transmits the generated sales forecast to the store's management terminal and notifies it.

[0321] Coupon distribution and sale announcements

[0322] Step 21:

[0323] The server periodically collects and analyzes sales information from nearby stores.

[0324] Step 22:

[0325] The server generates appropriate coupons based on store visit predictions (e.g., "10% off coupon when purchasing 2 liters of milk").

[0326] Step 23:

[0327] The server transmits the generated coupon information to the user terminal and performs a push notification.

[0328] Inventory management

[0329] Step 24:

[0330] The server calculates the appropriate inventory quantity based on the sales forecast.

[0331] Step 25:

[0332] The server connects with the inventory management system and suggests appropriate purchase quantities to the store (e.g., "Purchase an additional 200 tomatoes").

[0333] Emotion Engine Functions

[0334] User sentiment analysis

[0335] Step 26:

[0336] While the user is using the app, the emotion engine collects the user's facial expression data, typing speed and patterns.

[0337] Step 27:

[0338] The user terminal transmits the collected data to the server.

[0339] Step 28:

[0340] The server uses an emotion engine to analyze the user's emotions in real time.

[0341] Providing personalized product and service recommendations

[0342] Step 29:

[0343] The server determines personalized recommended products and services based on the analysis results of the emotion engine.

[0344] Step 30:

[0345] The server sends information about recommended products and services to the user terminal.

[0346] Step 31:

[0347] The user's device displays personalized product and service recommendations (e.g., if the user is feeling stressed, it suggests products that have a relaxing effect).

[0348] Providing coupon and sale information at the optimal time

[0349] Step 32:

[0350] The server generates coupon and sale information at the optimal time based on the analysis results of the emotion engine.

[0351] Step 33:

[0352] The server transmits the generated coupon and sale information to the user terminal.

[0353] Step 34:

[0354] The user's device notifies and displays coupons and sales information (e.g., offering special discount coupons when the user is in a good mood).

[0355] Example 2

[0356] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0357] Conventional price comparison systems simply collect price information from multiple stores and are unable to make suggestions tailored to consumer sentiment or individual needs. Furthermore, they lacked a mechanism for integrating information from flyer images taken by consumers into the system, making it difficult to reflect the latest price information. Furthermore, they lacked the functionality to integrate the collected information into a database and provide related recipes based on the shopping list entered by the consumer.

[0358] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0359] In this invention, the server includes: a means for a consumer to input their place of residence; a means for a consumer to input a list of products they wish to purchase; a means for collecting information from multiple local sales locations; a means for analyzing the collected information and extracting and integrating price information; a means for searching for the cheapest products based on the list of products they wish to purchase; a means for displaying the search results on the consumer's terminal; a means including an engine for analyzing consumer sentiment; and a means for providing personalized recommended products and services based on the analysis results. This enables the provision of personalized services according to the consumer's sentiments and needs, enabling more advanced price comparisons and shopping optimization. Furthermore, the server can analyze flyer images provided by consumers and integrate the latest price information into the database, thereby providing more accurate and up-to-date price information.

[0360] A "consumer" is an individual who purchases goods or services.

[0361] "Residence" is geographic information that indicates where a consumer currently lives.

[0362] "Point of sale" means a store or market that offers goods or services.

[0363] "Information gathering" is the process of gathering data from different sources.

[0364] "Analysis" refers to examining and processing collected information to extract meaningful data.

[0365] A "product list" is a list of products that a consumer wishes to purchase.

[0366] "Price information" is data regarding the selling price of a product or service.

[0367] "Search" is the act of finding information that matches specific conditions from a vast amount of data.

[0368] A "consumer terminal" is a digital device used by a consumer, such as a computer or smartphone.

[0369] An "emotion engine" is a technology for analyzing consumer emotions and their state.

[0370] "Recommended Products" are products that are specifically recommended based on a consumer's needs and preferences.

[0371] "Services" refers to support activities provided to consumers other than the products sold.

[0372] "Integration" refers to bringing together different data into one system or dataset.

[0373] "OCR technology" is a technology that converts scanned documents into machine-readable text data.

[0374] A "flyer image" is an image file of a paper printed for advertising or promotion.

[0375] A "recipe" is a set of instructions that lists the ingredients and steps for making a particular dish.

[0376] "Related recipes" are recipes for dishes related to the user's shopping list.

[0377] This invention is a system that enables consumers to efficiently purchase products at the optimal price. Specifically, the system allows consumers to input their place of residence and the product they wish to purchase, and combines price information collected from multiple local sales locations with an emotion engine to suggest the most suitable product.

[0378] This system mainly involves the exchange of information between the server, the device, and the consumer. The main hardware used to implement the system is the server, the consumer device (smartphone or computer), and network equipment. The software used includes Python, BeautifulSoup, Scrapy, MySQL, Tesseract OCR, OpenCV, analysis tools (pandas, scikit-learn), and web scraping tools.

[0379] First, a user accesses the smartphone app and enters the postal code or city where they live. This information is sent from the device to the server, which stores the residence information in a database. Next, the user enters a list of items they wish to purchase into the app, and the device sends this list to the server.

[0380] The server collects the latest flyer information from multiple local stores based on the registered residential address information. This collection process uses web scraping technology using Python's BeautifulSoup and Scrapy. In addition, images of paper flyers provided by consumers are analyzed using OCR technology (Tesseract OCR), and price information is integrated into the database.

[0381] The server then searches the database for the cheapest products based on the product list entered by the user, using the MySQL database management system, and sends the search results to the consumer's terminal, where the best price information is displayed to the user.

[0382] Furthermore, based on the shopping list entered by the user, the server searches for related recipes from the recipe database and displays them on the user's terminal. For example, if the user enters "tomato, milk, egg" in the list, the server will suggest a recipe for "tomato omelette."

[0383] For retail stores, the server collects shopping list data from consumers and uses analytical tools to predict sales. The analysis results are sent to the store's management terminal, which then implements appropriate inventory management and sales strategies. The server also collects and analyzes sales information from nearby stores, and generates appropriate coupons based on the predictions to provide to consumers.

[0384] Furthermore, the emotion engine analyzes data collected while the user is using the app (such as facial recognition data and input speed) to evaluate the user's emotions. This analysis is performed using OpenCV and other analytical tools. Based on the analysis results, the server provides personalized product and service recommendations according to the user's emotions. For example, if the user is feeling stressed, it will suggest products and services that have a relaxing effect.

[0385] As a concrete example, suppose a user uses a smartphone app to register their place of residence and enter "tomatoes, milk, and eggs" as part of their weekend shopping list, and the emotion engine detects the user's stress during this process. The server collects the latest flyer information from sales locations in Shinjuku Ward, searches for the lowest prices, and provides them to the user. At the same time, recipes and products with a relaxing effect are suggested, and special coupons based on sale information are notified to the user. Based on this data, the sales locations can implement appropriate inventory management and sales strategies.

[0386] In this way, the system, combined with the emotion engine, provides more advanced support to both consumers and points of sale, improving personalization and efficiency.

[0387] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0388] Step 1:

[0389] The user accesses the smartphone app and enters the postal code or city where they live.

[0390] Input: Postal code or city of residence

[0391] Output: Input data is saved on the user's device

[0392] Specific operation: The user enters the postal code and city / town name into the app's residence input form and presses the "Submit" button.

[0393] Step 2:

[0394] The terminal sends this information to the server.

[0395] Input: User-entered residential address information

[0396] Output: Location information is sent to the server

[0397] Specific operation: The device sends data to the server via the Internet.

[0398] Step 3:

[0399] The server stores the residence information in a database.

[0400] Input: Residential information

[0401] Output: Residence information is saved in the database

[0402] Specific operation: The server executes an SQL query to store the received residence information in a database.

[0403] Step 4:

[0404] A user enters a list of items they want to buy into the app (e.g., "tomatoes, milk, eggs").

[0405] Input: List of items you wish to purchase

[0406] Output: Input data is saved on the user's device

[0407] Specific operation: The user enters the product name into the app's shopping list input form and presses the "Submit" button.

[0408] Step 5:

[0409] The terminal sends this list to the server.

[0410] Input: A list of items the user wishes to purchase

[0411] Output: A list of items to purchase is sent to the server

[0412] Specific operation: The device sends data to the server via the Internet.

[0413] Step 6:

[0414] The server collects the latest flyer information from multiple sales locations in the area based on the registered residential address information.

[0415] Input: Residential information

[0416] Output: Collected flyer information

[0417] Specific operation: Based on the specified residence information, the server performs web scraping using Python's BeautifulSoup and Scrapy to collect flyer information from the websites of each sales location.

[0418] Step 7:

[0419] The server receives an image of a paper flyer provided by a consumer.

[0420] Input: Flyer image

[0421] Output: Flyer images are saved on the server

[0422] Specific operation: A user takes a photo of a flyer with their smartphone and uploads the image through the app. The server receives the image.

[0423] Step 8:

[0424] The server analyzes the flyer image using OCR technology (e.g., Tesseract OCR) and integrates the price information into a database.

[0425] Input: Flyer image

[0426] Output: Price information in the database

[0427] Specific operation: The server performs OCR analysis, extracts text data from the flyer image, and parses price information from it. The parsed data is then integrated into a database.

[0428] Step 9:

[0429] The server searches the database for the cheapest products based on the product list entered by the user.

[0430] Input: List of products you wish to purchase, price information

[0431] Output: Information on the cheapest product

[0432] What it does: The server queries the database for pricing information to find the cheapest item. It uses Python data analysis tools to calculate the lowest price.

[0433] Step 10:

[0434] The server sends the search results to the user's terminal and displays the lowest price information.

[0435] Input: Information on the lowest priced item

[0436] Output: The lowest price information displayed on the user's terminal

[0437] Specific operation: The server sends the search results to the user's device, and the app displays them.

[0438] Step 11:

[0439] The server searches for related recipes from a recipe database based on the shopping list entered by the user and displays them on the user terminal.

[0440] Input: List of items you wish to purchase

[0441] Output: Related recipe information

[0442] Specific operation: The server searches the recipe database for relevant recipes based on the shopping list. The searched recipe information is sent to the user's terminal and displayed.

[0443] Step 12:

[0444] The server collects past consumer shopping list data and begins analyzing it.

[0445] Input: Past shopping list data

[0446] Output: Analysis results (sales forecast)

[0447] What it does: The server runs a Python analytics tool to analyze historical consumer shopping list data in the database and generate sales forecasts.

[0448] Step 13:

[0449] The server periodically collects and analyzes sales information from nearby stores.

[0450] Input:Sale information

[0451] Output: Parsed sale information, generated coupons

[0452] What it does: The server periodically runs a web script to collect and analyze sales information from nearby stores.

[0453] Step 14:

[0454] The server generates appropriate coupons based on the store visit prediction and provides them to the consumer terminal.

[0455] Input: store visit prediction, sales information

[0456] Output: Generated coupon

[0457] Specific operation: The server executes an algorithm to generate appropriate coupons based on store visit predictions. The generated coupons are sent to the consumer's terminal and notified.

[0458] Step 15:

[0459] The emotion engine analyzes data collected while users are using the app.

[0460] Input: User operation data (face recognition data, input speed, etc.)

[0461] Output: User sentiment analysis results

[0462] Specific operation: The emotion engine collects user operation data and analyzes the emotional state using OpenCV and data analysis tools.

[0463] Step 16:

[0464] The server provides personalized recommended products and services based on the user's emotions analyzed by the emotion engine.

[0465] Input: Sentiment analysis results

[0466] Output: Personalized product and service recommendations

[0467] Specific operation: The server refers to the sentiment analysis results and executes a personalized recommendation algorithm to suggest appropriate products and services. The suggestions are sent to the user's device and displayed.

[0468] Step 17:

[0469] Based on the analysis results of the emotion engine, the server generates and provides coupon and sale information at the optimal time.

[0470] Input: Sentiment analysis results, sales information

[0471] Output: Generated coupons and sales information

[0472] Specific operation: The server uses the results of the sentiment analysis to run an algorithm that generates coupon and sale information at the optimal time. The generated information is then sent to the user's device and notified.

[0473] (Application example 2)

[0474] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0475] In the past, consumers had to visit multiple stores to find the best price, which was time-consuming and labor-intensive. Furthermore, the means for researching price information were limited, making efficient shopping difficult. Furthermore, personalized product and service recommendations based on the consumer's emotional state were not provided, making it difficult to improve the consumer experience.

[0476] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: a means for a consumer to input their place of residence; a means for inputting a list of products the consumer wants to buy; a means for collecting flyer information from multiple stores in the area based on the place of residence; a means for searching for the cheapest products from the collected flyer information based on the list of products the consumer wants to buy; a means for displaying the search results on the consumer terminal; a means for analyzing the consumer's emotions; a means for providing personalized recommended products and services based on the results of the emotion analysis; and a means for displaying the personalized recommended products and services on the consumer terminal. This enables consumers to efficiently purchase products at optimal prices and receive personalized suggestions based on their emotional state.

[0477] "Means for entering residential address" refers to the interface that allows consumers to register their residential address information in the system. Examples of such interfaces include smart glasses and smartphone apps.

[0478] The "means for inputting a list of products to be purchased" refers to an interface that allows consumers to register a list of products they wish to purchase in the system. This includes a touch screen and voice recognition.

[0479] "Methods for collecting flyer information" are methods for collecting the latest price and sale information from multiple stores in the area. Flyer images are analyzed using web scraping and OCR technology.

[0480] The "means for searching for the cheapest products" refers to algorithms or software that search for the lowest priced products from among the products entered by the consumer based on collected flyer information.

[0481] "Means for displaying search results on consumer devices" refers to an interface for presenting the lowest price information found on the consumer's device (such as smart glasses or a smartphone).

[0482] "Emotion analysis tools" are software or algorithms that analyze a consumer's facial expressions and behavioral data to determine their current emotional state. This may include cameras and sensors.

[0483] "Means for providing personalized product and service recommendations" refers to algorithms and software that suggest the most appropriate products and services based on the consumer's emotional state.

[0484] The "means for displaying personalized recommended products and services on a consumer terminal" is an interface for displaying recommended products and services related to sentiment analysis on a terminal used by a consumer.

[0485] The present invention is a system for supporting consumers in purchasing products efficiently at optimal prices in their daily lives. Specific embodiments of the system will be described below.

[0486] Implementation system configuration

[0487] Consumer Devices

[0488] Consumers use smart glasses or a smartphone app to input their location and a list of items they want to buy. The device has an interface that allows input via touchscreen or voice recognition.

[0489] server

[0490] The server receives information on the consumer's location and product list from the consumer's device and collects the latest flyer information from multiple stores in the area. This information is collected by integrating flyer information from online and paper sources using web scraping and OCR technologies. An algorithm is then run to search for the cheapest product among the items the consumer wants to buy, based on the price information stored in the database.

[0491] Furthermore, the server is equipped with an emotion analysis engine that recognizes emotions in real time by analyzing data such as consumer facial images and input patterns. Based on the results of this emotion analysis, personalized product and service recommendations are generated and displayed on the consumer's device.

[0492] Data processing and calculation

[0493] Sentiment Analysis Engine

[0494] This engine analyzes consumer facial image data collected using cameras and sensors, as well as the speed and patterns of their actions. Major technologies used include OpenCV (for facial recognition) and TensorFlow (for emotion recognition). Consumer emotions (such as stress or happiness) are detected in real time by this engine and provided as data to a server.

[0495] Price information collection and integration

[0496] The server automatically collects price information from websites and paper sources using web scraping and OCR technologies, specifically BeautifulSoup (for web scraping) and Tesseract OCR (for character recognition), and stores it in a database.

[0497] Best price search and recommendations

[0498] The server uses the product list and address information entered by the consumer to screen price information in the database and find the cheapest products. Data processing libraries such as Python's pandas and SQL are used. In addition, by combining consumer sentiment and individual product information, an algorithm is run that individually recommends the best products and services for each specific consumer.

[0499] Specific examples

[0500] Suppose a consumer uses smart glasses to input their place of residence (e.g., "Shinjuku Ward") and a list of products they wish to buy (e.g., "tomatoes, milk, eggs"). During this process, the smart glasses' camera captures the consumer's facial expression and sends the data to a server. If the emotion analysis engine detects stress, the server will suggest products or services with a relaxing effect (e.g., "aromatherapy oil") in addition to the usual lowest price information.

[0501] Prompt Sentence Examples

[0502] "I live in Shinjuku Ward, and I'd like to buy tomatoes, milk, and eggs. Next, I'll provide an image of my facial expression taken by the camera on my smart glasses. Based on this information, please tell me the lowest prices in the area and recommend products based on my emotions."

[0503] In this way, the embodiment of the invention realizes a system that reflects the consumer's input information and emotional state, collects and analyzes optimal price information from multiple stores, and provides personalized recommended products in real time.

[0504] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0505] Step 1:

[0506] A user uses a consumer device (smart glasses or a smartphone app) to input their location and a list of products they wish to purchase. The input information is then sent from the device to a server. Specifically, the user uses the device's interface to input information using a touch screen or voice recognition. The input data is in the form of text, which is then sent to the server.

[0507] Step 2:

[0508] Based on the received residential address information, the server collects the latest flyer and price information from multiple stores in the area. This collection is performed using web scraping and OCR technologies. BeautifulSoup is used for web scraping to extract price information from online store websites. Tesseract OCR is also used to recognize and extract characters from paper flyer images. The collected data is then stored in a database.

[0509] Step 3:

[0510] The server searches for the cheapest products that match the product list entered by the user based on the price information stored in the database. This process involves filtering the data using Python's pandas library and executing database queries using SQL. The search results are returned in text format as the lowest price information for the target products and are sent to the consumer's device.

[0511] Step 4:

[0512] The consumer device displays the lowest price information sent from the server to the user. In the case of smart glasses, the price information is displayed in the user's field of view using holographic display technology. In the case of smartphones, the price information is displayed in text and graphical format on the screen, allowing the user to visually check the cheapest product information.

[0513] Step 5:

[0514] Consumer emotion data is collected in real time by the camera and sensors on the consumer device. Specifically, facial image data captured by the camera is processed within the device, and the results are sent to a server. This data is analyzed using an emotion recognition model that combines OpenCV and TensorFlow on the edge device. The analysis results are sent to the server as text data.

[0515] Step 6:

[0516] The server receives the results of the emotion analysis and generates personalized product and service recommendations based on them. If the emotion indicates stress, products and services with a relaxing effect will be recommended. This recommendation process is performed by an algorithm using a generative AI model, and the recommendation results are obtained as text data.

[0517] Step 7:

[0518] The server sends the generated information about recommended products and services to the consumer's device, which then displays the received information to the user. In the case of smart glasses, the recommended product information is displayed in the user's field of vision using holographic display technology. In the case of smartphones, the recommended information is displayed in text and graphical format on the screen, allowing the user to learn about the best products and services based on their emotional state.

[0519] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0520] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0521] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0522] [Second embodiment]

[0523] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0524] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0525] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0526] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0527] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0528] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0529] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0530] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0531] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0532] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0533] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0534] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0535] This invention is a system that allows consumers to efficiently purchase products at optimal prices in their daily lives. The system allows consumers to input their location and the product they wish to purchase, and uses price information collected from multiple stores in the area to search for and recommend the cheapest products. The system can also analyze the collected data to suggest recipes, forecast sales, and manage inventory.

[0536] Consumer Features

[0537] Residence registration

[0538] The user registers their local area (e.g., Shinjuku-ku, Tokyo). The user enters their area using the smartphone app interface.

[0539] Enter a list of products

[0540] Users register a list of items they want to buy, such as "tomatoes, milk, eggs," through the app.

[0541] Collecting flyer information

[0542] The server collects the latest flyer information from multiple stores in the area based on the registered residential address. This includes web scraping and social media information gathering. The server also analyzes images of paper flyers provided by consumers using OCR technology and integrates price information into the database.

[0543] Search for the lowest price

[0544] The server searches for the cheapest product entered by the user from the flyer information collected by the server. The server sends the search results to the user's terminal and displays them. For example, if the cheapest tomato price is "100 yen per unit at Ito-Yokado," that information is provided to the user.

[0545] Recipe suggestions

[0546] The server searches for relevant recipes based on the shopping list entered by the user and displays them on the user's device. For example, if "tomato, milk, egg" is entered, the server will suggest a recipe for "tomato omelette" and display its ingredients list.

[0547] Retail Features

[0548] Sales forecast

[0549] The server collects and analyzes consumer shopping list data and makes sales forecasts. For example, if many users plan to buy tomatoes, the server predicts that demand for tomatoes will be high and notifies stores.

[0550] Coupon distribution and sale announcements

[0551] The server obtains sales information for nearby stores, generates coupons based on predicted store visits, and provides them to consumers. For example, based on sales information for a 2-liter pack of milk, a 10% off coupon for purchasing milk is generated and notified to the user.

[0552] Inventory management

[0553] The server calculates the appropriate inventory quantity based on the sales forecast and connects to the inventory management system to suggest the appropriate amount to purchase. For example, if the sales forecast for tomatoes is "300 units next Monday," the server will suggest to the store to "purchase an additional 200 tomatoes."

[0554] Specific examples

[0555] As a concrete example, suppose a user uses a smartphone app to register the area where they live and enter "tomatoes, milk, eggs" as part of their weekend shopping list. The server collects the latest flyer information from stores in Shinjuku Ward, searches for the lowest prices, and generates a list such as "tomatoes for 100 yen at Ito-Yokado, milk for 150 yen at Aeon, and eggs for 180 yen at a nearby greengrocer." It also suggests a recipe for "tomato omelette." Based on this data, retailers can distribute sale coupons and manage inventory appropriately.

[0556] In this way, the system provides optimal support for both consumers and retailers.

[0557] The processing flow will be explained below.

[0558] Consumer Features

[0559] Residence registration

[0560] Step 1:

[0561] The user accesses the smartphone app and enters the postal code or city where they live.

[0562] Step 2:

[0563] The user terminal transmits the input information to the server.

[0564] Step 3:

[0565] The server stores the user's location information in a database.

[0566] Enter a list of products

[0567] Step 4:

[0568] A user enters a list of items they want to buy into the app (e.g., "tomatoes, milk, eggs").

[0569] Step 5:

[0570] The user terminal transmits the input product list to the server.

[0571] Collecting flyer information

[0572] Step 6:

[0573] The server collects flyer information from each store's website and social media based on a pre-registered list of local stores.

[0574] Step 7:

[0575] The server uses web scraping technology to extract flyer information and store it in a database.

[0576] Step 8:

[0577] The user takes a photo of a paper flyer and uploads it to the app.

[0578] Step 9:

[0579] The user terminal transmits the uploaded flyer image to the server.

[0580] Step 10:

[0581] The server uses OCR (Optical Character Recognition) technology to extract price information from the image.

[0582] Step 11:

[0583] The server integrates the extracted price information into a database.

[0584] Search for the lowest price

[0585] Step 12:

[0586] The server searches the database for the cheapest products based on the product list entered by the user.

[0587] Step 13:

[0588] The server sends the search results to the user terminal.

[0589] Step 14:

[0590] The user's device displays the lowest price information (e.g., "Tomatoes: Ito-Yokado 100 yen / each").

[0591] Recipe suggestions

[0592] Step 15:

[0593] The server searches a recipe database for relevant recipes based on the user's shopping list.

[0594] Step 16:

[0595] The server sends the found recipe and its ingredient list to the user terminal.

[0596] Step 17:

[0597] The user device displays the recipe information (e.g., "Tomato Omelette: Tomato, Egg, Salt, Pepper").

[0598] Retail Features

[0599] Sales forecast

[0600] Step 18:

[0601] The server collects past consumer shopping list data and begins analyzing it.

[0602] Step 19:

[0603] The server generates a sales forecast based on the analysis results (e.g., "Tomato sales are predicted to increase next Monday").

[0604] Step 20:

[0605] The server transmits the generated sales forecast to the store's management terminal and notifies it.

[0606] Coupon distribution and sale announcements

[0607] Step 21:

[0608] The server periodically collects and analyzes sales information from nearby stores.

[0609] Step 22:

[0610] The server generates appropriate coupons based on predicted store visits (e.g., "10% off coupon when purchasing 2 liters of milk").

[0611] Step 23:

[0612] The server transmits the generated coupon information to the user terminal and performs a push notification.

[0613] Inventory management

[0614] Step 24:

[0615] The server calculates the appropriate inventory quantity based on sales forecasts.

[0616] Step 25:

[0617] The server connects with the inventory management system and suggests appropriate purchase quantities to the store (e.g., "Purchase an additional 200 tomatoes").

[0618] The above processing steps enable consumers to purchase products at the best price and enable retailers to efficiently manage inventory and forecast sales.

[0619] Example 1

[0620] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0621] Today's consumers are required to browse numerous stores and online platforms to find the cheapest prices for products. However, manually collecting and comparing advertising information from individual stores is time-consuming and inefficient. Furthermore, there is a lack of ways to integrate and analyze multiple advertising information, making it difficult for consumers to find the cheapest products. Furthermore, there is a lack of related cooking suggestions, discount information to promote purchases, and inventory optimization in retail stores.

[0622] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0623] In this invention, the server includes a means for inputting a consumer's place of residence, a means for inputting a list of products the consumer wishes to buy, a means for collecting advertising information from multiple stores in the area based on the consumer's place of residence, a means for searching for the cheapest products from the collected advertising information based on the list of products the consumer wishes to buy, a means for displaying the search results on the consumer's terminal, and a means for analyzing the advertising information and integrating the price information into a database. This allows consumers to efficiently find the cheapest products and receive related cooking suggestions and discount information to promote purchases based on the results. It also enables retail stores to optimize inventory based on sales forecasts.

[0624] A "means for inputting a place of residence" is a means for providing an interface for a consumer to input the area in which they live.

[0625] The "means for inputting a list of products to be purchased" is a means for providing an interface for a consumer to input products that he or she intends to purchase.

[0626] The "means for collecting advertising information" is a means for automatically acquiring the latest advertising information from a plurality of local stores based on residential location information.

[0627] The "means for analyzing advertising information" is a means for deciphering the collected advertising information and converting it into usable price data.

[0628] "Means for integrating price information into a database" refers to a means for organizing and storing analyzed price data in a single database.

[0629] The "means for searching for the cheapest product" is a means for searching for the cheapest product among the products input by the user from the collected and analyzed advertising information.

[0630] The "means for displaying search results on a consumer terminal" refers to a means for displaying the searched lowest price product information on the consumer's electronic device.

[0631] The "means for searching for a recipe" is a means for searching for a related recipe based on the product list entered by the user and the searched lowest price product information.

[0632] The "means for providing discount coupons or promotional information for promoting purchases" is a means for the server to notify consumers of discount coupons or special sale information based on predicted data.

[0633] "Means for optimizing inventory" refers to means for calculating and proposing the appropriate inventory amount for a store based on predicted sales data.

[0634] This invention is a system that enables consumers to efficiently purchase products at optimal prices in their daily lives. The system allows consumers to input their location and the product they wish to purchase, and uses advertising information collected from multiple stores in the area to search for and recommend the cheapest products. The system can also analyze the collected data to suggest related recipes, forecast sales, and manage inventory.

[0635] Consumer Features

[0636] Residence registration

[0637] The user registers their local area (e.g., Shinjuku-ku, Tokyo). The user enters their area using the smartphone app interface. This information is sent to the server and stored.

[0638] Enter a list of products

[0639] The user registers a list of items they want to buy. The user inputs items such as "tomatoes, milk, eggs" through the app. This information is also sent to the server and stored.

[0640] Collecting advertising information

[0641] The server collects the latest advertising information from multiple stores in the area based on the registered residential address. This includes web scraping technology, APIs, and social media information collection. The server also analyzes images of print advertisements provided by consumers using OCR (optical character recognition) technology and integrates the price information into a database. This database centrally manages the price information collected from multiple stores.

[0642] Search for the lowest price

[0643] The server searches for the cheapest product entered by the user from the advertising information collected. For example, if the cheapest price for tomatoes is "100 yen per unit at a specific supermarket," that information is sent to the user's device and displayed.

[0644] Recipe suggestions

[0645] The server searches for related recipes based on the shopping list entered by the user and displays them on the user's terminal. For example, if "tomato, milk, egg" is entered, the server will suggest a recipe for "tomato omelette" and display its ingredients list.

[0646] Retail Features

[0647] Sales forecast

[0648] The server collects and analyzes consumer shopping list data and makes sales forecasts. For example, if many users plan to buy tomatoes, the server predicts that demand for tomatoes will be high and notifies stores.

[0649] Coupon distribution and sale announcements

[0650] The server obtains sales information for nearby stores, generates discount coupons based on predicted store visits, and provides them to consumers. For example, based on sales information for a 2-liter pack of milk, a 10% discount coupon for purchasing milk is generated and notified to the user.

[0651] Inventory management

[0652] The server calculates the appropriate inventory quantity based on the sales forecast and connects to the inventory management system to suggest the appropriate amount to purchase. For example, if the sales forecast for tomatoes is "300 units next Monday," the server will suggest to the store to "purchase an additional 200 tomatoes."

[0653] Specific examples

[0654] As a concrete example, suppose a user uses a smartphone app to register the area where they live and enter "tomatoes, milk, eggs" as part of their weekend shopping list. The server collects the latest advertising information from multiple stores in the registered area and searches for the lowest prices. In this example, a list is generated such as "tomatoes for 100 yen at a specific supermarket, milk for 150 yen at another supermarket, and eggs for 180 yen at a local store." In addition, a recipe for "tomato omelette" is suggested. Retailers can use this data to distribute discount coupons and manage inventory appropriately.

[0655] Example prompts for generative AI models

[0656] "I live in a particular area. I want to buy tomatoes, milk, and eggs on the weekend. Which store is the cheapest?"

[0657] "Please tell me some recipes that use tomatoes, milk, and eggs."

[0658] In this way, the system provides optimal support for both consumers and retailers.

[0659] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0660] Step 1: Register your residence

[0661] Input: The user uses a smartphone app to enter the area where they live (e.g., Shinjuku-ku, Tokyo).

[0662] Specific operation: The user enters their place of residence into the form within the app and presses the registration button. The device captures this information and sends it to the server.

[0663] Data processing or data calculation: The server stores the received residence information in a database and associates it with the user profile.

[0664] Output: The server stores the residence information and prepares it for use in the next processing step.

[0665] Step 2: Enter your shopping list

[0666] Input: The user enters a list of items they want to buy (e.g., tomatoes, milk, eggs).

[0667] Specific operation: The user enters a list of items into the shopping list entry screen within the app and presses the submit button. The device acquires this information and sends it to the server.

[0668] Data processing or data calculation: The server stores the received product list in a database and associates it with the user profile.

[0669] Output: The server stores the product listing information and prepares it for use in the next processing step.

[0670] Step 3: Collect advertising information

[0671] Input: Server gets location and product listing information.

[0672] How it works: The server uses web scraping, APIs, and social media information gathering technology to collect the latest advertising information from multiple stores in the area. Images of print advertisements provided by users are analyzed using OCR technology.

[0673] Data processing or data calculation: Analyzing the collected advertising data, extracting the necessary price information and integrating it into the database.

[0674] Output: The server stores the consolidated pricing information data in a database.

[0675] Step 4: Search for the best price

[0676] Input: The server retrieves the consolidated pricing data and the user's shopping list.

[0677] What happens: The server searches the price information in the database to find the cheapest price for the product entered by the user.

[0678] Data processing or data crunching: Using search algorithms to identify the cheapest products and organize their details.

[0679] Output: The server lists the cheapest products and sends them to the user's device.

[0680] Step 5: Recipe suggestions

[0681] Input: The server retrieves the user's shopping list information.

[0682] What happens: The server searches its recipe database to find recipes related to the user's shopping list.

[0683] Data processing or data calculation: Selecting the best recipe from the recipe database and preparing its details.

[0684] Output: The server sends the relevant recipe information to the user's device.

[0685] Step 6: View results and redeem coupons

[0686] Input: Receives the lowest price product information and cooking method information sent to the user's terminal.

[0687] Specific operation: The user browses the lowest price product information and cooking method information on the smartphone app. In addition, the user checks and uses the provided coupon information.

[0688] Data processing or data calculation: verifying the validity of coupon codes and recording their usage in a database.

[0689] Output: The user can efficiently purchase the cheapest products and use coupons.

[0690] As a result, the present invention provides optimal support to both consumers and retailers, and creates an environment in which consumers can shop efficiently.

[0691] (Application example 1)

[0692] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0693] To enable consumers to purchase products efficiently, it is necessary not only to collect product price information and find the cheapest products, but also to suggest recipes, provide navigation to the cheapest stores, and present related coupon information. However, existing systems have difficulty providing all of these in a centralized manner, making it difficult for consumers to make optimal purchases without hassle. Therefore, an objective of the present invention is to provide a comprehensive system that enables consumers to efficiently and optimally conduct purchasing activities.

[0694] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0695] In this invention, the server includes means for a consumer to input their place of residence, means for inputting a list of products the consumer wants to buy, means for collecting price information from a plurality of stores in the area based on the place of residence, means for searching for the cheapest products from the collected price information based on the list of products the consumer wants to buy, means for displaying the search results on the consumer terminal, means for suggesting recipes related to the consumer, and means for providing route guidance to the store with the cheapest products. This enables the consumer to efficiently purchase products at the best price with a single operation, and to obtain related recipe information, route guidance to the store, and even coupon information.

[0696] "Consumer" refers to an individual or household who purchases goods or services.

[0697] "Place of residence" refers to the area where a consumer lives and shops on a daily basis.

[0698] "List of Products" refers to a specific list of products that a consumer wishes to purchase.

[0699] "Store" means a physical location or online sales site where products are sold.

[0700] "Price information" refers to the sales price set by each store for the product.

[0701] "Collect" refers to gathering specified information and integrating it into a single piece of data.

[0702] "Searching" refers to finding the desired information based on input data and conditions.

[0703] "Displaying" refers to outputting information on a terminal in a form that can be viewed by a consumer.

[0704] A "recipe" refers to a document or data that explains how to prepare a dish using specific ingredients.

[0705] "Navigation" refers to providing directions or routes to reach a specified destination.

[0706] "Coupon information" refers to information that allows you to receive discounts on products or services under certain conditions.

[0707] A "generative AI model" refers to an artificial intelligence model that automatically generates new information or suggestions based on given data.

[0708] This system follows the steps below to enable consumers to purchase products efficiently at the best price. First, the user uses a smartphone application to input their address and a list of products they wish to purchase. This address and product list information is then sent to the server.

[0709] The server uses web scraping and OCR (Optical Character Recognition) technologies to collect price information from multiple local stores based on the collected information. Specifically, the requests and BeautifulSoup libraries are used for web scraping, and an appropriate OCR library is used for OCR analysis.

[0710] The collected price information is stored and integrated in a database that includes paper flyer images provided by consumers and data obtained through web scraping, allowing price information from a variety of sources to be managed in a unified manner.

[0711] The server searches the collected price information for the cheapest products based on the product list entered by the consumer and displays the search results on the consumer's smartphone, allowing the consumer to easily decide which store to purchase which products.

[0712] The server then suggests related recipes based on the input product list and the cheapest product information. For example, if a user inputs tomatoes, milk, and eggs, the server will search for related "tomato omelette" recipes and display them on the consumer's smartphone. This recipe suggestion uses a generative AI model to automatically generate the optimal recipe.

[0713] Additionally, the server utilizes a Geographic Information System (GIS) to provide route guidance to the store offering the cheapest product. Specifically, it uses the geopy library and the Nominatim service to generate and present to the consumer a navigation link to the store offering the cheapest product.

[0714] The server also provides coupon information related to the cheapest products, which is retrieved using the coupon service API and notified to the consumer.

[0715] As a concrete example, suppose a user launches a smartphone application and inputs their address ("Shinjuku Ward, Tokyo") and a list of items they wish to purchase ("Tomatoes, Milk, Eggs"). The server collects price information from stores in Shinjuku Ward, searches for the cheapest prices, and generates and presents a list such as "Tomatoes for 100 yen at Store A, Milk for 150 yen at Store B, and Eggs for 180 yen at Store C." It also suggests a recipe for "Tomato Omelette" and provides route guidance links to Stores A and B. The user is then notified of a "10% off coupon that can be used at Store A."

[0716] Example prompts to input to a generative AI model:

[0717] "Suppose a user living in Shinjuku Ward, Tokyo wants to buy tomatoes, milk, and eggs, and we want to collect information on the cheapest prices from local stores. As a result, we want to show the cheapest store and price. We also want to suggest recipes using these ingredients. We also want to provide a navigation link to the cheapest store and coupon information."

[0718] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0719] Step 1:

[0720] The user inputs their address and a list of products they wish to purchase. Using a smartphone application, the user inputs their address as "Shinjuku-ku, Tokyo" and then adds "tomatoes, milk, and eggs" to the list of products they wish to purchase. This information is sent from the device to the server. The input data includes the user's address and product list. The output is the user's address and product list information stored on the server.

[0721] Step 2:

[0722] The server collects price information from multiple stores in the area. Specifically, it uses web scraping technology (using requests and the BeautifulSoup library) to collect price information from store websites. It then uses OCR technology to analyze paper flyer images provided by consumers and extract price information. The input is the URL of the store website and the paper flyer image. The output is a dataset containing the collected price information.

[0723] Step 3:

[0724] The server saves and integrates the collected price information in a database. The collected price information is stored in a database and integrated based on the product list entered by the consumer. This database centralizes store information and its price information. The input is the price information collected in step 2. The output is the integrated database.

[0725] Step 4:

[0726] The server searches the integrated database for the cheapest products. Based on the product list entered by the consumer, the server runs an algorithm to find the cheapest price for each product based on the price information in the database. The input is the consumer's product list and the price information in the database. The output is a list of the cheapest products.

[0727] Step 5:

[0728] The server displays the search results on the consumer's smartphone. The server sends the list of cheapest products obtained in step 4 to the consumer's smartphone and displays it on the application. The input is the list of cheapest products. The output is the displayed information on the cheapest products.

[0729] Step 6:

[0730] The server suggests related recipes based on the input product list and information on the cheapest products. Using a generative AI model, it analyzes the consumer's product list and the ingredient information in the database to generate the optimal recipe. For example, it suggests a recipe for "tomato omelette" for a list of "tomatoes, milk, eggs." The input is the list of cheapest products and the generative AI model. The output is the suggested recipe.

[0731] Step 7:

[0732] The server provides route guidance to the store with the cheapest product. The server uses the geopy library and Nominatim service to calculate the route from the consumer's current location to the store offering the cheapest product and generate a navigation link. The input is the consumer's location and the store's address. The output is the generated navigation link.

[0733] Step 8:

[0734] The server provides coupon information related to the cheapest product. It calls the coupon service API to obtain the coupon related to the cheapest product and notifies the smartphone. The input is the cheapest product and store information. The output is the obtained coupon information.

[0735] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0736] This invention is a system that combines an emotion engine that recognizes the user's emotions to enable consumers to efficiently purchase products at optimal prices in their daily lives. This system allows consumers to input their place of residence and the product they wish to purchase, and uses price information collected from multiple stores in the area to search for and recommend the cheapest products. Furthermore, the emotion engine can analyze the user's emotions and provide personalized recommended products and services based on the results.

[0737] Consumer Features

[0738] Residence registration

[0739] A user accesses the smartphone app and enters the postal code or city where they live. The user's device sends this information to the server, which then stores the residential address information in a database.

[0740] Enter a list of products

[0741] The user enters a list of items they wish to purchase into the app (e.g., "tomatoes, milk, eggs"), and the user's device sends this list to the server.

[0742] Collecting flyer information

[0743] The server collects the latest flyer information from multiple stores in the area based on the registered residential address, including through web scraping and social media. The server also analyzes images of paper flyers provided by consumers using OCR technology and integrates price information into the database.

[0744] Search for the lowest price

[0745] The server searches the database for the lowest priced items based on the product list entered by the user, and sends the search results to the user's terminal, where the lowest price information is displayed.

[0746] Recipe suggestions

[0747] The server searches for relevant recipes from a recipe database based on the shopping list entered by the user and displays them on the user's terminal. For example, if the user enters "tomato, milk, egg," the server will suggest a "tomato omelette" recipe and display its ingredient list.

[0748] Retail Features

[0749] Sales forecast

[0750] The server collects past shopping list data from consumers and begins analysis. Based on the analysis results, it generates sales forecasts and notifies the store's management terminal.

[0751] Coupon distribution and sale announcements

[0752] The server periodically collects and analyzes sales information from nearby stores. It then generates appropriate coupons based on store visit predictions and provides them to consumers. For example, based on sales information for a 2-liter carton of milk, a "10% off coupon when purchasing milk" is generated and notified to the user.

[0753] Inventory management

[0754] The server calculates the appropriate inventory level based on sales forecasts and, in cooperation with the inventory management system, suggests the appropriate amount of stock to the store. For example, if the sales forecast for tomatoes is "300 units next Monday," the server will suggest to the store that they "purchase an additional 200 tomatoes."

[0755] Emotion Engine Functions

[0756] User sentiment analysis

[0757] The emotion engine analyzes the user's emotions using data collected while the user is using the app (e.g., facial recognition data, typing speed and patterns, etc.). The analysis results reflect the user's state in real time and are reflected in the system.

[0758] Providing personalized product and service recommendations

[0759] The server then provides personalized product recommendations and services based on the user's emotions analyzed by the emotion engine. For example, if the user is feeling stressed, the server suggests products and services that have a relaxing effect.

[0760] Providing coupon and sale information at the optimal time

[0761] The server then provides coupon and sale information at the optimal time based on the analysis results of the emotion engine. For example, it executes marketing strategies according to the user's emotional state, such as providing special discount coupons only when the user is in a good mood.

[0762] Specific examples

[0763] As a concrete example, suppose a user uses a smartphone app to register their place of residence and enter "tomatoes, milk, eggs" as part of their weekend shopping list, and the emotion engine detects the user's stress during this process. The server collects the latest flyer information from stores in Shinjuku Ward, searches for the lowest prices, and provides them to the user. At the same time, recipes and products with a relaxing effect are suggested, and special coupons based on sale information are provided to the user. Based on this data, the store can implement appropriate inventory management and sales strategies.

[0764] In this way, the system, combined with the emotion engine, provides more advanced support to both consumers and retailers, improving personalization and efficiency.

[0765] The processing flow will be explained below.

[0766] Consumer Features

[0767] Residence registration

[0768] Step 1:

[0769] The user accesses the smartphone app and enters the postal code or city where they live.

[0770] Step 2:

[0771] The user terminal transmits the entered residence information to the server.

[0772] Step 3:

[0773] The server stores the user's location information in a database.

[0774] Enter a list of products

[0775] Step 4:

[0776] A user enters a list of items they want to buy into the app (e.g., "tomatoes, milk, eggs").

[0777] Step 5:

[0778] The user terminal transmits the input product list to the server.

[0779] Collecting flyer information

[0780] Step 6:

[0781] The server collects flyer information from each store's website and social media based on a pre-registered list of local stores.

[0782] Step 7:

[0783] The server uses web scraping technology to extract flyer information and store it in a database.

[0784] Step 8:

[0785] The user takes a photo of a paper flyer and uploads it to the app.

[0786] Step 9:

[0787] The user terminal transmits the uploaded flyer image to the server.

[0788] Step 10:

[0789] The server uses OCR (Optical Character Recognition) technology to extract price information from the image.

[0790] Step 11:

[0791] The server integrates the extracted price information into a database.

[0792] Search for the lowest price

[0793] Step 12:

[0794] The server searches the database for the cheapest products based on the product list entered by the user.

[0795] Step 13:

[0796] The server sends the search results to the user terminal.

[0797] Step 14:

[0798] The user's device displays the lowest price information (e.g., "Tomatoes: Ito-Yokado 100 yen / each").

[0799] Recipe suggestions

[0800] Step 15:

[0801] The server searches the recipe database for relevant recipes based on the user's shopping list.

[0802] Step 16:

[0803] The server sends the found recipe and its ingredient list to the user terminal.

[0804] Step 17:

[0805] The user device displays the recipe information (e.g., "Tomato Omelette: Tomato, Egg, Salt, Pepper").

[0806] Retail Features

[0807] Sales forecast

[0808] Step 18:

[0809] The server collects past consumer shopping list data and begins analyzing it.

[0810] Step 19:

[0811] The server generates a sales forecast based on the analysis results (e.g., "Tomato sales are predicted to increase next Monday").

[0812] Step 20:

[0813] The server transmits the generated sales forecast to the store's management terminal and notifies it.

[0814] Coupon distribution and sale announcements

[0815] Step 21:

[0816] The server periodically collects and analyzes sales information from nearby stores.

[0817] Step 22:

[0818] The server generates appropriate coupons based on store visit predictions (e.g., "10% off coupon when purchasing 2 liters of milk").

[0819] Step 23:

[0820] The server transmits the generated coupon information to the user terminal and performs a push notification.

[0821] Inventory management

[0822] Step 24:

[0823] The server calculates the appropriate inventory quantity based on the sales forecast.

[0824] Step 25:

[0825] The server connects with the inventory management system and suggests appropriate purchase quantities to the store (e.g., "Purchase an additional 200 tomatoes").

[0826] Emotion Engine Functions

[0827] User sentiment analysis

[0828] Step 26:

[0829] While the user is using the app, the emotion engine collects the user's facial expression data, typing speed and patterns.

[0830] Step 27:

[0831] The user terminal transmits the collected data to the server.

[0832] Step 28:

[0833] The server uses an emotion engine to analyze the user's emotions in real time.

[0834] Providing personalized product and service recommendations

[0835] Step 29:

[0836] The server determines personalized recommended products and services based on the analysis results of the emotion engine.

[0837] Step 30:

[0838] The server sends information about recommended products and services to the user terminal.

[0839] Step 31:

[0840] The user's device displays personalized product and service recommendations (e.g., if the user is feeling stressed, it suggests products that have a relaxing effect).

[0841] Providing coupon and sale information at the optimal time

[0842] Step 32:

[0843] The server generates coupon and sale information at the optimal time based on the analysis results of the emotion engine.

[0844] Step 33:

[0845] The server transmits the generated coupon and sale information to the user terminal.

[0846] Step 34:

[0847] The user's device notifies and displays coupons and sales information (e.g., offering special discount coupons when the user is in a good mood).

[0848] Example 2

[0849] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0850] Conventional price comparison systems simply collect price information from multiple stores and are unable to make suggestions tailored to consumer sentiment or individual needs. Furthermore, they lacked a mechanism for integrating information from flyer images taken by consumers into the system, making it difficult to reflect the latest price information. Furthermore, they lacked the functionality to integrate the collected information into a database and provide related recipes based on the shopping list entered by the consumer.

[0851] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0852] In this invention, the server includes: a means for a consumer to input their place of residence; a means for a consumer to input a list of products they wish to purchase; a means for collecting information from multiple local sales locations; a means for analyzing the collected information and extracting and integrating price information; a means for searching for the cheapest products based on the list of products they wish to purchase; a means for displaying the search results on the consumer's terminal; a means including an engine for analyzing consumer sentiment; and a means for providing personalized recommended products and services based on the analysis results. This enables the provision of personalized services according to the consumer's sentiments and needs, enabling more advanced price comparisons and shopping optimization. Furthermore, the server can analyze flyer images provided by consumers and integrate the latest price information into the database, thereby providing more accurate and up-to-date price information.

[0853] A "consumer" is an individual who purchases goods or services.

[0854] "Residence" is geographic information that indicates where a consumer currently lives.

[0855] "Point of sale" means a store or market that offers goods or services.

[0856] "Information gathering" is the process of gathering data from different sources.

[0857] "Analysis" refers to examining and processing collected information to extract meaningful data.

[0858] A "product list" is a list of products that a consumer wishes to purchase.

[0859] "Price information" is data regarding the selling price of a product or service.

[0860] "Search" is the act of finding information that matches specific conditions from a vast amount of data.

[0861] A "consumer terminal" is a digital device used by a consumer, such as a computer or smartphone.

[0862] An "emotion engine" is a technology for analyzing consumer emotions and their state.

[0863] "Recommended Products" are products that are specifically recommended based on a consumer's needs and preferences.

[0864] "Services" refers to support activities provided to consumers other than the products sold.

[0865] "Integration" refers to bringing together different data into one system or dataset.

[0866] "OCR technology" is a technology that converts scanned documents into machine-readable text data.

[0867] A "flyer image" is an image file of a paper printed for advertising or promotion.

[0868] A "recipe" is a set of instructions that lists the ingredients and steps for making a particular dish.

[0869] "Related recipes" are recipes for dishes related to the user's shopping list.

[0870] This invention is a system that enables consumers to efficiently purchase products at the optimal price. Specifically, the system allows consumers to input their place of residence and the product they wish to purchase, and combines price information collected from multiple local sales locations with an emotion engine to suggest the most suitable product.

[0871] This system mainly involves the exchange of information between the server, the device, and the consumer. The main hardware used to implement the system is the server, the consumer device (smartphone or computer), and network equipment. The software used includes Python, BeautifulSoup, Scrapy, MySQL, Tesseract OCR, OpenCV, analysis tools (pandas, scikit-learn), and web scraping tools.

[0872] First, a user accesses the smartphone app and enters the postal code or city where they live. This information is sent from the device to the server, which stores the residence information in a database. Next, the user enters a list of items they wish to purchase into the app, and the device sends this list to the server.

[0873] The server collects the latest flyer information from multiple local stores based on the registered residential address information. This collection process uses web scraping technology using Python's BeautifulSoup and Scrapy. In addition, images of paper flyers provided by consumers are analyzed using OCR technology (Tesseract OCR), and price information is integrated into the database.

[0874] The server then searches the database for the cheapest products based on the product list entered by the user, using the MySQL database management system, and sends the search results to the consumer's terminal, where the best price information is displayed to the user.

[0875] Furthermore, based on the shopping list entered by the user, the server searches for related recipes from the recipe database and displays them on the user's terminal. For example, if the user enters "tomato, milk, egg" in the list, the server will suggest a recipe for "tomato omelette."

[0876] For retail stores, the server collects shopping list data from consumers and uses analytical tools to predict sales. The analysis results are sent to the store's management terminal, which then implements appropriate inventory management and sales strategies. The server also collects and analyzes sales information from nearby stores, and generates appropriate coupons based on the predictions to provide to consumers.

[0877] Furthermore, the emotion engine analyzes data collected while the user is using the app (such as facial recognition data and input speed) to evaluate the user's emotions. This analysis is performed using OpenCV and other analytical tools. Based on the analysis results, the server provides personalized product and service recommendations according to the user's emotions. For example, if the user is feeling stressed, it will suggest products and services that have a relaxing effect.

[0878] As a concrete example, suppose a user uses a smartphone app to register their place of residence and enter "tomatoes, milk, and eggs" as part of their weekend shopping list, and the emotion engine detects the user's stress during this process. The server collects the latest flyer information from sales locations in Shinjuku Ward, searches for the lowest prices, and provides them to the user. At the same time, recipes and products with a relaxing effect are suggested, and special coupons based on sale information are notified to the user. Based on this data, the sales locations can implement appropriate inventory management and sales strategies.

[0879] In this way, the system, combined with the emotion engine, provides more advanced support to both consumers and points of sale, improving personalization and efficiency.

[0880] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0881] Step 1:

[0882] The user accesses the smartphone app and enters the postal code or city where they live.

[0883] Input: Postal code or city of residence

[0884] Output: Input data is saved on the user's device

[0885] Specific operation: The user enters the postal code and city / town name into the app's residence input form and presses the "Submit" button.

[0886] Step 2:

[0887] The terminal sends this information to the server.

[0888] Input: User-entered residential address information

[0889] Output: Location information is sent to the server

[0890] Specific operation: The device sends data to the server via the Internet.

[0891] Step 3:

[0892] The server stores the residence information in a database.

[0893] Input: Residential information

[0894] Output: Residence information is saved in the database

[0895] Specific operation: The server executes an SQL query to store the received residence information in a database.

[0896] Step 4:

[0897] A user enters a list of items they want to buy into the app (e.g., "tomatoes, milk, eggs").

[0898] Input: List of items you wish to purchase

[0899] Output: Input data is saved on the user's device

[0900] Specific operation: The user enters the product name into the app's shopping list input form and presses the "Submit" button.

[0901] Step 5:

[0902] The terminal sends this list to the server.

[0903] Input: A list of items the user wishes to purchase

[0904] Output: A list of items to purchase is sent to the server

[0905] Specific operation: The device sends data to the server via the Internet.

[0906] Step 6:

[0907] The server collects the latest flyer information from multiple sales locations in the area based on the registered residential address information.

[0908] Input: Residential information

[0909] Output: Collected flyer information

[0910] Specific operation: Based on the specified residence information, the server performs web scraping using Python's BeautifulSoup and Scrapy to collect flyer information from the websites of each sales location.

[0911] Step 7:

[0912] The server receives an image of a paper flyer provided by a consumer.

[0913] Input: Flyer image

[0914] Output: Flyer images are saved on the server

[0915] Specific operation: A user takes a photo of a flyer with their smartphone and uploads the image through the app. The server receives the image.

[0916] Step 8:

[0917] The server analyzes the flyer image using OCR technology (e.g., Tesseract OCR) and integrates the price information into a database.

[0918] Input: Flyer image

[0919] Output: Price information in the database

[0920] Specific operation: The server performs OCR analysis, extracts text data from the flyer image, and parses price information from it. The parsed data is then integrated into a database.

[0921] Step 9:

[0922] The server searches the database for the cheapest products based on the product list entered by the user.

[0923] Input: List of products you wish to purchase, price information

[0924] Output: Information on the cheapest product

[0925] What it does: The server queries the database for pricing information to find the cheapest item. It uses Python data analysis tools to calculate the lowest price.

[0926] Step 10:

[0927] The server sends the search results to the user's terminal and displays the lowest price information.

[0928] Input: Information on the lowest priced item

[0929] Output: The lowest price information displayed on the user's terminal

[0930] Specific operation: The server sends the search results to the user's device, and the app displays them.

[0931] Step 11:

[0932] The server searches for related recipes from a recipe database based on the shopping list entered by the user and displays them on the user terminal.

[0933] Input: List of items you wish to purchase

[0934] Output: Related recipe information

[0935] Specific operation: The server searches the recipe database for relevant recipes based on the shopping list. The searched recipe information is sent to the user's terminal and displayed.

[0936] Step 12:

[0937] The server collects past consumer shopping list data and begins analyzing it.

[0938] Input: Past shopping list data

[0939] Output: Analysis results (sales forecast)

[0940] What it does: The server runs a Python analytics tool to analyze historical consumer shopping list data in the database and generate sales forecasts.

[0941] Step 13:

[0942] The server periodically collects and analyzes sales information from nearby stores.

[0943] Input:Sale information

[0944] Output: Parsed sale information, generated coupons

[0945] What it does: The server periodically runs a web script to collect and analyze sales information from nearby stores.

[0946] Step 14:

[0947] The server generates appropriate coupons based on the store visit prediction and provides them to the consumer terminal.

[0948] Input: store visit prediction, sales information

[0949] Output: Generated coupon

[0950] Specific operation: The server executes an algorithm to generate appropriate coupons based on store visit predictions. The generated coupons are sent to the consumer's terminal and notified.

[0951] Step 15:

[0952] The emotion engine analyzes data collected while users are using the app.

[0953] Input: User operation data (face recognition data, input speed, etc.)

[0954] Output: User sentiment analysis results

[0955] Specific operation: The emotion engine collects user operation data and analyzes the emotional state using OpenCV and data analysis tools.

[0956] Step 16:

[0957] The server provides personalized recommended products and services based on the user's emotions analyzed by the emotion engine.

[0958] Input: Sentiment analysis results

[0959] Output: Personalized product and service recommendations

[0960] Specific operation: The server refers to the sentiment analysis results and executes a personalized recommendation algorithm to suggest appropriate products and services. The suggestions are sent to the user's device and displayed.

[0961] Step 17:

[0962] Based on the analysis results of the emotion engine, the server generates and provides coupon and sale information at the optimal time.

[0963] Input: Sentiment analysis results, sales information

[0964] Output: Generated coupons and sales information

[0965] Specific operation: The server uses the results of the sentiment analysis to run an algorithm that generates coupon and sale information at the optimal time. The generated information is then sent to the user's device and notified.

[0966] (Application example 2)

[0967] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0968] In the past, consumers had to visit multiple stores to find the best price, which was time-consuming and labor-intensive. Furthermore, the means for researching price information were limited, making efficient shopping difficult. Furthermore, personalized product and service recommendations based on the consumer's emotional state were not provided, making it difficult to improve the consumer experience.

[0969] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: a means for a consumer to input their place of residence; a means for inputting a list of products the consumer wants to buy; a means for collecting flyer information from multiple stores in the area based on the place of residence; a means for searching for the cheapest products from the collected flyer information based on the list of products the consumer wants to buy; a means for displaying the search results on the consumer terminal; a means for analyzing the consumer's emotions; a means for providing personalized recommended products and services based on the results of the emotion analysis; and a means for displaying the personalized recommended products and services on the consumer terminal. This enables consumers to efficiently purchase products at optimal prices and receive personalized suggestions based on their emotional state.

[0970] "Means for entering residential address" refers to the interface that allows consumers to register their residential address information in the system. Examples of such interfaces include smart glasses and smartphone apps.

[0971] The "means for inputting a list of products to be purchased" refers to an interface that allows consumers to register a list of products they wish to purchase in the system. This includes a touch screen and voice recognition.

[0972] "Methods for collecting flyer information" are methods for collecting the latest price and sale information from multiple stores in the area. Flyer images are analyzed using web scraping and OCR technology.

[0973] The "means for searching for the cheapest products" refers to algorithms or software that search for the lowest priced products from among the products entered by the consumer based on collected flyer information.

[0974] "Means for displaying search results on consumer devices" refers to an interface for presenting the lowest price information found on the consumer's device (such as smart glasses or a smartphone).

[0975] "Emotion analysis tools" are software or algorithms that analyze a consumer's facial expressions and behavioral data to determine their current emotional state. This may include cameras and sensors.

[0976] "Means for providing personalized product and service recommendations" refers to algorithms and software that suggest the most appropriate products and services based on the consumer's emotional state.

[0977] The "means for displaying personalized recommended products and services on a consumer terminal" is an interface for displaying recommended products and services related to sentiment analysis on a terminal used by a consumer.

[0978] The present invention is a system for supporting consumers in purchasing products efficiently at optimal prices in their daily lives. Specific embodiments of the system will be described below.

[0979] Implementation system configuration

[0980] Consumer Devices

[0981] Consumers use smart glasses or a smartphone app to input their location and a list of items they want to buy. The device has an interface that allows input via touchscreen or voice recognition.

[0982] server

[0983] The server receives information on the consumer's location and product list from the consumer's device and collects the latest flyer information from multiple stores in the area. This information is collected by integrating flyer information from online and paper sources using web scraping and OCR technologies. An algorithm is then run to search for the cheapest product among the items the consumer wants to buy, based on the price information stored in the database.

[0984] Furthermore, the server is equipped with an emotion analysis engine that recognizes emotions in real time by analyzing data such as consumer facial images and input patterns. Based on the results of this emotion analysis, personalized product and service recommendations are generated and displayed on the consumer's device.

[0985] Data processing and calculation

[0986] Sentiment Analysis Engine

[0987] This engine analyzes consumer facial image data collected using cameras and sensors, as well as the speed and patterns of their actions. Major technologies used include OpenCV (for facial recognition) and TensorFlow (for emotion recognition). Consumer emotions (such as stress or happiness) are detected in real time by this engine and provided as data to a server.

[0988] Price information collection and integration

[0989] The server automatically collects price information from websites and paper sources using web scraping and OCR technologies, specifically BeautifulSoup (for web scraping) and Tesseract OCR (for character recognition), and stores it in a database.

[0990] Best price search and recommendations

[0991] The server uses the product list and address information entered by the consumer to screen price information in the database and find the cheapest products. Data processing libraries such as Python's pandas and SQL are used. In addition, by combining consumer sentiment and individual product information, an algorithm is run that individually recommends the best products and services for each specific consumer.

[0992] Specific examples

[0993] Suppose a consumer uses smart glasses to input their place of residence (e.g., "Shinjuku Ward") and a list of products they wish to buy (e.g., "tomatoes, milk, eggs"). During this process, the smart glasses' camera captures the consumer's facial expression and sends the data to a server. If the emotion analysis engine detects stress, the server will suggest products or services with a relaxing effect (e.g., "aromatherapy oil") in addition to the usual lowest price information.

[0994] Prompt Sentence Examples

[0995] "I live in Shinjuku Ward, and I'd like to buy tomatoes, milk, and eggs. Next, I'll provide an image of my facial expression taken by the camera on my smart glasses. Based on this information, please tell me the lowest prices in the area and recommend products based on my emotions."

[0996] In this way, the embodiment of the invention realizes a system that reflects the consumer's input information and emotional state, collects and analyzes optimal price information from multiple stores, and provides personalized recommended products in real time.

[0997] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0998] Step 1:

[0999] A user uses a consumer device (smart glasses or a smartphone app) to input their location and a list of products they wish to purchase. The input information is then sent from the device to a server. Specifically, the user uses the device's interface to input information using a touch screen or voice recognition. The input data is in the form of text, which is then sent to the server.

[1000] Step 2:

[1001] Based on the received residential address information, the server collects the latest flyer and price information from multiple stores in the area. This collection is performed using web scraping and OCR technologies. BeautifulSoup is used for web scraping to extract price information from online store websites. Tesseract OCR is also used to recognize and extract characters from paper flyer images. The collected data is then stored in a database.

[1002] Step 3:

[1003] The server searches for the cheapest products that match the product list entered by the user based on the price information stored in the database. This process involves filtering the data using Python's pandas library and executing database queries using SQL. The search results are returned in text format as the lowest price information for the target products and are sent to the consumer's device.

[1004] Step 4:

[1005] The consumer device displays the lowest price information sent from the server to the user. In the case of smart glasses, the price information is displayed in the user's field of view using holographic display technology. In the case of smartphones, the price information is displayed in text and graphical format on the screen, allowing the user to visually check the cheapest product information.

[1006] Step 5:

[1007] Consumer emotion data is collected in real time by the camera and sensors on the consumer device. Specifically, facial image data captured by the camera is processed within the device, and the results are sent to a server. This data is analyzed using an emotion recognition model that combines OpenCV and TensorFlow on the edge device. The analysis results are sent to the server as text data.

[1008] Step 6:

[1009] The server receives the results of the emotion analysis and generates personalized product and service recommendations based on them. If the emotion indicates stress, products and services with a relaxing effect will be recommended. This recommendation process is performed by an algorithm using a generative AI model, and the recommendation results are obtained as text data.

[1010] Step 7:

[1011] The server sends the generated information about recommended products and services to the consumer's device, which then displays the received information to the user. In the case of smart glasses, the recommended product information is displayed in the user's field of vision using holographic display technology. In the case of smartphones, the recommended information is displayed in text and graphical format on the screen, allowing the user to learn about the best products and services based on their emotional state.

[1012] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1013] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1014] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1015] [Third embodiment]

[1016] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1017] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1019] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1020] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1021] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1023] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1024] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1026] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1027] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1028] This invention is a system that allows consumers to efficiently purchase products at optimal prices in their daily lives. The system allows consumers to input their location and the product they wish to purchase, and uses price information collected from multiple stores in the area to search for and recommend the cheapest products. The system can also analyze the collected data to suggest recipes, forecast sales, and manage inventory.

[1029] Consumer Features

[1030] Residence registration

[1031] The user registers their local area (e.g., Shinjuku-ku, Tokyo). The user enters their area using the smartphone app interface.

[1032] Enter a list of products

[1033] Users register a list of items they want to buy, such as "tomatoes, milk, eggs," through the app.

[1034] Collecting flyer information

[1035] The server collects the latest flyer information from multiple stores in the area based on the registered residential address. This includes web scraping and social media information gathering. The server also analyzes images of paper flyers provided by consumers using OCR technology and integrates price information into the database.

[1036] Search for the lowest price

[1037] The server searches for the cheapest product entered by the user from the flyer information collected by the server. The server sends the search results to the user's terminal and displays them. For example, if the cheapest tomato price is "100 yen per unit at Ito-Yokado," that information is provided to the user.

[1038] Recipe suggestions

[1039] The server searches for relevant recipes based on the shopping list entered by the user and displays them on the user's device. For example, if "tomato, milk, egg" is entered, the server will suggest a recipe for "tomato omelette" and display its ingredients list.

[1040] Retail Features

[1041] Sales forecast

[1042] The server collects and analyzes consumer shopping list data and makes sales forecasts. For example, if many users plan to buy tomatoes, the server predicts that demand for tomatoes will be high and notifies stores.

[1043] Coupon distribution and sale announcements

[1044] The server obtains sales information for nearby stores, generates coupons based on predicted store visits, and provides them to consumers. For example, based on sales information for a 2-liter pack of milk, a 10% off coupon for purchasing milk is generated and notified to the user.

[1045] Inventory management

[1046] The server calculates the appropriate inventory quantity based on the sales forecast and connects to the inventory management system to suggest the appropriate amount to purchase. For example, if the sales forecast for tomatoes is "300 units next Monday," the server will suggest to the store to "purchase an additional 200 tomatoes."

[1047] Specific examples

[1048] As a concrete example, suppose a user uses a smartphone app to register the area where they live and enter "tomatoes, milk, eggs" as part of their weekend shopping list. The server collects the latest flyer information from stores in Shinjuku Ward, searches for the lowest prices, and generates a list such as "tomatoes for 100 yen at Ito-Yokado, milk for 150 yen at Aeon, and eggs for 180 yen at a nearby greengrocer." It also suggests a recipe for "tomato omelette." Based on this data, retailers can distribute sale coupons and manage inventory appropriately.

[1049] In this way, the system provides optimal support for both consumers and retailers.

[1050] The processing flow will be explained below.

[1051] Consumer Features

[1052] Residence registration

[1053] Step 1:

[1054] The user accesses the smartphone app and enters the postal code or city where they live.

[1055] Step 2:

[1056] The user terminal transmits the input information to the server.

[1057] Step 3:

[1058] The server stores the user's location information in a database.

[1059] Enter a list of products

[1060] Step 4:

[1061] A user enters a list of items they want to buy into the app (e.g., "tomatoes, milk, eggs").

[1062] Step 5:

[1063] The user terminal transmits the input product list to the server.

[1064] Collecting flyer information

[1065] Step 6:

[1066] The server collects flyer information from each store's website and social media based on a pre-registered list of local stores.

[1067] Step 7:

[1068] The server uses web scraping technology to extract flyer information and store it in a database.

[1069] Step 8:

[1070] The user takes a photo of a paper flyer and uploads it to the app.

[1071] Step 9:

[1072] The user terminal transmits the uploaded flyer image to the server.

[1073] Step 10:

[1074] The server uses OCR (Optical Character Recognition) technology to extract price information from the image.

[1075] Step 11:

[1076] The server integrates the extracted price information into a database.

[1077] Search for the lowest price

[1078] Step 12:

[1079] The server searches the database for the cheapest products based on the product list entered by the user.

[1080] Step 13:

[1081] The server sends the search results to the user terminal.

[1082] Step 14:

[1083] The user's device displays the lowest price information (e.g., "Tomatoes: Ito-Yokado 100 yen / each").

[1084] Recipe suggestions

[1085] Step 15:

[1086] The server searches a recipe database for relevant recipes based on the user's shopping list.

[1087] Step 16:

[1088] The server sends the found recipe and its ingredient list to the user terminal.

[1089] Step 17:

[1090] The user device displays the recipe information (e.g., "Tomato Omelette: Tomato, Egg, Salt, Pepper").

[1091] Retail Features

[1092] Sales forecast

[1093] Step 18:

[1094] The server collects past consumer shopping list data and begins analyzing it.

[1095] Step 19:

[1096] The server generates a sales forecast based on the analysis results (e.g., "Tomato sales are predicted to increase next Monday").

[1097] Step 20:

[1098] The server transmits the generated sales forecast to the store's management terminal and notifies it.

[1099] Coupon distribution and sale announcements

[1100] Step 21:

[1101] The server periodically collects and analyzes sales information from nearby stores.

[1102] Step 22:

[1103] The server generates appropriate coupons based on predicted store visits (e.g., "10% off coupon when purchasing 2 liters of milk").

[1104] Step 23:

[1105] The server transmits the generated coupon information to the user terminal and performs a push notification.

[1106] Inventory management

[1107] Step 24:

[1108] The server calculates the appropriate inventory quantity based on sales forecasts.

[1109] Step 25:

[1110] The server connects with the inventory management system and suggests appropriate purchase quantities to the store (e.g., "Purchase an additional 200 tomatoes").

[1111] The above processing steps enable consumers to purchase products at the best price and enable retailers to efficiently manage inventory and forecast sales.

[1112] Example 1

[1113] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1114] Today's consumers are required to browse numerous stores and online platforms to find the cheapest prices for products. However, manually collecting and comparing advertising information from individual stores is time-consuming and inefficient. Furthermore, there is a lack of ways to integrate and analyze multiple advertising information, making it difficult for consumers to find the cheapest products. Furthermore, there is a lack of related cooking suggestions, discount information to promote purchases, and inventory optimization in retail stores.

[1115] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1116] In this invention, the server includes a means for inputting a consumer's place of residence, a means for inputting a list of products the consumer wishes to buy, a means for collecting advertising information from multiple stores in the area based on the consumer's place of residence, a means for searching for the cheapest products from the collected advertising information based on the list of products the consumer wishes to buy, a means for displaying the search results on the consumer's terminal, and a means for analyzing the advertising information and integrating the price information into a database. This allows consumers to efficiently find the cheapest products and receive related cooking suggestions and discount information to promote purchases based on the results. It also enables retail stores to optimize inventory based on sales forecasts.

[1117] A "means for inputting a place of residence" is a means for providing an interface for a consumer to input the area in which they live.

[1118] The "means for inputting a list of products to be purchased" is a means for providing an interface for a consumer to input products that he or she intends to purchase.

[1119] The "means for collecting advertising information" is a means for automatically acquiring the latest advertising information from a plurality of local stores based on residential location information.

[1120] The "means for analyzing advertising information" is a means for deciphering the collected advertising information and converting it into usable price data.

[1121] "Means for integrating price information into a database" refers to a means for organizing and storing analyzed price data in a single database.

[1122] The "means for searching for the cheapest product" is a means for searching for the cheapest product among the products input by the user from the collected and analyzed advertising information.

[1123] The "means for displaying search results on a consumer terminal" refers to a means for displaying the searched lowest price product information on the consumer's electronic device.

[1124] The "means for searching for a recipe" is a means for searching for a related recipe based on the product list entered by the user and the searched lowest price product information.

[1125] The "means for providing discount coupons or promotional information for promoting purchases" is a means for the server to notify consumers of discount coupons or special sale information based on predicted data.

[1126] "Means for optimizing inventory" refers to means for calculating and proposing the appropriate inventory amount for a store based on predicted sales data.

[1127] This invention is a system that enables consumers to efficiently purchase products at optimal prices in their daily lives. The system allows consumers to input their location and the product they wish to purchase, and uses advertising information collected from multiple stores in the area to search for and recommend the cheapest products. The system can also analyze the collected data to suggest related recipes, forecast sales, and manage inventory.

[1128] Consumer Features

[1129] Residence registration

[1130] The user registers their local area (e.g., Shinjuku-ku, Tokyo). The user enters their area using the smartphone app interface. This information is sent to the server and stored.

[1131] Enter a list of products

[1132] The user registers a list of items they want to buy. The user inputs items such as "tomatoes, milk, eggs" through the app. This information is also sent to the server and stored.

[1133] Collecting advertising information

[1134] The server collects the latest advertising information from multiple stores in the area based on the registered residential address. This includes web scraping technology, APIs, and social media information collection. The server also analyzes images of print advertisements provided by consumers using OCR (optical character recognition) technology and integrates the price information into a database. This database centrally manages the price information collected from multiple stores.

[1135] Search for the lowest price

[1136] The server searches for the cheapest product entered by the user from the advertising information collected. For example, if the cheapest price for tomatoes is "100 yen per unit at a specific supermarket," that information is sent to the user's device and displayed.

[1137] Recipe suggestions

[1138] The server searches for related recipes based on the shopping list entered by the user and displays them on the user's terminal. For example, if "tomato, milk, egg" is entered, the server will suggest a recipe for "tomato omelette" and display its ingredients list.

[1139] Retail Features

[1140] Sales forecast

[1141] The server collects and analyzes consumer shopping list data and makes sales forecasts. For example, if many users plan to buy tomatoes, the server predicts that demand for tomatoes will be high and notifies stores.

[1142] Coupon distribution and sale announcements

[1143] The server obtains sales information for nearby stores, generates discount coupons based on predicted store visits, and provides them to consumers. For example, based on sales information for a 2-liter pack of milk, a 10% discount coupon for purchasing milk is generated and notified to the user.

[1144] Inventory management

[1145] The server calculates the appropriate inventory quantity based on the sales forecast and connects to the inventory management system to suggest the appropriate amount to purchase. For example, if the sales forecast for tomatoes is "300 units next Monday," the server will suggest to the store to "purchase an additional 200 tomatoes."

[1146] Specific examples

[1147] As a concrete example, suppose a user uses a smartphone app to register the area where they live and enter "tomatoes, milk, eggs" as part of their weekend shopping list. The server collects the latest advertising information from multiple stores in the registered area and searches for the lowest prices. In this example, a list is generated such as "tomatoes for 100 yen at a specific supermarket, milk for 150 yen at another supermarket, and eggs for 180 yen at a local store." In addition, a recipe for "tomato omelette" is suggested. Retailers can use this data to distribute discount coupons and manage inventory appropriately.

[1148] Example prompts for generative AI models

[1149] "I live in a particular area. I want to buy tomatoes, milk, and eggs on the weekend. Which store is the cheapest?"

[1150] "Please tell me some recipes that use tomatoes, milk, and eggs."

[1151] In this way, the system provides optimal support for both consumers and retailers.

[1152] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1153] Step 1: Register your residence

[1154] Input: The user uses a smartphone app to enter the area where they live (e.g., Shinjuku-ku, Tokyo).

[1155] Specific operation: The user enters their place of residence into the form within the app and presses the registration button. The device captures this information and sends it to the server.

[1156] Data processing or data calculation: The server stores the received residence information in a database and associates it with the user profile.

[1157] Output: The server stores the residence information and prepares it for use in the next processing step.

[1158] Step 2: Enter your shopping list

[1159] Input: The user enters a list of items they want to buy (e.g., tomatoes, milk, eggs).

[1160] Specific operation: The user enters a list of items into the shopping list entry screen within the app and presses the submit button. The device acquires this information and sends it to the server.

[1161] Data processing or data calculation: The server stores the received product list in a database and associates it with the user profile.

[1162] Output: The server stores the product listing information and prepares it for use in the next processing step.

[1163] Step 3: Collect advertising information

[1164] Input: Server gets location and product listing information.

[1165] How it works: The server uses web scraping, APIs, and social media information gathering technology to collect the latest advertising information from multiple stores in the area. Images of print advertisements provided by users are analyzed using OCR technology.

[1166] Data processing or data calculation: Analyzing the collected advertising data, extracting the necessary price information and integrating it into the database.

[1167] Output: The server stores the consolidated pricing information data in a database.

[1168] Step 4: Search for the best price

[1169] Input: The server retrieves the consolidated pricing data and the user's shopping list.

[1170] What happens: The server searches the price information in the database to find the cheapest price for the product entered by the user.

[1171] Data processing or data crunching: Using search algorithms to identify the cheapest products and organize their details.

[1172] Output: The server lists the cheapest products and sends them to the user's device.

[1173] Step 5: Recipe suggestions

[1174] Input: The server retrieves the user's shopping list information.

[1175] What happens: The server searches its recipe database to find recipes related to the user's shopping list.

[1176] Data processing or data calculation: Selecting the best recipe from the recipe database and preparing its details.

[1177] Output: The server sends the relevant recipe information to the user's device.

[1178] Step 6: View results and redeem coupons

[1179] Input: Receives the lowest price product information and cooking method information sent to the user's terminal.

[1180] Specific operation: The user browses the lowest price product information and cooking method information on the smartphone app. In addition, the user checks and uses the provided coupon information.

[1181] Data processing or data calculation: verifying the validity of coupon codes and recording their usage in a database.

[1182] Output: The user can efficiently purchase the cheapest products and use coupons.

[1183] As a result, the present invention provides optimal support to both consumers and retailers, and creates an environment in which consumers can shop efficiently.

[1184] (Application example 1)

[1185] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1186] To enable consumers to purchase products efficiently, it is necessary not only to collect product price information and find the cheapest products, but also to suggest recipes, provide navigation to the cheapest stores, and present related coupon information. However, existing systems have difficulty providing all of these in a centralized manner, making it difficult for consumers to make optimal purchases without hassle. Therefore, an objective of the present invention is to provide a comprehensive system that enables consumers to efficiently and optimally conduct purchasing activities.

[1187] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1188] In this invention, the server includes means for a consumer to input their place of residence, means for inputting a list of products the consumer wants to buy, means for collecting price information from a plurality of stores in the area based on the place of residence, means for searching for the cheapest products from the collected price information based on the list of products the consumer wants to buy, means for displaying the search results on the consumer terminal, means for suggesting recipes related to the consumer, and means for providing route guidance to the store with the cheapest products. This enables the consumer to efficiently purchase products at the best price with a single operation, and to obtain related recipe information, route guidance to the store, and even coupon information.

[1189] "Consumer" refers to an individual or household who purchases goods or services.

[1190] "Place of residence" refers to the area where a consumer lives and shops on a daily basis.

[1191] "List of Products" refers to a specific list of products that a consumer wishes to purchase.

[1192] "Store" means a physical location or online sales site where products are sold.

[1193] "Price information" refers to the sales price set by each store for the product.

[1194] "Collect" refers to gathering specified information and integrating it into a single piece of data.

[1195] "Searching" refers to finding the desired information based on input data and conditions.

[1196] "Displaying" refers to outputting information on a terminal in a form that can be viewed by a consumer.

[1197] A "recipe" refers to a document or data that explains how to prepare a dish using specific ingredients.

[1198] "Navigation" refers to providing directions or routes to reach a specified destination.

[1199] "Coupon information" refers to information that allows you to receive discounts on products or services under certain conditions.

[1200] A "generative AI model" refers to an artificial intelligence model that automatically generates new information or suggestions based on given data.

[1201] This system follows the steps below to enable consumers to purchase products efficiently at the best price. First, the user uses a smartphone application to input their address and a list of products they wish to purchase. This address and product list information is then sent to the server.

[1202] The server uses web scraping and OCR (Optical Character Recognition) technologies to collect price information from multiple local stores based on the collected information. Specifically, the requests and BeautifulSoup libraries are used for web scraping, and an appropriate OCR library is used for OCR analysis.

[1203] The collected price information is stored and integrated in a database that includes paper flyer images provided by consumers and data obtained through web scraping, allowing price information from a variety of sources to be managed in a unified manner.

[1204] The server searches the collected price information for the cheapest products based on the product list entered by the consumer and displays the search results on the consumer's smartphone, allowing the consumer to easily decide which store to purchase which products.

[1205] The server then suggests related recipes based on the input product list and the cheapest product information. For example, if a user inputs tomatoes, milk, and eggs, the server will search for related "tomato omelette" recipes and display them on the consumer's smartphone. This recipe suggestion uses a generative AI model to automatically generate the optimal recipe.

[1206] Additionally, the server utilizes a Geographic Information System (GIS) to provide route guidance to the store offering the cheapest product. Specifically, it uses the geopy library and the Nominatim service to generate and present to the consumer a navigation link to the store offering the cheapest product.

[1207] The server also provides coupon information related to the cheapest products, which is retrieved using the coupon service API and notified to the consumer.

[1208] As a concrete example, suppose a user launches a smartphone application and inputs their address ("Shinjuku Ward, Tokyo") and a list of items they wish to purchase ("Tomatoes, Milk, Eggs"). The server collects price information from stores in Shinjuku Ward, searches for the cheapest prices, and generates and presents a list such as "Tomatoes for 100 yen at Store A, Milk for 150 yen at Store B, and Eggs for 180 yen at Store C." It also suggests a recipe for "Tomato Omelette" and provides route guidance links to Stores A and B. The user is then notified of a "10% off coupon that can be used at Store A."

[1209] Example prompts to input to a generative AI model:

[1210] "Suppose a user living in Shinjuku Ward, Tokyo wants to buy tomatoes, milk, and eggs, and we want to collect information on the cheapest prices from local stores. As a result, we want to show the cheapest store and price. We also want to suggest recipes using these ingredients. We also want to provide a navigation link to the cheapest store and coupon information."

[1211] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1212] Step 1:

[1213] The user inputs their address and a list of products they wish to purchase. Using a smartphone application, the user inputs their address as "Shinjuku-ku, Tokyo" and then adds "tomatoes, milk, and eggs" to the list of products they wish to purchase. This information is sent from the device to the server. The input data includes the user's address and product list. The output is the user's address and product list information stored on the server.

[1214] Step 2:

[1215] The server collects price information from multiple stores in the area. Specifically, it uses web scraping technology (using requests and the BeautifulSoup library) to collect price information from store websites. It then uses OCR technology to analyze paper flyer images provided by consumers and extract price information. The input is the URL of the store website and the paper flyer image. The output is a dataset containing the collected price information.

[1216] Step 3:

[1217] The server saves and integrates the collected price information in a database. The collected price information is stored in a database and integrated based on the product list entered by the consumer. This database centralizes store information and its price information. The input is the price information collected in step 2. The output is the integrated database.

[1218] Step 4:

[1219] The server searches the integrated database for the cheapest products. Based on the product list entered by the consumer, the server runs an algorithm to find the cheapest price for each product based on the price information in the database. The input is the consumer's product list and the price information in the database. The output is a list of the cheapest products.

[1220] Step 5:

[1221] The server displays the search results on the consumer's smartphone. The server sends the list of cheapest products obtained in step 4 to the consumer's smartphone and displays it on the application. The input is the list of cheapest products. The output is the displayed information on the cheapest products.

[1222] Step 6:

[1223] The server suggests related recipes based on the input product list and information on the cheapest products. Using a generative AI model, it analyzes the consumer's product list and the ingredient information in the database to generate the optimal recipe. For example, it suggests a recipe for "tomato omelette" for a list of "tomatoes, milk, eggs." The input is the list of cheapest products and the generative AI model. The output is the suggested recipe.

[1224] Step 7:

[1225] The server provides route guidance to the store with the cheapest product. The server uses the geopy library and Nominatim service to calculate the route from the consumer's current location to the store offering the cheapest product and generate a navigation link. The input is the consumer's location and the store's address. The output is the generated navigation link.

[1226] Step 8:

[1227] The server provides coupon information related to the cheapest product. It calls the coupon service API to obtain the coupon related to the cheapest product and notifies the smartphone. The input is the cheapest product and store information. The output is the obtained coupon information.

[1228] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1229] This invention is a system that combines an emotion engine that recognizes the user's emotions to enable consumers to efficiently purchase products at optimal prices in their daily lives. This system allows consumers to input their place of residence and the product they wish to purchase, and uses price information collected from multiple stores in the area to search for and recommend the cheapest products. Furthermore, the emotion engine can analyze the user's emotions and provide personalized recommended products and services based on the results.

[1230] Consumer Features

[1231] Residence registration

[1232] A user accesses the smartphone app and enters the postal code or city where they live. The user's device sends this information to the server, which then stores the residential address information in a database.

[1233] Enter a list of products

[1234] The user enters a list of items they wish to purchase into the app (e.g., "tomatoes, milk, eggs"), and the user's device sends this list to the server.

[1235] Collecting flyer information

[1236] The server collects the latest flyer information from multiple stores in the area based on the registered residential address, including through web scraping and social media. The server also analyzes images of paper flyers provided by consumers using OCR technology and integrates price information into the database.

[1237] Search for the lowest price

[1238] The server searches the database for the lowest priced items based on the product list entered by the user, and sends the search results to the user's terminal, where the lowest price information is displayed.

[1239] Recipe suggestions

[1240] The server searches for relevant recipes from a recipe database based on the shopping list entered by the user and displays them on the user's terminal. For example, if the user enters "tomato, milk, egg," the server will suggest a "tomato omelette" recipe and display its ingredient list.

[1241] Retail Features

[1242] Sales forecast

[1243] The server collects past shopping list data from consumers and begins analysis. Based on the analysis results, it generates sales forecasts and notifies the store's management terminal.

[1244] Coupon distribution and sale announcements

[1245] The server periodically collects and analyzes sales information from nearby stores. It then generates appropriate coupons based on store visit predictions and provides them to consumers. For example, based on sales information for a 2-liter carton of milk, a "10% off coupon when purchasing milk" is generated and notified to the user.

[1246] Inventory management

[1247] The server calculates the appropriate inventory level based on sales forecasts and, in cooperation with the inventory management system, suggests the appropriate amount of stock to the store. For example, if the sales forecast for tomatoes is "300 units next Monday," the server will suggest to the store that they "purchase an additional 200 tomatoes."

[1248] Emotion Engine Functions

[1249] User sentiment analysis

[1250] The emotion engine analyzes the user's emotions using data collected while the user is using the app (e.g., facial recognition data, typing speed and patterns, etc.). The analysis results reflect the user's state in real time and are reflected in the system.

[1251] Providing personalized product and service recommendations

[1252] The server then provides personalized product recommendations and services based on the user's emotions analyzed by the emotion engine. For example, if the user is feeling stressed, the server suggests products and services that have a relaxing effect.

[1253] Providing coupon and sale information at the optimal time

[1254] The server then provides coupon and sale information at the optimal time based on the analysis results of the emotion engine. For example, it executes marketing strategies according to the user's emotional state, such as providing special discount coupons only when the user is in a good mood.

[1255] Specific examples

[1256] As a concrete example, suppose a user uses a smartphone app to register their place of residence and enter "tomatoes, milk, eggs" as part of their weekend shopping list, and the emotion engine detects the user's stress during this process. The server collects the latest flyer information from stores in Shinjuku Ward, searches for the lowest prices, and provides them to the user. At the same time, recipes and products with a relaxing effect are suggested, and special coupons based on sale information are provided to the user. Based on this data, the store can implement appropriate inventory management and sales strategies.

[1257] In this way, the system, combined with the emotion engine, provides more advanced support to both consumers and retailers, improving personalization and efficiency.

[1258] The processing flow will be explained below.

[1259] Consumer Features

[1260] Residence registration

[1261] Step 1:

[1262] The user accesses the smartphone app and enters the postal code or city where they live.

[1263] Step 2:

[1264] The user terminal transmits the entered residence information to the server.

[1265] Step 3:

[1266] The server stores the user's location information in a database.

[1267] Enter a list of products

[1268] Step 4:

[1269] A user enters a list of items they want to buy into the app (e.g., "tomatoes, milk, eggs").

[1270] Step 5:

[1271] The user terminal transmits the input product list to the server.

[1272] Collecting flyer information

[1273] Step 6:

[1274] The server collects flyer information from each store's website and social media based on a pre-registered list of local stores.

[1275] Step 7:

[1276] The server uses web scraping technology to extract flyer information and store it in a database.

[1277] Step 8:

[1278] The user takes a photo of a paper flyer and uploads it to the app.

[1279] Step 9:

[1280] The user terminal transmits the uploaded flyer image to the server.

[1281] Step 10:

[1282] The server uses OCR (Optical Character Recognition) technology to extract price information from the image.

[1283] Step 11:

[1284] The server integrates the extracted price information into a database.

[1285] Search for the lowest price

[1286] Step 12:

[1287] The server searches the database for the cheapest products based on the product list entered by the user.

[1288] Step 13:

[1289] The server sends the search results to the user terminal.

[1290] Step 14:

[1291] The user's device displays the lowest price information (e.g., "Tomatoes: Ito-Yokado 100 yen / each").

[1292] Recipe suggestions

[1293] Step 15:

[1294] The server searches the recipe database for relevant recipes based on the user's shopping list.

[1295] Step 16:

[1296] The server sends the found recipe and its ingredient list to the user terminal.

[1297] Step 17:

[1298] The user device displays the recipe information (e.g., "Tomato Omelette: Tomato, Egg, Salt, Pepper").

[1299] Retail Features

[1300] Sales forecast

[1301] Step 18:

[1302] The server collects past consumer shopping list data and begins analyzing it.

[1303] Step 19:

[1304] The server generates a sales forecast based on the analysis results (e.g., "Tomato sales are predicted to increase next Monday").

[1305] Step 20:

[1306] The server transmits the generated sales forecast to the store's management terminal and notifies it.

[1307] Coupon distribution and sale announcements

[1308] Step 21:

[1309] The server periodically collects and analyzes sales information from nearby stores.

[1310] Step 22:

[1311] The server generates appropriate coupons based on store visit predictions (e.g., "10% off coupon when purchasing 2 liters of milk").

[1312] Step 23:

[1313] The server transmits the generated coupon information to the user terminal and performs a push notification.

[1314] Inventory management

[1315] Step 24:

[1316] The server calculates the appropriate inventory quantity based on the sales forecast.

[1317] Step 25:

[1318] The server connects with the inventory management system and suggests appropriate purchase quantities to the store (e.g., "Purchase an additional 200 tomatoes").

[1319] Emotion Engine Functions

[1320] User sentiment analysis

[1321] Step 26:

[1322] While the user is using the app, the emotion engine collects the user's facial expression data, typing speed and patterns.

[1323] Step 27:

[1324] The user terminal transmits the collected data to the server.

[1325] Step 28:

[1326] The server uses an emotion engine to analyze the user's emotions in real time.

[1327] Providing personalized product and service recommendations

[1328] Step 29:

[1329] The server determines personalized recommended products and services based on the analysis results of the emotion engine.

[1330] Step 30:

[1331] The server sends information about recommended products and services to the user terminal.

[1332] Step 31:

[1333] The user's device displays personalized product and service recommendations (e.g., if the user is feeling stressed, it suggests products that have a relaxing effect).

[1334] Providing coupon and sale information at the optimal time

[1335] Step 32:

[1336] The server generates coupon and sale information at the optimal time based on the analysis results of the emotion engine.

[1337] Step 33:

[1338] The server transmits the generated coupon and sale information to the user terminal.

[1339] Step 34:

[1340] The user's device notifies and displays coupons and sales information (e.g., offering special discount coupons when the user is in a good mood).

[1341] Example 2

[1342] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1343] Conventional price comparison systems simply collect price information from multiple stores and are unable to make suggestions tailored to consumer sentiment or individual needs. Furthermore, they lacked a mechanism for integrating information from flyer images taken by consumers into the system, making it difficult to reflect the latest price information. Furthermore, they lacked the functionality to integrate the collected information into a database and provide related recipes based on the shopping list entered by the consumer.

[1344] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1345] In this invention, the server includes: a means for a consumer to input their place of residence; a means for a consumer to input a list of products they wish to purchase; a means for collecting information from multiple local sales locations; a means for analyzing the collected information and extracting and integrating price information; a means for searching for the cheapest products based on the list of products they wish to purchase; a means for displaying the search results on the consumer's terminal; a means including an engine for analyzing consumer sentiment; and a means for providing personalized recommended products and services based on the analysis results. This enables the provision of personalized services according to the consumer's sentiments and needs, enabling more advanced price comparisons and shopping optimization. Furthermore, the server can analyze flyer images provided by consumers and integrate the latest price information into the database, thereby providing more accurate and up-to-date price information.

[1346] A "consumer" is an individual who purchases goods or services.

[1347] "Residence" is geographic information that indicates where a consumer currently lives.

[1348] "Point of sale" means a store or market that offers goods or services.

[1349] "Information gathering" is the process of gathering data from different sources.

[1350] "Analysis" refers to examining and processing collected information to extract meaningful data.

[1351] A "product list" is a list of products that a consumer wishes to purchase.

[1352] "Price information" is data regarding the selling price of a product or service.

[1353] "Search" is the act of finding information that matches specific conditions from a vast amount of data.

[1354] A "consumer terminal" is a digital device used by a consumer, such as a computer or smartphone.

[1355] An "emotion engine" is a technology for analyzing consumer emotions and their state.

[1356] "Recommended Products" are products that are specifically recommended based on a consumer's needs and preferences.

[1357] "Services" refers to support activities provided to consumers other than the products sold.

[1358] "Integration" refers to bringing together different data into one system or dataset.

[1359] "OCR technology" is a technology that converts scanned documents into machine-readable text data.

[1360] A "flyer image" is an image file of a paper printed for advertising or promotion.

[1361] A "recipe" is a set of instructions that lists the ingredients and steps for making a particular dish.

[1362] "Related recipes" are recipes for dishes related to the user's shopping list.

[1363] This invention is a system that enables consumers to efficiently purchase products at the optimal price. Specifically, the system allows consumers to input their place of residence and the product they wish to purchase, and combines price information collected from multiple local sales locations with an emotion engine to suggest the most suitable product.

[1364] This system mainly involves the exchange of information between the server, the device, and the consumer. The main hardware used to implement the system is the server, the consumer device (smartphone or computer), and network equipment. The software used includes Python, BeautifulSoup, Scrapy, MySQL, Tesseract OCR, OpenCV, analysis tools (pandas, scikit-learn), and web scraping tools.

[1365] First, a user accesses the smartphone app and enters the postal code or city where they live. This information is sent from the device to the server, which stores the residence information in a database. Next, the user enters a list of items they wish to purchase into the app, and the device sends this list to the server.

[1366] The server collects the latest flyer information from multiple local stores based on the registered residential address information. This collection process uses web scraping technology using Python's BeautifulSoup and Scrapy. In addition, images of paper flyers provided by consumers are analyzed using OCR technology (Tesseract OCR), and price information is integrated into the database.

[1367] The server then searches the database for the cheapest products based on the product list entered by the user, using the MySQL database management system, and sends the search results to the consumer's terminal, where the best price information is displayed to the user.

[1368] Furthermore, based on the shopping list entered by the user, the server searches for related recipes from the recipe database and displays them on the user's terminal. For example, if the user enters "tomato, milk, egg" in the list, the server will suggest a recipe for "tomato omelette."

[1369] For retail stores, the server collects shopping list data from consumers and uses analytical tools to predict sales. The analysis results are sent to the store's management terminal, which then implements appropriate inventory management and sales strategies. The server also collects and analyzes sales information from nearby stores, and generates appropriate coupons based on the predictions to provide to consumers.

[1370] Furthermore, the emotion engine analyzes data collected while the user is using the app (such as facial recognition data and input speed) to evaluate the user's emotions. This analysis is performed using OpenCV and other analytical tools. Based on the analysis results, the server provides personalized product and service recommendations according to the user's emotions. For example, if the user is feeling stressed, it will suggest products and services that have a relaxing effect.

[1371] As a concrete example, suppose a user uses a smartphone app to register their place of residence and enter "tomatoes, milk, and eggs" as part of their weekend shopping list, and the emotion engine detects the user's stress during this process. The server collects the latest flyer information from sales locations in Shinjuku Ward, searches for the lowest prices, and provides them to the user. At the same time, recipes and products with a relaxing effect are suggested, and special coupons based on sale information are notified to the user. Based on this data, the sales locations can implement appropriate inventory management and sales strategies.

[1372] In this way, the system, combined with the emotion engine, provides more advanced support to both consumers and points of sale, improving personalization and efficiency.

[1373] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1374] Step 1:

[1375] The user accesses the smartphone app and enters the postal code or city where they live.

[1376] Input: Postal code or city of residence

[1377] Output: Input data is saved on the user's device

[1378] Specific operation: The user enters the postal code and city / town name into the app's residence input form and presses the "Submit" button.

[1379] Step 2:

[1380] The terminal sends this information to the server.

[1381] Input: User-entered residential address information

[1382] Output: Location information is sent to the server

[1383] Specific operation: The device sends data to the server via the Internet.

[1384] Step 3:

[1385] The server stores the residence information in a database.

[1386] Input: Residential information

[1387] Output: Residence information is saved in the database

[1388] Specific operation: The server executes an SQL query to store the received residence information in a database.

[1389] Step 4:

[1390] A user enters a list of items they want to buy into the app (e.g., "tomatoes, milk, eggs").

[1391] Input: List of items you wish to purchase

[1392] Output: Input data is saved on the user's device

[1393] Specific operation: The user enters the product name into the app's shopping list input form and presses the "Submit" button.

[1394] Step 5:

[1395] The terminal sends this list to the server.

[1396] Input: A list of items the user wishes to purchase

[1397] Output: A list of items to purchase is sent to the server

[1398] Specific operation: The device sends data to the server via the Internet.

[1399] Step 6:

[1400] The server collects the latest flyer information from multiple sales locations in the area based on the registered residential address information.

[1401] Input: Residential information

[1402] Output: Collected flyer information

[1403] Specific operation: Based on the specified residence information, the server performs web scraping using Python's BeautifulSoup and Scrapy to collect flyer information from the websites of each sales location.

[1404] Step 7:

[1405] The server receives an image of a paper flyer provided by a consumer.

[1406] Input: Flyer image

[1407] Output: Flyer images are saved on the server

[1408] Specific operation: A user takes a photo of a flyer with their smartphone and uploads the image through the app. The server receives the image.

[1409] Step 8:

[1410] The server analyzes the flyer image using OCR technology (e.g., Tesseract OCR) and integrates the price information into a database.

[1411] Input: Flyer image

[1412] Output: Price information in the database

[1413] Specific operation: The server performs OCR analysis, extracts text data from the flyer image, and parses price information from it. The parsed data is then integrated into a database.

[1414] Step 9:

[1415] The server searches the database for the cheapest products based on the product list entered by the user.

[1416] Input: List of products you wish to purchase, price information

[1417] Output: Information on the cheapest product

[1418] What it does: The server queries the database for pricing information to find the cheapest item. It uses Python data analysis tools to calculate the lowest price.

[1419] Step 10:

[1420] The server sends the search results to the user's terminal and displays the lowest price information.

[1421] Input: Information on the lowest priced item

[1422] Output: The lowest price information displayed on the user's terminal

[1423] Specific operation: The server sends the search results to the user's device, and the app displays them.

[1424] Step 11:

[1425] The server searches for related recipes from a recipe database based on the shopping list entered by the user and displays them on the user terminal.

[1426] Input: List of items you wish to purchase

[1427] Output: Related recipe information

[1428] Specific operation: The server searches the recipe database for relevant recipes based on the shopping list. The searched recipe information is sent to the user's terminal and displayed.

[1429] Step 12:

[1430] The server collects past consumer shopping list data and begins analyzing it.

[1431] Input: Past shopping list data

[1432] Output: Analysis results (sales forecast)

[1433] What it does: The server runs a Python analytics tool to analyze historical consumer shopping list data in the database and generate sales forecasts.

[1434] Step 13:

[1435] The server periodically collects and analyzes sales information from nearby stores.

[1436] Input:Sale information

[1437] Output: Parsed sale information, generated coupons

[1438] What it does: The server periodically runs a web script to collect and analyze sales information from nearby stores.

[1439] Step 14:

[1440] The server generates appropriate coupons based on the store visit prediction and provides them to the consumer terminal.

[1441] Input: store visit prediction, sales information

[1442] Output: Generated coupon

[1443] Specific operation: The server executes an algorithm to generate appropriate coupons based on store visit predictions. The generated coupons are sent to the consumer's terminal and notified.

[1444] Step 15:

[1445] The emotion engine analyzes data collected while users are using the app.

[1446] Input: User operation data (face recognition data, input speed, etc.)

[1447] Output: User sentiment analysis results

[1448] Specific operation: The emotion engine collects user operation data and analyzes the emotional state using OpenCV and data analysis tools.

[1449] Step 16:

[1450] The server provides personalized recommended products and services based on the user's emotions analyzed by the emotion engine.

[1451] Input: Sentiment analysis results

[1452] Output: Personalized product and service recommendations

[1453] Specific operation: The server refers to the sentiment analysis results and executes a personalized recommendation algorithm to suggest appropriate products and services. The suggestions are sent to the user's device and displayed.

[1454] Step 17:

[1455] Based on the analysis results of the emotion engine, the server generates and provides coupon and sale information at the optimal time.

[1456] Input: Sentiment analysis results, sales information

[1457] Output: Generated coupons and sales information

[1458] Specific operation: The server uses the results of the sentiment analysis to run an algorithm that generates coupon and sale information at the optimal time. The generated information is then sent to the user's device and notified.

[1459] (Application example 2)

[1460] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1461] In the past, consumers had to visit multiple stores to find the best price, which was time-consuming and labor-intensive. Furthermore, the means for researching price information were limited, making efficient shopping difficult. Furthermore, personalized product and service recommendations based on the consumer's emotional state were not provided, making it difficult to improve the consumer experience.

[1462] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: a means for a consumer to input their place of residence; a means for inputting a list of products the consumer wants to buy; a means for collecting flyer information from multiple stores in the area based on the place of residence; a means for searching for the cheapest products from the collected flyer information based on the list of products the consumer wants to buy; a means for displaying the search results on the consumer terminal; a means for analyzing the consumer's emotions; a means for providing personalized recommended products and services based on the results of the emotion analysis; and a means for displaying the personalized recommended products and services on the consumer terminal. This enables consumers to efficiently purchase products at optimal prices and receive personalized suggestions based on their emotional state.

[1463] "Means for entering residential address" refers to the interface that allows consumers to register their residential address information in the system. Examples of such interfaces include smart glasses and smartphone apps.

[1464] The "means for inputting a list of products to be purchased" refers to an interface that allows consumers to register a list of products they wish to purchase in the system. This includes a touch screen and voice recognition.

[1465] "Methods for collecting flyer information" are methods for collecting the latest price and sale information from multiple stores in the area. Flyer images are analyzed using web scraping and OCR technology.

[1466] The "means for searching for the cheapest products" refers to algorithms or software that search for the lowest priced products from among the products entered by the consumer based on collected flyer information.

[1467] "Means for displaying search results on consumer devices" refers to an interface for presenting the lowest price information found on the consumer's device (such as smart glasses or a smartphone).

[1468] "Emotion analysis tools" are software or algorithms that analyze a consumer's facial expressions and behavioral data to determine their current emotional state. This may include cameras and sensors.

[1469] "Means for providing personalized product and service recommendations" refers to algorithms and software that suggest the most appropriate products and services based on the consumer's emotional state.

[1470] The "means for displaying personalized recommended products and services on a consumer terminal" is an interface for displaying recommended products and services related to sentiment analysis on a terminal used by a consumer.

[1471] The present invention is a system for supporting consumers in purchasing products efficiently at optimal prices in their daily lives. Specific embodiments of the system will be described below.

[1472] Implementation system configuration

[1473] Consumer Devices

[1474] Consumers use smart glasses or a smartphone app to input their location and a list of items they want to buy. The device has an interface that allows input via touchscreen or voice recognition.

[1475] server

[1476] The server receives information on the consumer's location and product list from the consumer's device and collects the latest flyer information from multiple stores in the area. This information is collected by integrating flyer information from online and paper sources using web scraping and OCR technologies. An algorithm is then run to search for the cheapest product among the items the consumer wants to buy, based on the price information stored in the database.

[1477] Furthermore, the server is equipped with an emotion analysis engine that recognizes emotions in real time by analyzing data such as consumer facial images and input patterns. Based on the results of this emotion analysis, personalized product and service recommendations are generated and displayed on the consumer's device.

[1478] Data processing and calculation

[1479] Sentiment Analysis Engine

[1480] This engine analyzes consumer facial image data collected using cameras and sensors, as well as the speed and patterns of their actions. Major technologies used include OpenCV (for facial recognition) and TensorFlow (for emotion recognition). Consumer emotions (such as stress or happiness) are detected in real time by this engine and provided as data to a server.

[1481] Price information collection and integration

[1482] The server automatically collects price information from websites and paper sources using web scraping and OCR technologies, specifically BeautifulSoup (for web scraping) and Tesseract OCR (for character recognition), and stores it in a database.

[1483] Best price search and recommendations

[1484] The server uses the product list and address information entered by the consumer to screen price information in the database and find the cheapest products. Data processing libraries such as Python's pandas and SQL are used. In addition, by combining consumer sentiment and individual product information, an algorithm is run that individually recommends the best products and services for each specific consumer.

[1485] Specific examples

[1486] Suppose a consumer uses smart glasses to input their place of residence (e.g., "Shinjuku Ward") and a list of products they wish to buy (e.g., "tomatoes, milk, eggs"). During this process, the smart glasses' camera captures the consumer's facial expression and sends the data to a server. If the emotion analysis engine detects stress, the server will suggest products or services with a relaxing effect (e.g., "aromatherapy oil") in addition to the usual lowest price information.

[1487] Prompt Sentence Examples

[1488] "I live in Shinjuku Ward, and I'd like to buy tomatoes, milk, and eggs. Next, I'll provide an image of my facial expression taken by the camera on my smart glasses. Based on this information, please tell me the lowest prices in the area and recommend products based on my emotions."

[1489] In this way, the embodiment of the invention realizes a system that reflects the consumer's input information and emotional state, collects and analyzes optimal price information from multiple stores, and provides personalized recommended products in real time.

[1490] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1491] Step 1:

[1492] A user uses a consumer device (smart glasses or a smartphone app) to input their location and a list of products they wish to purchase. The input information is then sent from the device to a server. Specifically, the user uses the device's interface to input information using a touch screen or voice recognition. The input data is in the form of text, which is then sent to the server.

[1493] Step 2:

[1494] Based on the received residential address information, the server collects the latest flyer and price information from multiple stores in the area. This collection is performed using web scraping and OCR technologies. BeautifulSoup is used for web scraping to extract price information from online store websites. Tesseract OCR is also used to recognize and extract characters from paper flyer images. The collected data is then stored in a database.

[1495] Step 3:

[1496] The server searches for the cheapest products that match the product list entered by the user based on the price information stored in the database. This process involves filtering the data using Python's pandas library and executing database queries using SQL. The search results are returned in text format as the lowest price information for the target products and are sent to the consumer's device.

[1497] Step 4:

[1498] The consumer device displays the lowest price information sent from the server to the user. In the case of smart glasses, the price information is displayed in the user's field of view using holographic display technology. In the case of smartphones, the price information is displayed in text and graphical format on the screen, allowing the user to visually check the cheapest product information.

[1499] Step 5:

[1500] Consumer emotion data is collected in real time by the camera and sensors on the consumer device. Specifically, facial image data captured by the camera is processed within the device, and the results are sent to a server. This data is analyzed using an emotion recognition model that combines OpenCV and TensorFlow on the edge device. The analysis results are sent to the server as text data.

[1501] Step 6:

[1502] The server receives the results of the emotion analysis and generates personalized product and service recommendations based on them. If the emotion indicates stress, products and services with a relaxing effect will be recommended. This recommendation process is performed by an algorithm using a generative AI model, and the recommendation results are obtained as text data.

[1503] Step 7:

[1504] The server sends the generated information about recommended products and services to the consumer's device, which then displays the received information to the user. In the case of smart glasses, the recommended product information is displayed in the user's field of vision using holographic display technology. In the case of smartphones, the recommended information is displayed in text and graphical format on the screen, allowing the user to learn about the best products and services based on their emotional state.

[1505] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1506] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1507] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1508] [Fourth embodiment]

[1509] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1510] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1511] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1512] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1513] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1514] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1515] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1516] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1517] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1518] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1519] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1520] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1521] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1522] This invention is a system that allows consumers to efficiently purchase products at optimal prices in their daily lives. The system allows consumers to input their location and the product they wish to purchase, and uses price information collected from multiple stores in the area to search for and recommend the cheapest products. The system can also analyze the collected data to suggest recipes, forecast sales, and manage inventory.

[1523] Consumer Features

[1524] Residence registration

[1525] The user registers their local area (e.g., Shinjuku-ku, Tokyo). The user enters their area using the smartphone app interface.

[1526] Enter a list of products

[1527] Users register a list of items they want to buy, such as "tomatoes, milk, eggs," through the app.

[1528] Collecting flyer information

[1529] The server collects the latest flyer information from multiple stores in the area based on the registered residential address. This includes web scraping and social media information gathering. The server also analyzes images of paper flyers provided by consumers using OCR technology and integrates price information into the database.

[1530] Search for the lowest price

[1531] The server searches for the cheapest product entered by the user from the flyer information collected by the server. The server sends the search results to the user's terminal and displays them. For example, if the cheapest tomato price is "100 yen per unit at Ito-Yokado," that information is provided to the user.

[1532] Recipe suggestions

[1533] The server searches for relevant recipes based on the shopping list entered by the user and displays them on the user's device. For example, if "tomato, milk, egg" is entered, the server will suggest a recipe for "tomato omelette" and display its ingredients list.

[1534] Retail Features

[1535] Sales forecast

[1536] The server collects and analyzes consumer shopping list data and makes sales forecasts. For example, if many users plan to buy tomatoes, the server predicts that demand for tomatoes will be high and notifies stores.

[1537] Coupon distribution and sale announcements

[1538] The server obtains sales information for nearby stores, generates coupons based on predicted store visits, and provides them to consumers. For example, based on sales information for a 2-liter pack of milk, a 10% off coupon for purchasing milk is generated and notified to the user.

[1539] Inventory management

[1540] The server calculates the appropriate inventory quantity based on the sales forecast and connects to the inventory management system to suggest the appropriate amount to purchase. For example, if the sales forecast for tomatoes is "300 units next Monday," the server will suggest to the store to "purchase an additional 200 tomatoes."

[1541] Specific examples

[1542] As a concrete example, suppose a user uses a smartphone app to register the area where they live and enter "tomatoes, milk, eggs" as part of their weekend shopping list. The server collects the latest flyer information from stores in Shinjuku Ward, searches for the lowest prices, and generates a list such as "tomatoes for 100 yen at Ito-Yokado, milk for 150 yen at Aeon, and eggs for 180 yen at a nearby greengrocer." It also suggests a recipe for "tomato omelette." Based on this data, retailers can distribute sale coupons and manage inventory appropriately.

[1543] In this way, the system provides optimal support for both consumers and retailers.

[1544] The processing flow will be explained below.

[1545] Consumer Features

[1546] Residence registration

[1547] Step 1:

[1548] The user accesses the smartphone app and enters the postal code or city where they live.

[1549] Step 2:

[1550] The user terminal transmits the input information to the server.

[1551] Step 3:

[1552] The server stores the user's location information in a database.

[1553] Enter a list of products

[1554] Step 4:

[1555] A user enters a list of items they want to buy into the app (e.g., "tomatoes, milk, eggs").

[1556] Step 5:

[1557] The user terminal transmits the input product list to the server.

[1558] Collecting flyer information

[1559] Step 6:

[1560] The server collects flyer information from each store's website and social media based on a pre-registered list of local stores.

[1561] Step 7:

[1562] The server uses web scraping technology to extract flyer information and store it in a database.

[1563] Step 8:

[1564] The user takes a photo of a paper flyer and uploads it to the app.

[1565] Step 9:

[1566] The user terminal transmits the uploaded flyer image to the server.

[1567] Step 10:

[1568] The server uses OCR (Optical Character Recognition) technology to extract price information from the image.

[1569] Step 11:

[1570] The server integrates the extracted price information into a database.

[1571] Search for the lowest price

[1572] Step 12:

[1573] The server searches the database for the cheapest products based on the product list entered by the user.

[1574] Step 13:

[1575] The server sends the search results to the user terminal.

[1576] Step 14:

[1577] The user's device displays the lowest price information (e.g., "Tomatoes: Ito-Yokado 100 yen / each").

[1578] Recipe suggestions

[1579] Step 15:

[1580] The server searches a recipe database for relevant recipes based on the user's shopping list.

[1581] Step 16:

[1582] The server sends the found recipe and its ingredient list to the user terminal.

[1583] Step 17:

[1584] The user device displays the recipe information (e.g., "Tomato Omelette: Tomato, Egg, Salt, Pepper").

[1585] Retail Features

[1586] Sales forecast

[1587] Step 18:

[1588] The server collects past consumer shopping list data and begins analyzing it.

[1589] Step 19:

[1590] The server generates a sales forecast based on the analysis results (e.g., "Tomato sales are predicted to increase next Monday").

[1591] Step 20:

[1592] The server transmits the generated sales forecast to the store's management terminal and notifies it.

[1593] Coupon distribution and sale announcements

[1594] Step 21:

[1595] The server periodically collects and analyzes sales information from nearby stores.

[1596] Step 22:

[1597] The server generates appropriate coupons based on predicted store visits (e.g., "10% off coupon when purchasing 2 liters of milk").

[1598] Step 23:

[1599] The server transmits the generated coupon information to the user terminal and performs a push notification.

[1600] Inventory management

[1601] Step 24:

[1602] The server calculates the appropriate inventory quantity based on sales forecasts.

[1603] Step 25:

[1604] The server connects with the inventory management system and suggests appropriate purchase quantities to the store (e.g., "Purchase an additional 200 tomatoes").

[1605] The above processing steps enable consumers to purchase products at the best price and enable retailers to efficiently manage inventory and forecast sales.

[1606] Example 1

[1607] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1608] Today's consumers are required to browse numerous stores and online platforms to find the cheapest prices for products. However, manually collecting and comparing advertising information from individual stores is time-consuming and inefficient. Furthermore, there is a lack of ways to integrate and analyze multiple advertising information, making it difficult for consumers to find the cheapest products. Furthermore, there is a lack of related cooking suggestions, discount information to promote purchases, and inventory optimization in retail stores.

[1609] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1610] In this invention, the server includes a means for inputting a consumer's place of residence, a means for inputting a list of products the consumer wishes to buy, a means for collecting advertising information from multiple stores in the area based on the consumer's place of residence, a means for searching for the cheapest products from the collected advertising information based on the list of products the consumer wishes to buy, a means for displaying the search results on the consumer's terminal, and a means for analyzing the advertising information and integrating the price information into a database. This allows consumers to efficiently find the cheapest products and receive related cooking suggestions and discount information to promote purchases based on the results. It also enables retail stores to optimize inventory based on sales forecasts.

[1611] A "means for inputting a place of residence" is a means for providing an interface for a consumer to input the area in which they live.

[1612] The "means for inputting a list of products to be purchased" is a means for providing an interface for a consumer to input products that he or she intends to purchase.

[1613] The "means for collecting advertising information" is a means for automatically acquiring the latest advertising information from a plurality of local stores based on residential location information.

[1614] The "means for analyzing advertising information" is a means for deciphering the collected advertising information and converting it into usable price data.

[1615] "Means for integrating price information into a database" refers to a means for organizing and storing analyzed price data in a single database.

[1616] The "means for searching for the cheapest product" is a means for searching for the cheapest product among the products input by the user from the collected and analyzed advertising information.

[1617] The "means for displaying search results on a consumer terminal" refers to a means for displaying the searched lowest price product information on the consumer's electronic device.

[1618] The "means for searching for a recipe" is a means for searching for a related recipe based on the product list entered by the user and the searched lowest price product information.

[1619] The "means for providing discount coupons or promotional information for promoting purchases" is a means for the server to notify consumers of discount coupons or special sale information based on predicted data.

[1620] "Means for optimizing inventory" refers to means for calculating and proposing the appropriate inventory amount for a store based on predicted sales data.

[1621] This invention is a system that enables consumers to efficiently purchase products at optimal prices in their daily lives. The system allows consumers to input their location and the product they wish to purchase, and uses advertising information collected from multiple stores in the area to search for and recommend the cheapest products. The system can also analyze the collected data to suggest related recipes, forecast sales, and manage inventory.

[1622] Consumer Features

[1623] Residence registration

[1624] The user registers their local area (e.g., Shinjuku-ku, Tokyo). The user enters their area using the smartphone app interface. This information is sent to the server and stored.

[1625] Enter a list of products

[1626] The user registers a list of items they want to buy. The user inputs items such as "tomatoes, milk, eggs" through the app. This information is also sent to the server and stored.

[1627] Collecting advertising information

[1628] The server collects the latest advertising information from multiple stores in the area based on the registered residential address. This includes web scraping technology, APIs, and social media information collection. The server also analyzes images of print advertisements provided by consumers using OCR (optical character recognition) technology and integrates the price information into a database. This database centrally manages the price information collected from multiple stores.

[1629] Search for the lowest price

[1630] The server searches for the cheapest product entered by the user from the advertising information collected. For example, if the cheapest price for tomatoes is "100 yen per unit at a specific supermarket," that information is sent to the user's device and displayed.

[1631] Recipe suggestions

[1632] The server searches for related recipes based on the shopping list entered by the user and displays them on the user's terminal. For example, if "tomato, milk, egg" is entered, the server will suggest a recipe for "tomato omelette" and display its ingredients list.

[1633] Retail Features

[1634] Sales forecast

[1635] The server collects and analyzes consumer shopping list data and makes sales forecasts. For example, if many users plan to buy tomatoes, the server predicts that demand for tomatoes will be high and notifies stores.

[1636] Coupon distribution and sale announcements

[1637] The server obtains sales information for nearby stores, generates discount coupons based on predicted store visits, and provides them to consumers. For example, based on sales information for a 2-liter pack of milk, a 10% discount coupon for purchasing milk is generated and notified to the user.

[1638] Inventory management

[1639] The server calculates the appropriate inventory quantity based on the sales forecast and connects to the inventory management system to suggest the appropriate amount to purchase. For example, if the sales forecast for tomatoes is "300 units next Monday," the server will suggest to the store to "purchase an additional 200 tomatoes."

[1640] Specific examples

[1641] As a concrete example, suppose a user uses a smartphone app to register the area where they live and enter "tomatoes, milk, eggs" as part of their weekend shopping list. The server collects the latest advertising information from multiple stores in the registered area and searches for the lowest prices. In this example, a list is generated such as "tomatoes for 100 yen at a specific supermarket, milk for 150 yen at another supermarket, and eggs for 180 yen at a local store." In addition, a recipe for "tomato omelette" is suggested. Retailers can use this data to distribute discount coupons and manage inventory appropriately.

[1642] Example prompts for generative AI models

[1643] "I live in a particular area. I want to buy tomatoes, milk, and eggs on the weekend. Which store is the cheapest?"

[1644] "Please tell me some recipes that use tomatoes, milk, and eggs."

[1645] In this way, the system provides optimal support for both consumers and retailers.

[1646] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1647] Step 1: Register your residence

[1648] Input: The user uses a smartphone app to enter the area where they live (e.g., Shinjuku-ku, Tokyo).

[1649] Specific operation: The user enters their place of residence into the form within the app and presses the registration button. The device captures this information and sends it to the server.

[1650] Data processing or data calculation: The server stores the received residence information in a database and associates it with the user profile.

[1651] Output: The server stores the residence information and prepares it for use in the next processing step.

[1652] Step 2: Enter your shopping list

[1653] Input: The user enters a list of items they want to buy (e.g., tomatoes, milk, eggs).

[1654] Specific operation: The user enters a list of items into the shopping list entry screen within the app and presses the submit button. The device acquires this information and sends it to the server.

[1655] Data processing or data calculation: The server stores the received product list in a database and associates it with the user profile.

[1656] Output: The server stores the product listing information and prepares it for use in the next processing step.

[1657] Step 3: Collect advertising information

[1658] Input: Server gets location and product listing information.

[1659] How it works: The server uses web scraping, APIs, and social media information gathering technology to collect the latest advertising information from multiple stores in the area. Images of print advertisements provided by users are analyzed using OCR technology.

[1660] Data processing or data calculation: Analyzing the collected advertising data, extracting the necessary price information and integrating it into the database.

[1661] Output: The server stores the consolidated pricing information data in a database.

[1662] Step 4: Search for the best price

[1663] Input: The server retrieves the consolidated pricing data and the user's shopping list.

[1664] What happens: The server searches the price information in the database to find the cheapest price for the product entered by the user.

[1665] Data processing or data crunching: Using search algorithms to identify the cheapest products and organize their details.

[1666] Output: The server lists the cheapest products and sends them to the user's device.

[1667] Step 5: Recipe suggestions

[1668] Input: The server retrieves the user's shopping list information.

[1669] What happens: The server searches its recipe database to find recipes related to the user's shopping list.

[1670] Data processing or data calculation: Selecting the best recipe from the recipe database and preparing its details.

[1671] Output: The server sends the relevant recipe information to the user's device.

[1672] Step 6: View results and redeem coupons

[1673] Input: Receives the lowest price product information and cooking method information sent to the user's terminal.

[1674] Specific operation: The user browses the lowest price product information and cooking method information on the smartphone app. In addition, the user checks and uses the provided coupon information.

[1675] Data processing or data calculation: verifying the validity of coupon codes and recording their usage in a database.

[1676] Output: The user can efficiently purchase the cheapest products and use coupons.

[1677] As a result, the present invention provides optimal support to both consumers and retailers, and creates an environment in which consumers can shop efficiently.

[1678] (Application example 1)

[1679] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1680] To enable consumers to purchase products efficiently, it is necessary not only to collect product price information and find the cheapest products, but also to suggest recipes, provide navigation to the cheapest stores, and present related coupon information. However, existing systems have difficulty providing all of these in a centralized manner, making it difficult for consumers to make optimal purchases without hassle. Therefore, an objective of the present invention is to provide a comprehensive system that enables consumers to efficiently and optimally conduct purchasing activities.

[1681] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1682] In this invention, the server includes means for a consumer to input their place of residence, means for inputting a list of products the consumer wants to buy, means for collecting price information from a plurality of stores in the area based on the place of residence, means for searching for the cheapest products from the collected price information based on the list of products the consumer wants to buy, means for displaying the search results on the consumer terminal, means for suggesting recipes related to the consumer, and means for providing route guidance to the store with the cheapest products. This enables the consumer to efficiently purchase products at the best price with a single operation, and to obtain related recipe information, route guidance to the store, and even coupon information.

[1683] "Consumer" refers to an individual or household who purchases goods or services.

[1684] "Place of residence" refers to the area where a consumer lives and shops on a daily basis.

[1685] "List of Products" refers to a specific list of products that a consumer wishes to purchase.

[1686] "Store" means a physical location or online sales site where products are sold.

[1687] "Price information" refers to the sales price set by each store for the product.

[1688] "Collect" refers to gathering specified information and integrating it into a single piece of data.

[1689] "Searching" refers to finding the desired information based on input data and conditions.

[1690] "Displaying" refers to outputting information on a terminal in a form that can be viewed by a consumer.

[1691] A "recipe" refers to a document or data that explains how to prepare a dish using specific ingredients.

[1692] "Navigation" refers to providing directions or routes to reach a specified destination.

[1693] "Coupon information" refers to information that allows you to receive discounts on products or services under certain conditions.

[1694] A "generative AI model" refers to an artificial intelligence model that automatically generates new information or suggestions based on given data.

[1695] This system follows the steps below to enable consumers to purchase products efficiently at the best price. First, the user uses a smartphone application to input their address and a list of products they wish to purchase. This address and product list information is then sent to the server.

[1696] The server uses web scraping and OCR (Optical Character Recognition) technologies to collect price information from multiple local stores based on the collected information. Specifically, the requests and BeautifulSoup libraries are used for web scraping, and an appropriate OCR library is used for OCR analysis.

[1697] The collected price information is stored and integrated in a database that includes paper flyer images provided by consumers and data obtained through web scraping, allowing price information from a variety of sources to be managed in a unified manner.

[1698] The server searches the collected price information for the cheapest products based on the product list entered by the consumer and displays the search results on the consumer's smartphone, allowing the consumer to easily decide which store to purchase which products.

[1699] The server then suggests related recipes based on the input product list and the cheapest product information. For example, if a user inputs tomatoes, milk, and eggs, the server will search for related "tomato omelette" recipes and display them on the consumer's smartphone. This recipe suggestion uses a generative AI model to automatically generate the optimal recipe.

[1700] Additionally, the server utilizes a Geographic Information System (GIS) to provide route guidance to the store offering the cheapest product. Specifically, it uses the geopy library and the Nominatim service to generate and present to the consumer a navigation link to the store offering the cheapest product.

[1701] The server also provides coupon information related to the cheapest products, which is retrieved using the coupon service API and notified to the consumer.

[1702] As a concrete example, suppose a user launches a smartphone application and inputs their address ("Shinjuku Ward, Tokyo") and a list of items they wish to purchase ("Tomatoes, Milk, Eggs"). The server collects price information from stores in Shinjuku Ward, searches for the cheapest prices, and generates and presents a list such as "Tomatoes for 100 yen at Store A, Milk for 150 yen at Store B, and Eggs for 180 yen at Store C." It also suggests a recipe for "Tomato Omelette" and provides route guidance links to Stores A and B. The user is then notified of a "10% off coupon that can be used at Store A."

[1703] Example prompts to input to a generative AI model:

[1704] "Suppose a user living in Shinjuku Ward, Tokyo wants to buy tomatoes, milk, and eggs, and we want to collect information on the cheapest prices from local stores. As a result, we want to show the cheapest store and price. We also want to suggest recipes using these ingredients. We also want to provide a navigation link to the cheapest store and coupon information."

[1705] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1706] Step 1:

[1707] The user inputs their address and a list of products they wish to purchase. Using a smartphone application, the user inputs their address as "Shinjuku-ku, Tokyo" and then adds "tomatoes, milk, and eggs" to the list of products they wish to purchase. This information is sent from the device to the server. The input data includes the user's address and product list. The output is the user's address and product list information stored on the server.

[1708] Step 2:

[1709] The server collects price information from multiple stores in the area. Specifically, it uses web scraping technology (using requests and the BeautifulSoup library) to collect price information from store websites. It then uses OCR technology to analyze paper flyer images provided by consumers and extract price information. The input is the URL of the store website and the paper flyer image. The output is a dataset containing the collected price information.

[1710] Step 3:

[1711] The server saves and integrates the collected price information in a database. The collected price information is stored in a database and integrated based on the product list entered by the consumer. This database centralizes store information and its price information. The input is the price information collected in step 2. The output is the integrated database.

[1712] Step 4:

[1713] The server searches the integrated database for the cheapest products. Based on the product list entered by the consumer, the server runs an algorithm to find the cheapest price for each product based on the price information in the database. The input is the consumer's product list and the price information in the database. The output is a list of the cheapest products.

[1714] Step 5:

[1715] The server displays the search results on the consumer's smartphone. The server sends the list of cheapest products obtained in step 4 to the consumer's smartphone and displays it on the application. The input is the list of cheapest products. The output is the displayed information on the cheapest products.

[1716] Step 6:

[1717] The server suggests related recipes based on the input product list and information on the cheapest products. Using a generative AI model, it analyzes the consumer's product list and the ingredient information in the database to generate the optimal recipe. For example, it suggests a recipe for "tomato omelette" for a list of "tomatoes, milk, eggs." The input is the list of cheapest products and the generative AI model. The output is the suggested recipe.

[1718] Step 7:

[1719] The server provides route guidance to the store with the cheapest product. The server uses the geopy library and Nominatim service to calculate the route from the consumer's current location to the store offering the cheapest product and generate a navigation link. The input is the consumer's location and the store's address. The output is the generated navigation link.

[1720] Step 8:

[1721] The server provides coupon information related to the cheapest product. It calls the coupon service API to obtain the coupon related to the cheapest product and notifies the smartphone. The input is the cheapest product and store information. The output is the obtained coupon information.

[1722] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1723] This invention is a system that combines an emotion engine that recognizes the user's emotions to enable consumers to efficiently purchase products at optimal prices in their daily lives. This system allows consumers to input their place of residence and the product they wish to purchase, and uses price information collected from multiple stores in the area to search for and recommend the cheapest products. Furthermore, the emotion engine can analyze the user's emotions and provide personalized recommended products and services based on the results.

[1724] Consumer Features

[1725] Residence registration

[1726] A user accesses the smartphone app and enters the postal code or city where they live. The user's device sends this information to the server, which then stores the residential address information in a database.

[1727] Enter a list of products

[1728] The user enters a list of items they wish to purchase into the app (e.g., "tomatoes, milk, eggs"), and the user's device sends this list to the server.

[1729] Collecting flyer information

[1730] The server collects the latest flyer information from multiple stores in the area based on the registered residential address, including through web scraping and social media. The server also analyzes images of paper flyers provided by consumers using OCR technology and integrates price information into the database.

[1731] Search for the lowest price

[1732] The server searches the database for the lowest priced items based on the product list entered by the user, and sends the search results to the user's terminal, where the lowest price information is displayed.

[1733] Recipe suggestions

[1734] The server searches for relevant recipes from a recipe database based on the shopping list entered by the user and displays them on the user's terminal. For example, if the user enters "tomato, milk, egg," the server will suggest a "tomato omelette" recipe and display its ingredient list.

[1735] Retail Features

[1736] Sales forecast

[1737] The server collects past shopping list data from consumers and begins analysis. Based on the analysis results, it generates sales forecasts and notifies the store's management terminal.

[1738] Coupon distribution and sale announcements

[1739] The server periodically collects and analyzes sales information from nearby stores. It then generates appropriate coupons based on store visit predictions and provides them to consumers. For example, based on sales information for a 2-liter carton of milk, a "10% off coupon when purchasing milk" is generated and notified to the user.

[1740] Inventory management

[1741] The server calculates the appropriate inventory level based on sales forecasts and, in cooperation with the inventory management system, suggests the appropriate amount of stock to the store. For example, if the sales forecast for tomatoes is "300 units next Monday," the server will suggest to the store that they "purchase an additional 200 tomatoes."

[1742] Emotion Engine Functions

[1743] User sentiment analysis

[1744] The emotion engine analyzes the user's emotions using data collected while the user is using the app (e.g., facial recognition data, typing speed and patterns, etc.). The analysis results reflect the user's state in real time and are reflected in the system.

[1745] Providing personalized product and service recommendations

[1746] The server then provides personalized product recommendations and services based on the user's emotions analyzed by the emotion engine. For example, if the user is feeling stressed, the server suggests products and services that have a relaxing effect.

[1747] Providing coupon and sale information at the optimal time

[1748] The server then provides coupon and sale information at the optimal time based on the analysis results of the emotion engine. For example, it executes marketing strategies according to the user's emotional state, such as providing special discount coupons only when the user is in a good mood.

[1749] Specific examples

[1750] As a concrete example, suppose a user uses a smartphone app to register their place of residence and enter "tomatoes, milk, eggs" as part of their weekend shopping list, and the emotion engine detects the user's stress during this process. The server collects the latest flyer information from stores in Shinjuku Ward, searches for the lowest prices, and provides them to the user. At the same time, recipes and products with a relaxing effect are suggested, and special coupons based on sale information are provided to the user. Based on this data, the store can implement appropriate inventory management and sales strategies.

[1751] In this way, the system, combined with the emotion engine, provides more advanced support to both consumers and retailers, improving personalization and efficiency.

[1752] The processing flow will be explained below.

[1753] Consumer Features

[1754] Residence registration

[1755] Step 1:

[1756] The user accesses the smartphone app and enters the postal code or city where they live.

[1757] Step 2:

[1758] The user terminal transmits the entered residence information to the server.

[1759] Step 3:

[1760] The server stores the user's location information in a database.

[1761] Enter a list of products

[1762] Step 4:

[1763] A user enters a list of items they want to buy into the app (e.g., "tomatoes, milk, eggs").

[1764] Step 5:

[1765] The user terminal transmits the input product list to the server.

[1766] Collecting flyer information

[1767] Step 6:

[1768] The server collects flyer information from each store's website and social media based on a pre-registered list of local stores.

[1769] Step 7:

[1770] The server uses web scraping technology to extract flyer information and store it in a database.

[1771] Step 8:

[1772] The user takes a photo of a paper flyer and uploads it to the app.

[1773] Step 9:

[1774] The user terminal transmits the uploaded flyer image to the server.

[1775] Step 10:

[1776] The server uses OCR (Optical Character Recognition) technology to extract price information from the image.

[1777] Step 11:

[1778] The server integrates the extracted price information into a database.

[1779] Search for the lowest price

[1780] Step 12:

[1781] The server searches the database for the cheapest products based on the product list entered by the user.

[1782] Step 13:

[1783] The server sends the search results to the user terminal.

[1784] Step 14:

[1785] The user's device displays the lowest price information (e.g., "Tomatoes: Ito-Yokado 100 yen / each").

[1786] Recipe suggestions

[1787] Step 15:

[1788] The server searches the recipe database for relevant recipes based on the user's shopping list.

[1789] Step 16:

[1790] The server sends the found recipe and its ingredient list to the user terminal.

[1791] Step 17:

[1792] The user device displays the recipe information (e.g., "Tomato Omelette: Tomato, Egg, Salt, Pepper").

[1793] Retail Features

[1794] Sales forecast

[1795] Step 18:

[1796] The server collects past consumer shopping list data and begins analyzing it.

[1797] Step 19:

[1798] The server generates a sales forecast based on the analysis results (e.g., "Tomato sales are predicted to increase next Monday").

[1799] Step 20:

[1800] The server transmits the generated sales forecast to the store's management terminal and notifies it.

[1801] Coupon distribution and sale announcements

[1802] Step 21:

[1803] The server periodically collects and analyzes sales information from nearby stores.

[1804] Step 22:

[1805] The server generates appropriate coupons based on store visit predictions (e.g., "10% off coupon when purchasing 2 liters of milk").

[1806] Step 23:

[1807] The server transmits the generated coupon information to the user terminal and performs a push notification.

[1808] Inventory management

[1809] Step 24:

[1810] The server calculates the appropriate inventory quantity based on the sales forecast.

[1811] Step 25:

[1812] The server connects with the inventory management system and suggests appropriate purchase quantities to the store (e.g., "Purchase an additional 200 tomatoes").

[1813] Emotion Engine Functions

[1814] User sentiment analysis

[1815] Step 26:

[1816] While the user is using the app, the emotion engine collects the user's facial expression data, typing speed and patterns.

[1817] Step 27:

[1818] The user terminal transmits the collected data to the server.

[1819] Step 28:

[1820] The server uses an emotion engine to analyze the user's emotions in real time.

[1821] Providing personalized product and service recommendations

[1822] Step 29:

[1823] The server determines personalized recommended products and services based on the analysis results of the emotion engine.

[1824] Step 30:

[1825] The server sends information about recommended products and services to the user terminal.

[1826] Step 31:

[1827] The user's device displays personalized product and service recommendations (e.g., if the user is feeling stressed, it suggests products that have a relaxing effect).

[1828] Providing coupon and sale information at the optimal time

[1829] Step 32:

[1830] The server generates coupon and sale information at the optimal time based on the analysis results of the emotion engine.

[1831] Step 33:

[1832] The server transmits the generated coupon and sale information to the user terminal.

[1833] Step 34:

[1834] The user's device notifies and displays coupons and sales information (e.g., offering special discount coupons when the user is in a good mood).

[1835] Example 2

[1836] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1837] Conventional price comparison systems simply collect price information from multiple stores and are unable to make suggestions tailored to consumer sentiment or individual needs. Furthermore, they lacked a mechanism for integrating information from flyer images taken by consumers into the system, making it difficult to reflect the latest price information. Furthermore, they lacked the functionality to integrate the collected information into a database and provide related recipes based on the shopping list entered by the consumer.

[1838] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1839] In this invention, the server includes: a means for a consumer to input their place of residence; a means for a consumer to input a list of products they wish to purchase; a means for collecting information from multiple local sales locations; a means for analyzing the collected information and extracting and integrating price information; a means for searching for the cheapest products based on the list of products they wish to purchase; a means for displaying the search results on the consumer's terminal; a means including an engine for analyzing consumer sentiment; and a means for providing personalized recommended products and services based on the analysis results. This enables the provision of personalized services according to the consumer's sentiments and needs, enabling more advanced price comparisons and shopping optimization. Furthermore, the server can analyze flyer images provided by consumers and integrate the latest price information into the database, thereby providing more accurate and up-to-date price information.

[1840] A "consumer" is an individual who purchases goods or services.

[1841] "Residence" is geographic information that indicates where a consumer currently lives.

[1842] "Point of sale" means a store or market that offers goods or services.

[1843] "Information gathering" is the process of gathering data from different sources.

[1844] "Analysis" refers to examining and processing collected information to extract meaningful data.

[1845] A "product list" is a list of products that a consumer wishes to purchase.

[1846] "Price information" is data regarding the selling price of a product or service.

[1847] "Search" is the act of finding information that matches specific conditions from a vast amount of data.

[1848] A "consumer terminal" is a digital device used by a consumer, such as a computer or smartphone.

[1849] An "emotion engine" is a technology for analyzing consumer emotions and their state.

[1850] "Recommended Products" are products that are specifically recommended based on a consumer's needs and preferences.

[1851] "Services" refers to support activities provided to consumers other than the products sold.

[1852] "Integration" refers to bringing together different data into one system or dataset.

[1853] "OCR technology" is a technology that converts scanned documents into machine-readable text data.

[1854] A "flyer image" is an image file of a paper printed for advertising or promotion.

[1855] A "recipe" is a set of instructions that lists the ingredients and steps for making a particular dish.

[1856] "Related recipes" are recipes for dishes related to the user's shopping list.

[1857] This invention is a system that enables consumers to efficiently purchase products at the optimal price. Specifically, the system allows consumers to input their place of residence and the product they wish to purchase, and combines price information collected from multiple local sales locations with an emotion engine to suggest the most suitable product.

[1858] This system mainly involves the exchange of information between the server, the device, and the consumer. The main hardware used to implement the system is the server, the consumer device (smartphone or computer), and network equipment. The software used includes Python, BeautifulSoup, Scrapy, MySQL, Tesseract OCR, OpenCV, analysis tools (pandas, scikit-learn), and web scraping tools.

[1859] First, a user accesses the smartphone app and enters the postal code or city where they live. This information is sent from the device to the server, which stores the residence information in a database. Next, the user enters a list of items they wish to purchase into the app, and the device sends this list to the server.

[1860] The server collects the latest flyer information from multiple local stores based on the registered residential address information. This collection process uses web scraping technology using Python's BeautifulSoup and Scrapy. In addition, images of paper flyers provided by consumers are analyzed using OCR technology (Tesseract OCR), and price information is integrated into the database.

[1861] The server then searches the database for the cheapest products based on the product list entered by the user, using the MySQL database management system, and sends the search results to the consumer's terminal, where the best price information is displayed to the user.

[1862] Furthermore, based on the shopping list entered by the user, the server searches for related recipes from the recipe database and displays them on the user's terminal. For example, if the user enters "tomato, milk, egg" in the list, the server will suggest a recipe for "tomato omelette."

[1863] For retail stores, the server collects shopping list data from consumers and uses analytical tools to predict sales. The analysis results are sent to the store's management terminal, which then implements appropriate inventory management and sales strategies. The server also collects and analyzes sales information from nearby stores, and generates appropriate coupons based on the predictions to provide to consumers.

[1864] Furthermore, the emotion engine analyzes data collected while the user is using the app (such as facial recognition data and input speed) to evaluate the user's emotions. This analysis is performed using OpenCV and other analytical tools. Based on the analysis results, the server provides personalized product and service recommendations according to the user's emotions. For example, if the user is feeling stressed, it will suggest products and services that have a relaxing effect.

[1865] As a concrete example, suppose a user uses a smartphone app to register their place of residence and enter "tomatoes, milk, and eggs" as part of their weekend shopping list, and the emotion engine detects the user's stress during this process. The server collects the latest flyer information from sales locations in Shinjuku Ward, searches for the lowest prices, and provides them to the user. At the same time, recipes and products with a relaxing effect are suggested, and special coupons based on sale information are notified to the user. Based on this data, the sales locations can implement appropriate inventory management and sales strategies.

[1866] In this way, the system, combined with the emotion engine, provides more advanced support to both consumers and points of sale, improving personalization and efficiency.

[1867] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1868] Step 1:

[1869] The user accesses the smartphone app and enters the postal code or city where they live.

[1870] Input: Postal code or city of residence

[1871] Output: Input data is saved on the user's device

[1872] Specific operation: The user enters the postal code and city / town name into the app's residence input form and presses the "Submit" button.

[1873] Step 2:

[1874] The terminal sends this information to the server.

[1875] Input: User-entered residential address information

[1876] Output: Location information is sent to the server

[1877] Specific operation: The device sends data to the server via the Internet.

[1878] Step 3:

[1879] The server stores the residence information in a database.

[1880] Input: Residential information

[1881] Output: Residence information is saved in the database

[1882] Specific operation: The server executes an SQL query to store the received residence information in a database.

[1883] Step 4:

[1884] A user enters a list of items they want to buy into the app (e.g., "tomatoes, milk, eggs").

[1885] Input: List of items you wish to purchase

[1886] Output: Input data is saved on the user's device

[1887] Specific operation: The user enters the product name into the app's shopping list input form and presses the "Submit" button.

[1888] Step 5:

[1889] The terminal sends this list to the server.

[1890] Input: A list of items the user wishes to purchase

[1891] Output: A list of items to purchase is sent to the server

[1892] Specific operation: The device sends data to the server via the Internet.

[1893] Step 6:

[1894] The server collects the latest flyer information from multiple sales locations in the area based on the registered residential address information.

[1895] Input: Residential information

[1896] Output: Collected flyer information

[1897] Specific operation: Based on the specified residence information, the server performs web scraping using Python's BeautifulSoup and Scrapy to collect flyer information from the websites of each sales location.

[1898] Step 7:

[1899] The server receives an image of a paper flyer provided by a consumer.

[1900] Input: Flyer image

[1901] Output: Flyer images are saved on the server

[1902] Specific operation: A user takes a photo of a flyer with their smartphone and uploads the image through the app. The server receives the image.

[1903] Step 8:

[1904] The server analyzes the flyer image using OCR technology (e.g., Tesseract OCR) and integrates the price information into a database.

[1905] Input: Flyer image

[1906] Output: Price information in the database

[1907] Specific operation: The server performs OCR analysis, extracts text data from the flyer image, and parses price information from it. The parsed data is then integrated into a database.

[1908] Step 9:

[1909] The server searches the database for the cheapest products based on the product list entered by the user.

[1910] Input: List of products you wish to purchase, price information

[1911] Output: Information on the cheapest product

[1912] What it does: The server queries the database for pricing information to find the cheapest item. It uses Python data analysis tools to calculate the lowest price.

[1913] Step 10:

[1914] The server sends the search results to the user's terminal and displays the lowest price information.

[1915] Input: Information on the lowest priced item

[1916] Output: The lowest price information displayed on the user's terminal

[1917] Specific operation: The server sends the search results to the user's device, and the app displays them.

[1918] Step 11:

[1919] The server searches for related recipes from a recipe database based on the shopping list entered by the user and displays them on the user terminal.

[1920] Input: List of items you wish to purchase

[1921] Output: Related recipe information

[1922] Specific operation: The server searches the recipe database for relevant recipes based on the shopping list. The searched recipe information is sent to the user's terminal and displayed.

[1923] Step 12:

[1924] The server collects past consumer shopping list data and begins analyzing it.

[1925] Input: Past shopping list data

[1926] Output: Analysis results (sales forecast)

[1927] What it does: The server runs a Python analytics tool to analyze historical consumer shopping list data in the database and generate sales forecasts.

[1928] Step 13:

[1929] The server periodically collects and analyzes sales information from nearby stores.

[1930] Input:Sale information

[1931] Output: Parsed sale information, generated coupons

[1932] What it does: The server periodically runs a web script to collect and analyze sales information from nearby stores.

[1933] Step 14:

[1934] The server generates appropriate coupons based on the store visit prediction and provides them to the consumer terminal.

[1935] Input: store visit prediction, sales information

[1936] Output: Generated coupon

[1937] Specific operation: The server executes an algorithm to generate appropriate coupons based on store visit predictions. The generated coupons are sent to the consumer's terminal and notified.

[1938] Step 15:

[1939] The emotion engine analyzes data collected while users are using the app.

[1940] Input: User operation data (face recognition data, input speed, etc.)

[1941] Output: User sentiment analysis results

[1942] Specific operation: The emotion engine collects user operation data and analyzes the emotional state using OpenCV and data analysis tools.

[1943] Step 16:

[1944] The server provides personalized recommended products and services based on the user's emotions analyzed by the emotion engine.

[1945] Input: Sentiment analysis results

[1946] Output: Personalized product and service recommendations

[1947] Specific operation: The server refers to the sentiment analysis results and executes a personalized recommendation algorithm to suggest appropriate products and services. The suggestions are sent to the user's device and displayed.

[1948] Step 17:

[1949] Based on the analysis results of the emotion engine, the server generates and provides coupon and sale information at the optimal time.

[1950] Input: Sentiment analysis results, sales information

[1951] Output: Generated coupons and sales information

[1952] Specific operation: The server uses the results of the sentiment analysis to run an algorithm that generates coupon and sale information at the optimal time. The generated information is then sent to the user's device and notified.

[1953] (Application example 2)

[1954] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1955] In the past, consumers had to visit multiple stores to find the best price, which was time-consuming and labor-intensive. Furthermore, the means for researching price information were limited, making efficient shopping difficult. Furthermore, personalized product and service recommendations based on the consumer's emotional state were not provided, making it difficult to improve the consumer experience.

[1956] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: a means for a consumer to input their place of residence; a means for inputting a list of products the consumer wants to buy; a means for collecting flyer information from multiple stores in the area based on the place of residence; a means for searching for the cheapest products from the collected flyer information based on the list of products the consumer wants to buy; a means for displaying the search results on the consumer terminal; a means for analyzing the consumer's emotions; a means for providing personalized recommended products and services based on the results of the emotion analysis; and a means for displaying the personalized recommended products and services on the consumer terminal. This enables consumers to efficiently purchase products at optimal prices and receive personalized suggestions based on their emotional state.

[1957] "Means for entering residential address" refers to the interface that allows consumers to register their residential address information in the system. Examples of such interfaces include smart glasses and smartphone apps.

[1958] The "means for inputting a list of products to be purchased" refers to an interface that allows consumers to register a list of products they wish to purchase in the system. This includes a touch screen and voice recognition.

[1959] "Methods for collecting flyer information" are methods for collecting the latest price and sale information from multiple stores in the area. Flyer images are analyzed using web scraping and OCR technology.

[1960] The "means for searching for the cheapest products" refers to algorithms or software that search for the lowest priced products from among the products entered by the consumer based on collected flyer information.

[1961] "Means for displaying search results on consumer devices" refers to an interface for presenting the lowest price information found on the consumer's device (such as smart glasses or a smartphone).

[1962] "Emotion analysis tools" are software or algorithms that analyze a consumer's facial expressions and behavioral data to determine their current emotional state. This may include cameras and sensors.

[1963] "Means for providing personalized product and service recommendations" refers to algorithms and software that suggest the most appropriate products and services based on the consumer's emotional state.

[1964] The "means for displaying personalized recommended products and services on a consumer terminal" is an interface for displaying recommended products and services related to sentiment analysis on a terminal used by a consumer.

[1965] The present invention is a system for supporting consumers in purchasing products efficiently at optimal prices in their daily lives. Specific embodiments of the system will be described below.

[1966] Implementation system configuration

[1967] Consumer Devices

[1968] Consumers use smart glasses or a smartphone app to input their location and a list of items they want to buy. The device has an interface that allows input via touchscreen or voice recognition.

[1969] server

[1970] The server receives information on the consumer's location and product list from the consumer's device and collects the latest flyer information from multiple stores in the area. This information is collected by integrating flyer information from online and paper sources using web scraping and OCR technologies. An algorithm is then run to search for the cheapest product among the items the consumer wants to buy, based on the price information stored in the database.

[1971] Furthermore, the server is equipped with an emotion analysis engine that recognizes emotions in real time by analyzing data such as consumer facial images and input patterns. Based on the results of this emotion analysis, personalized product and service recommendations are generated and displayed on the consumer's device.

[1972] Data processing and calculation

[1973] Sentiment Analysis Engine

[1974] This engine analyzes consumer facial image data collected using cameras and sensors, as well as the speed and patterns of their actions. Major technologies used include OpenCV (for facial recognition) and TensorFlow (for emotion recognition). Consumer emotions (such as stress or happiness) are detected in real time by this engine and provided as data to a server.

[1975] Price information collection and integration

[1976] The server automatically collects price information from websites and paper sources using web scraping and OCR technologies, specifically BeautifulSoup (for web scraping) and Tesseract OCR (for character recognition), and stores it in a database.

[1977] Best price search and recommendations

[1978] The server uses the product list and address information entered by the consumer to screen price information in the database and find the cheapest products. Data processing libraries such as Python's pandas and SQL are used. In addition, by combining consumer sentiment and individual product information, an algorithm is run that individually recommends the best products and services for each specific consumer.

[1979] Specific examples

[1980] Suppose a consumer uses smart glasses to input their place of residence (e.g., "Shinjuku Ward") and a list of products they wish to buy (e.g., "tomatoes, milk, eggs"). During this process, the smart glasses' camera captures the consumer's facial expression and sends the data to a server. If the emotion analysis engine detects stress, the server will suggest products or services with a relaxing effect (e.g., "aromatherapy oil") in addition to the usual lowest price information.

[1981] Prompt Sentence Examples

[1982] "I live in Shinjuku Ward, and I'd like to buy tomatoes, milk, and eggs. Next, I'll provide an image of my facial expression taken by the camera on my smart glasses. Based on this information, please tell me the lowest prices in the area and recommend products based on my emotions."

[1983] In this way, the embodiment of the invention realizes a system that reflects the consumer's input information and emotional state, collects and analyzes optimal price information from multiple stores, and provides personalized recommended products in real time.

[1984] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1985] Step 1:

[1986] A user uses a consumer device (smart glasses or a smartphone app) to input their location and a list of products they wish to purchase. The input information is then sent from the device to a server. Specifically, the user uses the device's interface to input information using a touch screen or voice recognition. The input data is in the form of text, which is then sent to the server.

[1987] Step 2:

[1988] Based on the received residential address information, the server collects the latest flyer and price information from multiple stores in the area. This collection is performed using web scraping and OCR technologies. BeautifulSoup is used for web scraping to extract price information from online store websites. Tesseract OCR is also used to recognize and extract characters from paper flyer images. The collected data is then stored in a database.

[1989] Step 3:

[1990] The server searches for the cheapest products that match the product list entered by the user based on the price information stored in the database. This process involves filtering the data using Python's pandas library and executing database queries using SQL. The search results are returned in text format as the lowest price information for the target products and are sent to the consumer's device.

[1991] Step 4:

[1992] The consumer device displays the lowest price information sent from the server to the user. In the case of smart glasses, the price information is displayed in the user's field of view using holographic display technology. In the case of smartphones, the price information is displayed in text and graphical format on the screen, allowing the user to visually check the cheapest product information.

[1993] Step 5:

[1994] Consumer emotion data is collected in real time by the camera and sensors on the consumer device. Specifically, facial image data captured by the camera is processed within the device, and the results are sent to a server. This data is analyzed using an emotion recognition model that combines OpenCV and TensorFlow on the edge device. The analysis results are sent to the server as text data.

[1995] Step 6:

[1996] The server receives the results of the emotion analysis and generates personalized product and service recommendations based on them. If the emotion indicates stress, products and services with a relaxing effect will be recommended. This recommendation process is performed by an algorithm using a generative AI model, and the recommendation results are obtained as text data.

[1997] Step 7:

[1998] The server sends the generated information about recommended products and services to the consumer's device, which then displays the received information to the user. In the case of smart glasses, the recommended product information is displayed in the user's field of vision using holographic display technology. In the case of smartphones, the recommended information is displayed in text and graphical format on the screen, allowing the user to learn about the best products and services based on their emotional state.

[1999] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2000] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2001] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2002] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2003] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2004] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2005] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2006] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2007] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2008] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2009] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2010] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external...

Claims

1. a means for the consumer to input their place of residence; a means for consumers to input a list of products they wish to purchase; A means for collecting flyer information from a plurality of stores in the area based on the residence; A means for searching for the cheapest product from the collected flyer information based on the list of products desired to be purchased; means for displaying the search results on a consumer terminal; A system including:

2. a means for integrating the collected flyer information and store price information and storing the information in a database; means for analyzing flyer images provided by consumers, extracting price information, and integrating the information into the database; The system of claim 1 further comprising:

3. 2. The system according to claim 1, further comprising means for searching for related recipes from the list of products desired to be purchased and information on the cheapest product, and displaying the recipes on the consumer terminal.

Images