system

Abstract

Provide a system. A means for a user to input a home address, travel distance, and price difference that the user sets in advance; A means for collecting image data from advertising media of a plurality of stores and uploading the collected image data to a server; A server converts the image data into text data and extracts product and price information; a means for selecting an optimal store based on the extracted product and price information set by a user; A means for generating an optimal navigation route to a store and notifying the user terminal of the route; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Today's consumers want to maximize time and cost efficiency when shopping daily. However, comparing sales information from multiple stores and selecting the most suitable store is a time-consuming and labor-intensive task. Furthermore, managing the ingredients needed based on pre-planned meals can be a burden for consumers. Given these circumstances, there is a need for a system that improves shopping efficiency and enables users to effectively purchase the items they need at the store with the best deals. [Means for solving the problem]

[0005] The present invention provides a means for users to input their home address, travel distance, and price difference in advance, and a means for collecting image data from advertising media of multiple stores and uploading it to a server. The server then converts the image data into text data and extracts product and price information. Based on the extracted product and price information, the system selects the optimal store based on the user's settings, generates a navigation route, and notifies the user's terminal. It also creates a shopping list based on the user's weekly menu and missing ingredient information, and notifies the user of this shopping list and information on the optimal store, significantly improving shopping efficiency. Furthermore, accurate product information can be obtained by analyzing advertising media images using an image recognition algorithm. This allows users to shop most efficiently and economically.

[0006] "User" refers to a person who uses the system to input their home address and purchase plans in order to efficiently carry out their own shopping.

[0007] "Home address" refers to the specific address of the place where the user lives, and indicates the location from which shopping begins.

[0008] The "travel distance" refers to the maximum distance a user is allowed to travel for shopping purposes.

[0009] "Price differential" refers to the acceptable difference in price for the same product between different stores.

[0010] "Advertising media" refers to paper and electronic flyers used by supermarkets and drugstores to communicate sales information and product prices to consumers.

[0011] "Image data" refers to photographs or scanned images of advertising media.

[0012] "Server" refers to a computer system that receives and processes image data sent by a user.

[0013] "Text data" refers to character information extracted from image data, such as product names and prices.

[0014] "Products" refer to items sold in stores, such as food ingredients or household goods that a user is considering purchasing.

[0015] "Price information" refers to the prices of products sold at each store.

[0016] "Navigation route" refers to the route that the user should follow to reach the selected optimal store.

[0017] "Terminal" refers to an electronic device such as a smartphone or tablet that a user uses to access the system.

[0018] A "purchase list" refers to a list of necessary ingredients and products that the user has entered in advance.

[0019] "Image recognition algorithm" refers to the technology or method for extracting text data from image data. [Brief explanation of the drawings]

[0020] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0021] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0022] First, the terms used in the following description will be explained.

[0023] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0024] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0025] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0026] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0027] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0028] [First embodiment]

[0029] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0030] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0031] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0032] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0033] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0034] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0035] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0036] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0037] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0038] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0039] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0040] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0041] The present invention is a system that provides the best deals by allowing users to input their home address, travel distance, and price difference, and then collects image data from the advertising media of multiple stores and uploads it to a server. This system is designed to improve the user's shopping experience and promote store promotional activities.

[0042] Overview of the overall system operation

[0043] Setup Phase

[0044] 1. Users

[0045] Enter your home address into the app, and also set the allowable travel distance (e.g., 2km) and price difference (e.g., 50 yen).

[0046] You can also enter your weekly menu and a list of ingredients you're missing, such as "Monday: curry, Tuesday: spaghetti" or "carrots, potatoes, onions, tomato sauce."

[0047] 2. Server

[0048] Load and prepare the analysis model for image recognition. Collect location information and the latest flyer data of nearby supermarkets and drugstores and store them in a database.

[0049] Save user settings information as a profile.

[0050] Image Recognition Phase

[0051] 3. Users

[0052] Take a photo of a flyer from a local supermarket or drugstore with your smartphone.

[0053] The image data of the photographed flyer is uploaded to the server via the app.

[0054] 4. Server

[0055] The received flyer image data is passed to an image recognition module, which converts the image data into text data. For example, the product name, price, and special sale information are extracted from the flyer.

[0056] The extracted text data is used to create a list of product types and their price information, which is then stored in a database.

[0057] Optimal store proposal phase

[0058] 5. Server

[0059] Based on the extracted price information, product prices are compared taking into consideration the travel distance and the tolerable range of price difference specified by the user.

[0060] The optimal store is selected for each product and calculations are performed to minimize the total cost to the user.

[0061] 6. Server

[0062] Based on the selected optimal store, a navigation route is generated using a map service, such as Google (registered trademark) Maps API.

[0063] A shopping list is created based on the information input by the user and is sent to the user terminal together with the navigation route.

[0064] 7. Terminal

[0065] It receives information from the server and notifies the user, for example, displaying the name and location of the best store, navigation route, shopping list, etc.

[0066] Specific examples

[0067] Setup and Initial Preparation

[0068] The user opens the app, enters their home address (Minato Ward, Tokyo), the travel distance (2 km), and the price difference (50 yen). They then enter the menu (curry and spaghetti) and the missing ingredients (carrots, potatoes, onions, and tomato sauce).

[0069] The server receives the information, collects the latest flyer data from nearby supermarkets and drugstores, and registers it in a database.

[0070] Image Recognition and Data Analysis

[0071] The user takes a photo of flyers from nearby supermarkets A and B and uploads them to the app.

[0072] The server extracts the data "Carrots: 100 yen, Potatoes: 80 yen, Onions: 60 yen, Tomato sauce: 120 yen" from flyer A and "Carrots: 90 yen, Potatoes: 100 yen, Onions: 70 yen, Tomato sauce: 110 yen" from flyer B, lists them, and saves them in a database.

[0073] Selection and notification of optimal stores

[0074] The server compares the extracted price information and determines that Supermarket A has a lower total cost (360 yen). Based on this, Supermarket A is selected as the optimal store.

[0075] The server uses the Google Maps API to generate a navigation route to Supermarket A and sends it to the user's device along with the shopping list.

[0076] The terminal notifies the user of the information received from the server and displays a shopping list along with navigation for heading to Supermarket A.

[0077] In this way, the present invention provides a system that allows users to shop most efficiently and economically, while also enhancing the effectiveness of store promotions.

[0078] The processing flow will be explained below.

[0079] Step 1:

[0080] The user launches the app and enters their home address (Minato Ward, Tokyo), the distance they can travel (2 km), and the price difference (50 yen). They then enter their weekly menu (curry and spaghetti) and a list of ingredients they are missing (carrots, potatoes, onions, and tomato sauce).

[0081] Step 2:

[0082] The server receives the user's input information and saves it as a profile. It also collects location information and the latest flyer data for nearby supermarkets and drugstores and registers them in a database.

[0083] Step 3:

[0084] The user takes a photo of flyers from nearby supermarkets A and B with their smartphone and uploads the flyer image data to the server via the app.

[0085] Step 4:

[0086] The server passes the received flyer image data to an image recognition module, which converts the image data into text data. For example, the product name, price, and special sale information are extracted from the flyer.

[0087] Step 5:

[0088] The server creates a list of product types and their price information from the extracted text data and stores the list in a database.

[0089] Step 6:

[0090] The server compares product prices based on the accumulated price information, taking into consideration the user's travelable distance and the acceptable range of price difference.

[0091] Step 7:

[0092] The server selects the best store for each product and calculates the minimum total cost for the user. For example, it checks that the total cost for supermarket A is 360 yen and the total cost for supermarket B is 370 yen.

[0093] Step 8:

[0094] The server obtains the optimal route using a map service (e.g., Google Maps API) to generate a navigation route to the selected optimal store (e.g., Supermarket A).

[0095] Step 9:

[0096] The server creates a shopping list based on the information input by the user and transmits it to the user terminal together with the navigation route.

[0097] Step 10:

[0098] The device receives information from the server and notifies the user of the best store information, navigation route, and shopping list. For example, it might display "Route to Supermarket A" with "Carrots: 100 yen, Potatoes: 80 yen, Onions: 60 yen, Tomato sauce: 120 yen."

[0099] The above steps realize a system that allows users to efficiently shop at the most cost-effective store and also enhances the store's sales promotion effects.

[0100] Example 1

[0101] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0102] Users need to expend a lot of time and effort to purchase the products they need efficiently and economically. Manually comparing advertising media and flyers from multiple stores is extremely tedious. Furthermore, it is difficult to optimally select stores to visit and navigate them along efficient routes. This degrades the user's shopping experience and reduces the effectiveness of store promotional activities. To solve this problem, a system is needed that allows users to easily input information and automatically provides optimal store information and navigation routes.

[0103] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0104] In this invention, the server includes means for inputting a user's preset home address, travel distance, and price difference, means for collecting image data from advertising media of multiple stores and uploading it to the server, means for converting the image data into text data and extracting product and price information, means for selecting the optimal store based on the extracted product and price information based on the user's settings, means for generating a navigation route to the optimal store and notifying the user terminal, means for inputting a shopping list preset by the user, and means for selecting the optimal store based on the shopping list and generating a navigation route. This allows users to shop efficiently and economically and also enhances the effectiveness of store promotions.

[0105] "User" refers to an individual who uses the system to input setting information and flyer images and receives optimal store information and navigation routes.

[0106] "Home address" refers to the specific location information of the place where the user lives.

[0107] "Moveable distance" refers to the maximum distance a user can move from their home.

[0108] "Price difference" refers to the difference in product price that a user can accept.

[0109] "Advertising media" refers to print or digital sources containing product information and special offers provided by a store.

[0110] "Image data" refers to a digital image file that captures the content of advertising media.

[0111] "Server" refers to a computer system that processes collected image data and provides optimal information to users.

[0112] "Text data" refers to character information extracted from image data.

[0113] "Product" refers to a specific item that a user is considering purchasing.

[0114] "Price information" refers to price information corresponding to each product.

[0115] The "optimal store" refers to a store that meets the user's set conditions and offers the most economical products.

[0116] A "navigation route" refers to the path a user takes to reach the optimal store.

[0117] "Notifying" refers to sending information to a user terminal to notify the user.

[0118] A "purchase list" refers to a list of products and quantities that a user plans to purchase.

[0119] "Image recognition algorithm" refers to a computational method or program for extracting text data from image data.

[0120] "Terminal" refers to an electronic device that a user operates and receives information from.

[0121] This technology allows users to input their home address, travel distance, and price difference, collect image data from advertising media of multiple stores, upload it to a server, and provide the store with the best deal. The system aims to improve the user's shopping experience and promote store promotional activities.

[0122] Overall system configuration

[0123] This system includes a terminal (e.g., a smartphone) that allows users to input information, photograph flyers, and upload them to a server, a server, and various program modules that run on the server. Potential hardware used includes a smartphone, a server, and a high-performance CPU or GPU for image processing. Software used includes an image processing library (e.g., OpenCV), an image recognition model (e.g., TENSORFLOW (registered trademark), Google Cloud Vision API), and a database management system (e.g., MySQL (registered trademark)).

[0124] Specific operating procedures

[0125] Entering User Preferences Information

[0126] The user launches the smartphone app and first enters their home address. For example, the user enters "Minato Ward, Tokyo," sets the travel distance to "2 km," and the price difference to "50 yen." The user also enters the weekly menu as "Monday: curry, Tuesday: spaghetti," and the missing ingredients as "carrots, potatoes, onions, tomato sauce."

[0127] Collect flyer images and upload them to the server

[0128] The user takes a picture of a flyer from a nearby store (e.g., a supermarket or drugstore) with their smartphone camera. The captured image data is sent to the server by pressing the "Upload" button in the app. The program notifies the user that the upload is complete.

[0129] Image data analysis

[0130] The server analyzes the received image data. TensorFlow and Google Cloud Vision API are used as image recognition models. This allows text data such as product names, prices, and sale information to be extracted from the image. For example, information such as "Carrots: 100 yen, Potatoes: 80 yen, Onions: 60 yen, Tomato sauce: 120 yen" can be extracted from a flyer. The extracted data is stored in a database.

[0131] Selecting the best store and generating navigation routes

[0132] The server selects the most suitable store based on the price information in the database, taking into account the user's settings (home address, travel distance, price difference). Once the most suitable store is determined, the server generates a navigation route using a map service (e.g., Google Maps API). At the same time, it also takes into account the shopping list entered by the user and sets a route that allows the user to efficiently purchase the necessary products.

[0133] User Notification and Guidance

[0134] The server sends the generated navigation route and shopping list to the user's smartphone. The user receives a notification from the device and can follow the information displayed on the screen to find the optimal store. The displayed information includes the store name, address, list of products to be purchased, and navigation route.

[0135] Specific examples

[0136] The user opens the app and enters their home address (Minato Ward, Tokyo), the travel distance (2 km), and the price difference (50 yen). They also enter their menu (curry and spaghetti) and the missing ingredients (carrots, potatoes, onions, and tomato sauce). Next, the user takes photos of flyers for nearby supermarkets A and B and uploads them to the app. The server extracts the following data from flyer A: "Carrots: 100 yen, potatoes: 80 yen, onions: 60 yen, tomato sauce: 120 yen"; from flyer B: "Carrots: 90 yen, potatoes: 100 yen, onions: 70 yen, tomato sauce: 110 yen." These data are then compiled into a list and stored in a database. The server compares the extracted price information and determines that supermarket A offers the best total cost (360 yen). The server uses the Google Maps API to generate a navigation route to supermarket A and sends it to the user's device along with the shopping list. The device notifies the user of the received information and displays the shopping list along with navigation to supermarket A. This allows users to shop in the most efficient and economical way possible.

[0137] In this way, the present invention provides a system that enables users to shop efficiently and economically, and also enhances the effectiveness of store promotions.

[0138] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0139] Step 1:

[0140] The user enters the configuration information

[0141] A user launches the app and enters their home address (e.g., "Minato Ward, Tokyo"), the distance they can travel (e.g., "2 km"), the price difference (e.g., "50 yen"), and a shopping list (e.g., "carrots, potatoes, onions, tomato sauce"), which then creates a personalized shopping profile for the user.

[0142] Input: Home address, travel distance, price difference, shopping list

[0143] Output: User profile data

[0144] Specific behavior: The user enters information into the app using form input fields and then clicks the "Save Settings" button to send it to the server.

[0145] Step 2:

[0146] The server stores user information

[0147] The profile data sent by the user is received by the server, which stores it in a database and updates the user profile.

[0148] Input: User profile data

[0149] Output: Updated database user profile

[0150] Specific operation: The server parses the received data and stores it in a structured database (e.g., MySQL).

[0151] Step 3:

[0152] The user takes and uploads a flyer image

[0153] Users take a photo of a flyer from a nearby store using their smartphone camera, then use the app's upload function to upload the flyer image data to the server.

[0154] Input: Flyer image data

[0155] Output: Flyer image uploaded to the server

[0156] Specific operation: The user takes a photo of a flyer with the camera and presses the "Upload image" button to send the image to the server.

[0157] Step 4:

[0158] The server analyzes the flyer image

[0159] The server analyzes the received flyer image data using an image recognition algorithm (e.g., TensorFlow, Google Cloud Vision API). As a result of the analysis, product names, prices, special sale information, etc. are extracted as text data.

[0160] Input: Flyer image data

[0161] Output: Product information text data

[0162] Specific operation: The server loads the image recognition model, inputs a flyer image, and extracts text data including product name and price information.

[0163] Step 5:

[0164] The server stores the product information in a database

[0165] The extracted text data of the product information is stored in a database, and price information for each product is saved.

[0166] Input: Text data of product information

[0167] Output: Product information stored in the database

[0168] Specific operation: The server structures the text data and stores it by adding a new entry to the database.

[0169] Step 6:

[0170] The server selects the best store

[0171] The server selects the most cost-effective store within the specified travel distance and price range based on the user's purchase list and price information.

[0172] Input: Purchase list, user profile data, product information

[0173] Output: Optimal store information

[0174] How it works: The server searches and compares product prices from a database and uses an algorithm to identify the best store.

[0175] Step 7:

[0176] The server generates the navigation route

[0177] A navigation route to the selected optimal store is generated using a map service (e.g., Google Maps API).

[0178] Input: Best store information, home address

[0179] Output: Navigation route

[0180] Specific behavior: The server queries the Google Maps API to obtain a navigation route and processes it.

[0181] Step 8:

[0182] The server notifies the user of the navigation route and shopping list.

[0183] The server sends the generated navigation route and shopping list to the user's smartphone.

[0184] Input: navigation route, purchase list

[0185] Output: User notification

[0186] Specific operation: The server sends information to the user's device, and the app receives and displays the notification.

[0187] (Application example 1)

[0188] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0189] With so many choices available to modern consumers, it can be difficult to determine where and which products are most economical to purchase. Stores also have limited means of efficiently advertising to consumers. This creates a need for a system that supports optimal purchasing choices tailored to individual consumer needs while simultaneously enhancing store promotional effectiveness.

[0190] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0191] In this invention, the server includes means for inputting the user's preset home address, travel distance, and price difference, means for taking images of advertising media for multiple stores with a smartphone and uploading them to the server, means for converting the image data into text data using an image recognition algorithm and extracting product and price information, means for selecting the optimal store from the extracted product and price information based on the user's settings, and means for generating a navigation route to the optimal store using a map service and notifying the user terminal. This allows the user to receive suggestions for optimal stores in real time, enabling efficient and economical purchasing activities.

[0192] "User" refers to a consumer who uses this system.

[0193] "Home address" refers to the specific address information of the home specified by the user.

[0194] The "travelable distance" refers to the distance that indicates the range that the user can travel from their home.

[0195] "Price difference" refers to the price difference that a user is willing to accept.

[0196] "Multiple stores" refers to multiple target commercial facilities.

[0197] "Advertising media" refers to distributed advertisements such as flyers and pamphlets provided by stores.

[0198] "Image data" refers to image files of advertising media taken with a smartphone or other device.

[0199] "Smartphone" refers to a portable information terminal equipped with mobile communication functions.

[0200] "Server" refers to a computer on a network that processes, stores, and distributes image data.

[0201] An "image recognition algorithm" refers to a program for extracting text information and product information from image data.

[0202] "Text data" refers to a character string extracted from image data.

[0203] "Product and price information" refers to information about the product name and price listed in the advertising media.

[0204] The "best store" refers to the store that best meets the user's requirements.

[0205] "Navigation route" refers to a route showing directions from the current location to the destination.

[0206] "Map Service" refers to a service that provides digital maps and supports navigation functions.

[0207] "User terminal" refers to a connected device such as a smartphone or PC used by a user.

[0208] The system for implementing this invention automates a series of steps for users to find the best store. Users input their home address, travel distance, and price difference into the application, and the system begins operation based on this information.

[0209] Hardware and Software Overview

[0210] Server: A computing device for large-scale data processing and analysis. The server contains software for running image recognition algorithms (e.g., pytesseract), a database management system (e.g., PostgreSQL), and a map service API for generating navigation routes (e.g., Google Maps API).

[0211] User device: A smartphone or tablet used by a user. An application is installed on this device, which receives user input, photographs advertising media in nearby stores, and uploads them to a server.

[0212] Process Overview

[0213] User input of settings

[0214] Through the application, users input information such as their home address, travel distance, price difference, etc. They can also input a weekly menu and a list of ingredients they are missing, and the system then creates and saves a user profile.

[0215] Specific input example:

[0216] Home address: "Minato-ku, Tokyo"

[0217] Travel distance: "2km"

[0218] Price difference: "50 yen"

[0219] Menu: "Curry and Spaghetti"

[0220] Missing ingredients list: "Carrots, potatoes, onions, tomato sauce."

[0221] Photograph and upload flyers

[0222] Users use their smartphones to take photos of advertising flyers from nearby stores and upload the image data to a server via the application.

[0223] Image data analysis

[0224] The server receives the uploaded image data and uses an image recognition algorithm (e.g., pytesseract) to extract product names, prices, and special offers, which are then stored in a database as text data.

[0225] Selection of optimal stores

[0226] The server compares product prices based on the product information stored in the database, taking into account the travel distance and price difference set by the user, and selects the optimal store for each product, performing calculations to minimize the user's total cost.

[0227] Navigation route generation and notification

[0228] The server uses a map service API (e.g., Google Maps API) to generate a navigation route to the best store. The generated route is sent to the user's device and notified along with the shopping list.

[0229] Specific prompt examples:

[0230] "I live in Minato Ward, Tokyo. Please find a supermarket within a 2km radius where the price difference is within 50 yen. I would like to buy carrots, potatoes, onions, and tomato sauce."

[0231] In this way, users can find the most economical shopping destination in real time, enabling them to shop efficiently, and stores can also advertise effectively to consumers.

[0232] The above is a specific description of the embodiment of the invention.

[0233] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0234] Step 1: User inputs settings

[0235] Input: The user enters their home address, travel distance, price difference, weekly meal plan, and missing ingredients into the application.

[0236] Data processing: The information entered by the user is sent to the server and saved in a database as a profile.

[0237] Output: The server creates a user profile and outputs it as saved data.

[0238] Step 2: Photograph and upload your flyer

[0239] Input: A user uses their smartphone to take a photo of a flyer for a nearby store.

[0240] Data processing: The captured image data is uploaded to the server via the application.

[0241] Output: The server receives and stores the uploaded image data.

[0242] Step 3: Analyzing the image data

[0243] Input: Image data received by the server.

[0244] Data processing: The server uses an image recognition algorithm (e.g., pytesseract) to convert the image data into text data, specifically extracting product names, prices, and special offers.

[0245] Output: The extracted product name, price, and sale information are saved as text data in a database.

[0246] Step 4: Select the best store

[0247] Input: User profile information (home address, travel distance, price difference) and product information stored in the database.

[0248] Data processing: The server compares product prices based on the user's settings and calculates the best store to minimize the user's total cost.

[0249] Output: Information on the optimal store and calculation results are generated.

[0250] Step 5: Generate and notify navigation route

[0251] Input: User's home address and best store information.

[0252] Data processing: The server uses a map service API (e.g., Google Maps API) to generate a navigation route to the optimal store.

[0253] Output: The generated navigation route and shopping list are sent to the user's device.

[0254] Specific behavior:

[0255] In step 1, the user enters the required information into the application, which is then saved as a profile on the server.

[0256] In step 2, the user takes a photo of the flyer with their smartphone and uploads it to the server via the application.

[0257] In step 3, the server uses an image recognition algorithm to convert the received image data into text data, specifically extracting product names, prices, and sale information, and storing it in a database.

[0258] In step 4, the server performs calculations to select the most suitable store based on the user's profile information and the product information stored in the database.

[0259] In step 5, the server uses the map service API to generate a navigation route and sends it to the user's terminal together with the shopping list.

[0260] This series of processes enables users to make the most efficient and economical purchases in real time.

[0261] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0262] The present invention is a system that streamlines the user's shopping experience by allowing the user to input their home address, travel distance, and price difference in advance. Furthermore, by combining this system with an emotion engine that recognizes the user's emotions, it is possible to select the most appropriate store according to the user's emotional state.

[0263] Overview of the overall system operation

[0264] Setup Phase

[0265] 1. Users

[0266] Enter your home address into the app, and set the allowable travel distance (e.g., 2km) and price difference (e.g., 50 yen).

[0267] You can also enter your weekly menu and a list of ingredients you're missing, such as "Monday: curry, Tuesday: spaghetti" or "carrots, potatoes, onions, tomato sauce."

[0268] As you use the app to consider ingredients or your shopping list, the emotion engine recognizes your emotions in real time.

[0269] 2. Server

[0270] It receives user input information and saves it as a profile. It also collects location information and the latest flyer data for nearby supermarkets and drugstores and stores them in a database.

[0271] The emotion engine analyzes and stores emotional data entered by the user in real time.

[0272] Image Recognition Phase

[0273] 3. Users

[0274] Take a photo of a flyer from a local supermarket or drugstore with your smartphone.

[0275] The image data of the photographed flyer is uploaded to the server via the app.

[0276] 4. Server

[0277] The received flyer image data is passed to an image recognition module, which converts the image data into text data. For example, the product name, price, and special sale information are extracted from the flyer.

[0278] The extracted text data is used to create a list of product types and their price information, which is then stored in a database.

[0279] Optimal store proposal phase

[0280] 5. Server

[0281] Product prices are compared based on accumulated price information, the travel distance set by the user, and the acceptable range of price differences.

[0282] The emotion engine assesses the user's emotional state and applies an algorithm that prioritizes the selection of the nearest store if, for example, stress levels are high.

[0283] 6. Server

[0284] The optimal store is selected for each product, and calculations are performed to minimize the total cost for the user. For example, it is confirmed that the total cost for supermarket A is 360 yen and the total cost for supermarket B is 370 yen.

[0285] The emotion engine also takes into account past emotional data and makes suggestions optimized for the user's shopping style.

[0286] 7. Server

[0287] To generate a navigation route to the selected optimal store (e.g., Supermarket A), a map service (e.g., Google Maps API) is used to obtain the optimal route.

[0288] A shopping list is created based on the information input by the user and is sent to the user terminal together with the navigation route.

[0289] 8. Terminal

[0290] It receives information from the server and notifies the user of the best store information, navigation route, and shopping list. For example, it displays "Route to Supermarket A" and "Carrots: 100 yen, Potatoes: 80 yen, Onions: 60 yen, Tomato sauce: 120 yen."

[0291] In addition, the emotion engine will suggest less crowded times, for example, to help users shop in a less stressful environment.

[0292] Specific examples

[0293] Setup and Initial Preparation

[0294] The user opens the app and enters their home address (Minato-ku, Tokyo), the travel distance (2km), and the price difference (50 yen). They then enter the menu (curry and spaghetti) and the missing ingredients (carrots, potatoes, onions, and tomato sauce). The emotion engine reads the user's emotions while they are entering and considering their choices, and accumulates the data.

[0295] The server receives the information, collects the latest flyer data from nearby supermarkets and drug stores, and stores it in a database. The emotion engine also stores the user's emotion data and uses it for future optimization.

[0296] Image Recognition and Data Analysis

[0297] The user takes a photo of flyers from nearby supermarkets A and B and uploads them to the app.

[0298] The server extracts the data "Carrots: 100 yen, Potatoes: 80 yen, Onions: 60 yen, Tomato sauce: 120 yen" from flyer A and "Carrots: 90 yen, Potatoes: 100 yen, Onions: 70 yen, Tomato sauce: 110 yen" from flyer B, lists them, and saves them in a database.

[0299] Selection and notification of optimal stores

[0300] The server compares the extracted price information and determines that Supermarket A has a lower total cost (360 yen). Based on this, Supermarket A is selected as the optimal store. The server evaluates the user's emotional state, and if, for example, stress is high, prioritizes the nearest store.

[0301] The server uses the Google Maps API to generate a navigation route to Supermarket A and sends it to the user's device along with the shopping list. Using the emotion engine, it also includes suggestions for reducing stress (e.g., less crowded times).

[0302] The terminal receives information from the server and notifies the user of the navigation and purchase list for Supermarket A. For example, it displays "Route to Supermarket A" and "Carrots: 100 yen, Potatoes: 80 yen, Onions: 60 yen, Tomato sauce: 120 yen."

[0303] In this way, the present invention provides a system that allows for the most efficient and economical shopping experience, while also taking into account the user's emotional state.

[0304] The processing flow will be explained below.

[0305] Step 1:

[0306] The user launches the app and enters their home address (Minato Ward, Tokyo), the distance they can travel (2 km), and the price difference (50 yen). They also enter their weekly menu and a list of ingredients they are missing. For example, "Monday's dinner is curry, Tuesday's is spaghetti," and "Ingredients needed: carrots, potatoes, onions, tomato sauce."

[0307] Step 2:

[0308] While the user is using the app, the emotion engine analyzes the user's facial expressions, voice, etc. to obtain emotional data in real time. The emotional state (e.g., stress level, elation, etc.) is evaluated and stored in a database.

[0309] Step 3:

[0310] The server receives the user's settings and emotional data and stores them as a profile. It also collects location information and the latest flyer data for nearby supermarkets and drugstores and stores them in a database.

[0311] Step 4:

[0312] The user takes a photo of flyers from nearby supermarkets A and B with their smartphone, and uploads the flyer image data to the server via the app.

[0313] Step 5:

[0314] The server passes the received flyer image data to an image recognition module, which converts the image data into text data (e.g., extracting product names, prices, and sale information from the flyer). The extracted text data is analyzed, and product types and their prices are listed and stored in a database.

[0315] Step 6:

[0316] The server compares product prices based on the accumulated price information, the user's travel distance, price tolerance, and emotional data. If the user's stress level is high, an algorithm is applied that prioritizes the selection of the nearest store.

[0317] Step 7:

[0318] The server selects the optimal store for each product and calculates the lowest total cost (e.g., the total cost for supermarket A is 360 yen, and the total cost for supermarket B is 370 yen). The optimal store is selected based on the user's emotional data (e.g., supermarket B may be selected because it has low stress levels).

[0319] Step 8:

[0320] The server considers the user's emotional data and generates a navigation route to the optimal selected store (e.g., obtains the optimal route using Google Maps API). It also makes suggestions to reduce stress (e.g., less crowded times).

[0321] Step 9:

[0322] The server creates a shopping list based on the information input by the user and transmits it to the user terminal together with the navigation route.

[0323] Step 10:

[0324] The terminal notifies the user of the information received from the server (for example, displaying the name, location, navigation route, shopping list, etc. of the best store). Specifically, for example, it displays "Route to Supermarket A" and "Carrots: 100 yen, Potatoes: 80 yen, Onions: 60 yen, Tomato sauce: 120 yen."

[0325] Through the above steps, the user can shop efficiently at the most suitable store that takes into consideration the user's emotional state, achieving a less stressful shopping experience.

[0326] Example 2

[0327] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0328] The present invention aims to streamline the user's shopping experience and provide an optimal shopping environment that takes into account the user's emotional state. Specifically, the present invention aims to provide a system that selects the optimal store based on the user's specified address, travel distance, and price difference, and also evaluates the user's emotional state to support a stress-free shopping experience.

[0329] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for inputting the user's home address, travel distance, and price difference, which are set in advance by the user; a means for collecting image data from advertising media of multiple business facilities and uploading it to the server; a means for converting the image data into text data and extracting product and price information therefor; a means for selecting the optimal business facility from the extracted product and price information based on the user's settings; a means for evaluating the user's emotional state and optimizing store selection based on the evaluation results; and a means for generating a navigation route to the optimal business facility and notifying the user terminal. This allows the user to have an efficient and emotionally-conscious shopping experience.

[0330] "Address" refers to specific geographical information about the place where the user lives, and serves as a criterion for selecting a store within the user's permitted range of movement.

[0331] The "travelable distance" refers to the distance of the travelable range that the user is willing to travel, and serves as a criterion for selecting the most suitable store within this range.

[0332] "Price difference" refers to the difference in product prices that a user is willing to accept, and serves as a criterion for comparing prices within this range to select a store.

[0333] "Business facilities" refer to facilities such as stores, supermarkets, and drugstores that sell products, and are places that provide products for users to purchase.

[0334] "Advertising media" refers to advertising information provided by each business establishment, and primarily includes printed media such as flyers and posters.

[0335] "Image data" refers to visual information of advertising media photographed and uploaded by a user and stored in digital format.

[0336] "Text data" refers to character information extracted by analyzing image data, and includes product names, price information, and the like.

[0337] A "product" refers to an item that a user considers purchasing, and is accompanied by a specific name and price information.

[0338] "Price information" refers to the specific price associated with a product, and serves as a basis for users to select the most suitable place to purchase.

[0339] "Emotional state" refers to the user's psychological state and emotions, and by evaluating this, the system is able to select and suggest the most suitable store for the user.

[0340] A "navigation route" refers to a specific route for a user to travel to a selected business facility, and is generated using a map service.

[0341] "User terminal" refers to an electronic device such as a computer or smartphone that a user uses to input information and receive notifications from the system.

[0342] The present invention is a system that streamlines the user's shopping experience by allowing the user to input their home address, travel distance, and price difference in advance. In addition, it has the function of recognizing the user's emotional state in real time and selecting the most suitable store based on that state.

[0343] Hardware and Software Configuration

[0344] User side

[0345] User device: A device, such as a smartphone or computer, that a user uses to enter information and receive notifications from the server.

[0346] Camera: The camera attached to the user terminal is used to capture image data of the advertising medium.

[0347] Server side

[0348] Server: The central system that receives the information sent by the user and performs analysis and data processing. This includes the database management system, image recognition module, and emotion engine.

[0349] Database management systems: Used to store and manage user profile information and advertising data. Examples include MySQL and PostgreSQL.

[0350] Image recognition module: A software module for converting image data from advertising media into text data. Specific software examples include OpenCV and Tesseract OCR.

[0351] Emotion engine: A software module for analyzing and storing a user's emotional state in real time. It uses natural language processing and machine learning algorithms to evaluate the user's emotions.

[0352] System Operation Overview

[0353] User Settings

[0354] The user launches the app and enters their home address, travel distance, and price difference. They also enter a week's worth of meals and a list of ingredients they are missing. For example, they set their address to "Minato Ward, Tokyo," their travel distance to "2 km," and their price difference to "50 yen," and then enter "curry and spaghetti" as their meal plan and "carrots, potatoes, onions, and tomato sauce" as the ingredients they are missing. This information is saved in the app.

[0355] Data collection and analysis

[0356] Users take a photo of a flyer from a nearby supermarket or drugstore with their smartphone camera and upload it to the server via the app. The server passes the received flyer image to an image recognition module, which converts data such as product names and prices into text. For example, information such as "Carrots: 100 yen, Potatoes: 80 yen, Onions: 60 yen, Tomato sauce: 120 yen" is extracted and stored in a database.

[0357] Selection and proposal of optimal stores

[0358] The server selects the optimal store that meets the user's criteria based on price information stored in a database. It creates a shopping list based on a weekly menu and a list of ingredients that are missing, and uses an emotion engine to evaluate the user's emotional state. For example, if the user is feeling stressed, it will prioritize the nearest store. The server selects the optimal store based on this data and generates a navigation route using Google Maps API, etc. The generated route and shopping list are sent to the user's device.

[0359] Notifications and Support

[0360] Based on the information received from the server, the user device notifies the user of the optimal store information, navigation route, and shopping list. For example, it displays "Optimal store: Supermarket A, navigation route, 'Carrots: 100 yen, Potatoes: 80 yen, Onions: 60 yen, Tomato sauce: 120 yen'." The emotion engine supports shopping in a less stressful environment for the user by making suggestions that take into account, for example, less crowded times.

[0361] Specific examples

[0362] The user opens the app, sets their home address to "Minato-ku, Tokyo," the travel distance to "2 km," the price difference to "50 yen," and inputs the menu item "curry and spaghetti" and the list of missing ingredients: "carrots, potatoes, onions, and tomato sauce."

[0363] The server receives the information and collects the latest flyer data from nearby supermarkets and drugstores, and the emotion engine also stores the user's emotion data.

[0364] When a user takes a photo of flyers from nearby supermarkets A and B and uploads them to the app, the server extracts price data from flyer A, compares them, and evaluates the total cost. It uses an emotion engine to take into account the user's stress level and makes appropriate suggestions.

[0365] The server sends the generated shopping list and navigation route to the user's terminal, helping the user to shop efficiently at the most suitable store.

[0366] Prompt Sentence Examples

[0367] Example prompt 1: "My home address is in Minato Ward, Tokyo. Please tell me the best store to buy ingredients for curry and spaghetti within a 2km travel distance and within a price difference of 50 yen. Also, please help me have a less stressful shopping experience by taking my emotional state into consideration."

[0368] Example prompt 2: "Today's list of things I need is carrots, potatoes, onions, and tomato sauce. I've uploaded images of flyers from nearby supermarkets. Please suggest the store with the lowest cost and least stress."

[0369] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0370] Step 1:

[0371] The user launches the app and enters their home address, travel distance, and price difference. For example, the address can be set to "Minato-ku, Tokyo," the travel distance to "2 km," and the price difference to "50 yen." The input data also includes a week's worth of meals (e.g., "curry and spaghetti") and a list of missing ingredients (e.g., "carrots, potatoes, onions, tomato sauce"). This information is sent to the server from the device operated by the user. Input: Address, travel distance, price difference, meals, list of ingredients. Output: User profile information stored on the server.

[0372] Step 2:

[0373] The server receives the information sent by the user and stores it in a database management system (e.g., MySQL, PostgreSQL). At the same time, the emotion engine analyzes and accumulates the user's emotion data in real time. It also collects location information of nearby business facilities and the latest advertising data and stores them in the database. Input: User profile information. Output: Location information, advertising data, and emotion data stored in the database.

[0374] Step 3:

[0375] The user takes a photo of advertising media in nearby business establishments with their smartphone camera and uploads it to the server via the app. For example, they take photos of flyers from supermarkets A and B. This image data is sent directly to the server. Input: Image of advertising media. Output: Image data uploaded to the server.

[0376] Step 4:

[0377] The image data of the advertising media received by the server is passed to an image recognition module (e.g., OpenCV, Tesseract OCR) and converted into text data. Specifically, the image recognition module extracts the product name, price, and sale information, and stores them in a database. For example, it extracts data such as "carrots: 100 yen, potatoes: 80 yen, onions: 60 yen, tomato sauce: 120 yen." Input: Image data of the advertising media. Output: Text data stored in the database.

[0378] Step 5:

[0379] The server applies an algorithm to select the optimal store based on the product price information stored in the database, within the address, travel distance, and price difference range set by the user. Furthermore, an emotion engine is used to evaluate the user's emotional state, and if, for example, stress levels are high, the system prioritizes closer stores. Input: Text data, user profile information, emotion data. Output: Optimal store selection results.

[0380] Step 6:

[0381] The server calculates the total cost and selects the optimal business facility. For example, if the total cost of supermarket A is 360 yen and the total cost of supermarket B is 370 yen, supermarket A is selected. Based on this, supermarket A is determined to be the optimal store. Input: store price data, user setting information. Output: optimal store and total cost information.

[0382] Step 7:

[0383] The server uses a map service (e.g., Google Maps API) to generate a navigation route to the optimal store. It also sends the generated navigation route and shopping list to the user's device. Input: optimal store information, user location information. Output: navigation route, shopping list.

[0384] Step 8:

[0385] The terminal receives the information sent from the server and notifies the user. Specifically, it displays the optimal store information, navigation route, and shopping list for the user. For example, it notifies the user of information such as "Optimal store: Supermarket A, navigation route, 'Carrots: 100 yen, Potatoes: 80 yen, Onions: 60 yen, Tomato sauce: 120 yen'." It also makes suggestions such as less crowded times. Input: Store information from the server, navigation route, shopping list. Output: Notification and suggestions to the user.

[0386] (Application example 2)

[0387] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0388] The present invention relates to a system that allows users to shop efficiently. There is a particular need for a system that takes into account the user's emotional state and provides optimal store selection while reducing stress. Conventional systems select stores based solely on price and distance, ignoring the user's emotional state and potentially reducing the quality of the shopping experience. Furthermore, there is also the issue that users take a long time to find the optimal store because price information from advertising materials is not adequately extracted and optimized.

[0389] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for inputting the user's home address, travel distance, and price difference, which are set in advance by the user; means for collecting image data from advertising materials for multiple sales points and uploading it to the server; means for converting the image data into text data and extracting product and price information; means for selecting the optimal sales point from the extracted product and price information based on the user's settings; means for analyzing the user's emotional state in real time and optimizing store selection in accordance with the emotional state; and means for generating a route to the optimal sales point and notifying the user terminal. This allows for efficient and optimal shopping while reducing stress, taking the user's emotional state into consideration.

[0390] The "home address preset by the user" is information that the user inputs into the system in advance to indicate the place of residence of the user.

[0391] The "travelable distance" is information indicating the maximum distance that the user is allowed to travel for shopping.

[0392] "Price difference" is information indicating the allowable price difference for the same product.

[0393] "Sales point advertising materials" are materials that show flyers and campaign information issued by stores such as supermarkets and drugstores.

[0394] "Image data" refers to image data obtained from advertising materials at points of sale.

[0395] The "means for uploading to the server" is a method or function for transferring image data collected from the user terminal to the server.

[0396] "Text data" is character string information extracted and converted from image data.

[0397] "Product and price information" is information indicating the name of the product and its selling price listed in the advertising material.

[0398] The "means for selecting the most suitable sales point" is a method or function for automatically selecting the most suitable store for the user based on the user's setting information and price information.

[0399] The "means for generating a travel route" is a method or function for generating a route that enables a user to travel to the optimal sales point in the shortest time or with the least stress.

[0400] The "means for analyzing the user's emotional state in real time" is a method or function for recognizing the user's emotions in real time and analyzing the data.

[0401] The "means for optimizing store selection according to emotional state" is a method or function for selecting an optimal store by taking into account the emotional state of the user.

[0402] The "notification means" is a method or function for notifying the user terminal of information regarding the optimum sales point and its travel route.

[0403] The present invention is a system that streamlines the user's shopping experience and selects the optimal store based on the user's emotional state. The system works by combining and processing user input information, image data from point-of-sale advertising materials, and the user's emotional state.

[0404] The server implements the present invention using the following means.

[0405] 1. Enter your user settings information

[0406] Users input their home address, the distance they can travel, and the price difference, which allows the system to determine the range within which they can shop.

[0407] 2. Collecting and uploading advertising materials

[0408] Users take photos of advertising materials from multiple points of sale using a device such as a smartphone and upload them to a server, at which point image data is collected.

[0409] 3. Conversion to text data

[0410] The server uses an image recognition algorithm (e.g., Tesseract) to convert the uploaded image data into text data, from which product names and their prices are extracted.

[0411] 4. Selection of optimal sales locations

[0412] The server selects the optimal sales location based on the extracted price information and the user's settings (address, distance, price difference), and then analyzes the user's emotional state in real time (e.g., using TensorFlow) to optimize store selection according to the user's emotional state.

[0413] 5. Generate navigation route

[0414] The server generates the optimal travel route for the user to the selected optimal sales point. Specifically, it creates the route using a map service (e.g., Google Maps API).

[0415] 6. Notification to user devices

[0416] This information is then sent to the user's device, which includes the best shopping locations, product pricing information, and navigation routes.

[0417] This system allows users to have an efficient and optimal shopping experience, and also reduces stress by selecting stores according to their emotional state.

[0418] Specific examples of hardware and software used

[0419] Hardware: smartphones, servers, cameras

[0420] Software: TensorFlow (sentiment analysis), Tesseract (image recognition), Google Maps API (navigation)

[0421] Examples of prompt statements

[0422] Here is an example of inputting the following prompt into a generative AI model:

[0423] "The user enters their address, travel distance, price difference, meal plan information, and a list of missing ingredients into the app. Furthermore, if the user's emotional state is 'stressed,' please suggest the most efficient and economical store where they can shop, and why."

[0424] Examples of expected outputs include the following:

[0425] "Supermarket A is the best choice. It was selected for the following reasons: 1) Supermarket A is closest to the specified address. 2) Your current emotional state is stressed, so the travel distance is short and stress reduction is expected. 3) The total cost of the ingredients needed for the meal is the lowest. For these reasons, I recommend Supermarket A."

[0426] In this way, the present invention provides a system that allows for the most efficient and economical shopping experience, while also taking into account the user's emotional state.

[0427] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0428] Step 1:

[0429] The user enters their home address, travel distance, and price difference into the app.

[0430] Input: User's home address, travel distance, price difference

[0431] Output: User configuration information

[0432] How it works: A user launches the app and enters their home address, the distance they can travel (e.g., 2 km), and the price difference (e.g., 50 yen) into the interface. This information is then sent to the server.

[0433] Step 2:

[0434] A user takes pictures of advertising materials from multiple points of sale and uploads them to a server.

[0435] Input: Advertising image of the point of sale

[0436] Output: Image data stored on the server

[0437] How it works: Users use their smartphone camera to take a photo of a flyer from a nearby supermarket or drugstore, then upload the image data to a server via the app.

[0438] Step 3:

[0439] The server converts the received image data into text data and extracts product and price information.

[0440] Input: Image data

[0441] Output: Text data of products and their prices

[0442] How it works: The server uses an image recognition algorithm (e.g., Tesseract) to convert the uploaded image data into text data, then extracts product names and their prices from the text data and stores them in a database.

[0443] Step 4:

[0444] The server selects the optimal sales location based on the user's settings and the extracted price information.

[0445] Input: User settings, product and price information

[0446] Output: Optimal selling point

[0447] Operation: The server analyzes the price information in the database, taking into account the address, travel distance, and price difference set by the user, and selects the optimal sales location.

[0448] Step 5:

[0449] The server analyzes the user's emotional state in real time and optimizes store selection according to that emotional state.

[0450] Input: User's image data or voice data (emotional state)

[0451] Output: Optimal selling points based on emotional state

[0452] How it works: The server uses an emotion engine (e.g., TensorFlow) to analyze the user's emotional state in real time. If the emotional state is stressed, it prioritizes stores with shorter travel distances.

[0453] Step 6:

[0454] The server generates an optimal route to the sales point and notifies the user terminal.

[0455] Input: Best point of sale, user's current location

[0456] Output: Travel route

[0457] Operation: The server uses a map service (e.g., Google Maps API) to generate a route from the user's current location to the optimal sales point, and notifies the user's device of this route information.

[0458] Step 7:

[0459] The user checks the notified content on the terminal and carries out the purchase.

[0460] Input: Notification information from the server (optimal sales location, price information for each product, travel route)

[0461] Output: Actual shopping behavior

[0462] How it works: The user checks the notification on their device and efficiently shops based on the optimal sales point and travel route. For example, it displays "Route to Supermarket A" and shows "Carrots: 100 yen, Potatoes: 80 yen, Onions: 60 yen, Tomato sauce: 120 yen."

[0463] These steps allow users to shop efficiently and economically, and by taking into consideration their emotional state, they are able to reduce stress.

[0464] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0465] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0466] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0467] [Second embodiment]

[0468] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0469] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0470] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0471] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0472] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0473] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0474] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0475] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0476] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0477] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0478] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0479] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0480] The present invention is a system that provides the best deals by allowing users to input their home address, travel distance, and price difference, and then collects image data from the advertising media of multiple stores and uploads it to a server. This system is designed to improve the user's shopping experience and promote store promotional activities.

[0481] Overview of the overall system operation

[0482] Setup Phase

[0483] 1. Users

[0484] Enter your home address into the app, and also set the allowable travel distance (e.g., 2km) and price difference (e.g., 50 yen).

[0485] You can also enter your weekly menu and a list of ingredients you're missing, such as "Monday: curry, Tuesday: spaghetti" or "carrots, potatoes, onions, tomato sauce."

[0486] 2. Server

[0487] Load and prepare the analysis model for image recognition. Collect location information and the latest flyer data of nearby supermarkets and drugstores and store them in a database.

[0488] Save user settings information as a profile.

[0489] Image Recognition Phase

[0490] 3. Users

[0491] Take a photo of a flyer from a local supermarket or drugstore with your smartphone.

[0492] The image data of the photographed flyer is uploaded to the server via the app.

[0493] 4. Server

[0494] The received flyer image data is passed to an image recognition module, which converts the image data into text data. For example, the product name, price, and special sale information are extracted from the flyer.

[0495] The extracted text data is used to create a list of product types and their price information, which is then stored in a database.

[0496] Optimal store proposal phase

[0497] 5. Server

[0498] Based on the extracted price information, product prices are compared taking into consideration the travel distance and the tolerable range of price difference specified by the user.

[0499] The optimal store is selected for each product and calculations are performed to minimize the total cost to the user.

[0500] 6. Server

[0501] Based on the selected optimal store, a navigation route is generated using a map service, such as Google Maps API.

[0502] A shopping list is created based on the information input by the user and is sent to the user terminal together with the navigation route.

[0503] 7. Terminal

[0504] It receives information from the server and notifies the user, for example, displaying the name and location of the best store, navigation route, shopping list, etc.

[0505] Specific examples

[0506] Setup and Initial Preparation

[0507] The user opens the app, enters their home address (Minato Ward, Tokyo), the travel distance (2 km), and the price difference (50 yen). They then enter the menu (curry and spaghetti) and the missing ingredients (carrots, potatoes, onions, and tomato sauce).

[0508] The server receives the information, collects the latest flyer data from nearby supermarkets and drugstores, and registers it in a database.

[0509] Image Recognition and Data Analysis

[0510] The user takes a photo of flyers from nearby supermarkets A and B and uploads them to the app.

[0511] The server extracts the data "Carrots: 100 yen, Potatoes: 80 yen, Onions: 60 yen, Tomato sauce: 120 yen" from flyer A and "Carrots: 90 yen, Potatoes: 100 yen, Onions: 70 yen, Tomato sauce: 110 yen" from flyer B, lists them, and saves them in a database.

[0512] Selection and notification of optimal stores

[0513] The server compares the extracted price information and determines that Supermarket A has a lower total cost (360 yen). Based on this, Supermarket A is selected as the optimal store.

[0514] The server uses the Google Maps API to generate a navigation route to Supermarket A and sends it to the user's device along with the shopping list.

[0515] The terminal notifies the user of the information received from the server and displays a shopping list along with navigation for heading to Supermarket A.

[0516] In this way, the present invention provides a system that allows users to shop most efficiently and economically, while also enhancing the effectiveness of store promotions.

[0517] The processing flow will be explained below.

[0518] Step 1:

[0519] The user launches the app and enters their home address (Minato Ward, Tokyo), the distance they can travel (2 km), and the price difference (50 yen). They then enter their weekly menu (curry and spaghetti) and a list of ingredients they are missing (carrots, potatoes, onions, and tomato sauce).

[0520] Step 2:

[0521] The server receives the user's input information and saves it as a profile. It also collects location information and the latest flyer data for nearby supermarkets and drugstores and registers them in a database.

[0522] Step 3:

[0523] The user takes a photo of flyers from nearby supermarkets A and B with their smartphone and uploads the flyer image data to the server via the app.

[0524] Step 4:

[0525] The server passes the received flyer image data to an image recognition module, which converts the image data into text data. For example, the product name, price, and special sale information are extracted from the flyer.

[0526] Step 5:

[0527] The server creates a list of product types and their price information from the extracted text data and stores the list in a database.

[0528] Step 6:

[0529] The server compares product prices based on the accumulated price information, taking into consideration the user's travelable distance and the acceptable range of price difference.

[0530] Step 7:

[0531] The server selects the best store for each product and calculates the minimum total cost for the user. For example, it checks that the total cost for supermarket A is 360 yen and the total cost for supermarket B is 370 yen.

[0532] Step 8:

[0533] The server obtains the optimal route using a map service (e.g., Google Maps API) to generate a navigation route to the selected optimal store (e.g., Supermarket A).

[0534] Step 9:

[0535] The server creates a shopping list based on the information input by the user and transmits it to the user terminal together with the navigation route.

[0536] Step 10:

[0537] The device receives information from the server and notifies the user of the best store information, navigation route, and shopping list. For example, it might display "Route to Supermarket A" with "Carrots: 100 yen, Potatoes: 80 yen, Onions: 60 yen, Tomato sauce: 120 yen."

[0538] The above steps realize a system that allows users to efficiently shop at the most cost-effective store and also enhances the store's sales promotion effects.

[0539] Example 1

[0540] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0541] Users need to expend a lot of time and effort to purchase the products they need efficiently and economically. Manually comparing advertising media and flyers from multiple stores is extremely tedious. Furthermore, it is difficult to optimally select stores to visit and navigate them along efficient routes. This degrades the user's shopping experience and reduces the effectiveness of store promotional activities. To solve this problem, a system is needed that allows users to easily input information and automatically provides optimal store information and navigation routes.

[0542] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0543] In this invention, the server includes means for inputting a user's preset home address, travel distance, and price difference, means for collecting image data from advertising media of multiple stores and uploading it to the server, means for converting the image data into text data and extracting product and price information, means for selecting the optimal store based on the extracted product and price information based on the user's settings, means for generating a navigation route to the optimal store and notifying the user terminal, means for inputting a shopping list preset by the user, and means for selecting the optimal store based on the shopping list and generating a navigation route. This allows users to shop efficiently and economically and also enhances the effectiveness of store promotions.

[0544] "User" refers to an individual who uses the system to input setting information and flyer images and receives optimal store information and navigation routes.

[0545] "Home address" refers to the specific location information of the place where the user lives.

[0546] "Moveable distance" refers to the maximum distance a user can move from their home.

[0547] "Price difference" refers to the difference in product price that a user can accept.

[0548] "Advertising media" refers to print or digital sources containing product information and special offers provided by a store.

[0549] "Image data" refers to a digital image file that captures the content of advertising media.

[0550] "Server" refers to a computer system that processes collected image data and provides optimal information to users.

[0551] "Text data" refers to character information extracted from image data.

[0552] "Product" refers to a specific item that a user is considering purchasing.

[0553] "Price information" refers to price information corresponding to each product.

[0554] The "optimal store" refers to a store that meets the user's set conditions and offers the most economical products.

[0555] A "navigation route" refers to the path a user takes to reach the optimal store.

[0556] "Notifying" refers to sending information to a user terminal to notify the user.

[0557] A "purchase list" refers to a list of products and quantities that a user plans to purchase.

[0558] "Image recognition algorithm" refers to a computational method or program for extracting text data from image data.

[0559] "Terminal" refers to an electronic device that a user operates and receives information from.

[0560] This technology allows users to input their home address, travel distance, and price difference, collect image data from advertising media of multiple stores, upload it to a server, and provide the store with the best deal. The system aims to improve the user's shopping experience and promote store promotional activities.

[0561] Overall system configuration

[0562] This system includes a terminal (e.g., a smartphone) on which users input information, photograph flyers, and upload them to a server, a server, and various program modules that run on the server. Potential hardware used includes a smartphone, a server, and a high-performance CPU or GPU for image processing. Software used includes an image processing library (e.g., OpenCV), an image recognition model (e.g., TensorFlow, Google Cloud Vision API), and a database management system (e.g., MySQL).

[0563] Specific operating procedures

[0564] Entering User Preferences Information

[0565] The user launches the smartphone app and first enters their home address. For example, the user enters "Minato Ward, Tokyo," sets the travel distance to "2 km," and the price difference to "50 yen." The user also enters the weekly menu as "Monday: curry, Tuesday: spaghetti," and the missing ingredients as "carrots, potatoes, onions, tomato sauce."

[0566] Collect flyer images and upload them to the server

[0567] The user takes a picture of a flyer from a nearby store (e.g., a supermarket or drugstore) with their smartphone camera. The captured image data is sent to the server by pressing the "Upload" button in the app. The program notifies the user that the upload is complete.

[0568] Image data analysis

[0569] The server analyzes the received image data. TensorFlow and Google Cloud Vision API are used as image recognition models. This allows text data such as product names, prices, and sale information to be extracted from the image. For example, information such as "Carrots: 100 yen, Potatoes: 80 yen, Onions: 60 yen, Tomato sauce: 120 yen" can be extracted from a flyer. The extracted data is stored in a database.

[0570] Selecting the best store and generating navigation routes

[0571] The server selects the most suitable store based on the price information in the database, taking into account the user's settings (home address, travel distance, price difference). Once the most suitable store is determined, the server generates a navigation route using a map service (e.g., Google Maps API). At the same time, it also takes into account the shopping list entered by the user and sets a route that allows the user to efficiently purchase the necessary products.

[0572] User Notification and Guidance

[0573] The server sends the generated navigation route and shopping list to the user's smartphone. The user receives a notification from the device and can follow the information displayed on the screen to find the optimal store. The displayed information includes the store name, address, list of products to be purchased, and navigation route.

[0574] Specific examples

[0575] The user opens the app and enters their home address (Minato Ward, Tokyo), the travel distance (2 km), and the price difference (50 yen). They also enter their menu (curry and spaghetti) and the missing ingredients (carrots, potatoes, onions, and tomato sauce). Next, the user takes photos of flyers for nearby supermarkets A and B and uploads them to the app. The server extracts the following data from flyer A: "Carrots: 100 yen, potatoes: 80 yen, onions: 60 yen, tomato sauce: 120 yen"; from flyer B: "Carrots: 90 yen, potatoes: 100 yen, onions: 70 yen, tomato sauce: 110 yen." These data are then compiled into a list and stored in a database. The server compares the extracted price information and determines that supermarket A offers the best total cost (360 yen). The server uses the Google Maps API to generate a navigation route to supermarket A and sends it to the user's device along with the shopping list. The device notifies the user of the received information and displays the shopping list along with navigation to supermarket A. This allows users to shop in the most efficient and economical way possible.

[0576] In this way, the present invention provides a system that enables users to shop efficiently and economically, and also enhances the effectiveness of store promotions.

[0577] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0578] Step 1:

[0579] The user enters the configuration information

[0580] A user launches the app and enters their home address (e.g., "Minato Ward, Tokyo"), the distance they can travel (e.g., "2 km"), the price difference (e.g., "50 yen"), and a shopping list (e.g., "carrots, potatoes, onions, tomato sauce"), which then creates a personalized shopping profile for the user.

[0581] Input: Home address, travel distance, price difference, shopping list

[0582] Output: User profile data

[0583] Specific behavior: The user enters information into the app using form input fields and then clicks the "Save Settings" button to send it to the server.

[0584] Step 2:

[0585] The server stores user information

[0586] The profile data sent by the user is received by the server, which stores it in a database and updates the user profile.

[0587] Input: User profile data

[0588] Output: Updated database user profile

[0589] Specific operation: The server parses the received data and stores it in a structured database (e.g., MySQL).

[0590] Step 3:

[0591] The user takes and uploads a flyer image

[0592] Users take a photo of a flyer from a nearby store using their smartphone camera, then use the app's upload function to upload the flyer image data to the server.

[0593] Input: Flyer image data

[0594] Output: Flyer image uploaded to the server

[0595] Specific operation: The user takes a photo of a flyer with the camera and presses the "Upload image" button to send the image to the server.

[0596] Step 4:

[0597] The server analyzes the flyer image

[0598] The server analyzes the received flyer image data using an image recognition algorithm (e.g., TensorFlow, Google Cloud Vision API). As a result of the analysis, product names, prices, special sale information, etc. are extracted as text data.

[0599] Input: Flyer image data

[0600] Output: Product information text data

[0601] Specific operation: The server loads the image recognition model, inputs a flyer image, and extracts text data including product name and price information.

[0602] Step 5:

[0603] The server stores the product information in a database

[0604] The extracted text data of the product information is stored in a database, and price information for each product is saved.

[0605] Input: Text data of product information

[0606] Output: Product information stored in the database

[0607] Specific operation: The server structures the text data and stores it by adding a new entry to the database.

[0608] Step 6:

[0609] The server selects the best store

[0610] The server selects the most cost-effective store within the specified travel distance and price range based on the user's purchase list and price information.

[0611] Input: Purchase list, user profile data, product information

[0612] Output: Optimal store information

[0613] How it works: The server searches and compares product prices from a database and uses an algorithm to identify the best store.

[0614] Step 7:

[0615] The server generates the navigation route

[0616] A navigation route to the selected optimal store is generated using a map service (e.g., Google Maps API).

[0617] Input: Best store information, home address

[0618] Output: Navigation route

[0619] Specific behavior: The server queries the Google Maps API to obtain a navigation route and processes it.

[0620] Step 8:

[0621] The server notifies the user of the navigation route and shopping list.

[0622] The server sends the generated navigation route and shopping list to the user's smartphone.

[0623] Input: navigation route, purchase list

[0624] Output: User notification

[0625] Specific operation: The server sends information to the user's device, and the app receives and displays the notification.

[0626] (Application example 1)

[0627] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0628] With so many choices available to modern consumers, it can be difficult to determine where and which products are most economical to purchase. Stores also have limited means of efficiently advertising to consumers. This creates a need for a system that supports optimal purchasing choices tailored to individual consumer needs while simultaneously enhancing store promotional effectiveness.

[0629] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0630] In this invention, the server includes means for inputting the user's preset home address, travel distance, and price difference, means for taking images of advertising media for multiple stores with a smartphone and uploading them to the server, means for converting the image data into text data using an image recognition algorithm and extracting product and price information, means for selecting the optimal store from the extracted product and price information based on the user's settings, and means for generating a navigation route to the optimal store using a map service and notifying the user terminal. This allows the user to receive suggestions for optimal stores in real time, enabling efficient and economical purchasing activities.

[0631] "User" refers to a consumer who uses this system.

[0632] "Home address" refers to the specific address information of the home specified by the user.

[0633] The "travelable distance" refers to the distance that indicates the range that the user can travel from their home.

[0634] "Price difference" refers to the price difference that a user is willing to accept.

[0635] "Multiple stores" refers to multiple target commercial facilities.

[0636] "Advertising media" refers to distributed advertisements such as flyers and pamphlets provided by stores.

[0637] "Image data" refers to image files of advertising media taken with a smartphone or other device.

[0638] "Smartphone" refers to a portable information terminal equipped with mobile communication functions.

[0639] "Server" refers to a computer on a network that processes, stores, and distributes image data.

[0640] An "image recognition algorithm" refers to a program for extracting text information and product information from image data.

[0641] "Text data" refers to a character string extracted from image data.

[0642] "Product and price information" refers to information about the product name and price listed in the advertising media.

[0643] The "best store" refers to the store that best meets the user's requirements.

[0644] "Navigation route" refers to a route showing directions from the current location to the destination.

[0645] "Map Service" refers to a service that provides digital maps and supports navigation functions.

[0646] "User terminal" refers to a connected device such as a smartphone or PC used by a user.

[0647] The system for implementing this invention automates a series of steps for users to find the best store. Users input their home address, travel distance, and price difference into the application, and the system begins operation based on this information.

[0648] Hardware and Software Overview

[0649] Server: A computing device for large-scale data processing and analysis. The server contains software for running image recognition algorithms (e.g., pytesseract), a database management system (e.g., PostgreSQL), and a map service API for generating navigation routes (e.g., Google Maps API).

[0650] User device: A smartphone or tablet used by a user. An application is installed on this device, which receives user input, photographs advertising media in nearby stores, and uploads them to a server.

[0651] Process Overview

[0652] User input of settings

[0653] Through the application, users input information such as their home address, travel distance, price difference, etc. They can also input a weekly menu and a list of ingredients they are missing, and the system then creates and saves a user profile.

[0654] Specific input example:

[0655] Home address: "Minato-ku, Tokyo"

[0656] Travel distance: "2km"

[0657] Price difference: "50 yen"

[0658] Menu: "Curry and Spaghetti"

[0659] Missing ingredients list: "Carrots, potatoes, onions, tomato sauce."

[0660] Photograph and upload flyers

[0661] Users use their smartphones to take photos of advertising flyers from nearby stores and upload the image data to a server via the application.

[0662] Image data analysis

[0663] The server receives the uploaded image data and uses an image recognition algorithm (e.g., pytesseract) to extract product names, prices, and special offers, which are then stored in a database as text data.

[0664] Selection of optimal stores

[0665] The server compares product prices based on the product information stored in the database, taking into account the travel distance and price difference set by the user, and selects the optimal store for each product, performing calculations to minimize the user's total cost.

[0666] Navigation route generation and notification

[0667] The server uses a map service API (e.g., Google Maps API) to generate a navigation route to the best store. The generated route is sent to the user's device and notified along with the shopping list.

[0668] Specific prompt examples:

[0669] "I live in Minato Ward, Tokyo. Please find a supermarket within a 2km radius where the price difference is within 50 yen. I would like to buy carrots, potatoes, onions, and tomato sauce."

[0670] In this way, users can find the most economical shopping destination in real time, enabling them to shop efficiently, and stores can also advertise effectively to consumers.

[0671] The above is a specific description of the embodiment of the invention.

[0672] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0673] Step 1: User inputs settings

[0674] Input: The user enters their home address, travel distance, price difference, weekly meal plan, and missing ingredients into the application.

[0675] Data processing: The information entered by the user is sent to the server and saved in a database as a profile.

[0676] Output: The server creates a user profile and outputs it as saved data.

[0677] Step 2: Photograph and upload your flyer

[0678] Input: A user uses their smartphone to take a photo of a flyer for a nearby store.

[0679] Data processing: The captured image data is uploaded to the server via the application.

[0680] Output: The server receives and stores the uploaded image data.

[0681] Step 3: Analyzing the image data

[0682] Input: Image data received by the server.

[0683] Data processing: The server uses an image recognition algorithm (e.g., pytesseract) to convert the image data into text data, specifically extracting product names, prices, and special offers.

[0684] Output: The extracted product name, price, and sale information are saved as text data in a database.

[0685] Step 4: Select the best store

[0686] Input: User profile information (home address, travel distance, price difference) and product information stored in the database.

[0687] Data processing: The server compares product prices based on the user's settings and calculates the best store to minimize the user's total cost.

[0688] Output: Information on the optimal store and calculation results are generated.

[0689] Step 5: Generate and notify navigation route

[0690] Input: User's home address and best store information.

[0691] Data processing: The server uses a map service API (e.g., Google Maps API) to generate a navigation route to the optimal store.

[0692] Output: The generated navigation route and shopping list are sent to the user's device.

[0693] Specific behavior:

[0694] In step 1, the user enters the required information into the application, which is then saved as a profile on the server.

[0695] In step 2, the user takes a photo of the flyer with their smartphone and uploads it to the server via the application.

[0696] In step 3, the server uses an image recognition algorithm to convert the received image data into text data, specifically extracting product names, prices, and sale information, and storing it in a database.

[0697] In step 4, the server performs calculations to select the most suitable store based on the user's profile information and the product information stored in the database.

[0698] In step 5, the server uses the map service API to generate a navigation route and sends it to the user's terminal together with the shopping list.

[0699] This series of processes enables users to make the most efficient and economical purchases in real time.

[0700] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0701] The present invention is a system that streamlines the user's shopping experience by allowing the user to input their home address, travel distance, and price difference in advance. Furthermore, by combining this system with an emotion engine that recognizes the user's emotions, it is possible to select the most appropriate store according to the user's emotional state.

[0702] Overview of the overall system operation

[0703] Setup Phase

[0704] 1. Users

[0705] Enter your home address into the app, and set the allowable travel distance (e.g., 2km) and price difference (e.g., 50 yen).

[0706] You can also enter your weekly menu and a list of ingredients you're missing, such as "Monday: curry, Tuesday: spaghetti" or "carrots, potatoes, onions, tomato sauce."

[0707] As you use the app to consider ingredients or your shopping list, the emotion engine recognizes your emotions in real time.

[0708] 2. Server

[0709] It receives user input information and saves it as a profile. It also collects location information and the latest flyer data for nearby supermarkets and drugstores and stores them in a database.

[0710] The emotion engine analyzes and stores emotional data entered by the user in real time.

[0711] Image Recognition Phase

[0712] 3. Users

[0713] Take a photo of a flyer from a local supermarket or drugstore with your smartphone.

[0714] The image data of the photographed flyer is uploaded to the server via the app.

[0715] 4. Server

[0716] The received flyer image data is passed to an image recognition module, which converts the image data into text data. For example, the product name, price, and special sale information are extracted from the flyer.

[0717] The extracted text data is used to create a list of product types and their price information, which is then stored in a database.

[0718] Optimal store proposal phase

[0719] 5. Server

[0720] Product prices are compared based on accumulated price information, the travel distance set by the user, and the acceptable range of price differences.

[0721] The emotion engine assesses the user's emotional state and applies an algorithm that prioritizes the selection of the nearest store if, for example, stress levels are high.

[0722] 6. Server

[0723] The optimal store is selected for each product, and calculations are performed to minimize the total cost for the user. For example, it is confirmed that the total cost for supermarket A is 360 yen and the total cost for supermarket B is 370 yen.

[0724] The emotion engine also takes into account past emotional data and makes suggestions optimized for the user's shopping style.

[0725] 7. Server

[0726] To generate a navigation route to the selected optimal store (e.g., Supermarket A), a map service (e.g., Google Maps API) is used to obtain the optimal route.

[0727] A shopping list is created based on the information input by the user and is sent to the user terminal together with the navigation route.

[0728] 8. Terminal

[0729] It receives information from the server and notifies the user of the best store information, navigation route, and shopping list. For example, it displays "Route to Supermarket A" and "Carrots: 100 yen, Potatoes: 80 yen, Onions: 60 yen, Tomato sauce: 120 yen."

[0730] In addition, the emotion engine will suggest less crowded times, for example, to help users shop in a less stressful environment.

[0731] Specific examples

[0732] Setup and Initial Preparation

[0733] The user opens the app and enters their home address (Minato-ku, Tokyo), the travel distance (2km), and the price difference (50 yen). They then enter the menu (curry and spaghetti) and the missing ingredients (carrots, potatoes, onions, and tomato sauce). The emotion engine reads the user's emotions while they are entering and considering their choices, and accumulates the data.

[0734] The server receives the information, collects the latest flyer data from nearby supermarkets and drug stores, and stores it in a database. The emotion engine also stores the user's emotion data and uses it for future optimization.

[0735] Image Recognition and Data Analysis

[0736] The user takes a photo of flyers from nearby supermarkets A and B and uploads them to the app.

[0737] The server extracts the data "Carrots: 100 yen, Potatoes: 80 yen, Onions: 60 yen, Tomato sauce: 120 yen" from flyer A and "Carrots: 90 yen, Potatoes: 100 yen, Onions: 70 yen, Tomato sauce: 110 yen" from flyer B, lists them, and saves them in a database.

[0738] Selection and notification of optimal stores

[0739] The server compares the extracted price information and determines that Supermarket A has a lower total cost (360 yen). Based on this, Supermarket A is selected as the optimal store. The server evaluates the user's emotional state, and if, for example, stress is high, prioritizes the nearest store.

[0740] The server uses the Google Maps API to generate a navigation route to Supermarket A and sends it to the user's device along with the shopping list. Using the emotion engine, it also includes suggestions for reducing stress (e.g., less crowded times).

[0741] The terminal receives information from the server and notifies the user of the navigation and purchase list for Supermarket A. For example, it displays "Route to Supermarket A" and "Carrots: 100 yen, Potatoes: 80 yen, Onions: 60 yen, Tomato sauce: 120 yen."

[0742] In this way, the present invention provides a system that allows for the most efficient and economical shopping experience, while also taking into account the user's emotional state.

[0743] The processing flow will be explained below.

[0744] Step 1:

[0745] The user launches the app and enters their home address (Minato Ward, Tokyo), the distance they can travel (2 km), and the price difference (50 yen). They also enter their weekly menu and a list of ingredients they are missing. For example, "Monday's dinner is curry, Tuesday's is spaghetti," and "Ingredients needed: carrots, potatoes, onions, tomato sauce."

[0746] Step 2:

[0747] While the user is using the app, the emotion engine analyzes the user's facial expressions, voice, etc. to obtain emotional data in real time. The emotional state (e.g., stress level, elation, etc.) is evaluated and stored in a database.

[0748] Step 3:

[0749] The server receives the user's settings and emotional data and stores them as a profile. It also collects location information and the latest flyer data for nearby supermarkets and drugstores and stores them in a database.

[0750] Step 4:

[0751] The user takes a photo of flyers from nearby supermarkets A and B with their smartphone, and uploads the flyer image data to the server via the app.

[0752] Step 5:

[0753] The server passes the received flyer image data to an image recognition module, which converts the image data into text data (e.g., extracting product names, prices, and sale information from the flyer). The extracted text data is analyzed, and product types and their prices are listed and stored in a database.

[0754] Step 6:

[0755] The server compares product prices based on the accumulated price information, the user's travel distance, price tolerance, and emotional data. If the user's stress level is high, an algorithm is applied that prioritizes the selection of the nearest store.

[0756] Step 7:

[0757] The server selects the optimal store for each product and calculates the lowest total cost (e.g., the total cost for supermarket A is 360 yen, and the total cost for supermarket B is 370 yen). The optimal store is selected based on the user's emotional data (e.g., supermarket B may be selected because it has low stress levels).

[0758] Step 8:

[0759] The server considers the user's emotional data and generates a navigation route to the optimal selected store (e.g., obtains the optimal route using Google Maps API). It also makes suggestions to reduce stress (e.g., less crowded times).

[0760] Step 9:

[0761] The server creates a shopping list based on the information input by the user and transmits it to the user terminal together with the navigation route.

[0762] Step 10:

[0763] The terminal notifies the user of the information received from the server (for example, displaying the name, location, navigation route, shopping list, etc. of the best store). Specifically, for example, it displays "Route to Supermarket A" and "Carrots: 100 yen, Potatoes: 80 yen, Onions: 60 yen, Tomato sauce: 120 yen."

[0764] Through the above steps, the user can shop efficiently at the most suitable store that takes into consideration the user's emotional state, achieving a less stressful shopping experience.

[0765] Example 2

[0766] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0767] The present invention aims to streamline the user's shopping experience and provide an optimal shopping environment that takes into account the user's emotional state. Specifically, the present invention aims to provide a system that selects the optimal store based on the user's specified address, travel distance, and price difference, and also evaluates the user's emotional state to support a stress-free shopping experience.

[0768] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for inputting the user's home address, travel distance, and price difference, which are set in advance by the user; a means for collecting image data from advertising media of multiple business facilities and uploading it to the server; a means for converting the image data into text data and extracting product and price information therefor; a means for selecting the optimal business facility from the extracted product and price information based on the user's settings; a means for evaluating the user's emotional state and optimizing store selection based on the evaluation results; and a means for generating a navigation route to the optimal business facility and notifying the user terminal. This allows the user to have an efficient and emotionally-conscious shopping experience.

[0769] "Address" refers to specific geographical information about the place where the user lives, and serves as a criterion for selecting a store within the user's permitted range of movement.

[0770] The "travelable distance" refers to the distance of the travelable range that the user is willing to travel, and serves as a criterion for selecting the most suitable store within this range.

[0771] "Price difference" refers to the difference in product prices that a user is willing to accept, and serves as a criterion for comparing prices within this range to select a store.

[0772] "Business facilities" refer to facilities such as stores, supermarkets, and drugstores that sell products, and are places that provide products for users to purchase.

[0773] "Advertising media" refers to advertising information provided by each business establishment, and primarily includes printed media such as flyers and posters.

[0774] "Image data" refers to visual information of advertising media photographed and uploaded by a user and stored in digital format.

[0775] "Text data" refers to character information extracted by analyzing image data, and includes product names, price information, and the like.

[0776] A "product" refers to an item that a user considers purchasing, and is accompanied by a specific name and price information.

[0777] "Price information" refers to the specific price associated with a product, and serves as a basis for users to select the most suitable place to purchase.

[0778] "Emotional state" refers to the user's psychological state and emotions, and by evaluating this, the system is able to select and suggest the most suitable store for the user.

[0779] A "navigation route" refers to a specific route for a user to travel to a selected business facility, and is generated using a map service.

[0780] "User terminal" refers to an electronic device such as a computer or smartphone that a user uses to input information and receive notifications from the system.

[0781] The present invention is a system that streamlines the user's shopping experience by allowing the user to input their home address, travel distance, and price difference in advance. In addition, it has the function of recognizing the user's emotional state in real time and selecting the most suitable store based on that state.

[0782] Hardware and Software Configuration

[0783] User side

[0784] User device: A device, such as a smartphone or computer, that a user uses to enter information and receive notifications from the server.

[0785] Camera: The camera attached to the user terminal is used to capture image data of the advertising medium.

[0786] Server side

[0787] Server: The central system that receives the information sent by the user and performs analysis and data processing. This includes the database management system, image recognition module, and emotion engine.

[0788] Database management systems: Used to store and manage user profile information and advertising data. Examples include MySQL and PostgreSQL.

[0789] Image recognition module: A software module for converting image data from advertising media into text data. Specific software examples include OpenCV and Tesseract OCR.

[0790] Emotion engine: A software module for analyzing and storing a user's emotional state in real time. It uses natural language processing and machine learning algorithms to evaluate the user's emotions.

[0791] System Operation Overview

[0792] User Settings

[0793] The user launches the app and enters their home address, travel distance, and price difference. They also enter a week's worth of meals and a list of ingredients they are missing. For example, they set their address to "Minato Ward, Tokyo," their travel distance to "2 km," and their price difference to "50 yen," and then enter "curry and spaghetti" as their meal plan and "carrots, potatoes, onions, and tomato sauce" as the ingredients they are missing. This information is saved in the app.

[0794] Data collection and analysis

[0795] Users take a photo of a flyer from a nearby supermarket or drugstore with their smartphone camera and upload it to the server via the app. The server passes the received flyer image to an image recognition module, which converts data such as product names and prices into text. For example, information such as "Carrots: 100 yen, Potatoes: 80 yen, Onions: 60 yen, Tomato sauce: 120 yen" is extracted and stored in a database.

[0796] Selection and proposal of optimal stores

[0797] The server selects the optimal store that meets the user's criteria based on price information stored in a database. It creates a shopping list based on a weekly menu and a list of ingredients that are missing, and uses an emotion engine to evaluate the user's emotional state. For example, if the user is feeling stressed, it will prioritize the nearest store. The server selects the optimal store based on this data and generates a navigation route using Google Maps API, etc. The generated route and shopping list are sent to the user's device.

[0798] Notifications and Support

[0799] Based on the information received from the server, the user device notifies the user of the optimal store information, navigation route, and shopping list. For example, it displays "Optimal store: Supermarket A, navigation route, 'Carrots: 100 yen, Potatoes: 80 yen, Onions: 60 yen, Tomato sauce: 120 yen'." The emotion engine supports shopping in a less stressful environment for the user by making suggestions that take into account, for example, less crowded times.

[0800] Specific examples

[0801] The user opens the app, sets their home address to "Minato-ku, Tokyo," the travel distance to "2 km," the price difference to "50 yen," and inputs the menu item "curry and spaghetti" and the list of missing ingredients: "carrots, potatoes, onions, and tomato sauce."

[0802] The server receives the information and collects the latest flyer data from nearby supermarkets and drugstores, and the emotion engine also stores the user's emotion data.

[0803] When a user takes a photo of flyers from nearby supermarkets A and B and uploads them to the app, the server extracts price data from flyer A, compares them, and evaluates the total cost. It uses an emotion engine to take into account the user's stress level and makes appropriate suggestions.

[0804] The server sends the generated shopping list and navigation route to the user's terminal, helping the user to shop efficiently at the most suitable store.

[0805] Prompt Sentence Examples

[0806] Example prompt 1: "My home address is in Minato Ward, Tokyo. Please tell me the best store to buy ingredients for curry and spaghetti within a 2km travel distance and within a price difference of 50 yen. Also, please help me have a less stressful shopping experience by taking my emotional state into consideration."

[0807] Example prompt 2: "Today's list of things I need is carrots, potatoes, onions, and tomato sauce. I've uploaded images of flyers from nearby supermarkets. Please suggest the store with the lowest cost and least stress."

[0808] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0809] Step 1:

[0810] The user launches the app and enters their home address, travel distance, and price difference. For example, the address can be set to "Minato-ku, Tokyo," the travel distance to "2 km," and the price difference to "50 yen." The input data also includes a week's worth of meals (e.g., "curry and spaghetti") and a list of missing ingredients (e.g., "carrots, potatoes, onions, tomato sauce"). This information is sent to the server from the device operated by the user. Input: Address, travel distance, price difference, meals, list of ingredients. Output: User profile information stored on the server.

[0811] Step 2:

[0812] The server receives the information sent by the user and stores it in a database management system (e.g., MySQL, PostgreSQL). At the same time, the emotion engine analyzes and accumulates the user's emotion data in real time. It also collects location information of nearby business facilities and the latest advertising data and stores them in the database. Input: User profile information. Output: Location information, advertising data, and emotion data stored in the database.

[0813] Step 3:

[0814] The user takes a photo of advertising media in nearby business establishments with their smartphone camera and uploads it to the server via the app. For example, they take photos of flyers from supermarkets A and B. This image data is sent directly to the server. Input: Image of advertising media. Output: Image data uploaded to the server.

[0815] Step 4:

[0816] The image data of the advertising media received by the server is passed to an image recognition module (e.g., OpenCV, Tesseract OCR) and converted into text data. Specifically, the image recognition module extracts the product name, price, and sale information, and stores them in a database. For example, it extracts data such as "carrots: 100 yen, potatoes: 80 yen, onions: 60 yen, tomato sauce: 120 yen." Input: Image data of the advertising media. Output: Text data stored in the database.

[0817] Step 5:

[0818] The server applies an algorithm to select the optimal store based on the product price information stored in the database, within the address, travel distance, and price difference range set by the user. Furthermore, an emotion engine is used to evaluate the user's emotional state, and if, for example, stress levels are high, the system prioritizes closer stores. Input: Text data, user profile information, emotion data. Output: Optimal store selection results.

[0819] Step 6:

[0820] The server calculates the total cost and selects the optimal business facility. For example, if the total cost of supermarket A is 360 yen and the total cost of supermarket B is 370 yen, supermarket A is selected. Based on this, supermarket A is determined to be the optimal store. Input: store price data, user setting information. Output: optimal store and total cost information.

[0821] Step 7:

[0822] The server uses a map service (e.g., Google Maps API) to generate a navigation route to the optimal store. It also sends the generated navigation route and shopping list to the user's device. Input: optimal store information, user location information. Output: navigation route, shopping list.

[0823] Step 8:

[0824] The terminal receives the information sent from the server and notifies the user. Specifically, it displays the optimal store information, navigation route, and shopping list for the user. For example, it notifies the user of information such as "Optimal store: Supermarket A, navigation route, 'Carrots: 100 yen, Potatoes: 80 yen, Onions: 60 yen, Tomato sauce: 120 yen'." It also makes suggestions such as less crowded times. Input: Store information from the server, navigation route, shopping list. Output: Notification and suggestions to the user.

[0825] (Application example 2)

[0826] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0827] The present invention relates to a system that allows users to shop efficiently. There is a particular need for a system that takes into account the user's emotional state and provides optimal store selection while reducing stress. Conventional systems select stores based solely on price and distance, ignoring the user's emotional state and potentially reducing the quality of the shopping experience. Furthermore, there is also the issue that users take a long time to find the optimal store because price information from advertising materials is not adequately extracted and optimized.

[0828] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for inputting the user's home address, travel distance, and price difference, which are set in advance by the user; means for collecting image data from advertising materials for multiple sales points and uploading it to the server; means for converting the image data into text data and extracting product and price information; means for selecting the optimal sales point from the extracted product and price information based on the user's settings; means for analyzing the user's emotional state in real time and optimizing store selection in accordance with the emotional state; and means for generating a route to the optimal sales point and notifying the user terminal. This allows for efficient and optimal shopping while reducing stress, taking the user's emotional state into consideration.

[0829] The "home address preset by the user" is information that the user inputs into the system in advance to indicate the place of residence of the user.

[0830] The "travelable distance" is information indicating the maximum distance that the user is allowed to travel for shopping.

[0831] "Price difference" is information indicating the allowable price difference for the same product.

[0832] "Sales point advertising materials" are materials that show flyers and campaign information issued by stores such as supermarkets and drugstores.

[0833] "Image data" refers to image data obtained from advertising materials at points of sale.

[0834] The "means for uploading to the server" is a method or function for transferring image data collected from the user terminal to the server.

[0835] "Text data" is character string information extracted and converted from image data.

[0836] "Product and price information" is information indicating the name of the product and its selling price listed in the advertising material.

[0837] The "means for selecting the most suitable sales point" is a method or function for automatically selecting the most suitable store for the user based on the user's setting information and price information.

[0838] The "means for generating a travel route" is a method or function for generating a route that enables a user to travel to the optimal sales point in the shortest time or with the least stress.

[0839] The "means for analyzing the user's emotional state in real time" is a method or function for recognizing the user's emotions in real time and analyzing the data.

[0840] The "means for optimizing store selection according to emotional state" is a method or function for selecting an optimal store by taking into account the emotional state of the user.

[0841] The "notification means" is a method or function for notifying the user terminal of information regarding the optimum sales point and its travel route.

[0842] The present invention is a system that streamlines the user's shopping experience and selects the optimal store based on the user's emotional state. The system works by combining and processing user input information, image data from point-of-sale advertising materials, and the user's emotional state.

[0843] The server implements the present invention using the following means.

[0844] 1. Enter your user settings information

[0845] Users input their home address, the distance they can travel, and the price difference, which allows the system to determine the range within which they can shop.

[0846] 2. Collecting and uploading advertising materials

[0847] Users take photos of advertising materials from multiple points of sale using a device such as a smartphone and upload them to a server, at which point image data is collected.

[0848] 3. Conversion to text data

[0849] The server uses an image recognition algorithm (e.g., Tesseract) to convert the uploaded image data into text data, from which product names and their prices are extracted.

[0850] 4. Selection of optimal sales locations

[0851] The server selects the optimal sales location based on the extracted price information and the user's settings (address, distance, price difference), and then analyzes the user's emotional state in real time (e.g., using TensorFlow) to optimize store selection according to the user's emotional state.

[0852] 5. Generate navigation route

[0853] The server generates the optimal travel route for the user to the selected optimal sales point. Specifically, it creates the route using a map service (e.g., Google Maps API).

[0854] 6. Notification to user devices

[0855] This information is then sent to the user's device, which includes the best shopping locations, product pricing information, and navigation routes.

[0856] This system allows users to have an efficient and optimal shopping experience, and also reduces stress by selecting stores according to their emotional state.

[0857] Specific examples of hardware and software used

[0858] Hardware: smartphones, servers, cameras

[0859] Software: TensorFlow (sentiment analysis), Tesseract (image recognition), Google Maps API (navigation)

[0860] Examples of prompt statements

[0861] Here is an example of inputting the following prompt into a generative AI model:

[0862] "The user enters their address, travel distance, price difference, meal plan information, and a list of missing ingredients into the app. Furthermore, if the user's emotional state is 'stressed,' please suggest the most efficient and economical store where they can shop, and why."

[0863] Examples of expected outputs include the following:

[0864] "Supermarket A is the best choice. It was selected for the following reasons: 1) Supermarket A is closest to the specified address. 2) Your current emotional state is stressed, so the travel distance is short and stress reduction is expected. 3) The total cost of the ingredients needed for the meal is the lowest. For these reasons, I recommend Supermarket A."

[0865] In this way, the present invention provides a system that allows for the most efficient and economical shopping experience, while also taking into account the user's emotional state.

[0866] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0867] Step 1:

[0868] The user enters their home address, travel distance, and price difference into the app.

[0869] Input: User's home address, travel distance, price difference

[0870] Output: User configuration information

[0871] How it works: A user launches the app and enters their home address, the distance they can travel (e.g., 2 km), and the price difference (e.g., 50 yen) into the interface. This information is then sent to the server.

[0872] Step 2:

[0873] A user takes pictures of advertising materials from multiple points of sale and uploads them to a server.

[0874] Input: Advertising image of the point of sale

[0875] Output: Image data stored on the server

[0876] How it works: Users use their smartphone camera to take a photo of a flyer from a nearby supermarket or drugstore, then upload the image data to a server via the app.

[0877] Step 3:

[0878] The server converts the received image data into text data and extracts product and price information.

[0879] Input: Image data

[0880] Output: Text data of products and their prices

[0881] How it works: The server uses an image recognition algorithm (e.g., Tesseract) to convert the uploaded image data into text data, then extracts product names and their prices from the text data and stores them in a database.

[0882] Step 4:

[0883] The server selects the optimal sales location based on the user's settings and the extracted price information.

[0884] Input: User settings, product and price information

[0885] Output: Optimal selling point

[0886] Operation: The server analyzes the price information in the database, taking into account the address, travel distance, and price difference set by the user, and selects the optimal sales location.

[0887] Step 5:

[0888] The server analyzes the user's emotional state in real time and optimizes store selection according to that emotional state.

[0889] Input: User's image data or voice data (emotional state)

[0890] Output: Optimal selling points based on emotional state

[0891] How it works: The server uses an emotion engine (e.g., TensorFlow) to analyze the user's emotional state in real time. If the emotional state is stressed, it prioritizes stores with shorter travel distances.

[0892] Step 6:

[0893] The server generates an optimal route to the sales point and notifies the user terminal.

[0894] Input: Best point of sale, user's current location

[0895] Output: Travel route

[0896] Operation: The server uses a map service (e.g., Google Maps API) to generate a route from the user's current location to the optimal sales point, and notifies the user's device of this route information.

[0897] Step 7:

[0898] The user checks the notified content on the terminal and carries out the purchase.

[0899] Input: Notification information from the server (optimal sales location, price information for each product, travel route)

[0900] Output: Actual shopping behavior

[0901] How it works: The user checks the notification on their device and efficiently shops based on the optimal sales point and travel route. For example, it displays "Route to Supermarket A" and shows "Carrots: 100 yen, Potatoes: 80 yen, Onions: 60 yen, Tomato sauce: 120 yen."

[0902] These steps allow users to shop efficiently and economically, and by taking into consideration their emotional state, they are able to reduce stress.

[0903] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0904] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0905] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0906] [Third embodiment]

[0907] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0908] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0909] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0910] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0911] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0912] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0913] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0914] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0915] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0916] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0917] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0918] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0919] The present invention is a system that provides the best deals by allowing users to input their home address, travel distance, and price difference, and then collects image data from the advertising media of multiple stores and uploads it to a server. This system is designed to improve the user's shopping experience and promote store promotional activities.

[0920] Overview of the overall system operation

[0921] Setup Phase

[0922] 1. Users

[0923] Enter your home address into the app, and also set the allowable travel distance (e.g., 2km) and price difference (e.g., 50 yen).

[0924] You can also enter your weekly menu and a list of ingredients you're missing, such as "Monday: curry, Tuesday: spaghetti" or "carrots, potatoes, onions, tomato sauce."

[0925] 2. Server

[0926] Load and prepare the analysis model for image recognition. Collect location information and the latest flyer data of nearby supermarkets and drugstores and store them in a database.

[0927] Save user settings information as a profile.

[0928] Image Recognition Phase

[0929] 3. Users

[0930] Take a photo of a flyer from a local supermarket or drugstore with your smartphone.

[0931] The image data of the photographed flyer is uploaded to the server via the app.

[0932] 4. Server

[0933] The received flyer image data is passed to an image recognition module, which converts the image data into text data. For example, the product name, price, and special sale information are extracted from the flyer.

[0934] The extracted text data is used to create a list of product types and their price information, which is then stored in a database.

[0935] Optimal store proposal phase

[0936] 5. Server

[0937] Based on the extracted price information, product prices are compared taking into consideration the travel distance and the tolerable range of price difference specified by the user.

[0938] The optimal store is selected for each product and calculations are performed to minimize the total cost to the user.

[0939] 6. Server

[0940] Based on the selected optimal store, a navigation route is generated using a map service, such as Google Maps API.

[0941] A shopping list is created based on the information input by the user and is sent to the user terminal together with the navigation route.

[0942] 7. Terminal

[0943] It receives information from the server and notifies the user, for example, displaying the name and location of the best store, navigation route, shopping list, etc.

[0944] Specific examples

[0945] Setup and Initial Preparation

[0946] The user opens the app, enters their home address (Minato Ward, Tokyo), the travel distance (2 km), and the price difference (50 yen). They then enter the menu (curry and spaghetti) and the missing ingredients (carrots, potatoes, onions, and tomato sauce).

[0947] The server receives the information, collects the latest flyer data from nearby supermarkets and drugstores, and registers it in a database.

[0948] Image Recognition and Data Analysis

[0949] The user takes a photo of flyers from nearby supermarkets A and B and uploads them to the app.

[0950] The server extracts the data "Carrots: 100 yen, Potatoes: 80 yen, Onions: 60 yen, Tomato sauce: 120 yen" from flyer A and "Carrots: 90 yen, Potatoes: 100 yen, Onions: 70 yen, Tomato sauce: 110 yen" from flyer B, lists them, and saves them in a database.

[0951] Selection and notification of optimal stores

[0952] The server compares the extracted price information and determines that Supermarket A has a lower total cost (360 yen). Based on this, Supermarket A is selected as the optimal store.

[0953] The server uses the Google Maps API to generate a navigation route to Supermarket A and sends it to the user's device along with the shopping list.

[0954] The terminal notifies the user of the information received from the server and displays a shopping list along with navigation for heading to Supermarket A.

[0955] In this way, the present invention provides a system that allows users to shop most efficiently and economically, while also enhancing the effectiveness of store promotions.

[0956] The processing flow will be explained below.

[0957] Step 1:

[0958] The user launches the app and enters their home address (Minato Ward, Tokyo), the distance they can travel (2 km), and the price difference (50 yen). They then enter their weekly menu (curry and spaghetti) and a list of ingredients they are missing (carrots, potatoes, onions, and tomato sauce).

[0959] Step 2:

[0960] The server receives the user's input information and saves it as a profile. It also collects location information and the latest flyer data for nearby supermarkets and drugstores and registers them in a database.

[0961] Step 3:

[0962] The user takes a photo of flyers from nearby supermarkets A and B with their smartphone and uploads the flyer image data to the server via the app.

[0963] Step 4:

[0964] The server passes the received flyer image data to an image recognition module, which converts the image data into text data. For example, the product name, price, and special sale information are extracted from the flyer.

[0965] Step 5:

[0966] The server creates a list of product types and their price information from the extracted text data and stores the list in a database.

[0967] Step 6:

[0968] The server compares product prices based on the accumulated price information, taking into consideration the user's travelable distance and the acceptable range of price difference.

[0969] Step 7:

[0970] The server selects the best store for each product and calculates the minimum total cost for the user. For example, it checks that the total cost for supermarket A is 360 yen and the total cost for supermarket B is 370 yen.

[0971] Step 8:

[0972] The server obtains the optimal route using a map service (e.g., Google Maps API) to generate a navigation route to the selected optimal store (e.g., Supermarket A).

[0973] Step 9:

[0974] The server creates a shopping list based on the information input by the user and transmits it to the user terminal together with the navigation route.

[0975] Step 10:

[0976] The device receives information from the server and notifies the user of the best store information, navigation route, and shopping list. For example, it might display "Route to Supermarket A" with "Carrots: 100 yen, Potatoes: 80 yen, Onions: 60 yen, Tomato sauce: 120 yen."

[0977] The above steps realize a system that allows users to efficiently shop at the most cost-effective store and also enhances the store's sales promotion effects.

[0978] Example 1

[0979] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0980] Users need to expend a lot of time and effort to purchase the products they need efficiently and economically. Manually comparing advertising media and flyers from multiple stores is extremely tedious. Furthermore, it is difficult to optimally select stores to visit and navigate them along efficient routes. This degrades the user's shopping experience and reduces the effectiveness of store promotional activities. To solve this problem, a system is needed that allows users to easily input information and automatically provides optimal store information and navigation routes.

[0981] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0982] In this invention, the server includes means for inputting a user's preset home address, travel distance, and price difference, means for collecting image data from advertising media of multiple stores and uploading it to the server, means for converting the image data into text data and extracting product and price information, means for selecting the optimal store based on the extracted product and price information based on the user's settings, means for generating a navigation route to the optimal store and notifying the user terminal, means for inputting a shopping list preset by the user, and means for selecting the optimal store based on the shopping list and generating a navigation route. This allows users to shop efficiently and economically and also enhances the effectiveness of store promotions.

[0983] "User" refers to an individual who uses the system to input setting information and flyer images and receives optimal store information and navigation routes.

[0984] "Home address" refers to the specific location information of the place where the user lives.

[0985] "Moveable distance" refers to the maximum distance a user can move from their home.

[0986] "Price difference" refers to the difference in product price that a user can accept.

[0987] "Advertising media" refers to print or digital sources containing product information and special offers provided by a store.

[0988] "Image data" refers to a digital image file that captures the content of advertising media.

[0989] "Server" refers to a computer system that processes collected image data and provides optimal information to users.

[0990] "Text data" refers to character information extracted from image data.

[0991] "Product" refers to a specific item that a user is considering purchasing.

[0992] "Price information" refers to price information corresponding to each product.

[0993] The "optimal store" refers to a store that meets the user's set conditions and offers the most economical products.

[0994] A "navigation route" refers to the path a user takes to reach the optimal store.

[0995] "Notifying" refers to sending information to a user terminal to notify the user.

[0996] A "purchase list" refers to a list of products and quantities that a user plans to purchase.

[0997] "Image recognition algorithm" refers to a computational method or program for extracting text data from image data.

[0998] "Terminal" refers to an electronic device that a user operates and receives information from.

[0999] This technology allows users to input their home address, travel distance, and price difference, collect image data from advertising media of multiple stores, upload it to a server, and provide the store with the best deal. The system aims to improve the user's shopping experience and promote store promotional activities.

[1000] Overall system configuration

[1001] This system includes a terminal (e.g., a smartphone) on which users input information, photograph flyers, and upload them to a server, a server, and various program modules that run on the server. Potential hardware used includes a smartphone, a server, and a high-performance CPU or GPU for image processing. Software used includes an image processing library (e.g., OpenCV), an image recognition model (e.g., TensorFlow, Google Cloud Vision API), and a database management system (e.g., MySQL).

[1002] Specific operating procedures

[1003] Entering User Preferences Information

[1004] The user launches the smartphone app and first enters their home address. For example, the user enters "Minato Ward, Tokyo," sets the travel distance to "2 km," and the price difference to "50 yen." The user also enters the weekly menu as "Monday: curry, Tuesday: spaghetti," and the missing ingredients as "carrots, potatoes, onions, tomato sauce."

[1005] Collect flyer images and upload them to the server

[1006] The user takes a picture of a flyer from a nearby store (e.g., a supermarket or drugstore) with their smartphone camera. The captured image data is sent to the server by pressing the "Upload" button in the app. The program notifies the user that the upload is complete.

[1007] Image data analysis

[1008] The server analyzes the received image data. TensorFlow and Google Cloud Vision API are used as image recognition models. This allows text data such as product names, prices, and sale information to be extracted from the image. For example, information such as "Carrots: 100 yen, Potatoes: 80 yen, Onions: 60 yen, Tomato sauce: 120 yen" can be extracted from a flyer. The extracted data is stored in a database.

[1009] Selecting the best store and generating navigation routes

[1010] The server selects the most suitable store based on the price information in the database, taking into account the user's settings (home address, travel distance, price difference). Once the most suitable store is determined, the server generates a navigation route using a map service (e.g., Google Maps API). At the same time, it also takes into account the shopping list entered by the user and sets a route that allows the user to efficiently purchase the necessary products.

[1011] User Notification and Guidance

[1012] The server sends the generated navigation route and shopping list to the user's smartphone. The user receives a notification from the device and can follow the information displayed on the screen to find the optimal store. The displayed information includes the store name, address, list of products to be purchased, and navigation route.

[1013] Specific examples

[1014] The user opens the app and enters their home address (Minato Ward, Tokyo), the travel distance (2 km), and the price difference (50 yen). They also enter their menu (curry and spaghetti) and the missing ingredients (carrots, potatoes, onions, and tomato sauce). Next, the user takes photos of flyers for nearby supermarkets A and B and uploads them to the app. The server extracts the following data from flyer A: "Carrots: 100 yen, potatoes: 80 yen, onions: 60 yen, tomato sauce: 120 yen"; from flyer B: "Carrots: 90 yen, potatoes: 100 yen, onions: 70 yen, tomato sauce: 110 yen." These data are then compiled into a list and stored in a database. The server compares the extracted price information and determines that supermarket A offers the best total cost (360 yen). The server uses the Google Maps API to generate a navigation route to supermarket A and sends it to the user's device along with the shopping list. The device notifies the user of the received information and displays the shopping list along with navigation to supermarket A. This allows users to shop in the most efficient and economical way possible.

[1015] In this way, the present invention provides a system that enables users to shop efficiently and economically, and also enhances the effectiveness of store promotions.

[1016] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1017] Step 1:

[1018] The user enters the configuration information

[1019] A user launches the app and enters their home address (e.g., "Minato Ward, Tokyo"), the distance they can travel (e.g., "2 km"), the price difference (e.g., "50 yen"), and a shopping list (e.g., "carrots, potatoes, onions, tomato sauce"), which then creates a personalized shopping profile for the user.

[1020] Input: Home address, travel distance, price difference, shopping list

[1021] Output: User profile data

[1022] Specific behavior: The user enters information into the app using form input fields and then clicks the "Save Settings" button to send it to the server.

[1023] Step 2:

[1024] The server stores user information

[1025] The profile data sent by the user is received by the server, which stores it in a database and updates the user profile.

[1026] Input: User profile data

[1027] Output: Updated database user profile

[1028] Specific operation: The server parses the received data and stores it in a structured database (e.g., MySQL).

[1029] Step 3:

[1030] The user takes and uploads a flyer image

[1031] Users take a photo of a flyer from a nearby store using their smartphone camera, then use the app's upload function to upload the flyer image data to the server.

[1032] Input: Flyer image data

[1033] Output: Flyer image uploaded to the server

[1034] Specific operation: The user takes a photo of a flyer with the camera and presses the "Upload image" button to send the image to the server.

[1035] Step 4:

[1036] The server analyzes the flyer image

[1037] The server analyzes the received flyer image data using an image recognition algorithm (e.g., TensorFlow, Google Cloud Vision API). As a result of the analysis, product names, prices, special sale information, etc. are extracted as text data.

[1038] Input: Flyer image data

[1039] Output: Product information text data

[1040] Specific operation: The server loads the image recognition model, inputs a flyer image, and extracts text data including product name and price information.

[1041] Step 5:

[1042] The server stores the product information in a database

[1043] The extracted text data of the product information is stored in a database, and price information for each product is saved.

[1044] Input: Text data of product information

[1045] Output: Product information stored in the database

[1046] Specific operation: The server structures the text data and stores it by adding a new entry to the database.

[1047] Step 6:

[1048] The server selects the best store

[1049] The server selects the most cost-effective store within the specified travel distance and price range based on the user's purchase list and price information.

[1050] Input: Purchase list, user profile data, product information

[1051] Output: Optimal store information

[1052] How it works: The server searches and compares product prices from a database and uses an algorithm to identify the best store.

[1053] Step 7:

[1054] The server generates the navigation route

[1055] A navigation route to the selected optimal store is generated using a map service (e.g., Google Maps API).

[1056] Input: Best store information, home address

[1057] Output: Navigation route

[1058] Specific behavior: The server queries the Google Maps API to obtain a navigation route and processes it.

[1059] Step 8:

[1060] The server notifies the user of the navigation route and shopping list.

[1061] The server sends the generated navigation route and shopping list to the user's smartphone.

[1062] Input: navigation route, purchase list

[1063] Output: User notification

[1064] Specific operation: The server sends information to the user's device, and the app receives and displays the notification.

[1065] (Application example 1)

[1066] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1067] With so many choices available to modern consumers, it can be difficult to determine where and which products are most economical to purchase. Stores also have limited means of efficiently advertising to consumers. This creates a need for a system that supports optimal purchasing choices tailored to individual consumer needs while simultaneously enhancing store promotional effectiveness.

[1068] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1069] In this invention, the server includes means for inputting the user's preset home address, travel distance, and price difference, means for taking images of advertising media for multiple stores with a smartphone and uploading them to the server, means for converting the image data into text data using an image recognition algorithm and extracting product and price information, means for selecting the optimal store from the extracted product and price information based on the user's settings, and means for generating a navigation route to the optimal store using a map service and notifying the user terminal. This allows the user to receive suggestions for optimal stores in real time, enabling efficient and economical purchasing activities.

[1070] "User" refers to a consumer who uses this system.

[1071] "Home address" refers to the specific address information of the home specified by the user.

[1072] The "travelable distance" refers to the distance that indicates the range that the user can travel from their home.

[1073] "Price difference" refers to the price difference that a user is willing to accept.

[1074] "Multiple stores" refers to multiple target commercial facilities.

[1075] "Advertising media" refers to distributed advertisements such as flyers and pamphlets provided by stores.

[1076] "Image data" refers to image files of advertising media taken with a smartphone or other device.

[1077] "Smartphone" refers to a portable information terminal equipped with mobile communication functions.

[1078] "Server" refers to a computer on a network that processes, stores, and distributes image data.

[1079] An "image recognition algorithm" refers to a program for extracting text information and product information from image data.

[1080] "Text data" refers to a character string extracted from image data.

[1081] "Product and price information" refers to information about the product name and price listed in the advertising media.

[1082] The "best store" refers to the store that best meets the user's requirements.

[1083] "Navigation route" refers to a route showing directions from the current location to the destination.

[1084] "Map Service" refers to a service that provides digital maps and supports navigation functions.

[1085] "User terminal" refers to a connected device such as a smartphone or PC used by a user.

[1086] The system for implementing this invention automates a series of steps for users to find the best store. Users input their home address, travel distance, and price difference into the application, and the system begins operation based on this information.

[1087] Hardware and Software Overview

[1088] Server: A computing device for large-scale data processing and analysis. The server contains software for running image recognition algorithms (e.g., pytesseract), a database management system (e.g., PostgreSQL), and a map service API for generating navigation routes (e.g., Google Maps API).

[1089] User device: A smartphone or tablet used by a user. An application is installed on this device, which receives user input, photographs advertising media in nearby stores, and uploads them to a server.

[1090] Process Overview

[1091] User input of settings

[1092] Through the application, users input information such as their home address, travel distance, price difference, etc. They can also input a weekly menu and a list of ingredients they are missing, and the system then creates and saves a user profile.

[1093] Specific input example:

[1094] Home address: "Minato-ku, Tokyo"

[1095] Travel distance: "2km"

[1096] Price difference: "50 yen"

[1097] Menu: "Curry and Spaghetti"

[1098] Missing ingredients list: "Carrots, potatoes, onions, tomato sauce."

[1099] Photograph and upload flyers

[1100] Users use their smartphones to take photos of advertising flyers from nearby stores and upload the image data to a server via the application.

[1101] Image data analysis

[1102] The server receives the uploaded image data and uses an image recognition algorithm (e.g., pytesseract) to extract product names, prices, and special offers, which are then stored in a database as text data.

[1103] Selection of optimal stores

[1104] The server compares product prices based on the product information stored in the database, taking into account the travel distance and price difference set by the user, and selects the optimal store for each product, performing calculations to minimize the user's total cost.

[1105] Navigation route generation and notification

[1106] The server uses a map service API (e.g., Google Maps API) to generate a navigation route to the best store. The generated route is sent to the user's device and notified along with the shopping list.

[1107] Specific prompt examples:

[1108] "I live in Minato Ward, Tokyo. Please find a supermarket within a 2km radius where the price difference is within 50 yen. I would like to buy carrots, potatoes, onions, and tomato sauce."

[1109] In this way, users can find the most economical shopping destination in real time, enabling them to shop efficiently, and stores can also advertise effectively to consumers.

[1110] The above is a specific description of the embodiment of the invention.

[1111] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1112] Step 1: User inputs settings

[1113] Input: The user enters their home address, travel distance, price difference, weekly meal plan, and missing ingredients into the application.

[1114] Data processing: The information entered by the user is sent to the server and saved in a database as a profile.

[1115] Output: The server creates a user profile and outputs it as saved data.

[1116] Step 2: Photograph and upload your flyer

[1117] Input: A user uses their smartphone to take a photo of a flyer for a nearby store.

[1118] Data processing: The captured image data is uploaded to the server via the application.

[1119] Output: The server receives and stores the uploaded image data.

[1120] Step 3: Analyzing the image data

[1121] Input: Image data received by the server.

[1122] Data processing: The server uses an image recognition algorithm (e.g., pytesseract) to convert the image data into text data, specifically extracting product names, prices, and special offers.

[1123] Output: The extracted product name, price, and sale information are saved as text data in a database.

[1124] Step 4: Select the best store

[1125] Input: User profile information (home address, travel distance, price difference) and product information stored in the database.

[1126] Data processing: The server compares product prices based on the user's settings and calculates the best store to minimize the user's total cost.

[1127] Output: Information on the optimal store and calculation results are generated.

[1128] Step 5: Generate and notify navigation route

[1129] Input: User's home address and best store information.

[1130] Data processing: The server uses a map service API (e.g., Google Maps API) to generate a navigation route to the optimal store.

[1131] Output: The generated navigation route and shopping list are sent to the user's device.

[1132] Specific behavior:

[1133] In step 1, the user enters the required information into the application, which is then saved as a profile on the server.

[1134] In step 2, the user takes a photo of the flyer with their smartphone and uploads it to the server via the application.

[1135] In step 3, the server uses an image recognition algorithm to convert the received image data into text data, specifically extracting product names, prices, and sale information, and storing it in a database.

[1136] In step 4, the server performs calculations to select the most suitable store based on the user's profile information and the product information stored in the database.

[1137] In step 5, the server uses the map service API to generate a navigation route and sends it to the user's terminal together with the shopping list.

[1138] This series of processes enables users to make the most efficient and economical purchases in real time.

[1139] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1140] The present invention is a system that streamlines the user's shopping experience by allowing the user to input their home address, travel distance, and price difference in advance. Furthermore, by combining this system with an emotion engine that recognizes the user's emotions, it is possible to select the most appropriate store according to the user's emotional state.

[1141] Overview of the overall system operation

[1142] Setup Phase

[1143] 1. Users

[1144] Enter your home address into the app, and set the allowable travel distance (e.g., 2km) and price difference (e.g., 50 yen).

[1145] You can also enter your weekly menu and a list of ingredients you're missing, such as "Monday: curry, Tuesday: spaghetti" or "carrots, potatoes, onions, tomato sauce."

[1146] As you use the app to consider ingredients or your shopping list, the emotion engine recognizes your emotions in real time.

[1147] 2. Server

[1148] It receives user input information and saves it as a profile. It also collects location information and the latest flyer data for nearby supermarkets and drugstores and stores them in a database.

[1149] The emotion engine analyzes and stores emotional data entered by the user in real time.

[1150] Image Recognition Phase

[1151] 3. Users

[1152] Take a photo of a flyer from a local supermarket or drugstore with your smartphone.

[1153] The image data of the photographed flyer is uploaded to the server via the app.

[1154] 4. Server

[1155] The received flyer image data is passed to an image recognition module, which converts the image data into text data. For example, the product name, price, and special sale information are extracted from the flyer.

[1156] The extracted text data is used to create a list of product types and their price information, which is then stored in a database.

[1157] Optimal store proposal phase

[1158] 5. Server

[1159] Product prices are compared based on accumulated price information, the travel distance set by the user, and the acceptable range of price differences.

[1160] The emotion engine assesses the user's emotional state and applies an algorithm that prioritizes the selection of the nearest store if, for example, stress levels are high.

[1161] 6. Server

[1162] The optimal store is selected for each product, and calculations are performed to minimize the total cost for the user. For example, it is confirmed that the total cost for supermarket A is 360 yen and the total cost for supermarket B is 370 yen.

[1163] The emotion engine also takes into account past emotional data and makes suggestions optimized for the user's shopping style.

[1164] 7. Server

[1165] To generate a navigation route to the selected optimal store (e.g., Supermarket A), a map service (e.g., Google Maps API) is used to obtain the optimal route.

[1166] A shopping list is created based on the information input by the user and is sent to the user terminal together with the navigation route.

[1167] 8. Terminal

[1168] It receives information from the server and notifies the user of the best store information, navigation route, and shopping list. For example, it displays "Route to Supermarket A" and "Carrots: 100 yen, Potatoes: 80 yen, Onions: 60 yen, Tomato sauce: 120 yen."

[1169] In addition, the emotion engine will suggest less crowded times, for example, to help users shop in a less stressful environment.

[1170] Specific examples

[1171] Setup and Initial Preparation

[1172] The user opens the app and enters their home address (Minato-ku, Tokyo), the travel distance (2km), and the price difference (50 yen). They then enter the menu (curry and spaghetti) and the missing ingredients (carrots, potatoes, onions, and tomato sauce). The emotion engine reads the user's emotions while they are entering and considering their choices, and accumulates the data.

[1173] The server receives the information, collects the latest flyer data from nearby supermarkets and drug stores, and stores it in a database. The emotion engine also stores the user's emotion data and uses it for future optimization.

[1174] Image Recognition and Data Analysis

[1175] The user takes a photo of flyers from nearby supermarkets A and B and uploads them to the app.

[1176] The server extracts the data "Carrots: 100 yen, Potatoes: 80 yen, Onions: 60 yen, Tomato sauce: 120 yen" from flyer A and "Carrots: 90 yen, Potatoes: 100 yen, Onions: 70 yen, Tomato sauce: 110 yen" from flyer B, lists them, and saves them in a database.

[1177] Selection and notification of optimal stores

[1178] The server compares the extracted price information and determines that Supermarket A has a lower total cost (360 yen). Based on this, Supermarket A is selected as the optimal store. The server evaluates the user's emotional state, and if, for example, stress is high, prioritizes the nearest store.

[1179] The server uses the Google Maps API to generate a navigation route to Supermarket A and sends it to the user's device along with the shopping list. Using the emotion engine, it also includes suggestions for reducing stress (e.g., less crowded times).

[1180] The terminal receives information from the server and notifies the user of the navigation and purchase list for Supermarket A. For example, it displays "Route to Supermarket A" and "Carrots: 100 yen, Potatoes: 80 yen, Onions: 60 yen, Tomato sauce: 120 yen."

[1181] In this way, the present invention provides a system that allows for the most efficient and economical shopping experience, while also taking into account the user's emotional state.

[1182] The processing flow will be explained below.

[1183] Step 1:

[1184] The user launches the app and enters their home address (Minato Ward, Tokyo), the distance they can travel (2 km), and the price difference (50 yen). They also enter their weekly menu and a list of ingredients they are missing. For example, "Monday's dinner is curry, Tuesday's is spaghetti," and "Ingredients needed: carrots, potatoes, onions, tomato sauce."

[1185] Step 2:

[1186] While the user is using the app, the emotion engine analyzes the user's facial expressions, voice, etc. to obtain emotional data in real time. The emotional state (e.g., stress level, elation, etc.) is evaluated and stored in a database.

[1187] Step 3:

[1188] The server receives the user's settings and emotional data and stores them as a profile. It also collects location information and the latest flyer data for nearby supermarkets and drugstores and stores them in a database.

[1189] Step 4:

[1190] The user takes a photo of flyers from nearby supermarkets A and B with their smartphone, and uploads the flyer image data to the server via the app.

[1191] Step 5:

[1192] The server passes the received flyer image data to an image recognition module, which converts the image data into text data (e.g., extracting product names, prices, and sale information from the flyer). The extracted text data is analyzed, and product types and their prices are listed and stored in a database.

[1193] Step 6:

[1194] The server compares product prices based on the accumulated price information, the user's travel distance, price tolerance, and emotional data. If the user's stress level is high, an algorithm is applied that prioritizes the selection of the nearest store.

[1195] Step 7:

[1196] The server selects the optimal store for each product and calculates the lowest total cost (e.g., the total cost for supermarket A is 360 yen, and the total cost for supermarket B is 370 yen). The optimal store is selected based on the user's emotional data (e.g., supermarket B may be selected because it has low stress levels).

[1197] Step 8:

[1198] The server considers the user's emotional data and generates a navigation route to the optimal selected store (e.g., obtains the optimal route using Google Maps API). It also makes suggestions to reduce stress (e.g., less crowded times).

[1199] Step 9:

[1200] The server creates a shopping list based on the information input by the user and transmits it to the user terminal together with the navigation route.

[1201] Step 10:

[1202] The terminal notifies the user of the information received from the server (for example, displaying the name, location, navigation route, shopping list, etc. of the best store). Specifically, for example, it displays "Route to Supermarket A" and "Carrots: 100 yen, Potatoes: 80 yen, Onions: 60 yen, Tomato sauce: 120 yen."

[1203] Through the above steps, the user can shop efficiently at the most suitable store that takes into consideration the user's emotional state, achieving a less stressful shopping experience.

[1204] Example 2

[1205] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1206] The present invention aims to streamline the user's shopping experience and provide an optimal shopping environment that takes into account the user's emotional state. Specifically, the present invention aims to provide a system that selects the optimal store based on the user's specified address, travel distance, and price difference, and also evaluates the user's emotional state to support a stress-free shopping experience.

[1207] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for inputting the user's home address, travel distance, and price difference, which are set in advance by the user; a means for collecting image data from advertising media of multiple business facilities and uploading it to the server; a means for converting the image data into text data and extracting product and price information therefor; a means for selecting the optimal business facility from the extracted product and price information based on the user's settings; a means for evaluating the user's emotional state and optimizing store selection based on the evaluation results; and a means for generating a navigation route to the optimal business facility and notifying the user terminal. This allows the user to have an efficient and emotionally-conscious shopping experience.

[1208] "Address" refers to specific geographical information about the place where the user lives, and serves as a criterion for selecting a store within the user's permitted range of movement.

[1209] The "travelable distance" refers to the distance of the travelable range that the user is willing to travel, and serves as a criterion for selecting the most suitable store within this range.

[1210] "Price difference" refers to the difference in product prices that a user is willing to accept, and serves as a criterion for comparing prices within this range to select a store.

[1211] "Business facilities" refer to facilities such as stores, supermarkets, and drugstores that sell products, and are places that provide products for users to purchase.

[1212] "Advertising media" refers to advertising information provided by each business establishment, and primarily includes printed media such as flyers and posters.

[1213] "Image data" refers to visual information of advertising media photographed and uploaded by a user and stored in digital format.

[1214] "Text data" refers to character information extracted by analyzing image data, and includes product names, price information, and the like.

[1215] A "product" refers to an item that a user considers purchasing, and is accompanied by a specific name and price information.

[1216] "Price information" refers to the specific price associated with a product, and serves as a basis for users to select the most suitable place to purchase.

[1217] "Emotional state" refers to the user's psychological state and emotions, and by evaluating this, the system is able to select and suggest the most suitable store for the user.

[1218] A "navigation route" refers to a specific route for a user to travel to a selected business facility, and is generated using a map service.

[1219] "User terminal" refers to an electronic device such as a computer or smartphone that a user uses to input information and receive notifications from the system.

[1220] The present invention is a system that streamlines the user's shopping experience by allowing the user to input their home address, travel distance, and price difference in advance. In addition, it has the function of recognizing the user's emotional state in real time and selecting the most suitable store based on that state.

[1221] Hardware and Software Configuration

[1222] User side

[1223] User device: A device, such as a smartphone or computer, that a user uses to enter information and receive notifications from the server.

[1224] Camera: The camera attached to the user terminal is used to capture image data of the advertising medium.

[1225] Server side

[1226] Server: The central system that receives the information sent by the user and performs analysis and data processing. This includes the database management system, image recognition module, and emotion engine.

[1227] Database management systems: Used to store and manage user profile information and advertising data. Examples include MySQL and PostgreSQL.

[1228] Image recognition module: A software module for converting image data from advertising media into text data. Specific software examples include OpenCV and Tesseract OCR.

[1229] Emotion engine: A software module for analyzing and storing a user's emotional state in real time. It uses natural language processing and machine learning algorithms to evaluate the user's emotions.

[1230] System Operation Overview

[1231] User Settings

[1232] The user launches the app and enters their home address, travel distance, and price difference. They also enter a week's worth of meals and a list of ingredients they are missing. For example, they set their address to "Minato Ward, Tokyo," their travel distance to "2 km," and their price difference to "50 yen," and then enter "curry and spaghetti" as their meal plan and "carrots, potatoes, onions, and tomato sauce" as the ingredients they are missing. This information is saved in the app.

[1233] Data collection and analysis

[1234] Users take a photo of a flyer from a nearby supermarket or drugstore with their smartphone camera and upload it to the server via the app. The server passes the received flyer image to an image recognition module, which converts data such as product names and prices into text. For example, information such as "Carrots: 100 yen, Potatoes: 80 yen, Onions: 60 yen, Tomato sauce: 120 yen" is extracted and stored in a database.

[1235] Selection and proposal of optimal stores

[1236] The server selects the optimal store that meets the user's criteria based on price information stored in a database. It creates a shopping list based on a weekly menu and a list of ingredients that are missing, and uses an emotion engine to evaluate the user's emotional state. For example, if the user is feeling stressed, it will prioritize the nearest store. The server selects the optimal store based on this data and generates a navigation route using Google Maps API, etc. The generated route and shopping list are sent to the user's device.

[1237] Notifications and Support

[1238] Based on the information received from the server, the user device notifies the user of the optimal store information, navigation route, and shopping list. For example, it displays "Optimal store: Supermarket A, navigation route, 'Carrots: 100 yen, Potatoes: 80 yen, Onions: 60 yen, Tomato sauce: 120 yen'." The emotion engine supports shopping in a less stressful environment for the user by making suggestions that take into account, for example, less crowded times.

[1239] Specific examples

[1240] The user opens the app, sets their home address to "Minato-ku, Tokyo," the travel distance to "2 km," the price difference to "50 yen," and inputs the menu item "curry and spaghetti" and the list of missing ingredients: "carrots, potatoes, onions, and tomato sauce."

[1241] The server receives the information and collects the latest flyer data from nearby supermarkets and drugstores, and the emotion engine also stores the user's emotion data.

[1242] When a user takes a photo of flyers from nearby supermarkets A and B and uploads them to the app, the server extracts price data from flyer A, compares them, and evaluates the total cost. It uses an emotion engine to take into account the user's stress level and makes appropriate suggestions.

[1243] The server sends the generated shopping list and navigation route to the user's terminal, helping the user to shop efficiently at the most suitable store.

[1244] Prompt Sentence Examples

[1245] Example prompt 1: "My home address is in Minato Ward, Tokyo. Please tell me the best store to buy ingredients for curry and spaghetti within a 2km travel distance and within a price difference of 50 yen. Also, please help me have a less stressful shopping experience by taking my emotional state into consideration."

[1246] Example prompt 2: "Today's list of things I need is carrots, potatoes, onions, and tomato sauce. I've uploaded images of flyers from nearby supermarkets. Please suggest the store with the lowest cost and least stress."

[1247] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1248] Step 1:

[1249] The user launches the app and enters their home address, travel distance, and price difference. For example, the address can be set to "Minato-ku, Tokyo," the travel distance to "2 km," and the price difference to "50 yen." The input data also includes a week's worth of meals (e.g., "curry and spaghetti") and a list of missing ingredients (e.g., "carrots, potatoes, onions, tomato sauce"). This information is sent to the server from the device operated by the user. Input: Address, travel distance, price difference, meals, list of ingredients. Output: User profile information stored on the server.

[1250] Step 2:

[1251] The server receives the information sent by the user and stores it in a database management system (e.g., MySQL, PostgreSQL). At the same time, the emotion engine analyzes and accumulates the user's emotion data in real time. It also collects location information of nearby business facilities and the latest advertising data and stores them in the database. Input: User profile information. Output: Location information, advertising data, and emotion data stored in the database.

[1252] Step 3:

[1253] The user takes a photo of advertising media in nearby business establishments with their smartphone camera and uploads it to the server via the app. For example, they take photos of flyers from supermarkets A and B. This image data is sent directly to the server. Input: Image of advertising media. Output: Image data uploaded to the server.

[1254] Step 4:

[1255] The image data of the advertising media received by the server is passed to an image recognition module (e.g., OpenCV, Tesseract OCR) and converted into text data. Specifically, the image recognition module extracts the product name, price, and sale information, and stores them in a database. For example, it extracts data such as "carrots: 100 yen, potatoes: 80 yen, onions: 60 yen, tomato sauce: 120 yen." Input: Image data of the advertising media. Output: Text data stored in the database.

[1256] Step 5:

[1257] The server applies an algorithm to select the optimal store based on the product price information stored in the database, within the address, travel distance, and price difference range set by the user. Furthermore, an emotion engine is used to evaluate the user's emotional state, and if, for example, stress levels are high, the system prioritizes closer stores. Input: Text data, user profile information, emotion data. Output: Optimal store selection results.

[1258] Step 6:

[1259] The server calculates the total cost and selects the optimal business facility. For example, if the total cost of supermarket A is 360 yen and the total cost of supermarket B is 370 yen, supermarket A is selected. Based on this, supermarket A is determined to be the optimal store. Input: store price data, user setting information. Output: optimal store and total cost information.

[1260] Step 7:

[1261] The server uses a map service (e.g., Google Maps API) to generate a navigation route to the optimal store. It also sends the generated navigation route and shopping list to the user's device. Input: optimal store information, user location information. Output: navigation route, shopping list.

[1262] Step 8:

[1263] The terminal receives the information sent from the server and notifies the user. Specifically, it displays the optimal store information, navigation route, and shopping list for the user. For example, it notifies the user of information such as "Optimal store: Supermarket A, navigation route, 'Carrots: 100 yen, Potatoes: 80 yen, Onions: 60 yen, Tomato sauce: 120 yen'." It also makes suggestions such as less crowded times. Input: Store information from the server, navigation route, shopping list. Output: Notification and suggestions to the user.

[1264] (Application example 2)

[1265] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1266] The present invention relates to a system that allows users to shop efficiently. There is a particular need for a system that takes into account the user's emotional state and provides optimal store selection while reducing stress. Conventional systems select stores based solely on price and distance, ignoring the user's emotional state and potentially reducing the quality of the shopping experience. Furthermore, there is also the issue that users take a long time to find the optimal store because price information from advertising materials is not adequately extracted and optimized.

[1267] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for inputting the user's home address, travel distance, and price difference, which are set in advance by the user; means for collecting image data from advertising materials for multiple sales points and uploading it to the server; means for converting the image data into text data and extracting product and price information; means for selecting the optimal sales point from the extracted product and price information based on the user's settings; means for analyzing the user's emotional state in real time and optimizing store selection in accordance with the emotional state; and means for generating a route to the optimal sales point and notifying the user terminal. This allows for efficient and optimal shopping while reducing stress, taking the user's emotional state into consideration.

[1268] The "home address preset by the user" is information that the user inputs into the system in advance to indicate the place of residence of the user.

[1269] The "travelable distance" is information indicating the maximum distance that the user is allowed to travel for shopping.

[1270] "Price difference" is information indicating the allowable price difference for the same product.

[1271] "Sales point advertising materials" are materials that show flyers and campaign information issued by stores such as supermarkets and drugstores.

[1272] "Image data" refers to image data obtained from advertising materials at points of sale.

[1273] The "means for uploading to the server" is a method or function for transferring image data collected from the user terminal to the server.

[1274] "Text data" is character string information extracted and converted from image data.

[1275] "Product and price information" is information indicating the name of the product and its selling price listed in the advertising material.

[1276] The "means for selecting the most suitable sales point" is a method or function for automatically selecting the most suitable store for the user based on the user's setting information and price information.

[1277] The "means for generating a travel route" is a method or function for generating a route that enables a user to travel to the optimal sales point in the shortest time or with the least stress.

[1278] The "means for analyzing the user's emotional state in real time" is a method or function for recognizing the user's emotions in real time and analyzing the data.

[1279] The "means for optimizing store selection according to emotional state" is a method or function for selecting an optimal store by taking into account the emotional state of the user.

[1280] The "notification means" is a method or function for notifying the user terminal of information regarding the optimum sales point and its travel route.

[1281] The present invention is a system that streamlines the user's shopping experience and selects the optimal store based on the user's emotional state. The system works by combining and processing user input information, image data from point-of-sale advertising materials, and the user's emotional state.

[1282] The server implements the present invention using the following means.

[1283] 1. Enter your user settings information

[1284] Users input their home address, the distance they can travel, and the price difference, which allows the system to determine the range within which they can shop.

[1285] 2. Collecting and uploading advertising materials

[1286] Users take photos of advertising materials from multiple points of sale using a device such as a smartphone and upload them to a server, at which point image data is collected.

[1287] 3. Conversion to text data

[1288] The server uses an image recognition algorithm (e.g., Tesseract) to convert the uploaded image data into text data, from which product names and their prices are extracted.

[1289] 4. Selection of optimal sales locations

[1290] The server selects the optimal sales location based on the extracted price information and the user's settings (address, distance, price difference), and then analyzes the user's emotional state in real time (e.g., using TensorFlow) to optimize store selection according to the user's emotional state.

[1291] 5. Generate navigation route

[1292] The server generates the optimal travel route for the user to the selected optimal sales point. Specifically, it creates the route using a map service (e.g., Google Maps API).

[1293] 6. Notification to user devices

[1294] This information is then sent to the user's device, which includes the best shopping locations, product pricing information, and navigation routes.

[1295] This system allows users to have an efficient and optimal shopping experience, and also reduces stress by selecting stores according to their emotional state.

[1296] Specific examples of hardware and software used

[1297] Hardware: smartphones, servers, cameras

[1298] Software: TensorFlow (sentiment analysis), Tesseract (image recognition), Google Maps API (navigation)

[1299] Examples of prompt statements

[1300] Here is an example of inputting the following prompt into a generative AI model:

[1301] "The user enters their address, travel distance, price difference, meal plan information, and a list of missing ingredients into the app. Furthermore, if the user's emotional state is 'stressed,' please suggest the most efficient and economical store where they can shop, and why."

[1302] Examples of expected outputs include the following:

[1303] "Supermarket A is the best choice. It was selected for the following reasons: 1) Supermarket A is closest to the specified address. 2) Your current emotional state is stressed, so the travel distance is short and stress reduction is expected. 3) The total cost of the ingredients needed for the meal is the lowest. For these reasons, I recommend Supermarket A."

[1304] In this way, the present invention provides a system that allows for the most efficient and economical shopping experience, while also taking into account the user's emotional state.

[1305] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1306] Step 1:

[1307] The user enters their home address, travel distance, and price difference into the app.

[1308] Input: User's home address, travel distance, price difference

[1309] Output: User configuration information

[1310] How it works: A user launches the app and enters their home address, the distance they can travel (e.g., 2 km), and the price difference (e.g., 50 yen) into the interface. This information is then sent to the server.

[1311] Step 2:

[1312] A user takes pictures of advertising materials from multiple points of sale and uploads them to a server.

[1313] Input: Advertising image of the point of sale

[1314] Output: Image data stored on the server

[1315] How it works: Users use their smartphone camera to take a photo of a flyer from a nearby supermarket or drugstore, then upload the image data to a server via the app.

[1316] Step 3:

[1317] The server converts the received image data into text data and extracts product and price information.

[1318] Input: Image data

[1319] Output: Text data of products and their prices

[1320] How it works: The server uses an image recognition algorithm (e.g., Tesseract) to convert the uploaded image data into text data, then extracts product names and their prices from the text data and stores them in a database.

[1321] Step 4:

[1322] The server selects the optimal sales location based on the user's settings and the extracted price information.

[1323] Input: User settings, product and price information

[1324] Output: Optimal selling point

[1325] Operation: The server analyzes the price information in the database, taking into account the address, travel distance, and price difference set by the user, and selects the optimal sales location.

[1326] Step 5:

[1327] The server analyzes the user's emotional state in real time and optimizes store selection according to that emotional state.

[1328] Input: User's image data or voice data (emotional state)

[1329] Output: Optimal selling points based on emotional state

[1330] How it works: The server uses an emotion engine (e.g., TensorFlow) to analyze the user's emotional state in real time. If the emotional state is stressed, it prioritizes stores with shorter travel distances.

[1331] Step 6:

[1332] The server generates an optimal route to the sales point and notifies the user terminal.

[1333] Input: Best point of sale, user's current location

[1334] Output: Travel route

[1335] Operation: The server uses a map service (e.g., Google Maps API) to generate a route from the user's current location to the optimal sales point, and notifies the user's device of this route information.

[1336] Step 7:

[1337] The user checks the notified content on the terminal and carries out the purchase.

[1338] Input: Notification information from the server (optimal sales location, price information for each product, travel route)

[1339] Output: Actual shopping behavior

[1340] How it works: The user checks the notification on their device and efficiently shops based on the optimal sales point and travel route. For example, it displays "Route to Supermarket A" and shows "Carrots: 100 yen, Potatoes: 80 yen, Onions: 60 yen, Tomato sauce: 120 yen."

[1341] These steps allow users to shop efficiently and economically, and by taking into consideration their emotional state, they are able to reduce stress.

[1342] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1343] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1344] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1345] [Fourth embodiment]

[1346] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1347] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1348] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1349] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1350] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1351] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1352] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1353] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1354] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1355] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1356] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1357] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1358] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1359] The present invention is a system that provides the best deals by allowing users to input their home address, travel distance, and price difference, and then collects image data from the advertising media of multiple stores and uploads it to a server. This system is designed to improve the user's shopping experience and promote store promotional activities.

[1360] Overview of the overall system operation

[1361] Setup Phase

[1362] 1. Users

[1363] Enter your home address into the app, and also set the allowable travel distance (e.g., 2km) and price difference (e.g., 50 yen).

[1364] You can also enter your weekly menu and a list of ingredients you're missing, such as "Monday: curry, Tuesday: spaghetti" or "carrots, potatoes, onions, tomato sauce."

[1365] 2. Server

[1366] Load and prepare the analysis model for image recognition. Collect location information and the latest flyer data of nearby supermarkets and drugstores and store them in a database.

[1367] Save user settings information as a profile.

[1368] Image Recognition Phase

[1369] 3. Users

[1370] Take a photo of a flyer from a local supermarket or drugstore with your smartphone.

[1371] The image data of the photographed flyer is uploaded to the server via the app.

[1372] 4. Server

[1373] The received flyer image data is passed to an image recognition module, which converts the image data into text data. For example, the product name, price, and special sale information are extracted from the flyer.

[1374] The extracted text data is used to create a list of product types and their price information, which is then stored in a database.

[1375] Optimal store proposal phase

[1376] 5. Server

[1377] Based on the extracted price information, product prices are compared taking into consideration the travel distance and the tolerable range of price difference specified by the user.

[1378] The optimal store is selected for each product and calculations are performed to minimize the total cost to the user.

[1379] 6. Server

[1380] Based on the selected optimal store, a navigation route is generated using a map service, such as Google Maps API.

[1381] A shopping list is created based on the information input by the user and is sent to the user terminal together with the navigation route.

[1382] 7. Terminal

[1383] It receives information from the server and notifies the user, for example, displaying the name and location of the best store, navigation route, shopping list, etc.

[1384] Specific examples

[1385] Setup and Initial Preparation

[1386] The user opens the app, enters their home address (Minato Ward, Tokyo), the travel distance (2 km), and the price difference (50 yen). They then enter the menu (curry and spaghetti) and the missing ingredients (carrots, potatoes, onions, and tomato sauce).

[1387] The server receives the information, collects the latest flyer data from nearby supermarkets and drugstores, and registers it in a database.

[1388] Image Recognition and Data Analysis

[1389] The user takes a photo of flyers from nearby supermarkets A and B and uploads them to the app.

[1390] The server extracts the data "Carrots: 100 yen, Potatoes: 80 yen, Onions: 60 yen, Tomato sauce: 120 yen" from flyer A and "Carrots: 90 yen, Potatoes: 100 yen, Onions: 70 yen, Tomato sauce: 110 yen" from flyer B, lists them, and saves them in a database.

[1391] Selection and notification of optimal stores

[1392] The server compares the extracted price information and determines that Supermarket A has a lower total cost (360 yen). Based on this, Supermarket A is selected as the optimal store.

[1393] The server uses the Google Maps API to generate a navigation route to Supermarket A and sends it to the user's device along with the shopping list.

[1394] The terminal notifies the user of the information received from the server and displays a shopping list along with navigation for heading to Supermarket A.

[1395] In this way, the present invention provides a system that allows users to shop most efficiently and economically, while also enhancing the effectiveness of store promotions.

[1396] The processing flow will be explained below.

[1397] Step 1:

[1398] The user launches the app and enters their home address (Minato Ward, Tokyo), the distance they can travel (2 km), and the price difference (50 yen). They then enter their weekly menu (curry and spaghetti) and a list of ingredients they are missing (carrots, potatoes, onions, and tomato sauce).

[1399] Step 2:

[1400] The server receives the user's input information and saves it as a profile. It also collects location information and the latest flyer data for nearby supermarkets and drugstores and registers them in a database.

[1401] Step 3:

[1402] The user takes a photo of flyers from nearby supermarkets A and B with their smartphone and uploads the flyer image data to the server via the app.

[1403] Step 4:

[1404] The server passes the received flyer image data to an image recognition module, which converts the image data into text data. For example, the product name, price, and special sale information are extracted from the flyer.

[1405] Step 5:

[1406] The server creates a list of product types and their price information from the extracted text data and stores the list in a database.

[1407] Step 6:

[1408] The server compares product prices based on the accumulated price information, taking into consideration the user's travelable distance and the acceptable range of price difference.

[1409] Step 7:

[1410] The server selects the best store for each product and calculates the minimum total cost for the user. For example, it checks that the total cost for supermarket A is 360 yen and the total cost for supermarket B is 370 yen.

[1411] Step 8:

[1412] The server obtains the optimal route using a map service (e.g., Google Maps API) to generate a navigation route to the selected optimal store (e.g., Supermarket A).

[1413] Step 9:

[1414] The server creates a shopping list based on the information input by the user and transmits it to the user terminal together with the navigation route.

[1415] Step 10:

[1416] The device receives information from the server and notifies the user of the best store information, navigation route, and shopping list. For example, it might display "Route to Supermarket A" with "Carrots: 100 yen, Potatoes: 80 yen, Onions: 60 yen, Tomato sauce: 120 yen."

[1417] The above steps realize a system that allows users to efficiently shop at the most cost-effective store and also enhances the store's sales promotion effects.

[1418] Example 1

[1419] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1420] Users need to expend a lot of time and effort to purchase the products they need efficiently and economically. Manually comparing advertising media and flyers from multiple stores is extremely tedious. Furthermore, it is difficult to optimally select stores to visit and navigate them along efficient routes. This degrades the user's shopping experience and reduces the effectiveness of store promotional activities. To solve this problem, a system is needed that allows users to easily input information and automatically provides optimal store information and navigation routes.

[1421] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1422] In this invention, the server includes means for inputting a user's preset home address, travel distance, and price difference, means for collecting image data from advertising media of multiple stores and uploading it to the server, means for converting the image data into text data and extracting product and price information, means for selecting the optimal store based on the extracted product and price information based on the user's settings, means for generating a navigation route to the optimal store and notifying the user terminal, means for inputting a shopping list preset by the user, and means for selecting the optimal store based on the shopping list and generating a navigation route. This allows users to shop efficiently and economically and also enhances the effectiveness of store promotions.

[1423] "User" refers to an individual who uses the system to input setting information and flyer images and receives optimal store information and navigation routes.

[1424] "Home address" refers to the specific location information of the place where the user lives.

[1425] "Moveable distance" refers to the maximum distance a user can move from their home.

[1426] "Price difference" refers to the difference in product price that a user can accept.

[1427] "Advertising media" refers to print or digital sources containing product information and special offers provided by a store.

[1428] "Image data" refers to a digital image file that captures the content of advertising media.

[1429] "Server" refers to a computer system that processes collected image data and provides optimal information to users.

[1430] "Text data" refers to character information extracted from image data.

[1431] "Product" refers to a specific item that a user is considering purchasing.

[1432] "Price information" refers to price information corresponding to each product.

[1433] The "optimal store" refers to a store that meets the user's set conditions and offers the most economical products.

[1434] A "navigation route" refers to the path a user takes to reach the optimal store.

[1435] "Notifying" refers to sending information to a user terminal to notify the user.

[1436] A "purchase list" refers to a list of products and quantities that a user plans to purchase.

[1437] "Image recognition algorithm" refers to a computational method or program for extracting text data from image data.

[1438] "Terminal" refers to an electronic device that a user operates and receives information from.

[1439] This technology allows users to input their home address, travel distance, and price difference, collect image data from advertising media of multiple stores, upload it to a server, and provide the store with the best deal. The system aims to improve the user's shopping experience and promote store promotional activities.

[1440] Overall system configuration

[1441] This system includes a terminal (e.g., a smartphone) on which users input information, photograph flyers, and upload them to a server, a server, and various program modules that run on the server. Potential hardware used includes a smartphone, a server, and a high-performance CPU or GPU for image processing. Software used includes an image processing library (e.g., OpenCV), an image recognition model (e.g., TensorFlow, Google Cloud Vision API), and a database management system (e.g., MySQL).

[1442] Specific operating procedures

[1443] Entering User Preferences Information

[1444] The user launches the smartphone app and first enters their home address. For example, the user enters "Minato Ward, Tokyo," sets the travel distance to "2 km," and the price difference to "50 yen." The user also enters the weekly menu as "Monday: curry, Tuesday: spaghetti," and the missing ingredients as "carrots, potatoes, onions, tomato sauce."

[1445] Collect flyer images and upload them to the server

[1446] The user takes a picture of a flyer from a nearby store (e.g., a supermarket or drugstore) with their smartphone camera. The captured image data is sent to the server by pressing the "Upload" button in the app. The program notifies the user that the upload is complete.

[1447] Image data analysis

[1448] The server analyzes the received image data. TensorFlow and Google Cloud Vision API are used as image recognition models. This allows text data such as product names, prices, and sale information to be extracted from the image. For example, information such as "Carrots: 100 yen, Potatoes: 80 yen, Onions: 60 yen, Tomato sauce: 120 yen" can be extracted from a flyer. The extracted data is stored in a database.

[1449] Selecting the best store and generating navigation routes

[1450] The server selects the most suitable store based on the price information in the database, taking into account the user's settings (home address, travel distance, price difference). Once the most suitable store is determined, the server generates a navigation route using a map service (e.g., Google Maps API). At the same time, it also takes into account the shopping list entered by the user and sets a route that allows the user to efficiently purchase the necessary products.

[1451] User Notification and Guidance

[1452] The server sends the generated navigation route and shopping list to the user's smartphone. The user receives a notification from the device and can follow the information displayed on the screen to find the optimal store. The displayed information includes the store name, address, list of products to be purchased, and navigation route.

[1453] Specific examples

[1454] The user opens the app and enters their home address (Minato Ward, Tokyo), the travel distance (2 km), and the price difference (50 yen). They also enter their menu (curry and spaghetti) and the missing ingredients (carrots, potatoes, onions, and tomato sauce). Next, the user takes photos of flyers for nearby supermarkets A and B and uploads them to the app. The server extracts the following data from flyer A: "Carrots: 100 yen, potatoes: 80 yen, onions: 60 yen, tomato sauce: 120 yen"; from flyer B: "Carrots: 90 yen, potatoes: 100 yen, onions: 70 yen, tomato sauce: 110 yen." These data are then compiled into a list and stored in a database. The server compares the extracted price information and determines that supermarket A offers the best total cost (360 yen). The server uses the Google Maps API to generate a navigation route to supermarket A and sends it to the user's device along with the shopping list. The device notifies the user of the received information and displays the shopping list along with navigation to supermarket A. This allows users to shop in the most efficient and economical way possible.

[1455] In this way, the present invention provides a system that enables users to shop efficiently and economically, and also enhances the effectiveness of store promotions.

[1456] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1457] Step 1:

[1458] The user enters the configuration information

[1459] A user launches the app and enters their home address (e.g., "Minato Ward, Tokyo"), the distance they can travel (e.g., "2 km"), the price difference (e.g., "50 yen"), and a shopping list (e.g., "carrots, potatoes, onions, tomato sauce"), which then creates a personalized shopping profile for the user.

[1460] Input: Home address, travel distance, price difference, shopping list

[1461] Output: User profile data

[1462] Specific behavior: The user enters information into the app using form input fields and then clicks the "Save Settings" button to send it to the server.

[1463] Step 2:

[1464] The server stores user information

[1465] The profile data sent by the user is received by the server, which stores it in a database and updates the user profile.

[1466] Input: User profile data

[1467] Output: Updated database user profile

[1468] Specific operation: The server parses the received data and stores it in a structured database (e.g., MySQL).

[1469] Step 3:

[1470] The user takes and uploads a flyer image

[1471] Users take a photo of a flyer from a nearby store using their smartphone camera, then use the app's upload function to upload the flyer image data to the server.

[1472] Input: Flyer image data

[1473] Output: Flyer image uploaded to the server

[1474] Specific operation: The user takes a photo of a flyer with the camera and presses the "Upload image" button to send the image to the server.

[1475] Step 4:

[1476] The server analyzes the flyer image

[1477] The server analyzes the received flyer image data using an image recognition algorithm (e.g., TensorFlow, Google Cloud Vision API). As a result of the analysis, product names, prices, special sale information, etc. are extracted as text data.

[1478] Input: Flyer image data

[1479] Output: Product information text data

[1480] Specific operation: The server loads the image recognition model, inputs a flyer image, and extracts text data including product name and price information.

[1481] Step 5:

[1482] The server stores the product information in a database

[1483] The extracted text data of the product information is stored in a database, and price information for each product is saved.

[1484] Input: Text data of product information

[1485] Output: Product information stored in the database

[1486] Specific operation: The server structures the text data and stores it by adding a new entry to the database.

[1487] Step 6:

[1488] The server selects the best store

[1489] The server selects the most cost-effective store within the specified travel distance and price range based on the user's purchase list and price information.

[1490] Input: Purchase list, user profile data, product information

[1491] Output: Optimal store information

[1492] How it works: The server searches and compares product prices from a database and uses an algorithm to identify the best store.

[1493] Step 7:

[1494] The server generates the navigation route

[1495] A navigation route to the selected optimal store is generated using a map service (e.g., Google Maps API).

[1496] Input: Best store information, home address

[1497] Output: Navigation route

[1498] Specific behavior: The server queries the Google Maps API to obtain a navigation route and processes it.

[1499] Step 8:

[1500] The server notifies the user of the navigation route and shopping list.

[1501] The server sends the generated navigation route and shopping list to the user's smartphone.

[1502] Input: navigation route, purchase list

[1503] Output: User notification

[1504] Specific operation: The server sends information to the user's device, and the app receives and displays the notification.

[1505] (Application example 1)

[1506] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1507] With so many choices available to modern consumers, it can be difficult to determine where and which products are most economical to purchase. Stores also have limited means of efficiently advertising to consumers. This creates a need for a system that supports optimal purchasing choices tailored to individual consumer needs while simultaneously enhancing store promotional effectiveness.

[1508] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1509] In this invention, the server includes means for inputting the user's preset home address, travel distance, and price difference, means for taking images of advertising media for multiple stores with a smartphone and uploading them to the server, means for converting the image data into text data using an image recognition algorithm and extracting product and price information, means for selecting the optimal store from the extracted product and price information based on the user's settings, and means for generating a navigation route to the optimal store using a map service and notifying the user terminal. This allows the user to receive suggestions for optimal stores in real time, enabling efficient and economical purchasing activities.

[1510] "User" refers to a consumer who uses this system.

[1511] "Home address" refers to the specific address information of the home specified by the user.

[1512] The "travelable distance" refers to the distance that indicates the range that the user can travel from their home.

[1513] "Price difference" refers to the price difference that a user is willing to accept.

[1514] "Multiple stores" refers to multiple target commercial facilities.

[1515] "Advertising media" refers to distributed advertisements such as flyers and pamphlets provided by stores.

[1516] "Image data" refers to image files of advertising media taken with a smartphone or other device.

[1517] "Smartphone" refers to a portable information terminal equipped with mobile communication functions.

[1518] "Server" refers to a computer on a network that processes, stores, and distributes image data.

[1519] An "image recognition algorithm" refers to a program for extracting text information and product information from image data.

[1520] "Text data" refers to a character string extracted from image data.

[1521] "Product and price information" refers to information about the product name and price listed in the advertising media.

[1522] The "best store" refers to the store that best meets the user's requirements.

[1523] "Navigation route" refers to a route showing directions from the current location to the destination.

[1524] "Map Service" refers to a service that provides digital maps and supports navigation functions.

[1525] "User terminal" refers to a connected device such as a smartphone or PC used by a user.

[1526] The system for implementing this invention automates a series of steps for users to find the best store. Users input their home address, travel distance, and price difference into the application, and the system begins operation based on this information.

[1527] Hardware and Software Overview

[1528] Server: A computing device for large-scale data processing and analysis. The server contains software for running image recognition algorithms (e.g., pytesseract), a database management system (e.g., PostgreSQL), and a map service API for generating navigation routes (e.g., Google Maps API).

[1529] User device: A smartphone or tablet used by a user. An application is installed on this device, which receives user input, photographs advertising media in nearby stores, and uploads them to a server.

[1530] Process Overview

[1531] User input of settings

[1532] Through the application, users input information such as their home address, travel distance, price difference, etc. They can also input a weekly menu and a list of ingredients they are missing, and the system then creates and saves a user profile.

[1533] Specific input example:

[1534] Home address: "Minato-ku, Tokyo"

[1535] Travel distance: "2km"

[1536] Price difference: "50 yen"

[1537] Menu: "Curry and Spaghetti"

[1538] Missing ingredients list: "Carrots, potatoes, onions, tomato sauce."

[1539] Photograph and upload flyers

[1540] Users use their smartphones to take photos of advertising flyers from nearby stores and upload the image data to a server via the application.

[1541] Image data analysis

[1542] The server receives the uploaded image data and uses an image recognition algorithm (e.g., pytesseract) to extract product names, prices, and special offers, which are then stored in a database as text data.

[1543] Selection of optimal stores

[1544] The server compares product prices based on the product information stored in the database, taking into account the travel distance and price difference set by the user, and selects the optimal store for each product, performing calculations to minimize the user's total cost.

[1545] Navigation route generation and notification

[1546] The server uses a map service API (e.g., Google Maps API) to generate a navigation route to the best store. The generated route is sent to the user's device and notified along with the shopping list.

[1547] Specific prompt examples:

[1548] "I live in Minato Ward, Tokyo. Please find a supermarket within a 2km radius where the price difference is within 50 yen. I would like to buy carrots, potatoes, onions, and tomato sauce."

[1549] In this way, users can find the most economical shopping destination in real time, enabling them to shop efficiently, and stores can also advertise effectively to consumers.

[1550] The above is a specific description of the embodiment of the invention.

[1551] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1552] Step 1: User inputs settings

[1553] Input: The user enters their home address, travel distance, price difference, weekly meal plan, and missing ingredients into the application.

[1554] Data processing: The information entered by the user is sent to the server and saved in a database as a profile.

[1555] Output: The server creates a user profile and outputs it as saved data.

[1556] Step 2: Photograph and upload your flyer

[1557] Input: A user uses their smartphone to take a photo of a flyer for a nearby store.

[1558] Data processing: The captured image data is uploaded to the server via the application.

[1559] Output: The server receives and stores the uploaded image data.

[1560] Step 3: Analyzing the image data

[1561] Input: Image data received by the server.

[1562] Data processing: The server uses an image recognition algorithm (e.g., pytesseract) to convert the image data into text data, specifically extracting product names, prices, and special offers.

[1563] Output: The extracted product name, price, and sale information are saved as text data in a database.

[1564] Step 4: Select the best store

[1565] Input: User profile information (home address, travel distance, price difference) and product information stored in the database.

[1566] Data processing: The server compares product prices based on the user's settings and calculates the best store to minimize the user's total cost.

[1567] Output: Information on the optimal store and calculation results are generated.

[1568] Step 5: Generate and notify navigation route

[1569] Input: User's home address and best store information.

[1570] Data processing: The server uses a map service API (e.g., Google Maps API) to generate a navigation route to the optimal store.

[1571] Output: The generated navigation route and shopping list are sent to the user's device.

[1572] Specific behavior:

[1573] In step 1, the user enters the required information into the application, which is then saved as a profile on the server.

[1574] In step 2, the user takes a photo of the flyer with their smartphone and uploads it to the server via the application.

[1575] In step 3, the server uses an image recognition algorithm to convert the received image data into text data, specifically extracting product names, prices, and sale information, and storing it in a database.

[1576] In step 4, the server performs calculations to select the most suitable store based on the user's profile information and the product information stored in the database.

[1577] In step 5, the server uses the map service API to generate a navigation route and sends it to the user's terminal together with the shopping list.

[1578] This series of processes enables users to make the most efficient and economical purchases in real time.

[1579] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1580] The present invention is a system that streamlines the user's shopping experience by allowing the user to input their home address, travel distance, and price difference in advance. Furthermore, by combining this system with an emotion engine that recognizes the user's emotions, it is possible to select the most appropriate store according to the user's emotional state.

[1581] Overview of the overall system operation

[1582] Setup Phase

[1583] 1. Users

[1584] Enter your home address into the app, and set the allowable travel distance (e.g., 2km) and price difference (e.g., 50 yen).

[1585] You can also enter your weekly menu and a list of ingredients you're missing, such as "Monday: curry, Tuesday: spaghetti" or "carrots, potatoes, onions, tomato sauce."

[1586] As you use the app to consider ingredients or your shopping list, the emotion engine recognizes your emotions in real time.

[1587] 2. Server

[1588] It receives user input information and saves it as a profile. It also collects location information and the latest flyer data for nearby supermarkets and drugstores and stores them in a database.

[1589] The emotion engine analyzes and stores emotional data entered by the user in real time.

[1590] Image Recognition Phase

[1591] 3. Users

[1592] Take a photo of a flyer from a local supermarket or drugstore with your smartphone.

[1593] The image data of the photographed flyer is uploaded to the server via the app.

[1594] 4. Server

[1595] The received flyer image data is passed to an image recognition module, which converts the image data into text data. For example, the product name, price, and special sale information are extracted from the flyer.

[1596] The extracted text data is used to create a list of product types and their price information, which is then stored in a database.

[1597] Optimal store proposal phase

[1598] 5. Server

[1599] Product prices are compared based on accumulated price information, the travel distance set by the user, and the acceptable range of price differences.

[1600] The emotion engine assesses the user's emotional state and applies an algorithm that prioritizes the selection of the nearest store if, for example, stress levels are high.

[1601] 6. Server

[1602] The optimal store is selected for each product, and calculations are performed to minimize the total cost for the user. For example, it is confirmed that the total cost for supermarket A is 360 yen and the total cost for supermarket B is 370 yen.

[1603] The emotion engine also takes into account past emotional data and makes suggestions optimized for the user's shopping style.

[1604] 7. Server

[1605] To generate a navigation route to the selected optimal store (e.g., Supermarket A), a map service (e.g., Google Maps API) is used to obtain the optimal route.

[1606] A shopping list is created based on the information input by the user and is sent to the user terminal together with the navigation route.

[1607] 8. Terminal

[1608] It receives information from the server and notifies the user of the best store information, navigation route, and shopping list. For example, it displays "Route to Supermarket A" and "Carrots: 100 yen, Potatoes: 80 yen, Onions: 60 yen, Tomato sauce: 120 yen."

[1609] In addition, the emotion engine will suggest less crowded times, for example, to help users shop in a less stressful environment.

[1610] Specific examples

[1611] Setup and Initial Preparation

[1612] The user opens the app and enters their home address (Minato-ku, Tokyo), the travel distance (2km), and the price difference (50 yen). They then enter the menu (curry and spaghetti) and the missing ingredients (carrots, potatoes, onions, and tomato sauce). The emotion engine reads the user's emotions while they are entering and considering their choices, and accumulates the data.

[1613] The server receives the information, collects the latest flyer data from nearby supermarkets and drug stores, and stores it in a database. The emotion engine also stores the user's emotion data and uses it for future optimization.

[1614] Image Recognition and Data Analysis

[1615] The user takes a photo of flyers from nearby supermarkets A and B and uploads them to the app.

[1616] The server extracts the data "Carrots: 100 yen, Potatoes: 80 yen, Onions: 60 yen, Tomato sauce: 120 yen" from flyer A and "Carrots: 90 yen, Potatoes: 100 yen, Onions: 70 yen, Tomato sauce: 110 yen" from flyer B, lists them, and saves them in a database.

[1617] Selection and notification of optimal stores

[1618] The server compares the extracted price information and determines that Supermarket A has a lower total cost (360 yen). Based on this, Supermarket A is selected as the optimal store. The server evaluates the user's emotional state, and if, for example, stress is high, prioritizes the nearest store.

[1619] The server uses the Google Maps API to generate a navigation route to Supermarket A and sends it to the user's device along with the shopping list. Using the emotion engine, it also includes suggestions for reducing stress (e.g., less crowded times).

[1620] The terminal receives information from the server and notifies the user of the navigation and purchase list for Supermarket A. For example, it displays "Route to Supermarket A" and "Carrots: 100 yen, Potatoes: 80 yen, Onions: 60 yen, Tomato sauce: 120 yen."

[1621] In this way, the present invention provides a system that allows for the most efficient and economical shopping experience, while also taking into account the user's emotional state.

[1622] The processing flow will be explained below.

[1623] Step 1:

[1624] The user launches the app and enters their home address (Minato Ward, Tokyo), the distance they can travel (2 km), and the price difference (50 yen). They also enter their weekly menu and a list of ingredients they are missing. For example, "Monday's dinner is curry, Tuesday's is spaghetti," and "Ingredients needed: carrots, potatoes, onions, tomato sauce."

[1625] Step 2:

[1626] While the user is using the app, the emotion engine analyzes the user's facial expressions, voice, etc. to obtain emotional data in real time. The emotional state (e.g., stress level, elation, etc.) is evaluated and stored in a database.

[1627] Step 3:

[1628] The server receives the user's settings and emotional data and stores them as a profile. It also collects location information and the latest flyer data for nearby supermarkets and drugstores and stores them in a database.

[1629] Step 4:

[1630] The user takes a photo of flyers from nearby supermarkets A and B with their smartphone, and uploads the flyer image data to the server via the app.

[1631] Step 5:

[1632] The server passes the received flyer image data to an image recognition module, which converts the image data into text data (e.g., extracting product names, prices, and sale information from the flyer). The extracted text data is analyzed, and product types and their prices are listed and stored in a database.

[1633] Step 6:

[1634] The server compares product prices based on the accumulated price information, the user's travel distance, price tolerance, and emotional data. If the user's stress level is high, an algorithm is applied that prioritizes the selection of the nearest store.

[1635] Step 7:

[1636] The server selects the optimal store for each product and calculates the lowest total cost (e.g., the total cost for supermarket A is 360 yen, and the total cost for supermarket B is 370 yen). The optimal store is selected based on the user's emotional data (e.g., supermarket B may be selected because it has low stress levels).

[1637] Step 8:

[1638] The server considers the user's emotional data and generates a navigation route to the optimal selected store (e.g., obtains the optimal route using Google Maps API). It also makes suggestions to reduce stress (e.g., less crowded times).

[1639] Step 9:

[1640] The server creates a shopping list based on the information input by the user and transmits it to the user terminal together with the navigation route.

[1641] Step 10:

[1642] The terminal notifies the user of the information received from the server (for example, displaying the name, location, navigation route, shopping list, etc. of the best store). Specifically, for example, it displays "Route to Supermarket A" and "Carrots: 100 yen, Potatoes: 80 yen, Onions: 60 yen, Tomato sauce: 120 yen."

[1643] Through the above steps, the user can shop efficiently at the most suitable store that takes into consideration the user's emotional state, achieving a less stressful shopping experience.

[1644] Example 2

[1645] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1646] The present invention aims to streamline the user's shopping experience and provide an optimal shopping environment that takes into account the user's emotional state. Specifically, the present invention aims to provide a system that selects the optimal store based on the user's specified address, travel distance, and price difference, and also evaluates the user's emotional state to support a stress-free shopping experience.

[1647] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for inputting the user's home address, travel distance, and price difference, which are set in advance by the user; a means for collecting image data from advertising media of multiple business facilities and uploading it to the server; a means for converting the image data into text data and extracting product and price information therefor; a means for selecting the optimal business facility from the extracted product and price information based on the user's settings; a means for evaluating the user's emotional state and optimizing store selection based on the evaluation results; and a means for generating a navigation route to the optimal business facility and notifying the user terminal. This allows the user to have an efficient and emotionally-conscious shopping experience.

[1648] "Address" refers to specific geographical information about the place where the user lives, and serves as a criterion for selecting a store within the user's permitted range of movement.

[1649] The "travelable distance" refers to the distance of the travelable range that the user is willing to travel, and serves as a criterion for selecting the most suitable store within this range.

[1650] "Price difference" refers to the difference in product prices that a user is willing to accept, and serves as a criterion for comparing prices within this range to select a store.

[1651] "Business facilities" refer to facilities such as stores, supermarkets, and drugstores that sell products, and are places that provide products for users to purchase.

[1652] "Advertising media" refers to advertising information provided by each business establishment, and primarily includes printed media such as flyers and posters.

[1653] "Image data" refers to visual information of advertising media photographed and uploaded by a user and stored in digital format.

[1654] "Text data" refers to character information extracted by analyzing image data, and includes product names, price information, and the like.

[1655] A "product" refers to an item that a user considers purchasing, and is accompanied by a specific name and price information.

[1656] "Price information" refers to the specific price associated with a product, and serves as a basis for users to select the most suitable place to purchase.

[1657] "Emotional state" refers to the user's psychological state and emotions, and by evaluating this, the system is able to select and suggest the most suitable store for the user.

[1658] A "navigation route" refers to a specific route for a user to travel to a selected business facility, and is generated using a map service.

[1659] "User terminal" refers to an electronic device such as a computer or smartphone that a user uses to input information and receive notifications from the system.

[1660] The present invention is a system that streamlines the user's shopping experience by allowing the user to input their home address, travel distance, and price difference in advance. In addition, it has the function of recognizing the user's emotional state in real time and selecting the most suitable store based on that state.

[1661] Hardware and Software Configuration

[1662] User side

[1663] User device: A device, such as a smartphone or computer, that a user uses to enter information and receive notifications from the server.

[1664] Camera: The camera attached to the user terminal is used to capture image data of the advertising medium.

[1665] Server side

[1666] Server: The central system that receives the information sent by the user and performs analysis and data processing. This includes the database management system, image recognition module, and emotion engine.

[1667] Database management systems: Used to store and manage user profile information and advertising data. Examples include MySQL and PostgreSQL.

[1668] Image recognition module: A software module for converting image data from advertising media into text data. Specific software examples include OpenCV and Tesseract OCR.

[1669] Emotion engine: A software module for analyzing and storing a user's emotional state in real time. It uses natural language processing and machine learning algorithms to evaluate the user's emotions.

[1670] System Operation Overview

[1671] User Settings

[1672] The user launches the app and enters their home address, travel distance, and price difference. They also enter a week's worth of meals and a list of ingredients they are missing. For example, they set their address to "Minato Ward, Tokyo," their travel distance to "2 km," and their price difference to "50 yen," and then enter "curry and spaghetti" as their meal plan and "carrots, potatoes, onions, and tomato sauce" as the ingredients they are missing. This information is saved in the app.

[1673] Data collection and analysis

[1674] Users take a photo of a flyer from a nearby supermarket or drugstore with their smartphone camera and upload it to the server via the app. The server passes the received flyer image to an image recognition module, which converts data such as product names and prices into text. For example, information such as "Carrots: 100 yen, Potatoes: 80 yen, Onions: 60 yen, Tomato sauce: 120 yen" is extracted and stored in a database.

[1675] Selection and proposal of optimal stores

[1676] The server selects the optimal store that meets the user's criteria based on price information stored in a database. It creates a shopping list based on a weekly menu and a list of ingredients that are missing, and uses an emotion engine to evaluate the user's emotional state. For example, if the user is feeling stressed, it will prioritize the nearest store. The server selects the optimal store based on this data and generates a navigation route using Google Maps API, etc. The generated route and shopping list are sent to the user's device.

[1677] Notifications and Support

[1678] Based on the information received from the server, the user device notifies the user of the optimal store information, navigation route, and shopping list. For example, it displays "Optimal store: Supermarket A, navigation route, 'Carrots: 100 yen, Potatoes: 80 yen, Onions: 60 yen, Tomato sauce: 120 yen'." The emotion engine supports shopping in a less stressful environment for the user by making suggestions that take into account, for example, less crowded times.

[1679] Specific examples

[1680] The user opens the app, sets their home address to "Minato-ku, Tokyo," the travel distance to "2 km," the price difference to "50 yen," and inputs the menu item "curry and spaghetti" and the list of missing ingredients: "carrots, potatoes, onions, and tomato sauce."

[1681] The server receives the information and collects the latest flyer data from nearby supermarkets and drugstores, and the emotion engine also stores the user's emotion data.

[1682] When a user takes a photo of flyers from nearby supermarkets A and B and uploads them to the app, the server extracts price data from flyer A, compares them, and evaluates the total cost. It uses an emotion engine to take into account the user's stress level and makes appropriate suggestions.

[1683] The server sends the generated shopping list and navigation route to the user's terminal, helping the user to shop efficiently at the most suitable store.

[1684] Prompt Sentence Examples

[1685] Example prompt 1: "My home address is in Minato Ward, Tokyo. Please tell me the best store to buy ingredients for curry and spaghetti within a 2km travel distance and within a price difference of 50 yen. Also, please help me have a less stressful shopping experience by taking my emotional state into consideration."

[1686] Example prompt 2: "Today's list of things I need is carrots, potatoes, onions, and tomato sauce. I've uploaded images of flyers from nearby supermarkets. Please suggest the store with the lowest cost and least stress."

[1687] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1688] Step 1:

[1689] The user launches the app and enters their home address, travel distance, and price difference. For example, the address can be set to "Minato-ku, Tokyo," the travel distance to "2 km," and the price difference to "50 yen." The input data also includes a week's worth of meals (e.g., "curry and spaghetti") and a list of missing ingredients (e.g., "carrots, potatoes, onions, tomato sauce"). This information is sent to the server from the device operated by the user. Input: Address, travel distance, price difference, meals, list of ingredients. Output: User profile information stored on the server.

[1690] Step 2:

[1691] The server receives the information sent by the user and stores it in a database management system (e.g., MySQL, PostgreSQL). At the same time, the emotion engine analyzes and accumulates the user's emotion data in real time. It also collects location information of nearby business facilities and the latest advertising data and stores them in the database. Input: User profile information. Output: Location information, advertising data, and emotion data stored in the database.

[1692] Step 3:

[1693] The user takes a photo of advertising media in nearby business establishments with their smartphone camera and uploads it to the server via the app. For example, they take photos of flyers from supermarkets A and B. This image data is sent directly to the server. Input: Image of advertising media. Output: Image data uploaded to the server.

[1694] Step 4:

[1695] The image data of the advertising media received by the server is passed to an image recognition module (e.g., OpenCV, Tesseract OCR) and converted into text data. Specifically, the image recognition module extracts the product name, price, and sale information, and stores them in a database. For example, it extracts data such as "carrots: 100 yen, potatoes: 80 yen, onions: 60 yen, tomato sauce: 120 yen." Input: Image data of the advertising media. Output: Text data stored in the database.

[1696] Step 5:

[1697] The server applies an algorithm to select the optimal store based on the product price information stored in the database, within the address, travel distance, and price difference range set by the user. Furthermore, an emotion engine is used to evaluate the user's emotional state, and if, for example, stress levels are high, the system prioritizes closer stores. Input: Text data, user profile information, emotion data. Output: Optimal store selection results.

[1698] Step 6:

[1699] The server calculates the total cost and selects the optimal business facility. For example, if the total cost of supermarket A is 360 yen and the total cost of supermarket B is 370 yen, supermarket A is selected. Based on this, supermarket A is determined to be the optimal store. Input: store price data, user setting information. Output: optimal store and total cost information.

[1700] Step 7:

[1701] The server uses a map service (e.g., Google Maps API) to generate a navigation route to the optimal store. It also sends the generated navigation route and shopping list to the user's device. Input: optimal store information, user location information. Output: navigation route, shopping list.

[1702] Step 8:

[1703] The terminal receives the information sent from the server and notifies the user. Specifically, it displays the optimal store information, navigation route, and shopping list for the user. For example, it notifies the user of information such as "Optimal store: Supermarket A, navigation route, 'Carrots: 100 yen, Potatoes: 80 yen, Onions: 60 yen, Tomato sauce: 120 yen'." It also makes suggestions such as less crowded times. Input: Store information from the server, navigation route, shopping list. Output: Notification and suggestions to the user.

[1704] (Application example 2)

[1705] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1706] The present invention relates to a system that allows users to shop efficiently. There is a particular need for a system that takes into account the user's emotional state and provides optimal store selection while reducing stress. Conventional systems select stores based solely on price and distance, ignoring the user's emotional state and potentially reducing the quality of the shopping experience. Furthermore, there is also the issue that users take a long time to find the optimal store because price information from advertising materials is not adequately extracted and optimized.

[1707] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for inputting the user's home address, travel distance, and price difference, which are set in advance by the user; means for collecting image data from advertising materials for multiple sales points and uploading it to the server; means for converting the image data into text data and extracting product and price information; means for selecting the optimal sales point from the extracted product and price information based on the user's settings; means for analyzing the user's emotional state in real time and optimizing store selection in accordance with the emotional state; and means for generating a route to the optimal sales point and notifying the user terminal. This allows for efficient and optimal shopping while reducing stress, taking the user's emotional state into consideration.

[1708] The "home address preset by the user" is information that the user inputs into the system in advance to indicate the place of residence of the user.

[1709] The "travelable distance" is information indicating the maximum distance that the user is allowed to travel for shopping.

[1710] "Price difference" is information indicating the allowable price difference for the same product.

[1711] "Sales point advertising materials" are materials that show flyers and campaign information issued by stores such as supermarkets and drugstores.

[1712] "Image data" refers to image data obtained from advertising materials at points of sale.

[1713] The "means for uploading to the server" is a method or function for transferring image data collected from the user terminal to the server.

[1714] "Text data" is character string information extracted and converted from image data.

[1715] "Product and price information" is information indicating the name of the product and its selling price listed in the advertising material.

[1716] The "means for selecting the most suitable sales point" is a method or function for automatically selecting the most suitable store for the user based on the user's setting information and price information.

[1717] The "means for generating a travel route" is a method or function for generating a route that enables a user to travel to the optimal sales point in the shortest time or with the least stress.

[1718] The "means for analyzing the user's emotional state in real time" is a method or function for recognizing the user's emotions in real time and analyzing the data.

[1719] The "means for optimizing store selection according to emotional state" is a method or function for selecting an optimal store by taking into account the emotional state of the user.

[1720] The "notification means" is a method or function for notifying the user terminal of information regarding the optimum sales point and its travel route.

[1721] The present invention is a system that streamlines the user's shopping experience and selects the optimal store based on the user's emotional state. The system works by combining and processing user input information, image data from point-of-sale advertising materials, and the user's emotional state.

[1722] The server implements the present invention using the following means.

[1723] 1. Enter your user settings information

[1724] Users input their home address, the distance they can travel, and the price difference, which allows the system to determine the range within which they can shop.

[1725] 2. Collecting and uploading advertising materials

[1726] Users take photos of advertising materials from multiple points of sale using a device such as a smartphone and upload them to a server, at which point image data is collected.

[1727] 3. Conversion to text data

[1728] The server uses an image recognition algorithm (e.g., Tesseract) to convert the uploaded image data into text data, from which product names and their prices are extracted.

[1729] 4. Selection of optimal sales locations

[1730] The server selects the optimal sales location based on the extracted price information and the user's settings (address, distance, price difference), and then analyzes the user's emotional state in real time (e.g., using TensorFlow) to optimize store selection according to the user's emotional state.

[1731] 5. Generate navigation route

[1732] The server generates the optimal travel route for the user to the selected optimal sales point. Specifically, it creates the route using a map service (e.g., Google Maps API).

[1733] 6. Notification to user devices

[1734] This information is then sent to the user's device, which includes the best shopping locations, product pricing information, and navigation routes.

[1735] This system allows users to have an efficient and optimal shopping experience, and also reduces stress by selecting stores according to their emotional state.

[1736] Specific examples of hardware and software used

[1737] Hardware: smartphones, servers, cameras

[1738] Software: TensorFlow (sentiment analysis), Tesseract (image recognition), Google Maps API (navigation)

[1739] Examples of prompt statements

[1740] Here is an example of inputting the following prompt into a generative AI model:

[1741] "The user enters their address, travel distance, price difference, meal plan information, and a list of missing ingredients into the app. Furthermore, if the user's emotional state is 'stressed,' please suggest the most efficient and economical store where they can shop, and why."

[1742] Examples of expected outputs include the following:

[1743] "Supermarket A is the best choice. It was selected for the following reasons: 1) Supermarket A is closest to the specified address. 2) Your current emotional state is stressed, so the travel distance is short and stress reduction is expected. 3) The total cost of the ingredients needed for the meal is the lowest. For these reasons, I recommend Supermarket A."

[1744] In this way, the present invention provides a system that allows for the most efficient and economical shopping experience, while also taking into account the user's emotional state.

[1745] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1746] Step 1:

[1747] The user enters their home address, travel distance, and price difference into the app.

[1748] Input: User's home address, travel distance, price difference

[1749] Output: User configuration information

[1750] How it works: A user launches the app and enters their home address, the distance they can travel (e.g., 2 km), and the price difference (e.g., 50 yen) into the interface. This information is then sent to the server.

[1751] Step 2:

[1752] A user takes pictures of advertising materials from multiple points of sale and uploads them to a server.

[1753] Input: Advertising image of the point of sale

[1754] Output: Image data stored on the server

[1755] How it works: Users use their smartphone camera to take a photo of a flyer from a nearby supermarket or drugstore, then upload the image data to a server via the app.

[1756] Step 3:

[1757] The server converts the received image data into text data and extracts product and price information.

[1758] Input: Image data

[1759] Output: Text data of products and their prices

[1760] How it works: The server uses an image recognition algorithm (e.g., Tesseract) to convert the uploaded image data into text data, then extracts product names and their prices from the text data and stores them in a database.

[1761] Step 4:

[1762] The server selects the optimal sales location based on the user's settings and the extracted price information.

[1763] Input: User settings, product and price information

[1764] Output: Optimal selling point

[1765] Operation: The server analyzes the price information in the database, taking into account the address, travel distance, and price difference set by the user, and selects the optimal sales location.

[1766] Step 5:

[1767] The server analyzes the user's emotional state in real time and optimizes store selection according to that emotional state.

[1768] Input: User's image data or voice data (emotional state)

[1769] Output: Optimal selling points based on emotional state

[1770] How it works: The server uses an emotion engine (e.g., TensorFlow) to analyze the user's emotional state in real time. If the emotional state is stressed, it prioritizes stores with shorter travel distances.

[1771] Step 6:

[1772] The server generates an optimal route to the sales point and notifies the user terminal.

[1773] Input: Best point of sale, user's current location

[1774] Output: Travel route

[1775] Operation: The server uses a map service (e.g., Google Maps API) to generate a route from the user's current location to the optimal sales point, and notifies the user's device of this route information.

[1776] Step 7:

[1777] The user checks the notified content on the terminal and carries out the purchase.

[1778] Input: Notification information from the server (optimal sales location, price information for each product, travel route)

[1779] Output: Actual shopping behavior

[1780] How it works: The user checks the notification on their device and efficiently shops based on the optimal sales point and travel route. For example, it displays "Route to Supermarket A" and shows "Carrots: 100 yen, Potatoes: 80 yen, Onions: 60 yen, Tomato sauce: 120 yen."

[1781] These steps allow users to shop efficiently and economically, and by taking into consideration their emotional state, they are able to reduce stress.

[1782] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1783] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1784] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1785] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1786] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1787] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1788] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1789] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1790] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1791] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1792] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1793] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1794] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1795] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1796] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1797] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1798] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1799] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1800] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1801] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1802] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1803] The following is further disclosed regarding the above embodiment.

[1804] (Claim 1)

[1805] A means for inputting a home address, travel distance, and price difference preset by the user;

[1806] A means for collecting image data from advertising media of a plurality of stores and uploading the collected image data to a server;

[1807] A server converts the image data into text data and extracts product and price information;

[1808] a means for selecting an optimal store based on the extracted product and price information set by a user;

[1809] A means for generating an optimal navigation route to a store and notifying the user terminal of the route;

[1810] A system including:

[1811] (Claim 2)

[1812] 2. The system according to claim 1, further comprising means for a user to input a weekly menu and ingredients that are in short supply, and a purchasing list is created and notified based on the input.

[1813] (Claim 3)

[1814] 10. The system of claim 1, further comprising means for using an image recognition algorithm to analyze image data of the advertising medium.

[1815] "Example 1"

[1816] (Claim 1)

[1817] A means for inputting a home address, travel distance, and price difference preset by the user;

[1818] A means for collecting image data from advertising media of a plurality of stores and uploading the collected image data to a server;

[1819] A server converts the image data into text data and extracts product and price information;

[1820] a means for selecting an optimal store based on the extracted product and price information set by a user;

[1821] A means for generating an optimal navigation route to a store and notifying the user terminal of the route;

[1822] A means for inputting a purchase list preset by a user;

[1823] a means for selecting an optimal store based on the purchase list and generating a navigation route;

[1824] A system including:

[1825] (Claim 2)

[1826] 2. The system according to claim 1, further comprising means for a user to input a weekly menu and ingredients that are in short supply, and a purchasing list is created and notified based on the input.

[1827] (Claim 3)

[1828] 10. The system of claim 1, further comprising means for using an image recognition algorithm to analyze image data of the advertising medium.

[1829] "Application Example 1"

[1830] (Claim 1)

[1831] A means for inputting a home address, travel distance, and price difference preset by the user;

[1832] A method for taking pictures of image data from advertising media in multiple stores with a smartphone and uploading them to a server;

[1833] A server converts the image data into text data using an image recognition algorithm, and extracts product and price information;

[1834] a means for selecting an optimal store based on the extracted product and price information set by a user;

[1835] A means for generating an optimal navigation route to a store using a map service and notifying the user terminal of the route;

[1836] A system including:

[1837] (Claim 2)

[1838] 2. The system according to claim 1, further comprising means for a user to input a weekly menu and ingredients that are in short supply, and a purchasing list is created and notified based on the input.

[1839] (Claim 3)

[1840] 10. The system of claim 1, further comprising means for using an image recognition algorithm to analyze image data of the advertising medium.

[1841] "Example 2: Combining Emotion Engines"

[1842] (Claim 1)

[1843] A means for inputting a home address, travel distance, and price difference preset by the user;

[1844] A means for collecting image data from advertising media of a plurality of business facilities and uploading the image data to a server;

[1845] A server converts the image data into text data and extracts product and price information;

[1846] a means for selecting an optimum business facility based on the extracted product and price information set by a user;

[1847] means for evaluating the emotional state of the user and optimizing store selection based on the evaluation result;

[1848] A means for generating a navigation route to an optimal business facility and notifying the user terminal of the route;

[1849] A system including:

[1850] (Claim 2)

[1851] 2. The system according to claim 1, further comprising means for a user to input a weekly menu and missing items, and creates and notifies a purchase list based on the input.

[1852] (Claim 3)

[1853] 10. The system of claim 1, further comprising means for using an image recognition algorithm to analyze image data of the advertising medium.

[1854] "Application example 2 when combining emotion engines"

[1855] (Claim 1)

[1856] A means for inputting a home address, travel distance, and price difference preset by the user;

[1857] means for collecting image data from advertising materials at a plurality of points of sale and uploading the image data to a server;

[1858] A server converts the image data into text data and extracts product and price information;

[1859] means for selecting an optimal sales point based on the extracted product and price information set by a user;

[1860] A means for generating an optimal route to a sales point and notifying the user terminal of the route;

[1861] A means for analyzing the emotional state of a user in real time and optimizing store selection according to the emotional state;

[1862] A system including:

[1863] (Claim 2)

[1864] 2. The system according to claim 1, further comprising means for a user to input a weekly meal plan and ingredients that are missing, and creates and notifies a shopping list based on the input.

[1865] (Claim 3)

[1866] 10. The system of claim 1, further comprising means for using an image recognition algorithm to analyze image data of said advertising material. [Explanation of symbols]

[1867] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for inputting a home address, travel distance, and price difference preset by the user; A means for collecting image data from advertising media of a plurality of stores and uploading the collected image data to a server; A server converts the image data into text data and extracts product and price information; a means for selecting an optimal store based on the extracted product and price information set by a user; A means for generating an optimal navigation route to a store and notifying the user terminal of the route; A system including:

2. 2. The system according to claim 1, further comprising means for a user to input a weekly menu and ingredients that are in short supply, and a purchasing list is created and notified based on the input.

3. 10. The system of claim 1, further comprising means for using an image recognition algorithm to analyze image data of said advertising media.

Images