Menu suggestion system, information processing device, information processing method, and program

Abstract

This invention provides a menu suggestion system, information processing device, information processing method, and program that can recommend the purchase of the necessary ingredients in the order suggested by advertisements, ensuring neither too much nor too little. [Solution] The information processing device comprises an acquisition unit that acquires images of advertising media, a generation unit that uses the acquired images to acquire product information relating to products listed on the advertising media and generates a menu using the product information, and an output control unit that causes an output means to output output information including information indicating the menu and information relating to products to be purchased in connection with the menu.

Description

Technical Field

[0001] The present invention relates to a menu proposal system, an information processing apparatus, an information processing method, and a program.

Background Art

[0002] Patent Document 1 describes an apparatus that reduces the labor of a user who searches for a recipe for cooking. This apparatus extracts ingredient information regarding a specific ingredient listed in a flyer based on image data of the flyer in which information regarding ingredients is listed, and outputs recipe information regarding a recipe for a specific dish using the specific ingredient based on the extracted ingredient information.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the technique described in Patent Document 1 above, although it is possible to propose a recipe using the ingredients listed in the flyer, in many cases, only those ingredients are not sufficient for cooking according to the recipe.

Means for Solving the Problems

[0005] An information processing apparatus according to one aspect of the present disclosure includes an acquisition unit that acquires an image of an advertising medium, a generation unit that uses the acquired image to acquire product information regarding products listed in the advertising medium, and generates a menu using the product information, an output control unit that causes an output unit to output output information including information indicating the menu and information regarding products to be purchased in relation to the menu, and includes.

[0006] One aspect of this disclosure is the menu suggestion system, Communication devices used by the user, The communication device and the server device connected via a network are included, The server device is The communication device includes an acquisition means for acquiring images of advertising media, A generation means for generating a menu based on product information about products published on the advertising medium extracted from the acquired image, The system includes an output control means that causes the output means to output output information, including information about products to be purchased, in addition to the generated menu.

[0007] The information processing method in one aspect of this disclosure is: One or more computers, Obtain images from advertising media, Using the acquired image, product information relating to the product displayed on the advertising medium is obtained, and a menu is generated using that product information. This includes causing the output means to output output information that includes information indicating the menu and information regarding products to be purchased in relation to the menu.

[0008] One aspect of this disclosure is that the program is On the computer, The process of acquiring images from advertising media. Using the acquired image, product information relating to the product displayed on the advertising medium is obtained, and a generation process is performed to generate a menu using the product information. This is a program for causing an output control process to output output information to an output means, which includes information indicating the menu and information regarding products that should be purchased in relation to the menu. [Effects of the Invention]

[0009] According to one example of this disclosure, a menu suggestion system, information processing device, information processing method, and program are provided that can recommend purchasing the necessary ingredients for a menu suggested in an advertisement without any excess or deficiency.

Brief Description of the Drawings

[0010] [Figure 1] It is a functional block diagram showing a configuration example of an information processing apparatus according to the present disclosure. [Figure 2] It is a flowchart showing an example of a processing operation of an information processing apparatus according to the present disclosure. [Figure 3] It is a diagram conceptually showing a system configuration example of a menu proposal system according to the present disclosure. [Figure 4] It is a diagram showing an example of a configuration of a computer that realizes an information processing apparatus according to the present disclosure. [Figure 5] It is a diagram showing an example of a data structure of user information. [Figure 6] It is a diagram showing an example of a data structure of user setting information. [Figure 7] It is a diagram showing an example of a data structure of received information. [Figure 8] It is a diagram showing an example of a data structure of a product master. [Figure 9] It is a diagram showing an example of a data structure of advertisement information. [Figure 10] It is a diagram showing an example of a data structure of menu information. [Figure 11] It is a diagram showing an example of output information. [Figure 12] It is a diagram showing an example of a data structure of information on products scheduled for purchase. [Figure 13] It is a flowchart showing an example of a processing operation of a menu proposal system according to the present disclosure. [Figure 14] It is a diagram for explaining a specific area.

Embodiments for Carrying Out the Invention

[0011] Hereinafter, in the present disclosure, the drawings are associated with one or more embodiments. Also, in all the drawings, the same reference numerals are given to the same components, and the description thereof is omitted as appropriate. Further, in each of the following drawings, the configuration of parts not related to the essence of the present disclosure is omitted and not shown.

[0012] In this disclosure, "acquisition" includes at least one of the following: the device retrieving data or information stored in another device or storage medium (active acquisition), and the device inputting data or information output from another device into its own device (passive acquisition). Examples of active acquisition include making a request or inquiry to another device and receiving a response, and accessing and reading data from another device or storage medium. Examples of passive acquisition include receiving information that is delivered (or transmitted, push notification, etc.). Furthermore, "acquisition" may also mean selecting and acquiring data or information from received data or information, or selecting and receiving data or information that has been delivered.

[0013] <Example of Functional Configuration> As shown in Figure 1, the information processing device 100 includes an acquisition unit 102, a generation unit 104, and an output control unit 106. The acquisition unit 102 acquires images from the advertising medium. The generation unit 104 uses the acquired images to obtain product information about the products listed on the advertising medium, and uses that product information to generate a menu. The output control unit 106 causes the output means to output output information including information indicating the menu and information regarding products to be purchased in relation to the menu.

[0014] The output means is, for example, the display of a communication device used by the user.

[0015] <Example of operation> In the information processing device 100, as shown in the flowchart in Figure 2, first, the acquisition unit 102 acquires an image of the advertising medium (step S101). The generation unit 104 uses the acquired image to acquire product information about the products listed on the advertising medium and uses this product information to generate a menu (step S103). Then, the output control unit 106 causes the output means to output output information including information indicating the menu and information about products that should be purchased in relation to the menu (step S105).

[0016] According to this information processing device 100, it is possible to provide users with information about products they should purchase in relation to the menu suggested from the images in the advertising medium (information that leads to product purchases at stores), so it is possible to recommend purchasing the necessary ingredients for the menu suggested from the advertisement without any excess or deficiency.

[0017] (First Embodiment) <System Overview> The following describes a detailed example of menu suggestion system 1. As shown in Figure 3, the menu suggestion system 1 comprises a server device 10, a store server 200, and a communication device 20. The server device 10, the store server 200, and the communication device 20 are connected to each other via a communication network 3.

[0018] The server device 10 includes a storage device 120. The storage device 120 may be located inside the server device 10 or outside of it. In other words, the storage device 120 may be hardware integrated with the server device 10 or hardware separate from the server device 10.

[0019] The store server 200 includes a storage device 220. The storage device 220 may be located inside the store server 200 or outside of it. In other words, the storage device 220 may be hardware integrated with the store server 200 or hardware separate from the store server 200.

[0020] Server device 10 and store server 200 are computers such as personal computers and server computers.

[0021] The store server 200 is the server for stores that sell products listed on the advertising medium 30. The storage device 220 of the store server 200 contains a database including a product master. The server device 10 implements the information processing device 100 shown in Figure 1 in cooperation with the communication device 20. The server device 10 can also obtain information on products not listed on the advertising medium 30 by querying the store server 200 via the communication network 3.

[0022] User U is a customer who shops at a physical store. However, User U may also be a customer who shops online at a virtual store on a website. The store could be a retail store such as a supermarket or convenience store.

[0023] The communication device 20 is, for example, a terminal used by user U, and is a computer such as a smartphone, tablet, PDA (Personal Digital Assistant), or personal computer. The communication device 20 has a camera 22.

[0024] The menu suggestion system 1 provides a service to user U, who is the user of the communication device 20, that suggests menus along with products that should be purchased at the store in relation to those menus.

[0025] Advertising medium 30 contains information about bargain products. If you can buy bargain products and plan your meals accordingly, you can reduce your food expenses. However, it is difficult and troublesome to plan meals efficiently using the products you have bought and the ingredients you already have at home.

[0026] Therefore, the menu suggestion system 1 acquires product information about the products listed on the advertising medium 30 and generates a menu using this product information. The menu suggestion system 1 then provides the user U with a service that presents the menu and the ingredients necessary for the menu (i.e., the products that should be purchased at the store).

[0027] This allows user U to not only learn about menus using discounted products featured in advertising medium 30, but also to learn about the ingredients needed for those menus (products to purchase at the store). As a result, user U can purchase the necessary ingredients for the menu without any surplus or shortage. Furthermore, stores can propose products not featured in advertising medium 30 to customers, in addition to the products featured in advertising medium 30, thereby boosting sales of their products.

[0028] Before using the service, user U must register user information in advance, such as creating a user account (user ID, password, etc.). By logging into the menu suggestion system 1 (server device 10) on the communication device 20 using a designated application or website with the previously registered account information, user U can use the services provided by the menu suggestion system 1.

[0029] By installing and starting an application program on the communication device 20 in Figure 3 that implements at least some of the functions of the information processing device 100 in Figure 1, it is possible to realize the functions of the information processing device 100 in Figure 1 in cooperation with the server device 10 in Figure 3.

[0030] <Example Hardware Configuration> The information processing device 100 described herein is implemented by the computer 1000 shown in Figure 4. The server device 10, communication device 20, and store server 200 shown in Figure 3 are also implemented by the computer 1000. Furthermore, the functions of the information processing device 100 can be shared between the computer 1000 of the communication device 20 and the computer 1000 of the server device 10.

[0031] Computer 1000 has a bus 1010, a processor 1020, memory 1030, a storage device 1040, an input / output interface 1050, and a network interface 1060.

[0032] Bus 1010 is a data transmission path for the processor 1020, memory 1030, storage device 1040, input / output interface 1050, and network interface 1060 to send and receive data to and from each other. However, the method of connecting the processor 1020 and other components to each other is not limited to bus connection.

[0033] The 1020 processor is a processor implemented in components such as the CPU (Central Processing Unit) and GPU (Graphics Processing Unit).

[0034] Memory 1030 is a main memory device implemented using RAM (Random Access Memory), etc.

[0035] The storage device 1040 is an auxiliary storage device implemented as an HDD (Hard Disk Drive), SSD (Solid State Drive), memory card, or ROM (Read Only Memory). The storage device 1040 stores program modules that implement each function of the information processing device 100 (for example, the acquisition unit 102, the generation unit 104, the output control unit 106, etc.). The processor 1020 reads these program modules into the memory 1030 and executes them, thereby realizing each function corresponding to the program module. The storage device 1040 may also function as the storage device 120 of the server device 10, or as the storage device 220 of the store server 200.

[0036] The program module may be recorded on a recording medium. The recording medium on which the program module is recorded includes a non-temporary, tangible medium usable by a computer 1000, and program code that can be read by the computer 1000 (processor 1020) may be embedded in that medium.

[0037] The input / output interface 1050 is an interface for connecting the computer 1000 to various input / output devices. The input / output interface 1050 also functions as a communication interface for short-range wireless communication such as Bluetooth (registered trademark) and NFC (Near Field Communication). Furthermore, the input / output interface 1050 also functions as a wired communication interface such as USB (Universal Serial Bus) and HDMI (High Definition Multimedia Interface) (registered trademark).

[0038] The network interface 1060 is an interface for connecting the computer 1000 to the communication network 3. This communication network 3 is, for example, a LAN (Local Area Network) or a WAN (Wide Area Network). The method by which the network interface 1060 connects to the communication network 3 may be a wireless connection or a wired connection. Furthermore, the network interface 1060 also functions as a communication interface for communication using mobile communication systems such as CDMA (Code Division Multiple Access), LTE (Long Term Evolution), 4th generation communication (4G), 5th generation communication (5G), and 6th generation communication (6G) and later, via a mobile communication network.

[0039] The computer 1000 then connects to necessary devices (for example, the display, touch panel, operation buttons, touchpad, keyboard, mouse, speaker, microphone, camera, etc. of the server device 10 or the communication device 20) via the input / output interface 1050 or the network interface 1060.

[0040] Each component of the information processing device 100 shown in each figure can be realized by any combination of the hardware and software of the computer 1000 shown in Figure 4. Those skilled in the art will understand that there are various modifications to the implementation method and apparatus. The functional block diagrams showing the information processing device 100 shown in each figure represent blocks of logical functional units, not hardware-level configurations.

[0041] <Example of Functional Configuration> The following explanation will use Figure 1. As shown in Figure 1, the information processing device 100 includes an acquisition unit 102, a generation unit 104, and an output control unit 106. The acquisition unit 102 acquires images of the advertising media 30. The advertising media 30 is a medium on which advertisements for products sold in the store are recorded, and includes paper media such as flyers and digital advertisements posted on websites. The acquisition unit 102 acquires images of the advertising media 30 by having user U photograph the advertising media 30 with the camera 22 of the communication device 20. Alternatively, the acquisition unit 102 may acquire images of digital advertisements contained in the store's website, posts on SNS (Social Networking Service) made by the store, or emails sent from the store using the communication device 20.

[0042] The camera 22 is equipped with a lens and an image sensor such as a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor.

[0043] When acquiring an advertisement image from a flyer, the acquisition unit 102 first accepts the selection of an operation for reading the advertisement image from the operation menu of the application installed and running on the communication device 20. Then, the acquisition unit 102 displays an operation screen for reading the advertisement image on the display of the communication device 20. For example, the acquisition unit 102 may superimpose a guide for capturing the advertisement with the camera 22 onto the view screen of the camera 22.

[0044] User U can operate the camera 22 of the communication device 20 according to the guide to capture images of the advertising medium 30. The acquisition unit 102 acquires the images of the advertising medium 30 generated by the camera 22 of the communication device 20. For example, the acquisition unit 102 instructs the communication device 20 to transmit the images to the server device 10. The acquisition unit 102 receives the images from the communication device 20. The image data is preferably a still image, but it may also be a moving image or a frame image at predetermined intervals.

[0045] Alternatively, user U may launch a browser and download the displayed digital advertisement in advance, saving it as image data of the advertisement in the memory 1030 or storage device 1040 of the computer 1000 that implements the communication device 20. The acquisition unit 102 may accept an operation on the operation screen displayed on the communication device 20 to select the image data of the advertisement to be sent from the communication device 20 to the server device 10, and acquire the selected image data. The acquisition unit 102 instructs the communication device 20 to send the image data to the server device 10. The acquisition unit 102 receives the image data from the communication device 20.

[0046] As described above, when user U uses the services provided by the menu suggestion system 1, user U must complete a user registration procedure in advance. As shown in Figure 5, user information 130 includes, for each user U, information that can identify user U (user ID in the figure), associated with authentication information for authenticating user U. User information 130 is stored in the storage device 120. The authentication information may be, for example, a password, or biometric authentication information such as a face or fingerprint.

[0047] Furthermore, the user U is asked to input various information necessary for generating the menu in advance, and the acquisition unit 102 stores the input information as user setting information 132 in the storage device 120. For example, the acquisition unit 102 receives information about various information input using a setting menu or the like in the application of the communication device 20. The acquisition unit 102 stores the input information in user setting information 132. Alternatively, the acquisition unit 102 may obtain the information necessary for generating the menu by asking questions to the user U in a chat format and receiving input from the user U's answers.

[0048] The user settings information 132 may be registered in advance, but it may also be obtained by extracting product information from the advertising medium 30, inputting the product information into the generating AI (Artificial Intelligence), and then having the generating AI inquire about the information it deems necessary to generate a menu (such as budget and cuisine genre) from the user U in a chat format.

[0049] As shown in Figure 6, the user setting information 132 includes, for each user U, information that identifies the user U (user ID in the figure), information indicating the shopping budget, information indicating the genre of the menu, and information indicating the number of people the menu is intended for, associated with the user U. The genre is information that can identify the type of cuisine, such as Japanese, Western, Chinese, Italian, or Asian. However, the information necessary to generate the menu stored in the user setting information 132 is not limited to this and may include other information (for example, ingredients to be excluded).

[0050] The acquisition unit 102 stores the image data received from the communication device 20 as reception information 140 in the storage device 120. As shown in Figure 7, the reception information 140 includes information that can identify user U (user ID in the figure), an acceptance ID that is assigned each time image data is received, information indicating the date and time the image data was received, and the image data of the advertising medium 30, associated with each other. The actual image data file of the advertising medium 30 is stored in a predetermined directory of the storage device 120. The reception information 140 includes information indicating the storage location (directory name) and file name (e.g., path name) where the image data file of the advertising medium 30 is stored.

[0051] The generation unit 104 uses the acquired images to obtain product information about the products listed on the advertising medium 30, and uses that product information to generate a menu.

[0052] First, the generation unit 104 processes the images of the advertising medium 30 to extract and identify the products listed on the advertising medium 30. Image processing includes various image data analysis processes such as pattern recognition based on predetermined conditions, determining quality by comparing with reference values, recognizing objects by matching them with the features of predetermined objects, and measuring the size of those objects. Feature data extraction may be achieved using methods such as SIFT (Scale-Invariant Feature Transform) or HOG (Histograms of Oriented Gradients).

[0053] Product information includes, for example, information such as the product name (name of ingredient, food product, etc.) and price. The product information extracted by the generation unit 104 is stored in the storage device 120 as advertising information 160. As shown in Figure 9, the advertising information 160 includes, for each user U, information that can identify user U (user ID in the figure), a reception ID assigned each time an image of the advertising medium 30 is received, identification information that can identify the target store of the advertising medium 30 (store ID in the figure), and product information for each product extracted from the advertising medium 30, all associated with each user U. The target store information for the advertising medium 30 is also identified from the advertising medium 30 by image processing by the generation unit 104. The product information includes the product name and the advertised price of the product.

[0054] The generation unit 104 generates a menu based on product information extracted from the image. The menu generation process may be performed using, for example, a generation AI or a large-scale language model. For example, the generation unit 104 inputs the product information extracted from the image (product information from the advertising information 160 described above) and the user setting information 132 described above into the generation AI and outputs a menu. However, the generation unit 104 may generate a menu using a method that does not use a generation AI. For example, a menu database containing information on ingredients and cooking methods necessary for the menu may be prepared, and the generation unit 104 may generate a menu by searching the menu database using product information extracted from the advertising medium 30.

[0055] Furthermore, the generation unit 104 also acquires information about the ingredients necessary for the menu generated by the generation AI. This information about ingredients is not published in the advertising medium 30, but it contains information about products that should be purchased in relation to the menu.

[0056] The generation unit 104 stores the generated information as menu information 170 in the storage device 120. As shown in Figure 10, the menu information 170 includes information that can identify user U (user ID in the figure), an acceptance ID assigned each time an image from the advertising medium 30 is received, cooking information about the dishes included in the menu, information that can identify the target store of the advertising medium 30 (store ID in the figure), and information about products that should be purchased in relation to the menu (list of products to be purchased in the figure).

[0057] The dish information includes the dish name and recipe information showing the ingredients and preparation method. The recipe information includes, for example, links (URLs (Uniform Resource Locator)) to websites or video sites showing the ingredients and preparation method for the dish.

[0058] Furthermore, the menu suggestion system 1 may also provide support services for cooking the suggested dishes, even after the product has been purchased, by linking with other recipe websites.

[0059] In the example in Figure 10, the information about products to be purchased does not include information about products listed in the advertisement and products not listed in the advertisement. However, in other examples, the information about products to be purchased further includes information that can identify products listed in the advertisement and products not listed in the advertisement.

[0060] For products not listed in the advertisement, the generation unit 104 queries the store server 200 for product information and obtains product information including the product name and price. The store server 200 refers to the product master 230 in the storage device 220 and returns the product information for the product to the information processing device 100.

[0061] As shown in Figure 8, the product master 230 includes, for each store, store-specific identification information (store ID in the figure), and for each product, product-specific information (product code in the figure), along with information indicating the product name and information indicating the price.

[0062] The output control unit 106 causes the output means to output output information including information indicating the menu and information regarding products to be purchased in relation to the menu. The output means is the display of the communication device 20 used by user U. Alternatively, it is the display of the communication device 20 installed on the shopping cart 40 in the store used by user U. The display is, for example, a liquid crystal display or an organic EL (Electro-Luminescence) display.

[0063] The output control unit 106 outputs information about the product to be purchased to the output means, including information about the product to be purchased in a manner that allows the user to select whether or not to purchase the product (the purchase button 330 in Figure 11, which will be described later). The output control unit 106 then causes the output means to output a list of planned purchase items, including the items selected by user U as items to be purchased, as output information.

[0064] The output control unit 106 further causes the output means to output output information (the advertising flyer product display section 322 and the advertising flyer non-product display section 324 in Figure 11, which will be described later) that distinguishes between information about products listed in the advertising medium 30 and information about products not listed in the advertising medium 30 among the products to be purchased.

[0065] As shown in Figure 11, the output control unit 106 displays screen 300 on the display of the communication device 20 as an example of output information. Screen 300 includes a menu display area 310 and a product list display area 320. The menu display area 310 displays the names of the dishes included in the menu generated by the generation unit 104. The product list display area 320 includes information indicating the target store where the products will be purchased, an advertising flyer product display area 322, and an advertising flyer product display area 324. The advertising flyer product display area 322 displays product information for products extracted from the advertising medium 30 that are necessary for the menu, that is, products listed in the advertisement. The advertising flyer product display area 324 displays product information for products that are not listed in the advertisement but are necessary for the menu. Each product included in the product list display area 320 is provided with a purchase button 330. The purchase button 330 is a UI (User Interface) for accepting the selection of products that user U has decided to purchase at the target store.

[0066] Information regarding the product for which the purchase button 330 has been pressed is stored in the storage device 120 as the planned purchase product information 180 in Figure 12. The planned purchase product information 180 includes information that can identify user U (user ID in the figure), an acceptance ID that is assigned each time image data is received, and information indicating the product to be purchased (purchased product information in the figure) for each product for which the purchase button 330 has been pressed. The purchased product information includes the product name and price.

[0067] The product information stored in the planned purchase product information 180 is included in the output information as a planned purchase product list (for example, a shopping list) and displayed on the display of the communication device 20.

[0068] Furthermore, the output control unit 106 may obtain price information for products handled by the store from the product information (product master 230) and include it in the output information before outputting it to the output means. The price information for products handled by the store can be obtained by the generation unit 104 querying the store server 200. Alternatively, the generation unit 104 may obtain the product price by character recognition after processing the image of the advertising medium 30. The output control unit 106 can include this price information in the output information.

[0069] <Example of operation> The following describes an example of the processing operation of the menu suggestion system 1 using Figure 13. In the menu suggestion system 1, as shown in Figure 13, first, the communication device 20 generates an image of the advertising medium 30 using the camera 22 (or downloads and acquires it from a website) and transmits it to the information processing device 100 (step S201). In the information processing device 100, the acquisition unit 102 acquires the image of the advertising medium 30 (step S203) (corresponding to step S101 in Figure 2).

[0070] Then, in the information processing device 100, the generation unit 104 extracts products from the images of the advertising medium 30 by image analysis (step S205) (corresponding to step S103 in Figure 2). Then, the generation unit 104 generates a menu using the extracted product information (step S207) (corresponding to step S103 in Figure 2).

[0071] The generation unit 104 further identifies ingredients required for the menu that are not listed in the advertising medium 30, and queries the store server 200 for information on those products (step S209). The store server 200 searches the product master 230 and returns the product information to the information processing device 100 (step S211).

[0072] The output control unit 106 causes the communication device 20 to output output information including information indicating the menu and information regarding products to be purchased in relation to the menu (step S213) (corresponding to step S105 in Figure 2). The communication device 20 displays the screen 300, which includes the menu and shopping information received from the information processing device 100, on the display (step S215).

[0073] The timing at which the output control unit 106 outputs the output information can vary and is not limited to the examples given below. Multiple combinations of the following can be used. (1) When output information including the menu is generated (2) When the registration of items to be purchased is accepted and a shopping list is generated (3) When the operation of user U is received in the operation menu of the application of the communication device 20 (4) When it is detected that user U has entered the store (location information of the communication device 20, acceptance of the store entry operation (e.g., reading of member ID, facial recognition, etc.))

[0074] As described above, the information processing device 100 according to this disclosure comprises an acquisition unit 102 and a generation unit 104. The acquisition unit 102 acquires images of advertising media, and the generation unit 104 uses the acquired images to acquire product information about the products listed on the advertising media and generates a menu using that product information. The output control unit 106 then causes the output means to output output information including information indicating the menu and information about products that should be purchased in relation to the menu. In this way, the information processing device 100 according to this disclosure can provide the user with information about products that should be purchased in relation to the menu suggested from the images of the advertising media (information that leads to the purchase of products), so it is possible to recommend the purchase of all necessary ingredients for the menu suggested from the advertisement without any excess or deficiency. Furthermore, the information processing device 100 can also be used to promote the sale of products in stores.

[0075] The embodiments of the present invention have been described above with reference to the drawings, but these are merely examples of the present invention, and various other configurations can also be adopted. (Other Embodiment 1) The generation unit 104 extracts product information using images contained in a specific area 410 (Figure 14) of the acquired advertising medium 30. The generation unit 104 uses images from the specific area 410 of the advertising image 400 acquired by the acquisition unit 102, for example, images containing information about featured products. In other words, the generation unit 104 does not extract product information from areas 420 of the advertising image 400 other than the specific area 410.

[0076] There are various ways to specify the specific area 410, as exemplified below, but not limited to these. (1) Within the image of the advertising medium 30 captured by user U, a region is specified, for example, using a UI for specifying regions. (2) Within the advertising medium 30, the area containing limited-time sale information is identified from the image of the advertising medium 30 by image processing, and the area containing information for the period that includes the date specified by user U or the date the menu was created is identified as the specified area 410. (3) The area containing product information with a discount rate (e.g., 20% discount) or higher specified by user U is identified from the image of the advertising medium 30 by image processing and designated as the identified area 410. (4) Within the advertising medium 30, the area containing information about food is identified from the image of the advertising medium 30 by image processing and designated as the identified area 410.

[0077] With this configuration, instead of targeting all products in the advertising medium 30 for menu generation, only the necessary specific areas 410 are targeted, thereby reducing the processing load.

[0078] (Another Embodiment 2) The output control unit 106 causes the output means to output a map showing the display location in the store for the products to be purchased, which are included in the list of products to be purchased.

[0079] The output control unit 106 queries the store server 200 for the display location of the product to be purchased in the store. The output control unit 106 obtains the store's location information on the map from the store server 200 and displays the display location of the purchased product on the store's map.

[0080] The output control unit 106 may also accept a request from the user U to specify which products they wish to purchase from the list of products to be purchased, and display the display location of the specified products on the map.

[0081] Alternatively, the output control unit 106 may display information on a map indicating the display location of each product, along with a route for efficiently purchasing the products included in the planned purchase list.

[0082] This configuration allows user U to be presented with information about the display location of the items they plan to purchase in the store, enabling them to make purchases smoothly.

[0083] (Other Embodiment 3) The output control unit 106 may further include recommended products in the output information. Recommended products may include, for example, products that are not necessary for the menu. Recommended products may also include products that the store is promoting.

[0084] There are various possible timings for displaying recommended products on the output device, as exemplified below, but not limited to these. Furthermore, multiple combinations of the following can be used. (1) When a list of items to be purchased (shopping list) is generated (2) When the store map is displayed on the display of the communication device 20 For example, display items in stages (Level 1: items intended for purchase, Level 2: recommended items, etc.). (3) When an item is added to the shopping cart 40 in the store (a pop-up message is displayed on the display of the communication device 20 installed in the shopping cart 40)

[0085] This configuration is expected to further stimulate sales of products unrelated to the menu.

[0086] (Another Embodiment 4) The output control unit 106 may output an alert using the budget set in the user setting information 132. The output control unit 106 outputs an alert if the total price of the items that user U is trying to purchase exceeds the budget.

[0087] There are various possible timings for determining whether or not to issue an alert, as exemplified below, but these are not limited to those examples. Furthermore, multiple combinations of the following can be used. (1) When a list of items to be purchased (shopping list) is generated (2) When an item is added to the shopping cart 40 in the store

[0088] With this configuration, an alert is issued if user U attempts to purchase an item that exceeds their budget, thus ensuring that user U can shop within their budget.

[0089] (Other Embodiments 5) When the generation unit 104 generates the planned purchase product information 180, it may proceed to the product ordering site on the online supermarket website of the store in question.

[0090] With this configuration, user U can purchase the necessary ingredients for their meals without having to go to a store. Furthermore, since the products are delivered, there is no need to transport them themselves, making it very convenient.

[0091] Other examples of the disclosure may include a program that causes at least one computer to execute the method disclosed above, or a recording medium that is readable by a computer on which such a program is recorded. This recording medium may include non-temporary tangible media. This computer program includes computer program code that, when executed by a computer, causes the computer to perform an information processing method on an information processing device.

[0092] Furthermore, any combination of the above components, as well as any conversion of the expressions of this disclosure between methods, apparatus, systems, recording media, computer programs, etc., are also valid forms of this disclosure.

[0093] Furthermore, the various components of this disclosure do not necessarily have to be independent entities; multiple components may be formed as a single member, one component may be formed from multiple members, one component may be part of another component, or a part of one component may overlap with a part of another component, and so on.

[0094] Furthermore, although the methods and computer programs disclosed herein describe multiple steps in sequence, the order in which these steps are described does not limit the order in which they must be performed. Therefore, when implementing the methods and computer programs disclosed herein, the order of the steps can be changed to the extent that it does not impair the content.

[0095] Furthermore, the methods and computer programs described herein are not limited to being executed at individually different times. For example, other procedures may be executed while one procedure is being executed, or the timing of one procedure's execution may overlap with the timing of another procedure's execution in whole or in part.

[0096] Although the present disclosure has been described above with reference to embodiments, the present disclosure is not limited to the embodiments described above. Various modifications to the structure and details of the present disclosure can be made as can be understood by those skilled in the art within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate. Furthermore, any information regarding users (User U) obtained or used in this disclosure shall be conducted in a lawful manner.

[0097] Furthermore, while the flowcharts used in the above description show multiple steps (processes) in sequence, the execution order of the steps performed in each embodiment is not limited to the order in which they are described. In each embodiment, the order of the illustrated steps can be changed to the extent that it does not impede the content.

[0098] Some or all of the above embodiments may also be described as follows, but are not limited to the following: 1. A means for acquiring images from advertising media, A generation means that uses the acquired image to acquire product information relating to the products listed on the advertising medium and generates a menu using the product information, An output control means causes an output means to output output information including information indicating the menu and information regarding products to be purchased in relation to the menu, An information processing device equipped with the following features. 2. In the information processing device described in 1., The output control means is The information regarding the product to be purchased is included in the output information in a manner that allows the user to select whether or not to purchase the product, and the output is output to the output means. An information processing device that then outputs to the output means a list of planned purchase items, including the planned purchase items selected by the user as necessary for purchase, to the output information. 3. In the information processing device described in 1. or 2., The output control means is an information processing device that obtains price information of products from product information of products handled in a store, includes it in the output information, and outputs it to the output means. 4. In the information processing device described in any one of items 1 to 3, The output control means is an information processing device that causes the output means to output output information that distinguishes between information about products listed in the advertising medium and information about products not listed in the advertising medium among the products to be purchased. 5. In the information processing device described in any one of items 1 to 4, The acquisition means is an information processing device that acquires the advertising medium from a website. 6. In the information processing device described in any one of items 1 to 5, The generation means is an information processing device that extracts product information using an image contained in a specific area of ​​the acquired advertising medium. In the information processing device described in 7.2, The output control means is an information processing device that causes the output means to output a map showing the display location in a store for the products to be purchased that are included in the list of products to be purchased.

[0099] 8. Communication equipment used by the user, The communication device and the server device connected via a network are included, The server device is The communication device includes an acquisition means for acquiring images of advertising media, A generation means for generating a menu based on product information about products published on the advertising medium extracted from the acquired image, The system includes an output control means that causes the output means to output output information including information about products to be purchased, in addition to the generated menu. Menu suggestion system. 9. One or more computers, Obtain images from advertising media, Using the acquired image, product information relating to the product displayed on the advertising medium is obtained, and a menu is generated using that product information. An information processing method that causes an output means to output output information including information indicating the menu and information regarding products to be purchased in relation to the menu. 10. To the computer, The process of acquiring images from advertising media. Using the acquired image, product information relating to the product displayed on the advertising medium is obtained, and a generation process is performed to generate a menu using the product information. A program for causing an output control process to output output information to an output means, which includes information indicating the menu and information regarding products to be purchased in relation to the menu. 11. To the computer, The process of acquiring images from advertising media. Using the acquired image, product information relating to the product displayed on the advertising medium is obtained, and a generation process is performed to generate a menu using the product information. A computer-readable recording medium storing a program for causing an output control process to output output information to an output means, which includes information indicating the menu and information regarding products to be purchased in relation to the menu.

[0100] Furthermore, some or all of the configurations described in Appendices 2 to 7, which are subordinate to Appendice 1 (information processing device) as described above, may also be subordinate to Appendice 8 (menu suggestion system), Appendice 9 (information processing method), Appendice 10 (program), and Appendice 11 (recording medium) in the same subordinate relationship as in Appendices 2 to 7. Moreover, not limited to Appendices 8, 9, 10, and 11, some or all of the configurations described as appendices may also be subordinate to various hardware, software, various recording means for recording software, or systems, without departing from the embodiments described above. [Explanation of Symbols]

[0101] 1. Menu suggestion system 10 Server devices 20 Communication equipment 22 cameras 30 advertising media 40 Shopping Carts 100 Information Processing Devices 102 Acquisition Department 104 Generation part 106 Output Control Unit 120 Storage device 130 User Information 132 User Settings Information 140 Received Information 160 Advertisement Information 170 Menu Information 180 Information on items you plan to purchase 200 store servers 220 Storage device 230 Product Master 300 screens 1000 computers 1010 Bus 1020 Processor 1030 memory 1040 Storage Devices 1050 Input / Output Interface 1060 Network Interfaces

Claims

1. A means of acquiring images for advertising media, A generation means that uses the acquired image to acquire product information relating to the product displayed on the advertising medium and generates a menu using the product information, An output control means causes an output means to output output information including information showing the menu and information about products to be purchased in relation to the menu, An information processing device equipped with the following features.

2. In the information processing apparatus according to claim 1, The output control means is The information regarding the product to be purchased is included in the output information in a manner that allows the user to select whether or not to purchase the product, and the output is output to the output means. An information processing device that then outputs to the output means a list of planned purchase items, including the planned purchase items selected by the user as necessary for purchase, to the output information.

3. In the information processing apparatus according to claim 1 or 2, The output control means is an information processing device that obtains price information of products from product information of products handled in a store, includes it in the output information, and outputs it to the output means.

4. In the information processing apparatus according to claim 1 or 2, The output control means is an information processing device that causes the output means to output the output information which includes, in a manner that distinguishes between information about products listed in the advertising medium and information about products not listed in the advertising medium among the products to be purchased.

5. In the information processing apparatus according to claim 1 or 2, The acquisition means is an information processing device that acquires the advertising medium from a website.

6. In the information processing apparatus according to claim 1 or 2, The generation means is an information processing device that extracts product information using an image contained in a specific area of ​​the acquired advertising medium.

7. In the information processing apparatus according to claim 2, The output control means is an information processing device that causes the output means to output a map showing the display location in a store for the products to be purchased that are included in the list of products to be purchased.

8. Communication devices used by the user, The communication device and the server device connected via a network are included, The server device is The communication device includes an acquisition means for acquiring images of advertising media, A generation means for generating a menu based on product information about products published on the advertising medium extracted from the acquired image, The system includes an output control means that causes the output means to output output information including information about products to be purchased, in addition to the generated menu. Menu suggestion system.

9. One or more computers, Obtain images from advertising media, Using the acquired image, product information relating to the product displayed on the advertising medium is obtained, and a menu is generated using that product information. An information processing method that causes an output means to output output information including information showing the menu and information about products to be purchased in relation to the menu.

10. On the computer, The process of acquiring images from advertising media. Using the acquired image, product information relating to the product displayed on the advertising medium is obtained, and a generation process is performed to generate a menu using the product information. A program for causing an output control process to output output information to an output means, which includes information indicating the menu and information regarding products to be purchased in relation to the menu.

Images