Method of providing food and beverage information

Abstract

To provide food and beverage product information for providing a user with detailed information on food and beverage products provided at a food and beverage store.SOLUTION: In a system 1 that provides food and beverage product information, a management server 10 generates a machine learned model by machine learning on learning data of an image captured by capturing an image of a food model of each of food and beverage products displayed by a food and beverage store and an image with a food and beverage ID that identifies a food product that the food model represents and stores the food and beverage product information for each food and beverage product associated with the food and beverage ID. Furthermore, by inputting a user captured image of a food model displayed in a food and beverage store captured by a camera of a user terminal 30 into the learned model via an Internet N, the management server 10 outputs from the learned model the food and beverage product ID estimated to correspond to the food model an image of which is to be captured, extracts the food and beverage product information associated with the outputted food and beverage product ID, and provides it to the user terminal 30.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a food and drink information providing method for providing a user with information about food and drink served at a food and drink establishment such as a restaurant. [Background technology]

[0002] In recent years, an increasing number of people are seeking safety and peace of mind when it comes to food and beverages. As a result, there is a growing demand for information about the ingredients and origins of the food and beverages served at restaurants and other food establishments. For people with food allergies, it is extremely important to know whether or not a food or beverage contains allergens that cause allergic symptoms. Furthermore, for some people, for religious reasons, they cannot eat certain ingredients or can only tolerate food and beverages that have been processed and cooked in a specific way. For such people, it is extremely important to know whether the food and beverages served at restaurants meet these conditions.

[0003] However, it is difficult to obtain such detailed information about the food and beverages offered at food and beverage establishments. Summary of the Invention [Problem to be solved by the invention]

[0004] In view of the above-mentioned circumstances, the present invention aims to provide a food and beverage information providing method that can provide users with detailed information about food and beverages offered at food and beverage establishments. [Means for solving the problem]

[0005] In order to solve the above problems, the food and drink information providing method according to the present invention comprises: "The food and beverage store will create multiple food models using non-food materials to replicate the appearance of each of the various types of food and beverage it offers. , at the storefront of the food and beverage store so as to be visible from outside the store. It is exhibited, A trained model is generated by machine learning using as training data still images of each of the food models taken with a camera, each of which is accompanied by a food ID that identifies the food model that the food model imitates. For each of the foods and beverages, food and beverage information including at least one of ingredient information indicating at least the type of ingredients used in the foods and beverages and allergen information regarding allergens contained in the ingredients is stored in a store server as a database associated with the food and beverage ID, One of the food models displayed in the food store is photographed by a camera equipped in a user terminal carried by a user. From outside the store By inputting the user-captured image into the trained model via the Internet, the trained model outputs the food and beverage ID that is estimated to correspond to the food and beverage modeled by the food model that is the subject of the user-captured image, The food and drink information associated with the output food and drink ID is transmitted via the store server. By providing it to the user terminal, To the user The food and drink information is provided outside the food and drink store. It is something that provides.

[0006] Examples of "food and beverage establishments" include restaurants, coffee shops, and food courts in commercial complexes. Furthermore, confectionery shops and the food sections of department stores and supermarkets sometimes display food models to show the interior of products (food and beverages) that cannot be seen from the outside, and such establishments also fall under the category of "food and beverage establishments" of the present invention. In other words, establishments that offer and sell food and beverages and display food models also fall under the category of "food and beverage establishments" of the present invention. Furthermore, "ingredient information indicating at least the type of ingredients used in food and beverages" can include information on the origin of the ingredients in addition to the type of ingredients.

[0007] According to this method, users can receive food and beverage information, including at least either ingredient information or allergen information, via the store server simply by inputting user-captured images of food models displayed in a food and beverage store, taken with the camera on the user's device, into the trained model via the Internet. This food and beverage information was previously difficult to obtain at food and beverage stores, and meets the needs of users who seek high safety and peace of mind in food and beverages.

[0008] Furthermore, even in the past, there existed restaurants and the like that had menus (menus) with information on the ingredients of dishes (food and drink) inside the store, but even if people found out after entering that the food and drink they wanted was not available, many people hesitated to leave the restaurant without placing an order. In contrast, according to the food and drink information providing method of the present invention, when food models are displayed in front of the restaurant, detailed food and drink information about the food and drink served at the restaurant can be obtained outside the restaurant.

[0009] Furthermore, the task of photographing a food model to identify a food or drink is a fun and innovative task for users compared to tasks such as searching for information by entering text, etc. Therefore, it has the advantage of attracting users' interest and allowing them to acquire useful information while having fun. [Effects of the Invention]

[0010] As described above, according to the present invention, a food and drink information providing method can be provided that can provide detailed information about food and drink offered at a food and drink establishment to a user. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a schematic configuration diagram of a food and drink information providing system that uses a food and drink information providing method according to one embodiment of the present invention. [Figure 2] FIG. 10 is an explanatory diagram regarding transmission of a user-captured image of a food model captured by a camera of a user terminal. [Figure 3]FIG. 10 is an explanatory diagram of a screen of a user terminal displaying candidate food and drink information. [Figure 4] FIG. 10 is an explanatory diagram of a food and drink information selection screen of a user terminal. [Figure 5] FIG. 10 is an explanatory diagram of a screen of a user terminal displaying ingredient information. [Figure 6] FIG. 10 is an explanatory diagram of a screen of a user terminal displaying a list of allergens. [Figure 7] FIG. 2 is a schematic diagram of a food and drink information providing system having a different configuration from that shown in FIG. DETAILED DESCRIPTION OF THE INVENTION

[0012] A food and beverage information providing method according to one embodiment of the present invention, and a food and beverage information providing system 1 that uses this method, will be described below with reference to the drawings. As shown in Fig. 1, the food and beverage information providing system 1 (hereinafter sometimes simply referred to as "system 1") comprises a management server 10 managed by a system operator, a store terminal 20 which is a terminal used by a food and beverage establishment such as a restaurant, or by an agent such as an agency or service contractor who has been entrusted by the food and beverage establishment to perform work related to this system 1, and a user terminal 30 carried by a user who is considering using the food and beverage establishment. The management server 10, store terminal 20, and user terminal 30 are connected via the Internet N.

[0013] The management server 10 is a computer that mainly comprises a storage device consisting of a main storage device and an auxiliary storage device, a processor, and a communication device that communicates via the Internet N. The storage device of the management server 10 stores a program that causes the computer to function as system management means. The management server 10 comprises a general management server 11 for managing the entire system 1, store servers 12 assigned to each food and beverage store, and a learning server 13 for machine learning and image recognition. The system management means comprises a general management means that processes information in the general management server 11, individual management means that processes information for each store server 12, and learning management means that processes information in the learning server 13.

[0014] The user terminal 30 is a portable terminal such as a smartphone or tablet terminal, and is equipped with a memory device consisting of a main memory device and an auxiliary memory device, a processor, a communication device that communicates via the Internet N, a monitor that allows input by pressing the screen display with a finger or a stylus pen, and a camera.

[0015] The store terminal 20 may be a portable terminal or a non-portable terminal such as a desktop computer. If it is a portable terminal, it may have the same configuration as the above-mentioned user terminal 30. If it is a non-portable terminal, it may have a configuration including a storage device consisting of a main storage device and an auxiliary storage device, a processor, a communication device for communicating via the Internet N, an output device such as a monitor or printer, and an input device such as a keyboard or pointing device.

[0016] Next, the processing flow of the food and drink information providing method of this embodiment will be described.

[0017] <Register food and beverage stores and assign URLs> First, a food and beverage establishment (contract applicant) or its agent who wishes to use this system 1 accesses a website operated by management server 10. The website in this case is a website provided by comprehensive management server 11. A contract application is made by entering information about the contract applicant on the website screen, such as the name, address, telephone number, email address, login account, login password, and the name of the person in charge of managing information about the food and beverage establishment.

[0018] The integrated management means of the integrated management server 11 that receives the request assigns a store ID to the food and beverage store and stores a database associating the above input information with the store ID as a store basic database in a storage area for the integrated management server 11 in the storage device. At this time, the integrated management means creates a copy of the store server 12 and assigns it to the food and beverage store. The individual management means of the store server 12 has means for providing a store management website to which the store's staff can connect, and a user website to which users intending to use the food and beverage store can connect, and a URL is assigned to each of these websites. The integrated management means associates the URLs of each of these websites with the store ID and stores them in the store basic database. The store basic database can include the store's address as well as the latitude and longitude information of the store's location.

[0019] The comprehensive management means notifies the email address of the new subscriber food store of the URL of the store management website. Because the store management website is a website individually assigned to each food store, even subscribers of this system 1 cannot connect to the store management website of other food stores.

[0020] <Registering food and beverage information> The person in charge of each food and beverage establishment or its representative accesses the store management website assigned to them using the notified URL and logs in by entering their login account and login password. A food and beverage information management table is displayed on the store management website screen, and food and beverage information is entered there. Food and beverage information consists of food and beverage unit information registered for each food and beverage served at the food and beverage establishment, and store unit information registered for the entire food and beverage establishment.

[0021] The information per food and beverage unit may include the name of the food and beverage (the name of the dish listed on the menu or the name of the product being sold), basic information on the food and beverage including the furigana of the name (1), ingredient information including the type of ingredients used in the food and beverage and the origin of those ingredients (2), and allergen information (3) regarding allergens contained in the ingredients used in the food and beverage.

[0022] The basic information about food and drink (1) can include an actual image 80d of the food or drink taken with a camera, the total calories of the food or drink, and a description prepared by the store about the food or drink. The ingredient information (2) can include information about whether the food or drink is permissible for a particular religion, such as whether it is halal food (food permitted in Islam). The allergen information (3) can be divided into allergens that must be labeled and allergens that are recommended to be labeled, and can be entered by selecting from pre-set items.

[0023] The food and drink unit information may also include video information (4) about the food and drink, such as video footage of the food and drink being cooked. The food and drink unit information exemplified in (1) to (4) is stored in a storage area for the store server 12 in the storage device of the management server 10 as a database associated with the store ID and a food and drink ID assigned to each food and drink served at the food and drink store.

[0024] Examples of store-based information include ranking information (5) that lists the foods and drinks served at the restaurant in descending order of the number of orders, and food and drink serving environment information (6) that includes still and moving images of the kitchen where the foods and drinks are prepared and the dining area where the foods and drinks are served. The store-based information is stored in a storage area for the store server 12 as a database associated with the store ID.

[0025] <Machine learning of images of food models> A food or beverage establishment that is a contract holder of System 1 has food models made of non-edible materials such as resin that mimic the appearance and cross-section of the food or beverage it serves, and displays the food models on display shelves in its store. Alternatively, when entering into a contract for System 1, new food models are manufactured. For example, if the appearance of the main dish of multiple food or beverage dishes is similar, it is desirable to manufacture a food model that imitates this by varying the type and placement of side dishes, etc., to increase the difference in appearance of the dishes as a whole.

[0026] Each food service establishment or its agent photographs a food model that imitates each food or beverage offered at the establishment with a camera and acquires a still image. The still image of the food model is used as a training image for machine learning. It is desirable to acquire training images such as images taken under conditions equivalent to those in which a photographer standing in front of a display shelf installed in the storefront of the food service establishment photographs the food model on the display shelf, as well as multiple training images taken under different shooting conditions, such as ambient brightness. It is also desirable to acquire training images of a given food model, including multiple training images taken at slightly different angles and multiple training images in which the image of the food model being photographed varies slightly in proportion to the overall range of the still image and in which the position of the food model image varies slightly.

[0027] A staff member at the food and beverage store connects to the store management website assigned to the store from the store terminal 20 and creates and registers training data and verification data on a machine learning registration page. The training data is the above-mentioned teacher image with the food and beverage ID of the food and beverage model that the photographed food model imitates. The verification data is a teacher image without the food and beverage ID. The teacher image is sent from the store terminal 20 to the store management website, and on the machine learning registration page, the teacher image is associated with the food and beverage ID to create training data, and the teacher image is associated with the food and beverage ID to create verification data. By doing so, a training data file containing multiple training data and a verification data file containing multiple verification data are created. This process is performed for each food and beverage. In other words, the training data file and the verification data file are created in association with the food and beverage ID. As described above, individual verification data are not assigned a food and beverage ID, but the verification data file is associated with the food and beverage ID.

[0028] Then, when a button to start machine learning is pressed on the store management website screen, the learning data file and verification data file are sent to the learning server 13 along with the store ID and stored in the memory area for the learning server 13.

[0029] The learning management means of the learning server 13 uses learning data, which are teacher images with food and drink IDs, to perform machine learning using a neural network for a predetermined time to generate a trained model. Furthermore, by inputting verification data without food and drink IDs into the generated trained model, an estimated food and drink ID is output, and the weight coefficients between neurons are adjusted so that the probability that the output food and drink ID is the correct food and drink ID (the food and drink ID associated with the verification data file) is increased.

[0030] The trained model generated in this manner is associated with the store ID associated with the training data file and the verification data file and stored in the memory area for the training server 13.

[0031] <User use of the system> A user considering whether to visit a certain food or beverage store typically looks at a display shelf of food models installed in the storefront. The display shelf displays a two-dimensional code encoding the URL of a website provided by management server 10, along with an explanation that by photographing the food model with the camera of user terminal 30, information about the food or beverage can be obtained. By photographing the two-dimensional code with the camera of user terminal 30, the user can connect to the website provided by management server 10. If the two-dimensional code contains store ID information, the store ID can be referenced, and the user-oriented website provided by store server 12 to which the store ID is assigned can be viewed on user terminal 30.

[0032] Alternatively, the user can download application software for connecting to the website provided by the management server 10 via a website that allows various application software to be downloaded, or the user can connect to the website provided by the management server 10 by using a search engine.

[0033] In cases where the two-dimensional code displayed in the storefront contains only information about the URL of the website provided by the management server 10 but does not contain information about the store the user is considering using, or where the user has downloaded application software for connecting to the website provided by the management server 10 from a website that allows downloads of various application software and therefore does not contain information about the store the user is considering using, or where the user has connected to the website provided by the management server 10 through a search engine and therefore does not contain information about the store the user is considering using, the website provided by the management server 10 displays a list of available stores, from which the user can select the store the user is considering using. This allows the user terminal 30 to view the user-oriented website of the store the user is considering using. Alternatively, based on an acknowledgement input from the user terminal 30, current location information regarding the longitude and latitude of the user terminal 30 at that time can be transmitted from the user terminal 30 to the management server 10. In this case, the comprehensive management means identifies the food and beverage store that the user is considering using by comparing the longitude and latitude information in the basic store database of each food and beverage store that is a subscriber to system 1 with the current location information sent from the user terminal 30, and makes the user-oriented website provided by the store server 12 assigned to that food and beverage store available for viewing on the user terminal 30.

[0034] The user website first displays an image capture and transmission screen 51 for accepting transmission of a user-captured image 81d (see FIG. 2). The user points the camera of the user terminal 30 toward the display shelf 90 of the food model 81, makes sure that the entire food model 81, which represents the food or drink the user is considering ordering, is included within the range frame 61 for capturing the subject on the image capture and transmission screen 51, and presses the "Send captured image" button 71. This causes the user-captured image 81d of the food model 81 to be transmitted to the store server 12 and stored in a memory area for the store server 12 as a user-captured image database associated with the store ID.

[0035] Furthermore, the user-captured image 81d is transmitted from the store server 12 to the learning server 13 with the store ID attached, and is subjected to image recognition processing in the learning server 13. This image recognition processing is a process in which the user-captured image 81d is input into a trained model associated with the store ID attached to the user-captured image 81d, and the trained model outputs a food and drink ID of the food and drink that is estimated to be modeled by the food model 81, which is the subject of the user-captured image 81d. The trained model outputs multiple food and drink IDs in descending order of the probability that the user-captured image 81d corresponds to them. Here, an example is shown in which three food and drink IDs are output.

[0036] The food and drink IDs output from the trained model are sent to the store server 12 along with ranking IDs assigned in order of highest probability. The individual management means references the database of basic food and drink information (1) to extract the names of the food and drink associated with the food and drink IDs, and displays candidate food and drink information 52 on the screen of the user website, displaying the food and drink names ranked from first to third in order of highest probability of corresponding to the food model 81 photographed by the user (see FIG. 3). In the candidate food and drink information 52, each food and drink name is displayed on a food and drink selection button 72 that allows the user to select that food or drink. In addition, the screen displaying the candidate food and drink information 52 can also display a user-photographed image 81d transmitted from the user terminal 30. Here, the probability that the subject of the user-captured image 81d is a food model 81 of a food or beverage named "DX Hamburger" is 99%, which is the highest, and so that name is displayed on the first food or beverage selection button 72; the probability that the subject of the user-captured image 81d is a food model 81 of a food or beverage named "Demi-glace Hamburger" is 97%, which is the second highest, and so that name is displayed on the second food or beverage selection button 72; and the probability that the subject of the user-captured image 81d is "Cheese Hamburger" is 67%, which is the third highest, and so that name is displayed on the third food or beverage selection button 72.

[0037] In this way, by listing multiple foods and drinks, the user can select the foods and drinks that they really want to eat and drink. Note that in most cases, the subject of the user's photographed image 81d is considered to be the food and drink whose name is displayed in the first food and drink selection button 72, but by listing multiple foods and drinks in this way, the user's attention and interest is drawn to foods and drinks other than the food and drink for which the food model 81 has actually been photographed, and the user can carefully consider the foods and drinks to order.

[0038] When the user selects and presses a food / beverage selection button 72 displaying the name of the food / beverage being considered for ordering, a food / beverage information selection screen 53 is displayed, which includes a display 62 of the food / beverage name and a plurality of information selection buttons 73 for selecting types of food / beverage information (see FIG. 4). Here, an example is shown in which information selection buttons 73a, 73b, and 73c are displayed as information selection buttons 73 for selecting food / beverage unit information (2), allergen information (3), and video information (4), respectively. Also shown is an example in which an actual image 80d of the food / beverage is displayed on the screen together with the display 62 of the food / beverage name. This type of display is possible when data on the actual image 80d of the food / beverage is stored in association with a food / beverage ID in the database of basic food / beverage information (1), as described above.

[0039] When information selection button 73a for selecting to provide ingredient information (2) is pressed on this food and drink information selection screen 53, ingredient information associated with the food and drink ID of the selected food and drink is extracted from the ingredient information (2) database, and the types of ingredients used in the food and drink and the origins of those ingredients are displayed. At this time, if data on actual food and drink images 80d are stored in association with the food and drink ID in the food and drink basic information (1) database, and if coordinates indicating the positions of ingredients in actual food and drink images 80d are stored in association with each ingredient in the ingredient information (2) database, individual ingredient information 54, which is ingredient information for each ingredient, can be displayed at the position of each ingredient, superimposed on actual food and drink image 80d, as shown in Figure 5.

[0040] In addition, in the database of ingredient information (2), the process of storing the coordinates indicating the position of ingredients in the actual food and beverage image 80d in correspondence with each ingredient can be performed by displaying the actual food and beverage image 80d on the input screen for inputting information about the ingredients used in the food and beverage when registering ingredient information (2), and each time the name of an ingredient is input, using a pointing device to identify the point corresponding to that ingredient in the actual food and beverage image 80d and associating the coordinates.

[0041] Furthermore, when information selection button 73b for selecting provision of allergen information (3) is pressed on food and drink information selection screen 53, allergen information associated with the food and drink ID of the selected food and drink is extracted from the allergen information (3) database, and a list 55 of allergens contained in the food and drink is displayed (see FIG. 6). This display may be such that only allergen list 55 is displayed on the screen, but here, an example is shown in which allergen list 55 is displayed superimposed on actual food and drink image 80d. Also, here, an example is shown in which, in addition to allergen list 55, a food and drink name image 65 displaying the name of the food and drink and a character image 66 are displayed superimposed on actual food and drink image 80d.

[0042] The food and drink name images 65 and character images 66 can be moving images. Furthermore, the words spoken by the characters can be displayed in speech bubbles or output as audio through a speaker provided in the user terminal 30. The system operator can prepare a plurality of designs for the food and drink name images 65 and character images 66, and the words spoken by the characters, and store these in a database in the management server 10. A store staff member can select a preferred character image 66 or the like when connecting to the management server 10 from the store terminal 20 to register food and drink information. The selected character image 66 or the like is stored in a storage area for the store server 12 as a database associated with the store ID, and the individual management means references this database to generate a display screen for the user-oriented website.

[0043] Furthermore, when the information selection button 73c for selecting the provision of video information (4) is pressed on the food and beverage information selection screen 53, video information associated with the food and beverage ID of the selected food and beverage is extracted from the video information (4) database, and a video image of the food and beverage being cooked is played.

[0044] As described above, according to the food and beverage information providing method of this embodiment, after connecting to the management server 10 with the user terminal 30, food and beverage information including ingredient information and allergen information can be provided simply by taking a picture of the food model 81 on the display shelf 90 of the food and beverage store with the camera of the user terminal 30. This food and beverage information has been difficult to obtain at conventional food and beverage stores, and meets the needs of users who seek high safety and peace of mind in food and beverages.

[0045] Furthermore, even in the past, there existed restaurants and the like that had menus with information about the ingredients of food and beverages inside the restaurant, but even if they found out after entering that the food and beverage they wanted was not available, many people hesitated to leave the restaurant without placing an order. In contrast, according to the food and beverage information providing method of this embodiment, detailed food and beverage information about the food and beverages offered at the restaurant can be obtained outside the restaurant.

[0046] Furthermore, the task of photographing the food model 81 to identify food or drink is a fun and innovative task for the user compared to tasks such as searching for information by entering text, etc. Therefore, it has the advantage of attracting the user's interest and allowing them to acquire useful information while having fun.

[0047] In addition, because the user terminal 30 connects to the management server 10 and receives food and drink information by browsing a user-oriented website, it may be necessary to download an application to connect to the website provided by the management server 10, but there is no need to download an application to receive the main service of providing food and drink information to the user terminal 30. This has the advantage that users can use the system 1 easily.

[0048] The present invention has been described above by citing preferred embodiments, but the present invention is not limited to the above embodiments, and various improvements and design changes are possible as described below, without departing from the spirit of the present invention.

[0049] For example, using cookies, the management server 10 can track a user's use of the system 1. The system administrator managing the management server 10 can track information such as the date and time the user used the system 1, the number of times the user used the system 1, the restaurant visited, and the food model 81 that was the subject of the user-photographed image 81d. The system administrator can analyze this information and provide it to the restaurant, allowing the restaurant to tailor its food and beverage services to meet the user's needs. In this case, when a user first connects to a website provided by the management server 10 from their user terminal 30, they can input information about their attributes, such as their age group and gender. By including this information in a cookie file, the time of day they visit the restaurant, the type of restaurant they visit, their preferences and interests in food and beverages, and other information can be analyzed for each user attribute. If a user deletes cookies from the web browser of their user terminal 30, the user can be prompted to input information about their attributes again the next time they connect to a website provided by the management server 10.

[0050] Furthermore, when a user actually orders food or drink at a food or drink store, the information is sent from the user terminal 30 to the management server 10, and the store server 12 can issue a coupon to the user terminal 30 or send useful information to the user. In addition, the store server 12 records, aggregates, and analyzes the relationship between the food or drink represented by the food model 81 photographed by the user and the food or drink actually ordered, and this information can be used as useful information for the business of the food or drink store.

[0051] In the above example, the food and drink information selectable on the food and drink information selection screen 53 is only food and drink unit information, such as ingredient information (2), allergen information (3), and video information (4). However, the selectable food and drink unit information may include the total calories of the food and drink included in the basic food and drink information (1) and explanatory text prepared by the restaurant about the food and drink. Furthermore, in addition to the food and drink unit information, store unit information such as ranking information (5) and food and drink serving environment information (6) may also be selectable.

[0052] In the above example, the user-oriented website, i.e., the language displayed on the screen of the user terminal 30, is illustrated as Japanese. However, the user can select other languages, such as English, to display the website. Many foreign visitors to Japan are interested in the food models 81 displayed in front of restaurants. Furthermore, foreigners are often unfamiliar with the food and beverages offered at Japanese restaurants, and therefore have a high need for information about food and beverages. Therefore, by configuring the user-oriented website to display the website in a language other than Japanese, it is possible to provide necessary and useful food and beverage information to foreigners in an interesting way.

[0053] Additionally, in the above example, the learning server 13 is included in the management server 10. However, this is not limited to this. As shown in Figure 7, system 1b can be configured such that the learning server 40 is a separate server from the management server 10 and is connected via the Internet N, and machine learning and image recognition processing are performed in cooperation by sending and receiving data between the system management means of the management server 10 and the learning management means of the learning server 40.

[0054] It should be noted that "food and beverages provided at food and beverage establishments" are not limited to those eaten inside the establishment, i.e., those intended for eating in. Take-out food and beverages are also included in "food and beverages provided at food and beverage establishments" if they correspond to food models displayed at the establishment. [Explanation of symbols]

[0055] 1 Management Server 11. Integrated Management Server 12 Store Server 13,40 Learning Server 81 Food model 81d User-taken image N Internet

Claims

[Claim 1] The food and beverage store displays a plurality of types of food models that are made of non-food materials to resemble the appearance of each of the plurality of types of food and beverages that it provides at a storefront of the food and beverage store so that they can be seen from outside the store; A trained model is generated by machine learning using as training data still images of each of the food models taken with a camera, each of which is accompanied by a food ID that identifies the food model that the food model imitates. For each of the foods and beverages, food and beverage information including at least one of ingredient information indicating at least the type of ingredients used in the foods and beverages and allergen information regarding allergens contained in the ingredients is stored in a store server as a database associated with the food and beverage ID, A user-captured image of one of the food models displayed in the food and beverage store taken from outside the store with a camera provided on a user terminal carried by the user is input to the trained model via the Internet, and the trained model outputs the food and beverage ID that is estimated to correspond to the food and beverage modeled by the food model that is the subject of the user-captured image; The food and drink information associated with the output food and drink ID is provided to the user terminal via the store server, thereby providing the food and drink information to the user outside the food and drink store. A method for providing information on food and drink, comprising:

Images