Information processing device, control method for information processing device, and program.

Abstract

To provide an information processing device which outputs highly reliable cooking information from a web site etc.SOLUTION: An information processing device 100 includes: an information collection unit 110 which acquires a plurality of pieces of original cooking information 10 via a telecommunication line; a learned model 125 of a neural network 120 generated by learning correct answer data about a dish name included in the original cooking information in advance as teacher data 20; an estimation unit 130 which stores the learned model, and estimates the dish name by the learned model for dish images; first determination 151 which stores the cooking information as first cooking information, when a value obtained by subtracting a probability value of a next dish name from a probability value of a dish name with the highest estimation probability of the dish name estimated for each of the dish images is larger than a predetermined value; and a second determination 152 which stores the first cooking information as second cooking information, when the dish name included in the first cooking information matches the correct answer data about the dish name.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing device, a control method for an information processing device, and a program, and more particularly to an information processing device for obtaining cooking information via a telecommunications line, a control method for an information processing device, and a program. [Background technology]

[0002] Images and recipes of various dishes can be obtained via the Internet, and information about these dishes is used in ordinary households. There are a great number of websites that deal with cooking, some of which are managed by cooking experts, while others personally post information via Instagram, etc.

[0003] In order to efficiently acquire such dish information, a technology for automatic collection has been disclosed (see Patent Document 1). Also, in order to grasp the acquired information more objectively, a technology for calculating the amount of dish from a dish image has been disclosed (see Patent Document 2).

[0004] Meanwhile, a technology has been disclosed that further develops these technologies to provide useful recommandation information based on the nutritional intake desired by the website user (see Patent Document 3).

[0005] These technologies make extensive use of information processing techniques that use regression learning models based on neural networks.

[0006] In a study on food datasets, Lukas Bossard et al. disclosed that in order to measure the recognition performance using random forests, they selected the top 101 dishes from a food image posting site called foodspotting, and collected 101,000 images across 101 classes from the site to construct a dataset called Food-101 (see Non-Patent Document 1). [Prior art documents] [Patent documents]

[0007] [Patent Document 1] Patent Publication No. 2009-100351 [Patent Document 2] Patent Publication No. 2017-211691 [Patent Document 3] Patent Publication No. 2020-101932 [Non-patent literature]

[0008] [Non-Patent Document 1] Lukas Bossard, Matthieu Guillaumin and Luc Van Gool, “Food-101 - Mining Discriminative Components with Random Forests,” ECCV, pp. 446-461, 2014. Summary of the Invention [Problem to be solved by the invention]

[0009] However, as mentioned above, websites and posting sites such as SNS provide a wide variety of food information, each with different management contents, and food names that do not match the food image may be tagged. In addition, when inference is made using a trained model only on food images, the effects of presentation and color may cause other food names to be mistakenly assigned to the food image. Furthermore, people who post food information often tag their information with a broader range of food information to make their information easier to search.

[0010] In such a situation, if misrecognition or misjudgment occurs between a dish and the information associated with that dish when collecting and analyzing information, a lot of noise information may be mixed in, and the reliability of the results provided by the technology disclosed in the prior art document may be significantly reduced.

[0011] In addition, the dataset disclosed in Non-Patent Document 1 is considered to be highly versatile in Europe and the United States. However, when considering the application of this dataset to practical use in Japan, such as automatic billing and health management, there are cultural differences, and for example, there is a tendency that the menu does not include Japanese food, which is different from the menu of inexpensive and common restaurants such as company cafeterias and student cafeterias, or that there are few home-cooked dishes. This means that there is a risk that the reliability and accuracy of the dataset as a basis for automatic billing and calorie calculation in countries other than Europe and the United States will be low.

[0012] The present invention has been made in consideration of the above-mentioned actual circumstances, and has an object to provide an information processing device, a control method for an information processing device, and a program for removing noise from cooking information obtained from a website or the like, and outputting more reliable cooking information. [Means for solving the problem]

[0013] The information processing device of the present invention includes an information collecting unit that acquires a plurality of original dish information including at least a dish image and a dish name corresponding to the dish image via a telecommunication line, a trained model of a neural network generated by pre-learning using correct answer data for the dish image and the dish name corresponding to the dish image included in the original dish information as teacher data, an estimation unit that stores the trained model and infers a dish name for each of the dish images using the trained model, and an estimation unit that subtracts the probability value of the dish name with the next highest probability from the probability value of the dish name with the highest estimated probability of the dish name inferred for each of the dish images, and when the value is greater than a predetermined value set in advance, Based on the food name with the highest estimated probability A first decision to store dish information as first dish information, and a first decision to store a dish name and a second decision to store the dish information as first dish information. image and a second determination of comparing the first dish information with correct answer data for the first dish information and storing the first dish information as the second dish information if the two match.

[0014] The above configuration uses a trained model based on training data to infer food images automatically collected from websites, etc., and then makes a two-stage judgment based on the numerical difference in the estimated probability value for the top results inferred from this result, and on the consistency between the inferred food tag and correct answer data.

[0015] According to the above configuration, a considerable amount of noise can be eliminated by setting a predetermined threshold value for the numerical difference of the estimated probability value for the top result inferred in the first judgment. Furthermore, in the second judgment, information whose descriptive content is inconsistent with the information from which noise has been eliminated by the first judging unit can be eliminated as noise. By providing such a two-stage noise elimination means, a more reliable data set of cooking information can be provided without complicating the configuration.

[0016] In the above configuration, the original dish information can include a recipe and ingredients.

[0017] According to the above configuration, the user can obtain detailed information about a dish based on the dish information that is consistent between the dish image and the dish name.

[0018] The above-mentioned configuration may further include an output unit that outputs third cooking information that is a predetermined number of data randomly selected from the second cooking information.

[0019] According to the above configuration, the user can obtain highly reliable food information with consistent food images and food names without handling huge amounts of data, which improves user convenience. For example, the third food information is useful for users to intuitively judge the trends and situations of food information by viewing a certain number of food information by aligning the number of images of each class.

[0020] In the above configuration, for selected dish information arbitrarily selected from the third dish information, a dish image, recipe, and ingredients included in the selected dish information may be output.

[0021] According to the above configuration, it is possible to improve the convenience when the user actually cooks the dish selected by the user. Furthermore, based on this information, the cost of the corresponding dish can be calculated from the cost of ingredients and time, and the like, and the bill can be automatically settled, and calorie calculations can be performed.

[0022] In the above configuration, the second dish information having the same dish name may be sorted by the number of times it has been accumulated, and the dish information having the largest number of times it has been accumulated may be output as command information in a predetermined order.

[0023] According to the above-mentioned configuration, it is possible to grasp cooking trends and rankings. In addition, since it is possible to grasp information on dishes that users like, this information is useful for providers of ingredients and cooking utensils.

[0024] The control method of the information processing device of the present invention includes an information collecting step of acquiring a plurality of original dish information including at least a dish image and a dish name corresponding to the dish image via a telecommunication line, a trained model creating step of creating a trained model of a neural network generated by pre-learning using teacher data as correct answer data for the dish image and the dish name corresponding to the dish image included in the original dish information, and an estimation step of storing the trained model and inferring a dish name for each of the dish images using the trained model, and a step of subtracting the probability value of the dish name with the next highest probability from the probability value of the dish name with the highest probability of being inferred for each of the dish images, when the value is greater than a predetermined value set in advance, Based on the food name with the highest estimated probability A first determination step of accumulating dish information as first dish information, and a first determination step of accumulating a dish name and a first dish information included in the first dish information. image and a second determination step of comparing the first dish information with correct answer data for the first dish information and storing the first dish information as second dish information if the two match.

[0025] According to the above configuration, as a method invention, it is possible to provide a more reliable data set of cooking information without complicating the configuration by including a two-stage noise elimination means.

[0026] The present invention can be realized as a program for causing a computer including the above-mentioned configuration to function.

[0027] According to the above configuration, it is possible to provide a program that includes a two-stage noise elimination means and provides a highly reliable data set of cooking information without complicating the configuration. Effect of the Invention

[0028] The present invention can provide an information processing device, a control method for the information processing device, and a program for removing noise from cooking information acquired from websites, etc., and outputting more reliable cooking information. Furthermore, the output cooking information is useful information for businesses that provide ingredients, materials, and cooking utensils. It is also useful information for food providers to estimate cooking trends and the costs and time required for cooking. Furthermore, based on cooking information acquired from reliable websites, etc., it can be applied to practical uses such as automatic billing and health management (calorie calculation). [Brief description of the drawings]

[0029] [Figure 1] 1 is an overall configuration diagram according to an embodiment of the present invention; [Diagram 2] This is an example of a food image from a website that provides original food information (taken from the Cookpad website). [Diagram 3] This is an example of a search for a website that provides original cooking information (taken from the Cookpad website). [Figure 4] 13 is a flow chart for automatically collecting original dish information according to one embodiment of the present invention. [Diagram 5] This is an example of a cooking information search (search results from the Cookpad website). [Figure 6]13 is an example of a confusion matrix showing a recognition result when a determination unit is not provided. [Figure 7] This is an example of misrecognition and misjudgment of food images and food names. [Figure 8] This is an example of misrecognition and misjudgment of food images and food names. [Figure 9] 13 is a diagram showing an example of a first determination made by a determination unit. [Figure 10] 13 is a diagram showing an example of a second determination made by the determination unit. [Figure 11] 13 is an example of a confusion matrix showing a recognition result when a determination unit is provided. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0030] First, an embodiment of the present invention will be described based on the overall configuration diagram in Fig. 1. In the following description, configurations with the same reference numerals in different drawings are considered to be similar, and descriptions thereof may be omitted.

[0031] The information processing device 100 of the present invention includes an information collection unit 110 that acquires multiple original dish information 10 including at least dish images and dish names corresponding to the dish images via an electric communication line, a trained model 125 of a neural network 120 that is generated by pre-learning using teacher data 20 as correct answer data for the dish images and the dish names corresponding to the dish images included in the original dish information, and an estimation unit 130 that stores the trained model and infers a dish name for each of the dish images using the trained model, and an estimation unit 130 that subtracts the probability value of the dish name with the next highest probability from the probability value of the dish name with the highest estimation probability of the dish name inferred for each of the dish images, and when the value is greater than a predetermined value set in advance, Based on the food name with the highest estimated probability A first decision 151 for accumulating dish information as first dish information, and a dish name included in the first dish information and the dish image The specific form of the configuration may be any as long as it includes a second determination 152 that compares the first dish information with correct answer data for the first dish information and, if the two match, stores the first dish information as second dish information.

[0032] The information collecting unit 110, the neural network 120, the trained model 125, the estimation unit 130, the first judgment 151, the second judgment 152, the calculation process, the communication process, the data accumulation process, the output process, and the like, included in the information processing device 100 according to the present invention, are each composed of a microcomputer, and have a processor CPU that performs calculations, a ROM that stores a control program and lists, tables, and maps of various data, and a RAM that temporarily stores the calculation results by the CPU. These configurations are provided with a non-volatile memory, and necessary data and the like are stored in this non-volatile memory. The non-volatile memory can be composed of an EEPROM, which is a rewritable ROM, or a RAM with a backup function that supplies a holding current and holds the memory even when the power is turned off. In the following, these configurations will be described as individual means, but they may be configured to have the functions of one microcomputer, or may be realized as a program or a method.

[0033] The information collection unit 110 acquires multiple pieces of original dish information 10, which include at least dish images and dish names corresponding to the dish images, via an electric communication line. The electric communication line includes internet websites, SNS (Social Networking Service), LAN (Local Area Network), WAN (Wide Area Network), etc., and the original dish information 10 includes all dish information posted on posting sites via these electric communication lines.

[0034] The collection of information from websites will be described below with reference to Figs. 2 to 5. Fig. 2 shows an example of a dish image on a website that provides original dish information (quoted from Cookpad's website). Fig. 3 shows an example of a search on a website that provides original dish information (quoted from Cookpad's website). Fig. 4 shows a flow of automatic collection of original dish information according to one embodiment of the present invention. Fig. 5 shows an example of a search for dish information (search results from Cookpad's website).

[0035] The original recipe information 10 is recipe information posted on a posting site via an electric communication line, such as recipe information obtained from the Cookpad website (https: / / cookpad.com / ). Cookpad is a recipe community website operated by Cookpad Inc. Individuals can join the community by registering as members of this website. Members who join the community can post their own recipes and publish dishes that use recipes invented by others, complete with photos. An example of a recipe image published on the Cookpad website is shown in FIG. 2.

[0036] Figure 3 shows an example of the results of a search for the term "curry rice" on this website. The searched page (hereinafter sometimes referred to as the "recipe page") lists the name and recipe of curry rice along with an image of the dish. Furthermore, the searched recipe page stores tag information such as the title (name of the dish), dish image, recipe, and the category in which the recipe is registered as HTML (Hyper Text Markup Language) tags.

[0037] Next, automatic collection of dish images by the information collecting unit 110 will be described with reference to Fig. 4. The subject of automatic collection is the above-mentioned original dish information 10. First, a website that provides dish information is used to search by dish name (step S10).

[0038] Next, the URLs (Uniform Resource Locators) of all recipe pages are obtained from the searched result pages and stored in memory (step S11).

[0039] Next, the HTML tags of the recipe pages stored in the memory are obtained and stored in the memory (step S12).

[0040] Next, the recipe pages stored in the memory in step S12 are searched for title tags, which are acquired and stored in the memory (step S13).

[0041] Next, the recipe pages stored in the memory in step S13 are searched for cooking images, acquired, and stored in the memory (step S14).

[0042] Next, other tag information is searched for from the recipe page stored in the memory in step S14, acquired, and stored in the memory (step S15).

[0043] Next, in the title tag search result of step S13, it is determined whether there is any title tag that has not been searched for, and if there is any title that has not been searched for, steps S12 to S16 are repeated again (step S16).

[0044] When steps S10 to S16 have been completed for all recipe data, the tag information stored in memory is saved (step S17). In this way, recipe information can be collected automatically by crawling through websites. The collected recipe information becomes collected recipe information 12. The above is the procedure for automatic collection, but this is only one example and the procedure is not limited to this.

[0045] The original dish information 10 can be displayed on the recipe page as a recipe title, dish image, and tag information, for example, as shown in Fig. 5. The data to be finally saved may be, for example, a folder named after the dish name used in the search, with the dish image and other information saved in CSV (Comma Separated Value) format. Fig. 5 shows a portion of the saved CSV, and in this example, the saved original dish information 10 is, from the left, the dish name used in the search, the recipe title, the image name, and the tag information.

[0046] Next, a description will be given of the teacher data 20, the neural network 120, and the trained model 125. The teacher data 20 is data on example questions and answers that is given in advance to the neural network 120 of artificial intelligence in supervised learning of machine learning.

[0047] Neural networks 120 break down inputs into layers of abstraction, and by training them with example question-and-answer data such as training data 20, they become able to recognize patterns in speech and images, just as the human brain does. Neural networks 120 are operated by defining how the individual elements of the network are connected, and the strengths or weights of those connections. Note that the weights are typically adjusted according to specified learning rules until the neural network 120 performs the task correctly.

[0048] In the development of the training data 20 in this embodiment, in order to further increase the accuracy and improve the general-purpose performance, a means of dividing the data set into several groups (Train data, Validation data, Test data) and performing verification can be applied. Below, a brief description is given of these data groups.

[0049] The Train dataset is the dataset used for learning. Learning with this model (updating the weights) is done only with this dataset.

[0050] The validation dataset is a dataset for validation. It is used in machine learning to tune hyperparameters and to stop learning early. In other words, it is similar to the train dataset in that it improves the accuracy of the model, but the validation dataset is used for purposes other than updating weights.

[0051] The Test dataset is a dataset used for accuracy verification. Unlike the Validation dataset, the Test dataset is not used for learning, but is used to confirm the accuracy of the final model. By using this Test dataset, a more accurate accuracy verification can be performed on the trained model.

[0052] By training the neural network 120 using the teacher data 20 in this manner, it is possible to generate a trained model 125 that can predict the output for an input with high accuracy.

[0053] The trained model 125 is a model that uses the teacher data 20 as described above and performs regression learning by the neural network 120, and is generated by a neural network, etc. This trained model 125 is configured to be able to perform appropriate inference, prediction, and estimation for large amounts of data.

[0054] The estimation unit 130 inputs the collected dish information 12 collected by the automatic collection unit 110 to the trained model 125, thereby causing the trained model 125 to estimate quantitative information of the collected dish information 12. This estimated information is treated as estimated dish information 14, in which dish images are tagged with dish names, etc., and is sent to the determination unit 150.

[0055] The judgment unit 150 is composed of a first judgment 151 which subtracts the probability value of the dish name with the next highest probability of being inferred for each dish image in the estimated dish information 14 from the probability value of the dish name with the highest probability, and stores the dish information as first dish information 16 if the subtracted value is greater than a predetermined value set in advance, and a second judgment 152 which compares the dish name contained in this first dish information 16 with correct answer data for the dish name, and stores the first dish information 16 as new second dish information 18 if the two match.

[0056] The output unit 160 outputs a data set including the recipe and ingredients included in the second dish information 18. In this way, the user can obtain detailed information about the dish based on the dish information that is consistent between the dish image and the dish name. The output unit 160 may also output third dish information in which a predetermined number of data items are randomly selected from the second dish information 18. In this way, the user can obtain reliable dish information that is consistent between the dish image and the dish name without having to handle a huge amount of data, thereby improving user convenience.

[0057] Furthermore, the output unit 160 may be configured to output a dish image, recipe, and ingredients included in the selected dish information for the selected dish information arbitrarily selected from the third dish information. This can improve the convenience when the user actually cooks the dish selected by the user. Furthermore, based on this information, the cost can be calculated from the cost of ingredients and time for the corresponding dish, and automatic settlement can be performed, and calorie calculation can be performed.

[0058] In addition, the second dish information may be rearranged by the number of accumulated dishes, and the dish information having the same dish name as the second dish information may be output as the command information in the order of the number of accumulated dishes to a predetermined rank. With the above configuration, it is possible to grasp the trend and ranking of dishes. In addition, since it is possible to grasp the dish information that is preferred by the user, it is useful information for the provider of ingredients and cooking utensils.

[0059] <Description of judgment example> Specific judgment means for the first judgment 151 and the second judgment 152 will be described below with reference to Figs. 6 to 11. Fig. 6 is an example of a confusion matrix showing the recognition result when a judgment unit is not provided. Figs. 7 and 8 are examples of erroneous recognition and erroneous judgment of food images and food names. Fig. 9 is an example of judgment by the first judgment unit. Fig. 10 is an example of judgment by the second judgment unit. Fig. 11 is an example of a confusion matrix showing the recognition result when a judgment unit is provided.

[0060] <Description of an embodiment in which a determination unit is not provided> For the configuration not including the determination unit 150, first, the original dish information 10 was automatically collected from the Cookpad website (https: / / cookpad.com / ) using the collection procedure of the information collection unit 110 described in FIG. 4, and 274,392 dish images and information corresponding to the collected dish information 12 were obtained.

[0061] Here, the search name is "food name", and the results of searching for each food name are called "classes" to construct a dataset. For example, "classes" are "1. Croquettes", "2. Curry rice", "3. Fish fry", etc. In order to eliminate variations in the number of images between classes, each class is standardized to 1,000 images. As a result of this search, a dataset of 31 classes (information on 31 types of food), i.e. 31,000 images, was constructed.

[0062] For the training data set of 20, 720 images were used for training, 80 images for validation, and 200 images for testing out of the 1,000 images for each class in the dataset.

[0063] In this embodiment, learning and recognition were performed on these datasets using a convolutional neural network (CNN), which is one type of neural network. As the architecture that serves as the overall structure including the learning mechanism, EfficientNet (a model announced by Google Brain in May 2019: published paper: Mingxing Tan and Quoc V. Le, “EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks,” ICML, 2019.) was applied. Note that the architecture is not limited to EfficientNet described here, and multiple architectures can be used and appropriately selected based on the results of learning and recognition.

[0064] There are different types of EfficientNet, from B0 to B7, and the differences are in the size of the layers, the number of layers, and the resolution of the input image. In order to decide which architecture to adopt, we conducted a preliminary experiment. For the dataset, we used the dataset that had been constructed before sorting, and performed learning and recognition for each of B0 to B7.

[0065] In image recognition, the quantity called Top-k accuracy is often used to evaluate the recognition rate, and we will use this quantity here as well. Top-k accuracy is the probability that the correct answer is in the top k most likely classifications predicted by the model, and is expressed as a percentage.

[0066] As a result of training and recognition for B0 to B7, the top-1 recognition rates (%) were B0: 62.1%, B1: 63.7%, B2: 63.8%, B3: 66.1%, B4: 64.9%, B5: 65.3%, B6: 65.8%, B7: 66.1%, and B3 and B7 had the highest recognition rates at 66.1%. Considering the training time as well as the recognition rate, B3 can be trained in a short time, so B3 was applied for training and recognition from here on.

[0067] As a result of learning and recognition on a dataset constructed using 12 food images from collected food information automatically collected by EfficientNet B3, the recognition rates were 68.4% for Top-1, 84.2% for Top3, and 89.7% for Top5. In other words, the recognition results showed that the name of the dish with the highest probability was ranked 1st in 68.4% of cases, the name of this dish was included in the top three in 84.2% of cases, and the name of this dish was included in the top five in 89.7% of cases. From these results, it is difficult to say that the recognition is reliable enough to be applied to practical applications such as automatic bill settlement and health management (calorie calculation).

[0068] Figure 6 shows the confusion matrix at the time of recognition. A confusion matrix is ​​a table that organizes the number of correct answers and incorrect answers when classifying certain data. By examining the state of this confusion matrix, it is possible to infer what kind of false positives, etc. are occurring.

[0069] In this example, the recognition rate in a configuration without a judgment unit is low at 66.1% (Top-1) at best. The reason for this is presumably that there are two classes in Fig. 6 with a recognition rate of 50% or less. These correspond to "Hayashi rice" in the 8th row and "Soba" in the 22nd row of the matrix enclosed in a box in Fig. 6. This situation will be explained in more detail with reference to Figs. 7 and 8. Note that the correct class in Fig. 7 is "Hayashi rice," and in Fig. 8 is "Soba."

[0070] In the case of "Hayashi rice," Figure 7(a) was mistaken for "Omu-rice." This is presumably because the name of the dish, "Omu-hayashi," was added when the food image was posted. Next, Figure 7(b) was mistaken for "Yogurt." "Hayashi rice" is likely recognized as yogurt because white is often emphasized in appearance.

[0071] Next, for "soba," Figure 8(a) was mistaken for "udon." This is presumably because there were many toppings and the noodles, which are characteristic of "soba" and "udon," were not easily visible. Next, Figure 8(b) was mistaken for "yakisoba," and it is presumed that this was because the information on "ankake yakisoba" was saved as a search result for soba during the automatic collection of food images, resulting in the misrecognition.

[0072] As such, with conventional image recognition that does not have a judgment unit, even when training data is used, there is a risk that dishes that look similar may be in different classes, or may be misrecognized due to a lot of noise that is unnecessary information.

[0073] <Description of an embodiment in which a determination unit is provided> Next, a description will be given of a configuration including a judgment unit 150. The judgment unit 150 comprises a first judgment 151 and a second judgment 152 that performs a further judgment on a data set from which noise has been removed by the first judgment 151.

[0074] The first judgment 151 subtracts the probability value of the dish name with the next highest estimated probability inferred by the estimation unit 130 from the probability value of the dish name with the next highest estimated probability, and if the value is greater than a predetermined value, stores the dish information as the first dish information 16.

[0075] The second determination 152 compares the dish name contained in the first dish information 16 with the correct answer data for the dish name, and stores the first dish information 16 as new second dish information 18 when the two match.

[0076] To explain the details, first, a dataset of teacher data 20 is constructed. Here, the number of datasets is reduced compared to the case where a judgment unit is not provided, and the dataset of teacher data 20 includes 75 sheets for Train, 5 sheets for Validation, and 2 sheets for Test out of 100 sheets for each class of the dataset.

[0077] For these datasets, we performed learning and recognition using CNN, a type of neural network. As in the previous example, we applied EfficientNet as the overall architecture including the learning mechanism.

[0078] The test results of the trained model 125 identified by the neural network 120 using the data set of the training data 20 were 88.8% for Top-1 and 97.8% for Top-3, and it was determined that the trained model 125 could be applied.

[0079] In this embodiment, in order to make a comparison with a case where no judgment unit is provided, specific numerical values ​​are set for the first judgment 151 and the second judgment 152. A concrete example will be given below and briefly described.

[0080] In the first judgment 151, the difference in probability between the top 1st and 2nd estimated results (estimated probability) is calculated. In this embodiment, if this difference is 10% or less, the data is determined to not be the correct dish name and is noise (misrecognition, etc.), and is deleted. This is based on the knowledge that images with a small difference between the top 1st and 2nd estimated probabilities output during estimation often do not belong to the correct class.

[0081] An example is shown in Figure 9. That is, although the correct class is "hamburger steak" as shown on the left side of Figure 9, the result of the estimation probability by image recognition as shown on the right side of Figure 9 shows that the first estimation rank is "stir-fried vegetables" with an estimation probability of 40%, while "hamburger steak", which is the correct class, is second in the estimation rank with an estimation probability of 30%.

[0082] The difference in estimated probability is 40%-30%, or 10%, and this data is discarded as noise.

[0083] In the second judgment 152, when the correct class does not exist in the first place of the estimated result, it is judged to be noise.

[0084] An example is shown in Figure 10. That is, the correct class is "hamburger steak" as shown on the left side of Figure 10, and the first place in the estimated ranking is "hamburger steak" as shown on the right side of Figure 10, so this data is judged to be correct. For example, if the first place in the estimated ranking is "fried noodles" or "rice ball," this data is deleted as noise.

[0085] For the judgment unit 150 equipped with the first judgment 151 and the second judgment 152, similar to the embodiment without the judgment unit, the information collection unit 110 described in FIG. 4 automatically collected original dish information 10 from the Cookpad website (https: / / cookpad.com / ), and obtained 274,392 dish images and information corresponding to the collected dish information 12, and judgments were made on these.

[0086] As a result, 274,392 food images and information were reduced to 73,767 by providing the determination unit 150. The estimation results were 92.7% for Top-1 and 99.7% for Top-3, which was an improvement in the recognition rate compared to 68.4% for Top-1 and 84.2% for Top-3 when the determination unit was not provided.

[0087] In addition, when the judgment unit 150 is provided, the data set of teacher data 20 is 100 sheets per class, compared to 1,000 sheets when the judgment unit is not provided, which has significantly improved productivity in data construction.

[0088] Fig. 11 shows the confusion matrix at the time of recognition. Compared to Fig. 6, which shows the result when no determination unit is provided, Fig. 11 shows that there are no classes with a recognition rate of 50% or less, and the recognition rate has improved.

[0089] As described above, the present invention can provide an information processing device, a control method for the information processing device, and a program for removing noise from cooking information acquired from websites and the like, and outputting more reliable cooking information. Furthermore, the information is useful for businesses that provide ingredients, materials, and cooking utensils. Furthermore, the information is useful for food providers to estimate cooking trends and the costs and time required for cooking. Furthermore, the cooking information acquired from reliable websites and the like can be applied to practical uses such as automatic billing and health management (calorie calculation). [Explanation of symbols]

[0090] 10. Original dish information 12. Collecting food information 14. Estimated food information 16···First dish information 18···Second dish information 20. Training data 100.... Information processing device 110. Information Gathering Department 120 Neural Networks 125···Trained model 130... Estimation section 150...judgment section 151...1st judgment 152...Second judgment 160 Output section

Claims

1. an information collecting unit that acquires a plurality of pieces of original dish information including at least dish images and dish names corresponding to the dish images via an electric communication line; A trained model of a neural network generated by learning in advance using teacher data as correct answer data for the dish image included in the original dish information and the dish name corresponding to the dish image; An estimation unit that stores the trained model and infers a dish name for each of the dish images using the trained model; a first determination in which a probability value of a dish name with a next highest estimated probability is subtracted from a probability value of a dish name with a highest estimated probability inferred for each of the dish images, and when the subtracted value is greater than a predetermined value, the dish information based on the dish name with the highest estimated probability is stored as a first dish information; An information processing device characterized by including a second judgment that compares the dish name contained in the first dish information with correct answer data for the dish image, and when the two match, accumulates the first dish information as second dish information.

2. The information processing device according to claim 1 , wherein the original dish information includes a recipe and ingredients.

3. The information processing device according to claim 1 or 2, further comprising an output unit configured to output third cooking information obtained by randomly selecting a predetermined number of pieces of data from the second cooking information.

4. 4. The information processing device according to claim 3, wherein for selected dish information arbitrarily selected from the third dish information, a dish image, a recipe, and ingredients included in the selected dish information are output.

5. An information processing device according to any one of claims 1 to 4, characterized in that for the second dish information, dish information having the same dish name is sorted by the number of accumulations, and the information from the largest number of accumulations to a predetermined rank is output as command information.

6. an information collecting step of acquiring a plurality of pieces of original dish information including at least dish images and dish names corresponding to the dish images via an electric communication line; a trained model creation process for creating a trained model of a neural network generated by pre-learning the dish image included in the original dish information and correct answer data for the dish name corresponding to the dish image using teacher data; An estimation step of storing the trained model and inferring a dish name for each of the dish images using the trained model; a first determination step of subtracting a probability value of a dish name with a second highest probability from a probability value of a dish name with the highest estimation probability inferred for each of the dish images, and accumulating, when the subtracted value is greater than a predetermined value, dish information based on the dish name with the highest estimation probability as first dish information; and a second determination step of comparing a dish name included in the first dish information with correct answer data for the dish image, and storing the first dish information as second dish information when the two match.

7. 7. A program for causing a computer to function as each of the units of the information processing device according to claim 1.

Images