Diet record information identification method and device based on food image

Abstract

The invention relates to the technical field of diet record of food images, and discloses a diet record information identification method and device based on food images, food information of target food is obtained by obtaining a to-be-identified image containing at least one piece of food, and the food information comprises a target food name and an alternative food name; the target food is any food in the at least one food, the alternative food names are similar names of the target food, and the alternative number of the alternative food names is preset; obtaining a basic food name of the target food; and based on the target food name, the alternative food name and the basic food name, querying diet record information of the target food through a food warehouse. Finally, on the basis of identifying the target food name and the alternative food name of the target food, the diet record information of the target food is queried through the food library in combination with the basic food name of the target food, so that the accuracy of identifying the diet record information of the target food can be ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of food image dietary record technology, and in particular to a method and device for identifying dietary record information based on food images. Background Art

[0002] Food dietary record information can help people track the amount of food they consume daily, the nutrients and calories they eat, so dietary record information is important information for people to manage their diet in daily health. In daily life, dietary record information of food pictures is usually obtained by taking photos.

[0003] In the related art, the name of food is generally identified through image recognition algorithm, but this method not only cannot identify the weight of food in the image, but also is troublesome to expand the food category and the recognition result is not accurate enough. In order to solve this problem, a large model is used to identify the dietary record information of food. Although the large model can predict the calorie and nutrient content of food, the dietary record samples input into the large model lack uniformity, resulting in inaccurate prediction results of the large model. Summary of the invention

[0004] In view of this, the present invention provides a method and device for recognizing dietary record information based on food images, so as to solve the problem that the prediction results of the large model are inaccurate due to the lack of uniformity of dietary record samples input into the large model.

[0005] According to a first aspect, an embodiment of the present disclosure provides a method for identifying dietary record information based on food images, the method comprising:

[0006] Acquire an image to be identified containing at least one food;

[0007] Based on the image to be recognized, obtaining food information of the target food, wherein the food information includes: a target food name and an alternative food name, the target food is any one of at least one food, the alternative food name is a similar name to the target food, and the number of alternative food names is preset;

[0008] Get the basic food name of the target food;

[0009] Based on the target food name, alternative food name, and basic food name, the dietary record information of the target food is queried through the food library.

[0010] The disclosed embodiment, based on identifying the target food name and the alternative food names of the target food, combines the basic food name of the target food and queries the dietary record information of the target food through the food library, thereby ensuring the accuracy of identifying the dietary record information of the target food.

[0011] In some optional implementations, based on the target food name, the candidate food name, and the basic food name, querying the dietary record information of the target food through the food library includes:

[0012] Determine whether the target food name or the alternative food name is found in the food library;

[0013] If yes, obtain the first query result, and when the first query result includes the name of the target food, output the dietary record information of the target food through the food library;

[0014] If not, determine whether the basic food name is found from the food database;

[0015] If yes, obtain a second query result, and when the second query result includes the basic food name, output the dietary record information of the target food through the food library;

[0016] If not, it is determined that the query of the target food has failed.

[0017] The disclosed embodiment, based on identifying the target food name, alternative food names, and basic food names, further queries the target food in conjunction with the food library and verifies the first query result and the second query result of the food library, in order to ultimately accurately output the dietary record information of the target food and make up for the problem of inaccurate dietary record information recognition by existing large models.

[0018] In some optional implementations, the method for identifying dietary record information based on food images in the embodiment of the present disclosure further includes: when the first query result does not include the target food name, returning to the step of determining whether the basic food name is queried from the food library.

[0019] When the target food name cannot be found in the food library, the embodiment of the present disclosure continues to search the basic food name of the target food through the food library to prevent the target food from being missed, thereby achieving the purpose of accurately identifying the target food.

[0020] In some optional implementations, the method for identifying dietary record information based on food images in the embodiment of the present disclosure verifies whether the first query result includes the name of the target food, including:

[0021] Obtain a preset food name corresponding to the first query result, where the preset food name is a food name preset in the food library;

[0022] The preset food name and the target food name are input into a food name recognition network to obtain a name verification result, wherein the food name recognition network is a pre-trained neural network, and the name verification result includes: the preset food name and the target food name are consistent or the preset food name and the target food name are inconsistent.

[0023] The disclosed embodiment utilizes a food name recognition network to verify whether the name in the query result in the food library is consistent with the original name of the target food, in order to verify whether the food library can query the target food.

[0024] In some optional implementations, obtaining the basic food name of the target food includes:

[0025] Get the original food name of the target food;

[0026] The original food name is input into a basic name recognition network to obtain a basic food name corresponding to the original food name, wherein the basic name recognition network is a pre-trained neural network.

[0027] The disclosed embodiment uses the above implementation methods to accurately obtain the basic food name of the target food.

[0028] In some optional implementations, obtaining food information of the target food based on the image to be identified includes:

[0029] The image to be identified is input into a food information identification network to obtain a food information identification result corresponding to the target food, wherein the food information identification network is a pre-trained neural network, and the food information identification result includes: the target food name and the alternative food names.

[0030] The disclosed embodiment uses the above implementation methods to accurately obtain the target food name and the alternative food names of the target food.

[0031] In some optional embodiments, the food information recognition result also includes: food weight and food position, diet record information, including: food calories, nutrients, food detection frame information, wherein food calories are calculated by combining food weight with basic calorie units, and food detection frame information is calculated by combining food position with a preset ratio, wherein the preset ratio is the ratio of the target food to the image to be identified; and also includes: based on the food detection frame information, displaying the diet record information of the target food.

[0032] The disclosed embodiment organizes and displays the dietary record information of the target food through the above-mentioned implementation methods, so that the user can instantly know the relevant nutrition and calories of the target food, which is conducive to the user's healthy diet.

[0033] According to a second aspect, an embodiment of the present disclosure provides a device for identifying dietary record information based on food images, and the method includes:

[0034] A food image acquisition module, used to acquire an image to be identified containing at least one food;

[0035] A food information acquisition module, used to acquire food information of a target food based on the image to be recognized, wherein the food information includes: a target food name and an alternative food name, the target food is any one of at least one food, the alternative food name is a similar name to the target food, and the number of alternative food names is preset;

[0036] Food name acquisition module, used to obtain the basic food name of the target food;

[0037] The dietary information query module is used to query the dietary record information of the target food through the food library based on the target food name or the alternative food name or the basic food name.

[0038] According to a third aspect, an embodiment of the present disclosure provides a computer device, including:

[0039] The memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method for recognizing dietary record information based on food images of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0040] According to the fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, on which computer instructions are stored, and the computer instructions are used to enable a computer to execute the method for identifying dietary record information based on food images of the above-mentioned first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0042] Figure 1 is a flow chart of a method for recognizing dietary record information based on food images according to an embodiment of the present invention;

[0043] Figure 2 is a flow chart of another method for identifying dietary record information based on food images according to an embodiment of the present invention;

[0044] Figure 3 is a schematic diagram of the overall process of a method for recognizing dietary record information based on food images according to an embodiment of the present invention;

[0045] Figure 4 is a structural block diagram of a device for identifying dietary record information based on food images according to an embodiment of the present invention;

[0046] Figure 5 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0048] According to an embodiment of the present invention, an embodiment of a method for identifying dietary record information based on food images is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in an order different from that shown here.

[0049] In this embodiment, a method for recognizing dietary record information based on food images is provided, which can be used in mobile terminals such as mobile phones, tablet computers, etc. Figure 1 is a flow chart of a method for recognizing dietary record information based on food images according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0050] Step S101, obtaining an image to be identified containing at least one food.

[0051] Specifically, the food in the embodiment of the present disclosure may be fruit or vegetable or rice or beef or noodles, etc. The image to be identified may contain at least one food, and the number of foods may be at least 2, and generally the number of foods is more than 3. For example, if a fruit image contains bananas, apples, and oranges, then the image to be identified contains 3 foods. The image to be identified may be photographed by a mobile phone or a camera.

[0052] Step S102, based on the image to be identified, obtain food information of the target food, wherein the food information includes: a target food name and alternative food names, the target food is any one of at least one food, the alternative food name is a similar name to the target food, and the number of alternative food names is pre-set.

[0053] Specifically, the target food is any food in the image to be identified. For example, the image to be identified contains 3 foods, and the target food represents any food in the image. For example, the image to be identified contains the i-th food, the j-th food, and the k-th food. The target food can refer to the i-th food, the j-th food, or the k-th food. In the embodiment disclosed herein, the food information of the i-th food, the j-th food, or the k-th food can be obtained. In the embodiment disclosed herein, the food information of each of the three foods can also be obtained at the same time. The target food name is the actual name of the target food. For example, if the target food is an apple, its corresponding name is an apple. For example, when the target food is a beef steak, its corresponding name is a beef steak. The candidate food name is a plurality of similar names of the target food name. The candidate number of the similar name is preset. The preset number is usually 0-3, and the preset number can be set in combination with the actual situation. For example, when the target food is an apple, its corresponding candidate food name is 0; when the target food is a beef steak, its corresponding candidate food name can be 3, namely chicken steak, mutton steak, and pork steak.

[0054] In some optional embodiments, the above step S102, based on the image to be identified, obtains food information of the target food, including: inputting the image to be identified into a food information recognition network to obtain a food information recognition result corresponding to the target food, wherein the food information recognition network is a pre-trained neural network, and the food information recognition result includes: the target food name and the alternative food names.

[0055] Specifically, since the food information recognition network can accurately identify the name of food, the food information recognition result corresponding to the target food is obtained with the help of the general food information recognition network. The food information recognition network focuses on the raw materials and appearance characteristics of the food. If the raw materials are not certain enough, similar foods can be given, which can greatly improve the effect of outputting similar foods; for example, the image to be identified is a beef burger, but because the steak has been fried, it is not necessarily certain whether it is beef. Therefore, the target food name output by the food information recognition network is beef, and the similar names are chicken burgers, pork burgers, and cheeseburgers. The alternative food names are chicken burgers, pork burgers, and cheeseburgers. The food information recognition network is pre-trained with food name sample training data, and the food information recognition network has also been verified with sample test data. For example, through the test experiment of the prompt module of the food information recognition network, it is found that introducing location information in the prompt module can improve the accuracy of the food information recognition network in predicting food names and the effect of predicting localized names.

[0056] Furthermore, in some optional implementations, the food information recognition result also includes: food weight and food location.

[0057] Specifically, in addition to outputting the target food name and the candidate food name, the food information recognition network mentioned above also needs to output the food weight and food position. The food weight can be used to further calculate the calories in the food record information. The food position represents the location of the target food in the image to be recognized. The food detection frame information can be further calculated through the food position. The food detection frame information facilitates the display of the food record information of the target food. Therefore, the food information recognition result output by the food information recognition network in the embodiment of the present disclosure includes a set of data consisting of the target food name, the candidate food name, the food weight and the food position.

[0058] Step S103, obtaining the basic food name of the target food.

[0059] Specifically, for example, the target food is a McChicken Burger, and the corresponding basic food is called a Chicken Burger.

[0060] In some optional implementations, the above step S103, obtaining the basic food name of the target food, includes:

[0061] Step a1, obtaining the original food name of the target food.

[0062] Step a2, inputting the original food name into the basic name recognition network to obtain the basic food name corresponding to the original food name, wherein the basic name recognition network is a pre-trained neural network.

[0063] Specifically, since the basic name recognition network can accurately identify the basic name of the food, the original food name corresponding to the target food is obtained with the help of the general basic name recognition network. The basic name recognition network identifies the basic food name from the text sentence of the original food name of the target food. For example, the original food name is steamed white rice. The text sentence is learned to obtain the basic food name of rice. The basic name recognition network is pre-trained with food name sample training data, and the basic name recognition network has also been verified with sample test data. For example, through the prompt module test experiment of the basic name recognition network, it is found that introducing location information in the prompt module can improve the accuracy of the basic name recognition network in predicting the basic food name.

[0064] Step S104, based on the target food name, the candidate food name, and the basic food name, the dietary record information of the target food is searched through the food library.

[0065] In a specific example, the dietary record information of the target food includes: food weight, food calories, nutritional elements, and food detection frame information; wherein, food calories are calculated by combining food weight with basic calorie units, and food detection frame information is calculated by combining food position with a preset ratio, wherein the preset ratio is the ratio of the target food location to the image to be identified.

[0066] Specifically, for example, the target food is a beef burger, and a beef burger is 250g. The basic calorie unit of a beef burger is 100g of beef burger corresponding to 400 calories. The food calories of a beef burger are 250g / 100×400=1000 calories. On this basis, the total calories of the food can also be calculated in combination with the food quantity. For example, if the food quantity of beef burgers is 2, then 2 beef burgers are 1000×2=2000 calories. The food position represents the center position of the target food in the image to be identified. For example, the center position of the target food is (3,4), and the proportion of the area where the target food is located in the image to be identified is 0.5. Then the image area of ​​the target food is (1.5, 2). The image area is the food detection frame information, and the food detection frame information is (1.5, 2). The food detection frame information is convenient for the subsequent display of the target food's dietary record information (food weight, food calories, nutrients, food detection frame information).

[0067] Specifically, after testing and supporting papers, it was found that the big model has a high accuracy rate in predicting the name of food and an acceptable weight of food. However, when using dietary records such as nutrients, calories, and servings as samples to train the big model, these sample information are inconsistent in different databases. Therefore, training the big model with these inconsistent sample data will ultimately affect the accuracy of the big model. Therefore, the disclosed embodiment uses the big model to predict the food name and food weight of the target food, and then based on the target food name, alternative food name, and basic food name of the target food, the target food is matched with the food library to obtain the accurate nutrient elements and calorie content of the target food as the dietary record information of the target food, which is finally sorted and output to the user.

[0068] Therefore, the disclosed embodiment obtains an image to be identified that contains at least one food, obtains food information of the target food, wherein the food information includes: a target food name and an alternative food name, the target food is any one of the at least one food, the alternative food name is a similar name to the target food, and the number of alternative food names is preset; obtains the basic food name of the target food; based on the target food name, the alternative food name, and the basic food name, queries the target food's dietary record information through the food library. Finally, the disclosed embodiment, based on identifying the target food name and the alternative food name of the target food, in combination with the target food's basic food name, queries the target food's dietary record information through the food library, thereby ensuring the accuracy of identifying the target food's dietary record information.

[0069] In this embodiment, a method for recognizing dietary record information based on food images is provided, which can be used in mobile terminals such as mobile phones, tablet computers, etc. Figure 2 is a flow chart of a method for recognizing dietary record information based on food images according to an embodiment of the present invention. Figure 2 As shown, in the above step S104, based on the target food name, the candidate food name, and the basic food name, the dietary record information of the target food is queried through the food library. The process includes the following steps:

[0070] Step S1041, determining whether the target food name or the candidate food name is found in the food library.

[0071] Specifically, a search is performed in the food library to match the target food name or candidate food names (0 to 3), and the food corresponding to each food name (target food name) in the food library is matched.

[0072] Step S1042: If yes, obtain the first query result, and when the first query result includes the name of the target food, output the dietary record information of the target food through the food library.

[0073] Specifically, if the target food name and the alternative food names are queried from the food library. For example, the target food name is beef burger, and the alternative food names are chicken burger, pork burger, and cheese burger, that is, the first query result is beef burger, chicken burger, pork burger, and cheese burger. Since there are multiple similar names for the target food, that is, the beef burger in the image to be identified may be similar foods such as chicken burger, pork burger, and cheese burger. At this time, it cannot be confirmed whether the target food in the image to be identified is a beef burger. Therefore, it is necessary to further verify whether the first query result contains the target food name. When the query result contains the target food name, that is, the target food exists in the food library, and then the dietary record information of the target food is output through the food library, that is, the dietary record information (food weight, food calories, nutrients, food detection box information) of at least one food (target food) in the image to be identified is output.

[0074] In some optional implementations, the method for identifying dietary record information based on food images provided in this embodiment verifies whether the first query result includes the name of the target food, including:

[0075] Step b1, obtaining a preset food name corresponding to the first query result, where the preset food name is a food name pre-set in the food library.

[0076] Specifically, for example, the first query result is beef burger, chicken burger, pork burger, cheese burger, and these names are all preset food names pre-set in the food library.

[0077] Step b2, input the preset food name and the target food name into the food name recognition network to obtain a name verification result. The food name recognition network is a pre-trained neural network, and the name verification result includes: the preset food name and the target food name are consistent or the preset food name and the target food name are inconsistent.

[0078] Specifically, the target food name and the food name in the food library are not necessarily exactly the same, and the food name recognition network is needed to perform fuzzy matching judgment of the name. The preset food name and the target food name mentioned above are input into the food name recognition network at the same time. Since the food name recognition network can accurately identify the name of the food, the universal food name recognition network is used to verify whether each preset food name is consistent with the target food name. If they are consistent, it means that the first query result contains the target food name, that is, the target food exists in the food library. Otherwise, if they are inconsistent, it means that the query result does not contain the target food name, that is, the target food does not exist in the food library.

[0079] Step S1043: if not, determine whether the basic food name is found from the food library.

[0080] Specifically, for example, the target food is a McChicken Leg Burger, and the first query result does not include: McChicken Leg. At this time, it is determined whether the basic food name of McChicken Leg Burger, Chicken Leg Burger, exists in the food library.

[0081] Step S1044: if yes, obtain the second query result, and when the second query result includes the basic food name, output the dietary record information of the target food through the food library.

[0082] Specifically, the second query result is obtained, and further verified whether the second query result includes the basic food name. The method of verifying whether the second query result includes the basic food name is the same as the method of verifying whether the first query result includes the target food name, see the above steps b1-b2, which will not be repeated here.

[0083] Step S1045: If not, it is determined that the query of the target food has failed.

[0084] Specifically, if the basic food name is not found in the food library, the target food is considered lost.

[0085] The method for identifying dietary record information based on food images in this embodiment further includes: step S1046, when the first query result does not include the target food name, returning to step S1043 of determining whether the basic food name is queried from the food library.

[0086] In the embodiment of the present disclosure, by executing the above steps S1041 to S1046, on the basis of identifying the target food name, alternative food name, and basic food name, the target food is further queried in combination with the food library and the first query result and the second query result of the food library are verified, in order to ultimately accurately output the dietary record information of the target food and make up for the inaccuracy of the existing large model in identifying the dietary record information.

[0087] In some optional implementations, the method further includes: displaying dietary record information of the target food based on the food detection frame information.

[0088] Specifically, the food detection frame information indicates the location of the target food in the image to be identified, and based on the food detection frame information, the target food's dietary record information (food weight, food calories, nutritional elements, food detection frame information) is displayed. By displaying the dietary record information of the target food, it is convenient for the user to instantly know the relevant nutrition and calories of the target food, which is conducive to the user's healthy diet.

[0089] like Figure 3 FIG. 1 is an overall flow chart of a method for recognizing dietary record information based on food images in an embodiment of the present disclosure. Figure 3In the image url, the image to be identified is the image to be identified mentioned above. The image to be identified is predicted by the large model (food information identification network), and the structured information obtained is the estimated portion, the center position of the food, the food name, and any one food has 0-3 similar food names, that is, the food information identification result (food weight, food position, target food name, alternative food name). If the food name or similar food name is searched from the food library, if the search fails, the basic food name is extracted through the large model (basic name recognition network). If the search for the basic food name fails, it is prompted that no matching food is identified. If the search for the food name from the food library is successful, it is necessary to further use the large model (food name recognition network) to verify whether the above query is the same food before and after. If it is the same food, the dietary record information (food name, portion, calories and nutrients, food detection box information) of the target food (any one food) in the image to be identified is output from the food library. If it is not the same food, the basic food name is extracted through the big model (basic name recognition network). After extracting the basic food name, the basic food name is searched through the food library. If the search is successful, the big model (food name recognition network) is used to verify whether the food before and after the above query is the same. If it is the same food, the dietary record information (food name, portion, calories and nutrients, food detection box information) of the target food (any food) in the image to be identified is output from the food library.

[0090] In this embodiment, a device for recognizing dietary record information based on food images is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0091] This embodiment provides a device for recognizing dietary record information based on food images, such as Figure 4 As shown, including:

[0092] The food image acquisition module 401 is used to acquire an image to be identified containing at least one food;

[0093] The food information acquisition module 402 is used to acquire food information of a target food based on the image to be recognized, wherein the food information includes: a target food name and an alternative food name, the target food is any one of a plurality of foods, the alternative food name is a similar name to the target food, and the number of alternative food names is preset;

[0094] The food name acquisition module 403 is used to acquire the basic food name of the target food;

[0095] The dietary information query module 404 is used to query the dietary record information of the target food through the food library based on the target food name or the candidate food name or the basic food name.

[0096] In some optional implementations, the dietary information query module 404 includes:

[0097] The first judgment submodule is used to judge whether the target food name or the candidate food name is found in the food library;

[0098] A first determination submodule is used to obtain a first query result, and when the first query result includes the name of the target food, output the dietary record information of the target food through the food library;

[0099] The second determination submodule is used to determine whether the basic food name is found from the food library;

[0100] an information output submodule, for obtaining a second query result, and when the second query result includes a basic food name, outputting the dietary record information of the target food through the food library;

[0101] The third determination submodule is used to determine that the query of the target food fails if no.

[0102] In some optional implementations, the dietary information query module 404 includes: an action return submodule, which is used to return to the step of determining whether the basic food name is queried from the food library when the first query result does not include the target food name.

[0103] In some optional implementations, the dietary record information recognition device based on food images in the embodiment of the present disclosure, the dietary information query module 404 includes: an information verification submodule, wherein the information verification submodule includes:

[0104] An acquiring unit, used for acquiring a preset food name corresponding to the first query result, where the preset food name is a food name preset in a food library;

[0105] The verification unit is used to input the preset food name and the target food name into the food name recognition network to obtain the name verification result, wherein the food name recognition network is a pre-trained neural network, and the name verification result includes: the preset food name and the target food name are consistent or the preset food name and the target food name are inconsistent.

[0106] In some optional implementations, the food name acquisition module 403 includes:

[0107] The original name acquisition submodule is used to obtain the original food name of the target food;

[0108] The basic name acquisition submodule is used to input the original food name into the basic name recognition network to obtain the basic food name corresponding to the original food name, wherein the basic name recognition network is a pre-trained neural network.

[0109] In some optional implementations, the food information acquisition module 402 includes:

[0110] The food information recognition submodule is used to input the image to be recognized into the food information recognition network to obtain the food information recognition result corresponding to the target food, wherein the food information recognition network is a pre-trained neural network, and the food information recognition result includes: the target food name and the alternative food names.

[0111] In some optional embodiments, the food information recognition result further includes: food weight and food position, diet record information, including: food calories, nutritional elements, food detection frame information, wherein the food calories are calculated by combining the food weight with the basic calorie unit, and the food detection frame information is calculated by combining the food position with a preset ratio, wherein the preset ratio is the ratio of the area where the target food is located to the image to be recognized;

[0112] In some optional implementations, it also includes: a diet information display module, which is used to display the diet record information of the target food based on the food detection frame information.

[0113] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0114] The dietary record information recognition device for food images in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0115] The embodiment of the present invention further provides a computer device having the above-mentioned device for recognizing dietary record information on food images.

[0116] See also Figure 5 , Figure 5 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 5As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 5 A processor 10 is taken as an example.

[0117] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0118] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0119] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0120] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0121] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 5 The example of connecting through bus is taken in the following.

[0122] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0123] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0124] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for identifying dietary record information based on food images, characterized in that: The method comprises: Acquire an image to be identified containing at least one food; Based on the image to be identified, obtaining food information of the target food, wherein the food information includes: a target food name and an alternative food name, the target food is any one of the at least one food, the alternative food name is a similar name to the target food, and the number of alternatives for the alternative food name is preset; Obtaining the basic food name of the target food; Based on the target food name, the candidate food name, and the basic food name, the dietary record information of the target food is queried through the food library.

2. The method according to claim 1, characterized in that Based on the target food name, the candidate food name, and the basic food name, querying the dietary record information of the target food through the food library includes: Determine whether the target food name or the candidate food name is found in the food library; If yes, obtain a first query result, and when the first query result includes the target food name, output the dietary record information of the target food through the food library; If not, determining whether the basic food name is found from the food library; If yes, obtain a second query result, and when the second query result includes the basic food name, output the dietary record information of the target food through the food library; If not, it is determined that the query of the target food has failed.

3. The method according to claim 2, characterized in that Also includes: When the first query result does not include the target food name, the process returns to the step of determining whether the basic food name is found in the query from the food library.

4. The method according to claim 2 or 3, characterized in that: Verifying whether the first query result includes the target food name includes: Obtain a preset food name corresponding to the first query result, where the preset food name is a food name preset in the food library; The preset food name and the target food name are input into a food name recognition network to obtain a name verification result, wherein the food name recognition network is a pre-trained neural network, and the name verification result includes: the preset food name and the target food name are consistent or the preset food name and the target food name are inconsistent.

5. The method according to claim 1, characterized in that: Get the basic food name of the target food, including: Get the original food name of the target food; The original food name is input into a basic name recognition network to obtain a basic food name corresponding to the original food name, wherein the basic name recognition network is a pre-trained neural network.

6. The method according to claim 1, characterized in that Based on the image to be identified, obtaining food information of the target food includes: The image to be identified is input into a food information identification network to obtain a food information identification result corresponding to the target food, wherein the food information identification network is a pre-trained neural network, and the food information identification result includes: the target food name and the alternative food names.

7. The method according to claim 6, characterized in that The food information recognition result also includes: food weight and food position, and the diet record information includes: food calories, nutritional elements and food detection frame information, wherein the food calories are calculated by combining the food weight with the basic calorie unit, and the food detection frame information is calculated by combining the food position with a preset ratio, wherein the preset ratio is the ratio of the area where the target food is located to the image to be identified; and also includes: based on the food detection frame information, displaying the diet record information of the target food.

8. A device for recognizing dietary record information based on food images, characterized in that: The device comprises: A food image acquisition module, used to acquire an image to be identified containing at least one food; a food information acquisition module, configured to acquire food information of a target food based on the image to be identified, wherein the food information includes: a target food name and an alternative food name, wherein the target food is any one of the at least one food, the alternative food name is a similar name to the target food, and the number of alternative food names is preset; A food name acquisition module, used to acquire the basic food name of the target food; The dietary information query module is used to query the dietary record information of the target food through the food library based on the target food name or the candidate food name or the basic food name.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method for recognizing dietary record information based on food images as described in any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method for identifying dietary record information based on food images according to any one of claims 1 to 7.

Images