Food display method and device, cooking equipment and storage medium

Abstract

The invention relates to the technical field of cooking equipment, and provides a food display method and device, cooking equipment and a storage medium. The method comprises the steps that an image recognition result of a food image is acquired, the image recognition result comprises at least two similarity results, and any similarity result is used for representing the similarity between food to be displayed and any preset food; all similarity results in the image recognition results are screened, a target similarity result set is obtained, and the sum of all the similarity results in the target similarity result set is larger than or equal to a preset similarity threshold value; and displaying food information corresponding to each similarity result in the target similarity result set. According to the invention, the display accuracy can be improved, and the user experience is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of cooking equipment, and in particular to a food display method, device, cooking equipment and storage medium. Background Art

[0002] With the rapid development of smart home appliances and the improvement of people's living standards, people have higher and higher requirements for cooking equipment, so most existing cooking equipment has a food recognition function. The food recognition function is used to recognize the food in the cooking equipment, and in order to improve the user experience, the recognition result needs to be displayed.

[0003] At present, a food corresponding to the food recognition result is displayed on the display screen of the cooking device, so that the user can view the food currently being cooked by the cooking device. However, due to certain recognition errors in food recognition, only one food is displayed, and the displayed food may not be the food currently being cooked by the cooking device. That is, the current food display method has a high error display rate, which affects the user experience. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, the present invention proposes a food display method, which screens each similarity result in the image recognition result of the food image to obtain a target similarity result set, and can display more food information corresponding to the similarity results for the user to select and determine, thereby improving the display accuracy and enhancing the user experience.

[0005] The invention also provides a food display device.

[0006] The invention also provides a cooking device.

[0007] The present invention also provides a non-transitory computer-readable storage medium.

[0008] According to a first aspect of the present invention, a food display method includes:

[0009] Acquire an image recognition result of a food image, wherein the image recognition result includes at least two similarity results, and any of the similarity results is used to characterize the similarity between the food to be displayed and any preset food;

[0010] Screening each similarity result in the image recognition result to obtain a target similarity result set, wherein the sum of each similarity result in the target similarity result set is greater than or equal to a preset similarity threshold;

[0011] Display food information corresponding to each similarity result in the target similarity result set.

[0012] According to the food display method of an embodiment of the present invention, each similarity result in the image recognition result of the food image is screened to obtain a target similarity result set, and then the food information corresponding to each similarity result in the target similarity result set is displayed, so that more food information corresponding to the similarity results can be displayed for the user to select and determine, thereby improving the display accuracy and enhancing the user experience. At the same time, the screening process ensures that the sum of each similarity result in the target similarity result set is greater than or equal to the preset similarity threshold, thereby improving the display accuracy and enhancing the user experience. In summary, the present invention improves the accuracy of food display and improves the user experience.

[0013] According to an embodiment of the present invention, the target similarity result set includes the maximum similarity result among the image recognition results.

[0014] According to an embodiment of the present invention, the screening of the similarity results in the image recognition results to obtain a target similarity result set includes:

[0015] Sorting the similarity results in the image recognition results to obtain a sorted similarity result set;

[0016] The sorted similarity result sets are screened one by one in descending order until the sum of the similarity results in the screened similarity result sets is greater than or equal to the preset similarity threshold;

[0017] Based on the similarity result set after the last screening round, a target similarity result set is determined.

[0018] According to an embodiment of the present invention, determining a target similarity result set based on the similarity result set after the last screening round includes:

[0019] Determining that the number of each similarity result in the similarity result set after the last screening round is less than a preset similarity number, determining that the recognition is successful, and determining the similarity result set after the last screening round as the target similarity result set;

[0020] Determine that the number of each similarity result in the similarity result set after the last screening round is greater than or equal to the preset similarity number, and return to the step of obtaining the image recognition result of the food image until the recognition is successful.

[0021] According to an embodiment of the present invention, displaying the food information corresponding to each similarity result in the target similarity result set includes:

[0022] Based on the order of the similarity results in the target similarity result set, the food information corresponding to each similarity result in the target similarity result set is displayed in order from large to small.

[0023] According to an embodiment of the present invention, the step of obtaining an image recognition result of a food image includes:

[0024] Acquire the food image;

[0025] Performing image recognition on the food image to obtain the image recognition result; or,

[0026] The food image is sent to a server so that the server can perform image recognition on the food image to obtain an image recognition result, and the image recognition result sent by the server is received.

[0027] According to an embodiment of the present invention, the obtaining of the food image also includes:

[0028] Determining that a door of the cooking device is closed, and controlling an image acquisition device of the cooking device to operate so as to acquire the food image; or,

[0029] An image recognition start instruction is obtained to control the image acquisition device of the cooking device to operate so as to acquire the food image.

[0030] According to an embodiment of the present invention, the displaying of food information corresponding to each similarity result in the target similarity result set further includes:

[0031] A user operation instruction is obtained, and a selected target food is determined based on the user operation instruction.

[0032] A food display device according to a second aspect of the present invention includes:

[0033] An acquisition module, used for acquiring an image recognition result of a food image, wherein the image recognition result includes at least two similarity results, and any of the similarity results is used to characterize the similarity between the food to be displayed and any preset food;

[0034] A screening module, used for screening each similarity result in the image recognition result to obtain a target similarity result set, wherein the sum of each similarity result in the target similarity result set is greater than or equal to a preset similarity threshold;

[0035] The display module is used to display the food information corresponding to each similarity result in the target similarity result set.

[0036] According to an embodiment of the third aspect of the present invention, the cooking device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any of the above-mentioned food display methods is implemented.

[0037] According to the non-transitory computer-readable storage medium of the fourth aspect of the present invention, a computer program is stored thereon, and when the computer program is executed by a processor, any of the food display methods described above is implemented.

[0038] The above one or more technical solutions in the embodiments of the present invention have at least one of the following technical effects:

[0039] The similarity results in the image recognition results of the food images are screened to obtain a target similarity result set, and then the food information corresponding to each similarity result in the target similarity result set is displayed, so that more food information corresponding to the similarity results can be displayed for the user to select and determine, thereby improving the display accuracy and enhancing the user experience. At the same time, the screening process ensures that the sum of each similarity result in the target similarity result set is greater than or equal to the preset similarity threshold, thereby improving the display accuracy and enhancing the user experience. In summary, the present invention improves the accuracy of food display and improves the user experience.

[0040] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0042] Figure 1 This is one of the flow charts of the food display method provided by the embodiment of the present invention;

[0043] Figure 2 This is the second flow chart of the food display method provided by the embodiment of the present invention;

[0044] Figure 3 is a schematic diagram of a food display interface provided by an embodiment of the present invention;

[0045] Figure 4 is a structural schematic diagram of a food display device provided by an embodiment of the present invention;

[0046] Figure 5 It is a schematic diagram of the structure of a cooking device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the embodiments of the present application. In addition, the terms "first", "second", and "third" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradicting each other.

[0049] With the rapid development of smart home appliances and the improvement of people's living standards, people have higher and higher requirements for cooking equipment. Therefore, most existing cooking equipment has a food recognition function. The food recognition function is used to identify the food in the cooking equipment, and most users need to view the food currently being cooked by the cooking equipment. Therefore, in order to improve the user experience, the recognition results need to be displayed.

[0050] At present, a food corresponding to the food recognition result is displayed on the display screen of the cooking device so that the user can view the food currently being cooked by the cooking device. However, due to certain recognition errors in food recognition, if only one food is displayed, the displayed food may not be the food currently being cooked by the cooking device, that is, the current food display method has a high error display rate, which affects the user experience. For example, if the food currently being cooked by the cooking device is corn, and the food recognition result is "51% similar to potatoes, 47% similar to corn, 2% similar to eggs", then it displays potatoes instead of corn, which leads to display errors.

[0051] Based on the above problems, the present invention proposes the following embodiments. The following embodiments of the present application are further described in detail in conjunction with the accompanying drawings and embodiments. The following embodiments are used to illustrate the present application, but cannot be used to limit the scope of the present application.

[0052] Figure 1is one of the flow charts of the food display method provided by the embodiment of the present invention, such as Figure 1 As shown, the food display method includes steps 110 to 130.

[0053] The execution subject of the food display method provided in the embodiment of the present invention may be a cooking device, and further, the execution subject may be a controller (processor) of the cooking device.

[0054] Step 110, obtaining image recognition results of food images.

[0055] The food image is an image of food to be displayed in the cooking device, and the image recognition result includes at least two similarity results, and any of the similarity results is used to characterize the similarity between the food to be displayed and any preset food.

[0056] Here, the food image is acquired by an image acquisition device in the cooking device. The image acquisition device is deployed inside the cooking device, for example, the image acquisition device is deployed above the position for placing food in the cooking device, so as to better acquire the image of the food to be displayed in the cooking device. The embodiment of the present invention does not specifically limit the deployment position of the image acquisition device. The image acquisition device may include but is not limited to: a camera, a video camera, a video camera, an image acquisition sensor, etc., and the embodiment of the present invention does not specifically limit this.

[0057] Here, the food to be displayed may be food to be cooked, and the food to be displayed is placed at a position in the cooking device for placing food.

[0058] Here, the similarity result can be directly the similarity between the food to be displayed and the preset food, or can be obtained by further processing the similarity. The preset food is a pre-set food, and further, the preset food can be set as a culinary food.

[0059] Exemplarily, there are 10 preset foods, and the image recognition results may include 10 similarity results, for example, the similarity with the first food is 60%, the similarity with the second food is 30%, the similarity with the third food is 10%, the similarity with the fourth food is 5%, the similarity with the fifth food is 3%, the similarity with the sixth food is 1%, the similarity with the seventh food is 0.8%, the similarity with the eighth food is 0.1%, the similarity with the ninth food is 0.06%, and the similarity with the tenth food is 0.03%.

[0060] In one embodiment, the food image is displayed. Specifically, the food image is displayed on a display screen of the cooking device so that the user can know the food in the cooking device after viewing it.

[0061] Step 120 , screening each similarity result in the image recognition result to obtain a target similarity result set.

[0062] The target similarity result set includes at least one similarity result, and the sum of the similarity results in the target similarity result set is greater than or equal to a preset similarity threshold.

[0063] Here, the preset similarity threshold is a pre-set threshold, which can be set according to actual needs, such as 80%, and the embodiment of the present invention does not specifically limit this. For example, the preset similarity threshold can be set according to the recognition accuracy requirement for the image recognition result, or the preset similarity threshold can be set according to the accuracy of the image recognition algorithm corresponding to the image recognition result. Of course, other factors can also be considered, which will not be repeated here one by one.

[0064] Specifically, based on a preset similarity threshold, each similarity result in the image recognition result is screened to obtain a target similarity result set.

[0065] Exemplarily, the image recognition result may include 10 similarity results, and the preset similarity threshold may be 80%. For example, the similarity with the first food is 75%, the similarity with the second food is 60%, the similarity with the third food is 25%, the similarity with the fourth food is 5%, the similarity with the fifth food is 3%, the similarity with the sixth food is 1%, the similarity with the seventh food is 0.8%, the similarity with the eighth food is 0.1%, the similarity with the ninth food is 0.06%, and the similarity with the tenth food is 0.03%. Based on this, the target similarity result The target similarity result set may include: 75% similarity with the first food, 60% similarity with the second food; the target similarity result set may also include: 75% similarity with the first food, 25% similarity with the third food; the target similarity result set may include: 75% similarity with the first food, 60% similarity with the second food, 25% similarity with the third food; the target similarity result set may also include: 60% similarity with the second food, 25% similarity with the third food, 5% similarity with the fourth food; I will not go into details here.

[0066] In some embodiments, based on a preset similarity threshold, each similarity result in the image recognition result is screened one by one until the sum of each similarity result in the screened similarity result set is greater than or equal to the preset similarity threshold; based on the similarity result set after the last screening round, a target similarity result set is determined.

[0067] Exemplarily, the image recognition result may include 10 similarity results, the preset similarity threshold may be 80%, and the image recognition results include: 65% similarity to the first food, 60% similarity to the second food, 10% similarity to the third food, 10% similarity to the fourth food, 8% similarity to the fifth food, 6% similarity to the sixth food, 0.8% similarity to the seventh food, 0.1% similarity to the eighth food, 0.06% similarity to the ninth food, and 0.03% similarity to the tenth food. Then the similarity result set after the final screening round may include: similarity to the first food, similarity to the ninth food, similarity to the tenth food, similarity to the fifth food, similarity to the sixth food, similarity to the seventh food, similarity to the eighth food, similarity to the ninth food, and similarity to the tenth food. The similarity is 65% with the second food and 60% with the second food; the similarity result set after the final screening round may also include: a similarity of 65% with the first food, a similarity of 10% with the third food, and a similarity of 10% with the fourth food; the similarity result set after the final screening round may also include: a similarity of 60% with the second food, a similarity of 10% with the third food, and a similarity of 10% with the fourth food; the similarity result set after the final screening round may also include: a similarity of 65% with the first food, a similarity of 10% with the third food, and a similarity of 6% with the sixth food; they will not be repeated here.

[0068] The specific execution process of determining the target similarity result set based on the similarity result set after the last screening round screening can refer to the following embodiments, which will not be described in detail here.

[0069] In other embodiments, the similarity results in the image recognition results are sorted to obtain a sorted similarity result set; based on a preset similarity threshold, the sorted similarity result set is filtered in descending order; based on the similarity result set after the last filtering round, a target similarity result set is determined.

[0070] Exemplarily, the image recognition result may include 10 similarity results, the preset similarity threshold may be 80%, and the sorted similarity result set includes: 65% similarity to the first food, 10% similarity to the second food, 10% similarity to the third food, 10% similarity to the fourth food, 8% similarity to the fifth food, 6% similarity to the sixth food, 0.8% similarity to the seventh food, 0.1% similarity to the eighth food, 0.06% similarity to the ninth food, and 0.03% similarity to the tenth food. Then the similarity result set after the final screening round may include: The similarity with the first food is 65%, the similarity with the second food is 10%, and the similarity with the third food is 10%; the similarity result set after the final screening round may include: the similarity with the first food is 65%, the similarity with the second food is 10%, the similarity with the third food is 10%, and the similarity with the fourth food is 10%; the similarity result set after the final screening round may include: the similarity with the first food is 65%, the similarity with the second food is 10%, the similarity with the third food is 10%, the similarity with the fourth food is 10%, and the similarity with the fifth food is 8%; they will not be repeated here.

[0071] The specific execution process of determining the target similarity result set based on the similarity result set after the last screening round screening can refer to the following embodiments, which will not be described in detail here.

[0072] In other embodiments, it is determined that the number of similarity results in the similarity result set after the last screening round is less than the preset similarity number, the recognition is determined to be successful, and the similarity result set after the last screening round is determined as the target similarity result set; it is determined that the number of similarity results in the similarity result set after the last screening round is greater than or equal to the preset similarity number, and the step of obtaining the image recognition result of the food image is returned until the recognition is successful. Further, it is determined that the number of similarity results in the similarity result set after the last screening round is greater than or equal to the preset similarity number, and the recognition fails, and the step of obtaining the image recognition result of the food image is returned until the recognition is successful, that is, the image recognition function is restarted.

[0073] Step 130 , displaying food information corresponding to each similarity result in the target similarity result set.

[0074] Here, the food information may include but is not limited to at least one of the following: food name, food picture, recipe name, recipe, recipe picture, menu, food calories, food nutritional ingredients, cooking time required, etc.

[0075] The food information corresponding to each similarity result can be displayed in random order, or in descending order, or in order of cooking frequency, or in order of user's preference for the food. Of course, other arrangement methods can also be used for display, which will not be described here one by one.

[0076] In one embodiment, the target food corresponding to each similarity result is obtained, the cooking frequency of each target food is determined, and based on the order of each cooking frequency, the food information corresponding to each similarity result in the target similarity result set is displayed in order from large to small. It can be understood that the display accuracy can be further improved by arranging the display with reference to the cooking frequency.

[0077] In another embodiment, the target food corresponding to each similarity result is obtained, the user's preference for each target food is determined, and based on the order of the user's preference, the food information corresponding to each similarity result in the target similarity result set is displayed in order from large to small. It can be understood that arranging the display with reference to the user's preference can further improve the display accuracy and improve the user experience.

[0078] In a specific embodiment, the food information corresponding to each similarity result in the target similarity result set is displayed on a display screen of the cooking device. The display screen can be arranged on the outer surface of the cooking device for the convenience of the user to view, and the embodiment of the present invention does not specifically limit its deployment position.

[0079] In one embodiment, if the food information corresponding to each similarity result cannot be fully displayed in one interface, the food information corresponding to each similarity result may be displayed in the form of a scroll bar.

[0080] In another embodiment, if the food information corresponding to each similarity result cannot be fully displayed in one interface, the food information corresponding to each similarity result may be displayed in multiple pages.

[0081] The food display method provided in the embodiment of the present invention screens each similarity result in the image recognition result of the food image to obtain a target similarity result set, and then displays the food information corresponding to each similarity result in the target similarity result set, so that more food information corresponding to the similarity results can be displayed for the user to select and determine, thereby improving the display accuracy and enhancing the user experience. At the same time, the screening process ensures that the sum of each similarity result in the target similarity result set is greater than or equal to the preset similarity threshold, thereby improving the display accuracy and enhancing the user experience. In summary, the embodiment of the present invention improves the accuracy of food display and improves the user experience.

[0082] Based on any of the above embodiments, in the method, the target similarity result set includes the maximum similarity result among the image recognition results.

[0083] Specifically, each similarity result in the image recognition result is screened to obtain the maximum similarity result in the image recognition result; based on a preset similarity threshold, each similarity result in the image recognition result except the maximum similarity result is screened to obtain at least one similarity result, so that the sum of the at least one similarity result and the maximum similarity result is greater than or equal to the preset similarity threshold; based on the at least one similarity result and the maximum similarity result, a target similarity result set is determined, so that the target similarity result set includes the at least one similarity result and the maximum similarity result.

[0084] Exemplarily, the image recognition result may include 10 similarity results, and the preset similarity threshold may be 80%. For example, the similarity with the first food is 55%, the similarity with the second food is 30%, the similarity with the third food is 25%, the similarity with the fourth food is 5%, the similarity with the fifth food is 3%, the similarity with the sixth food is 1%, the similarity with the seventh food is 0.8%, the similarity with the eighth food is 0.1%, the similarity with the ninth food is 0.06%, and the similarity with the tenth food is 0.03%. Based on this, the target similarity result The target similarity result set may include: 55% similarity with the first food, 30% similarity with the second food; the target similarity result set may also include: 55% similarity with the first food, 25% similarity with the third food; the target similarity result set may include: 55% similarity with the first food, 30% similarity with the second food, 25% similarity with the third food; the target similarity result set may also include: 55% similarity with the first food, 25% similarity with the third food, 5% similarity with the fourth food; I will not go into details here.

[0085] The food display method provided by the embodiment of the present invention ensures that the target similarity result set includes the maximum similarity result in the image recognition results during the screening process of the image recognition results, thereby further improving the display accuracy and further enhancing the user experience.

[0086] Based on any of the above embodiments, Figure 2 FIG. 2 is a second flow chart of the food display method provided by an embodiment of the present invention. Figure 2 As shown, the above step 120 includes:

[0087] Step 121 , sorting the similarity results in the image recognition results to obtain a sorted similarity result set.

[0088] Specifically, the similarity results in the image recognition results are sorted in descending order of similarity to obtain a sorted similarity result set.

[0089] Step 122 , successively screening the sorted similarity result set in descending order until the sum of the similarity results in the screened similarity result set is greater than or equal to the preset similarity threshold.

[0090] Specifically, each similarity result in the image recognition result is screened to obtain the maximum similarity result in the image recognition result, and the maximum similarity result is compared with a preset similarity threshold. If the maximum similarity result is greater than or equal to the preset similarity threshold, the similarity result set after the final screening round only includes the maximum similarity result; if the maximum similarity result is less than the preset similarity threshold, based on the preset similarity threshold, each similarity result in the image recognition result except the maximum similarity result is screened in order from large to small to obtain at least one similarity result, so that the sum of the at least one similarity result and the maximum similarity result is greater than or equal to the preset similarity threshold, and the similarity result set after the final screening round includes the maximum similarity result and the at least one similarity result.

[0091] Exemplarily, the image recognition results may include 10 similarity results, the preset similarity threshold may be 80%, and the sorted similarity result set includes: 65% similarity to the first food, 10% similarity to the second food, 10% similarity to the third food, 10% similarity to the fourth food, 8% similarity to the fifth food, 6% similarity to the sixth food, 0.8% similarity to the seventh food, 0.1% similarity to the eighth food, 0.06% similarity to the ninth food, and 0.03% similarity to the tenth food. Then the similarity result set after the final screening round includes: 65% similarity to the first food, 10% similarity to the second food, and 10% similarity to the third food.

[0092] Step 123: Determine a target similarity result set based on the similarity result set after the last screening round.

[0093] Here, the last screening round is the last round of successive screening, the sum of each similarity result in the similarity result set after screening in the last screening round is greater than or equal to the preset similarity threshold, and the sum of each similarity result in the similarity result set after screening in the previous screening round of the last screening round is less than the preset similarity threshold.

[0094] In one embodiment, the similarity result set after the last screening round is directly determined as the target similarity result set.

[0095] In another embodiment, it is determined that the number of each similarity result in the similarity result set after the last screening round is less than the preset similarity number, and the similarity result set after the last screening round is directly determined as the target similarity result set; it is determined that the number of each similarity result in the similarity result set after the last screening round is greater than or equal to the preset similarity number, and the process returns to step 110 above until the number of each similarity result in the similarity result set after the last screening round is less than the preset similarity number.

[0096] The food display method provided by the embodiment of the present invention sorts the similarity results in the image recognition results to obtain a sorted similarity result set, and based on a preset similarity threshold, successively filters the sorted similarity result set in order from large to small, so that the similarity results included in the target similarity result set are all large values, thereby improving the display accuracy and enhancing the user experience; and the screening process is stopped until the sum of the similarity results in the screened similarity result set is greater than or equal to the preset similarity threshold, thereby reducing the number of similarity results included in the target similarity result set, thereby reducing the amount of food information displayed, preventing the display of redundant information, and ultimately improving the user experience.

[0097] Based on any of the above embodiments, in the method, the above step 123 includes:

[0098] Determining that the number of each similarity result in the similarity result set after the last screening round is less than a preset similarity number, determining that the recognition is successful, and determining the similarity result set after the last screening round as the target similarity result set;

[0099] Determine that the number of each similarity result in the similarity result set after the last screening round is greater than or equal to the preset similarity number, and return to the step of obtaining the image recognition result of the food image until the recognition is successful.

[0100] Specifically, it is determined that the number of each similarity result in the similarity result set after the last screening round is greater than or equal to the preset similarity number, and the process returns to the above step 110, i.e., restarts the image recognition function until the recognition is successful, i.e., until the number of each similarity result in the similarity result set after the last screening round is less than the preset similarity number. Further, it is determined that the number of each similarity result in the similarity result set after the last screening round is greater than or equal to the preset similarity number, and the recognition is determined to have failed.

[0101] Here, the preset number of similarities can be set according to actual needs, for example, 4. The preset number of similarities can be determined according to the number that can be displayed on the display screen, or determined in consideration of factors such as interface layout, and the embodiment of the present invention does not specifically limit this.

[0102] Exemplarily, if the preset similarity threshold is 80%, the sorted similarity result set includes T0P1, T0P2, T0P3, T0P4, T0P5, T0P1>T0P2>T0P3>T0P4>T0P5, then when T0P1>=80%, the food information corresponding to TOP1 is displayed; when T0P1<80%, T0P1+T0P2>=80%, the food information corresponding to TOP1 and the food information corresponding to TOP2 are displayed; when T0P1+T0P2<80%, T0P1+T0P2+T0P3>=80%, the food information corresponding to TOP1 and the food information corresponding to TOP2, as well as the food information corresponding to TOP3 are displayed.

[0103] The food display method provided by the embodiment of the present invention determines that the number of similarity results in the similarity result set after the last screening round is less than the preset similarity number, and then determines the similarity result set after the last screening round as the target similarity result set, thereby reducing the number of similarity results included in the target similarity result set, and then reducing the amount of food information displayed, preventing too much information from being displayed to the user, and ultimately improving the user experience.

[0104] Based on any of the above embodiments, in the method, the above step 130 includes:

[0105] Based on the order of the similarity results in the target similarity result set, the food information corresponding to each similarity result in the target similarity result set is displayed in order from large to small.

[0106] Specifically, based on the order of each similarity result in the target similarity result set, the food information corresponding to each similarity result in the target similarity result set is displayed in order from large to small similarity.

[0107] It should be noted that if the target similarity result set has been sorted in descending order, there is no need to sort the target similarity result set; if the target similarity result set is not sorted in descending order, the target similarity result set is sorted to obtain a sorted target similarity result set, and the food information corresponding to each similarity result in the sorted target similarity result set is displayed in descending order of similarity.

[0108] For ease of understanding, Figure 3 is a schematic diagram of a food display interface provided by an embodiment of the present invention, such as Figure 3 As shown, Figure 3 The food at the top is the food corresponding to the largest similarity result in the target similarity result set. Figure 3 The food in the middle is the food corresponding to the second largest similarity result in the target similarity result set. Figure 3The food at the bottom is the food corresponding to the smallest similarity result in the target similarity result set.

[0109] The food display method provided by an embodiment of the present invention displays the food information corresponding to each similarity result in the target similarity result set in order from large to small based on the order of each similarity result in the target similarity result set, so that the user can first view the food information with higher similarity, thereby improving the display accuracy, and allowing the user to quickly select the corresponding food, ultimately improving the user experience.

[0110] Based on any of the above embodiments, the above step 110 includes:

[0111] Acquire the food image;

[0112] Perform image recognition on the food image to obtain the image recognition result.

[0113] In one embodiment, the food image is compared with each preset image in a preset image set to obtain an image recognition result. The preset image set includes at least two preset images, and the preset image set can be stored in a database and can be determined by experiments.

[0114] In another embodiment, the food image is input into an image recognition model to obtain an image recognition result output by the image recognition model. The image recognition model is trained based on the sample food image and the image recognition result label corresponding to the sample food image.

[0115] The execution subject of the image recognition may be a cooking device, and further, it may be a controller (processor) of the cooking device.

[0116] The food display method provided by the embodiment of the present invention performs image recognition on food images on a local device without sending the food images or receiving image recognition results, thereby reducing the waste of network resources and improving the efficiency of image recognition.

[0117] Alternatively, the above step 110 includes:

[0118] Acquire the food image;

[0119] The food image is sent to a server so that the server can perform image recognition on the food image to obtain an image recognition result, and the image recognition result sent by the server is received.

[0120] The execution subject of the image recognition is a server, and the specific execution process of the image recognition can refer to the above embodiment. The server is a server corresponding to the cooking device.

[0121] In one embodiment, the cooking device includes a WIFI wireless communication module, which is used to enable the cooking device to transmit data to the server according to a specific communication protocol and a specific data encryption mode. Furthermore, the WIFI wireless communication module is also used to enable the cooking device to transmit data to the user terminal according to a specific communication protocol and a specific data encryption mode, so that the food information corresponding to each similarity result is displayed on the APP (Application) of the user terminal, thereby improving the user experience.

[0122] The food display method provided by the embodiment of the present invention performs image recognition on food images on a server, thereby reducing the use of computing resources of a local device.

[0123] Based on any of the above embodiments, in the method, before acquiring the food image, the method further includes:

[0124] It is determined that the door of the cooking device is closed, and an image acquisition device of the cooking device is controlled to operate so as to acquire the food image.

[0125] Here, whether the door is closed may be determined by a sensor disposed on the cooking device, or may be determined by a detection circuit disposed on the cooking device, and the embodiment of the present invention does not specifically limit this.

[0126] Here, the image acquisition device is deployed inside the cooking device. For example, the image acquisition device is deployed above the position for placing food in the cooking device to better acquire the image of the food to be displayed in the cooking device. The embodiment of the present invention does not specifically limit the deployment position of the image acquisition device. The image acquisition device may include but is not limited to: a camera, a video camera, a video camera, an image acquisition sensor, etc., which is not specifically limited in the embodiment of the present invention. Furthermore, the image acquisition device is connected to the controller of the cooking device so that the image acquisition device sends the food image it acquires to the controller.

[0127] In one embodiment, while determining that the door of the cooking device is closed, the image acquisition device of the cooking device is controlled to operate.

[0128] In another embodiment, when it is determined that the door of the cooking device is closed and it is determined that the door of the cooking device is open within a preset time period before the current moment, the image acquisition device of the cooking device is controlled to operate. For example, the user opens the door of the cooking device, triggers the image recognition function and is in a ready state, the user puts in the food to be displayed and closes the door of the cooking device, and the image acquisition device of the cooking device is controlled to operate, that is, the image acquisition device is started to capture images of the food in the cavity of the cooking device.

[0129] It should be noted that the image recognition function can only be turned on after the cooking device is powered on and operating normally.

[0130] The food display method provided by the embodiment of the present invention determines that the door of the cooking device is closed before controlling the image acquisition device of the cooking device to operate, thereby preventing the image acquisition device from operating all the time, thereby reducing the waste of electricity.

[0131] Based on any of the above embodiments, in the method, before acquiring the food image, the method further includes:

[0132] An image recognition start instruction is obtained to control the image acquisition device of the cooking device to operate so as to acquire the food image.

[0133] Here, the image recognition start instruction is triggered by the user, and it can be triggered through the display screen of the cooking device, or through a button of the cooking device, or through a voice recognition module on the cooking device, that is, to realize the voice control function, or through a motion recognition module on the cooking device, that is, to realize the gesture control function. The embodiment of the present invention does not make specific limitations on this.

[0134] It should be noted that the image recognition function can only be turned on after the cooking device is powered on and operating normally.

[0135] The food display method provided by the embodiment of the present invention obtains the image recognition start instruction and then controls the image acquisition device of the cooking equipment to run, thereby preventing the image acquisition device from running all the time, thereby reducing the waste of electricity.

[0136] Based on any of the above embodiments, in the method, before acquiring the food image, the method further includes:

[0137] Obtain weight detection data, determine that the weight detection data is greater than a preset weight threshold, and control the image acquisition device of the cooking device to operate so as to acquire the food image.

[0138] Here, the weight detection data is obtained by detecting a weight sensor, and the weight sensor may be disposed at a location where the food is placed.

[0139] The food display method provided by the embodiment of the present invention determines that the weight detection data is greater than a preset weight threshold, and then controls the image acquisition device of the cooking equipment to run, thereby preventing the image acquisition device from running all the time, thereby reducing the waste of electricity.

[0140] Based on any of the above embodiments, in the method, before acquiring the food image, the method further includes:

[0141] It is determined that the cooking device starts cooking, and an image acquisition device of the cooking device is controlled to operate so as to acquire the food image.

[0142] The food display method provided by the embodiment of the present invention determines that the cooking device starts cooking, and then controls the image acquisition device of the cooking device to run, thereby preventing the image acquisition device from running all the time, thereby reducing the waste of electricity.

[0143] Based on any of the above embodiments, after the above step 130, the method further includes:

[0144] A user operation instruction is obtained, and a selected target food is determined based on the user operation instruction.

[0145] Here, the user operation instruction is triggered by the user, which can be triggered through the display screen of the cooking device.

[0146] It is understandable that the user clicks on the food information corresponding to any similarity result, thereby triggering the user operation instruction, and the user operation instruction indicates that there is a target food selected by the user, that is, indicates that there is a target food corresponding to any similarity result.

[0147] In one embodiment, if the user clicks on a food name, the target food corresponding to the food name is determined based on the user's operation instruction. Further, the menu corresponding to the target food, the food calories, the food's nutritional ingredients, the cooking time required for cooking, etc. are displayed.

[0148] In another embodiment, if the user clicks on a food image, the target food corresponding to the food image is determined based on the user's operation instruction. Further, the menu corresponding to the target food, the food calories, the food's nutritional ingredients, the cooking time required for cooking, etc. are displayed.

[0149] In another embodiment, if the user clicks on a recipe name, the target food corresponding to the recipe name is determined based on the user's operation instruction, and the recipe corresponding to the recipe name is displayed.

[0150] In another embodiment, if the user clicks on a recipe image, the target food corresponding to the recipe image is determined based on the user's operation instruction, and the recipe corresponding to the recipe image is displayed.

[0151] In one embodiment, food images are displayed so that the user can view the food in the cooking device and select the corresponding food, thereby ultimately improving the accuracy of food recognition.

[0152] The food display method provided by the embodiment of the present invention obtains user operation instructions, and determines the selected target food based on the user operation instructions, so as to provide the user with subsequent operations after the food is displayed, thereby improving the user experience.

[0153] Based on the above embodiments, the present invention can effectively solve the problem of high misrecognition rate of image recognition, significantly improve the recognition accuracy of food recognition, and improve the user experience of cooking equipment. In other words, when the image recognition result is not ideal, more recognition results can be displayed to the user, which improves the recognition rate of image recognition and improves the user experience.

[0154] The food display device provided by the present invention is described below. The food display device described below and the food display method described above can be referred to each other.

[0155] Figure 4 is a schematic diagram of the structure of a food display device provided by an embodiment of the present invention. Figure 4 As shown, the food display device comprises:

[0156] An acquisition module 410 is used to acquire an image recognition result of a food image, wherein the image recognition result includes at least two similarity results, and any of the similarity results is used to characterize the similarity between the food to be displayed and any preset food;

[0157] A screening module 420 is used to screen the similarity results in the image recognition results to obtain a target similarity result set, wherein the sum of the similarity results in the target similarity result set is greater than or equal to a preset similarity threshold;

[0158] The display module 430 is used to display the food information corresponding to each similarity result in the target similarity result set.

[0159] The food display device provided by the embodiment of the present invention screens each similarity result in the image recognition result of the food image to obtain a target similarity result set, and then displays the food information corresponding to each similarity result in the target similarity result set, so that more food information corresponding to the similarity results can be displayed for the user to select and determine, thereby improving the display accuracy and enhancing the user experience. At the same time, the screening process ensures that the sum of each similarity result in the target similarity result set is greater than or equal to the preset similarity threshold, thereby improving the display accuracy and enhancing the user experience. In summary, the embodiment of the present invention improves the accuracy of food display and improves the user experience.

[0160] Based on any of the above embodiments, the target similarity result set includes the maximum similarity result among the image recognition results.

[0161] Based on any of the above embodiments, the screening module 420 includes:

[0162] A result sorting unit, used to sort the similarity results in the image recognition results to obtain a sorted similarity result set;

[0163] A result screening unit, used for screening the sorted similarity result set one by one in descending order until the sum of the similarity results in the screened similarity result set is greater than or equal to the preset similarity threshold;

[0164] The result determination unit is used to determine a target similarity result set based on the similarity result set after the last screening round.

[0165] Based on any of the above embodiments, the result determination unit is further used for:

[0166] Determining that the number of each similarity result in the similarity result set after the last screening round is less than a preset similarity number, determining that the recognition is successful, and determining the similarity result set after the last screening round as the target similarity result set;

[0167] Determine that the number of each similarity result in the similarity result set after the last screening round is greater than or equal to the preset similarity number, and return to the step of obtaining the image recognition result of the food image until the recognition is successful.

[0168] Based on any of the above embodiments, the display module 430 is further used for:

[0169] Based on the order of the similarity results in the target similarity result set, the food information corresponding to each similarity result in the target similarity result set is displayed in order from large to small.

[0170] Based on any of the above embodiments, the acquisition module 410 includes:

[0171] An image acquisition unit, used for acquiring the food image;

[0172] an image recognition unit, configured to perform image recognition on the food image to obtain the image recognition result; or,

[0173] The image transceiver unit is used to send the food image to the server so that the server can perform image recognition on the food image to obtain an image recognition result, and receive the image recognition result sent by the server.

[0174] Based on any of the above embodiments, the device further includes:

[0175] a device control module, configured to determine that a door of the cooking device is closed, and control the image acquisition device of the cooking device to operate so as to acquire the food image; or

[0176] The device control module is also used to obtain an image recognition start instruction and control the image acquisition device of the cooking device to operate so as to acquire the food image.

[0177] Based on any of the above embodiments, the device further includes:

[0178] The instruction acquisition module is used to acquire user operation instructions and determine the selected target food based on the user operation instructions.

[0179] Figure 5 An example of a physical structure diagram of a cooking device is shown in FIG. Figure 5 As shown, the cooking device may include: a processor 510, a communication interface 520, a memory 530 and a communication bus 540, wherein the processor 510, the communication interface 520 and the memory 530 communicate with each other through the communication bus 540. The processor 510 may call the logic instructions in the memory 530 to execute the following method: obtaining an image recognition result of a food image, wherein the image recognition result includes at least two similarity results, and any of the similarity results is used to characterize the similarity between the food to be displayed and any preset food; screening each similarity result in the image recognition result to obtain a target similarity result set, wherein the sum of each similarity result in the target similarity result set is greater than or equal to a preset similarity threshold; and displaying the food information corresponding to each similarity result in the target similarity result set.

[0180] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the relevant technology or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0181] On the other hand, an embodiment of the present invention discloses a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions, and when the program instructions are executed by a computer, the computer can execute the methods provided by the above-mentioned method embodiments, for example including: obtaining an image recognition result of a food image, the image recognition result including at least two similarity results, any of the similarity results being used to characterize the similarity between the food to be displayed and any preset food; screening each similarity result in the image recognition result to obtain a target similarity result set, the sum of each similarity result in the target similarity result set being greater than or equal to a preset similarity threshold; and displaying the food information corresponding to each similarity result in the target similarity result set.

[0182] On the other hand, an embodiment of the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the food display method provided by the above-mentioned embodiments, for example, including: obtaining an image recognition result of a food image, the image recognition result including at least two similarity results, any of the similarity results being used to characterize the similarity between the food to be displayed and any preset food; screening each similarity result in the image recognition result to obtain a target similarity result set, the sum of each similarity result in the target similarity result set being greater than or equal to a preset similarity threshold; and displaying the food information corresponding to each similarity result in the target similarity result set.

[0183] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.

[0184] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or some parts of the embodiment.

[0185] Finally, it should be noted that the above embodiments are only used to illustrate the present invention, rather than to limit the present invention. Although the present invention is described in detail with reference to the embodiments, it should be understood by those skilled in the art that various combinations, modifications or equivalent substitutions of the technical solutions of the present invention do not depart from the spirit and scope of the technical solutions of the present invention, and should all be included in the scope of the present invention.

Claims

1. A food display method, characterized in that: include: Acquire an image recognition result of a food image, wherein the image recognition result includes at least two similarity results, and any of the similarity results is used to characterize the similarity between the food to be displayed and any preset food; Screening each similarity result in the image recognition result to obtain a target similarity result set, wherein the sum of each similarity result in the target similarity result set is greater than or equal to a preset similarity threshold; Display food information corresponding to each similarity result in the target similarity result set.

2. The food display method according to claim 1, characterized in that: The target similarity result set includes the maximum similarity result among the image recognition results.

3. The food display method according to claim 2, characterized in that: The screening of the similarity results in the image recognition results to obtain a target similarity result set includes: Sorting the similarity results in the image recognition results to obtain a sorted similarity result set; The sorted similarity result sets are screened one by one in descending order until the sum of the similarity results in the screened similarity result sets is greater than or equal to the preset similarity threshold; Based on the similarity result set after the last screening round, a target similarity result set is determined.

4. The food display method according to claim 3, characterized in that: The determining of the target similarity result set based on the similarity result set after the last screening round screening includes: Determining that the number of each similarity result in the similarity result set after the last screening round is less than a preset similarity number, determining that the recognition is successful, and determining the similarity result set after the last screening round as the target similarity result set; Determine that the number of each similarity result in the similarity result set after the last screening round is greater than or equal to the preset similarity number, and return to the step of obtaining the image recognition result of the food image until the recognition is successful.

5. The food display method according to claim 1, characterized in that: The displaying of food information corresponding to each similarity result in the target similarity result set includes: Based on the order of the similarity results in the target similarity result set, the food information corresponding to each similarity result in the target similarity result set is displayed in order from large to small.

6. The food display method according to claim 1, characterized in that: The step of obtaining the image recognition result of the food image includes: Acquire the food image; Performing image recognition on the food image to obtain the image recognition result; or, The food image is sent to a server so that the server can perform image recognition on the food image to obtain an image recognition result, and the image recognition result sent by the server is received.

7. The food display method according to claim 6, characterized in that: The obtaining of the food image also includes: Determining that a door of the cooking device is closed, and controlling an image acquisition device of the cooking device to operate so as to acquire the food image; or, An image recognition start instruction is obtained to control the image acquisition device of the cooking device to operate so as to acquire the food image.

8. The food display method according to claim 1, characterized in that: The method further includes: displaying food information corresponding to each similarity result in the target similarity result set; and then: A user operation instruction is obtained, and a selected target food is determined based on the user operation instruction.

9. A food display device, characterized in that: include: An acquisition module, used to acquire an image recognition result of a food image, wherein the image recognition result includes at least two similarity results, and any of the similarity results is used to characterize the similarity between the food to be displayed and any preset food; A screening module, used for screening each similarity result in the image recognition result to obtain a target similarity result set, wherein the sum of each similarity result in the target similarity result set is greater than or equal to a preset similarity threshold; The display module is used to display the food information corresponding to each similarity result in the target similarity result set.

10. A cooking device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the food display method according to any one of claims 1 to 8 is implemented.

11. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the food display method according to any one of claims 1 to 8 is implemented.

Images