A method and device for identifying food maturity

Abstract

The present disclosure relates to a method and device for recognizing the maturity of food, which includes: obtaining at least two pieces of image information corresponding to food, wherein the food includes at least one ingredient; performing image recognition processing on the at least two pieces of image information to obtain recognition results corresponding to the at least two pieces of image information; calculating the ingredient screening parameters corresponding to at least one ingredient in the target image information based on the position information, confidence level and brightness value of the target area corresponding to each ingredient in the target image information, wherein the target image information is any one of the at least two pieces of image information; performing maturity recognition processing on candidate ingredients to obtain corresponding maturity information, wherein the candidate ingredients are ingredients corresponding to the maximum ingredient screening parameters corresponding to each piece of image information; and determining the maturity information of the food according to the corresponding maturity information. The present disclosure can improve the accuracy of recognizing the maturity of food.

Description

Technical Field

[0001] The present invention relates to the technical field of smart home appliances, and in particular to a method and device for identifying the maturity of food. Background Art

[0002] With the continuous development of smart home appliance technology, more and more food cooking relies on the automatic control of kitchen appliances, such as steam ovens, microwave ovens, etc. In related technologies, food is generally cooked by controlling the heating time and power level. Due to the large variety of ingredients, different ingredients have different cooking firepower requirements and cooking times. If only the heating time and power level are set, it is easy for the ingredients to be immature or burnt. Therefore, it is necessary to detect the maturity of the food in a timely manner during the cooking process to improve the accuracy of food cooking. Summary of the invention

[0003] In order to solve at least one of the above-mentioned technical problems, the present disclosure proposes a method and device for identifying food maturity.

[0004] In one aspect, the present disclosure provides a method for identifying food maturity, the method comprising:

[0005] Acquire at least two pieces of image information corresponding to food, where the food includes at least one ingredient, and the at least two pieces of image information respectively correspond to different acquisition times;

[0006] Performing image recognition processing on at least two pieces of image information to obtain recognition results corresponding to the at least two pieces of image information, wherein the recognition results include location information of at least one food ingredient, a confidence level, and a brightness value of a target area corresponding to the food ingredient, wherein the target area is an area within the image information indicated by the location information;

[0007] Calculate the food screening parameters corresponding to at least one food in the target image information based on the location information of each food in the target image information, the confidence level and the brightness value of the target area corresponding to the food, where the target image information is any one of the at least two pieces of image information;

[0008] Perform maturity recognition processing on candidate ingredients to obtain corresponding maturity information, where the candidate ingredients are ingredients corresponding to the maximum ingredient screening parameters corresponding to each image information;

[0009] The maturity information of the food is determined according to the corresponding maturity information.

[0010] In an optional embodiment, based on the position information of each ingredient in the target image information, the confidence level, and the brightness value of the target area corresponding to the ingredient, calculating the ingredient screening parameter corresponding to at least one ingredient in the target image information includes:

[0011] Calculate an area index corresponding to the target food material according to the position information corresponding to the target food material in the target image information and the image area corresponding to the target image information, wherein the area index represents the ratio of the area indicated by the position information corresponding to the target food material to the image area;

[0012] Calculate the illumination index corresponding to the target food according to the average brightness value, the first brightness value and the second brightness value, wherein the average brightness value is the average brightness of each brightness value, the first brightness value is the maximum brightness value among each brightness value, and the second brightness value is the minimum brightness value among each brightness value;

[0013] According to the area index corresponding to the target ingredient, the confidence level corresponding to the target ingredient and the light index corresponding to the target ingredient, the ingredient screening parameter corresponding to the target ingredient is calculated, and the target ingredient is any one of the at least one ingredient.

[0014] In an optional embodiment, the food screening parameter corresponding to the target food is calculated according to the area index corresponding to the target food, the confidence level corresponding to the target food, and the light index corresponding to the target food, including:

[0015] Multiply the area index corresponding to the target food, the confidence level corresponding to the target food, the light index corresponding to the target food, and 3 to obtain a first coefficient;

[0016] Adding the area index corresponding to the target food, the confidence level corresponding to the target food, and the light index corresponding to the target food to obtain a second coefficient;

[0017] The ratio of the first coefficient to the second coefficient is determined as the ingredient screening parameter corresponding to the target ingredient.

[0018] In an optional embodiment, calculating the illumination index corresponding to the target food according to the average brightness value, the first brightness value, and the second brightness value includes:

[0019] Subtract the average brightness value from the second brightness value to obtain a third coefficient;

[0020] Subtracting the first brightness value from the average brightness value to obtain a fourth coefficient;

[0021] Subtracting the first brightness value from the second brightness value to obtain a fifth coefficient;

[0022] The ratio of the product of the third coefficient and the fourth coefficient to the square of the fifth coefficient is determined as the illumination index corresponding to the target food.

[0023] In an optional embodiment, obtaining at least two pieces of image information corresponding to the food includes:

[0024] During the cooking process of food, one piece of image information is acquired at every preset time interval, and the preset time interval increases as the number of times the image information is acquired increases.

[0025] In an optional embodiment, the method further comprises:

[0026] In response to each acquisition of image information, the following operations are performed:

[0027] Performing image recognition processing on the image information to obtain a recognition result corresponding to the image information;

[0028] Calculate the food screening parameters corresponding to at least one food in the image information based on the position information corresponding to at least one food in the image information, the confidence level, and the brightness value corresponding to the target area;

[0029] Determine the food corresponding to the largest food screening parameter in the image information as the candidate food corresponding to the image information;

[0030] Perform maturity recognition processing on the candidate food materials corresponding to the image information to obtain corresponding maturity information;

[0031] In an optional embodiment, after obtaining the corresponding maturity information, the method further includes:

[0032] When the acquisition order corresponding to the image information is not greater than the preset storage value, the maturity information corresponding to the candidate food corresponding to the image information is stored in the storage array;

[0033] When the acquisition order corresponding to the image information is greater than the preset storage value, the maturity information first saved in the storage array is deleted, and the maturity information corresponding to the candidate food corresponding to the image information is stored in the storage array.

[0034] In an optional embodiment, the corresponding maturity information includes each maturity information stored in the storage array, and determining the maturity information of the food according to the corresponding maturity information includes:

[0035] Determine the initial time weights corresponding to the respective maturity information based on the collection order of the image information associated with each maturity information;

[0036] According to the ingredients associated with each piece of maturity information, the initial time weights corresponding to the maturity information pointing to the same ingredient are added together to obtain the time weights corresponding to the ingredients associated with each piece of maturity information;

[0037] The last maturity information corresponding to the ingredient with the largest time weight is determined as the maturity information corresponding to the food.

[0038] In an optional embodiment, based on the acquisition order of the image information associated with each piece of maturity information, determining the initial time weight corresponding to each piece of maturity information includes:

[0039] Obtaining the acquisition order corresponding to each associated image information;

[0040] Each acquisition order is evenly mapped to a preset value range in chronological order to obtain a mapping value corresponding to each acquisition order;

[0041] Based on a preset corresponding relationship, determining the time weight corresponding to each associated image information, wherein the preset corresponding relationship represents the corresponding relationship between the mapping value and the time weight, and the ratio between the time weight and the corresponding mapping value decreases as the mapping value increases;

[0042] The corresponding time weight is determined as the initial time weight of the maturity information associated with each associated image information.

[0043] In a second aspect, the present invention further provides a device for identifying food maturity, comprising:

[0044] An acquisition module, used to acquire at least two pieces of image information corresponding to food, the food comprising at least one ingredient, and the at least two pieces of image information respectively corresponding to different acquisition times;

[0045] An image recognition module is used to perform image recognition processing on at least two pieces of image information to obtain recognition results corresponding to the at least two pieces of image information, wherein the recognition results include location information of at least one food ingredient, a confidence level, and a brightness value of a target area corresponding to the food ingredient, wherein the target area is an area within the image information indicated by the location information;

[0046] a calculation module, configured to calculate an ingredient screening parameter corresponding to at least one ingredient in the target image information based on the position information of each ingredient in the target image information, the confidence level, and the brightness value of the target area corresponding to the ingredient, wherein the target image information is any one of the at least two pieces of image information;

[0047] A maturity recognition module is used to perform maturity recognition processing on candidate ingredients to obtain corresponding maturity information, wherein the candidate ingredients are the ingredients corresponding to the maximum ingredient screening parameters corresponding to each image information;

[0048] The maturity determination module is used to determine the maturity information of the food according to the corresponding maturity information.

[0049] In a third aspect, the present invention further provides an electronic device, comprising:

[0050] processor;

[0051] a memory for storing processor-executable instructions;

[0052] The processor is used to execute instructions to implement the above-mentioned food maturity identification method.

[0053] In a fourth aspect, the present invention further provides a storage medium, which, when instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to execute the above-mentioned food maturity identification method.

[0054] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

[0055] The implementation of this disclosure has the following beneficial effects:

[0056] Acquire at least two images corresponding to food, where the food includes at least one ingredient, and the collection time corresponding to each of the at least two images is different; perform image recognition processing on the at least two images to obtain recognition results corresponding to each of the at least two images, where the recognition results include location information, confidence and brightness value of a target area corresponding to the at least one ingredient, where the target area is an area within the image information indicated by the location information; calculate ingredient screening parameters corresponding to each of the at least one ingredient in the target image information based on the location information, confidence and brightness value of the target area corresponding to each ingredient in the target image information, where the target image information is any one of the at least two images; perform maturity recognition processing on candidate ingredients to obtain corresponding maturity information, where the candidate ingredients are ingredients corresponding to the maximum ingredient screening parameters corresponding to each of the image information; determine maturity information of the food based on the corresponding maturity information.

[0057] The present invention obtains at least two pieces of image information by acquiring images of food at different times, and can timely acquire images of different moments in the cooking process of food during the cooking process of food, so as to identify the maturity of food; by calculating the food screening parameters corresponding to at least one food in the target image information based on the position information of each food in the target image information, the confidence level and the brightness value of the target area corresponding to the food, the food screening parameters for indicating the size of the area occupied by each food and the degree of influence of light on each food can be obtained; by performing maturity recognition processing on the food corresponding to the maximum food screening parameter corresponding to each piece of image information, the corresponding maturity information can be obtained, and the maturity of the food suitable for maturity judgment can be recognized, and the influence of the food not suitable as the maturity analysis object on the food maturity recognition can be eliminated, so as to obtain reliable food maturity information.

[0058] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present application or the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the prior art description. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure, and do not constitute an improper limitation of the present disclosure.

[0060] Figure 1 is a schematic diagram of an implementation environment according to an exemplary embodiment;

[0061] Figure 2 is a flow chart of a method for identifying food maturity according to an exemplary embodiment;

[0062] Figure 3 is a schematic diagram showing a method of determining a preset duration according to an exemplary embodiment;

[0063] Figure 4 is a schematic diagram of a recognition process of an image recognition model according to an exemplary embodiment;

[0064] Figure 5 is a schematic diagram showing a method of determining a time weight corresponding to each piece of image information according to an exemplary embodiment;

[0065] Figure 6 is a schematic diagram of a device for identifying food maturity according to an exemplary embodiment;

[0066] Figure 7 It is a block diagram of an electronic device for identifying food maturity according to an exemplary embodiment. DETAILED DESCRIPTION

[0067] The following will be combined with the drawings in the embodiments of this specification to clearly and completely describe the technical solutions in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0068] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0069] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the drawings, the drawings are not necessarily drawn to scale unless otherwise indicated. The word "exemplary" is used specifically herein to mean "serving as an example, embodiment, or illustrative". Any embodiment described herein as "exemplary" is not necessarily to be construed as being superior or better than other embodiments.

[0070] The term "and / or" herein is only a description of the association relationship of the associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set consisting of A, B, and C.

[0071] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.

[0072] In the related art, the method of detecting the maturity of food is often to perform maturity recognition on the whole image of the food to determine the maturity information of the food. However, since food may include multiple ingredients, not all of them are suitable as objects of maturity recognition. For example, some ingredients occupy a small area in the image, have fewer color and texture features in the image, or are too deeply affected by the light of the steam oven, such as some ingredients that are too bright or too dark. Therefore, if the whole image is used as the recognition object, the above-mentioned objects that are not suitable for maturity recognition will interfere with the recognition results of the maturity of the food, resulting in a large gap between the actual maturity and the recognized maturity, affecting the cooking results.

[0073] In order to improve the accuracy of food maturity recognition, the present disclosure proposes a food maturity recognition method and device.

[0074] See also Figure 1 , Figure 1 is a schematic diagram of an application environment according to an exemplary embodiment. Figure 1 As shown, the application environment may include a server 01 and a terminal 02 .

[0075] In an optional embodiment, the server 01 can be used for calculation and processing in the method for identifying the maturity of food. Specifically, the server 01 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0076] In an optional embodiment, the terminal 02 can perform calculation processing in combination with the food maturity recognition method of the server 01. Specifically, the terminal 02 can include but is not limited to smart phones, desktop computers, tablet computers, laptop computers, smart speakers, digital assistants, augmented reality (AR) / virtual reality (VR) devices, smart wearable devices and other types of electronic devices. Optionally, the operating system running on the electronic device can include but is not limited to Android system, IOS system, Linux system, Windows system, Unix system and the like.

[0077] For example, at least two images of food corresponding to the terminal 02 are obtained and transmitted to the server 01, the food includes at least one ingredient, and the collection time corresponding to the at least two images is different; the server 01 performs image recognition processing on the at least two images to obtain the recognition results corresponding to the at least two images, the recognition results include the location information, confidence and brightness value of the target area corresponding to the at least one ingredient, the target area is the area in the image information indicated by the location information; based on the location information, confidence and brightness value of the target area corresponding to each ingredient in the target image information, the food screening parameter corresponding to at least one ingredient in the target image information is calculated, the target image information is any one of the at least two images; the maturity recognition processing is performed on the candidate ingredients to obtain the corresponding maturity information, the candidate ingredients are the ingredients corresponding to the maximum food screening parameters corresponding to each image information; the maturity information of the food is determined according to the corresponding maturity information. And the maturity information of the food is transmitted to the terminal 02.

[0078] In addition, it should be noted that Figure 1 What is shown is only one application environment provided by the present disclosure. In actual application, other application environments may also be included.

[0079] In the embodiments of this specification, the server 01 and the terminal 02 may be directly or indirectly connected via wired or wireless communication, which is not limited in this disclosure.

[0080] Figure 2 is a flow chart of a method for identifying food maturity according to an exemplary embodiment. Figure 2 As shown, the above method includes the following:

[0081] Step S201: obtaining at least two pieces of image information corresponding to food, where the food includes at least one ingredient, and the at least two pieces of image information respectively correspond to different collection times.

[0082] In the embodiment of the present disclosure, obtaining at least two pieces of image information corresponding to the food may be performed at different times during the cooking process of the food in a steam oven or a microwave oven. The food is composed of a variety of ingredients, and during the cooking process of the food, one piece of image information is obtained at a preset time interval, and the preset time increases as the number of times the image information is obtained increases.

[0083] Specifically, the total time for obtaining the above-mentioned at least two image information is ; Get the first Image information distance to obtain the first The time interval between each image (That is, the preset duration )and Number of times to obtain image information within a certain period of time (That is, the number of images ) is shown in the following formula (1):

[0084] (1)

[0085] like Figure 3 As shown, the embodiment of the present disclosure uses function Solve when the total detection time is , the number of detections is of Value; among them, for The inverse function can be obtained by the following formula (2):

[0086] (2)

[0087] pass Total time for judgment The value is obtained in The sampling length on the x-scale is based on Sampling at equal intervals can obtain a series of image information at the collection time that meets and . Increase It will gradually increase, and the image features collected at different collection time points will undergo significant changes. The final maturity judgment result obtained based on multiple detection results will also be more credible.

[0088] As i increases, the cooking time of food increases, the food will irreversibly change from raw to cooked, and the color and texture characteristics of the different ingredients in the food will become more obvious. The maturity judgment results at each moment are more credible.

[0089] In an optional embodiment, in response to each acquisition of image information, the following operations are performed:

[0090] Perform image recognition processing on the image information to obtain a recognition result corresponding to the image information; calculate an ingredient screening parameter corresponding to at least one ingredient in the image information based on the position information, confidence level and brightness value corresponding to the target area corresponding to each of the at least one ingredient in the image information; determine the ingredient corresponding to the largest ingredient screening parameter in the image information as the candidate ingredient corresponding to the image information; perform maturity recognition processing on the candidate ingredients corresponding to the image information to obtain corresponding maturity information.

[0091] In the disclosed embodiment, each time the image information is acquired, the above operations are implemented, and the maturity recognition processing of the candidate food corresponding to the image information can be performed by determining the image area of ​​the candidate food in the corresponding image based on the position information of the candidate food in the corresponding image, inputting the image content in the image area into the trained maturity analysis model, and the maturity analysis model performs maturity analysis on the above image content and outputs the corresponding maturity information. The specific implementation of the above operations is shown in the following embodiments.

[0092] In an optional embodiment, after obtaining the corresponding maturity information, the method further includes:

[0093] When the acquisition order corresponding to the image information is not greater than the preset storage value, the maturity information corresponding to the candidate ingredients corresponding to the image information is stored in the storage array; when the acquisition order corresponding to the image information is greater than the preset storage value, the maturity information first saved in the storage array is deleted, and the maturity information corresponding to the candidate ingredients corresponding to the image information is stored in the storage array.

[0094] In the disclosed embodiment, the preset storage value is the number of maturity information that can be stored in the storage array, that is, the storage array can store at most the maturity information corresponding to the candidate ingredients corresponding to the preset storage value of image information. In the case where the acquisition order corresponding to the image information is not greater than the preset storage value, the storage array can still store the maturity information, so the maturity information corresponding to the candidate ingredients corresponding to the image information is stored in the storage array. In the case where the acquisition order corresponding to the image information is greater than the preset storage value, there is no storage space in the storage array, and the maturity information first saved in the storage array, that is, the maturity information saved earliest in the storage array at this time, is deleted, and the maturity information corresponding to the candidate ingredients corresponding to the image information is stored in the storage array.

[0095] Step S202: Perform image recognition processing on at least two pieces of image information to obtain recognition results corresponding to the at least two pieces of image information, wherein the recognition results include location information of at least one ingredient, a confidence level, and a brightness value of a target area corresponding to the ingredient, and the target area is an area within the image information indicated by the location information.

[0096] In the embodiment of the present disclosure, image recognition processing of at least two pieces of image information may be performed by inputting the at least two pieces of image information into a trained image recognition model, and having the image recognition model perform image recognition on each piece of image information, and output recognition results corresponding to each of the at least two pieces of image information, wherein the recognition results include location information of at least one food ingredient, a confidence level, and a brightness value of a target area corresponding to the food ingredient, wherein the target area is an area within the image information indicated by the location information.

[0097] Specifically, Figure 4As shown in Figure 1, when an image P is input into the image recognition model, the corresponding recognition result output by the image recognition model is ,in, is the number of the food detected in the image, is the location information, including the coordinate information of the corner points of the rectangular box of a certain food. is the confidence level, Represents the probability that the corresponding food is in the corresponding rectangular box, is the food type information, and the target area is within the image P The image area enclosed by the coordinate information of the corner points of the indicated rectangular frame, the brightness value corresponding to the target area is the brightness value of each pixel in the area. Specifically, after inputting the first image information, the recognition result corresponding to the output of the image recognition model can be expressed as ; After inputting the second image information, the recognition result of the image recognition model can be expressed as ; After inputting the third image information, the recognition result of the image recognition model can be expressed as .

[0098] Step S203: Calculate the ingredient screening parameters corresponding to at least one ingredient in the target image information based on the location information, confidence level and brightness value of the target area corresponding to each ingredient in the target image information, where the target image information is any one of the at least two image information.

[0099] In an optional embodiment, based on the position information of each ingredient in the target image information, the confidence level, and the brightness value of the target area corresponding to the ingredient, calculating the ingredient screening parameter corresponding to at least one ingredient in the target image information includes:

[0100] Step S301: Calculate the area index corresponding to the target ingredient according to the position information corresponding to the target ingredient in the target image information and the image area corresponding to the target image information. The area index represents the ratio of the area indicated by the position information corresponding to the target ingredient to the image area.

[0101] In the embodiment of the present disclosure, according to the position information corresponding to the target food in the target image information and the image area corresponding to the target image information, the area index corresponding to the target food can be calculated according to the following formula (3):

[0102] (3)

[0103] In formula (3), is the area value of the rectangular frame where the target food is located in the target image information, which is calculated based on the position information corresponding to the target food. is the image area corresponding to the target image, is the area index corresponding to the target food.

[0104] Step S302: Calculate the illumination index corresponding to the target food according to the average brightness value, the first brightness value and the second brightness value. The average brightness value is the average brightness of each brightness value, the first brightness value is the maximum brightness value among each brightness value, and the second brightness value is the minimum brightness value among each brightness value.

[0105] In the embodiment of the present disclosure, the illumination index corresponding to the target food is calculated according to the average brightness value, the first brightness value and the second brightness value, including:

[0106] Subtract the average brightness value from the second brightness value to obtain the third coefficient; subtract the first brightness value from the average brightness value to obtain the fourth coefficient; subtract the first brightness value from the second brightness value to obtain the fifth coefficient; and determine the ratio of the product of the third coefficient and the fourth coefficient to the square of the fifth coefficient as the lighting index corresponding to the target food.

[0107] In the embodiment of the present disclosure, the above calculation process can be calculated by the following formula (4):

[0108] (4)

[0109] In formula (4), is the average brightness value, is the first brightness value, is the second brightness value, is the light index corresponding to the target food.

[0110] Based on the above, it can be known that in the embodiment of the present disclosure, the third coefficient is obtained by subtracting the average brightness value from the second brightness value; the fourth coefficient is obtained by subtracting the first brightness value from the average brightness value; the fifth coefficient is obtained by subtracting the first brightness value from the second brightness value; and the ratio of the product of the third coefficient and the fourth coefficient to the square of the fifth coefficient is determined as the light index corresponding to the target food. The light index corresponding to the target food can reflect the influence of light intensity on the target food.

[0111] Step S303: Calculate the food screening parameter corresponding to the target food according to the area index corresponding to the target food, the confidence level corresponding to the target food and the light index corresponding to the target food, where the target food is any one of the at least one food.

[0112] In the disclosed embodiment, the food screening parameter is used to characterize the index of the corresponding target food as a reference food for the maturity of the food, and the maturity of the reference food is used to represent the maturity of the food. According to the area index corresponding to the target food, the confidence level corresponding to the target food, and the light index corresponding to the target food, the food screening parameter corresponding to the target food is calculated, including:

[0113] Multiply the area index corresponding to the target ingredient, the confidence level corresponding to the target ingredient, the light index corresponding to the target ingredient, and 3 to obtain a first coefficient; add the area index corresponding to the target ingredient, the confidence level corresponding to the target ingredient, and the light index corresponding to the target ingredient to obtain a second coefficient; determine the ratio of the first coefficient to the second coefficient as the ingredient screening parameter corresponding to the target ingredient.

[0114] In the embodiment of the present disclosure, the above calculation process can be calculated by the following formula (5):

[0115] (5)

[0116] In formula (5), is the area index corresponding to the target food, is the light index corresponding to the target food, is the confidence level corresponding to the target ingredient, is the light index corresponding to the target food.

[0117] Based on the above, it can be known that in the embodiment of the present disclosure, the first coefficient is obtained by multiplying the area index corresponding to the target ingredient, the confidence level corresponding to the target ingredient, the light index corresponding to the target ingredient, and 3; the second coefficient is obtained by adding the area index corresponding to the target ingredient, the confidence level corresponding to the target ingredient, and the light index corresponding to the target ingredient; the ratio of the first coefficient to the second coefficient is determined as the ingredient screening parameter corresponding to the target ingredient, and the ingredient screening parameter corresponding to the target ingredient can reflect the index of the target ingredient that is suitable as a reference ingredient for food maturity under the influence of area factors and light factors.

[0118] Step S204: performing maturity recognition processing on the candidate food ingredients to obtain corresponding maturity information, wherein the candidate food ingredients are the food ingredients corresponding to the maximum food ingredient screening parameters corresponding to each piece of image information.

[0119] In the disclosed embodiment, each image information includes the same ingredients. Since the collection time of different image information is different, the ingredient screening parameters corresponding to each ingredient calculated according to different image information are also different. Therefore, the ingredients with the maximum ingredient screening parameters corresponding to different image information may also be different. The candidate ingredients are the ingredients corresponding to the maximum ingredient screening parameters corresponding to each image information respectively.

[0120] In the embodiment of the present disclosure, the corresponding maturity information includes each maturity information stored in the storage array, and the maturity identification processing is performed on the candidate food to obtain the corresponding maturity information, including:

[0121] Step S401: determining initial time weights corresponding to respective pieces of maturity information based on the order of collecting image information associated with respective pieces of maturity information.

[0122] In the embodiment of the present disclosure, the image information associated with each maturity information is the image information corresponding to each maturity information. Based on the collection order of the image information associated with each maturity information, the initial time weight corresponding to each maturity information is determined, including:

[0123] Step S4011: Obtain the acquisition order corresponding to each associated image information.

[0124] In the disclosed embodiment, the order of the first image information collected during the food cooking process is 0, the order of the second image information collected during the food cooking process is 1, the order of the third image information collected during the food cooking process is 2, and so on. The collection order corresponding to each associated image information is only related to the collection order during the food cooking process, and has nothing to do with the order in which the maturity information corresponding to the corresponding candidate ingredients is saved in the storage array, and does not change when the maturity information corresponding to the corresponding candidate ingredients is deleted from the storage array.

[0125] Step S4012: uniformly map each acquisition order to a preset value range according to the chronological order, and obtain a mapping value corresponding to each acquisition order.

[0126] In the embodiment of the present disclosure, the preset value interval is , the order of collecting each image information The uniform mapping becomes Sampling at equal intervals within the range The times obtained value, The value is the mapping value corresponding to each collection order.

[0127] Step S4013: Based on a preset correspondence relationship, determine the time weight corresponding to each associated image information. The preset correspondence relationship represents the correspondence between the mapping value and the time weight. The ratio between the time weight and the corresponding mapping value decreases as the mapping value increases.

[0128] In the embodiment of the present disclosure, the preset corresponding relationship is a sine function variation, through which the sine function variation can be calculated to obtain The corresponding time weight. The schematic diagram of the sine function is as follows Figure 5 As shown, the corresponding sine function variant is shown in formula (6):

[0129] (6)

[0130] In formula (6), the value of x is , A is a constant, is the time weight.

[0131] Step S4014: Determine the corresponding time weight as the initial time weight of the maturity information associated with each associated image information.

[0132] In the present disclosure, The corresponding time weight is recorded as , No. The time weight corresponding to the image information is The initial time weight of the maturity information associated with each image.

[0133] Based on the above, it can be known that in the embodiment of the present disclosure, the acquisition order of each associated image information is evenly mapped to a preset numerical range in chronological order to obtain the mapping value corresponding to each acquisition order, and then the time weight corresponding to each associated image information is determined according to the preset corresponding relationship, and the corresponding time weight is determined as the initial time weight of the maturity information associated with each associated image information, which can make the time weight of the reference food that is more suitable as the maturity of food greater.

[0134] Step S402: according to the ingredients associated with each piece of maturity information, the initial time weights corresponding to the maturity information pointing to the same ingredient are added together to obtain the time weights corresponding to the ingredients associated with each piece of maturity information.

[0135] In the disclosed embodiment, since the ingredients associated with different maturity information may be the same or different, the same ingredient may correspond to two initial weights. The initial time weights of the same ingredient corresponding to different maturity information are added together to obtain the time weight corresponding to each ingredient.

[0136] Step S403: Determine the last maturity information corresponding to the ingredient with the largest time weight as the maturity information corresponding to the food.

[0137] In the disclosed embodiment, since there may be multiple pieces of maturity information stored in the storage array for the same food ingredient, the corresponding last maturity information, that is, the maturity information with the latest storage time, is determined as the maturity information corresponding to the food.

[0138] Based on the above, it can be known that in the embodiment of the present disclosure, by determining the maturity reference ingredients, performing maturity analysis on the maturity reference ingredients, obtaining maturity information corresponding to the maturity reference ingredients, and determining the maturity information corresponding to the maturity reference ingredients as the maturity information corresponding to the food, the maturity of the entire food can be represented by the maturity of the ingredient in the food that is most suitable for maturity analysis, thereby eliminating the influence of ingredients that are not suitable as maturity analysis objects on food maturity identification, and improving the accuracy of food maturity identification.

[0139] Figure 6is a block diagram of a device for identifying food maturity according to an exemplary embodiment. Figure 6 The device includes an acquisition module 601, an image recognition module 602, a calculation module 603, a maturity recognition module 604 and a maturity determination module 605, wherein:

[0140] An acquisition module 601 is used to acquire at least two pieces of image information corresponding to food, where the food includes at least one ingredient, and the at least two pieces of image information respectively correspond to different acquisition times;

[0141] An image recognition module 602 is used to perform image recognition processing on at least two pieces of image information to obtain recognition results corresponding to the at least two pieces of image information, wherein the recognition results include location information of at least one food ingredient, a confidence level, and a brightness value of a target area corresponding to the food ingredient, wherein the target area is an area within the image information indicated by the location information;

[0142] A calculation module 603 is used to calculate the food screening parameter corresponding to at least one food in the target image information based on the location information of each food in the target image information, the confidence level and the brightness value of the target area corresponding to the food, where the target image information is any one of the at least two image information;

[0143] A maturity recognition module 604 is used to perform maturity recognition processing on candidate ingredients to obtain corresponding maturity information. The candidate ingredients are the ingredients corresponding to the maximum ingredient screening parameters corresponding to each image information.

[0144] The maturity determination module 605 is used to determine the maturity information of the food according to the corresponding maturity information.

[0145] In an optional embodiment, the calculation module 603 includes:

[0146] A first calculation submodule, for calculating an area index corresponding to the target food according to the position information corresponding to the target food in the target image information and the image area corresponding to the target image information, wherein the area index represents the ratio of the area indicated by the position information corresponding to the target food to the image area;

[0147] A second calculation submodule is used to calculate the illumination index corresponding to the target food according to the average brightness value, the first brightness value and the second brightness value, wherein the average brightness value is the brightness mean of each brightness value, the first brightness value is the maximum brightness value among each brightness value, and the second brightness value is the minimum brightness value among each brightness value;

[0148] The third calculation submodule is used to calculate the food screening parameter corresponding to the target food according to the area index corresponding to the target food, the confidence level corresponding to the target food and the light index corresponding to the target food, where the target food is any one of the at least one food.

[0149] In an optional embodiment, the third computing submodule includes:

[0150] a fourth calculation submodule, configured to multiply the area index corresponding to the target food, the confidence level corresponding to the target food, the light index corresponding to the target food, and 3 to obtain a first coefficient;

[0151] a fifth calculation submodule, configured to add the area index corresponding to the target food, the confidence level corresponding to the target food, and the light index corresponding to the target food to obtain a second coefficient;

[0152] The sixth calculation submodule is used to determine the ratio of the first coefficient to the second coefficient as the ingredient screening parameter corresponding to the target ingredient.

[0153] In an optional embodiment, the second computing submodule includes:

[0154] a seventh calculation submodule, configured to subtract the average brightness value from the second brightness value to obtain a third coefficient;

[0155] an eighth calculation submodule, configured to subtract the first brightness value from the average brightness value to obtain a fourth coefficient;

[0156] a ninth calculation submodule, configured to subtract the first brightness value from the second brightness value to obtain a fifth coefficient;

[0157] The tenth calculation submodule is used to determine the light index corresponding to the target food by taking the ratio of the product of the third coefficient and the fourth coefficient and the square of the fifth coefficient.

[0158] In an optional embodiment, the acquisition module 601 includes:

[0159] The first acquisition submodule is used to acquire a piece of image information at a preset time interval during the cooking process of food, and the preset time interval increases as the number of times the image information is acquired increases.

[0160] In an optional embodiment, the device further comprises:

[0161] The execution module is used to perform the following operations in response to each acquisition of image information: performing image recognition processing on the image information to obtain a recognition result corresponding to the image information; calculating an ingredient screening parameter corresponding to at least one ingredient in the image information based on the position information, confidence level and brightness value corresponding to the target area corresponding to each of the at least one ingredient in the image information; determining the ingredient corresponding to the largest ingredient screening parameter in the image information as the candidate ingredient corresponding to the image information; performing maturity recognition processing on the candidate ingredients corresponding to the image information to obtain corresponding maturity information.

[0162] In an optional embodiment, the device further comprises:

[0163] A first storage module, used for storing the maturity information corresponding to the candidate food corresponding to the image information into a storage array when the acquisition order corresponding to the image information is not greater than a preset storage value;

[0164] The second saving module is used to delete the maturity information first saved in the storage array when the collection order corresponding to the image information is greater than the preset storage value, and store the maturity information corresponding to the candidate food corresponding to the image information into the storage array.

[0165] In an optional embodiment, the corresponding maturity information includes each maturity information stored in the storage array, and the maturity determination module 605 includes:

[0166] A first determination module is used to determine the initial time weights corresponding to the respective maturity information based on the collection order of the image information associated with each maturity information;

[0167] A summing module, for adding the initial time weights corresponding to the maturity information pointing to the same food material according to the food material associated with each maturity information, to obtain the time weights corresponding to the food material associated with each maturity information;

[0168] The second determination module is used to determine the last maturity information corresponding to the ingredient with the largest time weight as the maturity information corresponding to the food.

[0169] In an optional embodiment, the first determining module includes:

[0170] The second acquisition submodule is used to acquire the acquisition order corresponding to each associated image information;

[0171] A mapping module is used to evenly map each acquisition order to a preset value range in chronological order to obtain a mapping value corresponding to each acquisition order;

[0172] A third determination module is used to determine the time weight corresponding to each associated image information based on a preset corresponding relationship, wherein the preset corresponding relationship represents the corresponding relationship between the mapping value and the time weight, and the ratio between the time weight and the corresponding mapping value decreases as the mapping value increases;

[0173] The fourth determination module is used to determine the corresponding time weight as the initial time weight of the maturity information associated with each associated image information.

[0174] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware such as processing circuits or memories, or a combination thereof. Similarly, one processor or multiple processors or memories can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0175] In an exemplary embodiment, an electronic device is also provided, including: a processor; and a memory for storing instructions executable by the processor; wherein the processor is used for the instructions to implement the method for identifying food maturity as in the embodiment of the present disclosure.

[0176] Figure 7 is a block diagram of an electronic device for identifying food maturity according to an exemplary embodiment. The electronic device may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 7 As shown. The electronic device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the electronic device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for identifying the maturity of food is implemented. The display screen of the electronic device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device can be a touch layer covering the display screen, or a key, trackball or touchpad set on the housing of the electronic device, or an external keyboard, touchpad or mouse, etc.

[0177] Those skilled in the art will understand that Figure 7 The structure shown in the figure is merely a block diagram of a partial structure related to the scheme of the present disclosure, and does not constitute a limitation on the electronic device to which the scheme of the present disclosure is applied. The specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0178] In an exemplary embodiment, a storage medium is also provided. When instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute the method for identifying food maturity in the embodiment of the present disclosure.

[0179] In an exemplary embodiment, a computer program product containing instructions is also provided. When the computer program product is run on a computer, the computer is caused to execute the method for identifying the maturity of food in the embodiment of the present disclosure.

[0180] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present disclosure can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0181] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0182] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for identifying food maturity, characterized in that: The method comprises: Acquire at least two pieces of image information corresponding to food, the food comprising at least one ingredient, and the at least two pieces of image information respectively corresponding to different collection times; Performing image recognition processing on the at least two pieces of image information to obtain recognition results corresponding to the at least two pieces of image information, the recognition results including location information of at least one food ingredient, a confidence level, and a brightness value of a target area corresponding to the food ingredient, the target area being an area within the image information indicated by the location information; Calculating the food screening parameters corresponding to at least one food in the target image information based on the location information of each food in the target image information, the confidence level and the brightness value of the target area corresponding to the food, wherein the target image information is any one of the at least two pieces of image information; Performing maturity recognition processing on candidate ingredients to obtain corresponding maturity information, wherein the candidate ingredients are ingredients corresponding to the maximum ingredient screening parameters corresponding to each piece of image information; Determining the maturity information of the food according to the corresponding maturity information; Among them, the method of calculating the ingredient screening parameters corresponding to at least one ingredient in the target image information based on the position information, confidence level and brightness value of the target area corresponding to each ingredient in the target image information includes: calculating the ingredient screening parameters corresponding to the target ingredient according to the area index corresponding to the target ingredient, the confidence level corresponding to the target ingredient and the light index corresponding to the target ingredient, the target ingredient being any one of the at least one ingredient, and the ingredient screening parameters being used to characterize the index of the corresponding target ingredient as a reference ingredient for food maturity, and the maturity of the reference ingredient being used to represent the maturity of the food.

2. The method according to claim 1, characterized in that The step of calculating the food screening parameters corresponding to at least one food in the target image information based on the location information of each food in the target image information, the confidence level, and the brightness value of the target area corresponding to the food, comprises: Calculating an area index corresponding to the target food according to the position information corresponding to the target food in the target image information and the image area corresponding to the target image information, wherein the area index represents a ratio of an area indicated by the position information corresponding to the target food to an image area; Calculate the illumination index corresponding to the target food according to the average brightness value, the first brightness value and the second brightness value, wherein the average brightness value is the brightness mean of the brightness values, the first brightness value is the maximum brightness value of the brightness values, and the second brightness value is the minimum brightness value of the brightness values; The ingredient screening parameter corresponding to the target ingredient is calculated according to the area index corresponding to the target ingredient, the confidence level corresponding to the target ingredient and the light index corresponding to the target ingredient, and the target ingredient is any one of the at least one ingredient.

3. The method according to claim 2, characterized in that The step of calculating the food screening parameter corresponding to the target food according to the area index corresponding to the target food, the confidence level corresponding to the target food, and the light index corresponding to the target food comprises: Multiplying the area index corresponding to the target food, the confidence level corresponding to the target food, the light index corresponding to the target food, and 3 to obtain a first coefficient; Adding the area index corresponding to the target food, the confidence level corresponding to the target food, and the light index corresponding to the target food to obtain a second coefficient; The ratio of the first coefficient to the second coefficient is determined as the ingredient screening parameter corresponding to the target ingredient.

4. The method according to claim 2, characterized in that: The step of calculating the illumination index corresponding to the target food according to the average brightness value, the first brightness value, and the second brightness value includes: Subtracting the average brightness value from the second brightness value to obtain a third coefficient; Subtracting the first brightness value from the average brightness value to obtain a fourth coefficient; Subtracting the first brightness value from the second brightness value to obtain a fifth coefficient; The ratio of the product of the third coefficient and the fourth coefficient to the square of the fifth coefficient is determined as the illumination index corresponding to the target food.

5. The method according to claim 1, characterized in that The obtaining of at least two pieces of image information corresponding to the food includes: During the cooking process of the food, one piece of image information is acquired at every preset time interval, and the preset time interval increases as the number of times the image information is acquired increases.

6. The method according to claim 5, characterized in that The method further comprises: In response to each acquisition of the image information, the following operations are performed: Performing image recognition processing on the image information to obtain a recognition result corresponding to the image information; Calculating an ingredient screening parameter corresponding to at least one ingredient in the image information based on the position information corresponding to at least one ingredient in the image information, the confidence level, and the brightness value corresponding to the target area; Determine the food corresponding to the largest food screening parameter in the image information as the candidate food corresponding to the image information; Perform maturity recognition processing on the candidate food ingredients corresponding to the image information to obtain corresponding maturity information.

7. The method according to claim 6, characterized in that After obtaining the corresponding maturity information, the method further includes: When the acquisition order corresponding to the image information is not greater than a preset storage value, storing the maturity information corresponding to the candidate food corresponding to the image information into a storage array; When the acquisition order corresponding to the image information is greater than the preset storage value, the maturity information first saved in the storage array is deleted, and the maturity information corresponding to the candidate food corresponding to the image information is stored in the storage array.

8. The method according to claim 7, characterized in that The corresponding maturity information includes each maturity information stored in the storage array, and determining the maturity information of the food according to the corresponding maturity information includes: Determining initial time weights corresponding to respective pieces of maturity information based on the order in which the image information associated with each piece of maturity information is collected; According to the ingredients associated with each piece of maturity information, initial time weights corresponding to the maturity information pointing to the same ingredient are added together to obtain time weights corresponding to the ingredients associated with each piece of maturity information; The last maturity information corresponding to the food with the largest time weight is determined as the maturity information corresponding to the food.

9. The method according to claim 8, characterized in that The determining of the initial time weights corresponding to the respective maturity information based on the collection order of the image information respectively associated with the respective maturity information comprises: Obtaining the acquisition order corresponding to each associated image information; Uniformly mapping each of the acquisition orders to a preset value range in chronological order to obtain a mapping value corresponding to each of the acquisition orders; Determine, based on a preset corresponding relationship, a time weight corresponding to each of the associated image information, wherein the preset corresponding relationship represents a corresponding relationship between the mapping value and the time weight, and a ratio between the time weight and the corresponding mapping value decreases as the mapping value increases; The corresponding time weight is determined as the initial time weight of the maturity information associated with each of the associated image information.

10. A food maturity identification device, characterized in that: The device comprises: An acquisition module, used to acquire at least two pieces of image information corresponding to food, wherein the food includes at least one ingredient, and the at least two pieces of image information respectively correspond to different acquisition times; an image recognition module, configured to perform image recognition processing on the at least two pieces of image information to obtain recognition results corresponding to the at least two pieces of image information, wherein the recognition results include location information of at least one food ingredient, a confidence level, and a brightness value of a target area corresponding to the food ingredient, wherein the target area is an area within the image information indicated by the location information; a calculation module, configured to calculate an ingredient screening parameter corresponding to at least one ingredient in the target image information based on the position information of each ingredient in the target image information, the confidence level, and the brightness value of the target area corresponding to the ingredient, wherein the target image information is any one of the at least two pieces of image information; A maturity recognition module, used to perform maturity recognition processing on candidate ingredients to obtain corresponding maturity information, wherein the candidate ingredients are ingredients corresponding to the maximum ingredient screening parameters corresponding to each image information; A maturity determination module, used to determine the maturity information of the food according to the corresponding maturity information; Among them, the calculation module includes: a third calculation sub-module, used to calculate the ingredient screening parameters corresponding to the target ingredient according to the area index corresponding to the target ingredient, the confidence corresponding to the target ingredient and the light index corresponding to the target ingredient, the target ingredient is any ingredient of the at least one ingredient, and the ingredient screening parameters are used to characterize the index of the corresponding target ingredient as a reference ingredient for food maturity, and the maturity of the reference ingredient is used to represent the maturity of the food.

Images