Food material management method, storage medium, system, and refrigerator

Abstract

The application provides a food material management method, a storage medium, a system and a refrigerator. The refrigerator food material management method comprises the following steps: obtaining a food material image stored in the refrigerator and a corresponding food material name; obtaining each freshness stage information of the food material; determining the freshness of several regions of the food material according to the food material image and the each freshness stage information of the food material; and generating and displaying a freshness image marked with the freshness of the several regions of the food material. The storage medium, the system and the refrigerator can realize the above food material management method. In this way, the freshness of different regions of the food material can be intuitively displayed, and the user can know the freshness of the food material without opening the door.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of household appliances, in particular to a food material management method, a storage medium, a system and a refrigerator. BACKGROUND

[0002] With the progress of science and technology, people's demand for the intelligentization of household appliances is getting higher and higher. As a special device for food storage and preservation, the refrigerator also needs to be transformed intelligently. The traditional refrigerator often needs the user to open the refrigerator door to check whether the food materials inside the refrigerator are fresh. However, this design has the following defects: it cannot intuitively display the freshness of the food materials.

[0003] At present, the function of voice reminding of expired food materials on the market is generally to obtain the storage time of food materials according to theoretical common sense, and food materials are considered to be expired when the storage time exceeds the theoretical common sense. However, the freshness of each food material is different at the moment when the food material is purchased and put into the refrigerator. If the expiration time is simply inferred according to the shelf life of the food material, it will obviously cause the situation that the food material has actually rotted but the shelf life has not been reached. SUMMARY

[0004] The present application aims to provide a food material management method, a storage medium, a system and a refrigerator, which can intuitively display the actual freshness of different areas of food materials by generating and displaying a freshness image indicating the freshness of several areas of food materials, and can provide a full range of food material preservation solutions for users by combining the voice reminding function and the temperature control system to feed back the real storage time to the user, real-time remind the damaged food material, and delay the deterioration of the food material through the temperature control system, which is beneficial to the preservation of food materials, avoids the waste of food materials, and provides a full range of food material preservation solutions for users.

[0005] To achieve the above-mentioned application purpose, an embodiment of the present application provides a refrigerator food material management method, which comprises the following steps:

[0006] Obtaining the food material image stored in the refrigerator and the corresponding food material name;

[0007] Obtaining the freshness stage information of the food material;

[0008] Determining the freshness of several areas of the food material according to the food material image and the freshness stage information of the food material;

[0009] Generating and displaying a freshness image indicating the freshness of several areas of the food material.

[0010] As a further improvement of the embodiment of the present application, the refrigerator food material management method further comprises the following steps:

[0011] Obtaining the type of the food material;

[0012] querying a database image corresponding to the food category;

[0013] generating a three-dimensional image of the food according to the food image and the database image;

[0014] selecting several regions of the three-dimensional image and obtaining region images corresponding to the regions;

[0015] determining the freshness of the regions according to the region images and the information of the freshness stages of the food;

[0016] labeling the freshness of the regions on the three-dimensional image, generating and displaying the freshness image.

[0017] As a further improvement of an embodiment of the present application, the refrigerator food management method, wherein the control method of "selecting several regions of the three-dimensional image and obtaining region images corresponding to the regions" comprises:

[0018] dividing the three-dimensional image into several collection regions;

[0019] obtaining the main color of the collection region, the main color being the color with the largest proportion in the collection region;

[0020] randomly selecting several non-intersecting regions from the collection region;

[0021] calculating the color difference between the regions and the main color of the collection region, respectively;

[0022] when the color difference exceeds a preset color difference value, selecting the region and obtaining the region image corresponding to the region.

[0023] As a further improvement of an embodiment of the present application, the refrigerator food management method, wherein the control method of "determining the freshness of the regions according to the region images and the information of the freshness stages of the food" comprises:

[0024] obtaining images of the freshness stages of the food;

[0025] comparing the region images with the images of the food in different freshness stages, respectively;

[0026] determining the freshness of the regions according to the comparison results, respectively.

[0027] As a further improvement of an embodiment of the present application, the refrigerator food management method, wherein the control method further comprises:

[0028] comprehensively determining whether the food is edible according to the freshness of all the regions;

[0029] If yes, edible time of the foodstuff is obtained according to freshness of all the regions and each freshness stage information of the foodstuff, and a prompt information is sent out;

[0030] If no, a prompt information of unedible is sent out.

[0031] As a further improvement of an embodiment of the present application, the refrigerator foodstuff management method, wherein the control method further comprises:

[0032] The suitable storage environment of the foodstuff is determined according to freshness of all the regions and each freshness stage information of the foodstuff;

[0033] The environment at the position where the foodstuff is stored is adjusted to the suitable storage environment.

[0034] As a further improvement of an embodiment of the present application, the refrigerator foodstuff management method, wherein,

[0035] When the refrigerator is in use, the edible information of the foodstuff is broadcasted by voice.

[0036] As a further improvement of an embodiment of the present application, the refrigerator foodstuff management method, wherein,

[0037] The stereoscopic image is a point cloud image, and the freshness of the several regions is indicated by different point densities.

[0038] To achieve the above-mentioned purposes, an embodiment of the present application provides a computer readable storage medium, which stores a computer program, wherein the computer program is executed by a processor to realize the steps of the foodstuff management method in any of the above-mentioned embodiments.

[0039] To achieve the above-mentioned purposes, an embodiment of the present application provides a foodstuff management system, which comprises a control module and an image recognition module, wherein the foodstuff management system further comprises a visual module, and the foodstuff management system further comprises a memory and a processor, the memory stores a computer program executable on the processor, and the processor executes the computer program to realize the steps of the foodstuff management method in any of the above-mentioned embodiments.

[0040] To achieve the above-mentioned purposes, an embodiment of the present application provides a refrigerator, which comprises a storage chamber for storing foodstuff and a foodstuff management system, the foodstuff management system comprises a control module, an image recognition module and a display module, wherein the foodstuff management system further comprises a memory and a processor, the memory stores a computer program executable on the processor, and the processor executes the computer program to realize the steps of the foodstuff management method in any of the above-mentioned embodiments.

[0041] Compared with the prior art, the food material management method can generate and display a freshness image marked with the freshness of several regions of the food material according to the food material image and the freshness stage information of the food material, intuitively display the actual freshness of different regions of the food material, feed back the real storage time to the user, combine the voice reminding function with the temperature control system, and can remind in real time and delay the aggravation of the damage of the food material through the temperature control system when the food material is damaged, so as to help to keep the food material fresh for a longer time, avoid the waste of food, and provide the user with a comprehensive food material storage solution. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 is a flowchart of a food material management method according to an embodiment of the present application;

[0043] Figure 2 is a detailed flowchart of the food material management method shown in Figure 1

[0044] Figure 3 is a detailed flowchart of the food material management method shown in Figure 2 DETAILED DESCRIPTION

[0045] The present application will be described in detail below with reference to the specific embodiments shown in the drawings. However, these embodiments do not limit the present application, and any changes in structure, method, or function made by those of ordinary skill in the art based on these embodiments are included in the protection scope of the present application.

[0046] In an embodiment of the present application, the refrigerator can include a storage compartment for storing food materials and a food material management system. The food material management system can include a control module, an image recognition module, and a display module. The food material management system can further include a memory and a processor. The memory stores a computer program executable on the processor, and the processor can execute the computer program.

[0047] In the present embodiment, the image recognition module can include a camera arranged around the storage compartment. The camera can obtain an image of the food material. The camera can capture the image of the food material after the food material is placed in the refrigerator, or can capture the image of the food material during the process of placing the food material in the refrigerator. The camera can be provided with multiple cameras to obtain multiple angle images of the food material.

[0048] In the present embodiment, the food material management system can further include a temperature control system and a voice interaction system. The temperature control system can be used to adjust the storage environment of the food material, such as the temperature and humidity for storing the food material. The voice interaction system can be used to recognize the voice instructions of the user and voice broadcast specific information to the user, such as the name of the food material stored in the refrigerator, the edible time of the food material, and other information.​​

[0049] The refrigerator can further include a display module connected to the control module, and the display module can include a display screen disposed on the door body of the refrigerator. The display screen can display text, images, and the like.

[0050] Referring to Figure 1 In an embodiment of the present application, a refrigerator food material management method can include:

[0051] Obtaining a food material image and a corresponding food material name stored in a refrigerator;

[0052] Obtaining freshness stage information of the food material;

[0053] Determining the freshness of a plurality of regions of the food material based on the food material image and the freshness stage information of the food material;

[0054] Generating and displaying a freshness image indicating the freshness of the plurality of regions of the food material.

[0055] In the present embodiment, the food material can refer to fruit and vegetable food materials such as apples, green vegetables, and the like.

[0056] The food material image can refer to a photo of the food material taken by a camera in the refrigerator.

[0057] The food material name can be a food material name obtained by recognizing the food material image, or can be a food material name input by a user when storing the food material in the refrigerator. For example, the food material name can be an orange.

[0058] The freshness stage can refer to stages such as unripe, ripe, slightly deteriorated, and severely deteriorated.

[0059] The freshness stage information can refer to text information and image information related to the freshness stage of the food material, and the like. For example, it can be image information of the food material being unripe, ripe, deteriorated, and damaged, it can be time information of the food material from the ripe stage to the deteriorated and damaged stage, it can be environmental information such as temperature and humidity suitable for storing the food material, and the like. The freshness stage information can be obtained by querying a local database or a cloud database.

[0060] The freshness of the region can refer to the freshness stage of a certain delineated range on the outer surface of the food material. Accordingly, the freshness of the plurality of regions of the food material can refer to a plurality of mutually disjoint regions delineated on the outer surface of the food material, and the freshness stages in which the different regions are respectively located.

[0061] Since the food material can be partially slightly deteriorated and unedible while the rest is not deteriorated and edible, or partially unripe while the rest is ripe, the freshness of the food material can be determined more comprehensively and accurately by marking different regions on the surface of the food material and determining the freshness of the regions, so as to facilitate more accurate management of the food material.

[0062] The freshness image can be an image capable of visually showing the freshness of different regions of the food material to the user. For example, it can be an image formed by directly marking the freshness information of different regions of the food material on the photographed image of the food material. The freshness image can be displayed on the display screen of the refrigerator, so that the user can intuitively obtain the freshness information of the food material.

[0063] In this way, the freshness of different positions of the food material can be determined more comprehensively and accurately, the food material can be managed more accurately, the freshness of the food material can be intuitively displayed, the user can view the food material without opening the door, understand the freshness and deterioration of the food material, the user can eat the food material in time before the food material deteriorates, and the user can accurately find the deteriorated part after the food material deteriorates, thereby avoiding waste and avoiding the user eating the deteriorated food material.

[0064] Reference Figure 2 In addition, in an embodiment of the present application, the refrigerator food material management method can further include:

[0065] acquiring the type of the food material;

[0066] querying the database image corresponding to the type of the food material;

[0067] generating a three-dimensional image of the food material according to the food material image and the database image;

[0068] selecting a plurality of regions of the three-dimensional image and acquiring the region image corresponding to the regions;

[0069] determining the freshness of the regions according to the region image and the freshness stage information of the food material, respectively;

[0070] respectively marking the freshness of the regions on the three-dimensional image to generate and display the freshness image.

[0071] In this embodiment, the type of the food material can be a further fine division of the food material based on the name of the food material. For example, the name of the food material can be orange, and correspondingly, the type of the food material can be sugar orange, blood orange, navel orange, etc.

[0072] The three-dimensional image can be a three-dimensional simulation image of the food material generated by a computer.

[0073] The database image can refer to an image that belongs to the same category as the food material and is most similar to the food material image as a whole. For example, the food material can be a mature but slightly deteriorated ice sugar orange in some parts, and accordingly, the queried database image should also be an image of an ice sugar orange with similar degree of deterioration.

[0074] Since some local positions of the food material in the food material image can be blocked, by querying the most similar image and using the most similar image to complete the blocked parts of the food material in the stereoscopic image, the real situation of the food material can be restored to the greatest extent, and the accuracy and reliability of the freshness judgment of different regions of the food material are improved.

[0075] In this way, the food material and the freshness of the food material can be more intuitively displayed by the stereoscopic image, so that the user can view the food material without opening the door and understand the freshness and deterioration of the food material.

[0076] Referring to Figure 2 S10, further, in an embodiment of the present application, the stereoscopic image can be a point cloud image, and the freshness of the regions can be indicated with different point densities. Further, in this embodiment, the freshness of different regions of the food material can also be further clearly presented with different colors. For example, the greater the density of the points, the deeper the color. In this way, the food material and the freshness of the food material can be more intuitively displayed, so that the user can view the food material without opening the door and understand the freshness and deterioration of the food material.

[0077] Referring to Figure 2 S7 and Figure 3 , further, in an embodiment of the present application, the refrigerator food material management method, wherein the control method of "selecting a plurality of regions of the stereoscopic image and obtaining the region images corresponding to the regions" can include:

[0078] dividing the stereoscopic image into a plurality of collection regions;

[0079] obtaining a main color of the collection region, the main color being the color with the largest proportion in the collection region;

[0080] randomly selecting a plurality of non-intersecting regions from the collection region;

[0081] calculating the color difference between the regions and the main color of the collection region, respectively;

[0082] when the color difference exceeds a preset color difference value, selecting the regions and obtaining the region images corresponding to the regions.

[0083] The collection area can be a divided area on the outer surface of the stereoscopic image. The sizes of the collection areas can be the same or different. For example, collection areas under six different perspectives of top, bottom, left, right, front, and back can be divided on the stereoscopic image of the food material.

[0084] Since the different freshness of the food material is often reflected by the change of the appearance color. For example, the mold spots of bluish black color can appear at the damaged place of the food material, and the color can be more saturated at the mature place. Therefore, we can select the area with color change as the focus. By comparing the color difference between the local and the whole, the most representative area can be obtained.

[0085] In this way, the accuracy of the selection of the area image for judging the freshness of the food material can be improved, so as to more comprehensively and accurately determine the freshness of different positions of the food material, and improve the efficiency and reliability of the identification.

[0086] With reference to Figure 2 S4, S8, S9, further, in an embodiment of the present application, the refrigerator food material management method, wherein the control method of "determining the freshness of the area according to the area image and the information that the food material is in different freshness" can include:

[0087] obtaining the image of each freshness stage of the food material;

[0088] respectively comparing the area image with the image of the food material in different freshness;

[0089] respectively determining the freshness of the area according to the comparison result.

[0090] In this way, the freshness of the specific area of the food material can be obtained by picture comparison, which is simple, fast, accurate, and easy to control.

[0091] With reference to Figure 2 S11-S16, further, in an embodiment of the present application, the refrigerator food material management method, wherein the control method can further include:

[0092] comprehensively determining whether the food material is edible according to the freshness of all the areas;

[0093] if yes, comprehensively obtaining the edible time of the food material according to the freshness of all the areas and the information of each freshness stage of the food material, and issuing a prompt information;

[0094] if no, issuing a prompt information that the food material is not edible.

[0095] This setup allows for a more accurate determination of an ingredient's edibility by comprehensively considering its overall freshness information. For example, it can determine whether an ingredient is entirely edible, partially edible, or inedible. Furthermore, if an ingredient is edible, the system can predict its shelf life based on its overall freshness information, making the prediction more accurate and reliable.

[0096] Reference Figure 2 Further, in one embodiment of the present invention, the refrigerator food management method, as described in S17-S18, may further include the control method comprising:

[0097] The appropriate storage environment for the ingredients is determined by comprehensively considering the freshness of all the regions and the freshness information of each stage of the ingredients.

[0098] Adjust the environment at the location where the ingredients are stored to a suitable storage environment.

[0099] This setup allows for more accurate adjustment of the storage environment by comprehensively considering the overall freshness information of the ingredients, thereby extending their shelf life and preventing waste. For example, the freshness information of different areas of the ingredients can be used to determine their ripeness, and then the humidity and temperature that will allow them to be stored for a longer period can be determined based on that ripeness.

[0100] Reference Figure 2 S13-S16, further, in one embodiment of the present invention, the refrigerator food management method, wherein,

[0101] When the refrigerator is in use, it will announce the consumption information of the ingredients via voice.

[0102] In this embodiment, the refrigerator can announce the consumption information of each ingredient to the user one by one after being activated by voice. It can also announce specific ingredients according to the user's request. For example, if the user says "orange," the refrigerator can announce according to the current freshness of the orange: "The orange is not spoiled at present and can be stored for three days. Please eat it as soon as possible." At the same time, it can display an image of the orange's freshness on the display screen so that the user can have a more comprehensive, intuitive, and accurate understanding of the current condition of the ingredients.

[0103] This setup allows users to check the ingredients without opening the door, understanding their freshness and spoilage status. This enables users to consume the ingredients before they spoil and accurately locate the spoiled parts if they have already spoiled, avoiding waste and preventing users from consuming spoiled ingredients, thus improving the user experience.

[0104] In an embodiment of the present application, a food material management system can comprise a control module, an image recognition module, wherein the food material management system can further comprise a visual module, the food material management system can further comprise a memory and a processor, the memory stores a computer program capable of running on the processor, and when the processor executes the computer program, the steps of the food material management method described in any of the above embodiments are implemented.

[0105] In this way, the freshness of food materials at different positions can be more comprehensively and accurately determined, the food materials can be more accurately managed, the freshness of the food materials can be intuitively displayed, and the user can view the food materials without opening the door, and understand the freshness and deterioration of the food materials.

[0106] In an embodiment of the present application, a computer readable storage medium has a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of the food material management method described in any of the above embodiments are implemented.

[0107] In this way, the freshness of food materials at different positions can be more comprehensively and accurately determined, the food materials can be more accurately managed, the freshness of the food materials can be intuitively displayed, and the user can view the food materials without opening the door, and understand the freshness and deterioration of the food materials.

[0108] In summary, the refrigerator of the present application can solve the problem that the function of voice reminding of expired food materials of the existing refrigerator can only obtain the storage time of food materials according to theoretical common sense, and cannot feedback the actual storage time of food materials.

[0109] By adopting the technical solution in the present application, the freshness of food materials at different positions can be more comprehensively and accurately determined, the food materials can be more accurately managed, the freshness of the food materials can be intuitively displayed, the voice reminding function and the temperature control system are combined, the damaged food materials can be reminded in real time, the user can eat the food materials in time before the food materials deteriorate, the position of the deteriorated food materials can be accurately found after the food materials deteriorate, and the deterioration of the food materials is delayed by the temperature control system, which is beneficial to keeping the food materials fresh for a longer time, avoids waste and the user eating the deteriorated food materials, and improves the user experience.

[0110] It should be understood that although the present specification is described in terms of embodiments, each embodiment does not necessarily contain only one independent technical solution, and the specification is described in this way only for the sake of clarity, and those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can be appropriately combined to form other embodiments that those skilled in the art can understand.

[0111] The above detailed description merely illustrates preferred and possible non-limiting implementations of the application, and is not intended to limit the scope of the application. Other equally effective embodiments and modifications will be apparent to those skilled in the art, and are intended to be included within the scope of the application.

Claims

1. A refrigerator foodstuff management method, characterized by, The method comprises: acquiring an image of a food material stored in a refrigerator and a corresponding name of the food material; acquiring information of each freshness stage of the food material; determining the freshness of several regions of the food material according to the image of the food material and the information of each freshness stage of the food material; generating and displaying a freshness image indicating the freshness of the several regions of the food material; acquiring a category of the food material; querying a database image corresponding to the category of the food material, the database image being an image of the same category as the food material and most similar to the image of the food material as a whole; generating a three-dimensional image of the food material according to the image of the food material and the database image, the three-dimensional image being a three-dimensional simulation image of the food material, and the database image being used to complete a part of the three-dimensional image that is blocked by the food material; selecting several regions of the three-dimensional image and acquiring a region image corresponding to each of the regions; determining the freshness of the regions according to the region images and the information of each freshness stage of the food material; indicating the freshness of the regions on the three-dimensional image to generate and display the freshness image. 2.The refrigerator food management method of claim 1, wherein, The control method of "selecting several regions of the three-dimensional image and acquiring a region image corresponding to each of the regions" comprises: dividing the three-dimensional image into several collection areas; acquiring a main color of the collection areas, the main color being a color with the largest proportion in the collection areas; randomly selecting several non-intersecting regions from the collection areas; calculating a color difference between the regions and the main color of the collection areas, respectively; when the color difference exceeds a preset color difference value, selecting the regions and acquiring the region image corresponding to each of the regions. 3.The refrigerator food management method of claim 1, wherein, The control method of "determining the freshness of the regions according to the region images and the information of each freshness stage of the food material" comprises: acquiring images of each freshness stage of the food material; comparing the region images with the images of each freshness stage of the food material, respectively; determining the freshness of the regions according to the comparison results, respectively. 4.The refrigerator food management method of claim 3, wherein, The control method further comprises: comprehensively determining whether the food material is edible according to the freshness of all the regions; if yes, comprehensively acquiring an edible time of the food material according to the freshness of all the regions and the information of each freshness stage of the food material and issuing a prompt information; if no, issuing a prompt information that the food material is not edible. 5.The refrigerator food management method of claim 3, wherein, The control method further comprises: comprehensively determining a suitable storage environment of the food material according to the freshness of all the regions and the information of each freshness stage of the food material; adjusting an environment at a location where the food material is stored to the suitable storage environment.

6. The refrigerator food material management method according to claim 4, wherein when the refrigerator is in a use state, the edible information of the food material is broadcasted by voice.

7. The refrigerator food material management method according to claim 1, wherein the three-dimensional image is a point cloud image, and different point densities of the several regions indicate the freshness of the several regions. The computer program is executed by a processor to implement the steps of the food material management method in any one of claims 1-7. ​ 8. A computer-readable storage medium having stored thereon a computer program, characterized in that, ​ 9. A food management system comprising a control module, an image recognition module, characterized in that, The food material management system further comprises a visual module, and further comprises a memory and a processor, the memory stores a computer program capable of running on the processor, and the processor implements the steps in the food material management method of any one of claims 1-7 when executing the computer program. 10.A refrigerator comprising a storage compartment for storing foodstuffs and a foodstuff management system, the foodstuff management system comprising a control module, an image recognition module, a display module, characterized in that, The food material management system further comprises a memory and a processor, and the memory stores a computer program capable of running on the processor, and the processor implements the steps in the food material management method of any one of claims 1-7 when executing the computer program.

Images