Food management methods, equipment and refrigerators

Abstract

The present invention discloses a method, device and refrigerator for managing food materials, the method comprising: after receiving an instruction to print a label, taking an image of a refrigerator area and detecting food materials; for each new food material detected, identifying and recording the name of each new food material one by one, wherein the new food material refers to food materials without a label; printing a label for each new food material, the label comprising a QR code and text recording the name of the corresponding new food material and the current date. Compared with the prior art, the food material management method of the present invention obtains the food material name by identifying the food material, and then prints the food material name and storage date in the form of a label, which is pasted on the top of the corresponding food material, so that the computer and the user can obtain the food material name and the storage time of the refrigerator through the label, which is convenient for the computer and the user to manage the food material.

Description

Technical Field

[0001] The present invention relates to the field of household appliances, and in particular to a food management method, equipment and refrigerator. Background Art

[0002] The Internet of Things (IoT) is a crucial component of the new generation of information technology and a crucial developmental stage in the "informatization" era. It's even been called the third wave of development in the global information industry, following computers and the internet. In recent years, with the rise of the IoT concept, the home appliance industry has rapidly shifted toward intelligent development, with many traditional appliance companies jumping into the smart home development game. Since the rise of the smart home, numerous smart home products have emerged on the market, with the smart refrigerator being one of them.

[0003] A smart refrigerator is one that offers intelligent control and food management. Specifically, it can automatically switch refrigerator modes to keep food in optimal storage conditions. Users can access the quantity and quality of food in the refrigerator anytime, anywhere via their phone or computer. It can also provide healthy recipes and nutritional recommendations, and remind users to restock their food regularly.

[0004] However, many problems still exist in the implementation of smart refrigerators. For example, when it comes to managing the expiration dates of ingredients, because the production date and expiration date are not fixed in place (some are located at the bottom of the ingredients), the computer cannot accurately obtain the production date and expiration date of each ingredient, thus hindering the user experience. Summary of the Invention

[0005] The object of the present invention is to provide a food management method, equipment and refrigerator.

[0006] To achieve one of the above-mentioned objectives, an embodiment of the present invention provides a method for managing food materials, the method comprising:

[0007] After receiving the label printing instruction, the refrigerator area is imaged and food ingredients are inspected;

[0008] For new ingredients detected, identify and record the name of each new ingredient one by one, wherein the new ingredient refers to an ingredient without a label;

[0009] A label is printed for each of the new ingredients, wherein the label includes a QR code and text recording the name of the corresponding new ingredient and the current date.

[0010] As a further improvement of one embodiment of the present invention, the method further includes:

[0011] Identify the storage date of the corresponding ingredients through the QR code on the label and calculate the storage time of the ingredients;

[0012] If the storage time reaches a predetermined period, the user is reminded to process the corresponding food.

[0013] As a further improvement of one embodiment of the present invention, the method further includes:

[0014] After the user stores new ingredients, he or she can issue label printing instructions through a specific button set on the refrigerator.

[0015] As a further improvement of one embodiment of the present invention, the method further includes:

[0016] After receiving the printed label, the user sticks the label on the top of the corresponding ingredient according to the name on the label.

[0017] As a further improvement of an embodiment of the present invention, the “capturing images of the refrigerator area and detecting food ingredients” includes:

[0018] Control the camera on the top of the refrigerator area to capture images and obtain the image to be detected;

[0019] Perform fixed region cropping on the image to be detected to obtain a special image region where dairy products are placed, thereby obtaining a first image region;

[0020] Comparing the first image region with the empty bottle seat image region to obtain a changed second image region;

[0021] Performing shape detection on the second image area, and passing the detected shape area into the YOLO algorithm for recognition detection to obtain multiple ingredients;

[0022] Label detection is performed on the multiple ingredients, and for ingredients whose labels are not detected, they are marked as new ingredients.

[0023] As a further improvement of an embodiment of the present invention, the step of “comparing the first image region with the empty bottle holder image region to obtain a changed second image region” specifically includes:

[0024] The RGB value of the first image area is compared with the RGB value of the empty bottle seat image area to obtain an area where the RGB value changes, that is, the second image area.

[0025] As a further improvement of an embodiment of the present invention, the step of “identifying and recording the name of each detected new ingredient one by one” includes:

[0026] Through the pre-trained image recognition model, each new ingredient is recognized by image, and the name of each new ingredient is obtained and recorded.

[0027] As a further improvement of an embodiment of the present invention, the label further includes an ingredient ID, and the ingredient ID corresponds one-to-one with the ingredient name and storage date corresponding to the label.

[0028] To achieve one of the above-mentioned purposes of the invention, one embodiment of the present invention provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the program, the steps in any one of the above-mentioned food management methods are implemented.

[0029] To achieve one of the above-mentioned objects of the invention, one embodiment of the present invention provides a refrigerator, which includes the electronic device as described above.

[0030] Compared with the prior art, the food management method of the present invention obtains the name of the ingredient by identifying the ingredient, and then prints the ingredient name and storage date in the form of a label and sticks it on the top of the corresponding ingredient, so that the computer and the user can obtain the ingredient name and the time of storage in the refrigerator through the label, which is convenient for the computer and the user to manage the ingredients. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a schematic flow chart of the food management method of the present invention;

[0032] Figure 2 is a detailed flow chart of step S100 of the present invention;

[0033] Figure 3 is a specific example of the image to be detected in the present invention;

[0034] Among them, 1. the entire area of ​​the image to be detected; 2. the bottle seat area; 3. the dairy products placed in the bottle seat area. DETAILED DESCRIPTION

[0035] The present invention will be described in detail below with reference to the specific embodiments shown in the accompanying drawings. However, these embodiments do not limit the present invention, and any structural, methodological, or functional changes made by those skilled in the art based on these embodiments are all within the scope of protection of the present invention.

[0036] A dedicated refrigerator section is set up in the refrigerator to store dairy products, allowing for centralized monitoring of these products. This section typically includes a bottle holder, where dairy products are placed. Monitoring the dairy products in the bottle holder requires capturing and identifying images of the refrigerator section. Currently, cameras in this section are typically located at the top of the refrigerator section. A pre-trained image recognition model then identifies images of the tops of dairy products (e.g., bottle caps) captured by the camera to determine the category or name of the dairy product.

[0037] Because dairy products have a short shelf life, after being stored in the refrigerator, users need to be reminded to dispose of the dairy products within the expiration date. Normally, the expiration of dairy products can be monitored by the production date and expiration date on the dairy products. However, since the production date and expiration date are not usually displayed on the top of the dairy products, the camera on the top of the refrigerator cannot obtain this information.

[0038] Therefore, the present invention provides a method for managing food ingredients, wherein the food ingredients refer to dairy products. The management method obtains the name of the food ingredients by identifying the food ingredients, and then prints the name of the food ingredients and the storage date in the form of a label, and sticks it on the top of the corresponding food ingredients, so that the computer and the user can obtain the name of the food ingredients and the time of storage in the refrigerator through the label, which is convenient for the computer and the user to manage the food ingredients.

[0039] like Figure 1 As shown, the method includes:

[0040] Step S100: After receiving the label printing instruction, the refrigerator area is imaged and food is detected.

[0041] After the user stores the food, he needs to issue a label printing instruction to the refrigerator. It should be noted that a specific button is provided on the refrigerator, through which the user can issue a label printing instruction.

[0042] After receiving the label printing instruction, it is necessary to identify the food that the user has stored in the refrigerator this time, that is, the new food, through the image captured by the camera. Since the food stored in the refrigerator before has been labeled, Figure 2 As shown, the refrigerator area is imaged and food is detected, including:

[0043] Step S110: Control the camera on the top of the refrigerator area to capture images to obtain images to be detected.

[0044] Step S120: performing fixed region cropping on the image to be detected, obtaining a special image region where dairy products are placed, and obtaining a first image region.

[0045] like Figure 3As shown, the image to be detected is the overall image 1 of the refrigerator area taken by the corresponding camera, which includes the bottle seat area 2 and the area around the bottle seat. In order to eliminate unnecessary data as much as possible, the image to be detected is cropped in a fixed area, retaining only the ROI area (i.e., the image area where dairy products are placed) to obtain the first image area.

[0046] It should be noted that, since the camera's shooting angle is fixed, the cropping of the first image area can be a fixed cropping (i.e., the same area is cropped each time). In addition, for images in which no objects are placed in the bottle holder, a fixed cropping can be performed to obtain a corresponding empty bottle holder image area for comparison with the first image area.

[0047] Step S130: Compare the first image region with the empty bottle holder image region to obtain a changed second image region.

[0048] Since the background of the special image area where dairy products are placed is the same (all are empty bottle holders), in order to quickly detect the location of the dairy products, the picture with dairy products (first picture area) can be compared with the picture without dairy products (empty bottle holder picture area) to obtain the changed area, which is the second picture area.

[0049] It should be noted that a fixed area of ​​the image without dairy products can be cropped to obtain the empty bottle holder image area.

[0050] In a preferred embodiment, the step of “comparing the first image region with the empty bottle holder image region to obtain a changed second image region” specifically includes:

[0051] The RGB value of the first image area is compared with the RGB value of the empty bottle seat image area to obtain an area where the RGB value changes, that is, the second image area.

[0052] The RGB color model is an industry color standard that creates a wide variety of colors by varying and superimposing the three color channels: red (R), green (G), and blue (B). RGB represents the colors of the three channels, red, green, and blue. This standard encompasses nearly every color perceived by human vision and is one of the most widely used color systems. Each pixel in each image has a corresponding value in the red, green, and blue channels, known as its RGB value. Therefore, for pixels located in the same position in the first image area and the empty bottle seat image area, their RGB values ​​are compared. The area containing pixels whose RGB values ​​have changed is the second image area.

[0053] Step S140: performing shape detection on the second image area, passing the detected shape area into the Yolo algorithm for recognition detection, and obtaining multiple ingredients.

[0054] The shape detection refers to detecting whether there are various shapes in the second image area, such as circle, rectangle, triangle, etc. Since the top shape of most dairy products on the market is circular or rectangular (such as Figure 3 As shown, the top shape of the milk product placed in the bottle holder is circular and rectangular), so the shape detection here is preferably circular detection and rectangular detection.

[0055] Directly performing shape detection on the changed area can greatly improve the detection efficiency.

[0056] Through circle detection and rectangle detection, the image area that meets the requirements is passed to the YOLO algorithm for recognition and detection to obtain the detection results.

[0057] Step S150: performing label detection on the plurality of ingredients, and marking the ingredients without detected labels as new ingredients.

[0058] Step S200: For the detected new ingredients, identify and record the name of each new ingredient one by one, wherein the new ingredients refer to ingredients without labels.

[0059] Through the pre-trained image recognition algorithm, each new ingredient detected is identified one by one and the name of each new ingredient is recorded.

[0060] Step S300: Print a label for each of the new ingredients, wherein the label includes a QR code and text recording the name of the corresponding new ingredient and the current date.

[0061] After obtaining the names of all new ingredients, the name of each new ingredient and the current date are printed out in the form of a label. The label includes a QR code and text. The QR code is convenient for the computer to identify, and the text is convenient for the user to view, thereby making it easier for the computer and the user to manage the ingredients.

[0062] It should be noted that, in order to print labels, a dedicated label printing system can be provided on the refrigerator, or a label printing system can be connected to the refrigerator in an external manner. The label printing system prints a corresponding label for each new ingredient based on the printing information received from the refrigerator.

[0063] In a preferred embodiment, the method further comprises:

[0064] After receiving the printed label, the user sticks the label on the top of the corresponding ingredient according to the name on the label.

[0065] In another preferred embodiment, the method further comprises:

[0066] Identify the storage date of the corresponding ingredients through the QR code on the label and calculate the storage time of the ingredients;

[0067] If the storage time reaches a predetermined period, the user is reminded to process the corresponding food.

[0068] For food stored in the refrigerator, you can set different deadlines based on the food type. For example, for yogurt with a shelf life of only seven days, you can set a reminder to consume it after three days. For yogurt with a shelf life of 14 days, you can set a reminder to consume it after one week.

[0069] In order to facilitate the computer to distinguish the same type of yogurt stored on the same day, in a preferred embodiment, the label also includes an ingredient ID, and the ingredient ID is unique, and the ingredient ID corresponds one-to-one with the ingredient name and storage date corresponding to the label.

[0070] In a specific embodiment, a refrigerator is provided with a dedicated area for storing dairy products, the dedicated area including a bottle holder, in which the dairy products are placed. A camera is provided on the top of the bottle holder for monitoring the dairy products in the bottle holder. After the user places a new dairy product, a label printing instruction is issued through a specific button provided on the refrigerator. After receiving the printing instruction, the refrigerator controls the camera to take images and detect the ingredients. For new ingredients detected without labels, the name of each new ingredient is identified and recorded one by one; then a label is printed for each new ingredient, the label including a QR code and text recording the name of the corresponding new ingredient and the current date. Finally, after receiving the printed label, the user sticks the label on the top of the corresponding ingredient using the name on the label.

[0071] The food management method of the present invention obtains the name of the food by identifying the food, and then prints the food name and storage date in the form of a label, which is pasted on the top of the corresponding food. In this way, the computer and the user can obtain the food name and the time of storage in the refrigerator through the label, which is convenient for the computer and the user to manage the food.

[0072] The present invention also provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the program, it implements any one of the steps in the above-mentioned food management method, that is, it implements the steps in any one of the technical solutions in the above-mentioned food management method.

[0073] The present invention also provides a refrigerator, which includes the electronic device described above.

[0074] It should be understood that although this specification is described in terms of implementation methods, not every implementation method contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each implementation method can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

[0075] The series of detailed descriptions listed above are only specific descriptions of feasible implementation methods of the present invention. They are not intended to limit the scope of protection of the present invention. Any equivalent implementation methods or changes that do not deviate from the technical spirit of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for managing food materials, characterized in that: The method comprises: After receiving the label printing instruction, the refrigerator area is imaged and food ingredients are inspected; For new ingredients detected, identify and record the name of each new ingredient one by one, wherein the new ingredient refers to an ingredient without a label; Printing a label for each new ingredient, wherein the label includes a QR code and text recording the name of the corresponding new ingredient and the current date; The "capturing images of the refrigerator area and detecting food ingredients" includes: Control the camera on the top of the refrigerator area to capture images and obtain the image to be detected; Perform fixed region cropping on the image to be detected to obtain a special image region where dairy products are placed, thereby obtaining a first image region; Comparing the first image region with the empty bottle seat image region to obtain a changed second image region; Performing shape detection on the second image area, and passing the detected shape area into the YOLO algorithm for recognition detection to obtain multiple ingredients; Label detection is performed on the multiple ingredients, and for ingredients whose labels are not detected, they are marked as new ingredients.

2. The food management method according to claim 1, characterized in that: The method further comprises: Identify the storage date of the corresponding ingredients through the QR code on the label and calculate the storage time of the ingredients; If the storage time reaches a predetermined period, the user is reminded to process the corresponding food.

3. The food management method according to claim 1, characterized in that: The method further comprises: After the user stores new ingredients, he or she can issue label printing instructions through a specific button set on the refrigerator.

4. The food management method according to claim 1, characterized in that: The method further comprises: After receiving the printed label, the user sticks the label on the top of the corresponding ingredient according to the name on the label.

5. The food management method according to claim 1, characterized in that: The “comparing the first image area with the empty bottle seat image area to obtain a changed second image area” specifically includes: The RGB value of the first image area is compared with the RGB value of the empty bottle seat image area to obtain an area where the RGB value changes, that is, the second image area.

6. The food management method according to claim 1, characterized in that: The “identifying and recording the name of each detected new ingredient one by one” includes: Through the pre-trained image recognition model, each new ingredient is recognized by image, and the name of each new ingredient is obtained and recorded.

7. The food management method according to claim 1, characterized in that: The label also includes an ingredient ID, and the ingredient ID corresponds one-to-one with the ingredient name and storage date corresponding to the label.

8. An electronic device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, wherein: When the processor executes the program, the steps of the food management method according to any one of claims 1 to 7 are implemented.

9. A refrigerator, characterized in that: The refrigerator comprises the electronic device according to claim 8.

Images