Food material identification method combining motion and stillness, storage medium and refrigeration equipment

Abstract

The application provides a dynamic and static combined food material recognition method, a storage medium and a refrigeration equipment. The dynamic and static combined food material recognition method comprises the following steps: detecting the position and movement direction of a door body; controlling a first camera to shoot a video, the shooting area of the first camera covering the inside space of the door body, obtaining first food material information of food materials stored and taken out from the inside space of the door body according to the video shot by the first camera; controlling a second camera to shoot a photo and store it, and obtaining second food material information of food materials stored and taken out from the inside space of the door body according to the photo shot by the second camera; and obtaining food material information stored and taken out from the inside space of the door body according to the first food material information and the second food material information. The storage medium and the refrigeration equipment can realize the above-mentioned dynamic and static combined food material recognition method. In this way, the accuracy and reliability of the food material recognition result of the refrigeration equipment can be improved.

Description

Technical Field

[0001] This invention relates to the field of household appliances, and more particularly to a method for identifying food ingredients that combines static and dynamic elements, a storage medium, and a refrigeration device. Background Technology

[0002] With technological advancements, users have increasingly higher demands for refrigeration equipment, making intelligent transformation a new research and development direction for refrigeration equipment. Refrigeration equipment typically incorporates an image recognition system. This system usually includes a camera installed within the refrigeration unit. By recognizing photos of food stored inside the refrigeration compartment taken by the camera, the system can identify the food stored within, enabling intelligent management of the food. For example, it can inform users about the practical uses of the food and remind them to consume it promptly.

[0003] Current image recognition systems for refrigeration equipment typically use a single camera to identify a specific area. For example, a single camera might be placed on top of the bottle holder to monitor the area, or a single camera on the cabinet or door to monitor the storage and retrieval of food on multiple shelves inside the cabinet. However, this design has the following drawbacks: when there is obstruction, problems such as incomplete or incorrect food identification may occur. Users may obstruct the camera during use, and food items may also obstruct each other. If only a single camera is used to identify a specific area, such as the refrigerator compartment, the refrigeration equipment may misidentify or miss items due to mutual obstruction.

[0004] Therefore, existing image recognition systems may cause errors in the management of food information within refrigeration equipment, resulting in inaccurate food information provided to users. Summary of the Invention

[0005] The purpose of this invention is to provide a method, storage medium, and refrigeration device for identifying food ingredients by combining static and dynamic methods. By setting different cameras to cross-identify food ingredients at the same location of the refrigeration device from different angles and with different shooting methods, the accuracy and reliability of the food ingredient identification results of the refrigeration device can be improved.

[0006] To achieve the above-mentioned objective, one embodiment of the present invention provides a method for identifying food ingredients that combines static and dynamic elements, wherein the identification method includes:

[0007] Detect the position and direction of movement of the gate;

[0008] When the door is detected to have moved to a first preset position along the opening direction, the first camera is controlled to capture video, and the shooting area of ​​the first camera covers the inner space of the door.

[0009] When the door is detected to have moved to a second preset position along the closing direction, the first camera is controlled to stop recording video.

[0010] The first ingredient information of the ingredients stored and retrieved in the inner space of the door is obtained based on the video captured by the first camera.

[0011] When the door is detected to have moved to a third preset position in the closing direction, the second camera is controlled to take a picture and store it. When the door moves to the third preset position, the inner space of the door enters the shooting area of ​​the second camera.

[0012] The second ingredient information of the ingredients stored and retrieved in the inner space of the door is obtained based on the photos taken by the second camera.

[0013] Based on the first ingredient information and the second ingredient information, obtain the ingredient information stored and retrieved from the inner space of the door.

[0014] As a further improvement to one embodiment of the present invention, it further includes:

[0015] When the door is detected to have moved to the fourth preset position along the opening direction, the second camera is controlled to capture video.

[0016] When the door is detected to have moved to the fifth preset position in the closing direction, the second camera is controlled to stop shooting video, and the shooting area of ​​the second camera covers the opening area of ​​the refrigeration equipment storage room;

[0017] Third food information is obtained from the video captured by the second camera, showing the food items stored and retrieved in the storage room.

[0018] The food information inside the refrigeration equipment is obtained based on the first food information, the second food information, and the third food information.

[0019] As a further improvement to one embodiment of the present invention, it further includes:

[0020] Obtain information about the ingredients stored in the storage room;

[0021] Obtain information about the ingredients retrieved from the inner space of the door;

[0022] Determine whether the food stored in the storage room contains food taken from the space inside the door;

[0023] If so, then delete the food items taken from the inner space of the door after deleting the food items stored in the storage room and mark them as stored food items information;

[0024] If not, the food stored in the storage room will be directly marked as the stored food information;

[0025] Obtain the food information stored in the inner space of the door;

[0026] Obtain information about the ingredients retrieved from the storage room;

[0027] Determine whether the food stored in the inner space of the door contains food taken from the storage room.

[0028] If so, after deleting the food information taken from the storage room from the food information stored in the food information inside the door, mark it as the stored food information;

[0029] If not, the food information stored in the inner space of the door body will be directly marked as the stored food information.

[0030] As a further improvement to one embodiment of the present invention, it further includes:

[0031] Obtain information about the ingredients retrieved from the storage room;

[0032] Obtain the food information stored in the inner space of the door;

[0033] Determine whether the food taken out from the storage room contains food stored in the space inside the door;

[0034] If so, the information on the ingredients taken from the storage room is deleted from the information on the ingredients stored in the space inside the door, and then marked as the information on the ingredients taken out;

[0035] If not, the information of the food ingredients taken from the storage room will be directly marked as the information of the taken-out food ingredients;

[0036] Obtain information about the ingredients retrieved from the inner space of the door;

[0037] Obtain information about the ingredients stored in the storage room;

[0038] Determine whether the food ingredients taken out from the inner space of the door contain the food ingredients stored in the storage room.

[0039] If so, the information on the ingredients taken out from the inner space of the door is deleted from the information on the ingredients stored in the storage room and then marked as the information on the ingredients taken out;

[0040] If not, the information of the food ingredients taken out from the inner space of the door will be directly marked as the information of the food ingredients taken out.

[0041] As a further improvement to one embodiment of the present invention, it further includes:

[0042] Mark the stored ingredient information obtained each time and delete the retrieved ingredient information each time to obtain the existing ingredient information.

[0043] As a further improvement to one embodiment of the present invention, the control method includes:

[0044] When the door is detected to have moved to the sixth preset position in the closing direction, the third camera is controlled to take a picture and store it. When the door moves to the sixth preset position, the shooting area of ​​the third camera covers the first storage space above the first shelf in the storage room.

[0045] The fourth ingredient information of the ingredients stored and retrieved in the first storage space is obtained based on the photos taken by the third camera.

[0046] Information on ingredients stored in and retrieved from the storage room is obtained based on the third and fourth ingredient information.

[0047] As a further improvement to one embodiment of the present invention, the control method includes:

[0048] When the door is detected to have moved to the seventh preset position along the opening direction, the fourth camera is controlled to capture video. The shooting area of ​​the fourth camera covers the opening area of ​​the drawer of the storage room when it is in the open state.

[0049] When the door is detected to have moved to the eighth preset position along the closing direction, the fourth camera is controlled to stop recording video.

[0050] The fifth ingredient information is obtained from the video captured by the fourth camera, which shows the ingredients stored and retrieved from the drawer.

[0051] Information on ingredients stored in and retrieved from the storage room is obtained based on the third, fourth, and fifth ingredient information.

[0052] To achieve the above-mentioned objectives, one embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the dynamic and static combined food identification method described in any of the above embodiments.

[0053] To achieve the above-mentioned objectives, one embodiment of the present invention provides a refrigeration device, which includes a storage compartment and a door for opening and closing the storage compartment. The device also includes a food identification system, comprising a control module, an image recognition module, and a detection module. The food identification system further includes a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the computer program, it implements the steps of the dynamic and static combined food identification method described in any of the above embodiments.

[0054] As a further improvement of one embodiment of the present invention, the first camera is disposed on the upper part of the inner side of the door body, the first preset position is any position where the door body and the opening of the storage room are at an angle of 0 degrees to 30 degrees, and the second preset position is any position where the door body and the opening of the storage room are at an angle of 0 degrees to 30 degrees.

[0055] As a further improvement of one embodiment of the present invention, the second camera is disposed on the top wall of the storage room and close to the opening of the storage room; the third preset position is any position where the door and the opening of the storage room are at an angle of 0 degrees to 90 degrees; the fourth preset position is any position where the door and the opening of the storage room are at an angle of 0 degrees to 30 degrees; and the fifth preset position is any position where the door and the opening of the storage room are at an angle of 0 degrees to 30 degrees.

[0056] As a further improvement of one embodiment of the present invention, the third camera is disposed in the storage room and above the first shelf, and the sixth preset position is any position where the door and the opening of the storage room are at an angle of 0 degrees to 30 degrees.

[0057] As a further improvement of one embodiment of the present invention, the third camera is disposed on the door and cooperates with the first shelf, and the sixth preset position is any position where the door and the opening of the storage room are at an angle of 0 degrees to 30 degrees.

[0058] As a further improvement of one embodiment of the present invention, the fourth camera is disposed on the left and right side walls of the storage room, the seventh preset position is any position where the door and the opening of the storage room are at an angle of 0 degrees to 60 degrees, and the eighth preset position is any position where the door and the opening of the storage room are at an angle of 0 degrees to 60 degrees.

[0059] Compared with existing technologies, the present invention uses different cameras to cross-identify food items at the same location of the refrigeration equipment from different angles and with different shooting methods. Its beneficial effect is that it can improve the accuracy and reliability of food identification results of the refrigeration equipment. Attached Figure Description

[0060] Figure 1 This is a schematic diagram of the structure of the refrigeration equipment according to the first embodiment of the present invention;

[0061] Figure 2 yes Figure 1 A schematic diagram of the refrigeration equipment from another perspective;

[0062] Figure 3 This is a schematic diagram of the structure of the refrigeration device according to the second embodiment of the present invention;

[0063] Figure 4 This is a schematic diagram of the structure of the refrigeration equipment according to the third embodiment of the present invention;

[0064] Figure 5 This is a schematic diagram of the structure of the refrigeration device according to the fourth embodiment of the present invention;

[0065] Figure 6 This is a schematic diagram of the structure of the refrigeration device according to the fifth embodiment of the present invention;

[0066] Figure 7 This is a flowchart of a method for identifying ingredients that combines static and dynamic elements according to one embodiment of the present invention;

[0067] Figure 8 This is a flowchart illustrating how an embodiment of the present invention obtains information about a specific ingredient;

[0068] Figure 9 This is a flowchart of an embodiment of the present invention for obtaining food information for the entire storage room. Detailed Implementation

[0069] The present invention will now be described in detail with reference to the specific embodiments shown in the accompanying drawings. However, these embodiments do not limit the present invention, and any structural, methodological, or functional modifications made by those skilled in the art based on these embodiments are included within the scope of protection of the present invention.

[0070] Reference Figure 1 and Figure 2 Furthermore, in the first embodiment of the present invention, this embodiment provides a refrigeration device 100. The refrigeration device 100 may be a refrigerator, freezer, commercial goods display cabinet, etc. This application uses a refrigerator as an example to describe the refrigeration device 100.

[0071] The refrigeration equipment 100 includes a storage room 11, which is used for storing items. The refrigeration equipment 100 also includes a door 20 for opening and closing the storage room 11. The refrigeration equipment 100 also includes a food identification system, which includes a control module, an image recognition module 30, and a detection module 40.

[0072] In this embodiment, the refrigeration device 100 may include a housing 10. The storage compartment 11 may be formed inside the housing 10. The storage compartment 11 may be a refrigerator compartment, a freezer compartment, or another compartment of the refrigeration device 100 used for storing food. For example, it may be a compartment formed by an openable and closable door 20 of the refrigeration device 100.

[0073] In this embodiment, the storage compartment 11 refers to the refrigerator compartment formed inside the box body 10. The storage compartment 11 may have a storage compartment opening 111 for taking out and putting in food. The door 20 may be pivotally mounted on the box body 10 and used to open and close the storage compartment opening 111.

[0074] In this embodiment, the height can refer to the dimension along the vertical direction of the refrigeration device 100. The width can refer to the dimension along the horizontal direction of the storage compartment 11 when facing the opening 111. The depth can refer to the dimension along the front-back direction of the storage compartment 11 when facing the opening 111.

[0075] In this embodiment, the image recognition module 30 may include a first camera 31 and a second camera 32. The position of the door 20 may include a first preset position, a second preset position, and a third preset position.

[0076] In this embodiment, the refrigeration equipment 100 may further include a first shelf 12 and a second shelf 13 for storing food ingredients.

[0077] In this embodiment, the first shelf 12 can be disposed within the storage compartment 11 and spaced a certain distance from the storage compartment opening 111. The second camera 32 can be disposed on the top wall of the storage compartment 11 and close to the storage compartment opening 111. The lens of the second camera 32 can be located between the first shelf 12 and the storage compartment opening 111, and the lens of the second camera 32 can be tilted downwards and towards the door 20 to cover the opening area of ​​the storage compartment 11. The opening area can include the storage compartment opening 111 and the inner area of ​​the door 10 located outside the storage compartment opening.

[0078] The third preset position can be any position between 0 degrees and 90 degrees between the door 20 and the storage room opening 111. When the door 20 moves to the third preset position, the inner space of the door 20 enters the shooting area of ​​the second camera 32.

[0079] In this embodiment, the second shelf 13 can be disposed inside the door 20. Specifically, the second shelf 13 may include a multi-layered horizontally arranged bottle holder structure.

[0080] In this embodiment, the inner side of the door 20 can be the side of the door 20 facing the storage room opening 111 and sharing the cold air with the storage room 11.

[0081] In this embodiment, the inner space of the door 20 can refer to the storage space for storing food ingredients formed by the second shelf 13 located inside the door 20.

[0082] In this embodiment, the first camera 31 can be disposed on the side of the door 20 facing the storage room opening 111 and above the second shelf 13. The lens of the first camera 31 can face the second shelf 13 and the shooting area can cover the second shelf 13.

[0083] The first preset position can be any position between 0 degrees and 30 degrees between the door 20 and the storage room opening 111, and the second preset position can be any position between 0 degrees and 30 degrees between the door 20 and the storage room opening 111.

[0084] In this embodiment, the detection module 40 may include a door detection device 41. The door detection device 41 may be disposed on or around the door 20 and used to detect the position and direction of movement of the door 20. In this embodiment, the door detection device 41 may be an angle sensor disposed on the door 20 capable of detecting changes in the opening and closing angle of the door 20, or a distance sensor disposed on the housing 10 capable of sensing changes in the distance between the door 20 and the housing 10.

[0085] In this embodiment, the movement direction of the door 20 may include an opening direction and a closing direction. Movement of the door 20 in the opening direction can mean that the door 20 moves away from the storage compartment opening 111. Movement of the door 20 in the closing direction can mean that the door 20 moves towards the storage compartment opening 111.

[0086] In this embodiment, the door 20 can be a double door that rotates to open to the left and right sides of the cabinet 10. The door 20 may include a left door 21 and a right door 22. The left door 21 can be used to open and close the left half of the storage compartment opening 111 area, and the right door 22 is used to open and close the right half of the storage compartment opening 111 area.

[0087] The second shelf 13 may include a left door shelf 131 disposed on the left door body 21 and a right door shelf 132 disposed on the right door body 22.

[0088] The door detection device 41 may include a left door detection device 411 and a right door detection device 412.

[0089] The first camera 31 may include a left door camera 311 and a right door camera 312. The left door camera 311 may be located at the top corner of the left door body 21 away from the pivot axis of the left door body 21, and the right door camera 312 may be located at the top corner of the right door body 22 away from the pivot axis of the right door body 22.

[0090] Reference Figure 7 Furthermore, in this embodiment, a method for identifying food ingredients that combines static and dynamic elements is provided, the identification method comprising:

[0091] Detect the position and direction of movement of the gate 20;

[0092] When the door 20 is detected to have moved to a first preset position in the opening direction, the first camera 31 is controlled to capture video, and the shooting area of ​​the first camera 31 covers the inner space of the door 20.

[0093] When the door 20 is detected to have moved to the second preset position in the closing direction, the first camera 31 is controlled to stop recording video.

[0094] First ingredient information of ingredients stored and retrieved in the inner space of the door 20 is obtained based on the video captured by the first camera 31.

[0095] When the door 20 is detected to have moved to the third preset position in the closing direction, the second camera 32 is controlled to take a picture and store it. When the door 20 moves to the third preset position, the inner space of the door 20 enters the shooting area of ​​the second camera 32.

[0096] The second ingredient information of the ingredients stored and retrieved in the inner space of the door 20 is obtained based on the photos taken by the second camera 32.

[0097] Based on the first ingredient information and the second ingredient information, obtain the ingredient information stored and retrieved from the inner space of the door 20.

[0098] In this embodiment, during the time period between the door 20 moving in the opening direction to the first preset position and the door 20 moving in the closing direction to the second preset position, the first camera 31 can be controlled to capture video of the user dynamically taking and placing food from the inner space of the door 20, so as to understand the storage and retrieval of food in the inner space of the door 20.

[0099] When the door 20 moves to the third preset position in the closing direction, the second camera 32 can be controlled to take a photo of the food placed statically inside the door 20. By comparing the before and after images, the storage and retrieval of the food inside the door 20 can be understood.

[0100] In actual use, when users access food inside the door 20, their hands may block the view, which may cause the food recognition results of the video of dynamic food handling captured by the first camera to be inaccurate. When food is statically placed, there may be obstruction between the food, which may cause the food recognition results of the static photos of food captured by the second camera to be inaccurate, resulting in missed recognition and recognition errors.

[0101] This embodiment, by simultaneously setting up a first camera 31 and a second camera 32, enables mutual confirmation and supplementation of the food identification results from the video of dynamic food handling captured by the first camera 31 and the static food placement photos captured by the second camera 32. The static photos and dynamic videos are mutually verified, and the situation of food storage and handling in the inner space of the door 20 is known based on the combination of the identification results captured by the second camera 32 and the first camera 31. This reduces missed identification and incorrect identification caused by hand obstruction or food obstruction, and ensures the accuracy of the identification results.

[0102] Furthermore, in this embodiment, the method for obtaining the first ingredient information may include:

[0103] Obtain hand information from the video captured by the first camera 31;

[0104] Based on the hand information, obtain the food information carried by the hand in the video captured by the first camera 31 and mark it as the first food information;

[0105] Obtain the movement direction information of the hand carrying the food ingredient in the video captured by the first camera 31;

[0106] Based on the hand movement direction information, obtain information about the food being stored in the inner space of the door 20 or being retrieved from the inner space of the door 20, and mark it as the first food information.

[0107] In actual identification, the information of the hands in the video can be used to quickly and accurately locate the food information carried by the hands.

[0108] The direction of hand movement carrying the food can be used to determine whether the food is stored or taken out of the inner space of door 20. When the hand moves closer to the inner space of door 20, it can be determined that the food is stored inside door 20. When the hand moves further away from the inner space of door 20, it can be determined that the food is taken out of the inner space of door 20.

[0109] This setup enables accurate acquisition of the first ingredient information, and the control method is simple and easy to implement. Further, in this embodiment, the method for acquiring the second ingredient information includes:

[0110] The information of the food items taken from the inner space of the door 20 is obtained by comparing the photo currently taken by the second camera 32 with the previously stored photo taken by the second camera 32, and this information is marked as the second food item information.

[0111] This setup enables accurate acquisition of information about the second ingredient, and the control method is simple and easy to implement.

[0112] Furthermore, in this embodiment, the position of the door 20 may include a fourth preset position, a fifth preset position, and a closed position of the door 20.

[0113] In this embodiment, the fourth preset position can be any position where the door 20 and the storage compartment opening 111 are at an angle of 0 degrees to 30 degrees; the fifth preset position can be any position where the door 20 and the storage compartment opening 111 are at an angle of 0 degrees to 30 degrees. The closed position of the door 20 can be the position where the door 20 is completely closed on the storage compartment opening 111.

[0114] Reference Figure 8 Furthermore, in this embodiment, the method for identifying ingredients that combines static and dynamic elements also includes:

[0115] When the door 20 is detected to have moved to the fourth preset position along the opening direction, the second camera 32 is controlled to capture video.

[0116] When the door 20 is detected to have moved to the fifth preset position in the closing direction, the second camera 32 is controlled to stop shooting video. The shooting area of ​​the second camera 32 covers the opening area of ​​the storage room 11 of the refrigeration equipment 100. The third preset position is the same as the fifth preset position or is located between the fifth preset position and the door closing position.

[0117] The third food information of the food that is stored and taken out of the storage room 11 is obtained from the video captured by the second camera 32.

[0118] The food information inside the refrigeration equipment 100 is obtained based on the first food information, the second food information, and the third food information.

[0119] In this embodiment, the food information in the refrigeration equipment 100 can be the food information of the entire storage room 11.

[0120] In this embodiment, the food information within the refrigeration equipment 100 can be the food information for the entire storage room 11. The entire storage room 11 can refer to a space that includes the area inside the storage room opening 111 and the area inside the door 20, and is relatively enclosed and independent from other rooms of the refrigeration equipment 100.

[0121] During the use of the refrigeration unit 100, users may remove food items placed inside the door 20 and store them in the storage compartment 11. If the refrigeration unit 100 only identifies a specific area, such as the area within the storage compartment 11, it may record the food items moved from inside the door 20 to the storage compartment 11 as newly stored food items. However, the total amount of food stored in the refrigeration unit 100 remains unchanged. Therefore, this could lead to errors in the food information management by the refrigeration unit 100 and inaccurate food information provided to the user.

[0122] In this embodiment, the second camera 32 can be controlled to capture video of the user taking and placing food in the opening area of ​​the storage room 11 during the time period between the door 20 moving in the opening direction to the fourth preset position and the door 20 moving in the closing direction to the fifth preset position. This allows for capturing video of food being taken and placed between the outside of the refrigeration equipment 100 and the storage room 11 or the inner space of the door 20, as well as between the storage room 11 and the inner space of the door 20, without misidentifying the interaction of food between the two as newly added food inside the refrigeration equipment.

[0123] Meanwhile, when the second camera 32 captures video of the user retrieving food from the opening area of ​​the storage compartment 11, if the door 20 is opened at a large angle, the distance between the door 20 and the second camera 32 may be too great. Therefore, the second camera 32 may not be able to capture the retrieval of food from the inner space of the door 20, especially the food stored at the top of the inner space. Similarly, when the first camera 31 captures video of food being retrieved from the inner space of the door 20, the retrieval of certain food items may not be captured due to the user's hand obstructing the view.

[0124] Therefore, the food recognition effect is not good if the dynamic loading and unloading of food in the inner space of the door 20 is obtained only through the second camera 32 or only through the first camera 31. The food recognition results can be supplemented and confirmed by the food dynamic loading and unloading video captured by the first camera 31, the food dynamic loading and unloading video captured by the second camera 32, and the food static placement photo captured by the second camera 32. This can comprehensively obtain the information on the food stored and unloaded in the inner space of the door 20, reduce the poor recognition effect caused by the distance from the camera, the inability to capture the image, or the obstruction, and thus achieve accurate recognition of the storage and unloading of food in the inner space of the door 20.

[0125] This setup allows for the separate monitoring of food storage and retrieval in the storage room 11 and the inner space of the door 20 using a combination of static and dynamic camera recording. It also enables the monitoring of food interaction between these two areas. Therefore, it provides a comprehensive overview of food storage and retrieval throughout the entire refrigeration unit, allowing for holistic and comprehensive management of the food within the refrigeration unit 100. Furthermore, the combined static and dynamic monitoring method verifies the identification results, making the food identification results of the refrigeration unit 100 more comprehensive and accurate. The system is simple, easy to install, and occupies minimal space.

[0126] Furthermore, in this embodiment, the method for obtaining the third ingredient information may include:

[0127] Obtain hand information from the video captured by the second camera 32;

[0128] Based on the hand information, obtain the food information carried by the hand in the video captured by the second camera 32 and mark it as the third food information;

[0129] Obtain the movement direction information of the hand carrying the food ingredient in the video captured by the second camera 32;

[0130] Based on the hand movement direction information, obtain information about the food being stored in the storage room 11 or being retrieved from the storage room 11, and mark it as the third food information.

[0131] This setup enables accurate acquisition of information about the third ingredient, and the control method is simple and easy to implement.

[0132] Reference Figure 9 Furthermore, in this embodiment, the method for identifying ingredients that combines static and dynamic elements may also include:

[0133] Obtain information about the ingredients stored in the storage room 11;

[0134] Obtain information about the ingredients retrieved from the inner space of the door 20;

[0135] Determine whether the food stored in the storage room 11 contains food that was taken out from the inner space of the door 20;

[0136] If so, the food items removed from the inner space of the door 20 will be deleted from the food items stored in the storage room 11 and then marked as stored food item information.

[0137] If not, the food items stored in the storage room 11 will be directly marked as the stored food item information;

[0138] Obtain the food information stored in the inner space of the door 20;

[0139] Obtain information about the ingredients retrieved from the storage room 11;

[0140] Determine whether the food stored in the inner space of the door 20 contains food taken from the storage room 11;

[0141] If so, after deleting the food information taken from the storage room 11 from the food information stored in the inner space of the door 20, mark it as the stored food information;

[0142] If not, the food information stored in the inner space of the door 20 will be directly marked as the stored food information.

[0143] In this embodiment, the stored food information can be all the food information newly stored in the entire space of the storage room 11 after the door 20 completes one opening and closing.

[0144] This setup enables accurate information about all food items newly stored in the storage room 11, eliminating interference from food items that only move between the inside of the door 20 and the storage room 11. This ensures the comprehensiveness, accuracy, and reliability of the food item identification results from the refrigeration equipment 100, achieving holistic and comprehensive management of the food items within the refrigeration equipment 100.

[0145] Reference Figure 9 Furthermore, in this embodiment, the method for identifying ingredients that combines static and dynamic elements also includes:

[0146] Obtain information about the ingredients retrieved from the storage room 11;

[0147] Obtain the food information stored in the inner space of the door 20;

[0148] Determine whether the food taken out from the storage room 11 contains food stored in the inner space of the door 20;

[0149] If so, the information of the food ingredients taken out from the storage room 11 is deleted from the information of the food ingredients stored in the inner space of the door 20 and then marked as the information of the food ingredients taken out;

[0150] If not, the information of the food ingredients taken from the storage room 11 will be directly marked as the information of the food ingredients taken out.

[0151] Obtain information about the ingredients retrieved from the inner space of the door 20;

[0152] Obtain information about the ingredients stored in the storage room 11;

[0153] Determine whether the food taken out from the inner space of the door 20 contains the food stored in the storage room 11;

[0154] If so, the information on the ingredients taken out from the inner space of the door 20 and then the information on the ingredients stored in the storage room 11 is deleted and marked as the taken-out information.

[0155] If not, the information of the ingredients taken out from the inner space of the door 20 will be directly marked as the information of the taken-out ingredients.

[0156] In this embodiment, the information on the ingredients to be retrieved can be all the information on the ingredients newly retrieved from the entire space of the storage room 11 after the door 20 has completed one opening and closing.

[0157] This setup enables accurate information on all food items newly retrieved from the entire storage room 11, eliminating interference from food items that only move between the inside of the door 20 and the storage room 11. This ensures the comprehensiveness, accuracy, and reliability of the food item identification results from the refrigeration equipment 100, achieving holistic and comprehensive management of the food items within the refrigeration equipment 100.

[0158] Reference Figure 9 Furthermore, in this embodiment, the food ingredient identification method further includes:

[0159] Mark the stored ingredient information obtained each time and delete the retrieved ingredient information each time to obtain the existing ingredient information.

[0160] In this embodiment, the existing food information can be all the food information currently actually stored in the entire space of the storage room 11 after the door 20 has completed one opening and closing.

[0161] This setup enables accurate information about all the food items currently stored in the storage room 11, eliminating interference from food items that only move between the inside of the door 20 and the storage room 11. This ensures the comprehensiveness, accuracy, and reliability of the food item identification results from the refrigeration equipment 100, and enables comprehensive, all-round management of the food items within the refrigeration equipment 100.

[0162] Reference Figure 3 Furthermore, in the second embodiment of the present invention, a refrigeration device 100 is provided. The refrigeration device 100 in this embodiment can be as described in the first embodiment of the present invention.

[0163] The second camera 32 may include a left-side camera 321 and a right-side camera 322. The left-side camera 321 may be located on the left half of the storage room 11, and the right-side camera 322 may be located on the right half of the storage room 11.

[0164] The shooting area of ​​the left-side camera 321 covers the left half of the storage room opening 111 and the inner space of the left door 21 when it is in the third preset position. The shooting area of ​​the right-side camera 322 covers the right half of the storage room opening 111 and the inner space of the right door 22 when it is in the third preset position.

[0165] The left chamber camera 321 and the right chamber camera 322 can both be installed in the width direction of the top wall of the storage room 11, close to the middle.

[0166] The lens of the left chamber camera 321 can be angled downwards and to the left towards the left half of the storage room opening 111, and the right chamber camera 322 can be angled downwards and to the right towards the right half of the storage room opening 111.

[0167] In this embodiment, a method for identifying ingredients that combines static and dynamic elements is also provided. The method for identifying ingredients that combines static and dynamic elements in this embodiment may include the method described in the first embodiment of the present invention.

[0168] This setup enables targeted imaging and recognition of the left and right halves of the entire storage room 11, reducing blind spots and minimizing missed or incorrect recognitions caused by hands or food obstructing the view. This improves the accuracy and reliability of food recognition results from the refrigeration equipment 100. The combined static and dynamic camera imaging allows for separate monitoring of food storage and retrieval in both the storage room 11 and the inner space of the door 20, as well as information on food interaction between these two areas. This provides a comprehensive view of food storage and retrieval throughout the entire refrigeration equipment, enabling holistic and all-encompassing management of the food within the refrigeration equipment 100. Furthermore, the combined static and dynamic imaging verifies the recognition results, making the food recognition results from the refrigeration equipment 100 more comprehensive and accurate. The system is simple, easy to install, and occupies minimal space.

[0169] Reference Figure 4 Furthermore, in the third embodiment of the present invention, a refrigeration device 100 is provided. The refrigeration device 100 in this embodiment can be as described in the first embodiment of the present invention.

[0170] The second camera 32 may include a left-side camera 321 and a right-side camera 322. The left-side camera 321 may be located on the left half of the storage room 11, and the right-side camera 322 may be located on the right half of the storage room 11.

[0171] The shooting area of ​​the left-side camera 321 covers the left half of the storage room opening 111 and the inner space of the left door 21 when it is in the third preset position. The shooting area of ​​the right-side camera 322 covers the right half of the storage room opening 111 and the inner space of the right door 22 when it is in the third preset position.

[0172] The left chamber camera 321 is located in the middle of the left half of the storage room 11, and the right chamber camera 322 is located in the middle of the right half of the storage room 11.

[0173] Since the storage compartment 11, which has double doors, is often equipped with a central vertical beam on its door 20, this design prevents the central vertical beam from obstructing the second camera 32. This allows for targeted imaging and identification of the left and right halves of the storage compartment 11, reducing blind spots and minimizing missed or incorrect identifications caused by hands or food items obstructing the view. Ultimately, this improves the accuracy and reliability of the food identification results from the refrigeration equipment 100.

[0174] In this embodiment, a method for identifying ingredients that combines static and dynamic elements is also provided. The method for identifying ingredients that combines static and dynamic elements in this embodiment may include the method described in the first embodiment of the present invention.

[0175] This setup enables targeted imaging and recognition of the left and right halves of the entire storage room 11, reducing blind spots and minimizing missed or incorrect recognitions caused by hands or food obstructing the view. This improves the accuracy and reliability of food recognition results from the refrigeration equipment 100. The combined static and dynamic camera imaging allows for separate monitoring of food storage and retrieval in both the storage room 11 and the inner space of the door 20, as well as information on food interaction between these two areas. This provides a comprehensive view of food storage and retrieval throughout the entire refrigeration equipment, enabling holistic and all-encompassing management of the food within the refrigeration equipment 100. Furthermore, the combined static and dynamic imaging verifies the recognition results, making the food recognition results from the refrigeration equipment 100 more comprehensive and accurate. The system is simple, easy to install, and occupies minimal space.

[0176] Reference Figure 5 Furthermore, in the fourth embodiment of the present invention, a refrigeration device 100 is provided. The refrigeration device 100 in this embodiment can be as described in the first embodiment of the present invention.

[0177] The image recognition module 30 may include a third camera 33. The position of the door 20 may include a sixth preset position.

[0178] In this embodiment, the first shelf 12 may include three horizontally spaced shelves 121, each with a first storage space 122 formed above it. The third camera 33 may be mounted on the door 20 and correspond to the first storage space 122 above the first shelf 12. Three first storage spaces 122 are formed on the three first shelves, therefore there are three third cameras. The sixth preset position can be any position between 0 degrees and 30 degrees between the door 20 and the storage room opening 111.

[0179] In this embodiment, the height of the lens of the third camera 33 can be located in the upper middle part of the first storage space 122. When the door 20 closes the storage room opening 111, the width is located in the middle of the width direction of the first storage space 122. The lens of the third camera 32 can face the first storage space 122. When the door 20 moves to the sixth preset position, the shooting area of ​​the third camera 33 covers the corresponding first storage space 122.

[0180] In this embodiment, when the door 20 moves in the closing direction to the sixth preset position, the third camera 33 can be controlled to take a picture of the food placed by the user in the corresponding first storage space 122.

[0181] In this embodiment, a method for identifying ingredients that combines static and dynamic elements is also provided. The method for identifying ingredients that combines static and dynamic elements in this embodiment may include the method described in the first embodiment of the present invention.

[0182] Reference Figure 8 Furthermore, in this embodiment, the method for identifying ingredients that combines static and dynamic elements may also include:

[0183] When the door 20 is detected to move to the sixth preset position in the closing direction, the third camera 33 is controlled to take a picture and store it. When the door 20 moves to the sixth preset position, the shooting area of ​​the third camera 33 covers the first storage space 122 above the first shelf 12 in the storage room 11.

[0184] The fourth ingredient information of the ingredients stored and retrieved in the first storage space 122 is obtained based on the photos taken by the third camera 33.

[0185] Information on ingredients stored in and retrieved from the storage room 11 is obtained based on the third and fourth ingredient information.

[0186] Since there may be issues with recognition due to hand obstruction during the video recording process of the second camera 32, a third camera 33 is added. The static images captured by the third camera 33 are combined with the dynamic videos captured by the second camera. By combining static and dynamic images, the recognition results are cross-verified to ensure the accuracy of the recognition results of the food in the storage room.

[0187] Furthermore, in this embodiment, the method for obtaining the fourth ingredient information includes:

[0188] The information of the food items taken from the first storage space 122 is obtained by comparing the photo currently taken by the second camera 32 with the previously stored photo taken by the second camera 32, and this information is marked as the fourth food item information.

[0189] This setup allows for the use of both static and dynamic camera recording to monitor the storage and retrieval of food items in the storage room 11, the inner space of the door 20, and the first shelf area. It also enables the monitoring of food interaction between different areas. Therefore, it provides a comprehensive overview of the storage and retrieval of food items throughout the entire refrigeration unit, enabling holistic and all-encompassing management of the food items within the refrigeration unit 100. Furthermore, the combination of static and dynamic recordings allows for cross-verification of the identification results, resulting in more comprehensive and accurate food identification of the refrigeration unit 100. The setup is simple in structure, easy to install, and occupies minimal space.

[0190] Reference Figure 6 Furthermore, in the fifth embodiment of the present invention, this embodiment provides a refrigeration device 100. The refrigeration device 100 in this embodiment can be as described in the first and second embodiments of the present invention.

[0191] The image recognition module 30 of the refrigeration equipment 100 may also include a third camera 33. The position of the door 20 may include a sixth preset position.

[0192] The first shelf 12 may include three horizontally spaced shelves 121, each with a first storage space 122 formed above it. The third camera 33 may be disposed in the middle of the top wall of each first storage space 122, with the lens of the third camera 33 facing downwards and its shooting area covering the corresponding first storage space 122. Three first storage spaces 122 are formed on the three first shelves, therefore there are three third cameras.

[0193] The sixth preset position can be any position between 0 degrees and 30 degrees between the door 20 and the opening 111 of the storage room.

[0194] In this embodiment, a method for identifying ingredients that combines static and dynamic elements is also provided. This method may include the methods described in the first and fourth embodiments of the present invention.

[0195] Furthermore, in this embodiment, the image recognition module 30 may include a fourth camera 34. The position of the door 20 may include a seventh preset position and an eighth preset position.

[0196] The refrigeration equipment 100 may further include a third shelf 14, which may be located in the storage room 11 and below the first shelf 12. The third shelf 14 may include a pull-out horizontal drawer 141, with the upper part of the drawer 141 being open.

[0197] In this embodiment, the fourth camera 34 can be located above the third shelf 14. The fourth camera 34 can be installed on the left or right side walls of the storage compartment 11, and the fourth camera 34 can face the drawer 141, and the shooting area of ​​the fourth camera 34 can cover the opening area of ​​the drawer 141 of the storage compartment 11 when it is open.

[0198] In this embodiment, the drawer 141 may include a left drawer 1411 and a right drawer 1412. The left drawer 1411 may be located on the left half of the storage compartment 11, and the right drawer 1412 may be located on the right half of the storage compartment 11. The fourth camera 34 may include a left box camera and a right box camera. The lens of the left box camera may be directed toward the left drawer 1411, and the lens of the right box camera may be directed toward the right drawer 1412.

[0199] In this embodiment, two layers of left drawers 1411 and right drawers 1412 can be provided along the height direction of the refrigeration equipment 100 to expand the storage space of the refrigeration equipment 100 and facilitate users to store food ingredients.

[0200] The seventh preset position can be any position between 0 degrees and 60 degrees between the door 20 and the storage compartment opening 111, and the eighth preset position can be any position between 0 degrees and 60 degrees between the door 20 and the storage compartment opening 111. Further, in this embodiment, the seventh and eighth preset positions can be the door 20 position coordinates preset before the refrigeration equipment 100 leaves the factory.

[0201] Reference Figure 8 Furthermore, in this embodiment, the method for identifying ingredients that combines static and dynamic elements may also include:

[0202] When the door 20 is detected to have moved to the seventh preset position in the opening direction, the fourth camera 34 is controlled to capture video. The shooting area of ​​the fourth camera 34 covers the opening area of ​​the drawer 141 of the storage room 11 when it is in the open state.

[0203] When the door 20 is detected to have moved to the eighth preset position in the closing direction, the fourth camera 34 is controlled to stop recording video.

[0204] The fifth ingredient information is obtained from the video captured by the fourth camera 34, which shows the ingredients stored and taken out from the space inside the drawer 141.

[0205] Information on ingredients stored in and retrieved from the storage room 11 is obtained based on the third, fourth, and fifth ingredient information.

[0206] In this embodiment, the fourth camera 34 can be controlled to capture video of the user taking food out of the drawer 141 during the time period between the door 20 moving in the opening direction to the seventh preset position and the door 20 moving in the closing direction to the eighth preset position.

[0207] Refrigeration drawers are typically located at the bottom of the refrigerator compartment, far from the first camera, resulting in poor recognition performance. Therefore, a fourth camera 34 is added to specifically capture video of the food inside drawer 141. This dynamic video is combined with the dynamic video captured by the second camera 32 to cross-verify the recognition results, thus achieving accurate identification of the food inside drawer 141. Furthermore, by combining the dynamic video captured by the fourth camera 34 with the static images captured by the third camera 33 and the dynamic video captured by the second camera, this combination of static and dynamic data ensures accurate identification of the food inside the storage room.

[0208] Furthermore, in this embodiment, the method for obtaining the fifth ingredient information includes:

[0209] Obtain hand information from the video captured by the fourth camera 34;

[0210] Based on the hand information, obtain the food information carried by the hand in the video captured by the fourth camera 34 and mark it as the fifth food information;

[0211] Obtain the movement direction information of the hand carrying the food ingredient in the video captured by the fourth camera 34;

[0212] Based on the hand movement direction information, obtain information about the food being stored in or taken out of the drawer 141 and mark it as the fifth food information.

[0213] This setup enables accurate acquisition of information about the fifth ingredient, and the control method is simple and easy to implement.

[0214] This setup allows for the use of both static and dynamic camera recording to monitor the storage and retrieval of food in each area: the storage room 11, the inner space of the door 20, the drawer 141, and the first shelf 13. It also provides information on the interaction between food items in different areas. Therefore, it enables comprehensive monitoring of the entire refrigeration unit's food storage and retrieval, achieving holistic and all-encompassing management of the food within the refrigeration unit 100. Furthermore, the combined static and dynamic recording method verifies the identification results, making the food identification results of the refrigeration unit 100 more comprehensive and accurate. The system is simple, easy to install, and occupies minimal space.

[0215] Reference Figures 1-6 Furthermore, in other embodiments of the present invention, a refrigeration device 100 may be provided. The refrigeration device 100 in this embodiment may include some or all of the first shelf 12, second shelf 13, and third shelf 14 from any of the above embodiments. The image recognition module 30 of the refrigeration device 100 may include some or all of the first camera 31, second camera 32, third camera 33, and fourth camera 34 from any of the above embodiments, enabling their combination to achieve mutual confirmation and supplementation of dynamic and static shooting and recognition results of the storage room 11 or the inner space of the door 20, or the first shelf 13 or drawer 141 using different cameras, thereby improving the accuracy and reliability of camera shooting and recognition.

[0216] Furthermore, in this embodiment, the control method for food ingredient recognition may include some or all of the control methods for obtaining first, second, third, fourth, and fifth food ingredient information in any of the above embodiments. It can obtain food ingredient information inside the refrigeration equipment 100 based on some or all of the first, second, third, fourth, and fifth food ingredient information, enabling the combination of different cameras to mutually confirm and supplement the dynamic and static shooting and recognition results of areas such as the inner space of the door 20 and the storage room 11, thereby improving the accuracy and reliability of camera shooting and recognition.

[0217] Furthermore, in any of the above embodiments of the present invention, the refrigeration device 100 also includes a voice interaction module. The refrigeration device 100 can use the voice interaction module to inform the user of the food information in the space inside the door 20, the storage room 11, the first storage space 122, the drawer 141, and the entire space of the storage room 11. The user can correct the above food information through voice dialogue.

[0218] The food ingredient identification method also includes:

[0219] The product information is read aloud via voice.

[0220] Obtain user feedback information;

[0221] The accuracy of the reported ingredient information is determined based on user feedback.

[0222] If not, change the ingredient information based on user feedback.

[0223] For example, when the door 20 is closed, the refrigeration device 100 can announce the type and quantity of food that has been stored or retrieved, such as "store one apple". The user can provide feedback on the information announced by the refrigeration device 100 through voice reply, such as "change to store one pear". The refrigeration device 100 can then correct the above food information based on the user's feedback.

[0224] In practical applications, food detection algorithms based on videos or images are prone to misidentification or omission.

[0225] This setup allows users to easily understand the condition of the food in the refrigeration unit 100, and also enables interaction with users to correct misjudgments such as incorrect or missed identifications caused by food detection algorithms based on video or images.

[0226] Furthermore, in any of the above embodiments of the present invention, the refrigeration device 100 may further include a display screen module. The refrigeration device 100 can display the food information of the refrigeration device 100 to the user through the display screen, and the user can modify the food information of the refrigeration device 100 by operating the display screen.

[0227] This setup allows users to easily understand the condition of the food in the refrigeration unit 100, and also enables interaction with users to correct misjudgments such as incorrect or missed identifications caused by food detection algorithms based on video or images.

[0228] Furthermore, in one embodiment of the present invention, a computer-readable storage medium is provided, on which a computer program is stored, characterized in that, when the computer program is executed by a processor, it implements the steps of the dynamic and static combined food identification method described in any of the above embodiments.

[0229] In summary, the dynamic and static combined food identification method, storage medium, and refrigeration device 100 of the present invention, by setting up a camera to obtain food information in the inner space of the door 20, can solve the problems of incomplete food identification and identification errors that may occur when food is obscured by the image recognition system of the refrigeration device 100 in the prior art, which usually uses a single camera to identify a specific area.

[0230] By adopting the technical solution in this application, it is possible to mutually confirm and supplement the shooting and recognition results of cameras at different positions and angles, and to combine the shooting and recognition results to know the storage and retrieval of food in areas such as the inner space of the door 20 and the storage room 11, thereby reducing missed recognition and incorrect recognition caused by hand obstruction or food obstruction, and improving the accuracy and reliability of food recognition results of the refrigeration equipment 100.

[0231] By adopting the technical solution in this application, it is possible to mutually confirm and supplement the food identification results by using video of dynamic food handling and photos of static food placement. The static photos and dynamic videos are mutually verified, and the storage and handling of food in each area of ​​the storage room 11, door 20, first shelf 13, and drawer 141 can be accurately determined based on the combination of the shooting and identification results. This reduces omissions and errors caused by hand obstruction or food obstruction, ensuring the accuracy of the identification results. At the same time, it can also understand the interaction of food between different areas. Therefore, it can achieve a holistic understanding of the storage and handling of food in the entire space of the refrigeration equipment, realizing comprehensive and all-round management of food in the refrigeration equipment 100. By combining dynamic and static methods to mutually verify the identification results, the food identification results of the refrigeration equipment 100 are more comprehensive and accurate. The structure is simple, easy to install, and occupies little space.

[0232] It should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This way of describing the specification is only for clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

[0233] The detailed descriptions listed above are merely specific descriptions of feasible embodiments of the present invention, and are not intended to limit the scope of protection of the present invention. All equivalent embodiments or modifications made without departing from the spirit of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for identifying food ingredients that combines static and dynamic elements, characterized in that, The identification method includes: Detect the position and direction of movement of the gate; When the door is detected to have moved to a first preset position along the opening direction, the first camera is controlled to capture video, and the shooting area of ​​the first camera covers the inner space of the door. When the door is detected to have moved to a second preset position along the closing direction, the first camera is controlled to stop recording video. The first ingredient information of the ingredients stored and retrieved in the inner space of the door is obtained based on the video captured by the first camera. When the door is detected to have moved to a third preset position in the closing direction, the second camera is controlled to take a picture and store it. When the door moves to the third preset position, the inner space of the door enters the shooting area of ​​the second camera. The second ingredient information of the ingredients stored and retrieved in the inner space of the door is obtained based on the photos taken by the second camera. Based on the first ingredient information and the second ingredient information, obtain the ingredient information stored and retrieved from the inner space of the door; When the door is detected to have moved to the fourth preset position along the opening direction, the second camera is controlled to capture video. When the door is detected to have moved to the fifth preset position in the closing direction, the second camera is controlled to stop shooting video. The shooting area of ​​the second camera covers the opening area of ​​the refrigeration equipment storage room. The third preset position is the same as the fifth preset position, or the third preset position is located between the fifth preset position and the door closing position. Third food information is obtained from the video captured by the second camera, showing the food items stored and retrieved in the storage room. The food information inside the refrigeration equipment is obtained based on the first food information, the second food information, and the third food information.

2. The method for identifying ingredients combining static and dynamic elements as described in claim 1, characterized in that, Also includes: Obtain information about the ingredients stored in the storage room; Obtain information about the ingredients retrieved from the inner space of the door; Determine whether the food stored in the storage room contains food taken from the space inside the door; If so, then delete the food items taken from the inner space of the door after deleting the food items stored in the storage room and mark them as stored food items information; If not, the food stored in the storage room will be directly marked as the stored food information; Obtain the food information stored in the inner space of the door; Obtain information about the ingredients retrieved from the storage room; Determine whether the food stored in the inner space of the door contains food taken from the storage room. If so, after deleting the food information taken from the storage room from the food information stored in the food information inside the door, mark it as the stored food information; If not, the food information stored in the inner space of the door body will be directly marked as the stored food information.

3. The method for identifying ingredients combining static and dynamic elements as described in claim 2, characterized in that, Also includes: Obtain information about the ingredients retrieved from the storage room; Obtain the food information stored in the inner space of the door; Determine whether the food taken out from the storage room contains food stored in the space inside the door; If so, after deleting the food information stored in the inner space of the door body from the food information retrieved from the storage room, mark it as retrieved food information; If not, the information of the food ingredients taken from the storage room will be directly marked as the information of the taken-out food ingredients; Obtain information about the ingredients retrieved from the inner space of the door; Obtain information about the ingredients stored in the storage room; Determine whether the food ingredients taken out from the inner space of the door contain the food ingredients stored in the storage room. If so, the information on the ingredients taken out from the inner space of the door is deleted from the information on the ingredients stored in the storage room and then marked as the information on the ingredients taken out; If not, the information of the food ingredients taken out from the inner space of the door will be directly marked as the information of the food ingredients taken out.

4. The method for identifying ingredients combining static and dynamic elements as described in claim 3, characterized in that, Also includes: Mark the stored ingredient information obtained each time and delete the retrieved ingredient information each time to obtain the existing ingredient information.

5. The method for identifying ingredients combining static and dynamic elements as described in claim 1, characterized in that, The control method includes: When the door is detected to have moved to the sixth preset position in the closing direction, the third camera is controlled to take a picture and store it. When the door moves to the sixth preset position, the shooting area of ​​the third camera covers the first storage space above the first shelf in the storage room. The fourth ingredient information of the ingredients stored and retrieved in the first storage space is obtained based on the photos taken by the third camera. Information on ingredients stored in and retrieved from the storage room is obtained based on the third and fourth ingredient information.

6. The method for identifying ingredients combining static and dynamic elements as described in claim 5, characterized in that, The control method includes: When the door is detected to have moved to the seventh preset position along the opening direction, the fourth camera is controlled to capture video. The shooting area of ​​the fourth camera covers the opening area of ​​the drawer of the storage room when it is in the open state. When the door is detected to have moved to the eighth preset position along the closing direction, the fourth camera is controlled to stop recording video. The fifth ingredient information is obtained from the video captured by the fourth camera, which shows the ingredients stored and retrieved from the drawer. Information on ingredients stored in and retrieved from the storage room is obtained based on the third, fourth, and fifth ingredient information.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the dynamic and static combined food identification method according to any one of claims 1-6.

8. A refrigeration device, the refrigeration device including a storage compartment, the refrigeration device further including a door for opening and closing the storage compartment, the refrigeration device further including a food identification system, the food identification system including a control module, an image recognition module, and a detection module, characterized in that, The food ingredient recognition system further includes a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the computer program, it implements the steps of the dynamic and static combined food ingredient recognition method according to any one of claims 1-6.

9. The refrigeration equipment as described in claim 8, characterized in that, The first camera is located on the upper part of the inner side of the door. The first preset position is any position where the door and the opening of the storage room are at an angle of 0 degrees to 30 degrees. The second preset position is any position where the door and the opening of the storage room are at an angle of 0 degrees to 30 degrees.

10. The refrigeration equipment as described in claim 8, characterized in that, The second camera is installed on the top wall of the storage room and close to the opening of the storage room. The third preset position is any position where the door and the opening of the storage room are at an angle of 0 to 90 degrees. The fourth preset position is any position where the door and the opening of the storage room are at an angle of 0 to 30 degrees. The fifth preset position is any position where the door and the opening of the storage room are at an angle of 0 to 30 degrees.

11. The refrigeration equipment as described in claim 8, characterized in that, The third camera is installed in the storage room and above the first shelf. The sixth preset position is any position where the door and the opening of the storage room are at an angle of 0 to 30 degrees.

12. The refrigeration equipment as described in claim 8, characterized in that, The third camera is installed on the door and works in conjunction with the first shelf. The sixth preset position is any position where the door and the opening of the storage room are at an angle of 0 to 30 degrees.

13. The refrigeration equipment as described in claim 8, characterized in that, The fourth camera is installed on the left and right side walls of the storage room. The seventh preset position is any position where the door and the opening of the storage room are at an angle of 0 to 60 degrees. The eighth preset position is any position where the door and the opening of the storage room are at an angle of 0 to 60 degrees.

Images