Food identification method for refrigerator, refrigerator, and storage medium

Abstract

A food identification method for a refrigerator, a refrigerator, and a storage medium, relating to the technical field of intelligent refrigerators. The refrigerator is provided with a radar, and the food identification method comprises: by means of a radar reflection microwave signal, acquiring basic data of food in the refrigerator; identifying the position of the food on the basis of the basic data; generating a radar image of the food on the basis of the basic data; and identifying the type and quantity of the food on the basis of the radar image of the food, the position of the food, and a food identification model. The accuracy of food identification and the reliability of intelligent management of refrigerator food are improved.

Description

Food identification method for refrigerator, refrigerator and storage medium

[0001] This application is based on the Chinese patent application with application number 202311041766.X and application date August 17, 2023, and claims the priority of the Chinese patent application. The entire content of the Chinese patent application is hereby introduced into this application as a reference. Technical Field

[0002] The present application relates to the technical field of smart refrigerators, for example, to a food identification method for refrigerators, a refrigerator, and a storage medium. Background Art

[0003] With the advancement of science and technology, refrigerators have become increasingly intelligent, boasting a wide range of functions. However, at their core, the primary function of a refrigerator is to store and manage food. However, achieving this intelligent food management remains a pressing issue. Currently, manufacturers often use a combination of manual data entry and food data collection using optical cameras, followed by corresponding food management software, to achieve intelligent food management in refrigerators. However, data collection through optical cameras is significantly affected by the refrigerator's internal environment, such as lighting conditions and food placement, resulting in low food recognition accuracy. The manual data entry process is tedious, time-consuming, and prone to errors and omissions. Therefore, combining manual data entry with data collection using optical cameras to achieve intelligent food management in refrigerators is not very practical.

[0004] In the related art, a solution to the above problem is to provide a refrigerator control method. The method includes: obtaining a thermal image of the interior of the refrigerator and determining whether there are areas of a preset color in the thermal image; if there are areas of the preset color, determining the area of ​​each area and its location within the refrigerator; obtaining image data of the interior of the refrigerator; marking the image data based on the location of each area within the refrigerator to obtain target image data; and identifying each marked area in the target image data based on a preset food recognition model to determine the food type of each marked area.

[0005] During the implementation of the embodiments of the present disclosure, it was found that at least the following problems exist in the related art:

[0006] The above refrigerator control method uses thermal images and image analysis of the refrigerator interior to identify food types. Generating thermal images requires temperature differences between ingredients and between ingredients and the surrounding environment, which are then used to generate thermal images. However, the refrigerator interior maintains a constant temperature, and most vegetable products have similar temperature requirements. Foods of the same type have similar temperature requirements. Therefore, the generated thermal images of food have low resolution and cannot identify the number of foods of the same type. Consequently, this food identification method has low accuracy, making intelligent refrigerator food management less practical.

[0007] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of this application, and therefore may include information that does not constitute prior art known to ordinary technicians in this field.

[0008] Summary of the Invention

[0009] In order to provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. The summary is not an extensive review, nor is it intended to identify key / critical elements or delineate the scope of protection of these embodiments, but rather serves as a prelude to the detailed description that follows.

[0010] The embodiments of the present disclosure provide a food identification method for a refrigerator, a refrigerator, and a storage medium, which improve the accuracy of food identification and the reliability of intelligent food management in the refrigerator compared to related technologies.

[0011] In some embodiments, a food identification method for a refrigerator is provided. The refrigerator is equipped with a radar. The food identification method includes: obtaining basic data of food in the refrigerator through the radar; identifying the location of the food based on the basic data; generating a radar image of the food based on the basic data; and identifying the type and quantity of the food based on the radar image of the food, the location of the food and a food identification model.

[0012] Optionally, the step of identifying the type and quantity of food based on the radar image of the food, the location of the food and the food recognition model includes: preprocessing the radar image of the food; extracting feature data based on the preprocessed radar image of the food; and identifying the type and quantity of food based on the feature data, the location of the food and the food recognition model.

[0013] Optionally, the step of obtaining basic data of food in the refrigerator through radar includes: obtaining an initial reflected microwave signal of the food in the refrigerator through radar; and obtaining basic data of the food based on the initial reflected microwave signal.

[0014] Optionally, the step of obtaining the initial reflected microwave signal of the food in the refrigerator through radar includes: obtaining the first reflected microwave signal at each relative position of the refrigerator; obtaining the second reflected microwave signal of the food in the refrigerator; and filtering out the first reflected microwave signal from the second reflected microwave signal to obtain the initial reflected microwave signal of the food.

[0015] Optionally, the step of filtering out the first reflected microwave signal from the second reflected microwave signal to obtain the initial reflected microwave signal of the food includes: obtaining a third reflected microwave signal from the food packaging in the refrigerator; filtering out the first reflected microwave signal from the second reflected microwave signal to obtain a filtered reflected microwave signal; and filtering out the third reflected microwave signal from the filtered reflected microwave signal to obtain the initial reflected microwave signal of the food in the refrigerator.

[0016] Optionally, before obtaining basic data of food in the refrigerator through radar, the method further includes: obtaining data information of multiple preset categories of food through radar; and constructing a food recognition model based on the data information.

[0017] Optionally, the step of constructing a food recognition model based on the data information includes: generating preset radar images of multiple preset categories of food based on the data information; constructing a food recognition model through a deep learning method based on the preset radar images; or constructing a food recognition model through training a classifier based on the preset radar images.

[0018] Optionally, the step of identifying the location of the food based on the basic data includes: obtaining a reflected microwave signal of the food; and determining the relative position of the food and the radar based on the reflected microwave signal.

[0019] An embodiment of the present disclosure provides a refrigerator, comprising a processor and a memory storing program instructions, wherein the processor is configured to execute the food identification method for a refrigerator described in any of the above embodiments when running the program instructions.

[0020] An embodiment of the present disclosure provides a storage medium storing program instructions, which, when run, executes the food identification method for a refrigerator described in any of the above embodiments.

[0021] The food identification method for a refrigerator, the refrigerator, and the storage medium provided by the embodiments of the present disclosure can achieve the following technical effects:

[0022] The present disclosure provides a food identification method for a refrigerator equipped with a radar. The food identification method includes: obtaining basic data of food in the refrigerator through the radar; identifying the location of the food based on the basic data; generating a radar image of the food based on the basic data; and identifying the type and quantity of the food based on the radar image of the food, the location of the food, and a food identification model.

[0023] The food identification method for refrigerators provided in this disclosure uses radar microwave signals to obtain basic data about the food inside the refrigerator. This basic data is used to identify the food's position relative to the radar and generate a radar image of the food. Based on a preset food identification model and the food's position, the radar image of the food is analyzed to achieve food identification. Based on the food's own radar-reflected microwave signal, the radar microwave signal and radar imaging are used to identify the food, improving the accuracy of refrigerator food identification and the reliability of intelligent refrigerator food management.

[0024] The above general description and the following description are exemplary and explanatory only and are not intended to limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] One or more embodiments are exemplarily described by corresponding drawings. These exemplary descriptions and drawings do not limit the embodiments. Elements with the same reference numerals in the drawings are shown as similar elements. The drawings do not constitute a scale limitation. In addition,

[0026] FIG1 is a schematic diagram of a food identification method for a refrigerator provided by an embodiment of the present disclosure;

[0027] FIG2 is a schematic diagram of a food identification method for a refrigerator provided by another embodiment of the present disclosure;

[0028] FIG3 is a schematic diagram of a food identification method for a refrigerator provided by another embodiment of the present disclosure;

[0029] FIG4 is a schematic diagram of a food identification method for a refrigerator provided by another embodiment of the present disclosure;

[0030] FIG5 is a schematic diagram of a food identification method for a refrigerator provided by another embodiment of the present disclosure;

[0031] FIG6 is a schematic diagram of a food identification method for a refrigerator provided by another embodiment of the present disclosure;

[0032] FIG7 is a schematic diagram of a food identification method for a refrigerator provided by another embodiment of the present disclosure;

[0033] FIG8 is a schematic diagram of a control device for food identification in a refrigerator provided by one embodiment of the present disclosure;

[0034] FIG9 is a schematic diagram of a refrigerator provided by one embodiment of the present disclosure;

[0035] FIG10 is a schematic structural diagram of a refrigerator provided by one embodiment of the present disclosure. DETAILED DESCRIPTION

[0036] In order to be able to understand the features and technical content of the embodiments of the present disclosure in more detail, the implementation of the embodiments of the present disclosure is described in detail below in conjunction with the accompanying drawings. The accompanying drawings are for reference only and are not used to limit the embodiments of the present disclosure. In the following technical description, for the sake of convenience of explanation, a full understanding of the disclosed embodiments is provided through multiple details. However, one or more embodiments can still be implemented without these details. In other cases, to simplify the drawings, well-known structures and devices can be simplified for display.

[0037] In the description and claims of the embodiments of the present disclosure, as well as in the accompanying drawings, the terms "first," "second," and the like are used to distinguish similar items and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate to describe the embodiments of the present disclosure herein. In addition, the terms "including," "having," and any variations thereof are intended to cover non-exclusive inclusions.

[0038] Unless otherwise stated, the term "plurality" means two or more.

[0039] In the embodiment of the present disclosure, the character " / " indicates that the preceding and following objects are in an "or" relationship. For example, A / B means: A or B.

[0040] The term "and / or" describes an association between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or A and B.

[0041] The term "correspondence" may refer to an association relationship or a binding relationship. The correspondence between A and B means that there is an association relationship or a binding relationship between A and B.

[0042] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0043] A system for executing a food identification method for a refrigerator includes a processor and a memory storing program instructions. It may also include a communication interface and a bus. The processor, communication interface, and memory can communicate with each other via the bus. The communication interface can be used for information transmission. The processor is configured to execute the food identification method for a refrigerator when executing the program instructions.

[0044] 1 , an embodiment of the present disclosure provides a food identification method for a refrigerator, including:

[0045] S101, the processor obtains basic data of food in the refrigerator through radar;

[0046] S102, the processor identifies the location of the food based on the basic data;

[0047] S103, the processor generates a radar image of the food based on the basic data;

[0048] S104: The processor identifies the type and quantity of the food based on the radar image of the food, the location of the food, and the food recognition model.

[0049] The food recognition method for refrigerators provided by the embodiments of the present disclosure can obtain basic data about the food inside the refrigerator based on microwave signals reflected by radar. Using this basic data, the location of the food can be identified and a radar image of the food can be generated. Based on a preset food recognition model and the food's location, the radar image of the food can be analyzed to achieve food recognition.

[0050] For example, a refrigerator contains a variety of fruits. Radar reflects microwave signals to obtain basic data about the fruits inside the refrigerator. Using this data, the relative positions of the fruits in the refrigerator and the radar are identified, and a radar image of the fruits is generated. Based on a pre-set food recognition model and the positions of the fruits, the radar image is analyzed to identify the types, quantities, or locations of the fruits.

[0051] In addition, compared with the related art, based on the temperature difference between foods and between foods and the environment, thermal images and image analysis of the interior of the refrigerator are used to identify the type of food. This application identifies food by radar reflection of microwave signals and radar imaging. Radar imaging is based on the basic data of the food itself, which can be the dielectric constant, loss tangent or scattering point set of the food. Based on the basic data of the food itself, food recognition is performed through radar images of the food, the location of the food and the food recognition model, which can improve the accuracy of refrigerator food recognition, thereby improving the reliability of intelligent management of refrigerator food and the practicality of the refrigerator.

[0052] 2 , the present disclosure provides another method for identifying food in a refrigerator, including:

[0053] S201, the processor obtains basic data of food in the refrigerator through radar;

[0054] S202, the processor identifies the location of the food based on the basic data;

[0055] S203, the processor generates a radar image of the food based on the basic data;

[0056] S204, the processor pre-processes the radar image of the food;

[0057] S205, the processor extracts feature data based on the pre-processed radar image of the food;

[0058] S206: The processor identifies the type and quantity of the food according to the feature data, the location of the food, and the food identification model.

[0059] In this embodiment, the quality of food radar images is improved by preprocessing them. The preprocessing method can be one or more of normalization, filtering, and contrast enhancement. For example, normalization and filtering are used to preprocess the food radar images to reduce interference signals in the food radar images. Contrast enhancement is also used to preprocess the food radar images to improve the data resolution of the radar images.

[0060] Extract the required feature data from the preprocessed food radar image. For example, the feature data may include the radar image's shape, texture, and signal strength. By extracting this feature data, the amount of data required for food recognition is reduced, thereby reducing the complexity of the food radar image recognition process.

[0061] Based on a preset food recognition model and the location of the food, the food's type and quantity can be identified based on feature data. Specifically, the shape and texture of the radar images of different foods vary. The radar signal strength of foods at different distances from the radar varies. Based on the preset food recognition model, the shape and texture of the food's radar image can be used to determine the type of food. The radar signal strength varies in different areas of the radar image of a single food, and each food has an area with the strongest radar signal. Based on the preset food recognition model, the quantity of the food can be determined by comparing the regional radar signal strength with a regional strength threshold.

[0062] It should be noted that the radar reflected microwave signal area intensity thresholds corresponding to different locations or types of food are different. The specific area intensity thresholds are derived from the trained preset food recognition model and are not specifically specified here.

[0063] 3 , the present disclosure provides another method for identifying food in a refrigerator, including:

[0064] S301, the processor obtains an initial reflected microwave signal of food in the refrigerator through a radar;

[0065] S302, the processor obtains basic data of the food according to the initial reflected microwave signal;

[0066] S303, the processor identifies the location of the food based on the basic data;

[0067] S304, the processor generates a radar image of the food based on the basic data;

[0068] S305: The processor identifies the type and quantity of the food according to the radar image of the food, the location of the food, and the food recognition model.

[0069] In this embodiment, the initial reflected microwave signal from the food refers to the microwave signal emitted by the radar, which is reflected by the food and forms a reflected microwave signal. Because the initial reflected microwave signal from the food is formed by reflection from the food itself, it contains the physical properties of the food itself. By analyzing the initial reflected microwave signal, basic food data can be obtained. Exemplarily, the basic food data includes one or more of the following: dielectric constant, loss tangent, scattering point set, motion trajectory, or relative position. Based on the physical properties of the food, the food's location and radar image are obtained. The food is then identified using the food's location, radar image, and food recognition model, thereby improving the accuracy of food recognition.

[0070] It should be noted that the refrigerator can obtain the initial reflected microwave signal of the food at a fixed time or in real time to obtain the basic data of the food in the refrigerator. The time of the fixed time can be flexibly set by the user and is not specifically limited in this application.

[0071] 4 , the present disclosure provides another method for identifying food in a refrigerator, including:

[0072] S401, the processor obtains first reflected microwave signals at respective relative positions of the refrigerator;

[0073] S402, the processor obtains a second reflected microwave signal of the food in the refrigerator;

[0074] S403, the processor filters out the first reflected microwave signal from the second reflected microwave signal to obtain an initial reflected microwave signal of the food;

[0075] S404, the processor obtains basic data of the food according to the initial reflected microwave signal;

[0076] S405, the processor identifies the location of the food based on the basic data;

[0077] S406, the processor generates a radar image of the food based on the basic data;

[0078] S407: The processor identifies the type and quantity of the food based on the radar image of the food, the location of the food, and the food recognition model.

[0079] In this embodiment, the first reflected microwave signal is the radar-reflected microwave signal from various relative positions of the refrigerator. The second reflected microwave signal is the radar-reflected microwave signal from the food stored in the refrigerator. By filtering the first reflected microwave signal from the second reflected microwave signal, interference from the refrigerator's radar-reflected microwave signal on the food's initial reflected microwave signal is eliminated, improving the accuracy of the food's initial reflected microwave signal and, consequently, the accuracy of refrigerator food identification.

[0080] For example, consider apples stored in a refrigerator. First reflected microwave signals are obtained from various relative positions within the refrigerator. Second reflected microwave signals are then obtained from the apples within the refrigerator. Based on the second reflected microwave signals, the relative positions of the apples are determined. Based on the relative positions of the apples, the first reflected microwave signals are filtered out from the second reflected microwave signals. This eliminates interference from the refrigerator radar's reflected microwave signals on the apples' initial reflected microwave signals, improving the accuracy of the initial reflected microwave signals, and thus improving the accuracy of refrigerator food identification.

[0081] Optionally, the step of filtering out the first reflected microwave signal from the second reflected microwave signal to obtain the initial reflected microwave signal of the food includes: obtaining a third reflected microwave signal from the food packaging in the refrigerator; filtering out the first reflected microwave signal from the second reflected microwave signal to obtain a filtered reflected microwave signal; and filtering out the third reflected microwave signal from the filtered reflected microwave signal to obtain the initial reflected microwave signal of the food in the refrigerator.

[0082] In this embodiment, the third reflected microwave signal is the radar reflected microwave signal from the packaging structure of the food stored in the refrigerator. The packaging structure can be made of plastic, paper, or glass. The dielectric constant and loss tangent of the packaging structure itself are fixed depending on the material. For packaging structures of different materials, the corresponding radar reflected microwave signal remains constant. By filtering out the third reflected microwave signal from the filtered reflected microwave signal, interference from the radar reflected microwave signal from the food packaging structure on the initial reflected microwave signal from the food is eliminated, further improving the accuracy of the initial reflected microwave signal from the food, thereby improving the accuracy of refrigerator food identification.

[0083] For example, consider apples stored in a refrigerator, wrapped in plastic bags. The dielectric constant and loss tangent of the plastic are fixed, and the corresponding third reflected microwave signal is fixed. By filtering out the third reflected microwave signal from the filtered reflected microwave signal, interference from the plastic bag's radar-reflected microwave signal on the apple's initial reflected microwave signal is eliminated, improving the accuracy of the initial reflected microwave signal and, consequently, the accuracy of refrigerator food identification.

[0084] 5 , the present disclosure provides another method for identifying food in a refrigerator, including:

[0085] S501, the processor obtains data information of multiple preset categories of food through radar;

[0086] S502, the processor builds a food recognition model based on the data information;

[0087] S503, the processor obtains basic data of food in the refrigerator through radar;

[0088] S504, the processor identifies the location of the food based on the basic data;

[0089] S505, the processor generates a radar image of the food based on the basic data;

[0090] S506: The processor identifies the type and quantity of the food based on the radar image of the food, the location of the food, and the food recognition model.

[0091] In this embodiment, a food recognition model is constructed to directly identify food based on its radar image and location. Data information on multiple preset food categories is acquired via radar, and a food recognition model is constructed based on this data to increase the diversity of the food recognition model.

[0092] Optionally, the step of constructing a food recognition model based on the data information includes: generating preset radar images of multiple preset categories of food based on the data information; constructing a food recognition model through a deep learning method based on the preset radar images; or constructing a food recognition model based on the preset radar images and a classifier.

[0093] In this embodiment, data information of multiple preset food categories is used to generate preset radar images of multiple preset food categories, thereby enriching the diversity of the preset food recognition model. The food recognition model can be trained to form a food recognition model through deep learning.

[0094] Specifically, pre-processed radar images of various food categories are pre-processed to improve their quality and recognizability. Feature data is then extracted from these pre-processed radar images. This feature data can be extracted using deep learning methods, such as convolutional neural networks (CNNs). Based on this feature data, a food recognition model is then trained using deep learning methods, such as end-to-end training.

[0095] Building a food recognition model can form a food recognition model by training a classifier.

[0096] Specifically, pre-processed radar images of various food categories are pre-processed to improve their quality and recognizability. Feature data is then extracted from the pre-processed radar images of the various food categories. This feature data can be extracted using deep learning methods, such as convolutional neural networks (CNNs). A classifier is then trained based on the feature data and corresponding food data to form a food recognition model. Exemplarily, the classifier can be a support vector machine (SVM), random forest, or neural network.

[0097] Optionally, based on the data information, the step of constructing a food recognition model also includes: using the radar image to be tested and the basic data to be tested of the test food to detect the food recognition model; when the food recognition model is normal, completing the food recognition model detection; or when the food recognition model is abnormal, adjusting the number of preset radar images, the type of preset radar images, the type of extracted feature data or the classifier.

[0098] In this embodiment, by detecting the food recognition model and, if the food recognition model is abnormal, adjusting the number of preset radar images, the type of preset radar images, the type of extracted feature data or the classifier, the accuracy and reference value of the preset food recognition model are improved.

[0099] Specifically, the food recognition model is tested using the radar images to be tested and the basic data to be tested of the test food that does not participate in the food recognition model. The output results of the food recognition model are compared with the actual situation of the test food. The output results of the food recognition model include the type, number and location of the food. When the output results are consistent with the actual situation, the food recognition model is normal and the food recognition model detection is completed. When there is a discrepancy between the output results and the actual situation, the food recognition model is abnormal. When the food recognition model is abnormal, data information is added to increase the number of preset radar images, the types of preset radar images or the types of extracted feature data. Alternatively, when the food recognition model is abnormal, the type of classifier is replaced. By adding data information or replacing the classifier, the accuracy and reference value of the trained food recognition model can be improved.

[0100] 6 , the present disclosure provides another method for identifying food in a refrigerator, including:

[0101] S601, the processor obtains basic data of food in the refrigerator through radar;

[0102] S602, the processor obtains a reflected microwave signal of the food;

[0103] S603, the processor determines the relative position of the food and the radar based on the reflected microwave signal;

[0104] S604, the processor generates a radar image of the food based on the basic data;

[0105] S605: The processor identifies the type and quantity of the food based on the radar image of the food, the location of the food, and the food recognition model.

[0106] In this embodiment, the food's position and direction relative to the radar vary, and the phase information contained in the microwave signal reflected by the radar varies. The phase information in the reflected microwave signal can be used to determine the relative position of the food and the radar. Since the radar is fixed in position within the refrigerator, the food's position within the refrigerator can be determined.

[0107] For example, a refrigerator contains a variety of fruits, including apples and bananas. The fruits' positions and directions relative to the radar vary, resulting in different phase information in the microwave signals reflected by the radar. The phase information in the microwave signals reflected by the radar is used to determine the relative position of the fruits to the radar. Since the radar is fixed in the refrigerator, the relative position of the fruits to the radar is used to determine their location within the refrigerator.

[0108] Furthermore, the basic food data includes one or more of the following parameters: dielectric constant, loss tangent, and scattering point set. Based on the basic food data, a corresponding radar image of the food is generated. The radar image of the food is a three-dimensional information image of the food that reproduces the shape of the radar-reflected microwave signal from the food. The radar-reflected microwave signal shape refers to the three-dimensional shape of the radar-reflected microwave signal set formed by aggregating the radar-reflected microwave signals from the food. Different foods have different radar-reflected microwave signal shapes. Based on a preset food recognition model, the radar-reflected microwave signal shape of the food can be used to preliminarily determine the type of the food. Different foods have different dielectric constants and loss tangents, resulting in different absorption intensities for radar microwave signals. Therefore, the physical penetration information contained in the radar-reflected microwave signals from different foods varies. Different foods have different shapes and surface materials, resulting in different scattering point sets. Therefore, the surface scattering information contained in the radar-reflected microwave signals from different foods varies. Based on the preset food recognition model, the accuracy of food type recognition can be improved based on the food's location, physical penetration information, surface scattering information, and radar-reflected microwave signal shape.

[0109] For example, a refrigerator contains a variety of fruits, including apples and bananas. Radar images reproduce the shapes of radar-reflected microwave signals from different fruits. The shape of the radar-reflected microwave signal from apples is approximately circular, while that from bananas is approximately rectangular. Based on a preset food recognition model, the shape of the fruit's radar-reflected microwave signal can be used to preliminarily determine its type. Because radar microwave signals are penetrating, and apples and bananas have different dielectric constants and loss tangents, their absorption strengths differ. Consequently, the physical penetration information contained in the radar-reflected microwave signals from apples and bananas differs. Due to their different shapes and surface materials, the scattering point sets of apples and bananas differ, and the surface scattering information contained in the radar-reflected microwave signals from apples and bananas differs. Based on the preset food recognition model, the accuracy of fruit type recognition can be improved based on the fruit's location, physical penetration information, surface scattering information, and radar-reflected microwave signal shape.

[0110] Furthermore, the radar-reflected microwave signal intensity varies across different areas of a single food item. A single food item has an area with the strongest radar-reflected microwave signal. The radar-reflected microwave signal intensity varies at different distances from the radar. Based on the food recognition model and the relative position of the food item to the radar, the number of food items is determined by comparing the radar-reflected microwave signal intensity across the area with the corresponding regional intensity threshold.

[0111] For example, a refrigerator contains a variety of fruits, including apples and bananas. After determining the type of fruit, the number of fruits of the same type is determined, using apples as an example. The radar-reflected microwave signal of a single apple has the strongest region at its center. The intensity of the radar-reflected microwave signal varies at different distances from the radar. Based on a preset food recognition model and the relative position of the apple to the radar, the radar-reflected microwave signal intensity at each apple region is compared with an apple region intensity threshold to determine the number of apples.

[0112] 7 , the present disclosure provides another method for identifying food in a refrigerator, including:

[0113] S701, the processor obtains basic data of food in the refrigerator through radar;

[0114] S702, the processor identifies the location of the food based on the basic data;

[0115] S703, the processor generates a radar image of the food based on the basic data;

[0116] S704, the processor identifies the type and quantity of the food based on the radar image of the food, the location of the food, and the food recognition model;

[0117] S705: The processor updates the food according to the basic data.

[0118] In this embodiment, the refrigerator acquires basic data about the food inside in real time. This data includes the food's movement trajectory and its relative position to a radar. As a user takes food in and out of the refrigerator, the food moves relative to the refrigerator. The relative position of the food inside the refrigerator to the radar is detected in real time using the radar's microwave reflection signal. The food's movement trajectory is determined by changes in the relative position of the food to the radar. Upon detecting a change in the relative position or movement trajectory of the food to the radar, the processor executes steps S701 to S704 to update the food inside the refrigerator, improving the reliability of the refrigerator's intelligent food management and enhancing the refrigerator's practicality.

[0119] For example, take the case of a user taking an apple and putting it in. The refrigerator obtains the basic data of the food in the refrigerator in real time. When the user takes the apple in and out, the apple moves relative to the refrigerator. When the radar microwave reflection signal detects that the relative position or motion trajectory of the food in the refrigerator and the radar changes, the processor obtains the basic data of the food in the refrigerator through the radar. The relative position of the food and the radar refers to the final relative position of the food and the radar determined based on the final position of the motion trajectory. Then, based on the radar image of the food and the position of the food, the food is re-identified based on the food recognition model. The basic data of the food in the refrigerator is obtained in real time through the refrigerator, and the identified food is updated to improve the reliability of the intelligent management of refrigerator ingredients and improve the practicality of the refrigerator.

[0120] As shown in FIG8 , an embodiment of the present disclosure provides a control device 200 for identifying food in a refrigerator, comprising a data acquisition module 21, an image generation module 22, and an analysis and identification module 23. The data acquisition module 21 is configured to acquire basic data of food in the refrigerator via radar; the image generation module 22 is configured to generate a radar image of the food based on the basic data; and the analysis and identification module 23 is configured to identify the food based on the radar image and / or the basic data.

[0121] Using the control device 200 for identifying food in a refrigerator provided by an embodiment of the present disclosure, the data acquisition module 21 obtains basic data of the food in the refrigerator based on the radar reflected microwave signal. The image generation module 22 generates a radar image of the food in the refrigerator based on the basic data. The analysis and identification module 23 analyzes and identifies the radar image and / or basic data of the food based on a preset food recognition model, thereby achieving food recognition. The control device 200 for identifying food in a refrigerator provided by an embodiment of the present disclosure identifies food through radar reflected microwave signals and radar imaging analysis to improve the accuracy and reliability of the control device's recognition.

[0122] As shown in Figure 9, an embodiment of the present disclosure provides a refrigerator 300, including a processor 100 and a memory 101. Optionally, the refrigerator may further include a communication interface 102 and a bus 103. The processor 100, the communication interface 102, and the memory 101 may communicate with each other via the bus 103. The communication interface 102 may be used for information transmission. The processor 100 may call the logic instructions in the memory 101 to execute the food identification method for the refrigerator of the above embodiment.

[0123] In addition, the logic instructions in the memory 101 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

[0124] Memory 101, as a computer-readable storage medium, can be used to store software programs and computer-executable programs, such as the program instructions / modules corresponding to the methods in the embodiments of the present disclosure. Processor 100 executes the program instructions / modules stored in memory 101 to perform functional applications and data processing, thereby implementing the food identification method for a refrigerator in the above-described embodiments.

[0125] The memory 101 may include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function; the data storage area may store data generated based on the use of the terminal device. Furthermore, the memory 101 may include high-speed random access memory and non-volatile memory.

[0126] As shown in Figure 10, an embodiment of the present disclosure provides a refrigerator 300, including: a refrigerator body, and the above-mentioned control device 200 for food identification in the refrigerator. The control device 200 for food identification in the refrigerator is installed on the product body. The installation relationship described here is not limited to placement inside the product, but also includes installation connections with other components of the product, including but not limited to physical connections, electrical connections or signal transmission connections. It can be understood by those skilled in the art that the control device 200 for food identification in the refrigerator can be adapted to a feasible product body, thereby realizing other feasible embodiments.

[0127] An embodiment of the present disclosure provides a storage medium storing computer-executable instructions, wherein the computer-executable instructions are configured to execute the above-mentioned food identification method for a refrigerator.

[0128] An embodiment of the present disclosure provides a computer program, which, when executed by a computer, enables the computer to implement the above-mentioned food identification method for a refrigerator.

[0129] An embodiment of the present disclosure provides a computer program product, which includes computer instructions stored on a computer-readable storage medium. When the program instructions are executed by a computer, the computer implements the above-mentioned food identification method for a refrigerator.

[0130] The aforementioned computer-readable storage medium may be a transient computer-readable storage medium or a non-transitory computer-readable storage medium.

[0131] The technical solution of the embodiments of the present disclosure may be embodied in the form of a software product, which is stored in a storage medium and includes one or more instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the embodiments of the present disclosure. The aforementioned storage medium may be a non-transitory storage medium, including: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program code, or a transient storage medium.

[0132] The above description and the accompanying drawings fully illustrate the embodiments of the present disclosure so that those skilled in the art can practice them. Other embodiments may include structural, logical, electrical, process and other changes. The embodiments represent only possible variations. Unless explicitly required, individual components and functions are optional, and the order of operations may vary. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. Moreover, the words used in this application are only used to describe the embodiments and are not used to limit the claims. As used in the description of the embodiments and claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to also include plural forms. Similarly, the term "and / or" as used in this application refers to any and all possible combinations of one or more associated listings. In addition, when used in this application, the term "comprise" and its variations "comprises" and / or comprising refer to the presence of stated features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups of these. In the absence of further restrictions, an element defined by the sentence "comprising a..." does not exclude the presence of other identical elements in the process, method or device that includes the element. In this article, each embodiment may focus on the differences from other embodiments, and the same and similar parts between the various embodiments can be referenced to each other. For the methods, products, etc. disclosed in the embodiments, if they correspond to the method part disclosed in the embodiments, then the relevant parts can be found in the description of the method part.

[0133] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software may depend on the specific application and design constraints of the technical solution. The technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the embodiments of the present disclosure. The technicians will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0134] In the embodiments disclosed herein, the disclosed methods and products (including but not limited to devices, equipment, etc.) can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units can be merely a logical functional division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, and can be electrical, mechanical or other forms. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected to implement this embodiment according to actual needs. In addition, the functional units in the embodiments of the present disclosure may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0135] The flowcharts and block diagrams in the accompanying drawings show the possible implementation architectures, functions and operations of the systems, methods and computer program products according to the embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment or part of the code, and the module, program segment or part of the code contains one or more executable instructions for implementing the specified logical functions. In some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, or they can sometimes be executed in the opposite order, which can depend on the functions involved. In the descriptions corresponding to the flowcharts and block diagrams in the accompanying drawings, the operations or steps corresponding to different boxes can also occur in an order different from that disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps can actually be executed substantially in parallel, or they can sometimes be executed in the opposite order, which can depend on the functions involved. Each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified function or action, or may be implemented by a combination of dedicated hardware and computer instructions.

[0136] The reference to any prior art in the specification is not and should not be taken as an admission or any form of suggestion that the prior art forms part of the common general knowledge in the application area or any other jurisdiction, or that the prior art could reasonably be understood and regarded as relevant by a person skilled in the art.

Claims

1. A food identification method for a refrigerator, wherein the refrigerator is provided with a radar, characterized in that: Food identification methods include: Obtain basic data of food in the refrigerator through radar; Based on the basic data, identify the location of food; Generate radar images of food based on basic data; The type and quantity of food are identified based on the radar image of the food, the location of the food and the food recognition model.

2. The food identification method for a refrigerator according to claim 1, wherein: The steps of identifying the type and quantity of food according to the radar image of the food, the location of the food and the food recognition model include: Preprocessing radar images of food; Extracting feature data based on the pre-processed radar image of the food; Identify the type and quantity of food based on feature data, location of food and food recognition model.

3. The food identification method for a refrigerator according to claim 1 or 2, wherein: The steps to obtain basic data of food in the refrigerator through radar include: Acquire initial reflected microwave signals of food in the refrigerator by radar; The basic data of the food is obtained based on the initial reflected microwave signal.

4. The food identification method for a refrigerator according to claim 3, wherein: The steps of obtaining the initial reflected microwave signal of the food in the refrigerator by radar include: Acquire first reflected microwave signals at various relative positions of the refrigerator; Acquire a second reflected microwave signal of food in the refrigerator; The first reflected microwave signal is filtered out from the second reflected microwave signal to obtain an initial reflected microwave signal of the food.

5. The food identification method for a refrigerator according to claim 4, wherein: The step of filtering the first reflected microwave signal from the second reflected microwave signal to obtain the initial reflected microwave signal of the food comprises: acquiring a third reflected microwave signal of a food package in the refrigerator; filtering the first reflected microwave signal from the second reflected microwave signal to obtain a filtered reflected microwave signal; The third reflected microwave signal is filtered out from the filtered reflected microwave signal to obtain the initial reflected microwave signal of the food in the refrigerator.

6. The food identification method for a refrigerator according to any one of claims 1 to 5, wherein: Before obtaining basic data of food in the refrigerator through radar, it also includes: Obtain data information of various preset categories of food through radar; Build a food recognition model based on data information.

7. The food identification method for a refrigerator according to claim 6, wherein: According to the data information, the steps of building a food recognition model include: Generate preset radar images of multiple preset categories of food based on the data information; Based on the preset radar images, a food recognition model is constructed through deep learning methods; or Based on the preset radar images, a food recognition model is built by training the classifier.

8. The food identification method for a refrigerator according to any one of claims 1 to 7, wherein: Based on the basic data, the steps to identify the location of food include: Acquiring reflected microwave signals of food; The relative position of the food and the radar is determined based on the reflected microwave signal.

9. A refrigerator, comprising a processor and a memory storing program instructions, characterized in that: The processor is configured to execute the food identification method for a refrigerator according to any one of claims 1 to 8 when running the program instructions.

10. A storage medium storing program instructions, characterized in that: When the program instructions are executed, the food identification method for a refrigerator as claimed in any one of claims 1 to 8 is executed.

11. A computer program, when executed by a computer, enables the computer to implement the food identification method for a refrigerator according to any one of claims 1 to 8.

12. A computer program product, comprising computer instructions stored on a computer-readable storage medium, and when the program instructions are executed by a computer, the computer is enabled to implement the food identification method for a refrigerator as claimed in any one of claims 1 to 8.

Images