Method, device and equipment for identifying freshness of food materials in refrigerator and storage medium

Abstract

The invention relates to the technical field of intelligent refrigerators, and discloses a refrigerator food material freshness identification method, device and equipment and a storage medium. The method comprises the following steps: acquiring first spectral data information of a current food material in a refrigerator, and inverting a corresponding current red, yellow and blue (RGB) image; determining second spectral data information corresponding to the packaging material in the current RGB image, and stripping the second spectral data information from the first spectral data information to obtain third spectral data information; and identifying the current freshness of the current food material through a food material freshness artificial intelligence AI model according to the third spectrum data information. Thus, intelligent monitoring of the freshness of the food materials in the refrigerator is achieved, the accuracy of freshness recognition of the food materials is improved, and the intelligence of the refrigerator is also improved.

Description

Technical Field

[0001] The present application relates to the technical field of smart refrigerators, for example, to methods, devices, equipment and storage media for identifying the freshness of refrigerator food. Background Art

[0002] Refrigerators are now essential appliances for home life, preserving food through refrigeration and freezing functions. However, it is still difficult for users to accurately test the freshness of food stored in the refrigerator, especially packaged food. They often use intuitive sensory evaluation, but sensory evaluation often does not have a high degree of control over the freshness of food. Accurate methods for testing freshness with physical and chemical indicators often only exist in laboratories, and users cannot test physical and chemical indicators at home.

[0003] Storing food with poor freshness in the refrigerator, such as food that has expired, may not only contaminate other food and waste resources, but may also cause physical harm to users.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present application, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] In order to provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. The summary is not an extensive review, nor is it intended to identify key / critical components or delineate the scope of protection of these embodiments, but rather serves as a prelude to the detailed description that follows.

[0006] The embodiments of the present disclosure provide a method, device, equipment and storage medium for identifying the freshness of refrigerator food, so as to solve the technical problem of refrigerator resource waste.

[0007] In some embodiments, the method comprises:

[0008] Obtain the first spectrum data information of the current food in the refrigerator, and invert the corresponding current red, yellow, and blue RGB image;

[0009] Determine second spectral data information corresponding to the packaging material in the current RGB image, and strip the second spectral data information from the first spectral data information to obtain third spectral data information;

[0010] According to the third spectrum data information, the current freshness of the current ingredients is identified through the ingredient freshness artificial intelligence AI model.

[0011] In some embodiments, it also includes:

[0012] Obtaining food spectrum data sample information and corresponding freshness sample information of the food sample;

[0013] Obtaining packaging material spectrum data sample information of packaging material samples;

[0014] According to the packaging material spectral data sample information, the food material spectral data sample information and their corresponding freshness sample information, the food material freshness artificial intelligence AI model is obtained and saved through the AI ​​intelligent algorithm.

[0015] In some embodiments, the inverted corresponding current red, yellow, and blue (RGB) image includes:

[0016] Normalizing the first spectrum data information to obtain normalized data information;

[0017] Through the response function, the normalized data information is mapped to the red, yellow and blue RGB channels, and image processing is performed to obtain the corresponding RGB image, wherein the response function is constructed based on the spectral sample data corresponding to different wavelengths and the corresponding RGB color response data.

[0018] In some embodiments, determining the second spectral data information corresponding to the packaging material in the current RGB image includes:

[0019] Perform image recognition based on the current RGB image to determine the current packaging material corresponding to the current food;

[0020] According to the packaging material spectrum data sample information, second spectrum data information corresponding to the current packaging material is determined.

[0021] In some embodiments, obtaining the third spectrum data information includes:

[0022] Determine a peak band in which a peak of the spectral emissivity in the second spectral data information appears;

[0023] The data information corresponding to the peak band in the first spectrum data information is deducted to obtain the third spectrum data information.

[0024] In some embodiments, after identifying the current freshness of the current food, the method further includes:

[0025] According to the current freshness, control the refrigerator to remind the user;

[0026] When the current freshness is at a level corresponding to the availability of the food, the refrigerator is controlled to lower the temperature of the compartment where the current food is located.

[0027] In some embodiments, the apparatus comprises:

[0028] An acquisition and inversion module is configured to acquire first spectral data information of current food in the refrigerator and invert the corresponding current red, yellow and blue (RGB) image;

[0029] a determination stripping module, configured to determine second spectral data information corresponding to the packaging material in the current RGB image, and strip the second spectral data information from the first spectral data information to obtain third spectral data information;

[0030] The recognition module is configured to identify the current freshness of the current food according to the third spectral data information through the food freshness artificial intelligence AI model.

[0031] In some embodiments, the device for identifying the freshness of refrigerator ingredients includes a processor and a memory storing program instructions, and the processor is configured to execute the above-mentioned method for identifying the freshness of refrigerator ingredients when executing the program instructions.

[0032] In some embodiments, the device includes a device body; the above-mentioned device for identifying the freshness of refrigerator food is installed on the device body.

[0033] In some embodiments, the storage medium stores program instructions, and when the program instructions are run, the method for identifying the freshness of refrigerator ingredients is executed.

[0034] The method, device and apparatus for identifying the freshness of refrigerator food provided by the embodiments of the present disclosure can achieve the following technical effects:

[0035] The first spectral data information corresponding to the food in the refrigerator is obtained, and the second spectral data information corresponding to the packaging material of the food can also be obtained. The second spectral data information is stripped from the first spectral data information to obtain the third spectral data information. Then, the third spectral data information is identified through the food freshness AI model to obtain the freshness information of the food. In this way, the freshness of the food in the refrigerator is intelligently monitored, the accuracy of food freshness identification is improved, the chance of food waste is reduced, and the user's dietary health is further guaranteed, which improves the intelligence and user experience of the refrigerator.

[0036] The above general description and the following description are exemplary and explanatory only and are not intended to limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] One or more embodiments are exemplarily described by corresponding drawings, which do not limit the embodiments. Elements with the same reference numerals in the drawings are shown as similar elements, and the drawings do not constitute a scale limitation, and wherein:

[0038] Figure 1It is a structural schematic diagram of a refrigerator food freshness identification system provided by an embodiment of the present disclosure;

[0039] Figure 2 It is a flow chart of a method for identifying the freshness of refrigerator food provided by an embodiment of the present disclosure;

[0040] Figure 3 is a schematic diagram of spectral data information provided by an embodiment of the present disclosure;

[0041] Figure 4 It is a flow chart of a method for identifying the freshness of refrigerator food provided by an embodiment of the present disclosure;

[0042] Figure 5 It is a structural schematic diagram of a device for identifying the freshness of refrigerator food provided by an embodiment of the present disclosure;

[0043] Figure 6 It is a structural schematic diagram of a device for identifying the freshness of refrigerator food provided by an embodiment of the present disclosure;

[0044] Figure 7 It is a structural schematic diagram of a device for identifying the freshness of refrigerator food provided by an embodiment of the present disclosure;

[0045] Figure 8 It is a schematic diagram of a device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0046] In order to be able to understand the features and technical contents of the embodiments of the present disclosure in more detail, the implementation of the embodiments of the present disclosure is described in detail below in conjunction with the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present disclosure. In the following technical description, for the convenience of explanation, a full understanding of the disclosed embodiments is provided through multiple details. However, one or more embodiments can still be implemented without these details. In other cases, to simplify the drawings, well-known structures and devices can be simplified for display.

[0047] The terms "first", "second", etc. in the specification and claims of the embodiments of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged where appropriate, so that the embodiments of the embodiments of the present disclosure described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions.

[0048] Unless otherwise stated, the term "plurality" means two or more.

[0049] In the embodiment of the present disclosure, the character " / " indicates that the preceding and following objects are in an "or" relationship. For example, A / B indicates: A or B.

[0050] The term "and / or" is a description of the association relationship between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or, A and B.

[0051] In the disclosed embodiment, the first spectral data information corresponding to the food in the refrigerator can be obtained by hyperspectral technology, and the corresponding RGB image can be inverted. In this way, after the packaging material corresponding to the food is identified according to the RGB image, the corresponding second spectral data information is obtained, and the second spectral data information is stripped from the first spectral data information. After obtaining the third spectral data information, the third spectral data information can be identified through the food freshness artificial intelligence (AI) model to obtain the food freshness information, thereby realizing intelligent monitoring of the freshness of the food in the refrigerator, reducing the probability of food waste, and further ensuring the user's dietary health, improving the intelligence and user experience of the refrigerator. In addition, through the food freshness AI model, the shelf life of the food can also be calculated, and the corresponding refrigerator compartment temperature control can be performed, further reducing the probability of food waste and further improving the intelligence of the refrigerator.

[0052] Among them, the AI ​​model of food freshness is obtained through training and learning through AI intelligent algorithm based on multiple food spectral data sample information and its corresponding freshness sample information. In the embodiment of the present disclosure, the device used to identify the freshness of refrigerator food can be a refrigerator or a sports device. Since self-learning training is performed to obtain the AI ​​model of food freshness, obtaining the spectral sample data corresponding to different wavelengths, and the corresponding RGB color response data to construct the corresponding function between RGB and spectral data, etc. all require large memory support, therefore, in some embodiments, the device used to identify the freshness of refrigerator food can be a sports device. Figure 1 Schematic diagram of a system for identifying the freshness of food in a refrigerator provided by an embodiment of the present disclosure. Figure 1 As shown, the system for identifying the freshness of refrigerator food includes: a cloud server 100 and a refrigerator 200, wherein the cloud server 100 can perform data communication with the refrigerator 200, and the refrigerator 200 can include a spectral device 210.

[0053] In this way, the cloud server 100 can obtain the food spectral data sample information and the corresponding freshness sample information of the food sample, and train and learn these samples through the AI ​​intelligent algorithm to obtain the corresponding food freshness AI model. Among them, the AI ​​intelligent algorithm may include: machine learning algorithms, such as support vector machine (SVM), random forest, K nearest neighbor and other classification algorithms, or AI intelligent algorithms include: convolutional neural network (CNN), recurrent neural network (RNN) and convolution-recurrent neural network (CRNN) and other deep learning architecture algorithms.

[0054] The food sample includes as many types of food as possible. The cloud server 100 may obtain the food spectrum data sample information of the food sample and its corresponding freshness sample information in various ways, such as directly obtaining the food spectrum data sample information of the food sample and its corresponding freshness sample information through network communication; or obtaining the food spectrum data sample information of the food sample by communicating with a device having a spectral device, and obtaining the corresponding freshness sample information of the food sample by communicating with a device having chemical or microbiological analysis capabilities, etc.

[0055] In some embodiments, the cloud server 100 can obtain the food spectral data information and the corresponding freshness information corresponding to the same food at different storage times, and use them as sample information. For example, a piece of fresh pork is selected and frozen, and then the hyperspectral image of the unpackaged meat is obtained every 12 hours as the corresponding food spectral data sample information, and the corresponding freshness sample information is obtained at the same time. Among them, the freshness of the meat can be accurately obtained by chemical and microbiological analysis methods, which is used as a one-to-one correspondence with the hyperspectral image of each stage obtained above. In this way, each food spectral data sample information corresponds to a storage time label.

[0056] The cloud server 100 can obtain the packaging material spectrum data sample information of the packaging material sample. Among them, the packaging material may include: plastic wrap, transparent packaging bags, or transparent plastic boxes, etc. Different packaging materials correspond to different packaging material spectrum data information, and the packaging material spectrum data information can also be used as a sample for model training, that is, the packaging material spectrum data sample information. That is, in some embodiments, the cloud server 100 can train and learn through the AI ​​intelligent algorithm based on the packaging material spectrum data sample information, the food spectrum data sample information and its corresponding freshness sample information, to obtain and save the food freshness artificial intelligence AI model.

[0057] In this way, after the current food is stored in the refrigerator 200, the first spectral data information of the current food can be obtained periodically or in real time through the spectral device 210, and sent to the cloud server 100, so that the cloud server 200 can invert the corresponding current red, yellow, and blue RGB image. In this way, after identifying the packaging material corresponding to the food according to the RGB image, the corresponding second spectral data information is obtained, and the second spectral data information is stripped from the first spectral data information. After obtaining the third spectral data information, the third spectral data information is identified through the food freshness AI model to obtain the freshness information of the food, thereby realizing intelligent monitoring of the freshness of the food in the refrigerator.

[0058] Of course, in some embodiments, the refrigerator 200 can obtain the first spectral data information of the current food periodically or in real time through the spectral device 210, and can locally invert the corresponding current red, yellow, and blue (RGB) image. In this way, after identifying the packaging material corresponding to the food according to the RGB image, the corresponding second spectral data information is obtained according to the locally stored material spectral data sample information, and the second spectral data information is stripped from the first spectral data information. After obtaining the third spectral data information, the third spectral data information is identified through the locally stored food freshness AI model to obtain the freshness information of the current food.

[0059] In some embodiments, the refrigerator food freshness identification system may further include: a terminal, a cloud server or a refrigerator identifies the freshness of the current food and sends it to a terminal bound to the refrigerator, so that the user can learn about the freshness of the food in a timely manner without being in the refrigerator's operating area. Figure 1 As shown, the terminal 300 can communicate with the cloud server 100. After the cloud server 100 obtains the freshness information of the food, it can send it to the terminal 300, further improving the intelligence of the refrigerator.

[0060] Figure 2 FIG. 1 is a flow chart of a method for identifying the freshness of refrigerator food provided by an embodiment of the present disclosure. Figure 2 As shown in the figure, the process of identifying the freshness of refrigerator ingredients includes:

[0061] Step 201: Obtain first spectrum data information of current food in the refrigerator, and invert the corresponding current red, yellow, and blue (RGB) image.

[0062] The refrigerator is equipped with a spectral module. In this way, there are multiple ingredients in the refrigerator, one of which can be the current ingredient. Through the spectral module, the device for identifying the freshness of refrigerator ingredients can obtain the first spectral data information of the current ingredient. Of course, the device for identifying the freshness of refrigerator ingredients can be a cloud server or a refrigerator.

[0063] After the first spectral data information is obtained, the corresponding current red, yellow, blue (RGB) image can be inverted through the first spectral data information. Of course, in some embodiments, a response function has been constructed, so that the corresponding current red, yellow, blue (RGB) image is inverted including: normalizing the first spectral data information to obtain normalized data information; mapping the normalized data information to the red, yellow, blue (RGB) channels through the response function, and performing image processing to obtain the corresponding current RGB image, wherein the response function is constructed based on the spectral sample data corresponding to different wavelengths and the corresponding RGB color response data.

[0064] The spectrum sample data corresponding to different wavelengths can be collected by the spectrum device, and the corresponding RGB color can be obtained. Therefore, according to the spectrum sample data and the corresponding RGB color response data, the response function between the spectrum data and the RGB color data can be constructed. In this way, the first spectrum data information can be normalized by linear stretching or other adapted normalization algorithms to obtain normalized data information within an appropriate range. Then, the normalized data information can be mapped to the red, yellow, and blue RGB channels through the response function to obtain a to-be-processed RGB image. Then, according to the application scenario or user needs, the to-be-processed RGB image can be processed, such as contrast enhancement, denoising, etc., to obtain the corresponding current RGB image. Of course, in some embodiments, image processing may not be performed, and the to-be-processed RGB image can be directly determined as the corresponding current RGB image.

[0065] Step 202: Determine second spectral data information corresponding to the packaging material in the current RGB image, and strip the second spectral data information from the first spectral data information to obtain third spectral data information.

[0066] Image recognition can be performed based on the current RGB image to determine the current packaging material corresponding to the current food; and the second spectral data information corresponding to the current packaging material can be determined. The current RGB image can be identified by image comparison or neural network to determine the current packaging material corresponding to the current food, wherein the packaging materials include: no packaging, plastic wrap packaging, transparent packaging bag, transparent plastic box packaging, etc.

[0067] The equipment used for identifying the freshness of refrigerator food obtains the packaging material spectrum of the packaging material sample and the data sample information, and obtains the packaging material spectrum data information of each packaging material through training and learning through AI intelligent algorithm, thereby determining the current packaging material, that is, the packaging material spectrum data information that is basically saved, and obtaining the second spectrum data information corresponding to the current packaging material, including: determining the second spectrum data information corresponding to the current packaging material according to the packaging material spectrum data sample information. Of course, if the current packaging material is unpackaged, the second spectrum data information is blank information.

[0068] After obtaining the first spectrum data information and the second spectrum data information, the second spectrum data information may be stripped from the first spectrum data information to obtain the third spectrum data information.

[0069] Among them, when the current food is not packaged, the second spectral data information is blank information, that is, the first spectral data information is the third spectral data information, and when the current food is packaged, in some embodiments, the peak band where the spectral emissivity peak appears in the second spectral data information is determined; the data information corresponding to the peak band in the first spectral data information is deducted to obtain the third spectral data information.

[0070] Figure 3 is a schematic diagram of spectral data information provided by an embodiment of the present disclosure. Figure 3 (a) is a schematic diagram of first spectral data information corresponding to the current packaged food, Figure 3 (b) is a schematic diagram of the second spectrum data information corresponding to the package. Figure 3 As shown in (a), there are three peaks in reflectivity within 300-900nm; Figure 3 As shown in (b), the spectral reflectance peak appears in the band of 750-900nm. The two spectral data information are compared, the band of 750-900nm is deducted, and the spectral data of the packaged food in the band of 300-750nm is selected as the third spectral data information.

[0071] Step 203: According to the third spectral data information, the current freshness of the current food is identified through the food freshness artificial intelligence AI model.

[0072] The AI ​​model of food freshness is generated by training and learning a large number of food spectral data sample information and its corresponding freshness sample information through AI intelligent algorithms. In some embodiments, the food spectral data sample information and its corresponding freshness sample information of the food samples are obtained; the packaging material spectral data sample information of the packaging material samples is obtained; according to the packaging material spectral data sample information, the food spectral data sample information and its corresponding freshness sample information, the AI ​​intelligent algorithm is trained and learned to obtain and save the food freshness artificial intelligence AI model.

[0073] Among them, AI intelligent algorithms may include: machine learning algorithms, such as support vector machines (SVM), random forests, K nearest neighbors and other classification algorithms, or AI intelligent algorithms include: convolutional neural networks (CNN), recurrent neural networks (RNN) and convolution-recurrent neural networks (CRNN) and other deep learning architecture algorithms. Freshness includes: the corresponding degree of freshness of ingredients, the corresponding degree of edibility of ingredients, and the corresponding degree of expiration of ingredients, etc. The freshness of the food samples can be determined by chemical or microbiological analysis, that is, the corresponding freshness sample information is obtained.

[0074] For example, freshness can be assessed by measuring certain chemical indicators in meat. This can include pH, amino acid content, volatile odor substances, etc. These specific chemical characteristics will change over time. Alternatively, fresh meat usually contains a lower number of microorganisms, while the growth of bacteria and other microorganisms over time will cause the number to increase, so the freshness of meat can be assessed by testing the microbial load in the meat.

[0075] The food freshness AI model is generated and saved through training. In this way, after the third spectral data information is obtained in step 202, it can be input into the food freshness AI model, and the food freshness AI model can output the current freshness corresponding to the current food, and can identify whether the current food is fresh, edible or expired.

[0076] In some embodiments, during the process of training an AI model for food freshness, food spectral data information corresponding to different storage times of the same food and its corresponding freshness information can be obtained and used as sample information. In this way, each food spectral data sample information corresponds to a storage time label. For example, the food spectral data information and its corresponding freshness information of a frozen chicken are obtained every 8 hours. In this way, there is food spectral data sample information carrying a storage time label and its corresponding freshness information. Therefore, the food freshness AI model generated after training and learning through AI intelligent algorithms can also calculate the shelf life, that is, the freshness output by the food freshness AI model may include shelf life information.

[0077] Among them, when the third spectral data information is input through the food freshness AI model, it can find the matching food spectral data sample information with a storage time label, output the corresponding current freshness, and estimate the corresponding shelf life information based on multiple food spectral data sample information with a storage time label corresponding to the same food.

[0078] For example: the current freshness corresponding to the AI ​​model output of food freshness is the corresponding degree of edibility of the food, and the corresponding preservation time label of the food spectral data sample information is 25 days, while the preservation time label of the food spectral data sample information corresponding to the freshness of the same food is the corresponding degree of expiration is 30 days. Therefore, the corresponding shelf life information can be estimated to be 5 days.

[0079] Of course, after identifying the current freshness of the current ingredients, it also includes: controlling the refrigerator to perform user reminder processing according to the current freshness. Among them, if the device used for identifying the freshness of refrigerator ingredients is a refrigerator, that is, the refrigerator is controlled to perform user reminder processing according to the current freshness, including: one or more of sound, light, electricity, and application APP push, etc. For example: if the current freshness is the corresponding degree of freshness of the ingredients or the corresponding degree of edibility of the ingredients, only voice broadcast is performed through the voice module; if the current freshness is the corresponding degree of expired ingredients, not only voice broadcast is performed through the voice module, but also APP push can be performed to the terminal bound to the refrigerator.

[0080] Of course, the device used to identify the freshness of refrigerator ingredients is a cloud server, and the cloud server controls the refrigerator to remind the user according to the current freshness. For example, the cloud server generates corresponding voice information according to the current freshness and sends it to the refrigerator, so that the voice module of the refrigerator broadcasts it.

[0081] In this way, users can obtain the freshness information of current ingredients in a timely manner, reducing the chance of food waste and further ensuring the health of users' diet.

[0082] It can be seen that in the disclosed embodiment, the first spectral data information corresponding to the food in the refrigerator can be obtained through hyperspectral technology, and the corresponding RGB image can be inverted. In this way, after the packaging material corresponding to the food is identified according to the RGB image, the corresponding second spectral data information is obtained, and the second spectral data information is stripped from the first spectral data information. After obtaining the third spectral data information, the third spectral data information can be identified through the food freshness AI model to obtain the food freshness information, thereby realizing the intelligent monitoring of the freshness of the food in the refrigerator and improving the intelligence of the refrigerator. Of course, it can also reduce the probability of food waste, further ensure the user's dietary health, and improve the user experience.

[0083] Of course, in some embodiments, the current freshness output by the AI ​​model of food freshness also includes the estimated shelf life information, and the voice module of the refrigerator can also be controlled to broadcast it. Alternatively, in some embodiments, after identifying the current freshness of the current food, it also includes: controlling the refrigerator to perform user reminder processing according to the current freshness; if the current freshness is the corresponding degree of edible food, controlling the refrigerator to lower the temperature of the compartment where the current food is located.

[0084] Among them, the current freshness includes the estimated shelf life information, and the target temperature of the compartment where the current food is located can be inferred based on the shelf life information. Then, when the current freshness is at the corresponding level of edible food, the refrigerator is controlled to lower the temperature of the compartment to the target temperature. This further reduces the probability of food waste and further improves the intelligence of the refrigerator.

[0085] Of course, in some embodiments, after the refrigerator performs user reminder processing, the user can feedback the food freshness correction information through the terminal, the refrigerator's interactive interface, etc. In this way, the cloud server or the refrigerator can obtain the food freshness correction information corresponding to the current food, and can train and adjust the food freshness AI model according to the current food and the corresponding food freshness correction information, thereby further improving the accuracy of the refrigerator's food freshness recognition.

[0086] The operation flow is summarized into a specific embodiment below to illustrate the process for identifying the freshness of refrigerator food provided by the embodiment of the present invention.

[0087] In one embodiment of the present disclosure, a refrigerator food freshness identification system may include a cloud server, a refrigerator, and a terminal bound to the refrigerator. The cloud server has trained and learned multiple food spectral data sample information and its corresponding freshness sample information through the algorithm of the convolutional neural network (CNN) deep learning architecture, and obtained and saved the food freshness AI model. In addition, the cloud server also constructs and saves the response function between the spectrum and the RGB image. The refrigerator includes a spectral device and can communicate with the cloud server. Of course, the cloud server can also communicate with the terminal.

[0088] Figure 4 FIG. 1 is a flow chart of a method for identifying the freshness of refrigerator food provided by an embodiment of the present disclosure. Figure 4 As shown in the figure, the refrigerator food freshness identification process includes:

[0089] Step 401: The refrigerator obtains first spectral data information of current food through a spectral device and sends it to a cloud server.

[0090] The refrigerator can obtain the spectral data information of the food in the refrigerator at a fixed time, or, when a set trigger condition is reached, for example, the refrigerator door is opened, the refrigerator obtains the first spectral data information of the current food through the spectral device. The refrigerator can communicate with the cloud server, so the cloud server can receive the first spectral data information sent by the refrigerator.

[0091] Step 402: The cloud server normalizes the first spectral data information to obtain normalized data information, maps the normalized data information to the red, yellow and blue (RGB) channels through a response function, and performs image processing to obtain a corresponding current RGB image.

[0092] In this way, the cloud server inverts the corresponding current RGB image.

[0093] Step 403: The cloud server performs image recognition based on the current RGB image to identify the corresponding current packaging material, and determines the second spectrum data information corresponding to the current packaging material based on the stored packaging material spectrum data sample information of the packaging material sample.

[0094] Step 404: Determine whether the second spectrum data information is blank information. If so, execute step 405; otherwise, execute step 406.

[0095] When the current food is packaged, the second spectrum data information is blank information, and step 405 is executed. When the current food is packaged, step 406 can be executed.

[0096] Step 405 : The cloud server determines the first spectrum data information as the third spectrum data information. Then, the process proceeds to step 407 .

[0097] Step 406: The cloud server determines the peak band where the spectral emissivity peak appears in the second spectral data information, and deducts the spectral data information corresponding to the peak band in the first spectral data information to obtain the third spectral data information. Then proceed to step 407.

[0098] That is, the cloud server separates the second spectral data information from the first spectral data information to obtain the third spectral data information.

[0099] Step 407: The cloud server inputs the third spectral data information into the food freshness AI model for training and learning, obtains the current freshness of the current food, and can estimate the shelf life of the food, carrying the shelf life information in the current freshness.

[0100] Step 408: When the current freshness is at a level corresponding to the freshness of the food, the cloud server sends a first voice command to the refrigerator to control the refrigerator to make a voice announcement indicating the freshness of the food.

[0101] Step 409: When the current freshness corresponds to the edible degree of the food, the cloud server determines the target temperature of the compartment where the current food is located in the refrigerator according to the shelf life information included in the current freshness, sends a second voice command to the refrigerator, controls the refrigerator to voice broadcast the information about the edible nature and shelf life of the food, and controls the refrigerator to lower the temperature of the compartment where the current food is located to the target temperature.

[0102] Step 410: When the current freshness is at a level corresponding to the expiration of the food, the cloud server sends a third voice command to the refrigerator to control the refrigerator to make a voice announcement of the expiration of the food, and sends the information of the expiration of the food to the terminal.

[0103] It can be seen that in this embodiment, the AI ​​model of food freshness is trained and maintained in the cloud server, and the response function between the spectrum and the RGB image is also saved. In this way, after obtaining the first spectral data information corresponding to the food in the refrigerator, the corresponding RGB image can be inverted through the response function. In this way, after identifying the packaging material corresponding to the food according to the RGB image, the corresponding second spectral data information is obtained, and the second spectral data information is stripped from the first spectral data information. After obtaining the third spectral data information, the third spectral data information is identified through the AI ​​model of food freshness, and the freshness information of the food can be obtained. Therefore, the intelligent monitoring of the freshness of the food in the refrigerator is realized, the probability of food waste is reduced, and the user's dietary health is further guaranteed, and the intelligence and user experience of the refrigerator are improved. In addition, through the AI ​​model of food freshness, the shelf life of the food can also be calculated, and the corresponding refrigerator compartment temperature control is performed, which further reduces the probability of food waste and further improves the intelligence of the refrigerator.

[0104] According to the above process for identifying the freshness of refrigerator food, a device for identifying the freshness of refrigerator food can be constructed.

[0105] Figure 5 Schematic diagram of a device for identifying the freshness of food in a refrigerator provided by an embodiment of the present disclosure. Figure 5 As shown, the device 500 for identifying the freshness of refrigerator food includes: an acquisition inversion module 510 , a determination stripping module 520 , and an identification module 530 .

[0106] The acquisition and inversion module 510 is configured to acquire the first spectrum data information of the current food in the refrigerator and invert the corresponding current red, yellow and blue (RGB) image.

[0107] The determination stripping module 520 is configured to determine the second spectral data information corresponding to the packaging material in the current RGB image, and strip the second spectral data information from the first spectral data information to obtain third spectral data information.

[0108] The identification module 530 is configured to identify the current freshness of the current food according to the third spectral data information through the food freshness artificial intelligence AI model.

[0109] In some embodiments, it also includes: a training and saving module, which is configured to obtain food spectral data sample information and its corresponding freshness sample information of food samples; obtain packaging material spectral data sample information of packaging material samples; and train and learn through AI intelligent algorithms based on the packaging material spectral data sample information, food spectral data sample information and its corresponding freshness sample information to obtain and save an artificial intelligence AI model for food freshness.

[0110] In some embodiments, the acquisition inversion module 510 includes: an inversion unit, configured to normalize the first spectral data information to obtain normalized data information; through a response function, mapping the normalized data information to the red, yellow, and blue (RGB) channels, and performing image processing to obtain the corresponding current RGB image, wherein the response function is constructed based on the spectral sample data corresponding to different wavelengths and the corresponding RGB color response data.

[0111] In some embodiments, the stripping module 520 includes: a determination unit configured to perform image recognition based on the current RGB image to determine the current packaging material corresponding to the current food; and determine the second spectral data information corresponding to the current packaging material based on the packaging material spectral data sample information.

[0112] In some embodiments, the determination stripping module 520 includes: a stripping unit configured to determine a peak band where a spectral emissivity peak appears in the second spectral data information; and deduct the spectral data information corresponding to the peak band in the first spectral data information to obtain third spectral data information.

[0113] In some embodiments, it also includes:

[0114] The reminder module is configured to control the refrigerator to perform user reminder processing according to the current freshness.

[0115] The control module is configured to control the refrigerator to lower the temperature of the compartment where the current food is located when the current freshness is at a corresponding level of edible food.

[0116] The following further describes the refrigerator food freshness identification process used in the refrigerator food freshness identification device in conjunction with the embodiments.

[0117] In this embodiment, the device for identifying the freshness of refrigerator food can be applied to a cloud server.

[0118] Figure 6 Schematic diagram of a device for identifying the freshness of food in a refrigerator provided by an embodiment of the present disclosure. Figure 6As shown, the device 500 for identifying the freshness of refrigerator ingredients includes: an acquisition inversion module 510, a determination stripping module 520, an identification module 530, a training and preservation module 540, a reminder module 550 and a control module 560, wherein the acquisition inversion module 510 includes an inversion unit 511, and the determination stripping module 520 includes: a determination unit 521 and a stripping unit 522.

[0119] The training and saving module 540 has trained and learned multiple food spectral data sample information and their corresponding freshness sample information through the support vector machine (SVM) machine learning algorithm, obtained and saved the food freshness AI model, and also constructed and saved the response function between the spectrum and the RGB image.

[0120] In this way, the inversion module 510 obtains the first spectral data information of the current food sent by the refrigerator, so that the inversion unit 511 in the inversion module 510 normalizes the first spectral data information to obtain normalized data information, and maps the normalized data information to the red, yellow, and blue RGB channels through the response function, and performs image processing to obtain the corresponding current RGB image.

[0121] In this way, the determination unit 521 in the determination stripping module 520 performs image recognition according to the current RGB image to identify the corresponding current packaging material, and determines the second spectral data information corresponding to the current packaging material according to the packaging material spectral data sample information of the stored packaging material sample.

[0122] Among them, when the second spectral data information is blank information, that is, the current food is not packaged, the stripping unit 522 can directly determine the first spectral data information as the third spectral data information. Of course, when the second spectral data information is not blank information, the stripping unit 522 determines the peak band where the spectral emissivity peak appears in the second spectral data information; deducts the spectral data information corresponding to the peak band in the first spectral data information to obtain the third spectral data information.

[0123] Thus, the recognition module 530 can input the third spectral data information into the food freshness AI model for training and learning, obtain the current freshness of the current food, and estimate the shelf life of the food, carrying the shelf life information in the current freshness.

[0124] When the current freshness is at the level corresponding to the edible food, the reminder module 550 sends a first voice command to the refrigerator to control the refrigerator to make a voice announcement of the freshness of the food. When the current freshness is at the level corresponding to the edible food, the reminder module 550 sends a second voice command to the refrigerator to control the refrigerator to make a voice announcement of the edible food and the shelf life information, and the control module 560 determines the target temperature of the compartment where the current food is located in the refrigerator according to the shelf life information included in the current freshness, and controls the refrigerator to lower the temperature of the compartment where the current food is located to the target temperature. When the current freshness is at the level corresponding to the expired food, the reminder module 550 sends a third voice command to the refrigerator to control the refrigerator to make a voice announcement of the expired food, and sends the expired food information to the terminal.

[0125] It can be seen that in this embodiment, the device for identifying the freshness of refrigerator ingredients can train and save the AI ​​model of the freshness of ingredients, and also save the response function between the spectrum and the RGB image. In this way, after obtaining the first spectral data information corresponding to the ingredients in the refrigerator, the corresponding RGB image can be inverted through the response function. In this way, after identifying the packaging material corresponding to the ingredients according to the RGB image, the corresponding second spectral data information is obtained, and the second spectral data information is stripped from the first spectral data information. After obtaining the third spectral data information, the third spectral data information is identified through the AI ​​model of the freshness of ingredients, and the freshness information of the ingredients can be obtained. Therefore, the intelligent monitoring of the freshness of the ingredients in the refrigerator is realized, the probability of food waste is reduced, and the dietary health of users is further guaranteed, and the intelligence and user experience of the refrigerator are improved. In addition, through the AI ​​model of the freshness of ingredients, the shelf life of the ingredients can also be calculated, and the corresponding refrigerator compartment temperature control can be performed, which further reduces the probability of food waste and further improves the intelligence of the refrigerator.

[0126] Combination Figure 7 The embodiment of the present disclosure provides a device 700 for identifying the freshness of refrigerator food, comprising:

[0127] The processor 1000 and the memory 1001 may also include a communication interface 1002 and a bus 1003. The processor 1000, the communication interface 1002, and the memory 1001 may communicate with each other through the bus 1003. The communication interface 1002 may be used for information transmission. The processor 1000 may call the logic instructions in the memory 1001 to execute the method for identifying the freshness of refrigerator food in the above embodiment.

[0128] In addition, the logic instructions in the above-mentioned memory 1001 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

[0129] The memory 1001 is a computer-readable storage medium that can be used to store software programs and computer executable programs, such as program instructions / modules corresponding to the method in the embodiment of the present disclosure. The processor 1000 executes the function application and data processing by running the program instructions / modules stored in the memory 1001, that is, the method for identifying the freshness of refrigerator ingredients in the above method embodiment is implemented.

[0130] The memory 1001 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function; the data storage area may store data created according to the use of the terminal device, etc. In addition, the memory 1001 may include a high-speed random access memory and may also include a non-volatile memory.

[0131] An embodiment of the present disclosure provides a device for identifying the freshness of refrigerator ingredients, comprising: a processor and a memory storing program instructions, wherein the processor is configured to execute a method for identifying the freshness of refrigerator ingredients when executing the program instructions.

[0132] Combination Figure 8 , the embodiment of the present disclosure provides a device 800, which can be a cloud server or a refrigerator, including: a device body, and the above-mentioned device 500 (700) for identifying the freshness of refrigerator ingredients. The device 500 (700) for identifying the freshness of refrigerator ingredients is installed on the device body. The installation relationship described here is not limited to placement inside the product, but also includes installation connections with other components of the product, including but not limited to physical connections, electrical connections, or signal transmission connections. It can be understood by those skilled in the art that the device 500 (700) for identifying the freshness of refrigerator ingredients can be adapted to a feasible device body, thereby realizing other feasible embodiments.

[0133] An embodiment of the present disclosure provides a storage medium storing program instructions, which, when run, execute the above-mentioned method for identifying the freshness of refrigerator ingredients.

[0134] An embodiment of the present disclosure provides a computer program product, which includes a computer program stored on a storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer executes the above-mentioned method for identifying the freshness of refrigerator ingredients.

[0135] The above-mentioned storage medium may be a transient computer-readable storage medium or a non-transitory computer-readable storage medium.

[0136] The technical solution of the embodiment of the present disclosure can be embodied in the form of a software product, which is stored in a storage medium and includes one or more instructions for enabling a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in the embodiment of the present disclosure. The aforementioned storage medium may be a non-transient storage medium, including: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes, or a transient storage medium.

[0137] The above description and the accompanying drawings fully illustrate the embodiments of the present disclosure so that those skilled in the art can practice them. Other embodiments may include structural, logical, electrical, process and other changes. The embodiments represent only possible changes. Unless explicitly required, individual components and functions are optional, and the order of operation may vary. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. The scope of the embodiments of the present disclosure includes the entire scope of the claims, and all available equivalents of the claims. When used in this application, although the terms "first", "second", etc. may be used in this application to describe each element, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, without changing the meaning of the description, the first element can be called the second element, and similarly, the second element can be called the first element, as long as all occurrences of the "first element" are renamed consistently and all occurrences of the "second element" are renamed consistently. The first element and the second element are both elements, but may not be the same element. Moreover, the words used in this application are only used to describe the embodiments and are not used to limit the claims. As used in the description of the embodiments and claims, unless the context clearly indicates, the singular forms "a", "an" and "the" are intended to include the plural forms as well. Similarly, the term "and / or" as used in this application refers to any and all possible combinations of one or more associated listings. In addition, when used in this application, the term "comprise" and its variants "comprises" and / or comprising refer to the presence of stated features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups of these. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the presence of other identical elements in the process, method or device including the elements. In this article, each embodiment may focus on the differences from other embodiments, and the same and similar parts between the embodiments may refer to each other. For the methods, products, etc. disclosed in the embodiments, if they correspond to the method part disclosed in the embodiments, then the relevant parts can refer to the description of the method part.

[0138] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software may depend on the specific application and design constraints of the technical solution. The technicians may use different methods for each specific application to implement the described functions, but such implementations should not be considered to exceed the scope of the embodiments of the present disclosure. The technicians may clearly understand that, for the convenience and simplicity of description, the specific working processes of the systems, devices and units described above may refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.

[0139] In the embodiments disclosed herein, the disclosed methods and products (including but not limited to devices, equipment, etc.) can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units can be only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to implement this embodiment. In addition, each functional unit in the embodiment of the present disclosure may be integrated in a processing unit, or each unit may exist physically alone, or two or more units may be integrated in one unit.

[0140] The flowchart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to the embodiment of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. In the description corresponding to the flowchart and the block diagram in the accompanying drawings, the operations or steps corresponding to different boxes can also occur in a different order from the order disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. Each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified functions or actions, or may be implemented by a combination of dedicated hardware and computer instructions.

Claims

1. A method for identifying the freshness of refrigerator food, characterized in that: include: Obtain the first spectrum data information of the current food in the refrigerator, and invert the corresponding current red, yellow, and blue RGB image; Determine second spectral data information corresponding to the packaging material in the current RGB image, and strip the second spectral data information from the first spectral data information to obtain third spectral data information; According to the third spectrum data information, the current freshness of the current ingredients is identified through the ingredient freshness artificial intelligence AI model.

2. The method according to claim 1, characterized in that Also includes: Obtaining food spectrum data sample information and corresponding freshness sample information of the food sample; Obtaining packaging material spectrum data sample information of packaging material samples; According to the packaging material spectral data sample information, food material spectral data sample information and their corresponding freshness sample information, AI intelligent algorithm is used for training and learning to obtain and save an artificial intelligence AI model for food material freshness.

3. The method according to claim 1, characterized in that The inversion corresponding to the current red, yellow and blue RGB image includes: Normalizing the first spectrum data information to obtain normalized data information; Through the response function, the normalized data information is mapped to the red, yellow and blue RGB channels, and image processing is performed to obtain the corresponding current RGB image, wherein the response function is constructed based on the spectral sample data corresponding to different wavelengths and the corresponding RGB color response data.

4. The method according to claim 1, characterized in that: The determining of the second spectrum data information corresponding to the packaging material in the current RGB image comprises: Perform image recognition based on the current RGB image to determine the current packaging material corresponding to the current food; According to the packaging material spectrum data sample information, second spectrum data information corresponding to the current packaging material is determined.

5. The method according to claim 1, characterized in that The obtaining of the third spectrum data information includes: Determine a peak band in which a peak of the spectral emissivity in the second spectral data information appears; The spectrum data information corresponding to the peak band in the first spectrum data information is deducted to obtain the third spectrum data information.

6. The method according to any one of claims 1 to 5, characterized in that: After identifying the current freshness of the current food, the method further includes: According to the current freshness, control the refrigerator to remind the user; When the current freshness is at a level corresponding to the edible level of the food, the refrigerator is controlled to lower the temperature of the compartment where the current food is located.

7. A device for identifying the freshness of refrigerator food, characterized in that: The device includes: An acquisition and inversion module is configured to acquire first spectral data information of current food in the refrigerator and invert the corresponding current red, yellow and blue (RGB) image; a determination stripping module, configured to determine second spectral data information corresponding to the packaging material in the current RGB image, and strip the second spectral data information from the first spectral data information to obtain third spectral data information; The recognition module is configured to identify the current freshness of the current food according to the third spectral data information through the food freshness artificial intelligence AI model.

8. A device for identifying the freshness of refrigerator food, the device comprising a processor and a memory storing program instructions, characterized in that: The processor is configured to execute the method for identifying the freshness of refrigerator ingredients as claimed in any one of claims 1 to 6 when executing the program instructions.

9. A device, characterized in that: include: Equipment body; The device for identifying the freshness of refrigerator food as described in claim 7 or 8 is installed on the device body.

10. A storage medium storing program instructions, characterized in that: When the program instructions are executed, the method for identifying the freshness of refrigerator ingredients as described in any one of claims 1 to 6 is executed.

Images