Method and device for determining spoiled food, storage medium and electronic device

Abstract

The present invention discloses a method and device for determining spoiled food, a storage medium, and an electronic device. The method comprises: obtaining a target image obtained by photographing food stored in a refrigerator using a target image acquisition device, and obtaining target gas detection information obtained by detecting gas in the refrigerator using a target gas sensor, wherein the target image acquisition device and the target gas sensor are located inside the refrigerator; performing image recognition on the target image to obtain an image recognition result, and performing gas recognition on the target gas detection information to obtain a gas recognition result; and determining whether spoiled food exists in the food stored in the refrigerator based on the image recognition result and the gas recognition result. The above technical solution solves the problem of low accuracy in determining whether food stored in a refrigerator is spoiled using an image acquisition device or a gas sensor alone.

Description

Technical Field

[0001] The present invention relates to the field of communications, and in particular to a method and device for determining spoiled food, a storage medium, and an electronic device. Background Art

[0002] Currently, refrigerators on the market have powerful preservation technology to delay the spoilage of food. However, putting food in the refrigerator does not mean that the food will not spoil permanently. Refrigerators usually have a lot of food and they are piled up in a messy way. If they are stored for a long time and forgotten to be handled, it will be difficult to detect when they spoil. Therefore, a technical solution is needed to detect the spoilage of food in the refrigerator in time.

[0003] In existing solutions, an image acquisition device is usually used alone to capture images of food in the refrigerator to determine whether the food has spoiled, or a gas sensor is used alone to determine whether there is gas from spoiled food in the refrigerator to determine whether the food in the refrigerator has spoiled.

[0004] However, if detection is performed solely through images or gas, inaccurate identification may occur. For example, after a banana turns black, the image may indicate that the refrigerator is spoiled, but in fact the banana is not spoiled. If there is vinegar in the refrigerator and the gas sensor detects the presence of acetic acid, the gas sensor alone may determine that there is spoiled food in the refrigerator.

[0005] Regarding the related art, there is currently no effective solution to the problem of low accuracy in determining whether food stored in a refrigerator has spoiled using image acquisition devices or gas sensors alone.

[0006] Therefore, it is necessary to improve the related technology to overcome the above-mentioned defects in the related technology. Summary of the Invention

[0007] Embodiments of the present invention provide a method and device for determining spoiled food, a storage medium, and an electronic device to at least solve the problem of low accuracy in determining whether food stored in a refrigerator has spoiled using an image acquisition device or a gas sensor alone.

[0008] According to one aspect of an embodiment of the present invention, a method for determining spoiled food is provided, comprising: obtaining a target image obtained by photographing food stored in a refrigerator using a target image acquisition device, and obtaining target gas detection information obtained by detecting gas in the refrigerator using a target gas sensor, wherein the target image acquisition device and the target gas sensor are located inside the refrigerator; performing image recognition on the target image to obtain an image recognition result, and performing gas recognition on the target gas detection information to obtain a gas recognition result; and determining whether there is spoiled food in the food stored in the refrigerator based on the image recognition result and the gas recognition result.

[0009] According to another aspect of an embodiment of the present invention, a device for determining spoiled food is provided, including: an acquisition module for acquiring a target image obtained by a target image acquisition device shooting food stored in a refrigerator, and acquiring target gas detection information obtained by a target gas sensor detecting gas in the refrigerator, wherein the target image acquisition device and the target gas sensor are located inside the refrigerator; an identification module for performing image recognition on the target image to obtain an image recognition result, and performing gas recognition on the target gas detection information to obtain a gas recognition result; and a determination module for determining whether there is spoiled food in the food stored in the refrigerator based on the image recognition result and the gas recognition result.

[0010] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored. The computer program is configured to execute the above-mentioned method for determining spoiled food when running.

[0011] According to another aspect of an embodiment of the present invention, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the method for determining spoiled food through the computer program.

[0012] The present invention obtains a target image obtained by capturing food stored in a refrigerator using a target image acquisition device, and obtains target gas detection information obtained by detecting gas in the refrigerator using a target gas sensor. The target image is then subjected to image recognition to obtain an image recognition result, and gas recognition is performed on the target gas detection information to obtain a gas recognition result. Finally, based on the image recognition result and the gas recognition result, the presence of spoiled food in the food stored in the refrigerator is determined. In other words, the above technical solution improves recognition accuracy by jointly determining the presence of spoiled food in the refrigerator using both the image acquisition device and the gas sensor, thereby resolving the issue of low accuracy in determining whether food stored in the refrigerator is spoiled using either the image acquisition device or the gas sensor alone. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0014] Figure 1 1 is a hardware structure block diagram of a computer terminal for a method for determining spoiled food according to an embodiment of the present invention;

[0015] Figure 2 is a flow chart (1) of a device for determining spoiled food according to an embodiment of the present invention;

[0016] Figure 3 is a characteristic schematic diagram of spoiled food according to an embodiment of the present invention (I);

[0017] Figure 4 is a characteristic schematic diagram of spoiled food according to an embodiment of the present invention (II);

[0018] Figure 5 is a flow chart (II) of a device for determining spoiled food according to an embodiment of the present invention;

[0019] Figure 6 1 is a structural block diagram of a device for determining spoiled food according to an embodiment of the present invention (I);

[0020] Figure 7 2 is a structural block diagram of a device for determining spoiled food according to an embodiment of the present invention. DETAILED DESCRIPTION

[0021] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0022] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0023] The method embodiments provided in the embodiments of the present application can be executed in a computer terminal or similar computing device. Taking running on a computer terminal as an example, Figure 1 FIG. 1 is a hardware block diagram of a computer terminal for determining spoiled food according to an embodiment of the present invention. Figure 1 As shown, the computer terminal may include one or more ( Figure 1 Only one is shown in the figure) processor 102 (processor 102 may include but is not limited to a microprocessor (Microprocessor Unit, referred to as MPU) or a programmable logic device (Programmable logic device, referred to as PLD)) and a memory 104 for storing data. In an exemplary embodiment, the computer terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above-mentioned computer terminal. For example, the computer terminal may also include Figure 1 More or fewer components than shown, or with Figure 1 Equivalent functions or comparisons shown Figure 1 Shown are different configurations with more functionality.

[0024] Memory 104 can be used to store computer programs, such as software programs and modules of application software, such as the computer program corresponding to the method for determining spoiled food in the embodiment of the present invention. Processor 102 executes the computer program stored in memory 104 to execute various functional applications and data processing, thereby implementing the above-mentioned method. Memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, memory 104 may further include memory remotely located relative to processor 102, and these remote memories may be connected to the computer terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0025] The transmission device 106 is used to receive or send data via a network. A specific example of the aforementioned network may include a wireless network provided by a computer terminal's communications provider. In one embodiment, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0026] In order to solve the above problems, a method for determining spoiled food is provided in this embodiment. Figure 2 FIG. 1 is a flow chart of a method for determining spoiled food according to an embodiment of the present invention (I), the flow chart comprising the following steps:

[0027] Step S202: acquiring a target image obtained by photographing food stored in a refrigerator using a target image acquisition device, and acquiring target gas detection information obtained by detecting gas in the refrigerator using a target gas sensor, wherein the target image acquisition device and the target gas sensor are located inside the refrigerator;

[0028] In an exemplary embodiment, the target image acquisition device is a camera, and the target gas sensor includes but is not limited to an ethanol sensor and an acetic acid sensor.

[0029] It should be noted that there are one or more target image acquisition devices and one or more target gas sensors in the refrigerator.

[0030] Step S204, performing image recognition on the target image to obtain an image recognition result, and performing gas recognition on the target gas detection information to obtain a gas recognition result;

[0031] Step S206: Determine whether there is spoiled food in the food stored in the refrigerator based on the image recognition result and the gas recognition result.

[0032] Through the above steps, a target image is obtained by capturing food stored in the refrigerator using a target image acquisition device, and target gas detection information is obtained by detecting gas in the refrigerator using a target gas sensor. Image recognition is then performed on the target image to obtain an image recognition result, and gas recognition is performed on the target gas detection information to obtain a gas recognition result. Finally, based on the image recognition result and the gas recognition result, a determination is made as to whether spoiled food is present in the food stored in the refrigerator. In other words, the above technical solution improves recognition accuracy by jointly determining whether spoiled food is present in the refrigerator using both the image acquisition device and the gas sensor, thereby resolving the issue of low accuracy in determining whether spoiled food is present in the refrigerator using either the image acquisition device or the gas sensor alone.

[0033] In an exemplary embodiment, performing image recognition on the target image to obtain an image recognition result can be achieved by the following steps, specifically:

[0034] Step S1: when the target image includes a first set of images obtained by photographing the first food at multiple moments, acquiring feature data of a set of food features for each image in the first set of images, to obtain multiple sets of feature data of the set of food features;

[0035] It should be noted that the first set of images includes images obtained by the target image acquisition device taking multiple shots of the first food after the first food is placed in the refrigerator for the first time, and the set of food features is used to determine whether the first food has deteriorated;

[0036] It should be noted that, assuming the first food is an apple, after the apple is placed in the refrigerator, the refrigerator will photograph the apple at preset time intervals, thereby obtaining a first set of images corresponding to the apple. For example, if the apple is photographed ten times, the first set of images will include images from ten different time periods. A set of food characteristics includes, but is not limited to, color, external contour, surface smoothness, and the presence of attachments. Therefore, each image in the first set of images has a set of feature data representing a set of food characteristics. In other words, the first set of images has multiple sets of feature data representing a set of food characteristics.

[0037] Figure 3 Schematic diagram of spoiled food according to an embodiment of the present invention (I). Figure 3The first set of images includes N images, and a set of food features includes feature X, feature Y, feature Z, ..., feature N. Each image has data for each feature in the set of food features. Specifically, feature X can be color, feature Y can be outline, and so on.

[0038] Step S2: Determine whether the first food has deteriorated based on multiple sets of feature data of the set of food features and preset reference feature data of the set of food features, wherein the reference feature data of the set of food features is the feature data of the set of food features when food of the same type as the first food has deteriorated.

[0039] It should be noted that, assuming a set of food features includes feature X, feature Y, feature Z ... feature N, the corresponding reference feature data is X0, Y0, Z0, ... N0. That is, when the first food deteriorates, it has the above reference feature data.

[0040] It should be noted that the above step S2 has multiple implementations. In an exemplary embodiment, it can be implemented in the following ways:

[0041] Step 21: When the first group of images includes M1 images, the group of food features includes N1 food features, and the multiple groups of feature data of the group of food features include M1 feature data of each of the N1 food features, determine whether there is a food feature with abnormal feature data among the N1 food features based on the M1 feature data of each of the N1 food features, where N1 is a positive integer greater than or equal to 2, and M1 is a positive integer greater than or equal to 2.

[0042] In an exemplary embodiment, according to the chronological order of shooting time of the M1 images, the difference between the last P1 feature data in the M1 feature data of the food feature with abnormal feature data and the reference feature data of the food feature with abnormal feature data is less than or equal to a first preset threshold, and P1 is equal to 1 or a positive integer greater than or equal to 2.

[0043] That is, after the first food is put into the refrigerator, the target image acquisition device will take multiple pictures of the first food according to the preset time interval, thereby obtaining a first set of images, and then analyzing the set of images to obtain multiple sets of feature data of a set of food features, such as Figure 3 As shown, for a feature X in a group of food features, as long as the difference between the last P1 feature data and the reference feature data of feature X is less than or equal to the first preset threshold, the food feature X can be determined to be a food feature with abnormal feature data.

[0044] In an exemplary embodiment, in the order of shooting time of the M1 images, the difference between the last P1 feature data in the M1 feature data of the food feature with abnormal feature data and the reference feature data of the food feature with abnormal feature data is less than or equal to a first preset threshold, and the difference between the P1 feature data and the reference feature data of the food feature with abnormal feature data becomes smaller and smaller in the order of shooting time, and P1 is equal to 1 or a positive integer greater than or equal to 2.

[0045] like Figure 3 As shown, for a feature X in a set of food features, assuming P1 is 3, the difference between the three feature data of feature X in the last three images and the reference feature data of feature X must be less than or equal to a first preset threshold, and the difference between the third-to-last feature data Xn-2 and the reference feature data must be less than the difference between the second-to-last feature data Xn-1 and the reference feature data. If the difference between the second-to-last feature data Xn-1 and the reference feature data is less than the difference between the first-to-last feature data Xn and the reference feature data, then the food feature X is determined to have abnormal feature data.

[0046] It should be noted that each time an image of the first food is captured, a judgment is made as to whether the food has deteriorated. However, if P1 is greater than M1, no judgment is made at this time. The process waits until there are more than or equal to P1 images in the first group of images before judging whether the food has deteriorated.

[0047] In an exemplary embodiment, according to the chronological order of shooting time of the M1 images, the similarity between the first characteristic curve fitted according to time of the M1 characteristic data of the food characteristics with abnormal characteristic data and the first target characteristic curve fitted according to time of the characteristic data of the food characteristics with abnormal characteristic data when the first food is not spoiled is less than a preset threshold.

[0048] For example, if the first food has food feature X, the feature data of M1 images about food feature X can be fitted into a feature curve in chronological order, and then compared with the target feature curve of feature X when the first food is not spoiled. If the similarity is less than a preset threshold, it means that the first food has spoiled.

[0049] Step 22: When there are Q1 food features with abnormal feature data among the N1 food features, it is determined that the first food has deteriorated, where Q1 is equal to 1 or a positive integer greater than or equal to 2.

[0050] In other words, if Q1 equals 1, then as long as any one of the N1 food characteristics has abnormal data, it can be determined that the first food has spoiled. For example, suppose the first food is A, and the food characteristics are: X, Y, Z. If the data for feature X, Y, or Z is abnormal, it means that food A has spoiled.

[0051] If Q1 equals 2, then two of the N1 food characteristics must show abnormal data for the first food to be considered spoiled. For example, let's assume the first food is B, and the food characteristics are still: X, Y, and Z. As long as two of these three food characteristics show abnormal data, food B is considered spoiled.

[0052] In an exemplary embodiment, performing gas identification on the target gas detection information to obtain a gas identification result can be achieved by the following steps, specifically:

[0053] Step S1: When the target gas detection information includes a first set of gas detection information obtained by performing gas detection on the first food at multiple times, acquiring a set of characteristic data of gas characteristics from each gas detection information in the first set of gas detection information, thereby obtaining multiple sets of characteristic data of the set of gas characteristics;

[0054] It should be noted that the first set of gas detection information includes gas detection information obtained by the target gas sensor performing multiple gas detections on the first food after the first food is placed in the refrigerator for the first time, and the set of gas characteristics is used to determine whether the first food has deteriorated;

[0055] It should be noted that, assuming the first food is an apple, after the apple is placed in the refrigerator, the refrigerator will perform gas detection on the apple at preset time intervals, thereby obtaining a first set of gas detection information corresponding to the apple. For example, if the apple is gas-tested ten times, the first set of gas detection information will include gas detection information from ten different time periods. A set of gas characteristics includes, but is not limited to, ethanol, acetic acid, etc. Furthermore, for each gas detection information in the first set of gas detection information, a set of characteristic data for the gas characteristic is generated. In other words, the first set of gas detection information contains multiple sets of characteristic data for a set of gas characteristics.

[0056] Figure 4 Schematic diagram (2) of the characteristics of spoiled food according to an embodiment of the present invention, such as Figure 4 The first set of gas detection information includes N gas detection information, and a set of gas characteristics includes gas X, gas Y, gas Z, and gas N. Each gas detection information has data for each characteristic in the corresponding set of gas characteristics. Specifically, gas X can be ethanol, gas Y can be acetic acid, and so on.

[0057] Step S2: Determine whether the first food has deteriorated based on multiple sets of characteristic data of the set of gas characteristics and preset reference characteristic data of the set of gas characteristics, wherein the reference characteristic data of the set of gas characteristics is the characteristic data of the set of gas characteristics when food of the same type as the first food has deteriorated.

[0058] It should be noted that the characteristic data of the gas characteristics are specifically expressed in terms of the content of the gas.

[0059] There are many implementations for the above step S2. In an exemplary embodiment, it can be implemented by the following steps:

[0060] Step S21: When the first group of gas detection information includes M2 gas detection information, the group of gas characteristics includes N2 gas characteristics, and the multiple groups of characteristic data of the group of gas characteristics include M2 ​​characteristic data of each gas characteristic in the N2 gas characteristics, determine whether there is a gas characteristic with abnormal characteristic data in the N2 gas characteristics based on the M2 characteristic data of each gas characteristic in the N2 gas characteristics, wherein N2 is a positive integer greater than or equal to 2, and M2 is a positive integer greater than or equal to 2.

[0061] In an example embodiment, according to the order of detection time of the M2 gas detection information, the difference between the last P2 characteristic data in the M2 characteristic data of the gas characteristic with abnormal characteristic data and the reference characteristic data of the gas characteristic with abnormal characteristic data is less than or equal to a second preset threshold, and P2 is equal to 1 or a positive integer greater than or equal to 2.

[0062] Assume that for a feature X in a set of gas features, as long as the difference between the last P2 feature data and the reference feature data of gas X is less than or equal to the first preset threshold, it can be determined that the food feature X is a food feature with abnormal feature data.

[0063] In an exemplary embodiment, in the order of detection time of the M2 gas detection information, the difference between the last P2 feature data among the M2 feature data of the gas characteristic with abnormal feature data and the reference feature data of the gas characteristic with abnormal feature data is less than or equal to a second preset threshold, and the difference between the P2 feature data and the reference feature data of the gas characteristic with abnormal feature data decreases in the order of detection time. P2 is equal to 1 or a positive integer greater than or equal to 2.

[0064] It should be noted that each time a gas detection is performed on the first food, a judgment will be made as to whether it has deteriorated. However, if P2 is greater than M2, no judgment will be made at this time. The process of waiting for the first set of gas detection information to include P2 gas detection information is used to determine whether the food has deteriorated.

[0065] In an exemplary embodiment, according to the order of detection time of the M2 gas detection information, the similarity between the second characteristic curve fitted according to time by the M2 characteristic data of the gas characteristics with abnormal characteristic data and the second target characteristic curve fitted according to time by the characteristic data of the gas characteristics with abnormal characteristic data when the first food is not spoiled is less than a preset threshold.

[0066] Step S22: When there are Q2 gas features with abnormal feature data among the N2 gas features, it is determined that the first food has deteriorated, where Q2 is equal to 1 or a positive integer greater than or equal to 2.

[0067] It should be noted that the gas detection data of the first food can be obtained by detecting a target gas sensor closest to the specific first food.

[0068] In an exemplary embodiment, the above-mentioned step S206 determines whether there is spoiled food in the food stored in the refrigerator based on the image recognition result and the gas recognition result, which can be implemented in the following manner: when the image recognition result indicates that the first food stored in the refrigerator has spoiled, and the gas recognition result indicates that the first food stored in the refrigerator has spoiled, it is determined that the first food stored in the refrigerator is spoiled food.

[0069] In an exemplary embodiment, when the image recognition result indicates that the first food stored in the refrigerator has deteriorated, or the gas recognition result indicates that the first food stored in the refrigerator has deteriorated, it can also be determined that the first food stored in the refrigerator is deteriorated food.

[0070] In other words, food must be identified as spoiled only if both the image recognition and gas recognition results indicate it is spoiled. If the user believes that any change in the food inside the refrigerator indicates spoilage, the food can be identified as spoiled only if either the image recognition result or the gas recognition result indicates it is spoiled. It should be noted that the specific method used can be determined based on the type of food, thereby improving recognition accuracy.

[0071] In an exemplary embodiment, different weights may be assigned to the image recognition result and the gas recognition result, and whether the food is spoiled may be calculated based on the weights. It should be noted that the weights may also be adjusted based on user feedback on whether the recognition is correct.

[0072] In an exemplary embodiment, when it is determined that the first food stored in the refrigerator is spoiled food, first spoilage prompt information is issued, wherein the first spoilage prompt information is used to prompt that there is spoiled food in the refrigerator.

[0073] It should be noted that the deterioration warning information can be sent to the user's mobile terminal or announced in the form of voice. Mobile terminals include but are not limited to: mobile phones and tablet computers.

[0074] In an exemplary embodiment, when it is determined that the first food stored in the refrigerator is spoiled food, the first food is photographed by the target image acquisition device to obtain a first image, and second spoilage prompt information is sent to the target device, wherein the second spoilage prompt information includes the first image, and the second spoilage prompt information is used to prompt the presence of spoiled food in the refrigerator.

[0075] That is, when the first food is determined to be spoiled food, a photo of the first food can also be sent to the user's mobile terminal to remind the user. It should be noted that the user can also determine whether the identification is accurate on the mobile terminal and feedback to the refrigerator.

[0076] In an exemplary embodiment, when it is determined that the first food stored in the refrigerator is spoiled food and it is detected that the refrigerator is opened, a warning light adjacent to the first food is turned on, wherein the warning light is used to indicate the presence of spoiled food in the refrigerator.

[0077] Obviously, the above-described embodiments are only part of the embodiments of the present invention, not all of them. To better understand the above-mentioned method for determining spoiled food, the above-mentioned process is described below with reference to the embodiments, but it is not intended to limit the technical solutions of the embodiments of the present invention. Specifically:

[0078] In an optional embodiment, Figure 5 This is a flow chart (II) of the apparatus for determining spoiled food according to an embodiment of the present invention, specifically:

[0079] Image sensor (equivalent to the target image acquisition device in the above embodiment):

[0080] (1) Input the appearance feature data of spoiled food (equivalent to the reference feature data of food features in the above embodiment), such as hairy, blackened, wrinkled, etc.

[0081] (2) Photographing food and extracting features. After fresh food is placed in the refrigerator, the refrigerator's built-in camera captures the image information of the fresh food and extracts the main features of the captured food information. At least one main feature is extracted. If the first image taken after the food is placed in the refrigerator directly has features corresponding to the database, such as color, external contour, whether the surface is smooth or has attachments, etc., it can be directly judged as spoiled.

[0082] (3) Intermittent shooting and data comparison. If the first shot after adding the food does not show any signs of spoilage, an image can be taken every 2 hours (equivalent to the preset time interval in the above embodiment) to detect changes in at least one major feature of the food. The image data is then compared with the appearance image data of spoiled food recorded in the database. If there is a consistent match, the food is considered spoiled.

[0083] Odor sensor (equivalent to the gas sensor in the above embodiment):

[0084] (1) Input the characteristic data of the odor of spoiled food, such as sour, smelly, moldy and other common spoilage odor data.

[0085] (2) Identify odors and extract features. The first odor identification is performed on the food just put in. If there is spoiled odor data after comparison, it can be directly judged as spoiled.

[0086] (3) Intermittent sensing and data comparison. After the first determination that there is no spoilage, odor monitoring is performed every 2 hours. If the odor detected later is consistent with the odor in the database, it can be determined that the food has spoiled by analyzing and comparing the gas data information.

[0087] It should be noted that, depending on the characteristics of the ingredients, some foods can be considered spoiled if either their appearance or smell changes. Other foods may require both changes to be considered spoiled. Multiple internal odor sensors can be installed. By comparing the data from different odor sensors and combining it with relevant image data, the location of spoiled ingredients can be determined. Once the location is determined, if the user opens the refrigerator door, an infrared device can directly project an infrared beam onto the spoiled food, alerting the user.

[0088] Furthermore, the technical solution of the embodiment of the present invention improves the accuracy of identifying spoiled food by combining an image sensor with an odor sensor, avoiding misjudgments that can occur with a single image or odor sensor. This simplifies the steps involved in identifying spoiled food using the image sensor and the food sensor. This fundamentally solves the problem of spoiled food being difficult to detect after prolonged storage, allowing users to open the refrigerator door and directly follow the infrared light to locate the spoiled food, allowing them to directly remove and process it, thus improving the user experience. This also prevents harmful substances and bacteria attached to spoiled food from corroding other fresh ingredients, thus avoiding waste and being more environmentally friendly.

[0089] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of various embodiments of the present invention.

[0090] This embodiment also provides a device for determining spoiled food, which is used to implement the above-mentioned embodiments and preferred embodiments. Details already described will not be repeated here. As used below, the term "module" may refer to a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.

[0091] Figure 6 FIG. 1 is a structural block diagram of a device for determining spoiled food according to an embodiment of the present invention (I), the device comprising:

[0092] an acquisition module 62, configured to acquire a target image obtained by capturing food stored in a refrigerator using a target image acquisition device, and to acquire target gas detection information obtained by detecting gas in the refrigerator using a target gas sensor, wherein the target image acquisition device and the target gas sensor are located inside the refrigerator;

[0093] The recognition module 64 is configured to perform image recognition on the target image to obtain an image recognition result, and perform gas recognition on the target gas detection information to obtain a gas recognition result;

[0094] The determination module 66 is configured to determine whether there is spoiled food in the food stored in the refrigerator based on the image recognition result and the gas recognition result.

[0095] The above-described device obtains a target image captured by a target image acquisition device of food stored in a refrigerator, and obtains target gas detection information detected by a target gas sensor detecting gas in the refrigerator. Image recognition is then performed on the target image to obtain an image recognition result, and gas recognition is performed on the target gas detection information to obtain a gas recognition result. Finally, based on the image recognition result and the gas recognition result, a determination is made as to whether spoiled food is present in the food stored in the refrigerator. In other words, the above-described technical solution improves recognition accuracy by jointly determining whether spoiled food is present in the refrigerator using both the image acquisition device and the gas sensor, thereby resolving the issue of low accuracy in determining whether spoiled food is present in the refrigerator using either the image acquisition device or the gas sensor alone.

[0096] In an exemplary embodiment, the recognition module 64 is further used to obtain feature data of a set of food features for each image in a first group of images when the target image includes a first group of images obtained by photographing the first food at multiple moments, thereby obtaining multiple sets of feature data of the set of food features; and determine whether the first food has deteriorated based on the multiple sets of feature data of the set of food features and preset reference feature data of the set of food features, wherein the reference feature data of the set of food features is the feature data of the set of food features when food of the same type as the first food has deteriorated.

[0097] In an exemplary embodiment, the identification module 64 is further used to, when the first group of images includes M1 images, the group of food features includes N1 food features, and the multiple groups of feature data of the group of food features include M1 feature data of each food feature in the N1 food features, determine whether there is a food feature with abnormal feature data in the N1 food features based on the M1 feature data of each food feature in the N1 food features, wherein N1 is a positive integer greater than or equal to 2, M1 is a positive integer greater than or equal to 2, and in the order of shooting time of the M1 images, the difference between the last P1 feature data in the M1 feature data of the food feature with abnormal feature data and the reference feature data of the food feature with abnormal feature data is less than or equal to a first preset threshold, and P1 is equal to 1 or a positive integer greater than or equal to 2; and when there are Q1 food features with abnormal feature data in the N1 food features, determine that the first food has deteriorated, wherein Q1 is equal to 1 or a positive integer greater than or equal to 2.

[0098] In an exemplary embodiment, the identification module 64 is further used to obtain characteristic data of a set of gas characteristics in each gas detection information in the first set of gas detection information, thereby obtaining multiple sets of characteristic data of the set of gas characteristics, when the target gas detection information includes a first set of gas detection information obtained by performing gas detection on the first food at multiple moments; and determine whether the first food has deteriorated based on the multiple sets of characteristic data of the set of gas characteristics and preset reference characteristic data of the set of gas characteristics, wherein the reference characteristic data of the set of gas characteristics is the characteristic data of the set of gas characteristics when food of the same type as the first food has deteriorated.

[0099] In an exemplary embodiment, the identification module 64 is further configured to, when the first group of gas detection information includes M2 gas detection information, the group of gas features includes N2 gas features, and the multiple sets of feature data of the group of gas features include M2 ​​feature data of each gas feature in the N2 gas features, determine whether there is a gas feature with abnormal feature data in the N2 gas features based on the M2 feature data of each gas feature in the N2 gas features, wherein N2 is a positive integer greater than or equal to 2, M2 is a positive integer greater than or equal to 2, and in order of detection time of the M2 gas detection information, a difference between last P2 feature data of the M2 feature data of the gas feature with abnormal feature data and a reference feature data of the gas feature with abnormal feature data is less than or equal to a second preset threshold, and P2 is equal to 1 or a positive integer greater than or equal to 2; and when Q2 gas features with abnormal feature data exist in the N2 gas features, determine that the first food has deteriorated, wherein Q2 is equal to 1 or a positive integer greater than or equal to 2.

[0100] In an exemplary embodiment, the determination module 66 is further used to determine that the first food stored in the refrigerator is spoiled food when the image recognition result indicates that the first food stored in the refrigerator has spoiled and the gas recognition result indicates that the first food stored in the refrigerator has spoiled.

[0101] Figure 7 2 is a structural block diagram of a device for determining spoiled food according to an embodiment of the present invention. The device includes: a reminder module 68.

[0102] In an exemplary embodiment, the reminder module 68 is used to, when it is determined that the first food stored in the refrigerator is spoiled food, issue a first spoilage prompt information, wherein the first spoilage prompt information is used to prompt that there is spoiled food in the refrigerator; or, when it is determined that the first food stored in the refrigerator is spoiled food, photograph the first food by the target image acquisition device to obtain a first image, and send a second spoilage prompt information to the target device, wherein the second spoilage prompt information includes the first image, and the second spoilage prompt information is used to prompt that there is spoiled food in the refrigerator; or, when it is determined that the first food stored in the refrigerator is spoiled food and it is detected that the refrigerator is opened, turn on a prompt light adjacent to the first food, wherein the prompt light is used to prompt that there is spoiled food in the refrigerator.

[0103] An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored. The computer program is configured to execute the steps of any one of the above method embodiments when run.

[0104] Optionally, in this embodiment, the storage medium may be configured to store a computer program for performing the following steps:

[0105] S1, obtaining a target image obtained by photographing food stored in a refrigerator by a target image acquisition device, and obtaining target gas detection information obtained by detecting gas in the refrigerator by a target gas sensor, wherein the target image acquisition device and the target gas sensor are located inside the refrigerator;

[0106] S2, performing image recognition on the target image to obtain an image recognition result, and performing gas recognition on the target gas detection information to obtain a gas recognition result;

[0107] S3, determining whether there is spoiled food among the food stored in the refrigerator based on the image recognition result and the gas recognition result.

[0108] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0109] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail here.

[0110] An embodiment of the present invention further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0111] Optionally, in this embodiment, the processor may be configured to execute the following steps through a computer program:

[0112] S1, obtaining a target image obtained by photographing food stored in a refrigerator by a target image acquisition device, and obtaining target gas detection information obtained by detecting gas in the refrigerator by a target gas sensor, wherein the target image acquisition device and the target gas sensor are located inside the refrigerator;

[0113] S2, performing image recognition on the target image to obtain an image recognition result, and performing gas recognition on the target gas detection information to obtain a gas recognition result;

[0114] S3, determining whether there is spoiled food among the food stored in the refrigerator based on the image recognition result and the gas recognition result.

[0115] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0116] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail here.

[0117] Obviously, those skilled in the art will appreciate that the various modules or steps of the present invention described above can be implemented using a general-purpose computing device, can be centralized on a single computing device, or can be distributed across a network of multiple computing devices. They can be implemented using program code executable by the computing device, and thus, can be stored in a storage device and executed by the computing device. In some cases, the steps shown or described herein can be performed in a different order than that shown, or can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0118] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A method for determining spoiled food, characterized in that: include: obtaining a target image obtained by photographing food stored in a refrigerator using a target image acquisition device, and obtaining target gas detection information obtained by detecting gas in the refrigerator using a target gas sensor, wherein the target image acquisition device and the target gas sensor are located inside the refrigerator; Performing image recognition on the target image to obtain an image recognition result, and performing gas recognition on the target gas detection information to obtain a gas recognition result; determining whether there is spoiled food in the food stored in the refrigerator based on the image recognition result and the gas recognition result; Wherein, performing image recognition on the target image to obtain an image recognition result includes: when the target image includes a first group of images obtained by photographing the first food at multiple moments, acquiring feature data of a group of food features for each image in the first group of images to obtain multiple sets of feature data of the group of food features; when the first group of images includes M1 images, the group of food features includes N1 food features, and the multiple sets of feature data of the group of food features include M1 feature data of each food feature in the N1 food features, judging the N1 food features according to the M1 feature data of each food feature in the N1 food features. Whether there are food features with abnormal feature data among the object features, wherein N1 is a positive integer greater than or equal to 2, M1 is a positive integer greater than or equal to 2, and according to the chronological order of shooting time of the M1 images, the similarity between a first feature curve fitted according to time by the M1 feature data of the food features with abnormal feature data and a first target feature curve fitted according to time by the feature data of the food features with abnormal feature data when the first food is not spoiled is less than a preset threshold; when there are Q1 food features with abnormal feature data among the N1 food features, it is determined that the first food is spoiled, wherein Q1 is equal to 1 or a positive integer greater than or equal to 2.

2. The method according to claim 1, characterized in that The performing gas identification on the target gas detection information to obtain a gas identification result includes: When the target gas detection information includes a first set of gas detection information obtained by performing gas detection on the first food at multiple times, acquiring feature data of a set of gas characteristics from each gas detection information in the first set of gas detection information to obtain multiple sets of feature data of the set of gas characteristics; Determine whether the first food has deteriorated based on multiple sets of characteristic data of the set of gas characteristics and preset reference characteristic data of the set of gas characteristics, wherein the reference characteristic data of the set of gas characteristics is the characteristic data of the set of gas characteristics when food of the same type as the first food has deteriorated.

3. The method according to claim 2, characterized in that The determining whether the first food has deteriorated based on the multiple sets of characteristic data of the set of gas characteristics and the preset reference characteristic data of the set of gas characteristics includes: In a case where the first group of gas detection information includes M2 gas detection information, the group of gas features includes N2 gas features, and the multiple groups of feature data of the group of gas features include M2 ​​feature data of each gas feature in the N2 gas features, judging whether there is a gas feature with abnormal feature data in the N2 gas features based on the M2 feature data of each gas feature in the N2 gas features, wherein N2 is a positive integer greater than or equal to 2, M2 is a positive integer greater than or equal to 2, and in a chronological order of detection time of the M2 gas detection information, a difference between the last P2 feature data of the M2 feature data of the gas feature with abnormal feature data and the reference feature data of the gas feature with abnormal feature data is less than or equal to a second preset threshold, and P2 is equal to 1 or a positive integer greater than or equal to 2; When there are Q2 gas features with abnormal feature data among the N2 gas features, it is determined that the first food has deteriorated, where Q2 is equal to 1 or a positive integer greater than or equal to 2.

4. The method according to any one of claims 1 to 3, characterized in that The determining, based on the image recognition result and the gas recognition result, whether there is spoiled food in the food stored in the refrigerator includes: When the image recognition result indicates that the first food stored in the refrigerator has deteriorated and the gas recognition result indicates that the first food stored in the refrigerator has deteriorated, it is determined that the first food stored in the refrigerator is deteriorated food.

5. The method according to claim 4, wherein The method further comprises: When it is determined that the first food stored in the refrigerator is spoiled food, issuing first spoilage prompt information, wherein the first spoilage prompt information is used to prompt that spoiled food exists in the refrigerator; or When it is determined that the first food stored in the refrigerator is spoiled food, the target image acquisition device captures the first food to obtain a first image, and sends second spoilage prompt information to the target device, wherein the second spoilage prompt information includes the first image and is used to prompt that spoiled food exists in the refrigerator; or When it is determined that the first food stored in the refrigerator is spoiled food and it is detected that the refrigerator is opened, a warning light adjacent to the first food is turned on, wherein the warning light is used to indicate that spoiled food exists in the refrigerator.

6. A device for determining spoiled food, characterized in that: include: an acquisition module, configured to acquire a target image obtained by photographing food stored in a refrigerator by a target image acquisition device, and to acquire target gas detection information obtained by detecting gas in the refrigerator by a target gas sensor, wherein the target image acquisition device and the target gas sensor are located inside the refrigerator; an identification module, configured to perform image recognition on the target image to obtain an image recognition result, and perform gas recognition on the target gas detection information to obtain a gas recognition result; a determination module, configured to determine whether there is spoiled food in the food stored in the refrigerator based on the image recognition result and the gas recognition result; Among them, the recognition module is further used to obtain feature data of a group of food features for each image in the first group of images when the target image includes a first group of images obtained by photographing the first food at multiple moments, so as to obtain multiple sets of feature data of the group of food features; when the first group of images includes M1 images, the group of food features includes N1 food features, and the multiple sets of feature data of the group of food features include M1 feature data of each food feature in the N1 food features, determine whether there is a feature in the N1 food features based on the M1 feature data of each food feature in the N1 food features. Food features with abnormal data, wherein N1 is a positive integer greater than or equal to 2, M1 is a positive integer greater than or equal to 2, and according to the chronological order of shooting time of the M1 images, the similarity between the first characteristic curve fitted according to time of the M1 feature data of the food features with abnormal feature data and the first target characteristic curve fitted according to time when the feature data of the food features with abnormal feature data is not spoiled is less than a preset threshold; when there are Q1 food features with abnormal feature data among the N1 food features, it is determined that the first food is spoiled, wherein Q1 is equal to 1 or a positive integer greater than or equal to 2.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein the program executes the method described in any one of claims 1 to 5 when executed.

8. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 5 through the computer program.

Images