Food storage system, method and computer-readable medium
By setting machine-readable identifiers and computer vision algorithms on storage bags, the identification and management problems in food storage are solved, cooking and edible reminders are provided, and users' food management efficiency is improved.
Patent Information
- Application Number
- CN202080014753.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2019-01-25
- Filing Date
- 2020-01-24
- Publication Date
- 2025-07-11
- Estimated Expiration
- 2040-01-24
AI Technical Summary
In the prior art, when food is stored in a container or bag, it is difficult to determine or remember the types of food stored in the bag, especially in the frozen state, causing the user to forget the stored food information.
By setting machine-readable identifiers, such as QR codes or RFIDs on the storage bag, combined with computer vision algorithms, identify and track food information in the storage bag, and generate cooking or edible reminders, data processing and notification sending is used to utilize processors and non-transitory processor-readable media.
It realizes accurate identification and management of food in storage bags, provides cooking recommendations and edible reminders, reduces food waste and improves user experience.
Smart Images

Figure CN113490958B_ABST
Abstract
Description
[0001] Related Applications
[0002] This application claims the benefit of priority of U.S. Provisional Patent Application No. 62 / 796,997, filed on Jan. 25, 2019, the entire content of which is incorporated herein by reference. BACKGROUND OF THE INVENTION
[0003] Food is often placed in containers or bags for storage in freezers, refrigerators, pantries, cabinets, etc. For example, people often scatter many foods in different bags in their freezers. The bags may contain sauces, proteins (e.g., fish, chicken, steak, etc.), marinades, vegetables, cooking ingredients, combinations thereof, etc. However, it may be difficult to determine or remember what foods are in what bags—especially when the foods are frozen in sealed bags. In addition, since foods can be frozen for a long time, people may easily forget that certain foods are stored in their freezers. SUMMARY OF THE INVENTION
[0004] An object of the present invention is to substantially overcome or at least ameliorate one or more disadvantages of the existing arrangements.
[0005] In one aspect, there is provided a food storage system, comprising: at least one processor; and at least one non-transitory processor-readable medium storing processor-executable instructions which, when executed by the at least one processor, cause the at least one processor to: receive data uniquely identifying a storage bag from a user device; receive data regarding the contents of the storage bag from the user device; associate the data regarding the storage bag with the data regarding the contents of the storage bag; generate a notification based on the data regarding the contents of the storage bag; and send the notification to the user device.
[0006] In certain embodiments, the at least one processor is configured to generate a notification to indicate: cooking recommendations based on the data regarding the contents of the storage bag; and / or eating recommendations based on the data regarding the contents of the storage bag.
[0007] In certain embodiments, the data regarding the storage bag indicates a machine-readable identifier printed on or attached to the storage bag, wherein the machine-readable identifier is a unique identifier.
[0008] In certain embodiments, the at least one processor is configured to: receive a query request from a user device indicating data uniquely identifying a storage bag in which contents are stored; query a data repository using the data uniquely identifying the storage bag to determine the contents of the storage container; and send data indicating the contents of the storage container to the user device.
[0009] In some embodiments, the at least one processor is configured to: receive input data regarding the contents of the storage bag from a user device; determine a notification sending date for sending a notification to the user device based on the input data; and schedule the delivery of the notification to the user device on the notification sending date.
[0010] In some embodiments, the at least one processor is configured to receive temperature data indicating the storage temperature of the storage bag from a computing device of a storage device in which the storage bag is stored, wherein the at least one processor further determines the notification sending data based on the temperature data.
[0011] In some embodiments, the storage device is a refrigerator or a freezer.
[0012] In some embodiments, the storage bag has a logo printed or attached to its outer surface, wherein the at least one processor is configured to: receive image data indicating the logo of the storage bag containing the contents; process the image data to determine the position or distortion of the logo in a three-dimensional (3D) space; determine one or more characteristics of the contents at least partially based on the position or distortion of the logo; and further determine the notification sending date based on the one or more characteristics of the contents of the storage bag.
[0013] In some embodiments, the at least one processor is configured to process the image data using a computer vision algorithm.
[0014] In some embodiments, the one or more characteristics include at least one of weight, thickness, volume, shape, and / or food type, wherein the contents of the storage bag are one or more food items.
[0015] In a second aspect, there is provided a system comprising: the food storage system of the first aspect; and a storage bag.
[0016] In a third aspect, there is provided a method for operating a food storage system, wherein the method comprises: receiving data uniquely identifying the storage bag from a user device; receiving data regarding the contents of the storage bag from the user device; correlating the data regarding the storage bag with the data regarding the contents of the storage bag; generating a notification based on the data regarding the contents of the storage bag; and sending the notification to the user device.
[0017] In some embodiments, the method further comprises generating a notification to indicate: cooking recommendations based on the data regarding the contents of the storage bag; and / or eating recommendations based on the data regarding the contents of the storage bag.
[0018] In certain embodiments, data regarding the storage bag indicates a machine-readable identifier printed on or attached to the storage bag, wherein the machine-readable identifier is a unique identifier.
[0019] In certain embodiments, the method further comprises: receiving, from a user device, a query request indicating data that uniquely identifies the storage bag in which the inclusion is stored; querying a data repository with the data that uniquely identifies the storage bag to determine the inclusion of the storage container; and sending, to the user device, data indicating the inclusion of the storage container.
[0020] In certain embodiments, the method further comprises: receiving input data regarding the inclusion of the storage bag from a user device; determining a notification sending date for sending a notification to the user device based on the input data; and scheduling delivery of the notification to the user device on the notification sending date.
[0021] In certain embodiments, the method further comprises receiving, from a computing device of a storage device in which the storage bag is stored, temperature data indicating the storage temperature of the storage bag, wherein the at least one processor further determines the notification sending data based on the temperature data.
[0022] In certain embodiments, the storage device is a refrigerator or a freezer.
[0023] In certain embodiments, the storage bag has a logo printed or attached to its outer surface, wherein the method further comprises: receiving image data indicating the logo of the storage bag containing the inclusion; processing the image data to determine the position or distortion of the logo in three-dimensional (3D) space; determining, at least in part, one or more characteristics of the inclusion based on the position or distortion of the logo; and further determining the notification sending date based on the one or more characteristics of the inclusion of the storage bag.
[0024] In certain embodiments, the method further comprises processing the image data using a computer vision algorithm.
[0025] In certain embodiments, the one or more characteristics include at least one of weight, thickness, volume, shape, and / or food type, wherein the inclusion of the storage bag includes one or more food items.
[0026] In a fourth aspect, there is provided one or more non-transitory processor-readable media storing processor-executable instructions that, when executed by the at least one processor, cause the at least one processor to: receive, from a user device, data uniquely identifying a storage bag; receive, from the user device, data regarding the contents of the storage bag; associate the data regarding the storage bag with the data regarding the contents of the storage bag; generate a notification based on the data regarding the contents of the storage bag; and send the notification to the user device.
[0027] In certain embodiments, the one or more non-transitory processor-readable media further include executable instructions that configure the at least one processor to generate a notification to indicate: a cooking recommendation based on the data regarding the contents of the storage bag; and / or an eating recommendation based on the data regarding the contents of the storage bag.
[0028] In certain embodiments, the data regarding the storage bag indicates a machine-readable identifier printed on or attached to the storage bag, where the machine-readable identifier is a unique identifier.
[0029] In certain embodiments, the one or more non-transitory processor-readable media include additional executable instructions that configure the at least one processor to: receive, from a user device, a query request indicating data uniquely identifying a storage bag in which contents are stored; query a data repository using the data uniquely identifying the storage bag to determine the contents of the storage container; and send to the user device data indicating the contents of the storage container.
[0030] In certain embodiments, the one or more non-transitory processor-readable media include additional executable instructions that configure the at least one processor to: receive, from a user device, input data regarding the contents of the storage bag; determine, based on the input data, a notification sending date for sending the notification to the user device; and schedule delivery of the notification to the user device on the notification sending date.
[0031] In certain embodiments, the one or more non-transitory processor-readable media include additional executable instructions that configure the at least one processor to: receive, from a computing device of a storage device in which the storage bag is stored, temperature data indicating the storage temperature of the storage bag, where the at least one processor further determines a notification sending date based on the temperature data.
[0032] In some embodiments, the storage bag has a logo printed or attached to its outer surface, wherein the one or more non-transitory processor-readable media include additional executable instructions that configure the at least one processor to: receive image data indicative of the logo of the storage bag containing the inclusion; process the image data to determine the position or distortion of the logo in three-dimensional (3D) space; determine one or more characteristics of the inclusion at least in part based on the position or distortion of the logo; and further determine a notification sending date based on the one or more characteristics of the inclusion in the storage bag.
[0033] In some embodiments, the one or more non-transitory processor-readable media include additional executable instructions that configure the at least one processor, which is configured to process the image data using a computer vision algorithm.
[0034] In some embodiments, the one or more characteristics include at least one of weight, thickness, volume, shape, and / or food type, wherein the inclusion in the storage bag includes one or more food items.
[0035] In a fifth aspect, there is provided a food storage system comprising: at least one processor; and at least one non-transitory processor-readable medium storing processor-executable instructions that, when executed by the at least one processor, cause the at least one processor to: receive data uniquely identifying the storage container from a user device, wherein the data uniquely identifying the storage container is obtained from at least one machine-readable symbol printed on the storage container or printed on a label attached to the storage container; receive data about the inclusion in the storage container from the user device; and associate the data about the storage bag with the data about the inclusion in the storage container.
[0036] In a sixth aspect, there is provided a system comprising the food storage system of the fifth aspect and the storage container.
[0037] In a seventh aspect, there is provided a method of operating a food storage system, the method comprising the steps of: receiving data uniquely identifying the storage container from a user device, wherein the data uniquely identifying the storage container is obtained from at least one machine-readable symbol printed on the storage container or printed on a label attached to the storage container; receiving data about the inclusion in the storage container from the user device; and associating the data about the storage bag with the data about the inclusion in the storage container.
[0038] In an eighth aspect, there is provided one or more non-transitory processor-readable media storing processor-executable instructions that, when executed by at least one processor, configure the at least one processor to: receive, from a user device, data that uniquely identifies a storage container, wherein the data that uniquely identifies the storage container is obtained from at least one machine-readable symbol printed on the storage container or printed on a label attached to the storage container; receive, from the user device, data regarding the contents of the storage container; and associate data regarding the storage bag with data regarding the contents of the storage container.
[0039] Other aspects and embodiments will be understood throughout the description of the one or more preferred embodiments. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] The example embodiments will become apparent from the following description given only by way of example of at least one preferred but non-limiting embodiment described in conjunction with the accompanying drawings. The components in the drawings are not necessarily drawn to scale. In fact, emphasis is placed on clearly illustrating the principles of the present disclosure.
[0041] Figure 1 is a block diagram of an example of a user device on which some embodiments of the disclosed technology may operate.
[0042] Figure 2 is a block diagram of an example of a distributed computing environment in which some embodiments of the disclosed technology may operate.
[0043] Figure 3 is a perspective view of an example of a storage bag in which food is placed.
[0044] Figure 4A is a front view of an example of a storage bag.
[0045] Figure 4B is one in which food is placed Figure 4A of a perspective view of a storage bag.
[0046] Figure 5 is a schematic illustration of an example of a food storage system.
[0047] Figure 6 is a flowchart of an example of a process or method for tracking the contents of a storage bag. DETAILED DESCRIPTION
[0048] Aspects of the present disclosure generally relate to systems and methods for sharing data regarding the contents of a storage bag or container with an external device. In several embodiments described below, a food storage system is operable to track / record data regarding the contents of a storage container (e.g., a rigid vessel) or a storage bag (e.g., a plastic bag) to allow a user to identify the contents therein. In a particular embodiment, when a user seals food in a storage bag, the user can scan a QR code or other machine-readable identifier on the storage bag or capture an image of the storage bag using a user device (e.g., a mobile phone or a tablet). The user can also provide information about the food, such as what the food is, how large it is, what it is marinated in, etc. The QR code or the image can uniquely identify the storage bag such that at a later date, the user can rescan or capture another image of the storage bag to retrieve the information about the food. In some embodiments, the food storage system can provide reminders, recommendations, or other tips regarding the use (e.g., cooking) of the contents in the storage container or bag. In some embodiments, at least some of the reminders, recommendations, or other tips regarding the food in the storage container or bag are provided before use (e.g., cooking, eating, etc.).
[0049] In the following detailed description, specific details are set forth to provide an understanding of the present technology. However, the inventive technology may be practiced without some of these specific details. In some instances, well-known structures and techniques are not shown in detail so as not to obscure the present technology. Although the terms used in the description presented below are used in the context of a detailed description of certain specific embodiments of the present disclosure, it is intended to be interpreted in the broadest reasonable manner. For example, although certain aspects of the operation of the system and method are described in the context of a storage bag for storing food in a freezer, the inventive technology is applicable to many different types of food storage items (e.g., rigid or semi-rigid containers) intended to be stored in many different types of environments (e.g., in a cooler, a cabinet, a pantry, etc.). Certain terms may even be emphasized below; however, any term that is intended to be interpreted in any restrictive manner will be disclosed and explicitly defined in this detailed description section.
[0050] The headings provided herein are for convenience only and do not necessarily affect the scope of the embodiments.
[0051] I. Selected Embodiments of a Suitable Computing Environment
[0052] Figure 1 and 2The following discussion provides a brief general description of a suitable computing environment in which aspects of the inventive systems and methods for sharing information / data regarding the contents of a storage container with an external device may be implemented. The inventive technology may be embodied as special purpose hardware (e.g., circuitry), programmable circuitry appropriately programmed with software and / or firmware, or a combination of special purpose circuitry and programmable circuitry. Accordingly, embodiments of the inventive technology may include a machine-readable medium having instructions stored thereon that may be used to cause a computer, microprocessor, processor, and / or microcontroller (or other electronic device) to perform a process. The machine-readable medium may include, but is not limited to, optical disks, compact disk read-only memory (CD-ROM), magneto-optical disks, ROM, random access memory (RAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic or optical cards, flash memory, or other types of media / machine-readable media suitable for storing electronic instructions.
[0053] The inventive technology may also be practiced in a distributed computing environment where tasks or modules are performed by remote processing devices linked via a communication network such as, for example, a local area network (“LAN”), a wide area network (“WAN”), or the Internet. In a distributed computing environment, program modules or subroutines may be located in both local and remote memory storage devices. Aspects of the technology described above may be stored or distributed on a computer-readable medium, or aspects of the invention may be distributed electronically over the Internet or other networks, including wireless networks. Those skilled in the relevant art will recognize that portions of the invention may reside on a server computer while corresponding portions reside on a client / user computer. Specific data structures and the transmission of data of aspects of the invention are also emphasized within the scope of the invention.
[0054] Figure 1 is a block diagram of a user device 100 on which some embodiments of the disclosed technology may operate. The user device 100 may be a smart phone, a tablet computer, or a personal computer. The user device 100 may include one or more input devices 102 that provide input to a CPU (one or more processors) 104, thereby notifying it to take an action. The action is typically mediated by a hardware controller that interprets the signals received from the input device 102 and conveys information to the CPU 104 using a communication protocol. The input devices 102 include, for example, a mouse, a keyboard, a touch screen, an infrared sensor, a touch pad, a wearable input device, a camera or image-based input device, a microphone, and / or other user input devices.
[0055] The CPU 104 can be a single processing unit or multiple processing units in the device, or distributed across multiple devices. For example, the CPU 104 can be coupled to other hardware devices through a bus such as a PCI bus or a SCSI bus. The CPU 104 can communicate with a hardware controller for a device such as the display 106. The display 106 can be used to display text and graphics. In some instances, the display 106 provides visual feedback of graphics and text to the user. In some embodiments, the display 106 includes one or more of the input devices 102 as part of the display, such as when the input device 102 includes a touch screen or an eye direction monitoring system. The display 106 can be separate from the input device 102. Examples of display devices include: LCD screens, LED displays, projections, holographic or augmented reality displays (e.g., head-up display devices or head-mounted devices), etc. Other I / O devices 108 can also be coupled to the processor, such as network cards, video cards, audio cards, USB, FireWire, or other types of external devices, cameras, printers, speakers, CD-ROM drives, DVD drives, disk drives, or Blu-ray devices.
[0056] The user device 100 can also include a communication device (e.g., a wireless transceiver) capable of communicating with a network node in a wireless or wire-based manner. The communication device can communicate with another device or server through a network using, for example, the TCP / IP protocol. The user device 100 can utilize the communication device to distribute operations across multiple network devices.
[0057] The CPU 104 may be able to access the memory 110. The memory 110 can include one or more of various hardware devices for volatile storage, non-volatile storage, or both, and can include both read-only and writable memories. For example, the memory 110 can include random access memory (RAM), CPU registers, read-only memory (ROM), and writable non-volatile memory such as flash memory, hard disk drives, floppy disks, CDs, DVDs, magnetic storage devices, tape drives, device buffers, etc. The memory 110 is not a propagated signal separate from the underlying hardware and is non-transitory. The memory 110 can include a program memory 112 that stores programs and software such as an operating system 114, a storage bag contents identification, tracking, reminder, and recommendation platform 116, and other application programs 118 (e.g., including a predictive cooking platform). As described in more detail below, the memory 110 can also include a data memory 119 that can store information about one or more storage bags and the contents (e.g., food) within the storage bags. The information in the data memory 119 can be received / captured from the input device 102 and / or other I / O devices 108 and can be provided to the program memory 112 or any element of the user device 100.
[0058] Some embodiments of the technology of the present invention can operate in many other general or special computing system environments or configurations. Examples of well-known computing systems, environments, and / or configurations suitable for use with the technology include, but are not limited to, personal computers, server computers, handheld or laptop devices, cellular phones, mobile phones, wearable electronics, gaming consoles, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like.
[0059] More specifically, Figure 2 is a block diagram of a distributed computing environment 220 in which some embodiments of the disclosed technology can operate. Environment 220 can include one or more user devices 200 (individually identified as user devices 200a - 200d), examples of which can include the user device 100 described in detail above with respect to Figure 1 The user device 200 can operate in a networked environment using a logical connection via a network 222 to one or more remote computers. For example, in the illustrated embodiment, the user device 200 is communicatively coupled to a remote server computing device 224 via the network 222.
[0060] The server computing device 224 can be an edge server that receives client / user requests and coordinates the fulfillment of those requests via other servers such as server computing devices 226 (individually identified as server computing devices 226a - 226c). The server computing devices 224 and 226 can include computing systems such as the user device 100. In the illustrated embodiment, each of the server computing devices 224 and 226 is logically shown as a single server. Alternatively, some or all of the server computing devices 224 and 226 can be a distributed computing environment encompassing multiple computing devices located at the same physical location or at geographically distinct physical locations. In some embodiments, each of the server computing devices 226 corresponds to a group of servers.
[0061] The user device 200 and the server computing devices 224 and 226 can each act as a server or a client to other server / client devices. The server computing device 224 can be connected to the database 225, and the server computing devices 226a - 226c can each be respectively connected to the corresponding databases 227a - 227c. As discussed above, each of the server computing devices 226 can correspond to a group of servers, and each of these servers can share a database or can have its own database. As described in more detail below, the databases 225 and 227 can store (e.g., hold) information such as the captured / received information about one or more storage bags and the contents of the storage bags. In the illustrated embodiment, the databases 225 and 227 are logically shown as a single unit. Alternatively, the databases 225 and 227 can each be a distributed computing environment encompassing multiple computing devices and can be located within their respective servers or can be located at the same physical location or at geographically different physical locations.
[0062] The network 222 can be a local area network (LAN) or a wide area network (WAN), but can also be other wired or wireless networks. The network 222 can be the Internet or some other public or private network. The user device 200 can be connected to the network 222 via a network interface, e.g., by wired or wireless communication. Although the connections between the server computing devices 224 and 226 are shown as separate connections, these connections can be any kind of local network, wide area network, wired network, or wireless network, including the network 222 or a separate public or private network.
[0063] II. Selected Embodiments of a Suitable Storage Bag
[0064] Figure 3 is a perspective view of a storage bag 330 having food 334 disposed therein. In the illustrated embodiment, the storage bag 330 is a plastic bag or pouch and includes a sealing member 332, such as a single - press zipper seal, a multi - press zipper seal, or an adhesive tape (initially covered with a release tape before the bag is closed), or other sealing members known in the art. The user can unseal the sealing member 332 to position the food 334 within the storage bag 330 and then reseal the sealing member 332 to provide a liquid - tight and / or air - tight seal, and then store the storage bag 330 and the food 334 in a refrigerator, freezer, or other suitable place. The bag 330 can be formed of a transparent, light - blocking, or opaque plastic material (e.g., a polymer or a laminated film). In the illustrated embodiment, the bag 330 has a generally rectangular shape, while in other embodiments, the bag 330 can have other shapes (e.g., circular, linear, oval, polygonal, triangular, etc.).
[0065] Food 334 may include one or more of protein (e.g., fish, steak, etc.), sauce, marinade, vegetables, cooking ingredients, etc. In some embodiments, the storage bag 330 may be a sous vide bag configured to be submerged in a water bath during a sous vide cooking process. For example, the storage bag 330 may be the same as or similar to any of the sous vide bags described in detail in the following documents: U.S. Provisional Patent Application No. 62 / 738,786, filed on September 28, 2018, and titled "Sous Vide Pouches and Systems and Methods for Tracking Sous Vide Pouches", and International Patent Application No. PCT / IB2019 / 058210, filed on September 27, 2019, and titled "Sous Vide Pouch, Assembly, System and Method", both of which are incorporated herein by reference in their entireties.
[0066] Alternatively, the storage bag 330 may be a vacuum bag that includes a vacuum port (not shown), or may be a vacuum bag that omits the sealing member 332 and is configured to be evacuated and sealed by an external vacuum sealing machine that heat seals the bag to melt the opposing bag surfaces (often after most of the air has been removed from the bag). For example, the material for the storage bag 330 may be provided as a continuous open-ended material tube positioned on a reel and may be custom cut and sealed by a user (e.g., using an external vacuum sealing machine).
[0067] As Figure 3 further shown, the code / identifier 336 may be pre-printed on or otherwise attached to the first (i.e., top) surface of the storage bag 330. The identifier 336 may be printed directly on the storage bag 330 using an ink established for use at low and high temperatures (e.g., low migration ink). Alternatively, the identifier 336 may be a sticker sheet that a user may attach to the storage bag 330 after the user has custom cut and sealed the storage bag 330 from the material reel. In yet another alternative, the user may print the identifier 336 on the storage bag 330 using a printer.
[0068] The identifier 336 can be any of the following: various types of barcodes (read linearly), any of the various codes read in two dimensions (e.g., QR codes), an identifier that can be read electromagnetically (e.g., radio frequency identifier (RFID)), or other automatically readable identifiers that can be scanned / read by a user device (e.g., via a camera and an application running on a mobile phone or tablet) for uniquely identifying the storage bag 330. As described in detail below, in some embodiments, the scanning or imaging of the identifier 336 can automatically open an application running on the user device, and the user can input information about the food 334 into the user device to associate the food 334 with the storage bag 330.
[0069] Figure 4A is a top view of an alternative storage bag 430. Figure 4B is Figure 4A is a perspective view of the alternative storage bag 430 with the food 434 disposed therein as shown in. For clarity, features of the storage bag 330 shared with the storage bag 430 include reference numerals incremented by 100. As shown, the storage bag 430 includes a sealing member 432, and a logo, which may be provided in the form of a pattern 438 printed on the first (i.e., top) surface of the storage bag 430 or otherwise attached to the first surface. The logo 438 may be printed on the storage bag 430 as an alternative or supplement to a QR or other machine-readable identifier (not shown). In some embodiments, a larger logo may be formed on the material web from which the storage bag 430 is cut, and the logo 438 may be a part or sub-segment of the larger logo. In the illustrated embodiment, the logo 438 includes an irregular arrangement of shapes 435, lines 437, and / or dots 439 that can be used to uniquely identify the storage bag 430. In other embodiments, the pattern 438 may include any combination of shapes, lines, dots, etc. that distinguish the storage bag 430 from one or more other storage bags. As described in detail below, in some embodiments, the user can capture an image of the logo 438 on the storage bag 430 via a user device (e.g., the camera of a smart phone or tablet). The user can then input information about the food 434 into the user device (e.g., via an application running thereon), and the information can be associated with the storage bag 430, as uniquely identified by the logo 438.
[0070] In some embodiments, the logo 438 can be configured to enable an estimation of various characteristics of the food 434. Specifically, placing the food 434 inside the storage bag 430 causes the pattern 438 to distort with respect to when the storage bag 430 is lying flat. Positioning the food 434 in the bag 430 causes the first surface 433 of the bag 430 to be generally non-planar - which causes the logo 438 to be distorted compared to Figure 4AThe planar indicia shown in is distorted. Optionally, the user may place the bag 430 in a vacuum device to remove air from the bag 430, or may manually remove some air from the bag 430 before sealing the bag. This distortion may provide information about the characteristics of the food 434, such as the thickness, volume, shape, or mass of the food 434. Also, if the storage bag 430 is transparent, the color, size, position, shape, etc. of the food 434 in the storage bag 430 may provide information about the characteristics of the food 434, complementary or alternative to any distortion of the pattern 438. For example, the position of the food 434 relative to the pattern 438 - even if the storage bag 430 is not tightened around the food 434 - may provide information about the size (e.g., shape, volume, thickness, etc.) or weight of the food 434. Even when the storage bag 430 is opaque and not tightened around the food 434, the distortion of the pattern 438 from the food 434 may provide information that permits accurate estimation of the thickness and shape of the food 434. More specifically, in some embodiments, the storage bag 430 may be the same as or similar to any of the vacuum sous vide bags described in detail in the following: U.S. Provisional Patent Application No. 62 / 768,776, filed November 16, 2018, and titled "METHODS FOR ESTIMATING CHARACTERISTICS OF FOOD ITEMS PLACED IN BAGS, SUCH AS FOR USE IN SOUS VIDE COOKING", and International Patent Application No. PCT / IB2019 / 059815, filed November 15, 2019, and titled "ESTIMATING A CHARACTERISTIC OF A FOOD ITEM".
[0071] In Figure 4A and 4B the illustrated embodiment of , the indicia 438 is positioned on the first surface 433 of the bag 430 such that it generally overlays or superposes over the food 434. In other embodiments, the indicia 438 may comprise another regular pattern and may extend more or less above the first surface 433 of the bag 430. For example, the pattern 438 may extend to the sides of the bag 430 and thus cover the entire first surface 433. In the embodiment shown in Figure 4, the bag 430 may also contain additional indicia (not depicted) on a second surface (Figure 4) opposite the first surface 433. The indicia on the first surface 433 and the second surface of the bag may have different colors such that they may be distinguished in an image taken of the bag 430. The indicia 438 on the first surface 433 and / or the indicia on the second surface may be imaged to provide information about the food 434.
[0072] In Figure 3 and 4A In the illustrated embodiments, the bags 330, 430 are formed of a transparent or light-transmissive material such that the food items 334, 434 are visible within the bags 330, 430. Alternatively, the bags 330, 430 may be formed of a different material such that the food items 334, 434 are not visible within the bag 430.
[0073] III. Selected Embodiments of a System for Sharing Data of Storage Bag Contents with an External Device
[0074] Figure 5 is a block diagram of a suitable food storage system 540 (“System 540”) for sharing information / data regarding the contents of a storage bag or container with an external device. The System 540 is operable to track / record data regarding the contents of a plurality of storage containers or storage bags. Although certain aspects of the operation of the System 540 are described below in the context of storing food items within a storage bag (e.g., a plastic bag), those skilled in the art will understand that the System 540 is configured to operate with respect to many different types of storage vessels capable of storing many different types of items. Additionally, the System 540 may operate on and / or between user devices 100 and 200 and / or other devices, systems, etc., and it may be distributed between such devices and, for example, server computing devices 224 and 226, as described in detail above with reference to Figure 1 and 2 described in detail.
[0075] In the illustrated embodiment, the System 540 is configured to receive data (e.g., via an application running on the user device 500) regarding / relating to a storage bag 530 (e.g., Figure 3 , 4A and the storage bags 330 and / or 430 shown in 4B) and the contents (e.g., food items placed within the storage bag 530) from the user device 500. Data regarding the storage bag 530 may include data that uniquely identifies each storage bag 530. For example, the user device 500 may capture image data of the storage bag 530 (e.g., image data that includes a unique identifier on the bag), read / scan a QR code on the storage bag 530, read an RFID tag embedded within the storage bag 530, etc. At the same time, the user device 500 may receive one or more user inputs that provide information regarding the food items stored within the storage bag 530. The System 540 may then associate the storage bag 530 with the food items placed therein. At a later date, when, for example, the user scans, takes a picture (etc.) of a particular one of the storage bags 530, the System 540 may display data regarding the food items stored within that particular one of the storage bags 530. The System 540 may further operate to provide reminders, recommendations, or other tips regarding the use (e.g., cooking) of the contents within the storage bag 530.
[0076] More specifically, system 540 may include a bag identification module 542, a bag contents identification module 544, a notification module 546, and a cooking recommendation module 548. System 540 may read, store, and / or access data from a bag and bag contents ("bag") database 550, a user database 552, a notification database 554, and a recommendation database 556. Other databases (not shown) may be connected to system 540. Also, although the databases are shown separately, the databases may be integrated into fewer databases or data structures (e.g., a single database or a unified data structure).
[0077] The bag identification module 542 is configured to receive data regarding the storage bag 530 from the user device 500 to uniquely identify the storage bag 530. For example, the user device 500 may capture an image of the storage bag 530, and / or read / scan machine-readable data directly stored on the storage bag 530, such as an OR code or a logo printed on or otherwise attached to the storage bag 530, or a radio frequency identification (RFID) or near field communication (NFC) chip or tag embedded in or otherwise attached to the storage bag 530. If the storage bag 530 has not been scanned previously, the bag identification module 542 may update the bag database 550 to add the newly scanned / identified storage bag 530. As described in more detail below, if the storage bag 530 has been scanned previously, the bag identification module 542 may retrieve data regarding the contents of the storage bag 530 from the bag database 550.
[0078] In one aspect of the present technology, the storage bag 530 only needs to be uniquely identifiable for an individual user. That is, the storage bag 530 does not need to be globally unique, but only locally unique for the user's specific storage environment (e.g., uniquely identifiable relative to other storage bags 530 in the same freezer of the same user). Accordingly, the bag identification module 542 may receive and store (e.g., in the user database 552) data that associates the user device 500 with the specific storage bag for which they have provided data (e.g., scanned, imaged, etc.).
[0079] The bag content identification module 544 is configured to receive data regarding food or other contents placed within the storage bag 530 from the user device 500. For example, a user may operate one of the user devices 500 to scan a QR code on one of the storage bags 530 in which they recently placed food. Scanning the QR code may automatically open an application running on the user device 500, prompting the user to enter data regarding the food. For example, the user may enter data regarding the food such as type (e.g., beef, chicken, fish, vegetables, sauce, marinade, etc.), size, thickness, weight, intended storage location (e.g., freezer, refrigerator, etc.), use or cooking expiration, marinade or sauce type, preferred marinade to use with the food, meal or recipe the food is preferably used for, cost of the food, and / or other notes or characteristics regarding the food. The application running on the user device 500 may provide a GUI that provides drop-down options to select common data fields such as food type (beef, fish, etc.), preferred end use (sous vide for one type of fish, pressure cooking for one type of beef), and other data fields. Additionally, an identifier on the storage bag 530 (e.g., Figure 3 identifier 336) may have data encoded therein such as the size of the bag, possible later uses for the bag (sous vide machine vs. slow cooker vs. pressure cooker), etc., such that a particular storage bag 530 may be adapted to be used with a particular food, adapted for a particular end use, etc. The bag content identification module 544 may then update the bag database 550 to associate the storage bag 530 with the data regarding the food placed therein. That is, the system 540 may update the bag database 550 to include a record of the particular food (and its characteristics) stored within each of the storage bags 530.
[0080] In addition to receiving data about the bag contents from the user, the bag contents identification module 544 may automatically determine specific data about the food placed in the storage bag 530. For example, the bag contents identification module 544 may automatically determine the date and time (e.g., the date and time the user device 500 scans the storage bag 530) that the food was sealed in one of the storage bags 530. In some embodiments, the bag contents identification module 544 may automatically estimate one or more characteristics of the food after the food is placed in the storage bag 530, such as the shape, thickness, volume, weight, type, quantity, surface heat transfer coefficient, and / or other characteristics of the food. For example, the system 540 may receive image data from one of the user devices 500 and may process the image data using one or more computer vision algorithms to estimate the one or more characteristics. More specifically, the system 540 may analyze the distortion of the pattern on the storage bag 530 (e.g., the pattern 438 shown in FIG. 4) to estimate the one or more characteristics, as described in detail in the following documents: U.S. Provisional Patent Application No. 62 / 768,776, filed Nov. 16, 2018, and titled "METHODS FOR ESTIMATING CHARACTERISTICS OF FOOD ITEMS PLACED IN BAGS, SUCH AS FOR USE IN SOUS VIDE COOKING," and International Patent Application No. PCT / IB2019 / 059815, filed Nov. 15, 2019, and titled "ESTIMATING A CHARACTERISTIC OF A FOOD ITEM." For example, the processor of the system 540 may execute the inclusion estimation module 544 stored in the memory, which employs one or more computer vision or augmented reality algorithms stored in the memory to process the image data to determine the position / orientation of the markers on the bag 530 in three-dimensional (3D) space.The processor of system 540 may employ algorithms described, for example, in the following documents: "Image Restoration of Arbitrarily Warped Documents", Brown, Michael S and W. Brent Seales, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 26, No. 10, October 2004, pp. 1295-306; and / or "A New Approach to Image-Based Estimation of Food Volume", Hassannejad et al., Algorithms, Vol. 10, No. 2, June 10, 2017. That is, the processor of system 540 may estimate / model the 3D shape of bag 530 based on the distortion of known markers on bag 530. For example, the processor of system 540 takes known data stored in the memory (e.g., a previous image of the marker, or the known geometry of the marker), and compares it with a newly received image of the marker distorted by the food at the inner bottom of bag 530. The known data may be locally stored in the memory of the user device. Those skilled in the art will understand that other computer vision algorithms may be used. In one form, augmented reality toolkits for Android TM (ARCore) and Apple TM iOS (ARKit) may be used to perform computer vision algorithms. Deep neural networks and machine learning techniques may also be used to determine the type of food (e.g., sirloin steak vs. lamb chop).
[0081] System 540 or user device 500 may additionally be configured to determine one or more characteristics of the food before and / or after removal from storage bag 530. For example, system 540 or user device 500 may use a trained image classification model stored in the memory and developed using machine learning to determine characteristics of the food, such as the shape, thickness, volume, weight, type, quantity, and / or other characteristics of the food. For example, the image classification model may use a neural network library called Keras TM to be developed. In another example, the image classification model may use Tensorflow TMDevelopment. In one form, a user may capture an image of food to estimate properties about the food using a trained image classification model. Results indicating one or more properties of the food determined by the trained image classification model are presented to the user for confirmation and / or adjustment. The confirmed properties are then stored in association with identifiers as discussed in the examples contained in the specification. Thus, the estimated properties can be used as input data for generating one or more notifications for a storage bag that will contain the food. Using this technique, the user does not need to input several properties of the food because the trained image classification model is automatically used to estimate these properties.
[0082] The bag identification module 542 is further configured to retrieve data about the contents of the storage bag 530 from the bag database 550. For example, after encapsulating one of the storage bags 530 with a specific food, scanning the storage bag 530 and inputting data about the food, the user may place the storage bag 530 in their freezer, refrigerator, or another suitable location. Sometimes, the user may not remember specific data about the food, such as what it is, why they stored it, when it was encapsulated, and so on. Accordingly, the user may re-scan the identifier (e.g., QR code) on the storage bag 530, take another photo of the storage bag 530, or otherwise identify the storage bag 530. The bag contents identification module 544 can then match the newly received data about the storage bag 530 with the data about the storage bag 530 previously stored in the bag database 550 and retrieve the associated data about the food therein. The data about the food can then be output to the user via one or more of the user devices 500. In this way, the system 540 facilitates the identification of the food stored in the storage bag 530.
[0083] The notification module 546 is configured to provide a user device 500 with notifications or reminders regarding a specific food item, such as a reminder to cook / consume a specific food item before its expiration date, or a reminder to cook / consume a specific food item within a time window of optimal flavor and / or texture or before an unwanted change in flavor and / or texture (i.e., freezer burn). The notification module 546 can be configured to generate a notification based on data regarding the contents of the storage bag to indicate a cooking recommendation. For example, the cooking recommendation can indicate that the food in the storage bag should be cooked by a specific date. Additionally or alternatively, the notification module can be configured to generate a notification based on data regarding the contents of the storage bag to indicate a consumption recommendation. For example, the food may have been cooked and then frozen. The notification module can be configured to generate a notification to indicate a consumption recommendation, thereby indicating that the cooked food should be consumed by a specific date to reduce the probability of the food being freezer burned. It should be understood that separate or dedicated notifications can be sent for cooking recommendations and consumption recommendations. However, in some instances, the generated notification can include a cooking notification and a consumption recommendation. In some examples, the notification can relate to multiple food items stored in separate storage bags in order to reduce the number of notifications sent to the user device.
[0084] The system 540 can be configured to receive input data regarding the contents of the storage bag from the user device 500. The notification module 546 of the system can be configured to determine a notification send date for sending a notification to the user device based on the input data. The notification module 546 can be configured to schedule the delivery of the notification to the user device 500 on the notification send date. The input data can be input by a user interacting with an input device of the user device 500. However, the input data can be received by the system 540 from other sources. In one form, the input data is received from a computing device of a storage device in which the storage bag is stored. The input data can include temperature data indicating the storage temperature of the storage bag, wherein the at least one processor further determines the notification send data based on the temperature data. In these instances, the storage device is a refrigerator or a freezer.
[0085] A notification or reminder can be sent to one or more of the user devices 500 before food contained in the cooking storage bag 530. In some embodiments, the system 540 can receive a user request to send a notification or reminder at a specific predetermined time and date. For example, the notification module 546 can prompt the user to enter a cooking expiration as part of the data entered regarding the food (e.g., at the time of encapsulation). Additionally or alternatively, the notification module 546 can prompt the user to enter a consumption expiration as part of the data entered regarding the food. The notification module 546 can further update the notification database 554 with the cooking expiration and / or consumption expiration, and can issue a notification to one of the user devices 500 associated with the user at or before the cooking expiration and / or consumption expiration. The notification can include a photo of the storage bag 530 in which the food is stored, or other data helpful in identifying the storage bag 530. In some embodiments, the notification can include instructions executable by the user device 500 to update a calendar application running thereon to include / display the cooking expiration.
[0086] In some embodiments, the notification module 546 is configured to automatically generate and send notifications based on (i) one or more predefined rules (e.g., rules stored in the notification database 554) and (ii) data regarding the contents of the storage bag 530. For example, if the system 540 receives input data that a particular one of the storage bags 530 contains fresh chicken and is stored in the freezer, the system 540 may issue a notification before the estimated expiration date of the chicken. In another example, if the system 540 receives input data from a user indicating that a particular one of the storage bags 530 contains fresh chicken and was initially stored in the refrigerator, the system 540 may issue a notification to relocate the storage bag 530 containing the food to the freezer. Additional input data indicating the relocation location of the storage bag 530 may be received by the system 540 from the user such that another notification may be set by the notification module 546 in response to the relocated storage bag 530. In each of these examples, the system 540 limits the waste and / or consumption of food that may have passed its expiration date. In one aspect of the present inventive technique, the system 540 may remind users of food that they have forgotten when they first sealed and stored it or even completely forgotten that they ever sealed and stored it. The notification module 546 may be configured to additionally receive input data indicating one or more storage temperatures of the refrigerator or freezer in which the storage bag is stored. For example, at least some of the one or more rules may depend on the input data. For example, the notification generation date may depend on the operating temperature of the refrigerator / freezer device. In one example, the rules used by the notification module 546 cause a notification to be generated within a predetermined period and issued to one or more of the user devices, such as when the storage bag is stored in a refrigerator operating at a predetermined temperature (e.g., 3 degrees Celsius) or a temperature less than the predetermined temperature for 14 days. However, the notification module 546 may be configured to generate a notification and issue the notification to one or more of the user devices within 3 days when the storage bag is stored in a refrigerator operating at 5 degrees Celsius. In this example, the refrigerator and / or freezer device includes a computing device that communicates data with the system 540 to provide input data indicating the operating temperature, allowing the notification module to generate a notification on the appropriate notification date.
[0087] The cooking recommendation module 548 is configured to provide cooking recommendations based on the received data regarding the food stored in the storage bag 530. For example, the cooking recommendation module 548 may receive data regarding the type of food (e.g., as input by the user, or automatically determined by the bag contents identification module 544), and compare the data with the data stored in the recommendation database 556 to provide recipe recommendations and / or associated cooking instructions. The recommendations may be in the form of text alerts, emails, pop-ups, etc. that are published to one or more of the user devices 500. In some embodiments, the cooking recommendation module 548 may compare data regarding the food stored in multiple storage bags within the storage bag 530 and associated with the same user. The cooking recommendation module 548 may then recommend recipes based on the entire inventory of food associated with the user. For example, if the system 540 determines that the same user has different storage bags 530 in their freezer that contain chicken and lemon garlic sauce, the cooking recommendation module 548 may publish a notification of a recipe that includes lemon garlic chicken pasta to one of the user devices 500 associated with the user.
[0088] In some embodiments, the system 540 may be communicatively coupled to one or more cooking devices, such as a refrigerator, a stovetop baking dish, a sous vide cooking device, a pressure cooker, etc. In such embodiments, the cooking recommendation module 548 may provide cooking instructions (e.g., including time, temperature, etc.) for cooking the food in one of the storage bags 530 to the cooking device.
[0089] IV. Selected Embodiments of a Process for Tracking Storage Bag Contents
[0090] The present invention's technology enables tracking of the contents (e.g., food) in a storage bag via an application running on a user device. Specifically, the user may use the user device (e.g., a mobile phone or a tablet) to scan or capture an image of the storage bag when they package the food into the storage bag, while also providing data regarding the food via the user device. The scanning or imaging may provide data that uniquely identifies the storage bag, such that at a later date, the user may rescan or capture another image of the storage bag to retrieve data regarding the food.
[0091] Specifically, the at least one processor of the system 540 may be configured to receive a query request from the user device indicating data that uniquely identifies the storage bag in which the contents are stored. The at least one processor of the system 540 may then query the data repository using the data that uniquely identifies the storage bag to determine the contents of the storage container. The at least one processor of the system 540 may be configured to send data indicating the contents of the storage container to the user device. More specifically, Figure 6 is a flowchart of a process or method 660 for tracking storage bag contents. For illustrative purposes, it will be in Figure 5Describe some features of process 660 in the context of the embodiments shown.
[0092] Method 660 begins at block 661 by receiving data about a storage bag or storage container from a user device. For example, before, during, or after food is packaged into a storage bag (e.g., vacuum sealed in a plastic bag), the user may activate an application running on one of the user devices 500 and then scan a QR code on the storage bag and / or take a photo of the storage bag. In other embodiments, scanning the QR code or imaging the bag may automatically cause the application to run on the user device 500. The machine-readable data associated with the QR code and / or image data is transmitted to and received by system 540, which may process the data to uniquely identify the storage bag such that it can be associated with the food in the storage bag and later identified.
[0093] At block 662, method 660 includes receiving data about the contents of the storage bag. For example, the user may input data about the food stored in the storage bag, such as type, size, thickness, weight, intended storage location, cooking duration, etc., via an application running on the user device 500. Additionally or alternatively, system 540 may automatically determine / estimate one or more characteristics of the food, such as the packaging date or time, type, size, etc.
[0094] At block 663, method 660 includes associating the data about the storage bag with the data about the contents of the storage bag. For example, system 540 may jointly store this data in the bag database 550. Because the data about the storage bag includes data that uniquely identifies the storage bag, the data about the food can be retrieved later by rescanning the storage bag, reimaging the storage bag, etc.
[0095] At block 664, method 660 optionally includes sending one or more reminders. For example, system 540 may receive user-defined reminder dates and times (e.g., at block 662) and may generate and send a reminder to the one of the user devices 500 associated with the user at the date and time. Additionally or alternatively, system 540 may generate and send a reminder based on predefined rules and the received data about the food (e.g., received at block 662). For example, system 540 may generate and send a notification at the estimated expiration date or best before date of the food to remind the user to take out and cook the food.
[0096] At block 665, method 660 includes retrieving data regarding the storage bag again. For example, a user who wishes to determine the contents of a storage bag in their freezer may scan a QR code on the storage bag and / or take a photo of the storage bag. At block 666, method 660 includes providing bag contents data associated with the storage bag to the user. For example, system 540 may compare the QR code and / or image with data stored in bag database 550, retrieve food data associated with the storage bag, and send the food data to user device 500 associated with the user. In one instance, a user may remove a storage bag from their freezer and not remember what food is in the bag, when the food was packaged, what the food was marinated in, what meal they had intended to use the food for, etc. However, by previously entering this data into system 540 (e.g., at the time of packaging), system 540 can present the desired data to the user after re-scanning the storage bag, re-imaging the storage bag, etc.
[0097] At block 667, method 660 optionally includes providing one or more cooking recommendations to the user. For example, if the user re-scans a storage bag containing a steak, system 540 may provide recommendations on how to season and cook the steak, foods or wines to pair with the steak, or other foods in other storage bags associated with the user that can be used in a meal with the steak. In some embodiments, system 540 may provide cooking instructions for cooking food using a sous vide cooking process.
[0098] V. Conclusion
[0099] In general, a detailed description of embodiments of the present invention is not intended to be exhaustive or to limit the invention to the precise forms disclosed above. Although specific embodiments and examples of the present invention have been described above for illustrative purposes, various equivalent modifications are possible within the scope of the present invention as will be recognized by those skilled in the relevant art. For example, although processes or blocks are presented in a given order, alternative embodiments may execute routines with steps in a different order or employ systems with blocks, and some processes or blocks may be deleted, moved, added, subdivided, combined, and / or modified. Each of these processes or blocks can be implemented in many different ways. Also, although processes or blocks are sometimes shown as being executed serially, these processes or blocks can instead be executed in parallel or at different times.
[0100] Aspects of the present invention may be stored or distributed on a computer-readable medium, which includes magnetic or optically readable computer disks, hardware-wired or pre-programmed chips (e.g., EEPROM semiconductor chips), nanotechnology memories, biological memories, or other data storage media. Alternatively, computer-implemented instructions, data structures, screen displays, and other data according to aspects of the present invention may be distributed over a propagation medium (e.g., (one or more) electromagnetic waves, sound waves, etc.) on a propagation signal over the Internet or other networks (including wireless networks) over a period of time, or they may be provided on any analog or digital network (packet-switching, circuit-switching, or other schemes). Those skilled in the relevant art will recognize that portions of the present invention reside on a server computer, while corresponding portions reside on a client computer such as a mobile device or a portable device. Thus, although certain hardware platforms are described herein, aspects of the present invention are equally applicable to nodes on a network.
[0101] Based on the above detailed embodiments, these and other changes can be made to the present invention. Although the above description details certain embodiments of the present invention and describes the expected best mode, no matter how detailed the above is shown in the text, the present invention can be practiced in many ways. The details of the present invention can vary greatly in its implementation details while still being covered by the invention disclosed herein. As described above, the specific terms used when describing certain features or aspects of the present invention should not be considered to imply that the term is redefined herein to be limited to any specific property, feature, or aspect of the present invention associated with that term. Generally, unless the terms are clearly defined in the above detailed embodiments section, such terms used in the appended claims should not be construed to limit the present invention to the specific embodiments disclosed in the specification. Therefore, the actual scope of the present invention not only covers the disclosed embodiments, but also covers all equivalent ways of practicing or implementing the present invention.
Claims
1. A food storage system, comprising: at least one processor; and at least one non-transitory processor-readable medium storing processor-executable instructions which, when executed by the at least one processor, cause the at least one processor to: receive, from a user device, data uniquely identifying the storage bag, wherein the storage bag has a label printed or attached to its outer surface; receive, from the user device, data regarding the contents of the storage bag; associate the data regarding the storage bag with the data regarding the contents of the storage bag; receive image data indicating the label of the storage bag containing the contents; process the image data to determine the position or distortion of the label in three-dimensional (3D) space; determine one or more characteristics of the contents at least in part based on the position or distortion of the label; generate a notification based on the data regarding the contents of the storage bag and the one or more characteristics; and send the notification to the user device.
2. The food storage system according to claim 1, wherein the at least one processor is further configured to generate the notification to indicate: a cooking recommendation based on the data regarding the contents of the storage bag; and / or an eating recommendation based on the data regarding the contents of the storage bag.
3. The food storage system according to claim 1, wherein the data regarding the storage bag indicates a machine-readable identifier printed on or attached to the storage bag, and the machine-readable identifier is a unique identifier.
4. The food storage system according to claim 1, wherein the at least one processor is configured to: receive, from the user device, a query request for data uniquely identifying the storage bag in which the contents are stored; query a data repository using the data uniquely identifying the storage bag to determine the contents of the storage container; and send, to the user device, data indicating the contents of the storage container.
5. The food storage system according to claim 1, wherein the at least one processor is configured to: receive, from the user device, input data regarding the contents of the storage bag; determine a notification sending date for sending the notification to the user device based on the input data; and schedule delivery of the notification to the user device on the notification sending date.
6. The food storage system according to claim 5, wherein the at least one processor is configured to receive temperature data indicating the storage temperature of the storage bag from a computing device of a storage device in which the storage bag is stored, and the at least one processor further determines the notification sending date based on the temperature data.
7. The food storage system according to claim 6, wherein the storage device is a refrigerator or a freezer.
8. The food storage system according to claim 5, wherein the at least one processor is configured to: Further determine a notification sending date based on one or more characteristics of the contents of the storage bag.
9. The food storage system according to claim 8, wherein the at least one processor is configured to process the image data using a computer vision algorithm.
10. The food storage system according to claim 9, wherein the one or more characteristics include at least one of weight, thickness, volume, shape, and / or food type, and the contents of the storage bag are one or more food products.
11. A method for operating a food storage system, the method comprising: Receiving, from a user device, data that uniquely identifies a storage bag, wherein the storage bag has a label printed or attached to its outer surface; Receiving, from the user device, data regarding the contents of the storage bag; Associating the data regarding the storage bag with the data regarding the contents of the storage bag; Receiving image data indicative of the label of the storage bag containing the contents; Processing the image data to determine the position or distortion of the label in three-dimensional (3D) space; Determining, at least in part, one or more characteristics of the contents based on the position or distortion of the label; Generating a notification based on the data regarding the contents of the storage bag and the one or more characteristics; And Sending the notification to the user device.
12. The method according to claim 11, wherein the method further comprises generating the notification to indicate: Cooking recommendations based on the data regarding the contents of the storage bag; and / or Consumption recommendations based on the data regarding the contents of the storage bag.
13. The method according to claim 11, wherein the data regarding the storage bag indicates a machine-readable identifier printed on or attached to the storage bag, and the machine-readable identifier is a unique identifier.
14. The method according to claim 12, wherein the method further comprises: Receiving, from the user device, a query request indicative of data that uniquely identifies the storage bag in which the contents are stored; Querying a data repository using the data that uniquely identifies the storage bag to determine the contents of the storage container; And Sending, to the user device, data indicative of the contents of the storage container.
15. The method according to claim 11, wherein the method further comprises: Receiving, from the user device, input data regarding the contents of the storage bag; Determining, based on the input data, a notification sending date for sending the notification to the user device; And Scheduling delivery of the notification to the user device on the notification sending date.
16. The method according to claim 15, wherein the method further comprises receiving, from a computing device of a storage device in which the storage bag is stored, temperature data indicative of the storage temperature of the storage bag, and the at least one processor further determines the notification sending date based on the temperature data.
17. The method according to claim 16, wherein the storage device is a refrigerator or a freezer.
18. The method according to claim 15, wherein the method further comprises: further determining a notification sending date based on one or more characteristics of the contents of the storage bag.
19. The method according to claim 18, wherein the method further comprises processing the image data using a computer vision algorithm.
20. The method according to claim 19, wherein the one or more characteristics comprise at least one of weight, thickness, volume, shape, and / or food type, wherein the contents of the storage bag are one or more food items.
21. One or more non-transitory processor-readable media storing processor-executable instructions that, when executed by the at least one processor, configure the at least one processor to: receive, from a user device, data uniquely identifying a storage bag, wherein the storage bag has a label printed or attached to its outer surface; receive, from the user device, data regarding the contents of the storage bag; associate the data regarding the storage bag with the data regarding the contents of the storage bag; receive image data indicating the label of the storage bag containing the contents; process the image data to determine the position or distortion of the label in three-dimensional (3D) space; determine one or more characteristics of the contents at least in part based on the position or distortion of the label; generate a notification based on the data regarding the contents of the storage bag and the one or more characteristics; and send the notification to the user device.
22. The one or more non-transitory processor-readable media according to claim 21, which comprise additional executable instructions that cause the at least one processor to generate the notification to indicate: a cooking recommendation based on the data regarding the contents of the storage bag; and / or an eating recommendation based on the data regarding the contents of the storage bag.
23. The one or more non-transitory processor-readable media according to claim 21, wherein the data regarding the storage bag indicates a machine-readable identifier printed on or attached to the storage bag, wherein the machine-readable identifier is a unique identifier.
24. The one or more non-transitory processor-readable media according to claim 21, which comprise additional executable instructions that cause the at least one processor to: receive, from the user device, a query request indicating data uniquely identifying the storage bag in which the contents are stored; query a data repository using the data uniquely identifying the storage bag to determine the contents of the storage container; and send, to the user device, data indicating the contents of the storage container.
25. One or more non-transitory processor-readable media according to claim 21, comprising additional executable instructions that cause the at least one processor to: Receive input data regarding the contents of the storage bag from the user device; Determine a notification sending date for sending the notification to the user device based on the input data; and Schedule delivery of the notification to the user device on the notification sending date.
26. One or more non-transitory processor-readable media according to claim 25, comprising additional executable instructions that cause the at least one processor to receive temperature data indicating a storage temperature of the storage bag from a computing device of a storage device in which the storage bag is stored, wherein the at least one processor further determines a notification sending date based on the temperature data.
27. One or more non-transitory processor-readable media according to claim 26, wherein the storage device is a refrigerator or a freezer.
28. One or more non-transitory processor-readable media according to claim 25, wherein the one or more non-transitory processor-readable media comprise additional executable instructions that configure the at least one processor to: Further determine a notification sending date based on one or more characteristics of the contents of the storage bag.
29. One or more non-transitory processor-readable media according to claim 28, wherein the one or more non-transitory processor-readable media comprise additional executable instructions that configure the at least one processor to be configured to process the image data using a computer vision algorithm.
30. One or more non-transitory processor-readable media according to claim 29, wherein the one or more characteristics include at least one of weight, thickness, volume, shape, and food type, wherein the contents of the storage bag are one or more food items.
Citation Information
Patent Citations
Fresh-keeping control method, system and refrigerator for food in refrigerator based on mobile terminal
CN103322767A
Connected food preparation system and method of use
CN107535024A
Informative systems and methods for perishable food storage
US9436771B1
Improvements in food storage
WO2017059480A1