Techniques for machine vision estimation of food container fill and nutrient content

Abstract

Techniques are disclosed for techniques for estimating the filling amount of a target food container. The image may capture at least a portion of the food container and any food in the food content space of the food container. The image may be segmented to provide a first food parameter and a first aspect parameter corresponding to the image. The first aspect parameter may be input into a machine learning model. A first angle reference corresponding to the image may be determined from a machine learning model based on the first aspect parameter. The first food parameter and the first angle reference may be input into a machine learning model. An estimated weight reference may be determined from a machine learning model based on the first food parameter and the first angle reference. The estimated weight reference may correspond to a fill amount of the food container.

Description

[0001] Cross-references to related applications

[0002] This application claims the benefit of U.S. Provisional Patent Application No. 63 / 589,184, filed October 10, 2023, the entire disclosure of which is incorporated herein by reference for all purposes. Background Technology

[0003] Objects (such as animals) can obtain nutrients (such as food and / or water) from various dispensers and / or food containers (such as pet food and water bowls). Pet owners may fill such pet bowls to various levels and / or at various times based on the pet's specific needs and / or habits. Pets may consume food from pet food bowls because the pet needs supplements and / or because the pet owner measures the pet's food, for example, for a pet on a certain type of diet.

[0004] For a variety of reasons, pets may be fed / consume the same food or different types of food. For example, for health reasons, pets may be fed specially prepared food. Or pet owners may believe that, over time, their pets may prefer one type of food over another. For these and other reasons, pets may be fed wet and / or dry food. And pets may be fed chunky, pellet, and / or mashed types of food. Summary of the Invention

[0005] One or more computer-implemented techniques / methods (and devices and / or systems for implementing these techniques / methods) for estimating the filling volume of a food container are disclosed.

[0006] The technique may include receiving at least a first image for a first food container, the first image capturing at least a portion of the food container and any food in the food content space of the food container. Segmentation may be performed on the first image. The segmentation may provide a first food parameter and a first aspect parameter corresponding to the first image.

[0007] The technique may include inputting first cross-sectional parameters into a machine learning model (e.g., and / or an algorithm, equation, and / or model). The technique may include determining a first angular reference corresponding to a first image from the machine learning model, at least in part, based on the first cross-sectional parameters. The first cross-sectional parameters and the first angular reference may be input into the machine learning model.

[0008] The technology may include determining, at least in part, an estimated weight reference and / or an estimated volume reference corresponding to the filling amount of the food container from a machine learning model, based on a first food parameter and a first angle reference.

[0009] In one or more scenarios, the technology may include determining, at least in part, a minimum estimated weight reference and / or a minimum estimated volume reference corresponding to a food container that is at least substantially full, a time period corresponding to when the food container is at least substantially full, a time period corresponding to when the food container is substantially empty, and / or the feeding rate of the object, and the frequency of object feeding and / or timing (e.g., different from the time rate), based on an estimated weight reference and / or an estimated volume reference. For example, an object may eat more in the morning and / or at specific times (such as 8 a.m., 10 a.m., etc.) and may eat less in the afternoon and / or at specific times (such as 2 p.m., 5 p.m., etc.).

[0010] In one or more contexts, segmentation may include at least food container segmentation and food segmentation.

[0011] In one or more scenarios, the first image may be provided by a digital camera and / or a machine vision camera.

[0012] In one or more scenarios, segmentation can produce a plurality of first pixels corresponding to a food container corresponding to a first image and / or a plurality of second pixels corresponding to food in the food content space of a food container corresponding to a first image.

[0013] In one or more scenarios, the first food parameter can be the ratio of the number of second pixels to the sum of the number of first pixels and the number of second pixels.

[0014] In one or more scenarios, segmentation produces a rotated bounding box width and height corresponding to a first image. The first x and y parameters can be the ratio of the rotated bounding box width to the rotated bounding box height. In one or more scenarios, segmentation can utilize one or more morphological geodesic active contours (MGACs), among other algorithms and / or techniques. For example, image processing (e.g., active contours), machine learning, and / or deep learning algorithms / models (e.g., convolutional neural networks (CNNs), etc.) can be used.

[0015] In one or more contexts, the technique may include determining the nutritional content of a food corresponding to an estimated weight reference and / or an estimated volume reference, at least in part, based on an estimated weight reference and / or an estimated volume reference.

[0016] In one or more contexts, the object may be a canine and / or feline. In one or more contexts, the food container may be a pet food bowl.

[0017] Techniques / methods implemented on one or more computers are disclosed for capturing machine learning training data to estimate the filling volume of an object food container (as well as devices and / or systems for implementing these techniques / methods).

[0018] The technology may include feeding a first type of food container into at least one processing device, feeding a first type of floor on which the food container is disposed into at least one processing device, feeding a first type of food into at least one processing device, and / or feeding a first predetermined amount of food disposed in the food content space of the food container into at least one processing device.

[0019] The technique may include placing a camera device in a first position relative to the food container within a vertical plane that cuts through at least a portion of the food container. The camera device may be positioned at the first position at a first angle relative to the food container. A first image of the food container can be captured from the first position.

[0020] This can be used to place the camera device in a second, third, fourth, and / or fifth position, repeating one or more of the prior art. The camera device can be arranged to capture a second image at a second angle, a third image at a third angle, a fourth image at a fourth angle, and / or a fifth image at a fifth angle.

[0021] One or more of the prior art can be repeated for at least a second predetermined filling amount and / or a third predetermined filling amount.

[0022] The technology may include storing a first image, a second image, a third image, a fourth image, and / or a fifth image into a machine learning training database.

[0023] The technique may include training at least one machine learning model using at least a first image, a second image, a third image, a fourth image, and / or a fifth image.

[0024] In one or more contexts, the type of food container may include the shape of the food container, the size of the food container, and / or the material of the food container.

[0025] In one or more contexts, the first type of flooring may include bare flooring, tile flooring, and / or carpet flooring.

[0026] In one or more contexts, the first type of food may include chunks, granules, mashed, wet, and / or dry.

[0027] In one or more contexts, the camera device may be a digital camera and / or a machine vision camera.

[0028] In one or more scenarios, one or more of the prior art can be repeated for a second predetermined filling amount, a third predetermined filling amount, a fourth predetermined filling amount, and / or a fifth predetermined filling amount.

[0029] In one or more contexts, the technology may include storing a first image, a second image, a third image, a fourth image, and / or a fifth image together with indications of a first type of food container, a first type of floor, and / or a first type of food.

[0030] In one or more scenarios, a first angle may be defined by the angular displacement of a first position relative to a horizontal plane that cuts through at least a portion of the food container. A second angle may be defined by the angular displacement of a second position relative to a horizontal plane. A third angle may be defined by the angular displacement of a third position relative to a horizontal plane. A fourth angle may be defined by the angular displacement of a fourth position relative to a horizontal plane. A fifth angle may be defined by the angular displacement of a fifth position relative to a horizontal plane.

[0031] In one or more scenarios, the first angle can be approximately (e.g., within 5% to 15%) ninety degrees. The second angle can be approximately (e.g., within 5% to 15%) sixty-five degrees. The third angle can be approximately (e.g., within 5% to 15%) forty-five degrees. The fourth angle can be approximately (e.g., within 5% to 15%) twenty-five degrees. The fifth angle can be approximately (e.g., within 5% to 15%) ten degrees.

[0032] In one or more scenarios, the technology may include feeding a second type of food container into at least one processing device. One or more of the prior technologies may be repeated for the second type of food container.

[0033] In one or more scenarios, the technology may include inputting a second type of floor on which food containers are placed into at least one processing device. One or more of the prior technologies may be repeated for the second type of floor on which food containers are placed.

[0034] In one or more scenarios, the technology may include feeding a second type of food into at least one processing device. One or more scenarios may be repeated for the second type of food. Attached Figure Description

[0035] The elements contained herein, along with other features, advantages, and disclosures, and ways of implementing them, will become apparent and will be better understood by referring to the following description of various examples of the present disclosure taken in conjunction with the accompanying drawings:

[0036] Figure 1 This is a block diagram illustrating an example object nutrient distribution monitoring communication network, which is operable to control one or more parts of an object nutrient distribution monitoring system via one or more devices, such as an object nutrient distribution monitoring control device (SNDMCD) and other devices.

[0037] Figure 2 This is an example illustration of a feline subject and two food containers with different filling amounts in the food content space of the two food containers.

[0038] Figure 3A and Figure 3B This is an example flowchart of at least one technique for estimating the filling volume of food containers in an object nutrient distribution monitoring system.

[0039] Figure 4 This is a block diagram of the hardware configuration of an example device that can control one or more parts of an object nutrient distribution monitoring system / communication network, such as... Figure 1 SNDMCD device.

[0040] Figure 5 These are illustrative examples of test canines and food containers with varying filling volumes in the food content space over time.

[0041] Figure 6 This is an example image of a food container set on the floor, where the food container's food contents space is mostly empty.

[0042] Figure 7 This is an example diagram of a food container set on the floor, where the food container's food content space has at least some filling.

[0043] Figure 8 This is an example illustration of an arrangement that captures training data for one or more machine learning models to estimate the filling volume of an object food container.

[0044] Figure 9 This is an example illustration depicting the correlation between the pixel ratio of a food container image and a weight reference value as a function of the image's angle reference.

[0045] Figure 10 This is an example illustration depicting the relationship between the aspect ratio of a food container image and the image's angular reference.

[0046] Figure 11 These are example illustrations of three types of food that may take up space within a food container.

[0047] Figure 12 This is an example illustration of segmenting an image of a food container into a food segment image and a food container segment image.

[0048] Figure 13 This is an example illustration showing the color segmentation of an image of a food container into hue, saturation, and value space segments.

[0049] Figure 14This is an example of the property that describes the correlation between the elliptic eccentricity of an image of a food container and an image angular reference.

[0050] Figure 15 This is an example illustration of a feature that depicts the correlation between the ratio of the inner wall height to the edge height of an image of a food container and a weight reference value as a function of the image's angle reference.

[0051] Figure 16 This is an example illustration of a feature example, depicting the correlation between the ratio of multiple food pixels to elliptical region pixels and the ratio of the eccentricity of the food container image and the weight reference value as a function of the image angle reference.

[0052] Figure 17 This is an example illustration of a feature example, depicting the correlation between the ratio of multiple food pixels to elliptical region pixels in a food container image and a weight reference value as a function of the image angle reference.

[0053] Figure 18 This is an example diagram of a food container that can be used with one or more estimation techniques described herein. Detailed Implementation

[0054] For the purpose of promoting an understanding of the principles of this disclosure, reference will now be made to the examples shown in the accompanying drawings, and these examples will be described using specific language. However, it will be understood that this is not intended to limit the scope of this disclosure in any way.

[0055] Figure 1 This is a block diagram illustrating an example Object Nutrition Distribution Monitoring System Network (SNDMSN) 100 operable to monitor and / or control one or more parts of a Nutrition Distribution Monitoring System (NDMS). One or more of the following—digital and / or analog control signals, electronic content, various input signals and / or various output signals, and other condition monitoring system information—can be transmitted from / across / between the Object Nutrition Distribution Monitoring System Network 100. One or more of the following—discrete and / or continuous control schemes, techniques, and / or algorithms—can be processed / executed by / across / from the Object Nutrition Distribution Monitoring System Network 100.

[0056] Electronic content can include media content, electronic documents, device-to-device communication, streaming media content, digital image still frames, digital streaming video, internet / cloud-based, edge-based electronic applications / services / databases, electronic communications / services (e.g., video / audio conferencing), internet-based electronic services, virtual reality content and / or services, augmented reality content and / or services, media captioning content and / or services, e-commerce, video components / elements of electronic content and / or audio components / elements of electronic content, and other types of electronic content.

[0057] In one or more scenarios, SNDMSN devices 110a-110d transmit / receive signals and / or communicate and / or receive data services from WAN 120 via a connection to an Object Nutrient Distribution Monitoring Network (SNDMN) 130. One or more nodes of the Object Nutrient Distribution Monitoring Network 130 and / or WAN 120 can communicate with one or more cloud-based nodes (not shown) via the Internet 124.

[0058] The SNDMSN device may include, for example, a modem 110a, a process control device / logic controller 110b, a wireless router including an embedded modem 110c, or a media gateway 110d, as well as many other devices (such as digital subscriber line (DSL) modems, VoIP terminal adapters, video game consoles, digital multifunction disc (DVD) players, communication equipment, hotspot devices, etc.). For example, the object nutrient distribution monitoring network 130 may be a hybrid fiber-coaxial (HFC) network, a local area network (LAN), a wireless local area network (WLAN), a cellular network, and / or a personal area network (PAN), as well as other networks. As used herein, for example, the object nutrient distribution monitoring and control device (SNDMCD) can be any of devices 110a-110d and / or 140a-140i, an internet gateway, a router device, a set-top box (STB), a process control device / logic controller, a smart media device (SMD), a cloud computing device, any type of SNDMCD, and / or any other suitable device (e.g., wired and / or wireless device) that can be configured to perform one or more of the technologies and / or functions disclosed herein.

[0059] The SNDMCD device facilitates communication between WAN 120 and devices 140a-140i. A cable modem or embedded MTA (eMTA) 110a facilitates communication between WAN 120 and computer 140a. A process control device / logic controller 110b facilitates communication between WAN 120 and television / monitor / display 140b (e.g., media presentation device, graphical user interface, process control interface, etc.) and / or digital video recorder (DVR). A wireless router 110c facilitates communication between computer 140c and WAN 120.

[0060] Media gateway 110d facilitates communication between mobile devices 140d (e.g., tablet computing devices, smartphones, personal digital assistant (PDA) devices, laptop computing devices, etc.; one or more PC-based, iOS-based, Linux-based, and / or Android-based devices, etc.) and WAN 120. One or more speaker devices (e.g., sound radiation devices / systems) 140e can communicate with the object nutrient distribution monitoring network 130, process control devices / logic controllers 110b, and / or televisions / monitors / displays 140b, etc. Camera devices 140g, 140h, and / or 140i can communicate with, for example, computers 140a, televisions / monitors / displays 140b, computers 140c, and / or the object nutrient distribution monitoring network 130, as well as other devices and networks.

[0061] One or more speaker devices 140e (e.g., surround sound speakers, home theater speakers, other external wired / wireless speakers, amplifiers, full-range drivers, subwoofer drivers, low-frequency drivers, mid-frequency drivers, high-frequency drivers, coaxial drivers, etc.) can broadcast at least the audio component of electronic content / media content, as well as other audio signals, processes, and / or applications. One or more speaker devices 140e may have the ability to radiate sound in a pre-configured acoustic / physical pattern (e.g., cone pattern, directional pattern, etc.). For example, audible alarms for condition monitoring of process control equipment / logic controllers can be communicated via one or more speaker devices 140e.

[0062] One or more microphone devices 140f may be external / standalone microphone devices. One or more microphone devices 140f may communicate with the object nutrient distribution monitoring network 130, process control equipment / logic controller 110b, television / monitor / display 140b, computer 140a, computer 140c, mobile device 140a, etc. Any of devices 110a-110d and / or devices 140a-140i may include internal microphone devices. One or more speaker devices 140e (e.g., “speakers”) and / or one or more microphone devices 140f (e.g., “microphones”), which may be “high-quality” devices such as far-field microphones, noise-canceling microphones, shotgun microphones, dynamic microphones, ribbon microphones, and / or diaphragm microphones of various sizes, Bluetooth-based... TM The remote / control device (RF4CE-based remote / control device, etc.) may have wired and / or wireless connections (e.g., Bluetooth, Wi-Fi, proprietary protocol communication network, etc.) to any of the following: other devices 140a-140i, object nutrient distribution monitoring network 130, WAN 120 and / or Internet 124.

[0063] Camera devices 140g-140i can provide digital video input / output capabilities for one or more of devices 110a-110d and / or devices 140a-140d. Camera devices 140g-140i can communicate with any of devices 110a-110d and / or devices 140a-140f, possibly via wired and / or wireless connections. One or more of camera devices 140g-140i can capture digital images, digital video streams, and / or can scan various types of images, such as Universal Product Code (UPC) codes and / or Quick Response (QR) codes, and other images. One or more of camera devices 140g-140i can provide video input / output for, for example, video surveillance and other video functions (e.g., acting as a webcam or the like).

[0064] Any of the camera devices 140g-140i may include a microphone device and / or a speaker device. The inputs / outputs of any of the camera devices 140g-140i may include audio signals / packets / components, which may be, for example, separate / separable from the video signals / packets / components of any of the camera devices 140g-140i, or combined with video signals / packets / components in some (e.g., separable) combinations.

[0065] One or more of the camera devices 140g-140i can detect the presence of one or more objects and / or things (e.g., food containers) that may be near the camera device 140g-140i and / or located in the same general space (e.g., same room, same space, same room, same defined area, etc.) as the camera device 140g-140i. One or more of the camera devices 140g-140i can measure the general activity level (e.g., high activity, medium activity, and / or low activity) of one or more objects that can be detected by the camera device 140g-140i. One or more of the camera devices 140g-140i can detect one or more general characteristics (e.g., height, body shape, skin color, pulse, heart rate, respiratory count, object size, object volume, etc.) of one or more objects and / or things detected by the camera device 140g-140i. For example, one or more of the camera devices 140g-140i can be configured to identify one or more specific objects and / or things. One or more of the camera devices 140g-140i can be configured to detect the attention / gaze of one object toward another (e.g., to detect an object and / or a thing, which may correspond to the attention / gaze of one object toward another).

[0066] One of the camera devices 140g-140i or any of the devices 110a-110d and / or 140a-140d can use wireless communication, such as Bluetooth. TM and / or Wi-Fi TM And other wireless communication protocols. One or more of the camera devices 140g-140i may be located outside of devices 110a-110d and / or any of devices 140a-140d. One or more of the camera devices 140g-140i may be located inside devices 110a-110d and / or any of devices 140a-140d.

[0067] One or more of the camera devices 140g-140i can be industrial vision camera devices. The vision camera can be a gigabit Ethernet compatible device (e.g., 10 GB Ethernet or similar). The vision camera can operate in black and white and / or color modes. The vision camera can have at least 8.8 megapixels (8.8 megapixels) or similar capacity. The vision camera can have a resolution of 4096 × 2160 pixels or similar. For example, the vision camera can be (e.g., manufactured by Baumer, such as the VLXT-90C.I LX series or similar / equivalent devices as mentioned herein) capable of capturing product images in various forms, such as digital still image frames and / or video streams, possibly from, for example, ninety-five (95) frames per second (fps). The vision camera can have one or more parameters that can be remotely and / or locally configured. Camera devices 140g-140i may include color / RGB, black and white / grayscale, infrared and / or devices capable of measuring the invisible spectrum, such as multispectral and / or hyperspectral and / or depth / 3D cameras.

[0068] SNDMCD devices (such as process control devices / logic controller devices, media gateway devices, and others) can support visual and / or voice interfaces with users, viewers, and / or operators of the object nutrient distribution monitoring network 130. This interface can support intelligent enhancements to the user / viewer / operator experience, for example, within the object nutrient distribution monitoring network environment or in any network environment. One or more traditional and / or current viewer experiences can be enriched to utilize the visual and / or voice interfaces, possibly, for example, to derive intelligent actions and / or results.

[0069] In one or more contexts, any of the devices 110a-110d, 140a-140i, and other devices may be used to implement the capabilities, techniques, methods, and / or any of the devices described herein.

[0070] The WAN network 120 and / or the object nutrient distribution monitoring network 130 can be implemented as any type of wired and / or wireless network, including a local area network (LAN), a wide area network (WAN), a global network (Internet), etc. Therefore, the WAN network 120 and / or the object nutrient distribution monitoring network 130 may include one or more communication-coupled network computing devices (not shown) for facilitating the flow and / or processing of network communication traffic via a series of wired and / or wireless interconnections. Such network computing devices may include, but are not limited to, one or more access points, routers, switches, servers, computing devices, and / or storage devices.

[0071] Without the capabilities, techniques, methods, systems, and / or devices described herein, those skilled in the art would not understand how to estimate the fill volume in a food container. This disclosure provides those skilled in the art with the capability, system, device, method, and / or technique to estimate the fill volume of the food content space of a food container based on one or more images of the food container. This disclosure also provides those skilled in the art with the capability, system, device, method, and / or technique to train one or more machine learning models for estimating the fill volume of the food content space of a food container based on one or more images of the food container. Such capabilities, systems, devices, methods, and / or techniques can be used for such purposes, as well as other purposes, such as providing a correlation between the estimated fill volume of the food content space of the food container and the nutritional content of the food.

[0072] Pet parents may be unable and / or unwilling to measure or weigh their pet's food and / or water. For example, the techniques described in this article can make it easier and / or more likely to measure and / or weigh pet's food and / or water by using imaging / range sensors instead of scales.

[0073] The nutritional content of pet food can be difficult to determine, perhaps because the amount and / or type of food dispensed into the bowl and / or consumed may be unknown and / or difficult to quantify. The techniques described herein can measure and / or infer nutritional content from one or more imaging sensors, potentially determining nutritional content, for example, rather than relying on laboratory tests.

[0074] Figure 2 This is an example illustration of a feline animal and two food containers with different fill levels in the food content spaces of the two food containers. Figure 2In this context, an object (e.g., an animal, such as a pet dog / canine and / or cat / feline) (such as a cat 204) can be fed from a first food container (e.g., a pet food bowl) 206 and / or a second food container 208. For various reasons, and at various time periods, pet owners may wish to estimate the fill level of the first food container 206 and / or the second food container 208 via indirect techniques such as based on one or more images of the first food container 206 and / or the second food container 208.

[0075] Figure 5 This is an example illustration of a canine animal 504 and a food container 506 having different fill levels in the food content space over time. For various reasons, and at various time periods, pet owners may wish to estimate the fill level of the food container 506 via indirect techniques such as from one or more images of the food container 506.

[0076] Figure 6 This is an example illustration of a food container 604 set on the floor 606, wherein the food content space 608 of the food container 604 is substantially empty.

[0077] Figure 7 This is an example illustration of a food container 704 set on a floor 706, wherein the food content space 708 of the food container 704 has at least some filling amount.

[0078] Figure 8 This is an example illustration of an arrangement 802 used to capture training data for one or more machine learning models to estimate the filling amount of a food container 804 for an object.

[0079] about Figure 8 The desired angle described can be used for food containers 804 of various shapes, such as basically circular food containers. For example, any of camera devices 806, 808, 810, 812 and / or 814 can be configured for wired and / or wireless transmission via object nutrient distribution monitoring communication network 130.

[0080] Techniques / methods implemented on one or more computers are disclosed for capturing machine learning training data to estimate the filling volume of an object food container (as well as devices and / or systems for implementing these techniques / methods).

[0081] The technology may include feeding a first type of food container into at least one processing device, feeding a first type of floor on which the food container is disposed into at least one processing device, feeding a first type of food into at least one processing device, and / or feeding a first predetermined amount of food disposed in the food content space of the food container into at least one processing device.

[0082] refer to Figure 8 The technique may include placing a camera device 806 in a first position relative to the food container 804 within a vertical plane (not shown) that cuts through at least a portion of the food container. The camera device 806 may be arranged at the first position at a first angle relative to the food container. A first image of the food container can be captured from the first position.

[0083] The camera devices can be positioned in a second position (e.g., camera device 808), a third position (e.g., camera device 810), a fourth position (e.g., camera device 812), and / or a fifth position (e.g., camera device 814) to repeat one or more of the prior art. Camera devices 806, 810, 812, and / or 814 can be arranged to capture a second image at a second angle, a third image at a third angle, a fourth image at a fourth angle, and / or a fifth image at a fifth angle.

[0084] In one or more scenarios, the food container 804 may be shaken between picture / image captures or during video capture (e.g., thus ensuring that the food container does not move for a period of time after shaking) to disturb the food in the food container 804.

[0085] One or more of the prior art can be repeated for at least a second predetermined filling amount and / or a third predetermined filling amount. Various methods exist for determining the filling amount of the container. For example, the container can be determined to have a full capacity of 60 ml or 35 ml. For example, the container can be determined to have 150^3 grams of feed. For example, the container can be determined to have 172 grams of feed.

[0086] The technology may include storing a first image, a second image, a third image, a fourth image, and / or a fifth image into a machine learning training database.

[0087] The technique may include training at least one machine learning model using at least a first image, a second image, a third image, a fourth image, and / or a fifth image.

[0088] In one or more contexts, the type of food container may include the shape of the food container, the size of the food container, and / or the material of the food container.

[0089] In one or more contexts, the first type of flooring may include bare flooring, tile flooring, and / or carpet flooring.

[0090] In one or more contexts, the first type of food may include chunks, granules, mashed, wet, and / or dry.

[0091] In one or more contexts, the camera device may be a digital camera and / or a machine vision camera.

[0092] In one or more scenarios, one or more of the prior art can be repeated for a second predetermined filling amount, a third predetermined filling amount, a fourth predetermined filling amount, and / or a fifth predetermined filling amount.

[0093] In one or more contexts, the technology may include storing a first image, a second image, a third image, a fourth image, and / or a fifth image together with indications of a first type of food container, a first type of floor, and / or a first type of food.

[0094] In one or more contexts, refer to Figure 8 The first angle can be defined by the angular displacement of a first position (e.g., camera device 806) relative to a horizontal plane (not shown) that cuts through at least a portion of the food container. The second angle can be defined by the angular displacement of a second position (e.g., camera device 808) relative to the horizontal plane. The third angle can be defined by the angular displacement of a third position (e.g., camera device 810) relative to the horizontal plane. The fourth angle can be defined by the angular displacement of a fourth position (e.g., camera device 812) relative to the horizontal plane. The fifth angle can be defined by the angular displacement of a fifth position (e.g., camera device 814) relative to the horizontal plane.

[0095] In one or more contexts, refer to Figure 8 The first angle can be approximately (e.g., within 5% to 15%) ninety degrees (e.g., camera device 806). The second angle can be approximately (e.g., within 5% to 15%) sixty-five degrees (e.g., camera device 808). The third angle can be approximately (e.g., within 5% to 15%) forty-five degrees (e.g., camera device 810). The fourth angle can be approximately (e.g., within 5% to 15%) twenty-five degrees (e.g., camera device 812). The fifth angle can be approximately (e.g., within 5% to 15%) ten degrees (e.g., camera device 814).

[0096] In one or more scenarios, the technology may include feeding a second type of food container into at least one processing device. One or more of the prior technologies may be repeated for the second type of food container.

[0097] In one or more scenarios, the technology may include inputting a second type of floor on which food containers are placed into at least one processing device. One or more of the prior technologies may be repeated for the second type of floor on which food containers are placed.

[0098] In one or more scenarios, the technology may include feeding a second type of food into at least one processing device. One or more scenarios may be repeated for the second type of food.

[0099] In one or more scenarios, one or more models of empty containers (e.g., food or water containers, bowls, plates, etc.) can be obtained. Containers can have many different shapes and sizes. The shape / dimension of the inner surface can be useful for measuring the contents of the container. For example, Figure 18 The image shown is an example container with a non-concave inner surface, which is not visible when full. To accurately measure the container, the shape / size of the inner surface can be obtained so that it can be used later to estimate the filling content.

[0100] Figure 18 The examples also illustrate one or more other problems that can be solved by one or more of the techniques described in this article. Figure 18 The container has translucent walls, which may affect the measurement of feed type. This could be because the translucent material may alter the feed (e.g., change its color, brightness, etc.), and / or the measurement of feed level may be inaccurate due to refraction. Obtaining a model of the complete bowl may be useful.

[0101] One or more techniques can be used to obtain the shape / size of an empty bowl (e.g., any empty container). For measurement, a 3D model can be constructed from one or more measurements using ranging sensors (such as depth camera sensors, LiDAR, etc.), among other reasons. The output of the measurement can be a distance (e.g., centimeters, inches) expressing the shape / size of the container under an absolutely true measurement. One or more measurements can help capture at least some or all of the container's perspectives. In one or more scenarios, the container can be rotated (e.g., on a turntable) while one or more sensor measurements are taken. In one or more scenarios, for example, a machine learning model can be developed that infers depth measurements from images and / or can convert pixel measurements into distance measurements.

[0102] For example, for one or more measurement techniques, it may be useful to stitch together multiple sensor measurements from one or more different views of a container into a single 3D model.

[0103] For one or more measurement techniques, it can be useful to use one or more segmentation algorithms / models to separate the data points (e.g., pixels, distance measurements, etc.) of a container from one or more or all other objects in the image and / or setting.

[0104] In one or more scenarios, the manufacturer can provide the shape / size. For example, this shape / measurement can take the form of design drawings, CAD drawings, 3D printer designs and / or 3D models.

[0105] In one or more scenarios, the user of the measurement system described herein can provide inputs such as the width, height, length, and / or curvature properties of the inner / outer container surface.

[0106] In one or more scenarios, one or more outlines / categories of bowls can be selected. For example, a user can select one or more inputs (e.g., via a user interface) that indicate whether the bottom (e.g., the inner surface) of the bowl is flat or curved, and / or that the container corresponds to a bowl of a specific color.

[0107] In one or more scenarios, one or more (e.g., known) bowl models can be selected. For example, a user can declare / input a bowl of a specific manufacturer's model that is being used. For example, a user can select either the "Acme ultraMkII" bowl or the "Nitro-Eco" feeding bowl.

[0108] In one or more contexts, the data / model describing the bowl can be a 3D model and / or a 3D dense model. The data / model can describe one or more (e.g., critical) dimensions that can be useful in other processing stages, such as width, height, length, and / or internal curvature. These can be provided by the user, prior knowledge, and / or inferred from the 3D dense model. The data / model can describe the container's appearance, such as color, translucency, etc. Data / model information can be stored in a database, such as a local and / or cloud-based database.

[0109] One or more technologies may include the measurement of containers containing food and / or water. The user / operator can obtain sensor measurements of the containers containing food / liquid from imaging and / or ranging sensors.

[0110] One or more techniques may include the measurement of the contents of various containers at one or more fill levels, which may include substantially full containers, substantially empty containers, and one or more different fill levels.

[0111] In one or more scenarios, measuring the filling of a container can describe the feed surface that can be observed in sensor measurements. For example, if the feed can be observed in an image, then it can be seen how far / near the top of the container the feed is. One or more, or a combination of, sensor measurements can be useful, perhaps, for example, to account for viewing angles and / or to improve the accuracy of filling measurements.

[0112] One or more techniques may include measuring the volume of a container with contents. This can be somewhat similar to measuring a substantially empty container / bowl. Container volume measurement can be performed using a camera and / or a depth sensor (e.g., a single depth camera device, a device integrated into a modern mobile phone, etc.). Measuring the volume difference between 3D measurements of an empty container and a filled container can be useful (e.g., possibly if the same container is used).

[0113] One or more techniques may include measurements of the container's fill level / height, such as the distance from the food / feed to the top of the container. For example, such measurements may be used in conjunction with a 3D model of an empty container to estimate its volume.

[0114] One or more techniques may include measuring the ratio of observable feed pixels to bowl pixels. This requires one or more images from different angles, which may be sampled from video.

[0115] One or more techniques may include measuring the shape of the container surface. Pets enthusiastically banging their faces against their food may leave marks. Measuring the shape of the surface can allow for a more accurate measurement of the fill level compared to, for example, measuring the height of the feed level alone. The feed may not be (e.g., completely) flat on top, one side of the container may contain a higher fill level than other sides, etc. For example, a range sensor and / or a color camera with one or more models can be used, which can infer distance from pixel data.

[0116] One or more techniques may include measuring one or more quantities of feed pellets / pellets / clumps. For example, if the shape / size of the feed pellets / pellets is known, the quantity in the container can be extrapolated. Techniques may include (e.g., possibly before extrapolation) identifying / classifying feed type and / or retrieving (e.g., known) pellet / pellet size, verifying and / or grading the quality of the pellets / pellets (e.g., whether they are completely / substantially homogeneous, or whether there are many broken pellets / pellets, which could be a source of extrapolation error as it may alter bulk density, etc.).

[0117] One or more sensors, or a combination thereof, can be used to measure to improve the accuracy of the measurements (e.g., by aggregating combined measurements) and / or to understand image perspective.

[0118] One or more techniques may include determining the type of feed and / or the properties of the container contents. Feed type and / or feed condition may be useful because they are related to bulk density, which may help infer the volume and / or nutrient content of the contents. The feed may be wet / dry or degraded. For example, feed may be dehydrated from nearby heat sources (e.g., a bowl placed next to a radiator), sunlight (e.g., a bowl placed directly in sunlight), etc.

[0119] In one or more scenarios, one or more machine learning (ML) models and / or algorithms can be used to classify / identify feed types and / or attributes. A known set of pellets / coarse grains can be used to allow the model to infer the type of feed used. One or more, or each type / category of pellets / coarse grains, can have known sizes / shapes stored in a database (e.g., local and / or cloud-based), which can be used to extrapolate the number of pellets / coarse grains.

[0120] In one or more scenarios, ML models or algorithms can be used to classify / score the consistency of feed content.

[0121] In one or more contexts, classification / identification / scoring can be achieved using the entire measurement of the feed (e.g., segmented pixels of the feed) and / or a sample of the measurement. Such a sample can be related to a cup / spoon, which can be application-specific, such as if the cup / spoon is used for refilling, dispensing food to a pet, etc.

[0122] In one or more contexts, classification / identification / scoring can be based on prior knowledge of feed type. Features of feed appearance (e.g., color, texture, reflectivity, etc.) can be generated and / or used to infer volume and / or weight, instead of using labels describing known container models.

[0123] In one or more contexts, hyperspectral and / or multispectral imaging can measure properties related to nutrient content and / or volume / weight / density (e.g., water content).

[0124] One or more techniques may include the measurement of spilled food / feed contents. It is possible for contents to spill from a container by a pet tapping it as it passes by, knocking contents out of the container during feeding / drinking, and / or spilling when the container is refilled, etc. Measuring and / or quantifying the spilled container contents can provide adjustments for (e.g., more accurate) measurement of consumed feed / food contents.

[0125] In one or more scenarios, pellets / coarse particles outside the container can be measured. This may be fundamentally similar to the techniques described herein for counting pellets / coarse particles, but applied to the outside of the container.

[0126] In one or more scenarios, pixels in an imaging sensor that can represent the contents of an overflowing container or measurement results in a ranging sensor can be segmented.

[0127] In one or more contexts, the similarity of objects identified as overflow content and / or container content can be evaluated. Similarity can be quantified by comparing feature vectors including one or more attributes between the overflow content and the container content using a distance metric. This can be a self-verification phase, in which an object detected as overflow content is determined to have overflowed from the container. Attributes to be compared can include one or more of texture, color, reflectivity and / or edge detection, other image processing techniques and / or hyperspectral / multispectral features.

[0128] One or more techniques may include inferences about volume and / or weight based on measurements of container contents, overflow contents, and / or empty container models.

[0129] In one or more scenarios, the volume and / or weight of feed can be estimated using measurements of one or more fill levels, measurements of one or more feed types / attributes, and / or measurements of one or more overflow contents.

[0130] In one or more scenarios, the number of coarse grains / pellets can be extrapolated from one or more stacking algorithms in a container storage chamber. This can be useful because it can (e.g., directly) handle the air gap between the pellets / coarse grains.

[0131] In one or more scenarios, the volume difference between a 3D model of an empty container and a filling measurement of a container with content can be used to measure volume.

[0132] In one or more scenarios, the volume of the container contents and the density of the container contents can determine the estimated weight reference.

[0133] One or more techniques may include estimating nutrient content. In one or more scenarios, the nutrient content in a container may be estimated by looking up the feed type in a database of nutrient data for known feed types. One or more ML models may be adapted to quantify the weight / percentage of nutrient content in the container for feed degradation (e.g., dehydration from sunlight, etc.) and / or nutrient content in the container (e.g., 10 cm^3 of feed, so there are 4.5 g of water, 1 g of protein, etc.).

[0134] In one or more contexts, for example, features / measurements from hyperspectral and / or multispectral imaging can measure food properties such as the ingredients used / quantified and / or nutrient content.

[0135] One or more technologies may include monitoring the container. Imaging / range sensors and / or other sensors may be used to monitor the container to determine when measurements are taken. Repeated measurements of volume, weight, and / or nutrient content, etc., may be performed at regular / periodic times (e.g., hourly), triggered by external events (e.g., in a connected environment where sensors detect that a door / gate has opened to allow an object to enter / leave the area where the container is located), and / or triggered by events detected / measured in the imaging / range sensors and / or other sensors (e.g., a pet accessing the container, a pet feeding / drinking from the container, etc.).

[0136] One or more alerts can be sent when certain conditions are met, such as when a threshold for filling / weight / volume can be met and / or the container contents / feed may be degraded beyond a threshold (e.g., food is dried out too much in the sun).

[0137] One or more techniques may include: obtaining a model of an empty container, measuring a container with feed using visual / range / other sensors and machine learning and / or algorithms, measuring spilled contents around the container using visual / range / other sensors and machine learning and / or algorithms, measuring properties of the container contents (food / water / other) using machine learning and / or algorithms, measuring the volume of the contents using machine learning and / or algorithms, and measuring feed parameters (such as weight and nutrient content) of the contents using machine learning and / or algorithms with data from one or more techniques described herein.

[0138] One or more technologies may include a model of an empty container for receiving a first food container, providing data on the container's inner surface, outer surface, transparency, refractive properties, and / or other data related to measurements of the substance within the container from any angle / viewpoint using sensors. The model may be provided directly by the container's manufacturer / creator (drawings, 3D printer designs, CAD, molds and / or dies, etc.). The model may be inferred from visual / range / other sensors using machine learning and / or algorithms from direct measurements of physical distance dimensions (such as stereo vision and / or time-of-flight cameras and / or LiDAR sensors) and / or indirect measurements from visual appearance (such as stereo vision cameras). The model may be provided by an operator based on operator-provided measurements (such as width, height, length, and / or curvature properties of the inner / outer surfaces). The model may be provided by an operator based on one or more predefined options of known containers, container outlines, and / or groups of containers stored in a database, attributing the container model / properties to user selection. The model may include data and refractive properties regarding the container's translucent portions. At least the first image or sensor measurement did not capture any substance / object / other in the container or food content space of the food container.

[0139] For example, the model may include one or more data types, such as volume-based (e.g., point cloud, 3D dense model, etc.), dimension-specific (width, height, and / or curvature, etc.) appearance (e.g., color, refractive properties, etc.), and more. User-provided model data may be stored in a database. Parameters of one or more models of the first container and / or a single model of the first container may be obtained to represent any variations in container appearance (e.g., lighting effects), container operation (e.g., the container may have doors / openings / channels for emptying / loading / consuming feed / water / substances), and / or container degradation, etc.

[0140] The technology may include receiving at least a first image or sensor measurement for a first food container, the first image or sensor measurement capturing at least a portion of the food container and / or any food in the food content space of the food container. Segmentation may be performed on the first image. The segmentation or sample may provide matching in a database of feed types and known feed attributes to identify food parameters.

[0141] The feed parameter database may contain the size and visual appearance of coarse grains / pellets / blocks / units; composition, quantity, and / or nutritional data of feed / liquid; images / videos / models of coarse grains / pellets / blocks / units; images / videos / models representing groups of coarse grains / pellets / blocks / units with bulk density, such as low / medium / high or quantity / volume of coarse grains / pellets / blocks / units in a sample area; material / matter density; calorific density; and visual characteristics such as color, refractive index, and / or texture.

[0142] When feed parameters may not be available in the database, they can be inferred directly from the segmentation of feed pixels. Feed parameters may be non-uniform, such as blocks of different sizes or coarse grains, so image segmentation (e.g., feed pixels) can be used to measure samples of physical units / blocks within a predefined region / shape / color that may match a cup / spoon.

[0143] As described in this article, one or more techniques can be used to measure the fill of a container. One or more of these techniques can be used to estimate the amount of food in / near the container in one or more ways (e.g., similar to estimating the correct value in different ways to triangulate that correct value).

[0144] Figure 9 An example illustration of feature 902 is provided, which depicts the correlation between the pixel ratio of a food container image and a weight reference value as a function of the image angle reference.

[0145] Figure 10 An example illustration of feature 1002 is provided, which depicts the correlation between the aspect ratio of a food container image and an image angular reference.

[0146] Figure 11 Illustration 1102 shows three types of food that may occupy space within a food container. Figure 11 In the middle, food container 1104 contains dry, pellet-shaped food. At 1106, food in block-shaped, moist form is shown. Food container 1108 contains food in mashed form.

[0147] Figure 12 This is an example illustration 1202 showing how an image of food container 1204 is segmented into food segmentation image 1026 and food container segmentation image 1208.

[0148] Figure 13 This is an example illustration 1302 showing the transformation from color segmentation (e.g., RGB color channel segmentation) of images of food containers 1304 and 1306 to Hue, Saturation, and Value Space (HSV) thresholding 1308. HSV thresholding can be used for clustering problems in the HSV (Hue, Saturation, and Value) color space for color-based pixel separation. For example, this could support semi-automatic color-based segmentation.

[0149] Figure 14 This is an example of feature 1402, which depicts the correlation between the elliptic eccentricity of a food container image (not shown) and an image angular reference. Figure 14 The sample constants can be a brown bowl, a carpeted floor, and a specific feed type. Sample variables can be food weight and / or angle. Eccentricity can characterize the shape of an ellipse, where, for example, the eccentricity of a circle is 0 and / or the maximum eccentricity is 1.

[0150] Figure 15 This is an exemplary illustration of feature 1502, which depicts the correlation between the ratio of the inner wall height to the edge height of an image of a food container and a weight reference value as a function of the image's angular reference. Figure 15 In this context, the feed level can be determined on the X-axis (wh_rh_e) (inner wall height / edge height)*e. For example, a point on feature 1502 can be indicated by angle reference 1-5.

[0151] Figure 16 This is an example illustration of feature 1602, which depicts the correlation between the ratio of multiple food pixels to elliptical region pixels and the ratio of the eccentricity of the food container image (not shown) and a weight reference value as a function of the image angle reference. Figure 16 In this context, the ratio / eccentricity of (multiple feed pixels / elliptical region pixels) can be correlated with a weight reference value. For example, a point on feature 1602 can be indicated by angle references 1-5.

[0152] Figure 17This is an example illustration of feature 1702, which depicts the correlation between the ratio and weight reference values ​​of multiple food pixels and elliptical region pixels in a food container image as a function of the image angle reference. Figure 17 In this context, the ratio of (multiple feed pixels / elliptical region pixels) can be related to a weight reference value. For example, a point on feature 1702 can be indicated by angle references 1-5.

[0153] Given Figures 1 to 18 Understandably, this document discloses techniques for monitoring object nutrient distribution systems. It also discloses one or more computer-implemented techniques / methods (and devices and / or systems for implementing these techniques / methods) for estimating the filling volume of food containers for objects.

[0154] The technique may include receiving at least a first image for a first food container, the first image capturing at least a portion of the food container and any food in the food content space of the food container. Segmentation may be performed on the first image. The segmentation may provide first food parameters and first x and y parameters corresponding to the first image.

[0155] The technique may include inputting first horizontal and vertical parameters into a machine learning model. The technique may also include determining a first angular reference corresponding to a first image from the machine learning model, at least in part, based on the first horizontal and vertical parameters. The first horizontal and vertical parameters and the first angular reference may be input into the machine learning model.

[0156] The technology may include determining, at least in part, an estimated weight reference and / or an estimated volume reference corresponding to the filling amount of the food container from a machine learning model, based on a first food parameter and a first angle reference.

[0157] In one or more scenarios, the technology may include determining, at least in part, a minimum estimated weight reference and / or a minimum estimated volume reference corresponding to a food container that is at least substantially full, a time period corresponding to when the food container is at least substantially full, a time period corresponding to when the food container is substantially empty, and / or the feeding rate of the object, based on an estimated weight reference and / or an estimated volume reference.

[0158] In one or more scenarios, segmentation may include at least food container segmentation and food segmentation. In one or more scenarios, the first image may be provided by a digital camera and / or a machine vision camera.

[0159] In one or more scenarios, segmentation can produce a plurality of first pixels corresponding to a food container corresponding to a first image and / or a plurality of second pixels corresponding to food in the food content space of a food container corresponding to a first image.

[0160] Pet owners / parents may be unable and / or unwilling to measure or weigh pet food. Being able to measure food volume and verify it against weight would allow for the calculation of nutritional content from images. The computer vision system described in this paper can automatically estimate the amount of food in a food container (e.g., a pet food bowl).

[0161] Two features are extracted from the image and fed into a neural network / machine learning model to infer / estimate the weight of the food.

[0162] For example, the food / (food container + food) pixel ratio can be expressed as: feed_bowl_pixel_ratio = n_feed_pixels / (n_feed_pixels + n_bowl_pixels).

[0163] In one or more scenarios, the first food parameter (as described above) can be the ratio of the number of second pixels to the sum of the number of first pixels and the number of second pixels.

[0164] In one or more scenarios, segmentation produces a rotated bounding box width and height corresponding to the first image. The first aspect ratio can be the ratio of the rotated bounding box width to its height. For example, the width / height aspect ratio can be expressed as:

[0165] aspect_ratio = rotated_bbox_width / rotated_bbox_height.

[0166] In one or more scenarios, segmentation can be performed using one or more morphological geodesic active profiles (MGACs), in addition to other segmentation algorithms / techniques.

[0167] In one or more contexts, the technology may include determining the nutrient content of a food corresponding to an estimated weight reference and / or an estimated volume reference, at least in part, based on an estimated weight reference and / or an estimated volume reference. For example, the estimated weight and / or estimated volume may be used in conjunction with the food’s known (e.g., predetermined, real-time accessed, post-accessed weight / volume estimates, etc.) nutrient density (e.g., kcal / 100g or kcal / cup, etc.) to calculate the nutrients / nutrients provided and / or consumed through the food content.

[0168] In one or more contexts, the object may be a canine and / or feline. In one or more contexts, the food container may be a pet food bowl.

[0169] Table 1 (below) shows the results of neural network / machine learning estimates of the filling amount from food containers (e.g., possibly used in conjunction with regression models and other models). Table 1: Results from neural network / machine learning model validation

[0170]

[0171] The data described in Table 1 (above) show that food weight can be estimated from at least one image of food in a food container, which can be taken from one or more angles or various angles. The accuracy of the measurement can be a function of the image angle. The data indicate that the system / apparatus / method / technique described herein can predict the amount of food / filling volume present in a food container with reasonable accuracy.

[0172] Now for reference Figure 3A and Figure 3B Figure 300 illustrates an example technique / process for estimating the filling amount of an object food container. This method can be performed by an Object Nutrition Distribution Monitoring and Control Device (SNDMCD) and other devices. For example, the Object Nutrition Distribution Monitoring and Control Device can be a process control device / logic controller 110b and other devices 110a-110d and / or 140a-140i and / or cloud computing devices. The Object Nutrition Distribution Monitoring and Control Device (SNDMCD) can communicate with any device in the Object Nutrition Distribution Monitoring System Network (SNDMSN) 100. At 302, the process can be started or restarted.

[0173] At 304, the object nutrient distribution monitoring and control device can receive at least a first image of the first food container, the first image capturing at least a portion of the food container and any food in the food content space of the food container. At 306, the object nutrient distribution monitoring and control device can perform segmentation on the first image. This segmentation can provide first food parameters and first longitudinal and transverse parameters corresponding to the first image.

[0174] At point 308, the object nutrient distribution monitoring and control device can input the first longitudinal and transverse parameters into the machine learning model. At point 310, the object nutrient distribution monitoring and control device can determine, at least partially, a first angular reference corresponding to the first image from the machine learning model based on the first longitudinal and transverse parameters.

[0175] At point 312, the object nutrient distribution monitoring and control device can input the first food parameter and the first angular reference into the machine learning model. At point 314, the object nutrient distribution monitoring and control device can determine, at least in part, an estimated weight reference and / or an estimated volume reference corresponding to the filling amount of the food container from the machine learning model, based on the first food parameter and the first angular reference. At point 316, this technique / process can be stopped or restarted.

[0176] Figure 4This is a block diagram of the hardware configuration of an example device that can act as a process control device / logic controller, such as... Figure 1 The object nutrient distribution monitoring and control device 110b and other devices such as any of devices 140a-140i and devices 110a-110d, for example. Hardware configuration 400 may be operable to facilitate the delivery of information from an internal server of the device. Hardware configuration 400 may include processor 410, memory 420, storage device 430 and / or input / output device 440. One or more of components 410, 420, 430 and 440 may be interconnected, for example, using system bus 450. Processor 410 may process instructions for execution within hardware configuration 400. Processor 410 may be a single-threaded processor or a multi-threaded processor. Processor 410 may be a single-core processor or a multi-core processor. Processor 410 may be able to process instructions stored in memory 420 and / or on storage device 430. Processor 410 may be a CPU, GPU, or hardware decoder (e.g., for a JPEG hardware decoder).

[0177] Memory 420 may store information within hardware configuration 400. Memory 420 may be a computer-readable medium (CRM), such as a non-transitory CRM. Memory 420 may be a volatile memory cell and / or a non-volatile memory cell.

[0178] Storage device 430 may provide mass storage for hardware configuration 400. Storage device 430 may be a computer-readable medium (CRM), such as a non-transitory CRM. Storage device 430 may include, for example, hard disk drives, optical disk drives, flash memory, and / or other mass storage devices. Storage device 430 may be a device located external to hardware configuration 400.

[0179] Input / output device 440 can provide input / output operations for hardware configuration 400. Input / output device 440 (e.g., transceiver device) may include one or more of the following: network interface devices (e.g., Ethernet cards), serial communication devices (e.g., RS-232 ports), one or more Universal Serial Bus (USB) interfaces (e.g., USB 2.0 ports), and / or wireless interface devices (e.g., 802.11 cards). Input / output devices may include those configured to output to one or more networks (e.g.,... Figure 1The object nutrient distribution monitoring network 130) is a driving device that sends and / or receives communications from it. Input / output device 400 can communicate with one or more input / output modules (not shown) that are accessible to and / or remote from hardware configuration 400. One or more output modules can provide input / output functionality in digital signal form, discrete signal form, TTL form, analog signal form, serial communication protocol, fieldbus protocol communication, and / or other open or proprietary communication protocols and / or similar protocols.

[0180] Camera device 460 can provide digital video input / output capabilities to hardware configuration 400. Camera device 460 can communicate with any component of hardware configuration 400, possibly via system bus 450. Camera device 460 can capture digital images and / or scan various types of images, such as Universal Product Code (UPC) codes and / or Quick Response (QR) codes, as well as other images as described herein. In one or more scenarios, camera device 460 can be identical and / or substantially similar to any other camera device described herein.

[0181] Camera device 460 may include at least one microphone device and / or at least one speaker device (not shown). The inputs / outputs of camera device 460 may include audio signals / packets / components, which may be separate / separable from the video signals / packets / components of camera device 460, or combined with the video signals / packets / components of camera device 460 in some (e.g., separable) combinations.

[0182] Camera device 460 can also detect the presence of one or more objects that may be near camera device 460 and / or located in the same general space as camera device 460 (e.g., the same room, feeding area, etc.). Camera device 460 can measure the general activity level (e.g., high activity, medium activity, and / or low activity) of one or more objects that can be detected by camera device 460. Camera device 460 can detect one or more general characteristics (e.g., height, body shape, skin color, pulse, heart rate, respiratory count, etc.) of one or more objects detected by camera device 460. Camera device 460 can be configured to identify one or more specific objects, for example. Camera device 460 can be configured to perform one or more food container filling level / filling volume techniques disclosed herein.

[0183] Camera device 460 can communicate with hardware configuration 400 via wired and / or wireless communication. In one or more scenarios, camera device 460 may be located outside of hardware configuration 400. In one or more scenarios, camera device 460 may be located inside hardware configuration 400.

[0184] The subject matter of this disclosure and its components can be implemented by instructions that, when executed, cause one or more processing devices to perform the processes and / or functions described herein. Such instructions may include, for example, interpreted instructions, such as scripting instructions (e.g., JavaScript or ECMAScript instructions) or executable code, and / or other instructions stored in a computer-readable medium.

[0185] The embodiments of the subject matter and / or functional operation described in this specification and / or the accompanying drawings may be provided in digital electronic circuits, computer software, firmware and / or hardware (including the structures disclosed in this specification and their structural equivalents) and / or combinations thereof. The subject matter described in this specification may be implemented as one or more computer program products, for example, one or more modules of computer program instructions encoded on a tangible program carrier for execution by a data processing apparatus and / or control of the operation of the data processing apparatus.

[0186] Computer programs (also known as programs, software, software applications, scripts, or code) can be written in any programming language, including compiled or interpreted languages ​​and / or declarative or procedural languages. They can be deployed in any form, including as standalone programs or as modules, components, subroutines, and / or other units suited to a computing environment. A computer program may or may not correspond to a file in a file system. A program may be stored as part of a file that holds other programs and / or data (e.g., one or more scripts stored in a markup language document), a single file dedicated to the program in question, and / or multiple co-files (e.g., a file storing one or more modules, subroutines, or portions of code). A computer program can be deployed to execute on a single computer or on multiple computers that may be located at one site or distributed across multiple sites and / or interconnected via a communication network.

[0187] The processes and / or logic flows described in this specification and / or the accompanying drawings can be executed by one or more programmable processors that execute one or more computer programs to perform functions by manipulating input data and / or generating outputs, thereby binding the processes to a specific machine (e.g., a machine programmed to perform the processes described herein). The processes and / or logic flows can also be executed by special-purpose logic circuitry, and the apparatus can also be implemented as special-purpose logic circuitry, such as FPGAs (Field-Programmable Gate Arrays) and / or ASICs (Application-Specific Integrated Circuits).

[0188] Computer-readable media suitable for storing computer program instructions and / or data can include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices (e.g., EPROM, EEPROM, and / or flash memory devices); magnetic disks (e.g., internal hard disks or removable disks); magneto-optical disks; and / or CD-ROMs and DVD-ROMs. The processor and / or memory may be supplemented by or incorporated into dedicated logic circuitry.

[0189] While this specification and accompanying drawings contain numerous specific details of implementation, these should not be construed as limiting the scope of any invention and / or what may be claimed, but rather as descriptions of features that may be specific to the exemplary embodiments described. Certain features described in the context of individual embodiments may also be implemented in combination in a possible single embodiment. Various features described in the context of a possible single embodiment may also be implemented individually in multiple combinations or in any suitable sub-combinations. Although features may be described above as functioning in certain combinations and / or possibly even (e.g., initially) so claimed, one or more features from a claimed combination may be removed from that combination in certain circumstances. Claimed combinations may be for sub-combinations and / or variations thereof.

[0190] Although operations may be depicted sequentially in the accompanying drawings, this should not be construed as requiring such operations to be performed in the specific order and / or sequential manner shown, and / or requiring all illustrated operations to be performed to achieve a useful result. The described program components and / or systems may typically be integrated into a single software product and / or packaged into multiple software products.

[0191] Examples of the subject matter described in this specification have been described. Unless otherwise expressly stated, the actions described in the claims can be performed in different orders and still achieve useful results. For example, the processes depicted in the figures do not require the specific order and / or sequential order shown to achieve useful results. In one or more situations, multitasking and parallel processing may be advantageous.

[0192] Non-restrictive and composable examples:

[0193] Example 1: A computer implementation method for estimating the fill volume of an object food container. The method includes receiving at least a first image for a first food container, the at least first image capturing at least a portion of the food container and any food in the food content space of the food container. The method includes performing segmentation on the first image, the segmentation providing a first food parameter and a first cross-sectional parameter corresponding to the first image. The method includes inputting the first cross-sectional parameter into a machine learning model. The method includes determining, at least partially based on the first cross-sectional parameter, a first angular reference corresponding to the first image from the machine learning model. The method includes inputting the first food parameter and the first angular reference into the machine learning model. The method includes determining, at least partially based on the first food parameter and the first angular reference, at the machine learning model at least one of the following: an estimated weight reference or an estimated volume reference corresponding to the fill volume of the food container. In one or more scenarios, one or more of the above are performed at least partially by one or more processors.

[0194] Example 2: The method according to Example 1 further includes: determining at least one of the following based at least in part on at least one of the following: an estimated weight reference or an estimated volume reference: a minimum estimated weight reference corresponding to a food container that is at least substantially full, a time period corresponding to when the food container is at least substantially full, a time period corresponding to when the food container is substantially empty, or a feeding rate of the object.

[0195] Example 3: The method according to any one of the foregoing examples, wherein the segmentation includes at least food container segmentation and food segmentation.

[0196] Example 4: The method according to any one of the preceding examples, wherein the first image is provided by at least one of the following: a digital camera, a depth camera, an infrared camera, or a machine vision camera.

[0197] Example 5: The method according to any one of the preceding claims, wherein the segmentation produces: a plurality of first pixels corresponding to the food container corresponding to the first image, and a plurality of second pixels corresponding to food in the food content space of the food container corresponding to the first image.

[0198] Example 6: According to the method described in Example 5, wherein the first food parameter is the ratio of the number of the second pixels to the sum of the number of the first pixels and the number of the second pixels.

[0199] Example 7: The method according to any one of the foregoing examples, wherein the segmentation generates a geometric description. The geometric description includes a rotated bounding box width and a rotated bounding box height corresponding to the first image.

[0200] Example 8: According to the method described in Example 7, wherein the first horizontal and vertical parameters are the ratio of the width of the rotated bounding box to the height of the rotated bounding box.

[0201] Example 9: The method according to any one of the preceding examples, wherein the segmentation uses one or more morphological geodesic active profiles (MACG).

[0202] Example 10: The method according to any one of the foregoing examples further includes: determining the nutritional content of the food corresponding to the estimated weight reference and / or the estimated volume reference, at least in part based on the estimated weight reference and / or the estimated volume reference.

[0203] Example 11: The method according to any one of the preceding examples, wherein the object is at least one of the following: a canine or a feline; and wherein the food container is a pet food bowl.

[0204] Example 12: A computer-implemented method for capturing machine learning training data to estimate the fill volume of an object food container, the method comprising: (a) inputting a first type of food container into at least one processing device. The method comprises: (b) inputting one or more colors of the food container into the at least one processing device. The method comprises: (c) inputting a first type of floor on which the food container is disposed into the at least one processing device. The method comprises: (d) inputting a first type of food into the at least one processing device. The method comprises: (e) inputting a first predetermined fill volume of the food disposed in a food content space of the food container into the at least one processing device. The method comprises: (f) placing a camera device at a first position relative to the food container in a vertical plane that bisects at least a portion of the food container. The camera device is arranged at the first position at a first angle relative to the food container. The method comprises: (g) capturing a first image of the food container from the first position.

[0205] The method includes (h) repeating elements (f) to (g) for placing the camera device in at least one of the following: a second position, a third position, a fourth position, and a fifth position. The camera device is arranged to capture at least one of the following: a second image at a second angle, a third image at a third angle, a fourth image at a fourth angle, and a fifth image at a fifth angle. The method includes (i) repeating elements (e) to (h) for at least a second predetermined fill amount and a third predetermined fill amount. The method includes (j) storing the first image, the second image, the third image, the fourth image, and the fifth image into a machine learning training database. The method includes (k) training at least one machine learning model using at least the first image, the second image, the third image, the fourth image, and the fifth image. In one or more scenarios, at least elements (a) to (e) and (g) to (k) are at least partially executed by one or more processors of the at least one processing device.

[0206] Example 13: According to the method of Example 12, the food container of the type includes at least one of the following: the shape of the food container, the size of the food container, or the material of the food container.

[0207] Example 14: The method according to Example 12 or Example 13, wherein the first type of flooring includes at least one of the following: bare flooring, tile flooring, or carpet flooring.

[0208] Example 15: The method according to any one of Examples 12 to 14, wherein the first type of food comprises at least one of the following: blocky, granular, mashed, wet or dry.

[0209] Example 16: The method according to any one of Examples 12 to 15, wherein the camera device is at least one of the following: a digital camera or a machine vision camera.

[0210] Example 17: The method according to any one of Examples 12 to 16, wherein at element (i), elements (e) to (h) are repeated for at least the second predetermined fill amount, the third predetermined fill amount, the fourth predetermined fill amount and the fifth predetermined fill amount.

[0211] Example 18: The method according to any one of Examples 12 to 17 further includes storing the first image, the second image, the third image, the fourth image, and the fifth image together with indications of the first type of food container, the first type of floor, and the first type of food.

[0212] Example 19: The method according to any one of Examples 12 to 18, wherein the first angle is defined by the angular displacement of the first position relative to a horizontal plane that cuts through at least a portion of the food container, the second angle is defined by the angular displacement of the second position relative to the horizontal plane, the third angle is defined by the angular displacement of the third position relative to the horizontal plane, the fourth angle is defined by the angular displacement of the fourth position relative to the horizontal plane, and the fifth angle is defined by the angular displacement of the fifth position relative to the horizontal plane.

[0213] Example 20: The method according to Example 19, wherein the first angle is approximately 90 degrees, the second angle is approximately 65 degrees, the third angle is approximately 45 degrees, the fourth angle is approximately 25 degrees, and the fifth angle is approximately 10 degrees.

[0214] Example 21: The method according to any one of Examples 12 to 20 further includes (1) feeding a second type of food container into the at least one processing device. The method includes (m) repeating elements (e) to (k) for the second type of food container.

[0215] Example 22: The method according to any one of Examples 12 to 21 further includes (n) inputting a second type of floor on which the food container is disposed into the at least one processing device. The method includes (o) repeating elements (e) to (k) for the second type of floor on which the food container is disposed.

[0216] Example 23: The method according to any one of Examples 12 to 22 further includes (p) inputting a second type of food into the at least one processing device. The method includes (q) repeating elements (e) to (k) for the second type of food.

[0217] Although this disclosure has been shown and described in detail in the accompanying drawings and the foregoing description, it should be considered illustrative rather than restrictive, and it should be understood that only certain examples have been shown and described, and protection is intended for all changes and modifications within the spirit of this disclosure.

Claims

1. A computer-implemented method for estimating the filling volume of a food container, the method comprising: (a) Receive at least a first image for a first food container, the at least first image capturing at least a portion of the food container and any food in the food content space of the food container; (b) Perform segmentation on the first image, the segmentation providing a first food parameter and a first cross-sectional parameter corresponding to the first image; (c) Input the first cross and vertical parameters into the machine learning model; (d) Determine a first angular reference corresponding to the first image from the machine learning model, at least in part, based on the first axial and lateral parameters; (e) Input the first food parameters and the first angle reference into the machine learning model; and (f) Based at least in part on the first food parameter and the first angle reference, determine from the machine learning model at least one of the following: an estimated weight reference or an estimated volume reference corresponding to the filling amount of the food container. Steps (a)-(f) are performed at least in part by one or more processors.

2. The method according to claim 1, further comprising: (g) Determine at least one of the following based at least in part on the estimated weight reference or the estimated volume reference: the minimum estimated weight reference corresponding to a food container that is at least substantially full, the time period corresponding to when the food container is at least substantially full, the time period corresponding to when the food container is substantially emptied, or the feeding rate of the object.

3. The method according to claim 1 or claim 2, wherein, The segmentation includes at least food container segmentation and food segmentation.

4. The method according to any one of the preceding claims, wherein, The first image is provided by at least one of the following: a digital camera, a depth camera, an infrared camera, or a machine vision camera.

5. The method according to any one of the preceding claims, wherein, The segmentation produces: a plurality of first pixels corresponding to the food container corresponding to the first image, and a plurality of second pixels corresponding to the food in the food content space of the food container corresponding to the first image.

6. The method according to claim 5, wherein, The first food parameter is the ratio of the number of the second pixels to the sum of the number of the first pixels and the number of the second pixels.

7. The method according to any one of the preceding claims, wherein, The segmentation generates a geometric description, which includes the width and height of the rotated bounding box corresponding to the first image.

8. The method according to claim 7, wherein, The first horizontal and vertical parameters are the ratio of the width of the rotated bounding box to the height of the rotated bounding box.

9. The method according to any one of the preceding claims, wherein, The segmentation uses one or more morphological geodesic active profiles (MACG).

10. The method according to any one of the preceding claims further comprises: The nutritional content of the food corresponding to the estimated weight reference and / or estimated volume reference is determined, at least in part, based on the estimated weight reference and / or estimated volume reference.

11. The method according to any one of the preceding claims, wherein, The object is at least one of the following: a canine or a feline; and the food container is a pet food bowl.

12. A computer-implemented method for capturing machine learning training data to estimate the fill volume of a food container for an object, the method comprising: (a) Inputting a food container of type 1 into at least one processing device; (b) Inputting one or more colors of the food container into the at least one processing device; (c) Inputting the food container onto a first type of floor into the at least one processing device; (d) Inputting the first type of food into the at least one processing device; (e) Inputting a first predetermined filling amount of the food disposed in the food content space of the food container into the at least one processing device; (f) The camera device is placed in a first position relative to the food container in a vertical plane that cuts through at least a portion of the food container, the camera device being arranged at the first position at a first angle relative to the food container; (g) Capture a first image of the food container from the first location; (h) Repeat elements (f) to (g) for placing the camera device in at least one of the following: a second position, a third position, a fourth position and a fifth position, wherein the camera device is arranged to capture at least one of the following: a second image at a second angle, a third image at a third angle, a fourth image at a fourth angle and a fifth image at a fifth angle; (i) Repeat elements (e) to (h) for at least the second and third predetermined fill amounts; (j) Store the first image, the second image, the third image, the fourth image, and the fifth image into a machine learning training database; and (k) Train at least one machine learning model using at least the first image, the second image, the third image, the fourth image, and the fifth image. At least elements (a) to (e) and (g) to (k) are executed at least partially by one or more processors of the at least one processing device.

13. The method according to claim 12, wherein, The type of food container includes at least one of the following: the shape of the food container, the size of the food container, or the material of the food container.

14. The method according to claim 12 or 13, wherein, The first type of flooring includes at least one of the following: bare flooring, tile flooring, or carpet flooring.

15. The method according to any one of claims 12 to 14, wherein, The first type of food includes at least one of the following: in chunks, in granules, mashed, wet, or dry.

16. The method according to any one of claims 12 to 15, wherein, The camera device is at least one of the following: a digital camera or a machine vision camera.

17. The method according to any one of claims 12 to 16, wherein, At element (i), elements (e) to (h) are repeated for at least the second predetermined fill amount, the third predetermined fill amount, the fourth predetermined fill amount and the fifth predetermined fill amount.

18. The method according to any one of claims 12 to 17, further comprising storing the first image, the second image, the third image, the fourth image, and the fifth image together with indications of the first type of food container, the first type of floor, and the first type of food.

19. The method according to any one of claims 12 to 18, wherein, The first angle is defined by the angular displacement of the first position relative to a horizontal plane that cuts through at least a portion of the food container, the second angle is defined by the angular displacement of the second position relative to the horizontal plane, the third angle is defined by the angular displacement of the third position relative to the horizontal plane, the fourth angle is defined by the angular displacement of the fourth position relative to the horizontal plane, and the fifth angle is defined by the angular displacement of the fifth position relative to the horizontal plane.

20. The method according to claim 19, wherein, The first angle is approximately 90 degrees, the second angle is approximately 65 degrees, the third angle is approximately 45 degrees, the fourth angle is approximately 25 degrees, and the fifth angle is approximately 10 degrees.

21. The method according to any one of claims 12 to 20, further comprising: (l) Inputting a second type of food container into the at least one processing device; and (m) Repeat elements (e) to (k) for the second type of food container.

22. The method according to any one of claims 12 to 21, further comprising: (n) Inputting a food container onto a second type of floor into the at least one processing device; and (o) Repeating elements (e) to (k) of the second type of floor on which the food container is placed.

23. The method according to any one of claims 12 to 22, further comprising: (p) Inputting the second type of food into the at least one processing device; and (q) Repeat elements (e) to (k) for the second type of food.

Images