SYSTEM AND METHOD FOR ASSESSING PRODUCT SHELF-LIFE USING TIME-LAPSE IMAGE DATA - Patent application

Abstract

Systems and techniques for identifying duration of a product using image data. The method may include receiving, by a computing system, from an imaging device, image data of the product captured at consistent time intervals during a time period, the image data including one or more processed products that have been coated with a shelf-life extending coating solution and one or more unprocessed products that have not been coated with a shelf-life extending coating solution during the time period, performing object detection on the image data to identify bounding boxes around each product in the image data, identifying quality attributes within each bounding box at each of time intervals during the time period, and determining, for each product, one or more duration scoring metrics based on comparing the identified quality attributes across the time period.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to U.S. patent application Ser. No. 63 / 309,876, filed February 14, 2022, the disclosure of which is incorporated by reference in its entirety.

[0002] Technical Field

[0002] This document generally describes devices, systems, and methods relating to determining the shelf life of food products, such as produce, using time lapse image data. [Background technology]

[0003] background

[0003] Food products such as produce, fruit, and meat can have different shelf lives that can affect their suitability for consumption and value in the supply chain. Many different stakeholders throughout the supply chain are interested in assessing the shelf life of produce. Shelf life can affect how long a produce can be displayed for purchase by a consumer before the produce is no longer salable or is no longer desirable for purchase by the consumer. Sometimes produce may move through the supply chain without being coated with a shelf life extending coating solution. As a result, the produce may reach the consumer with a short shelf life and / or with properties that make the produce less desirable for purchase by the consumer. As a result, the produce may not be purchased and / or may be wasted. Sometimes produce may be coated with some type of shelf life extending coating solution, but not with the correct amount and / or type of solution. As a result, the produce may still reach the consumer with a short shelf life and / or with properties that make the produce less desirable for purchase by the consumer.

[0004]

[0004] It can be difficult to objectively and quantitatively define the shelf life of a produce based on image data. Relevant stakeholders in the supply chain can observe and compare colors or other characteristics visible in or on the food product. However, the human eye can only distinguish color or other characteristic differences in extreme cases, such as when the differences are quite obvious. For example, the human eye may be able to distinguish when a lime is yellow, green, or brown, but may not be able to distinguish more subtle color differences that may be indicators of the shelf life of the produce. Sometimes the human eye can identify color changes, but may not be able to accurately quantify such changes. Subtle changes in color and other characteristics can represent quality changes in the food product, which may affect the shelf life and saleability of the produce. Summary of the Invention [Means for solving the problem]

[0005] overview

[0005] This document generally describes systems, methods, and techniques for assessing the shelf life of a produce using image data. More specifically, the shelf life of a produce can be determined from time-lapse image data, such as a time-lapse of hyperspectral images (HSI), which can be generated and captured by one or more imaging devices, such as a camera configured to capture images of the produce within and / or outside the visible light spectrum. In some implementations, the hyperspectral image data can be captured by a hyperspectral camera over some period of time with a fixed time difference. This image data can then be used to assess the shelf life of the produce. The image data can include images and other image-related data and / or metadata, including but not limited to histograms, RGB color models, hyperspectral data, multispectral data, and / or videos.

[0006]

[0006] To assess the shelf life of a produce, pixels representing the produce in the time-lapse image data can be extracted at one or more predefined time frames. Machine learning models and / or algorithms can then be applied to the extracted pixels to calculate the quality changes of the produce over time. Such quality changes can include color, volume, spoilage, hardness, wrinkles, mold, deterioration, infection, spoilage, dry matter, pH, Brix, sugar content, and / or drying. Thus, machine learning techniques can be used to identify and calculate both internal and external quality attributes of the produce and, in combination, quantify the shelf life, edibility, and / or saleability of the produce. The shelf life of the produce can then be determined based on an assessment of the identified quality changes.

[0007]

[0007] The determined shelf life can be used by relevant stakeholders in the supply chain to identify how ripe the produce is, the length of the produce's shelf life and / or saleability, and how a shelf life extension coating solution can affect the ripening and / or saleability of the produce. For example, the disclosed techniques can also be used to determine a shelf life extension factor between a produce that may be coated with a shelf life extension coating solution (e.g., a treated produce) and a produce that may not be coated with the solution (e.g., an untreated produce). In some embodiments, the shelf life extension factor can be determined at a given environmental condition and can be the average of the shelf life ratios of the treated and untreated produce over a statistically representative set of produce.

[0008]

[0008] The identified shelf life extension factor can be used by relevant stakeholders throughout the supply chain to modify the shelf life extension coating solution, the application of the solution to the produce, and / or the quantity of solution applied to the produce. By using the disclosed techniques described throughout this disclosure, the treated produce can have increased and / or improved ripeness, shelf life, edibility, and / or saleability, thereby benefiting stakeholders and consumers in the supply chain. The shelf life extension factor can also be used to compare different coating solutions and their effectiveness in extending the shelf life of the produce. The shelf life extension factor can also be used to compare treated and untreated produce under refrigerated, retail, ambient, and other environmental conditions.

[0009]

[0009] Certain embodiments described herein may include a method for identifying duration of a product using image data, the method including receiving, by a computing system, from an imaging device, image data of the products, which may be captured at consistent time intervals during a time period, the image data including one or more processed products that may be coated with a shelf life extending coating solution and one or more unprocessed products that may not be coated with a shelf life extending coating solution during the time period, wherein the processed products and the unprocessed products are the same product type; performing, by the computing system, an object detection technique on the image data to identify a bounding box around each product in the image data; identifying, by the computing system, quality attributes within each bounding box at each of the time intervals during the time period; determining, by the computing system, one or more duration scoring metrics for each of the products in the image data based on comparing the identified quality attributes over the time period; and transmitting, by the computing system, the one or more duration scoring metrics for each of the products to a user device for display on a graphical user interface (GUI).

[0010]

[0010] In some implementations, the embodiments described herein may include one or more of the following features. For example, the method may include storing, by the computing system, in a data store, for each of the products: (i) a bounding box around each product in the image data; (ii) quality attributes of each product; and (iii) a duration scoring metric for each product. The method may also include identifying, by the computing system, a grid structure in the image data; and assigning, by the computing system, a grid index within the grid structure to each bounding box, the grid index being used to identify the product within the bounding box.

[0011]

[0011] As another example, the method can include determining, by the computing system, a duration score for the product in the image data based on identifying an extended shelf-life duration from the one or more duration scoring metrics, the extended shelf-life duration experienced by the treated product compared to the untreated product over the time period. Additionally, the method can include determining, by the computing system, one or more modifications to make to the shelf-life extension coating solution based at least in part on the one or more duration scoring metrics and the duration score for the product. The method can include sending, by the computing system, instructions to a controller of the supply chain participant to modify the shelf-life extension coating solution based on the determined one or more modifications. The one or more changes, when executed by the supply chain participant, may include instructions to cause at least one of: (i) increasing a concentration of one or more components of the shelf-life extension coating solution; (ii) decreasing a concentration of one or more components of the shelf-life extension coating solution; (iii) applying the shelf-life extension coating solution to a batch of unprocessed product of the same product type; (iv) increasing an amount of the shelf-life extension coating solution applied to a subsequent batch of unprocessed product of the same product type; and (v) decreasing an amount of the shelf-life extension coating solution applied to a subsequent batch of unprocessed product of the same product type.

[0012] In some implementations, the quality attributes can include at least one of color, volume, hardness, wrinkles, spoilage, and mold. The method can also include mapping, by a computing system, RGB values ​​from the image data into a three-dimensional (3D) space, identifying, by the computing system, clusters of RGB values ​​in the 3D space, selecting, by the computing system, one or more of the clusters in the 3D space having a threshold median color, and extracting, by the computing system, the selected one or more clusters from the image data, where the selected one or more clusters represent processed and unprocessed products in the image data.

[0013]

[0013] In some embodiments, by the computing system, identifying the quality attribute within each bounding box can include identifying color values ​​of all pixels within the bounding box, and by the computing system, determining one or more duration scoring metrics for each product in the image data can include identifying a distance between each of the identified color values ​​and a statistical scale color value of the product in the image data, determining whether the distance between each of the identified color values ​​and the statistical scale color value in each of the time intervals exceeds a threshold, identifying a duration scoring metric for each product as a time interval during the time period during which the distance between the identified color value and the statistical scale color value exceeds a threshold level, and generating an output visually indicating the distance between each of the identified color values ​​and the statistical scale color value of the product in each time interval during the time period. Further, the statistical scale color value can be identified by the computing system based on receiving image data of the product in a first time interval during the time period from the imaging device, identifying color values ​​of all pixels representing the product in the image data in the first time interval, and calculating the statistical scale color value of the product in the image data based on averaging the identified color values. The statistical scale color value may represent an average color of the processed and unprocessed product when the product is imaged by the imaging device at a first time interval during the time period.

[0014]

[0014] In some embodiments, the one or more duration scoring metrics may correspond to at least one of the product's eating time, end of eating time, length of eating time, shelf life, end of shelf life, length of shelf life, edibility, end of edibility, length of edibility, saleability, end of saleability, and length of saleability for each of the products in the image data.

[0015]

[0015] Sometimes, by the computing system, identifying the quality attribute within each bounding box can include determining the area of ​​the product within the bounding box based on counting the number of pixels representing the product within the bounding box, determining the radius of the product based on the area and a formula of the area of ​​a circle, and determining the volume of the product at each time interval based on the radius and a formula of the volume of a sphere. By the computing system, determining one or more duration scoring metrics for each of the products in the image data can further include determining the change in the volume of the product at each time interval relative to the volume of the product at a first time interval, determining whether the volume change exceeds a threshold level, identifying the duration scoring metric of each product as the time interval during the time period during which the volume change exceeds the threshold level, and generating an output visually indicating the volume change of each product at each time period during the time period. In some embodiments, the threshold level can be a 10% volume shrinkage between the volume of the product at one of the time intervals and the volume of the product at the first time interval.

[0016]

[0016] In some embodiments, identifying quality attributes within each bounding box by the computing system can include identifying a grid structure of the product within the bounding box, retrieving a machine learning trained wrinkle model from the data store, and applying the wrinkle model to each grid cell within the grid structure to identify one or more wrinkle patches. Determining one or more duration scoring metrics for each of the products in the image data by the computing system can include counting a percentage of grid cells having wrinkle patches of the bounding box, determining whether the percentage of grid cells having wrinkle patches exceeds a threshold level, identifying a duration scoring metric of the product as a wrinkle coverage percentage based on a determination that the percentage of grid cells having wrinkle patches exceeds the threshold level, and generating an output visually indicative of the wrinkle coverage percentage of the product. The wrinkle model is trained using a set of training image data of other products of the same type, the set of training image data being annotated based on a previous identification of a first portion of the other products as wrinkled and a previous identification of a second portion of the other products as not wrinkled.

[0017]

[0017] In some embodiments, identifying quality attributes within each bounding box by the computing system can include identifying a grid structure of the produce within the bounding box, retrieving a machine learning trained quality feature model from the data store, and applying the quality feature model to each grid cell within the grid structure to identify one or more quality features. Determining one or more duration scoring metrics for each of the produce in the image data by the computing system can include counting a percentage of grid cells having the quality feature of the bounding box, determining whether the percentage of grid cells having the quality feature exceeds a threshold level, identifying a duration scoring metric for the produce as a percentage of coverage of the quality feature based on a determination that the percentage of grid cells having the quality feature exceeds the threshold level, and generating an output visually indicating the percentage of coverage of the quality feature of the produce. The quality feature model is trained using a set of training image data of other produce of the same type, the set of training image data being annotated based on a previous identification of a first portion of the other produce as having one or more of the quality features and a previous identification of a second portion of the other produce as not having the quality feature. Additionally, the quality characteristics may include at least one of hardness, internal spoilage, external spoilage, internal infection, external infection, internal spoilage, external spoilage, dry matter content, pH, and sugar content.

[0018]

[0018] In some embodiments, the produce in the image data can be an avocado, (i) the quality attribute can be at least one of color, volume, hardness, and ripeness, and (ii) the one or more duration scoring metrics can be at least one of a change in color over a time period, a change in volume over a time period, a change in hardness over a time period, and a change in ripeness over a time period. The produce in the image data can be a lime, and the quality attribute can be at least one of color and volume, and the one or more duration scoring metrics can be at least one of a change in color and a change in volume over a time period. The produce in the image data can be an apple, and (i) the quality attribute can be at least one of color and hardness, and (ii) the one or more duration scoring metrics can be at least one of a change in color over a time period and a change in hardness over a time period.

[0019]

[0019] As another example, the method may include identifying, by a computing system, a product type of the product in the image data at a first time interval during a time period; selecting, by the computing system, one or more types of duration scoring metrics from a plurality of duration scoring metrics for the identified product type; selecting, by the computing system, one or more threshold levels for the selected types of duration scoring metrics for the identified product type; identifying, by the computing system, quality attributes corresponding to each of the selected types of duration scoring metrics; and determining, by the computing system, the selected one or more duration scoring metrics based on a comparison of the identified quality attributes with the selected one or more threshold levels.

[0020]

[0020] In some embodiments, the method may also include retrieving, by a computing system, duration scoring metrics for the treated and untreated products over a period of time from a data store, identifying, by the computing system, one or more differences between the duration scoring metrics for the treated and untreated products, and generating, by the computing system, an output indicative of the identified one or more differences. The generated output may include one or more modifications of the shelf-life extension coating solution, the one or more modifications being determined by the computing system based on the identified one or more differences.

[0021] As another example, the produce can be at least one of an avocado, a lime, a lemon, an apple, and a mango. The image data can include at least one of color data, an RGB color model, hyperspectral image data, time lapse video data, and time lapse hyperspectral image data.

[0022]

[0022] One or more embodiments described herein may include a system for identifying a duration of a product using image data, the system may include a conveyor system that may be configured to transport the product along a path through a facility, one or more imaging devices located in proximity to the conveyor system and that may be configured to capture image data of the product at consistent time intervals during a time period, where the products are of the same product type, and at least one computing system in communication with the one or more imaging devices. At least one computing system can be configured to receive, from one or more imaging devices, image data of products captured at consistent time intervals during a time period, the image data including one or more processed products that may be coated with a shelf life extension coating solution and one or more unprocessed products that may not be coated with a shelf life extension coating solution during the time period, where the processed products and the unprocessed products are the same product type; perform object detection techniques on the image data to identify bounding boxes around each product in the image data; identify quality attributes within each bounding box at each of the time intervals during the time period; determine one or more duration scoring metrics for each of the products in the image data based on comparing the identified quality attributes over the time period; and transmit the one or more duration scoring metrics for each of the products to a user device for display on a graphical user interface (GUI).

[0023] In some implementations, the system may optionally include one or more of the above features.

[0024]

[0024] The disclosed technology may provide one or more of the following advantages. For example, the disclosed technology may be used to quantitatively assess the effect of a shelf-life extension coating solution on the ripeness, shelf life, edibility, and / or saleability of a produce. Thus, the disclosed technology may be used to improve the shelf-life extension coating solution and / or the application of the solution to the produce. Produce coated with the improved solution may experience improved ripeness, shelf life, edibility, and / or saleability. Further down the supply chain, consumers may be more inclined to purchase the produce, thereby reducing produce-based losses and thus improving the saleability of the produce.

[0025]

[0025] As another example, produce quality can be more accurately determined from subtle differences over time seen in image data. The human eye can be prone to errors in observing subtle changes in produce appearance. Furthermore, the human eye may not be able to quantify and correlate appearance changes with quality attributes such as ripeness, shelf life, edibility, and / or saleability of the produce. The disclosed techniques can provide automatic and accurate detection of different external and internal quality characteristics of produce from image data captured over time (e.g., time lapse image data such as time lapse HSI). More specifically, HSI data can be used to identify and quantify internal quality changes that are not easily visible to the human eye without the use of the present invention. Thus, the disclosed techniques provide a deeper analysis into produce quality while also reducing potential human errors that may result from visual observation and inspection of produce.

[0026]

[0026] As another example, the disclosed techniques can be used to modify shelf-life extension coating solutions to reduce produce-based waste and ensure that the produce delivered to the consumer is desirable for purchase and consumption. By analyzing the ripeness and other quality changes over time of treated produce (e.g., produce coated with a solution) and untreated produce (e.g., produce not coated with a solution), an analysis can be performed to determine improvements that can be made to the shelf-life coating solution. Such improvements can be implemented and applied before the produce is delivered to the consumer in a retail environment. Improving the shelf-life extension coating solution and / or applying the solution earlier in the supply chain can have the advantage of improving the ripeness, shelf life, edibility, and / or saleability of the produce by the time it reaches the consumer.

[0027]

[0027] Similarly, shelf life extension factors identified using the disclosed techniques can be advantageous for retailers to identify the return on investment (ROI) of produce. Such factors can be used by retailers and other relevant stakeholders to determine which produce to order (e.g., processed or unprocessed), which produce to display to customers, how to arrange produce on shelves to reduce produce-based waste (e.g., more ripe produce can be placed before less ripe produce), and / or how to price produce (e.g., based on whether the produce is processed, the ripeness of the produce, other quality characteristics). Thus, the disclosed techniques can be used to monitor the quality of produce and make changes that improve or maintain the quality of the produce throughout the supply chain.

[0028]

[0028] As described throughout, the disclosed techniques can also generate more robust quality assessments of products. Different algorithms, techniques, and machine learning models can be used to identify and score various quality features associated with different types of products. Thus, the disclosed techniques can be used to identify changes in quality features over time that may be difficult for a person to observe and associate with a product's quality features without the present invention.

[0029]

[0029] The details of one or more implementations are set forth in the accompanying drawings and the description below. Other features and advantages will be apparent from the description and drawings, and from the claims. [Brief description of the drawings]

[0030] BRIEF DESCRIPTION OF THE DRAWINGS [Figure 1]

[0030] Conceptual diagram of determining shelf life of produce using time lapse image data. [Figure 2A]

[0031] FIG. 1 is a conceptual diagram for training one or more models to identify quality characteristics of a product using machine learning techniques. [Figure 2B]

[0032] FIG. 1 is a conceptual diagram of training one or more models to identify the shelf life of a product based on identified changes in quality characteristics over time. [Figure 3A]

[0033] 1 is a flow chart of a process for determining the shelf life of a produce. [Figure 3B] 1 is a flow chart of a process for determining the shelf life of a product. [Figure 4]

[0034] 1 is a flowchart of a process for identifying color quality metrics for a production product. [Diagram 5]

[0035] 1 is a flow chart of a process for identifying statistical scale color values ​​for a product. [Figure 6]

[0036] 1 is a flowchart of a process for identifying a volumetric quality metric for a product. [Figure 7A]

[0037] 1 is a flow chart of a process for identifying wrinkle quality metrics for a product. [Figure 7B]

[0038] 7B illustrates the training and run-time application of a machine learning model for identifying wrinkle quality metrics for the product in FIG. 7A. [Figure 8]

[0039] 1 is a flowchart of a process for identifying a quality characteristic metric of a product. [Figure 9]

[0040] FIG. 1 is a system diagram of an example of components used to determine the shelf life of a produce using the techniques described herein. [Figure 10]

[0041] 1 is an example of an environment for capturing time lapse image data of a production product. [Figure 11]

[0042] 1 shows an example of time lapse image data comparing processed and unprocessed products. [Figure 12]

[0043] 1 is an example of a shelf life analysis of avocados using the disclosed techniques. [Figure 13]

[0044] FIG. 6 is an example of a statistically scaled color determination of an avocado using the process of FIG. 5. [Figure 14]

[0045] FIG. 6 is an example of avocado color analysis over time using the process of FIGS. 3-5. [Figure 15]

[0046] FIG. 7 is an example of volumetric analysis of an avocado over time using the process of FIG. 6. [Figure 16]

[0047] 1 is an example of a volume-based shelf life analysis of avocados using the disclosed techniques. [Figure 17]

[0048] 1 is an example of a shelf life analysis of limes using the disclosed techniques. [Figure 18]

[0049] 1 illustrates an example technique for extracting apples or other produce from image data. [Figure 19]

[0050] 1 is an example of shelf-life analysis of apples using the disclosed techniques. [Figure 20]

[0051] 13 is a flowchart of a process for determining a quality metric, machine learning model, and / or quality threshold to select for a type of product being imaged over time. [Figure 21]

[0052] 1 is a flow chart of a process for comparing the shelf life of a treated product to the shelf life of an untreated product to determine one or more modifications to a shelf-life extension coating solution. [Figure 22]

[0053] FIG. 1 is a block diagram of system components that can be used to implement a system for assessing the quality of one or more food products. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0031] Detailed Description

[0054] The present disclosure relates to a system, method, and computer program for assessing the quality of produce from time-lapse image data. The produce can be any type of produce, including but not limited to fruits and vegetables. The disclosed techniques can also be used for quality assessment of other foods, such as meat and fish. The image data can be HSI captured at certain pre-defined time points of the same batch of produce. The image data can be image data of other times, including but not limited to video. The disclosed techniques can provide for obtaining image data of processed and unprocessed produce and identifying the ripeness, shelf life, edibility, and / or saleability of the produce based on an evaluation of quality characteristics identified in the image data. The ripeness, shelf life, edibility, and / or saleability of the produce can be identified as a duration score of the produce. The duration score can indicate the duration for which the produce is or will be ripe to eat, the duration for which the produce may be edible before reaching the end of its shelf life and / or saleability, and other characteristics indicative of the duration for which the produce may be desired, purchased, and / or consumed by consumers. The shelf life score of the treated produce can be compared to the shelf life of the untreated produce to determine whether any changes should be made to the shelf life extension coating solution applied to the treated produce that can improve the coating solution and thereby improve the ripening, shelf life, edibility, and / or saleability of the produce treated with the coating solution.

[0032]

[0055] Referring to the figures, FIG. 1 is a conceptual diagram of determining the shelf life of a produce using time-lapse image data. A computer system 110, an imaging device 104, and a user device 112 can communicate (e.g., wired and / or wireless) via a network 114. The computer system 110 can be configured to assess the quality of the imaged produce to assess the duration of the produce (e.g., shelf life, edibility, saleability, ripeness to eat), as described throughout this disclosure. The imaging device 104 can be located in an imaging environment 100. The imaging environment 100 can be located in a storage facility or other environment along a supply chain. The imaging environment 100 can be an enclosure, such as a box, that can be moved around the storage facility (e.g., from room to room), as further described with reference to FIG. 10. In some implementations, the imaging environment 100 can be a room. Additionally, in some implementations, the computer system 110 can be an edge computing device deployed in the imaging environment 100. The computer system 110 can then perform processing of the image data in a fast and efficient manner, thereby reducing network bandwidth congestion and computational resource usage.

[0033]

[0056] The imaging environment 100 may include an imaging device 104 and at least one light source 106. The products 102A-N may be placed within the imaging environment 100 for a duration during which the imaging device 104 captures images of the products 102A-N. A processed 108A product and an unprocessed 108B product may be placed within the imaging environment 100. A batch of the processed 108A product may be selected for sampling, and a batch of the unprocessed 108B product may be selected for sampling. Thus, a sampling of the processed 108A product may represent an entire batch of the processed 108A product, and a sampling of the unprocessed 108B product may represent an entire batch of the unprocessed 108B product. As a result, the computer system 110 may compare the quality changes of both the processed 108A and unprocessed 108B products 102A-N to determine the effectiveness and potential modifications of the shelf life extension coating solution coating the processed 108A product.

[0034]

[0057] The imaging device 104 may be configured to capture (A) image data of the produce 102A-N over a predefined time period. As described herein, the image data may be HSI, thermal, 2D images, and / or video. The predefined time period may vary depending on the type of produce being imaged in the imaging environment 100. For example, avocados may be imaged over 30 days and apples may be imaged over 100 days. The amount of time the produce 102A-N is imaged may vary. The produce 102A-N may be imaged over an amount of time long enough to see a wide variation in at least one characteristic of the produce 102A-N. For example, the avocados may be imaged until the volume of the avocado no longer changes (e.g., no longer shrinks). As another example, the apples may be imaged until all the apples (e.g., both treated and untreated) change color completely. The produce 102A-N may be imaged at regular time lags and / or intervals during the predefined time period. For example, the produce 102A-N may be imaged every 12 hours, every 24 hours, etc., during a 30 day time period. The produce 102A-N may be imaged daily. The frequency of imaging may also be adjusted based on the time of change of the feature of interest of the produce 102A-N. For example, if an apple variety changes color consistently and / or smoothly over an average of 60 days, the sampling rate may be reduced from a daily sampling rate. Instead, for example, the apples may be sampled (e.g., imaged) every 10 days. As another example, if an avocado changes color very quickly, over several days (as shown in FIG. 14), a sampling rate of at least a daily sampling rate may be used. In some implementations, the sampling rate for avocados may be every 2 hours. Thus, the produce 102A-N may be imaged at a rate that may be determined based on the duration of the feature being captured by the image divided by the number of data points required to clearly see the signal indicative of the feature. In some implementations, the number of data points may be 10. One or more ranges may be used, including but not limited to 5 to 15 data points.One or more other ranges are possible. As described herein, the products 102A-N can be avocados, citrus fruits, apples, other types of fruit, and / or other food products that can be processed, unprocessed, stored in a storage facility, and / or transported through a supply chain to a consumer.

[0035]

[0058] The imaging device 104 may transmit the image data to the computer system 110 at time=t (B). The imaging device 104 may transmit the image data at predetermined time intervals over the time period during which the products 102A-N are being imaged. For example, the imaging device 104 may transmit the image data in batches every 24 hours, every 48 hours, etc., over a 30 day time period. The imaging device 104 may also transmit the image data in real time as the image data is captured. For example, if the image data was captured at time=1, the image data may be transmitted to the computer system 110 at time=1.

[0036]

[0059] The computer system 110 may then apply one or more models and / or algorithms to the image data to determine a duration scoring metric for the produce 102A-N at time=t (e.g., when the produce 102A-N was imaged) (C). The duration scoring metric may be a quality characteristic of the produce 102A-N that changes over time and affects the ripeness, shelf life, edibility, and / or saleability of the produce 102A-N. A duration score may be derived from the duration scoring metric using techniques described herein and may be any one or more of senescence, ripeness, shelf life, saleability, quality, duration of shelf life, duration of saleability, and / or days until the produce 102A-N reaches the end of its shelf life or saleability.

[0037]

[0060] By applying models and / or algorithms to the produce image data each time the image data is captured, the computer system 110 can quantify the change in quality of the treated 108A and untreated 108B produce 102A-N and thus the impact of the quality change on the shelf life (e.g., ripeness, shelf life, edibility, saleability) of the produce 102A-N. The impact of the quality change on the shelf life of the produce 102A-N can also be used by the computer system 110 to determine the effect / effectiveness of a shelf life extension coating solution on the treated 108A produce compared to the untreated 108B produce.

[0038]

[0061] As described further herein, the computer system 110 may select one or more models and / or algorithms to apply in step C based on the type of produce 102A-N being imaged. In some implementations, the computer system 110 may analyze one quality characteristic of the produce 102A-N and therefore apply one model and / or algorithm. As an example, the computer system 110 may analyze color metrics of apples to identify and determine a duration score for quality changes in apples. Thus, the computer system 110 may select and apply a particular model and / or algorithm that assesses color metrics of apples or a particular type of apple. In some implementations, the computer system 110 may analyze two or more quality characteristics of the produce 102A-N and therefore apply two or more models and / or algorithms. As an example, the computer system 110 may analyze color metrics and volumetric metrics of avocados to identify and determine a duration score for quality changes in avocados. Thus, the computer system 110 may select and apply one or more models and / or algorithms that assess both color metrics and volumetric metrics of avocados. The computer system 110 may also analyze one or more other quality characteristics, including, but not limited to, external and internal attributes of the produce, such as spoilage, drying, browning, wrinkling, and other quality characteristics.

[0039]

[0062] The computer system 110 may then determine a duration score for the products 102A-N in the image data at time=t (D). The computer system 110 may determine a duration score for the processed 108A product and a duration score for the unprocessed 108B product. As described herein, the duration score may be based on a quantitative assessment of the duration scoring metrics identified in step C.

[0040]

[0063] The computer system 110 may transmit the duration score to the user device 112 (E). The user device 112 may then output the duration score (F). In some implementations, the duration score may be output in the form of a histogram or other graphical diagram. For example, the histogram may provide a comparison of the quality changes (e.g., duration scoring metrics) of the processed 108A and unprocessed 108B produce over a predetermined time period. The duration scores determined in step D may be plotted at time=t over a predetermined time period. The histogram may also include thresholds indicating when a particular type of produce 102A-N is expected to reach the end of its duration (e.g., end of shelf life, saleability, edibility, and / or eating). The thresholds may be determined based on an analysis of edible shelf life, marketing, and saleability information for the particular type of produce 102A-N.

[0041]

[0064] In some embodiments, the duration score may be output in a variety of other ways in step F, including but not limited to a numerical value. For example, the duration score may be output as the number of days remaining of the shelf life of the treated 108A product and the untreated 108B product. The duration score may be output as the number of days of the total shelf life of the treated 108A product and the untreated 108B product. In some embodiments, the duration score may be output as a shelf life extension factor, which is a comparison of the shelf life of the treated 108A product to the shelf life of the untreated 108B product. Thus, the shelf life extension factor may indicate how much the shelf life of the treated 108A product is extended compared to the untreated 108B product because the treated 108A product has been coated with the shelf life extension coating solution.

[0042]

[0065] Optionally, a shelf-life extension coating solution change can be determined (G). Such a change can be automatically determined by the computer system 110 and transmitted and output to the user device 112. In some embodiments, the change can be determined by the relevant stakeholder at the user device 112. If such a change is determined, the change can be implemented automatically, semi-automatically, and / or manually. For example, the change can include coating the unprocessed product in a subsequent batch with the same quantity of shelf-life extension coating solution as was applied to the imaged processed product 108A. Another change can include adjusting the concentration of the shelf-life extension coating solution or otherwise modifying one or more properties / compositions of the solution. Another example change can include applying a different quantity of the solution to the subsequent batch of product. Yet another example change can include applying the solution at one or more different time periods during the aging process of the subsequent batch of product. One or more other changes may be determined and applied to improve the shelf life extension coating solution and / or improve the shelf life score of the structured batch of the product (thereby extending the eating, shelf life, edibility, and / or saleability of the product).

[0043]

[0066] The user device 112 can be a mobile device, smartphone, tablet, laptop, or other computer that can be used by relevant stakeholders in the supply chain. The stakeholders can view the duration scoring metrics and duration scores for each of the processed 108A and unprocessed 108B products over a given time period. As a result, the stakeholders can more accurately assess the effectiveness of the shelf-life extension coating solution applied to the processed 108A product and determine potential changes to be made to the supply chain that would improve the duration scores of subsequent batches of the product. Thus, the output metrics and / or scores can be used by the stakeholders to monitor the quality changes over time of the products 102A-N and, optionally, make one or more changes to the supply chain.

[0044]

[0067] FIG. 2A is a conceptual diagram of training one or more models to identify quality characteristics of a produce using machine learning techniques. Models can be generated for different produce, different types of produce, and different characteristics associated with different products. For example, a model can be generated and trained to identify color metrics for avocados. Another model can be generated and trained to identify color metrics for a specific type of apple, such as apples or Granny Smith apples. Yet another model can be generated and trained to identify color metrics for different types of produce, such as all citrus fruits. Yet another example model can be generated and trained to identify color metrics for any produce, regardless of type. One or more other models can be trained to identify other characteristics of the produce, including, but not limited to, volume, hardness, wrinkles, spoilage, dryness, sugar content, brix, and other internal and external characteristics of the produce. In some embodiments, instead of or in addition to a model, one or more algorithms can be developed to identify and quantify quality characteristics that change over time of the produce and thus affect the duration of the produce (e.g., shelf life, edibility, saleability, and / or eating time).

[0045]

[0068] The training and model generation may be performed by computer system 110. In some implementations, the training and model generation may be performed by one or more other computing systems, devices, cloud-based services, and / or networks of computers and / or devices.

[0046]

[0069] The computer system 110 may receive product training data 200 (A). The product training data 200 may optionally include one or more types of data. For example, the product training data 200 may include product image data 202, product spectral data 204, and / or destructive measurement data 206. Thus, the product training data 200 may include histograms, RGB color models, hyperspectral data, multispectral data, etc., representing a particular product, a product type, different products, different product types, a single product, and / or a batch of products. The product training data 200 may include images of a particular product having some particular characteristic to be modeled, such as color, and images of the same type of product without the particular characteristic to be modeled. The product training data 200 may also include images of a particular product treated with a shelf life extension coating solution and images of the same type of product not treated with the solution. In some embodiments, the product training data 200 may include images of the exterior of the product and / or the interior of the product. The produce training data 200 may also include images of a particular produce at different stages of ripeness and between stages of ripeness.

[0047]

[0070] The product training data 200 can be a robust collection of training data indicative of multiple different characteristics that may be present in one or more types of products over the life of the product. The destructive measurement data 206 can include measurements taken at different quality characteristics of the product. For example, product hardness can be measured using a penetrometer and / or a hardness meter. Measurements captured by the penetrometer and / or hardness meter can be correlated with the product image data 202 and / or the product spectrum data 204 to quantify the product quality characteristics and quality changes.

[0048]

[0071] The product training data 200 may be retrieved from one or more databases or data stores. In some embodiments, the product training data 200 may also be received from imaging data.

[0049]

[0072] The computer system 110 can identify characteristics of the produce from the image data (B). The computer system 110 can identify characteristics indicative of the quality of the produce. For example, the computer system 110 can identify the color of the produce, spoilage, mold, different types of texture, damage, etc. The identified characteristics can also be labeled. The computer system 110 can identify such characteristics at one or more different points throughout the life cycle and / or shelf life of the produce. For example, the produce training data 200 can include images of the produce at certain time intervals throughout the life cycle of the produce. The computer system 110 can then identify one or more characteristics indicative of the quality of the produce at each of the time intervals.

[0050]

[0073] The computer system 110 can then generate a machine learning model of the identified features (C). The model can be generated using machine learning techniques including, but not limited to, convolutional neural networks (CNN). One or more other machine learning techniques may be used to generate and train the model. The computer system 110 can generate a model for each of the identified and labeled features. Thus, during runtime use, each model can be trained to identify a particular feature from image data of the same type of produce. In some implementations, the computer system 110 can generate a model that identifies one or more features associated with a particular type of produce. Thus, each model can be trained to identify one or more features (e.g., color, volume, wrinkles, other quality features) from image data of a particular type of produce, such as avocados.

[0051]

[0074] The generated models may then be output by the computer system 110 (D). Outputting the models may include storing the models in a database, data store, and / or cloud storage. Outputting the models may also include maintaining the models in local memory and / or storage so that they can be easily and quickly retrieved and applied at run-time. Thus, during run-time, based on the type of produce being imaged over a given period of time, one or more of the models may be selected and applied to the image data. One or more of the models may be selected based on one or more features of interest (e.g., quality features) including, but not limited to, color, volume, wrinkles, internal decay, etc., as described throughout this disclosure.

[0052]

[0075] FIG. 2B is a conceptual diagram of training one or more models to identify the shelf life of a produce based on the identified changes in quality characteristics over time. For example, one or more models can be trained to analyze the changes in the quality characteristics identified by application of the model of FIG. 2A and determine a shelf life score for the produce. In some embodiments, instead of or in addition to a machine learning model, a computer system can develop an algorithm to analyze the changes in the quality characteristics over time and determine a shelf life score for the produce. The training and model generation can be performed by the computer system 110. In some embodiments, the training and model generation can be performed by one or more other computing systems, devices, cloud-based services, and / or networks of computers and / or devices.

[0053]

[0076] The computer system 110 can receive product training data 200 (A). The product training data 200 can optionally include product image data 202, product spectrum data 204, and destructive measurement data 206 (see, e.g., FIG. 2A). The product training data 200 can include output 208 of a product feature model. Thus, the computer system 110 can train one or more models to identify quality changes over time and corresponding duration scores of a particular product based on the output 208 of the product feature model (see, e.g., FIG. 2A).

[0054]

[0077] Thus, the computer system 110 can identify a duration score based on the output 208 of the produce characteristic model (B). The computer system 110 can identify duration scores for both processed and unprocessed produce of different types of produce. The computer system 110 can correlate the changes in the quality characteristics at different time periods with different stages and / or levels of duration (e.g., ripeness, shelf life, saleability, edibility) to thereby identify a duration score for the particular produce. The computer system 110 can also train one or more models to identify a shelf life extension factor based on the duration score in step C. The duration score can indicate how much shelf life is remaining for the produce, the total length of shelf life of the produce, etc., as described herein.

[0055]

[0078] The computer system 110 may then output the generated and trained models for the identified duration scoring (D). Outputting the models may include storing the models in a database, data store, and / or cloud storage. Outputting the models may also include maintaining the models in local memory and / or storage so that they can be easily and quickly retrieved and applied at run-time. Thus, during run-time, one or more of the models may be selected and applied to the image data based on the type of product being imaged over a given period of time.

[0056]

[0079] 3A and 3B are flow charts of a process 300 for determining the shelf life of a produce. Process 300 may be performed by computer system 110. Process 300 may also be performed by one or more other computers, computer systems, devices, cloud-based services, and / or networks of computers and / or devices. For example, process 300 may be performed by an edge computing device. For illustrative purposes, process 300 is described from the perspective of a computer system.

[0057]

[0080] Referring to process 300 in both FIG. 3A and FIG. 3B, a computer system may receive image data of a produce at 302 at time=t. Time=t may be a fixed time interval throughout the time period during which time lapse image data of the produce is being captured. In some implementations, citrus fruits may be imaged over a time period of 70 to 90 days. Avocados and similar produce may be imaged over a time period of 30 days. Apples may be imaged over a time period of 80 to 100 days. As discussed above, one or more other time periods may be used to image the produce based on the type of produce and / or particular characteristics of interest of that produce. For example, the produce may be imaged over a time period of 30 days. Image data of the produce may be captured every 24 hours during the 30 days. Thus, time=t may be 0:00, 24:00, 48:00, 72:00, etc. Time=t may be any other time interval during which image data of the produce may be captured during the time period (e.g., 5 times a day for the entire time period, 7 times a day for the entire time period, etc.).

[0058]

[0081] If the process 300 is started for the produce being imaged for the first time, then time=t can be 0 o'clock or when the produce is first imaged. As described herein, the process 300 can be repeated each time=t that image data of the produce is captured until the time period for imaging the produce is over (e.g., 30 days for some types of produce such as avocados). Throughout the time period that the produce is imaged, the produce does not move from the imaging environment. When image data is captured at each time=t, the lighting, camera conditions, and positioning of the produce remain constant. This can be advantageous to ensure that the produce is imaged similarly at each time interval, providing a more accurate comparison and analysis of quality changes over time.

[0059]

[0082] The image data can be any type of image data described herein, including, but not limited to, 2D images, thermal images, hyperspectral images, and / or video feeds. Hyperspectral images can be useful for identifying and quantifying produce quality characteristics that may not be readily visible to human observation. For example, hyperspectral images can be useful for identifying and quantifying internal and external spoilage, internal and external infection, spoilage, dry matter, and pH. RGB images (e.g., 2D images) can be useful for identifying and quantifying features such as color, volume, and wrinkles, for example.

[0060]

[0083] Treated and untreated products of the same type can be captured in the image data. The treated products can be coated with a shelf-life extension coating solution as described herein. Using the disclosed techniques, quality changes identified and quantified in the treated products can be compared to quality changes identified and quantified in the untreated products. Such comparisons can be useful for analyzing and / or modifying the shelf-life extension coating solution so that the duration (e.g., eating time, shelf life, edibility, saleability) of the treated products can be improved. This comparison can also be useful for benchmarking the performance of the shelf-life extension coating solution and objectively comparing the shelf-life extension coating solution to other solutions.

[0061]

[0084] At 304, the computer system may perform object detection techniques to identify a bounding box for each product in the image data. An object detection algorithm and / or a machine learning trained model may be applied to the image data to identify objects indicative of a product in the image data. A bounding box may be formed around each of the identified objects. In some implementations, the computer system may not identify a product category or type of product in the image data. Instead, the object detection algorithm and / or the machine learning trained model may detect any object that is a product in the image data. Additionally, in some implementations, the computer system may identify a product or type of product when identifying a bounding box for each product in the image data.

[0062]

[0085] The computer system may also identify a grid structure based on the bounding boxes at 306. Identifying the grid structure may include assigning an identifier to each bounding box in the image data. The identifier may be used to identify each of the artifacts in the image data and associate quality changes found in each bounding box with each artifact. For example, a first bounding box may be a first cell in the grid structure and may be assigned an identification value of 0. As another example, a second bounding box may be a second cell in the grid structure and may be assigned a value of 1. An artifact in the first bounding box may be identified via a grid index of 0, and an artifact in the second bounding box may be identified via a grid index of 1. In some implementations, a coordinate system may be used to annotate or index the location of each of the artifacts in the image data. Numeric values ​​may be used to represent the location of the artifact in the image data. For example, a bounding box of an artifact may have four numeric values, such as (xmin, xmax, ymin, ymax) or (x, y, width, height). Subsequent processing steps can use the numerical values ​​representing the product locations to identify the outer boundaries of each product and a product's particular duration scoring metric (eg, quality attribute).

[0063]

[0086] The computer system can then select a bounding box for the produce at 308. Ultimately, the computer system can identify a quality change for each produce in the image data at time=t. The computer system can determine an overall quality change for the produce based on an aggregation of the quality changes for each individual produce. More specifically, the computer system can determine an overall quality change for the treated produce and an overall quality change for the untreated produce. Those overall quality changes can then be compared to analyze the effect of the shelf-life extension coating solution on the duration of the produce (e.g., shelf life, the length of time the produce is fresh, ripe, or otherwise desirable for human consumption).

[0064]

[0087] The computer system can then extract the produce from the background within the bounding box (310). In some implementations, the computer system can implement techniques, algorithms, and / or models to identify channels where the foreground can be separated from the background by thresholding to obtain a mask. The mask can then be applied to the full image (e.g., a three channel image) to extract the foreground representing the produce. As an illustrative example, each channel can include a [0-255] numerical value for each x,y location in a grid structure. For example, the avocado does not have much blue, while the background has a lot of blue, so the blue channel can better separate the avocado from the background. Thus, in the blue channel, the avocado pixels can get a low value close to 0, while the background can get a high value, such as 150 and other values ​​closer to 255. A threshold can be identified around 80, as an illustrative example, and applied to create a binary mask of all pixels in the image that have a value less than 80. The mask can be applied to the full image to crop out the RGB pixels on the avocado.

[0065]

[0088] In some implementations, the computer system can optionally implement techniques, algorithms, and / or models that map all RGB values ​​within the bounding box to a 3D space or other color space, identify clusters of RGB values, select one or more clusters that are indicative of a product, and then extract the selected one or more clusters from the image data. For example, the computer system can train a clustering model to predict the number of clusters for each data point in the 3D space. Depending on the clustering technique used, the clustering can be based on spheres in the 3D space (e.g., k-means clustering) and / or more complex multi-dimensional Gaussians (e.g., Gaussian mixture models).

[0066]

[0089] At 312, the computer system can apply one or more machine learning trained models of the extracted produce. The computer system can select one or more models to apply to quantify the quality change of the produce. The computer system can select the model based on the type of produce being imaged. See FIG. 20 for further discussion. In some embodiments, the computer system can select one or more algorithms to apply to assess and quantify the quality change of the produce in addition to or instead of the machine learning model. The computer system can also identify one or more threshold levels of a particular type of produce that indicate a duration (e.g., end of shelf life, end of saleability, etc.), as described above. Once the model, algorithm, and / or threshold are selected, they can be applied to the extracted produce within the bounding box at block 312. In some embodiments, the model, algorithm, and / or threshold can be selected once when the produce is first imaged (e.g., at time=0 or time=1). The same model, algorithm, and / or threshold can then be applied to any subsequent image data of the produce captured during the time period the produce is imaged. Furthermore, the same models, algorithms, and / or thresholds may be applied to processed and unprocessed products of the image data.

[0067]

[0090] The computer system can determine 314 a duration scoring metric for the extracted produce at time=t. The duration scoring metric can be determined based on application of a model and / or algorithm for a particular produce. The duration scoring metric can be determined based on an analysis of pixels in the image data representing the extracted produce. As further described below, the duration scoring metric can be various quality attributes that change over time. Thus, the duration scoring metric can include firmness, color, volume, spoilage, dry matter, wrinkles, and other external and internal quality features. Color changes can indicate the quality of the produce, for example, when the produce reaches a color that discourages consumers from purchasing the produce (e.g., when a lime turns yellow, the consumer is unlikely to purchase). As another example, if the produce shrinks over time (e.g., the volume of the produce decreases), the produce is deteriorating and therefore becoming less desirable for purchase and consumption by consumers. These are examples of duration scoring metrics that can be quantified using the techniques described herein.

[0068]

[0091] In some implementations, the computer system may determine one duration scoring metric, such as color. In yet some implementations, the computer system may determine multiple duration scoring metrics, such as color and volume. The number of duration scoring metrics identified may depend on the type of artifact captured in the image data. See Figures 4, 6, 7A, 7B, and 8 for further discussion of identifying duration scoring metrics.

[0069]

[0092] The computer system can then store the duration scoring metrics of the produce using the grid index of the produce at 316. The computer system can later retrieve the duration scoring metrics to analyze changes in the duration scoring metrics over a time period for a particular produce. For example, the computer system can retrieve the duration scoring metrics of both treated and untreated avocados over a 30 day period during which the same batch of treated and untreated avocados are imaged. The computer system can graph the duration scoring metrics of both treated and untreated avocados in a curve plot, histogram, or similar graphical diagram. The computer system can then compare the graphed duration scoring metrics to identify how the duration of the treated produce changes or improves compared to the untreated produce (see, for example, Figures 12 and 14).

[0070]

[0093] Optionally, the computer system may output the duration scoring metric of the product (320). The computer system may transmit the duration scoring metric to the user device for presentation in a GUI on a display screen of the user device (see, e.g., FIGS. 1 and 10). In some implementations, the computer system may be the same as the user device, and the computer system may simply output the duration scoring metric on a display screen of the computer system.

[0071]

[0094] As described throughout this disclosure, the duration scoring metric can be output in a curve plot, histogram, or other similar graphical representation (see, e.g., FIG. 12, 14-18). The output of the duration scoring metric can be advantageous to have a visual representation of the quality change over time for both processed and unprocessed products of the same type over the time period the products are imaged. Such a visual representation can advantageously show how the duration of the processed product increases compared to the duration of the unprocessed product.

[0072]

[0095] In some embodiments, outputting a duration scoring metric may also include identifying and outputting an extension factor. The extension factor may be a multiplication factor indicating the extent to which the duration of a treated product is extended compared to an untreated product. In other words, the extension factor may be an average of the shelf life ratios (e.g., duration scoring metric) of treated and untreated products over a statistically representative set of products (see, e.g., Figures 16, 17, 19).

[0073]

[0096] Further, in some embodiments, outputting the duration scoring metric can include determining an overall duration score for the treated product and an overall duration score for the untreated product. The overall duration score can then be outputted and presented to relevant stakeholders and used to analyze how the shelf-life extension coating solution improves the duration of the treated product. For example, the overall duration score can be used to determine how the shelf life is extended at each individual duration metric. As an illustrative example, the relevant stakeholders can determine that in terms of color retention, the shelf-life extension coating solution performs well, but in terms of volume retention, the shelf-life coating solution can be improved to retain the volume of the treated product longer.

[0074]

[0097] Optionally, the computer system may also determine a change to the shelf-life extension coating solution of the product based on the duration scoring metric (322). Once the time period of imaging the product is complete, the computer system may determine whether a change can or should be made to the shelf-life extension coating solution applied to the processed product in the image data. For example, block 322 may be executed when time=t is the last day and / or last hour of the time period (e.g., day 30 of a 30 day period of imaging the product).

[0075]

[0098] In some embodiments, block 322 can be performed throughout the time period that the product is imaged, such as at predetermined time intervals. For example, if the product is imaged for 30 days, then every 5 days the computer system can determine whether a change should be made to the coating solution.

[0076]

[0099] If a change is determined in block 322, the change can be implemented (e.g., manually by relevant stakeholders in the supply chain and / or semi-automatically by the computer system or another computer system). The change can be implemented in real time as it is determined. The change may be implemented at a later time. Implementing the change can include changing the composition of the shelf-life extension coating solution. Implementing the change can also include applying a different concentration of coating solution to a subsequent batch of the same and / or different type of produce as the imaged produce. The coating solution change may be implemented on a batch of produce that is not imaged. For example, the modified coating solution may be applied to a produce that is being shipped from a storage facility and in transit to an end consumer in a retail environment. Application of the modified coating solution to a produce can be beneficial to improving the shelf life (e.g., ripeness, shelf life, edibility, saleability) of the produce. Improved shelf life can improve consumer satisfaction, be beneficial to a retail environment, and lead to less produce-based waste. Modifications to the shelf-life extension coating solution can be beneficial in optimizing the duration of the produce (e.g., shelf life, the length of time that the produce is fresh, ripe for eating, or otherwise desirable for human consumption). Furthermore, improvements in the duration score and shelf-life extension ratio strengthen the assertion that the shelf-life extension coating solution does in fact improve the duration of the produce.

[0077]

[0100] Still referring to process 300, the computer system can determine (324) whether there are more artifacts in the image data at time=t. If there are more artifacts in the image data, the computer system can return to block 308 and repeat blocks 308-322 for any remaining artifacts in the image data. Thus, the computer system can determine a duration scoring metric for each artifact in the image data. The computer system can also determine an aggregate duration scoring metric for processed artifacts and an aggregate duration scoring metric for unprocessed artifacts in the image data at time=t. If there are no more artifacts in the image data, the computer system can return to block 302 and repeat blocks 302-322 for all remaining time periods (e.g., t=24 hours, 48 ​​hours, 72 hours, etc.) during the time period during which artifacts are imaged (e.g., 30 days, 35 days, 40 days, 45 days, 50 days, etc.). Thus, the computer system can repeat process 300 until artifacts are imaged over the time period.

[0078]

[0101] FIG. 4 is a flow chart of a process 400 for identifying a color quality metric for a product. The process 400 may be performed as part of determining a duration scoring metric for an extracted product in block 314 of the process 300 of FIG. 3A and FIG. 3B. In other words, a duration scoring metric for a particular product may be a color quality metric, which is identified using the process 400. The process 400 may be performed by the computer system 110. The process 400 may also be performed by one or more other computers, computer systems, devices, cloud-based services, and / or networks of computers and / or devices. For example, the process 400 may be performed by an edge computing device. For illustrative purposes, the process 400 is described from the perspective of a computer system.

[0079]

[0102] Referring to process 400, a computer system may identify color values ​​of all pixels in the extracted product at time=t (402). The computer system may identify a color value for each pixel of the extracted product. In some implementations, the computer system may determine an average color value of all pixels representing the extracted product.

[0080]

[0103] The computer system may calculate 404 a distance between each color value of the artifact in the image data and the statistical scale (e.g., mean, mode, median, or quantile of some other distribution) color value at t=1. The distance may be calculated as a Euclidean distance. The computer system may compare the color values ​​on a pixel-by-pixel basis by comparing the color value of each pixel to the statistical scale color value. In some implementations, the computer system may compare the average color value of all pixels to the statistical scale color value at t=1.

[0081]

[0104] The statistical scale color value, in some embodiments, can represent the average color of the product when the product is first imaged, t=1. For further discussion of statistical scale color values ​​of a product, see FIG. 5. By calculating the distance between each color value of the product and the statistical scale color value, the computer system can determine how much the color of the product has changed at each time=t since t=1 (when the product was first imaged). The change in color over time can indicate the age of the product, as described herein.

[0082]

[0105] The computer system can then determine whether the distance between each color value and the statistical scale color value exceeds a threshold distance (406). A threshold range can be determined based on each produce category. An analysis can be performed to determine the range of colors within which a particular produce is edible and / or salable. A smaller distance can indicate a smaller change in color from t=1 (when the produce is first imaged) to time=t (whatever the time interval is when the images were captured). A smaller change in color can indicate a longer duration (e.g., longer shelf life, edibility, salability). A larger distance can indicate a larger change in color from t=1 to time=t. A larger change in color can indicate a shorter duration (e.g., shorter shelf life, edibility, salability). Thus, a larger change in color can indicate that the produce is reaching the end of its duration (e.g., the produce is stale and therefore reaching the end of its shelf life). When using color distance, a threshold range can be set based on the distance between the initial statistical scale color value and a threshold color (the threshold color can be determined based on an analysis of the produce category). For example, in the case of Granny Smith apples (as shown in FIG. 19), the end of shelf life can be identified as when the apple turns yellow. This information can be used to calculate the distance between the apple's green color on day 1 and the yellow color that determines the end of the apple's shelf life. This calculated distance can be used as the threshold range for the color distance described herein.

[0083]

[0106] In some embodiments, in addition to or instead of determining whether the distance between each color value and the statistical scale color value exceeds a threshold range, the computer system can output the distance in a graphical diagram (e.g., a curve plot, a histogram, a line graph, etc., as shown in Figures 15-19). The computer system can then identify the change in color value over time at each different time = t during the time period. In some embodiments, a user can analyze the graphical diagram to identify the change in color value over time. By visually showing the change in color value over time, a user can easily compare the duration of a treated product to an untreated product.

[0084]

[0107] If the distance between each color value and the statistical scale color value exceeds a threshold range at 406, the computer system can identify the end of shelf life and / or saleability of the extracted produce at 408. In other words, the distance may be so great that the produce is no longer desirable for purchase and / or consumption by consumers. In some embodiments, the computer system may not make this determination until after the produce has been imaged for the entire time period (e.g., for avocados, all 30 days). The computer system can then make this determination by comparing the distances identified at each time=t during the time period and output in a graphical diagram.

[0085]

[0108] In some implementations, the computer system may use absolute color as an alternative metric to the color distance discussed above, in which case the computer system can monitor how the color of a product moves in 3D color space to determine if the color has moved into an unsalable region.

[0086]

[0109] The computer system may also output the extracted color values ​​over time of the product at 410. As mentioned above, the computer system may output the color values ​​and / or the distance between each color value and the statistical scale color value in a graphical diagram, such as a line graph and / or a histogram. A graphical diagram may be preferred by a user to easily identify and quantify the differences in duration between treated and untreated products.

[0087]

[0110] Referring back to 406, if the distance does not exceed the threshold range, the computer system may store (412) the extracted color value of the product at time=t. In other words, the computer system may determine that the color of the product has not deviated significantly from the original color of the product at t=1 and identify the product as reaching the end of the duration. Optionally, as described above, the computer system may output the color value and / or the distance in a graphical diagram. The graphical diagram may show the color change over time of both the processed product and the unprocessed product. The computer system may then return to block 302 of process 300 of FIG. 3A and FIG. 3B. The process 400 may be repeated to show the color change over time of each of the products in the image data at time=t. As mentioned above, in some embodiments, the process 400 may be performed to identify the average color change over time of the processed product in the image data at time=t and the average color change over time of the unprocessed product in the image data at time=t.

[0088]

[0111] Additionally, in some implementations, process 400 may be performed using one or more machine learning models. Additionally, in some implementations, process 400 may be performed using one or more algorithms or other computer processes and / or techniques.

[0089]

[0112] 5 is a flow chart of a process 500 for identifying a statistically quantified color value of a produce, such as an average color value. As described above, the process 500 can be performed once when the produce is first imaged (e.g., at t=1, which can be the first day of imaging for a 30-day period). The first time t=1 can represent the beginning of the produce's lifespan, such as the beginning of a ripeness period, storage period, edibility, and / or saleability. The first time t=1 can also represent any other time during the lifespan of the produce that is first imaged using the techniques described herein. The statistically quantified color value can be used to compare how much the produce's color has deviated from t=1 at each time interval that the produce is imaged throughout the produce's lifespan (e.g., time period).

[0090]

[0113] Process 500 may be performed as part of block 404 of process 400 of Figure 4. Process 500 may be performed by computer system 110. Process 500 may also be performed by one or more other computers, computer systems, devices, cloud-based services, and / or networks of computers and / or devices. For example, process 500 may be performed by an edge computing device. For purposes of illustration, process 500 is described from the perspective of a computer system.

[0091]

[0114] Referring to process 500, a computer system may identify (502) color values ​​of all pixels of the product within all bounding boxes of the image data at t=1. For example, when the product is initially imaged, the computer system may apply one or more models and / or algorithms to the image data to extract pixels that identify the product. The computer system may identify color values ​​of pixels of both the processed and unprocessed product in the image data. Thus, subsequent color changes of both the processed and unprocessed product may be compared to the same statistically measured color values.

[0092]

[0115] The computer system can then calculate (504) a statistical scale color value for all of the products in the image data at t=1. The computer system can take the average of all of the identified color values ​​from block 502. In some implementations, the computer system can also find the average value of all of the identified color values ​​from block 502. In some implementations, the computer system can calculate the mean, median, quantile, or other metric for each product separately. The computer system can then track the distance of each product's metric from the statistical scale color value for the product at t=1. In yet another implementation, the computer system can calculate, at each time t, the median of each product's pixel-by-pixel distance from its own pixel-by-pixel color at t=1. Thus, instead of averaging colors over multiple pixels and then calculating the distance between the average values, the computer system can calculate the distance of each pixel from the value the pixel had at t=1, which can provide a distance matrix, which can then be reduced to a single number by taking the mean, median, or other quantile metric. One or more other mathematical processes may be used to determine the statistical scale color values ​​of the treated and untreated products.

[0093]

[0116] The computer system may output the statistical scale color value at 506. As described above, the statistical scale color value is then used in the process 400 to compare the aged color value of the product to the statistical scale color value. How the color of the treated and untreated products deviates from the statistical scale color value over time may indicate a duration score for each of the treated and untreated products, such as how much shelf life is remaining for the treated and / or untreated products. See FIG. 13 for further discussion of determining the statistical scale color value.

[0094]

[0117] FIG. 6 is a flow chart of a process 600 for identifying a volumetric quality metric of a product. The process 600 can be performed as part of determining a duration scoring metric of an extracted product in block 314 of the process 300 of FIG. 3A and FIG. 3B. For example, the duration scoring metric identified using the process 600 can be the volumetric change of the product (e.g., the % shrinkage of the product over time). In some embodiments, the process 600 can be performed to identify the volume of the product at t=1, which is the time when the product was first imaged (e.g., day 1). Using the process 600, the computer system can then identify the volume of the product at each time interval of imaging and then compare the identified volume to the volume of the product at t=1. The computer system can also output the volumetric change over time, for example, as a line graph, histogram, or other graphical representation, as described throughout this disclosure.

[0095]

[0118] The process 600 can be performed to identify the volume change of each of the produce in the image data. The process 600 can also be performed to identify the average volume change of the treated produce and the average volume change of the untreated produce. As described throughout, the average volume change of the treated produce can be compared to the average volume change of the untreated produce over time to quantify how application of the shelf-life extension coating solution to the treated produce affects (e.g., improves) the duration of the treated produce. The shelf-life extension coating solution can retain moisture and block oxygen from entering the produce. Thus, the shelf-life extension coating solution can prevent oxidation and moisture loss, meaning that the produce shrinks (e.g., loses volume) more slowly than the untreated produce. Identifying the volume change can be an important and useful factor for identifying the duration (e.g., shelf life) of the produce. Volume can be a good indicator of the duration of some types of produce, such as citrus fruits (e.g., lemons and limes) and produce such as avocados.

[0096]

[0119] Process 600 may be performed by computer system 110. Process 600 may also be performed by one or more other computers, computer systems, devices, cloud-based services, and / or networks of computers and / or devices. For example, process 600 may be performed by an edge computing device. For purposes of illustration, process 600 is described from the perspective of a computer system.

[0097]

[0120] Referring to process 600, the computer system may determine 602 an area of ​​the extracted product at time=t. The area may be determined based on counting the number of pixels in the extracted product. For example, the computer system may apply an algorithm to count the number of extracted pixels of a particular product once the product has been extracted from the background of the image data in process 300.

[0098]

[0121] Optionally, the computer system can determine a radius of the extracted product based on the area (604). The computer system can model the shape of the product to accurately calculate the volume of the product. For example, for a straight cucumber, the cucumber can be fitted to an ellipse in 2D space and then the volume of the ellipsoid can be calculated. Alternatively, the cucumber can be modeled as a rectangle in 2D space and then the volume of a rectangular prism can be calculated. In some example implementations, the radius is calculated as Area = πr 2The radius can be found using the formula: V=4 / 3πr This formula can be used for spherical produce or other produce that can approximate a sphere, such as citrus fruits, apples, and some avocados. The computer system can then use the formula to solve for this radius. The computer system can then determine 606 the volume of the extracted produce at time=t. For example, once the computer system optionally identifies the radius in 604, the computer system can determine 606 the volume of the extracted produce at time=t. As mentioned above, the computer system can use the formula for the volume of a sphere: V=4 / 3πr 3 The computer system may use a spherical area and volume formula for relatively round produce, such as avocados, apples, limes, lemons, and other similar produce. Depending on the relative shape of the produce, one or more other area and volume formulas may be selected and utilized.

[0099]

[0122] The computer system can compare the volume at time=t to the volume of the product at t=1 (607). As mentioned above, at t=1 when the product is first imaged, the product can be at 100% volume. Over time, the product shrinks. As an example, the product can be at 98% of its original volume when imaged on the second day. The product can be at 90% of its original volume when imaged on the fourth day, and so on. Thus, the computer system can analyze and / or determine the relative volume of the product compared to the starting volume over the time period the product is imaged. In some embodiments, comparing the volume at time=t to the volume of the product at t=1 can include outputting the volume per time period the product is imaged in a graphical representation, such as a line graph (see, e.g., FIG. 12, FIG. 15). Comparing the volume at time=t to the volume of the product at t=1 can also include outputting the percent volume change per time period the product is imaged in a graphical representation, such as a line graph.

[0100]

[0123] The computer system can determine whether the volume change exceeds a threshold range at 608. In other words, the computer system can determine whether the produce has shrunk by a predetermined amount indicating that the produce is approaching or at the end of its duration (e.g., eating time, shelf life, edibility, and / or saleability). Moreover, the computer system can make such determinations for both treated and untreated produce. As a result, the computer system and / or relevant stakeholders can compare the effect that shelf-life extension coating solutions have on improving the duration of the produce.

[0101]

[0124] If the volume change exceeds a threshold range, the computer system can identify the end of shelf life and / or saleability of the extracted product at 610. As mentioned above, the computer system can determine that the product has reached or is reaching the end of its lifespan, eventually shrinking or decreasing in volume / size such that it is no longer desirable for purchase and / or consumption by consumers.

[0102]

[0125] The computer system may also output the volume of the extracted product over time at 612. As mentioned above, the computer system may output the percent volume change from t=1 in a graphical plot, such as a line graph. See Figures 12 and 15 for further discussion.

[0103]

[0126] If the volume change does not exceed the threshold range at 608, the computer system may store (614) the volume of the extracted product at time=t. The stored volume change may later be retrieved and compared and / or analyzed with the volume changes of both treated and untreated products of the same type at different time periods. The computer system may output a graphical representation of the volume changes of the treated and untreated products to provide a visual comparison of the effect of treating the product with the shelf-life extension coating solution. The computer system may then return to block 302 of process 300.

[0104]

[0127] In some implementations, process 600 may be performed using one or more machine learning models. Additionally, in some implementations, process 600 may be performed using one or more algorithms or other computer processes and / or techniques.

[0105]

[0128] FIG. 7A is a flow chart of a process 700 for determining wrinkle quality metrics for a produce. The process 700 can be performed as part of determining the age scoring metric for the extracted produce in block 314 of the process 300 of FIG. 3A and FIG. 3B. Thus, the computer system can determine the age of the produce based on whether the produce shows wrinkles and / or the degree of wrinkles of the produce. The more wrinkled the produce is, the closer it is to the end of its age (e.g., end of ripeness, shelf life, edibility, saleability). As described throughout this disclosure, the computer system can determine how wrinkled each produce is, including processed and unprocessed produce. In some embodiments, the computer system can determine the average wrinkle indication of all of the processed produce that are imaged and the average wrinkle indication of all of the unprocessed produce that are imaged. Process 700 can be performed to identify how application of a shelf life extension coating solution to a product (e.g., a treated product) causes the product to wrinkle more slowly than an untreated product, thereby extending the life of the product.

[0106]

[0129] Process 700 may be performed by computer system 110. Process 700 may also be performed by one or more other computers, computer systems, devices, cloud-based services, and / or networks of computers and / or devices. For example, process 700 may be performed by an edge computing device. For purposes of illustration, process 700 is described from the perspective of a computer system.

[0107]

[0130] Referring to process 700 of FIG. 7A, a computer system can identify a grid structure of the extracted product at time=t (702). As another example, the computer system can annotate wrinkles across the product image to create a Boolean mask of wrinkle locations. An image segmentation model can be trained to predict the Boolean mask (and thus the wrinkle locations). The model prediction can be used to calculate the percentage of wrinkled area of ​​the product. Once the computer system extracts the product from the background in the image data in process 300 of FIG. 3A and FIG. 3B, the computer system can generate a grid structure of the product, where each grid is assigned a grid index and where each grid overlaps a portion of the product.

[0108]

[0131] The computer system can retrieve 704 the machine learning trained wrinkle models. Wrinkle models can be generated and trained for each different type of produce that may develop wrinkles. For example, one wrinkle model can be generated and trained for mangoes. Another wrinkle model can be generated and trained for bell peppers.

[0109]

[0132] At 706, the computer system may apply the model to each grid cell in the grid structure to identify one or more wrinkles at time=t. As further described in FIG. 7B, the model may be trained to identify whether one or more wrinkles are present in each grid in the grid structure that constitutes the image data of the product. If a wrinkle is identified in a grid, the grid may be assigned a corresponding value, such as 1 if wrinkles are present and 0 if wrinkles are not present. One or more other values ​​may also be assigned, including, but not limited to, string values ​​(e.g., yes if wrinkles are present and no if wrinkles are not present), Boolean values ​​(e.g., true if wrinkles are present and false if wrinkles are not present), and / or other numerical values ​​including percentages and / or proportions.

[0110]

[0133] Once the model is applied to each grid cell in the grid structure representing the produce, the computer system can calculate the proportion (or percentage) of grid cells that are wrinkled and determine whether the percentage exceeds some threshold range (708). The computer system can count how many grid cells are assigned a value of 1 or other value that indicates wrinkles. In some embodiments, the computer system can identify the percentage of the produce that is wrinkled. If the percentage of the produce that is wrinkled exceeds the threshold range, it can indicate that the produce is approaching or at the end of its lifespan. In some embodiments, the threshold range can vary depending on the type of produce and the characteristics specific to the produce. For example, for mangoes, the threshold range can be 5%, indicating that mangoes with more than 5% wrinkles on their surface are not salable.

[0111]

[0134] If the percentage of wrinkled grid cells exceeds a threshold range, the computer system can identify the end of shelf life and / or saleability of the extracted product (710). Eventually, the computer system can identify that the product is no longer desired for purchase and / or consumption by the consumer. The product may have too many wrinkles and is therefore at the end of its life. The computer system can also output a wrinkle indication of the extracted product at time=t (712). As described throughout this disclosure, the computer system can output a wrinkle indication of the product over the time period during which the product is imaged in a graphical illustration, such as a line graph. Thus, relevant stakeholders can compare how slowly the product wrinkles over time when treated with the shelf life extension coating solution compared to an untreated product.

[0112]

[0135] If the percentage of grid cells with wrinkles identified in 708 is less than the threshold range, the computer system may store (714) the percentage of grid cells with wrinkles at time=t of the product. The computer system may also output the stored wrinkle indication in a graphical diagram, such as a line graph. The computer system may then return to block 302 of process 300.

[0113]

[0136] FIG. 7B illustrates the training and run-time application of a machine learning model to identify wrinkle quality metrics for the produce in FIG. 7A. A wrinkle model can be trained to identify wrinkles in any type of produce (whether the produce is processed or unprocessed). A wrinkle model can also be trained to identify wrinkles in a particular type of produce. For example, a wrinkle model can be trained to identify wrinkles in mangoes. Another wrinkle model can be trained to identify wrinkles in bell peppers. One or more other wrinkle models can be trained to identify wrinkles in wrinkled produce, including, but not limited to, cucumbers, apples, pears, and passion fruit.

[0114]

[0137] The wrinkle model can be a convolutional neural network (CNN). In some implementations, the wrinkle model can be one or more other types of neural networks and / or machine learning techniques / algorithms. The wrinkle model can utilize a wrinkle classifier to identify patches of the product, and then use the classification results to calculate the percent coverage. As described throughout this disclosure, if the percent coverage exceeds a certain predefined threshold range, the product can be determined to be nearing the end of its lifespan. In other words, the product can be so covered with wrinkles that it is no longer desirable for purchase and / or consumption by a consumer.

[0115]

[0138] During wrinkle model training 720, a classifier in wrinkle model 721 may be trained to distinguish wrinkled product patches from non-wrinkled product patches. Model 721 may be trained using annotated / labeled image training data, such as wrinkled product image data 722 and non-wrinkled product image data 726. Image data 722 and 726 may be patches (e.g., grids) of a product. Such patches may be labeled (automatically by a computer system and / or manually by a human operator / user) as wrinkled (image data 722) and non-wrinkled (image data 726). Model 721 may then be trained to distinguish wrinkles in image data 722 compared to the absence of wrinkles in image data 726. Thus, if trained model 721 identifies a wrinkle in image data 722, etc., it may output a value 724, which may indicate a positive identification of wrinkles or the presence of wrinkles in a particular patch of the product. Value 724 may be a numeric / integer / float such as 1. Value 724 may be a string value and / or a Boolean value as described above. If model 721 does not identify a wrinkle in a patch of the product, such as in image data 726, it may output value 728, which may indicate that no wrinkles are present in the particular patch of the product. Value 728 may be a numeric / integer / float such as 0. Value 728 may be a string value and / or a Boolean value as described above.

[0116]

[0139] During runtime application 730, a bounding box 732 may be selected that represents a produce in the produce image data, as described above with reference to process 700 of FIG. 7A. Here, the produce is a mango, and bounding box 732 is a portion of the image data that represents an entire mango. A computer system, such as computer system 110, may generate a grid structure of the bounding box 732 of the mango (block 702 of process 700). Thus, a grid structure 734 may be overlaid on the bounding box 732 that represents the mango. The computer system may then apply model 721 to each grid (e.g., patch) in grid structure 734 to classify the grid as wrinkled or wrinkle-free (blocks 706-708 of process 700). The generated output 736 may include a visual representation of where wrinkles were identified on the produce and / or a wrinkle coverage percentage. For example, in the example output 736 of FIG. 7B, wrinkles were identified in four patches (e.g., grids). The four wrinkled patches are calculated to be about 5% wrinkle coverage. Depending on the predefined threshold range, 5% coverage may not be enough to indicate the end of the mango's lifespan. In other words, a mango with only 5% of its surface area wrinkled can still be purchased and consumed by a consumer. The mango may still taste good and / or be of a desirable ripeness. The predefined threshold range may vary for a particular produce. In the case of mango, for example, the threshold range may be 5% for one grade or class of mango and 20% for another grade or class of mango.

[0117]

[0140] FIG. 8 is a flow chart of a process 800 for identifying a quality feature metric for a product. The process 800 may be performed as part of determining the duration scoring metric for the extracted product in block 314 of the process 300 of FIG. 3A and FIG. 3B. The process 800 is similar to the process 700 of FIG. 7A, except that the process 700 is used to identify wrinkles as a quality feature, whereas the process 800 is used to identify any type of quality feature. The process 800 may be performed to identify one or more different types of quality features indicative of the quality change and thus duration of the product. One or more models and / or algorithms may be used to identify the quality features associated with a particular type of product. One or more models and / or algorithms may be used to identify the quality features associated with different types of products. Furthermore, in some embodiments, each model and / or algorithm may be used to identify a particular quality feature regardless of the type of product. Quality characteristics that can be identified and quantified using process 800 can include, but are not limited to, wrinkles, external spoilage, external damage, browning, spots, and other external visual attributes of the produce. Quality characteristics can also include internal attributes, including but not limited to damage, infection, spoilage, hardness, dry matter, pH, Brix, and / or sugar content. As described throughout this disclosure, process 800 can be used to identify quality characteristic changes in both treated and untreated produce. Thus, the quality characteristic changes in treated and untreated produce can be compared to identify differences in the shelf life of the treated produce. Such differences in shelf life can be attributed to the shelf life extension coating solution with which the produce was treated.

[0118]

[0141] Process 800 may be performed by computer system 110. Process 300 may also be performed by one or more other computers, computer systems, devices, cloud-based services, and / or networks of computers and / or devices. For example, process 300 may be performed by an edge computing device. For illustrative purposes, process 300 is described from the perspective of a computer system.

[0119]

[0142] Referring to process 800, similar to process 700 of FIG. 7A, the computer system can retrieve at 804 a machine learning trained feature model. As mentioned above, the model can be trained to identify different types of quality features of a particular type of product imaged. The model can also be trained to identify a particular type of quality feature of a particular type of product. The model can be trained to identify different types of quality features of any type of product. The model can also be trained to identify a particular type of quality feature of any type of product. One or more other variations of the model and / or algorithm are also possible. At 804, the computer system can retrieve one or more models that are selected based on the type of quality feature to identify and / or the type of product imaged.

[0120]

[0143] The computer system can then apply the model to the image data of the product to identify features at time=t (806). As described with reference to Figures 7A and 7B, when the model is applied to the image data, the model can output a value indicating whether the quality feature is present (e.g., detected) or not. The value can be a number / integer (such as 1 if present and 0 if not), a string, and / or a Boolean value. The model can be trained to identify the quality feature using training image data with the quality features so labeled and annotated.

[0121]

[0144] The computer system can determine whether the identified characteristic exceeds a threshold range at 808. The computer system can also identify a quality characteristic coverage percentage indicating how much of the produce is covered with the identified quality characteristic (e.g., see FIG. 7B). The computer system can determine whether the coverage percentage exceeds a threshold range. As an example, if the produce is a lemon that is covered with a significant amount of browning or spoilage (e.g., the coverage percentage in the lemon exceeds a certain predetermined threshold range of browning or spoilage), the lemon may no longer be desirable for purchase and / or consumption.

[0122]

[0145] Thus, if the identified characteristics exceed the threshold range, the computer system can identify the end of shelf life and / or saleability of the extracted product (810). As mentioned above, the product may be approaching or has reached the end of its shelf life. The product may no longer be desirable for purchase and / or consumption. The computer system can also output an indication of the characteristics of the extracted product at 812. For example, the computer system can output what type of quality characteristics are identified and / or how much of the quality characteristics are identified (e.g., percent coverage). The computer system can also output the extracted product using a grid structure. The grid cells having the quality characteristics can be represented with some indicia, such as a highlight. A user viewing this output on a user device can then review the identified quality characteristics and compare the identified quality characteristics of the treated and untreated products. In some embodiments, the computer system can also output a graphical illustration, such as a line graph, showing the percent coverage over time of the treated product and the percent coverage over time of the untreated product. This output can also be beneficially used by a user to compare the effectiveness of applying a shelf-life extension coating solution to treated produce versus untreated produce.

[0123]

[0146] If the identified features are below the threshold range, the computer system may store the identified features of the extracted product at time=t (814). As described above, the computer system may output an indication of the identified features. The computer system may then return to block 302 of process 300.

[0124]

[0147] 9 is a system diagram of an example of components that may be used to determine the shelf life of a produce using the techniques described herein. The computer system 110, the produce information data store 900, and the model data store 954 may be in communication (e.g., wired and / or wireless) over a network 114.

[0125]

[0148] Computer system 110 may include an object detection engine 902, an indexing engine 904, a model training engine 906, a duration scoring metric assessment engine 908, and a duration scoring specifier 910. Although not shown, computer system 110 may include a communications interface. In some implementations, one or more of components 902, 904, 906, 908, and / or 910 may be separate from computer system 110 and may be part of one or more other computer systems, computers, servers, cloud-based services, devices, and / or networks.

[0126]

[0149] The model training engine 906 can be configured to generate one or more models that can be used by the duration scoring metric assessment engine 908. The engine 906 can receive training image data of the produce. The engine 906 can train a model to identify features associated with the produce from the training image data, such as wrinkles and other quality features / attributes. The identified features can be annotated / labeled in the training image data. In some implementations, the trained model can be a CNN.

[0127]

[0150] The generated and trained models can be stored in the model data store 954 as duration scoring metric models 956A-N. The models 956A-N can be accessed and / or retrieved during run-time by one or more analyzers of the duration scoring metric assessment engine 908. As described throughout this disclosure, during run-time, only some of the models 956A-N can be selected based on the type of artifacts in the image data and / or the type of features being identified and quantified in the image data. In some implementations, the models 956A-N can be continually refined and / or updated based on run-time usage of the models.

[0128]

[0151] The object detection engine 902 may be configured to detect one or more products in the image data, as described throughout this disclosure (see, e.g., FIG. 3A and FIG. 3B). The engine 902 may receive the image data and perform object detection techniques to process the image data. The engine 902 may also apply one or more machine learning models trained to identify products in the image data. The engine 902 may generate a bounding box around each of the products in the image data. For each product 944A-N, the engine 902 may store a bounding box image of the product 950 in the product information data store 900. Optionally, in some implementations, the object detection engine 902 may also be configured to identify the type of product in the image data. Thus, the products 944A-N may include a bounding box image 950 of a processed product. Another product 944A-N may include a bounding box image 950 of an unprocessed product.

[0129]

[0152] The indexing engine 904 can be configured to apply a grid structure to the image data and index each bounding box within the structure (see, e.g., FIGS. 3A and 3B). Each bounding box can encompass the product represented in the image data. For each product 944A-N, the engine 904 can retrieve a bounding box image 950 from the product information data store 900 and assign an index value to the bounding box as a grid index 952. The engine 904 can also identify each bounding box as representing a processed or unprocessed product. Instructions for the processing of the product can be stored with the grid index 952 of each product 944A-N. Thus, each food item can be identified by its assigned index and whether the product is processed or unprocessed, which can be beneficial for future retrieval of information, analysis, and processing operations.

[0130]

[0153] The duration scoring metric assessment engine 908 can be configured to identify one or more features indicative of a quality change of the produce in the image data (see, for example, FIGS. 3-8). The engine 908 can retrieve a bounding box image 950 of each food item 944A-N from the produce information data store 900. The bounding box image 950 can be used by an analyzer in identifying the time-dependent characteristic change of the produce, as described throughout this disclosure. The engine 908 can also retrieve and execute one or more models 956A-N from the model data store 954 during runtime. The engine 908 can select the model 956A-N based on the identification made by the object detection engine 902. For example, the engine 902 can determine that the produce identified in the image data are treated apples and untreated apples. The engine 902 can inform the duration scoring metric assessment engine 908 that the produce is an apple. The engine 908 can then retrieve one or more models 956A-N associated with the apple from the model data store 954. The models can be used to analyze the quality changes of the processed and unprocessed produce over the time period during which the apple is imaged.

[0131]

[0154] The duration scoring metric assessment engine 908 can include multiple analyzers, each of which can be configured to identify a different quality characteristic in a produce and / or a different type of quality characteristic in one or more types of produce. Each model 956A-N can be performed by a different analyzer. In some embodiments, one or more of the models 956A-N can be performed by one or more analyzers. Although several example analyzers are shown in FIG. 9, additional, fewer, and / or different analyzers can be included in the engine 908. Thus, the engine 908 can include one or more additional or fewer analyses. Each of the analyzers can receive, for each food product 944A-N, a bounding box image 950 for that produce.

[0132]

[0155] By way of example, the duration scoring metric assessment engine 908 can include a color analyzer 912, a volume analyzer 914, a wrinkle analyzer 916, a hardness analyzer 918, a quality characteristic analyzer 920, a spoilage analyzer 922, an infection analyzer 924, a spoilage analyzer 926, a dry matter analyzer 928, a pH analyzer 930, a Brix analyzer 932, a sugar content analyzer 934, an avocado shelf life analyzer 936, a mango shelf life analyzer 938, an apple shelf life analyzer 940, and a lemon shelf life analyzer 942. Any one or more of the analyzers 912, 914, 916, 918, 920, 922, 924, 926, 928, 930, 932, 936, 938, 940, and 942 can retrieve and apply the duration scoring metric models 956A-N. The output from any one or more of the analyzers can be stored in the product information data store 900 as analyzer metric outputs 946A-N for each product 944A-N. Additionally, as described throughout this disclosure, the duration scoring metric assessment engine 908 can implement only one analyzer to identify the quality change of a product over time. In some implementations, the duration scoring metric assessment engine 908 can implement multiple analyzers to identify the overall quality change of a product over time.

[0133]

[0156] With reference to the exemplary analyzer shown in FIG. 9, the color analyzer 912 can be configured to analyze the bounding box image 950 to identify the color of the produce at each time interval the produce is imaged compared to a reference color (e.g., a statistically scaled color value representing the color of the produce when the produce is first imaged at t=1). The color analyzer 912 can be applied to any type of produce that is imaged. In some implementations, the color analyzer 912 can implement models and / or algorithms specific to the type of produce that is imaged. For example, a model trained to assess color change in lemons can be implemented, while another model can be trained to assess color change in Granny Smith apples. See FIGS. 4 and 5 for further discussion of identifying, quantifying, and outputting color change in imaged produce.

[0134]

[0157] The volume analyzer 914 can be configured to analyze the bounding box images 950 to identify the percent shrinkage of the product volume at each time interval compared to a reference volume (e.g., the volume of the product when the product is first imaged at t=1). The volume analyzer 914 can be used to analyze the volumes of different types of products. See FIG. 6 for further discussion on analyzing, quantifying, and outputting the volume change of an imaged product.

[0135]

[0158] The wrinkle analyzer 916 can be configured to identify and quantify wrinkles in the bounding box image 950 to determine the percent wrinkle coverage of the produce at each time interval. The wrinkle analyzer 916 can be used to analyze wrinkles in different types of produce. The analyzer 916 can implement a model to analyze wrinkles in a particular type of produce, such as mango. Another model can be implemented by the analyzer 916 to identify wrinkles in another type of produce, such as peppers, cucumbers, apples, passion fruit, or other types of produce that wrinkles occur in. See Figures 7A and 7B for further discussion of analysis, quantification, and output of wrinkle changes in imaged produce.

[0136]

[0159] The hardness analyzer 918 may be configured to identify and quantify the hardness of the produce within the bounding box image 950 to identify the change in hardness of the produce at each time interval relative to the reference volume. The hardness analyzer 918 may use one or more models and / or algorithms to identify the hardness of the produce. The hardness may be identified from hyperspectral image data. The hardness may be identified from a combination of image data and destructive measurements made on the produce. The hardness analyzer 918 may generate output data, such as a score, indicative of the hardness level. The output data may be a numeric value in a range such as 0 to 1. A value of 0 may indicate that the depicted product is hard and inedible, while a value greater than 0 and less than 1 may provide an indication of a less hard hardness. For example, an output value closer to 1 may indicate that the product is softer, less hard, and / or that the product is approaching the maximum softness it should be (e.g., approaching the end of the product's lifespan). In some embodiments, depending on the type of produce, maximum softness can indicate that the produce is of good quality and ready to be consumed by the consumer, while in some embodiments, maximum softness can indicate that the produce is of poor quality and is no longer of a desired firmness for purchase and / or consumption by the consumer.

[0137]

[0160] The quality feature analyzer 920 can be configured to identify and quantify one or more quality features in the bounding box image 950 to determine which and how many quality features are present in the produce at each time interval. The analyzer 920 can be used to analyze quality features of different types of produce. See FIG. 8 for further discussion of analyzing, quantifying, and outputting changes in quality features of imaged produce.

[0138]

[0161] The damage analyzer 922, like the quality feature analyzer 920, can be configured to identify and quantify internal and / or external damage found in the bounding box image 950 of the product at each time interval compared to a baseline damage quantity (when the product is first imaged at t=1).

[0139]

[0162] The infection analyzer 924, like the quality feature analyzer 920, can be configured to identify and quantify internal and / or external infections seen in the bounding box image 950 of the product at each time interval compared to a baseline infection quantity (when the product is first imaged at t=1).

[0140]

[0163] The spoilage analyzer 926, like the quality feature analyzer 920, can be configured to identify and quantify internal and / or external spoilage seen in the bounding box image 950 of the product at each time interval compared to a baseline spoilage quantity (when the product is first imaged at t=1).

[0141]

[0164] The dry matter analyzer 928, like the quality feature analyzer 920, can be configured to identify and quantify the dry matter found in the bounding box image 950 of the produce at each time interval compared to a reference dry matter quantity (when the produce is first imaged at t=1). The dry matter analyzer 928 can be used to assess the dry matter in an avocado.

[0142]

[0165] The pH analyzer 930, like the quality feature analyzer 920, can be configured to identify and quantify acid levels from the hyperspectral data, which may be derived from bounding box images 950 of the produce at each time interval. The acid levels can be compared to a baseline pH quantity (when the produce is first imaged at t=1) to identify the age of the produce.

[0143]

[0166] The Brix analyzer 932, like the quality feature analyzer 920, can be configured to identify and quantify the sugar content of the produce from the hyperspectral image, which may be derived from the bounding box image 950 of the produce at each time interval. The sugar content at each time interval can then be compared to a baseline sugar content (when the produce is first imaged at t=1) to identify the age of the produce. Thus, the analyzer 932 can be configured to identify the sweetness and / or astringency of the produce, which can indicate whether and when the produce is desirable for purchase and / or consumption by a consumer.

[0144]

[0167] The avocado shelf life analyzer 936 can be configured to determine the duration (e.g., ripeness, shelf life, edibility, saleability) of the avocados using one or more of the techniques described throughout this disclosure. For example, the avocado shelf life analyzer 936 can identify and quantify the change in color and volume of the avocado over the time period in which the avocado is imaged. The avocado shelf life analyzer 936 can then determine the duration of both the processed and unprocessed avocados based on the color change, the volume change, or a combination of both. The avocado shelf life analyzer 936 can also determine the duration of the processed and unprocessed produce using one or more additional techniques, such as analysis and quantification of wrinkles, hardness, quality characteristics, bruising, infection spoilage, and / or dry matter. Additionally, the avocado shelf life analyzer 936 can be configured to generate an output indicative of the duration of the processed and unprocessed produce. As described throughout this disclosure, the analyzer 936 can generate graphical diagrams, such as line graphs or histograms, that show the change in color and / or volume of both the treated and untreated products over time. This output can then be used by relevant stakeholders at the computer system 110 and / or user devices to determine whether any changes should be made to the shelf-life extension coating solution applied to the treated products.

[0145]

[0168] The mango shelf life analyzer 938 can be configured to determine the duration (e.g., ripeness, shelf life, edibility, saleability) of the mango using one or more of the techniques described throughout this disclosure. For example, the mango shelf life analyzer 938 can identify and quantify color, volume, and / or wrinkle changes of the mango over the time period in which the mango is imaged. The mango shelf life analyzer 938 can then determine the duration of both the treated and untreated mangoes based on color change, volume change, wrinkles, or many combinations of these quality characteristics. The mango shelf life analyzer 938 can also determine the duration of the treated and untreated produce using one or more additional techniques, such as analysis and quantification of firmness, quality characteristics, damage, infection, spoilage, pH, and / or Brix. Additionally, the mango shelf life analyzer 938 can be configured to generate an output indicative of the duration of the treated and untreated produce. As described throughout this disclosure, the analyzer 938 can generate graphical diagrams, such as line graphs or histograms, that show the change in color, volume, and / or wrinkle coverage over time for both treated and untreated products. This output can then be used by relevant stakeholders at the computer system 110 and / or user devices to determine whether any changes should be made to the shelf life extension coating solution applied to the treated products.

[0146]

[0169] The apple shelf-life analyzer 940 can be configured to identify the duration of an apple (e.g., ripeness, shelf life, edibility, saleability) using one or more of the techniques described throughout this disclosure. In some implementations, the analyzer 940 can identify the duration of different types of rings. In some implementations, the analyzer 940 can be configured to identify the duration of a particular type of apple. For example, one apple shelf-life analyzer can identify the duration of Granny Smith apples, and another apple shelf-life analyzer can identify the duration of Honey Crisp apples. The analyzer can use different threshold ranges and / or levels based on the type of apple. For example, the statistical scale color value can be a shade of green for Granny Smith apples and a mix of red and yellow for Honey Crisp apples. Additionally, Granny Smith and Honey Crisp apples can have different levels of sweetness and / or astringency, where the astringency of Granny Smith apples may be desirable to consumers, while a similar level of astringency may be undesirable to consumers for Honey Crisp.

[0147]

[0170] The apple shelf life analyzer 940 can identify and quantify one or more quality attributes that change over the time period during which the apples are imaged. The apple shelf life analyzer 940 can then identify the duration of both the treated and untreated apples based on the one or more quality attributes assessed during the time period. For example, the apple shelf life analyzer 940 can identify the duration of the treated and untreated produce based on the analysis and quantification of any one or more changes in color, volume, hardness, quality characteristics, damage, infection spoilage, pH, and / or sugar content. Additionally, the apple shelf life analyzer 940 can also be configured to generate an output indicative of the duration of the treated and untreated produce. As described throughout this disclosure, the analyzer 940 can generate graphical diagrams, such as line graphs and histograms, that show the change in the quality attributes over time for both the treated and untreated produce. This output can then be used by relevant stakeholders at the computer system 110 and / or user devices to determine whether any changes should be made to the shelf life extension coating solution applied to the treated produce.

[0148]

[0171] The lemon shelf life analyzer 942 can be configured to determine the duration of a lemon (e.g., ripeness, shelf life, edibility, saleability) using one or more of the techniques described throughout this disclosure. In some implementations, the analyzer 942 can also be used to determine the duration of limes or other similar citrus fruits. The lemon shelf life analyzer 942 can identify and quantify the change in color and volume of the lemon over the time period in which the lemon is imaged. The lemon shelf life analyzer 942 can then determine the duration of both treated and untreated lemons based on the color change, volume change, or a combination of both. The lemon shelf life analyzer 942 can also determine the duration of treated and untreated produce using one or more additional techniques such as analysis and quantification of wrinkles, hardness, quality characteristics, damage, infection, spoilage, dry matter, pH, and / or sugar content. Additionally, the lemon shelf life analyzer 942 can be configured to generate an output indicative of the duration of treated and untreated produce. As described throughout this disclosure, the analyzer 942 can generate graphical diagrams, such as line graphs or histograms, that show the change in color and / or volume of both the treated and untreated products over time. This output can then be used by relevant stakeholders at the computer system 110 and / or user devices to determine whether any changes should be made to the shelf-life extension coating solution applied to the treated products.

[0149]

[0172] The duration scoring specifier 910 can be configured to determine an overall duration score for the product imaged and analyzed using the techniques described herein. For example, the duration scoring specifier 910 can determine whether the change in the quality attribute is within a threshold range for a particular type of product and / or exceeds some threshold level. The duration scoring specifier 910 can retrieve and apply one or more duration scoring metric models 956A-N from the model data store 954 to determine the duration scores for the processed and unprocessed products being imaged. The duration scoring specifier 910 can retrieve duration scoring metrics from the duration scoring metric assessment engine 908 and use those metrics to determine the overall duration of the product. In some implementations, the duration scoring specifier 910 can generate outputs as described throughout this disclosure, such as graphical charts (e.g., line graphs) showing the change in one or more quality attributes of the processed and unprocessed products. As described throughout this disclosure, the duration score may be indicative of a produce's ripeness level, length of eating, remaining shelf life, length of shelf life, edibility, and / or saleability. The duration scores generated by the duration scoring specifier 910 may be stored in the produce information data store 900 for each produce 944A-N as a duration score 948. The duration scores 948 may later be retrieved and output to a user device for analysis and review by relevant stakeholders in the supply chain.

[0150]

[0173] As described throughout, the above example analyzers do not serve to limit the scope of the present disclosure. Instead, any type of analyzer may be used by the duration scoring metric assessment engine 908. Similarly, the input and output characteristics of each of the example analyzers should not be viewed as limiting. Analyzers may be configured and / or trained to generate other types of output data and receive other types of input data.

[0151]

[0174] FIG. 10 is an example of an environment for capturing time-lapse image data of a product. Components of the imaging environment 1000 can communicate with the computer system 110, the user device 112, and / or the product information data store 900 via the network 114. As described further below, components of the imaging environment 1000 can be remote from one or more of the computer system 110, the user device 112, and / or the data store 900. In some implementations, the computer system 110 can be part of the imaging environment 1000. For example, the computer system 110 can be an edge computing device configured to analyze and quantify in real time changes in quality attributes of a product being imaged in the imaging environment 1000. This configuration can be advantageous to avoid clogging network bandwidth and / or use computing resources more efficiently. In such implementations, the computer system 110 can be attached to or otherwise part of a camera 1004 in the imaging environment 1000. The imaging environment 1000 may further include a power source that can be used to provide power to components of the imaging environment 1000, including the computer system 110, the camera 1004, and other components such as one or more fans and / or one or more light sources.

[0152]

[0175] In some implementations, the user device 112 can be part of the computer system 110 and / or the imaging environment 1000. The user device 112 can include a display screen and one or more input devices, such as a mouse, keyboard, microphone, and / or touch screen. The user device 112 may be powered by the same or another power source as that of the imaging environment 1000. In operation, the computer system 110 can schedule when images are captured by the camera 1004. The computer system 110 can send a notification to the camera 1004 that triggers the camera 1004 to capture one or more images of the products 1010A-N. For example, every five hours for 30 days, the computer system 110 can send a notification to the camera 1004 prompting the camera 1004 to capture one or more images of the products 1010A-N. The images captured by the camera 1004 can then be sent to the computer system 110 for processing and output to the user device 112. For example, the computer system 110 may receive images of the products 1010A-N, analyze the images to identify quality attributes (e.g., duration scoring metrics) of the products 1010A-N, and determine a duration score for each of the products 1010A-N and / or a batch of the products 1010A-N. The identified duration scores and / or quality attributes may be transmitted to the user device 112 and presented on a display screen of the user device 112. The images may also be transmitted to and presented / outputted by the user device 112. Relevant stakeholders may then review the output information and determine whether and / or if any changes can and / or should be made to the shelf-life extension coating solution applied to one or more of the products 1010A-N.

[0153]

[0176] The imaging environment 1000 may include a fan 1002, a camera 1004, a rig 1006A-N, and at least one light source 1008A-N. The imaging environment 1000 may be a box with walls on all sides and top and bottom. The products 1010A-N may be located at the bottom of the imaging environment 1000, such as on a pallet or shallow box. The products 1010A-N may remain in such a position throughout the time period during which the products 1010A-N are imaged by the camera 1004. As a result, the camera 1004 may capture images of the products 1010A-N from a constant angle and lighting and at constant time intervals during the time period. This may be beneficial in increasing the accuracy of comparing changes in quality attributes of the products 1010A-N over time.

[0154]

[0177] The fan 1002 can be configured to circulate air within the imaging environment 1000. The fan 1002 can be attached to a rig 1006N, which can be located in a ceiling lining of the imaging environment 1000. The fan 1002 can also be located elsewhere within the imaging environment 1000. In some implementations, the imaging environment 1000 can have multiple fans.

[0155]

[0178] The camera 1004 may be mounted on a rig 1006N. The camera 1004 may be centered and positioned on the ceiling lining of the imaging environment 1000. Thus, the camera 1004 may have a field of view that includes all of the products 1010A-N from a fixed angle. In some implementations, a motor may be used to move the produce on a motorized tray near, under, or otherwise adjacent to the hyperspectral camera so that the hyperspectral camera can capture images of each line of the produce 1010A-N (e.g., line by line of pixels, thereby forming a full matrix that is a hyperspectral image of the produce 1010A-N). The camera 1004 may be configured to capture images of the produce 1010A-N at fixed time intervals throughout the time period that the produce 1010A-N is being imaged, as described herein. For example, the camera 1004 may capture multiple images of the produce 1010A-N five times a day for 30 days.

[0156]

[0179] As described herein, the camera 1004 can generate image data representative of attributes of the produce 1010A-N. In some implementations, the camera 1004 can include one or more hyperspectral sensors configured to capture hyperspectral images representative of the characteristics of the produce 1010A-N. In such implementations, each pixel of the hyperspectral image can correspond to a visible light spectrum that can be a portion of the near infrared (NIR) range of the hyperspectral data. As a result, one hyperspectral camera can capture and extract RGB images (and / or video) from the data without having to capture additional data with a separate RGB camera. In some implementations, the camera 1004 can be a low-resolution digital camera (e.g., 5M or less), a high-resolution digital camera (e.g., 5MP or more), or any other type of sensor capable of capturing image data.

[0157]

[0180] In some implementations, the imaging environment 1000 may include multiple sensors or cameras positioned at multiple angles relative to the products 1010A-N. For example, the imaging environment 1000 may include a first camera and at least one additional second camera, each capturing image data of the products 1010A-N from a different angle. In such a configuration, the one or more additional cameras may be used to generate image data based on light at different or additional wavelengths than those captured by the first camera.

[0158]

[0181] Each particular one of the one or more cameras can be configured to detect different or additional wavelengths of light in a number of different ways. For example, in some implementations, different sensors can be used in different cameras to detect different or additional wavelengths of light. Alternatively or additionally, each of the one or more cameras can be positioned at different heights, angles, etc. relative to one another to attempt to capture different wavelengths of light. In some implementations, one or more cameras can be positioned to at least partially capture a portion of the product 1010A-N that may be obscured from the view of a first camera.

[0159]

[0182] The light sources 1008A-N can be used to illuminate the products 1010A-N so that the camera 1004 can clearly capture image data of the product. The light sources 1008A-N can be mounted on rigs 1006A and 1006B located on the top / ceiling of the imaging environment 1000. The light sources 1008A-N can be positioned to provide consistent and uniform lighting to the products 1010A-N. This can be beneficial because the imaging environment 1000 can be a closed setting that prevents additional ambient lighting from spilling onto the products 1010A-N being imaged and does not cast shadows or create uneven lighting conditions.

[0160]

[0183] The light sources 1008A-N can include one or more light sources, each generating the same or different electromagnetic radiation. In some implementations, the light sources 1008A-N can be located at one or more locations proximate the camera 1004 to illuminate the product 1010A-N before and / or during capture of image data. In some implementations, the one or more light sources 1008A-N can be selected based on the frequency of the electromagnetic radiation output. For example, in some implementations, the one or more light sources 1008A-N can be halogen light sources. Alternatively or additionally, the one or more light sources 1008A-N can be a diode or a series of broadband light emitting diodes (LEDs) that can be used to provide light across the visible wavelength spectrum, the near infrared wavelength spectrum, the electromagnetic spectrum, or any other spectrum, such as the UV spectrum. In general, any light source can be used to provide any type of light to the camera 1004.

[0161]

[0184] In some implementations, one or more of the light sources 1008A-N or a control unit for one or more of the light sources 1008A-N can be communicatively connected to the camera 1004 or a control unit for the camera 1004. For example, the camera 1004 or a control unit for the camera 1004 can send a signal to the one or more of the light sources 1008A-N or the control unit for the one or more of the light sources 1008A-N that causes the light sources 1008A-N to illuminate the product 1010A-N with a particular power and / or with one or more particular wavelengths of light at a particular moment in time. In some implementations, the particular moment can be a predetermined amount of time before or during the capture of the image data.

[0162]

[0185] The image data may be comprised of a plurality of images, each of which corresponds to one or more of the products 1010A-N in the imaging environment 1000. The image data may also include a portion of the background surrounding the products 1010A-N, such as the bottom / ground / floor of the imaging environment 1000. As described throughout, the computer system 110 may process the image data and extract a portion of the image data indicative of one or more of the products 1010A-N. The extracted portion of the image data may be indicative of all of the products 1010A-N being imaged. In some implementations, the extracted portion of the image data may be indicative of one of the products 1010A-N. Thus, the computer system 110 may generate an extracted portion (e.g., a bounding box) of each of the products 1010A-N being imaged.

[0163]

[0186] The imaging environment 1000 may include any number of produce 1010A-N for imaging. The same produce 1010A-N placed in the imaging environment 1000 on the first day of imaging may remain in the imaging environment 1000 until the imaging time period is completed (e.g., 30 days for some produce, such as avocados). The imaging environment 1000 may be sized to accommodate different types and / or quantities of produce. As an illustrative example, the imaging environment 1000 may be sized to accommodate six avocados, three of which are treated (e.g., coated with a shelf-life extending coating solution) and three of which are untreated (e.g., not coated with any produce or solution). The imaging environment 1000 may be sized to accommodate 25 avocados in five rows, five per row. The same imaging environment 1000 may be sized to image four apples and used to image four rings, two processed and two unprocessed. The same imaging environment 1000 may be used to image eight lemons, four processed and four unprocessed. As discussed above, the imaging environment 1000 can accommodate any number of products 1010A-N desired for imaging. The products 1010A-N can be arranged in the imaging environment 1000 in a grid configuration. In some implementations, the products 1010A-N may simply be placed within the imaging environment 1000 without any particular configuration.

[0164]

[0187] The computer system 110 may utilize one or more machine learning algorithms, techniques, and / or models trained to detect produce in the image data and extract the detected produce. Such algorithms, techniques, and / or models may be trained to identify the type of produce detected in the image data. As an illustrative example, the produce 1010A-N may be avocados. The computer system 110 may utilize one or more models trained on a plurality of images of avocados to determine from the image data whether one or more avocados are shown and which regions of the image data contain avocados. The one or more models may be trained to generate output data in the form of one or more annotated images of the produce 1010A-N. In generating the annotated images, the computer system 110 may annotate or otherwise index each of the avocados identified in the image data. As described throughout this disclosure, quality attributes (e.g., duration scoring metrics) may be identified for each of the annotated / indexed avocados in the image data. As described throughout this disclosure, the imaging environment 1000 can be used to assess the quality and duration of different types of produce, including, but not limited to, citrus fruits, mangoes, apples, berries, stone fruits, tomatoes, meats, and / or vegetables.

[0165]

[0188] FIG. 11 shows an example of time lapse image data comparing treated and untreated produce. The image data in FIG. 11 shows an untreated avocado 1110 and a treated avocado 1112. As described herein, the treated avocado 1112 can be coated with a shelf-life extending coating solution. Image data 1100 shows the untreated avocado 1110 and the treated avocado 1112 on the first day of imaging. Image data 1102 shows the untreated avocado 1110 and the treated avocado 1112 on the sixth day of imaging. Image data 1104 shows the untreated avocado 1110 and the treated avocado 1112 on the fourteenth day of imaging. Image data 1106 shows the untreated avocado 1110 and the treated avocado 1112 on the twenty-sixth day of imaging.

[0166]

[0189] As shown in image data 1100, 1102, 1104, and 1106, the untreated avocados 1110 shrink in size and darken first (e.g., due to becoming less firm and / or more ripe). In other words, over the 26 days of imaging, the untreated avocados 1110 have a shorter duration (e.g., shorter shelf life, edibility, saleability) than the treated avocados 1112. In image data 1102, the first untreated avocado 1110 first darkened on day 6. The two untreated avocados 1112 did not darken until day 26, as shown in image data 1106. The quality attributes shown in image data 1100, 1102, 1104, and 1106 can be identified and quantified by a computer system (e.g., computer system 110) to identify the age of both the unprocessed avocado 1110 and the processed avocado 1112.

[0167]

[0190] 12 is an example shelf-life analysis 1200 of an avocado using the disclosed techniques. An unprocessed avocado 1110 and a processed avocado 1112 can be imaged using the techniques described herein. The shelf-life analysis 1200 can be performed by a computer system, such as computer system 110 described throughout this disclosure.

[0168]

[0191] As part of the shelf life analysis 1200, the computer system may identify 1202 the locations of the unprocessed avocados 1110 and the processed avocados 1112. As described with reference to process 300 of Figures 3A and 3B, the computer system may determine a bounding box around each of the avocados 1110 and 1112. The computer system may also identify a grid structure and assign a grid index to each bounding box of the avocados 1110 and 1112. For example, the first unprocessed avocado 1110 may be assigned grid index 0. The second unprocessed avocado 1112 may be assigned grid index 1. The first processed avocado 1112 may be assigned grid index 2. The second unprocessed avocado 1112 may be assigned grid index 3.

[0169]

[0192] The computer system may also extract 1204 pixels for each of the avocados 1110 and 1112 at each time the avocados 1110 and 1112 are imaged. See the discussion of 310 of process 300 in FIG. 3A for additional discussion.

[0170]

[0193] The computer system can then calculate (1206) quality features of the avocados 1110 and 1112. The computer system can identify color changes of the avocados 1110 and 1112 based on the analysis of the extracted pixels from 1204. The color changes over time can be compared to a statistical scale color value determined at the beginning of the imaging (e.g., when the avocados 1110 and 1112 were first imaged). The statistical scale color value can be the average color value of all of the avocados 1110 and 1112 on the first day of imaging. The computer system can compare the pixels of the avocados 1110 and 1112 to the statistical scale color value pixel by pixel. For example, a pixel by pixel basis can be used to calculate the distance of each pixel from the statistical scale color value determined at the beginning of the imaging. The distances can be used to generate a matrix of distances (e.g., one distance per pixel). The computer system can then calculate the mean, median, or other statistical measure of the distance distribution, which can be a duration metric for which threshold ranges can then be applied, as described above. As described above, one or more other techniques may also be used.

[0171]

[0194] The color change may be plotted over time in a line graph 1210. The computer system may identify line graphs 1210 for a group of untreated avocados 1110 and a group of treated avocados 1112. Thus, one of the line graphs 1210 may show the average change in color over time for the untreated avocados 1110 and another of the line graphs 1210 may show the average change in color over time for the treated avocados 1112. The computer system may also generate line graphs that show both the average change in color over time for the untreated avocados 1110 and the average change in color over time for the treated avocados 1112.

[0172]

[0195] The computer system may also identify a change in volume over time in 1206 of the avocados 1110 and 1112 based on an analysis of the extracted pixels from 1204. The change in volume over time may be compared to an original average volume of the avocados 1110 and 1112 identified at the first time of imaging (e.g., when the avocados 1110 and 1112 were first imaged). The original average volume may be the average volume value of all of the avocados 1110 and 1112 on the first day of imaging. The volume change may be plotted over time in a line graph 1212. The computer system may identify a line graph 1212 for a group of untreated avocados 1110 and a group of treated avocados 1112. Thus, one of the line graphs 1212 may show the average change in volume over time for unprocessed avocados 1110, and another of the line graphs 1210 may show the average change in volume over time for processed avocados 1112. The computer system may also generate line graphs that show both the average change in volume over time for unprocessed avocados 1110 and the average change in volume over time for processed avocados 1112.

[0173]

[0196] The computer system can then use the threshold to identify 1208 the shelf life of the avocados 1110 and 1112. As described herein, the shelf life can be a duration score. The computer system can also determine the end of shelf life, edibility, ripeness, and / or saleability of the avocados 1110 and 1112. The end of life threshold 1216 can also be included in the shelf life graph 1214. In some implementations, the shelf life graph 1214 can be a combination of the color change over time and the volume change over time from 1206. The end of life threshold 1216 can be used for both the unprocessed avocados 1110 and the processed avocados 1112. Thus, the end of life threshold 1216 can be included in the line graph 1210 showing the color change of either the avocados 1110 and 1112. An end of life threshold 1216 may be included in the line graph 1212 showing the volume change of either of the avocados 1110 and 1112 .

[0174]

[0197] As shown at 1208, a shelf life graph 1214 shows the change in volume over time of either the unprocessed avocados 1110 or the processed avocados 1112. The shelf life 1218 is identified as the point in time where the line of the graph 1214 crosses the end of life threshold 1216. As described further below, the shelf life 1218 can be a later point in time for the processed avocados 1112 compared to the shelf life of the unprocessed avocados 1110.

[0175]

[0198] As described throughout this disclosure, the computer system may determine shelf life 1218 using only color quality attributes or only volumetric quality attributes. In some implementations, the computer system may determine shelf life 1218 based on a combination of color and volumetric quality attributes. Additionally, in some implementations, the computer system may determine shelf life 1218 based on a combination of one or more other quality attributes, such as wrinkles, hardness, spoilage, infection, and / or other attributes identified and quantified using the techniques described herein.

[0176]

[0199] Figure 13 is an example of a statistically scaled color determination 1300 of an avocado using the process of Figure 5. The statistically scaled color determination 1300 can be performed by a computer system such as computer system 110. The computer system can receive image data 1100 of avocados 1110 and 1112 at a first time point when the avocados 1110 and 1112 were imaged using the techniques described herein. Here, the computer system receives image data 1100 of avocados 1110 and 1112 on day 1. For each of the avocados 1110 and 1112, the computer system can extract (1302) corresponding pixels. Thus, the computer system can generate masks 1304 of the avocados 1110 and 1112.

[0177]

[0200] The computer system may determine 1306 the average or mean of all color pixels of the avocados 1110 and 1112 in the mask 1304. The mask 1304 may be used to extract the RGB data of the avocados 1110 and 1112 from the RGB image data by extracting all RGB values ​​of pixels where the mask 1304 has a value of true or 1 (e.g., yellow). Thus, the computer system may determine and output a statistical measure color value 1308. The statistical measure color value 1308 may represent the average or mean color value of all the avocados 1110 and 1112 when first imaged on day 1. The computer system may then compare the detected colors of the avocados 1110 and 1112 in the image data at each subsequent time point of imaging. Comparing the detected colors of the avocados 1110 and 1112 to the statistical scale color value 1308 may include calculating a distance (e.g., Euclidean distance) between all pixels representing each of the avocados 1110 and 1112 at each time point and the statistical scale color value 1308. The calculated distances may be plotted on a graph over time to show how the colors of the avocados 1110 and 1112 change over time.

[0178]

[0201] Figure 14 is a time course color analysis 1400 of an example of avocados 1110 and 1112 using the process of Figures 3-5. The color analysis 1400 can be used to quantify color changes seen in the image data 1100 of the avocados 1110 and 1112 on day 1 and the image data 1104 of the avocados 1110 and 1112 on day 14. The color analysis 1400 may be performed to identify color changes seen in the image data of the avocados 1110 and 1112 over an entire imaging period, such as 30 days.

[0179]

[0202] 14, each of the avocados 1110 and 1112 can be indexed. Avocados 0 and 1 are both untreated avocados 1110. Avocados 2 and 3 are both treated avocados 1112. The identified colors of the avocados 1110 and 1112 at each time point imaged can be compared to the statistical scale color value 1308. The computer system can then generate a color over time graph 1402 that shows the change in color of each of the avocados 1110 and 1112: Avocados 0, 1, 2, and 3 over time.

[0180]

[0203] As shown in graph 1402, by approximately day 5, avocado 0 (untreated avocado 1110) reached a color significance distance. Avocado 0 started out at a color distance of 5 at the first time point of imaging (day 0). By day 5, avocado 0 reached a color distance of 25, while avocados 1, 2, and 3 remained around or below color distance 5. By approximately day 6, avocado 1 (untreated avocado 1110) reached a color significance distance. Avocado 1 started out at approximately color distance 1 at the first time point of imaging. By day 6, avocado 1 reached a color distance of 25, while avocados 2 and 3 remained around or below color distance 5. By approximately day 20, avocado 3 reaches approximately color distance 25. By approximately day 23, avocado 2 reaches approximately color distance 25. Thus, compared to the untreated avocado 1110, avocados 2 and 3, both treated avocados 1112, reach a significant color distance after approximately 15 days. This can be attributed to the application of the shelf life extension coating solution to the avocado 1112. Thus, the shelf life extension coating solution can extend the shelf life, edibility, and / or saleability (e.g., duration score) of the avocado 1112, as demonstrated in graph 1402.

[0181]

[0204] FIG. 15 is an example time-course volume analysis 1500 of an avocado using the process of FIG. 6. As described herein, a computer system, such as computer system 110, can count 1502 the number of pixels in image data representing an avocado. As shown in FIG. 15, the computer system counts 244,000 pixels. The computer system can represent the pixel count as the area of ​​the avocado and then solve for the radius 1504. Using the radius, the computer system can solve for the volume of the avocado 1506. Here, because avocados are relatively spherical / round objects, the computer system can use the sphere volume formula. The sphere volume formula can also be used for other products, such as apples, lemons, limes, and other relatively round products.

[0182]

[0205] The computer system may also plot the volume change over time in a line graph 1508. The computer system may plot and output the volume change over time for each avocado imaged. In some embodiments, the computer system may identify an average volume change over time for the treated avocados and an average volume change over time for the untreated avocados. The computer system may then output these average volume changes over time in the same line graph and / or in separate line graphs.

[0183]

[0206] Example line graph 1508 shows the volume change over time for an avocado labeled Avocado 1. Avocado 1 can be an untreated avocado, as shown in FIGS. 11-14. On day 0, which can be the first day of imaging, Avocado 1 is at 100% volume. In other words, the baseline volume of Avocado 1 can be the volume measured on the first day of imaging. Volumes measured on subsequent days can be compared to the baseline volume to determine the extent to which the volume has changed from the first day of imaging. The greater the volume change, the more the avocado has shrunk. The greater the volume change, the shorter the avocado will last.

[0184]

[0207] Here, as shown in line graph 1508, avocado 1 has declined to approximately 93% of its original volume (e.g., baseline volume on day 1) by day 5 (in some embodiments, the produce can decline to one or more other percentages of its original volume from day 1 over one or more other time periods). Avocado 1 has declined to approximately 88% of its original volume by day 10. By day 23, avocado 1 has declined to approximately 75% of its original volume. As described herein, applying a shelf-life extension coating solution to avocado 1 can be beneficial to slow down the amount and / or rate at which the avocado's volume changes over time. As a result, the shelf life of the treated avocados can be increased or otherwise improved, making the avocados more desirable for purchase and consumption by consumers.

[0185]

[0208] 16 is an example shelf life analysis 1600 of an avocado based on volume using the disclosed techniques. As shown in FIG. 16, the techniques described herein can be used to quantify the volume change of the avocados shown from day 1 in image data 1100 to day 26 (or later) in image data 1106. Quantifying the volume change of the untreated avocado 1110 compared to the volume change of the treated avocado 1112 is instructive in demonstrating that treating avocados 2 and 3 with a shelf life extension coating solution results in a 3.2+ / -0.6 fold increase in duration (e.g., shelf life, ripeness, edibility, and / or saleability) of the treated avocado 1112 over the untreated avocado 1110.

[0186]

[0209] The shelf life graph 1602 illustrates the shelf life of an untreated avocado 1110 and a treated avocado 1112 as each reaches a 10% volumetric shrinkage. For example, Avocado 0 (untreated avocado 1110) undergoes a 10% volumetric shrinkage when Avocado 0 reaches 8-9 days. The 8-9 day range can be identified as the shelf life of Avocado 0. Avocado 1 (untreated avocado 1110) undergoes a 10% volumetric shrinkage when Avocado 1 reaches approximately 10 days. On the other hand, Avocado 2 (treated avocado 1112) undergoes a 10% volumetric shrinkage after approximately 30 days, and Avocado 3 (treated avocado 1112) undergoes a 10% volumetric shrinkage after approximately 25 days. Avocados 2 and 3 have an increased shelf life (approximately 30 days and 25 days, respectively) because they have been treated with the shelf life extension coating solution. The shelf life of avocados 2 and 3 is approximately 3.2+ / -0.6 times longer than avocados 0 and 1. The techniques described herein are useful for quantifying and identifying such improvements in shelf life due to the shelf life extension coating solution. Thus, such techniques can be useful for determining additional improvements / modifications that can be made to the shelf life extension coating solution.

[0187]

[0210] The volume over time graph 1604 similarly indicates that the shelf life (e.g., duration, duration score) of avocados 2 and 3 is improved / prolonged over avocados 0 and 1 because avocados 2 and 3 were treated with the shelf life extension coating solution and avocados 0 and 1 were not treated. A threshold level 1606 is identified as the percentage of volume shrinkage that indicates the end of shelf life (e.g., end of duration, ripeness, edibility, saleability). The threshold level 1606 in this example is 90% of the original volume of the avocado. As described throughout this disclosure, the threshold level can vary based on the produce, the type of produce, and / or one or more other characteristics associated with the produce (e.g., geographic origin). The intersection of the volume change over time of the avocado with the threshold level 1606 indicates the amount of time of shelf life of that particular avocado.

[0188]

[0211] Thus, in the example graph 1604, Avocado 0 crosses the threshold level 1606 at approximately day 8-9 (1608). Avocado 1 crosses the threshold level 1606 at approximately day 9-10 (1608). Avocado 2 crosses the threshold level 1606 at approximately day 27 (1610). Avocado 3 crosses the threshold level 1606 at approximately day 25 (1610).

[0189]

[0212] Graphs 1602 and 1604 can be useful in visualizing the impact on the quality attributes of a produce when the produce is treated with a shelf-life extension coating solution. Such output can be useful in identifying the positive impact on the quality of the produce from the application of the shelf-life extension coating solution. Such output can also be useful in determining potential modifications that can be made to the shelf-life extension coating solution to further improve the benefits that the shelf-life extension coating solution may have on the quality of the produce.

[0190]

[0213] FIG. 17 is an example shelf-life analysis 1700 of limes using the disclosed techniques. The shelf-life analysis 1700 can be based on color change, volume change, or a combination of both. Image data 1702 of untreated lime 1704 and treated lime 1706 can be received by a computer system, such as computer system 110. The computer system can identify and extract pixels representing each of the limes 1704 and 1706. The computer system can also apply a grid structure to the image data 1702 to determine an index for each of the limes 1704 and 1706. Thus, the first lime can be indexed and labeled as lime 0. The second lime can be indexed and labeled as lime 1. Limes 0 and 1 are untreated limes 1704. The third lime can be indexed and labeled as lime 2. The fourth lime can be indexed and labeled as lime 3. Limes 2 and 3 are treated limes 1706. The computer system may implement one or more other indexing and labeling schemes. The index of each of the limes 1704 and 1706 can be used to identify changes in color and / or volume of each of the limes 1704 and 1706 over time.

[0191]

[0214] Using the techniques described herein, a computer system can determine that the duration (e.g., shelf life, maturity, edibility, saleability) of limes 2 and 3 (treated limes 1706) is 6+ / -2 times longer than limes 0 and 1 (untreated limes 1704).

[0192]

[0215] Three different types of outputs are visualized in Figure 17. Using the techniques described herein, one or more other types of outputs may be realized and identified to visually indicate the impact of coating the treated lime 1706 with a shelf life extension coating solution.

[0193]

[0216] Graph 1708 shows the shelf life of each of the limes 1704 and 1706 based on volume change. A 10% volume shrinkage is used as the threshold level for limes 1704 and 1706. Thus, lime 0 (untreated lime 1704) shrinks to 10% of its original volume in approximately 3 days. Thus, the shelf life (e.g., duration) of lime 0 is 3 days. Lime 1 (untreated lime 1704) shrinks to 10% volume in approximately 6 days. Thus, the shelf life of lime 1 is 6 days. Lime 2 (treated lime 1706) shrinks to 10% volume in approximately 25 days, and lime 3 (another treated lime 1706) shrinks to 10% volume in approximately 23 days. Thus, the shelf life of limes 2 and 3, both of which are treated limes 1706, is, on average, six times the shelf life of limes 0 and 1, which are untreated limes 1704. In some embodiments, the standard deviation can be 2.

[0194]

[0217] Graph 1710 shows the color change of each of the limes 1704 and 1706. Lime 0 undergoes the greatest and most rapid color change compared to the other limes 1, 2, and 3. Limes 2 and 3 undergo the least color change, which can be attributed to being treated with the shelf-life extension coating solution. In fact, as shown in graph 1710, the color of limes 2 and 3 remains relatively constant, thereby indicating that limes 2 and 3, both treated limes 1706, have a longer duration (e.g., edible, saleable, shelf life, ready to eat) than limes 0 and 1, both untreated limes 1704.

[0195]

[0218] Graph 1712 shows the shelf life of each of the limes 1704 and 1706 based on the volume change. As explained with reference to graph 1710, a 10% volume shrinkage can indicate the threshold level 1714. The intersection of each of the limes 1704 and 1706 with the threshold level 1714 indicates the shelf life of each of the limes 1704 and 1706. Thus, lime 0 crosses the threshold level 1714 at approximately day 3. Thus, lime 0 has a shelf life of 3 days. Lime 1 crosses the threshold level 1714 at approximately day 6. Thus, lime 1 has a shelf life of 6 days. Limes 2 and 3 cross the threshold level 1714 at approximately day 25. Thus, limes 2 and 3, both of which are treated limes 1706, have a shelf life of approximately 25 days.

[0196]

[0219] As mentioned throughout, graphs 1708, 1710, and 1712 are useful for visually illustrating the quality attribute differences between untreated lime 1704 and treated lime 1706. A user can view graphs 1708, 1710, and / or 1712 to quantify the improvement in duration resulting from coating treated lime 1706 with a shelf-life extension coating solution. A user can also use graphs 1708, 1710, and / or 1712 to determine whether and what type of changes should be made to the shelf-life extension coating solution.

[0197]

[0220] Figure 18 illustrates an example technique for extracting apples 1800 or other produce from image data. A computer system, such as computer system 110, can receive image data 1802 of apples on the first day of imaging. Untreated apples 1804 and treated apples 1806 can be captured in the image data 1802. In the example of Figure 18, 16 apples are imaged, of which 8 are untreated apples 1804 and 8 are treated apples 1806. The same 16 apples are imaged each day throughout the time period that apples 1804 and 1806 are imaged.

[0198]

[0221] The computer system can then use one or more different techniques to identify and extract each of the apples 1804 and 1806. For example, the computer system can use a blue channel histogram 1810 to identify and extract each of the apples 1804 and 1806. The computer system can also use clustering 1814 to identify and extract each of the apples 1804 and 1806.

[0199]

[0222] In the blue channel histogram 1810, a computer system can generate blue channel data 1808 from image data 1802. The blue channel is represented by values ​​from 0 to 255 shown in the blue channel data 1808 with colors ranging from purple (0) to yellow (255). Using a mask, RGB data for apples 1804 and 1806 can be extracted from image data 1802 by capturing all RGB values ​​of pixels where the mask is true / 1. For example, pixels in blue channel data 1808 having color values ​​from 0 to 75 (representing various hues of purple) can be extracted to represent apples 1804 and 1806. A significant amount of pixels in blue channel data 1808 (e.g., as many as 100,000 pixels of one color value) have values ​​from 0 to 75 and are therefore likely to represent apples 1804 and 1806. Another significant amount of pixels (e.g., as many as 145,000 pixels of one color value) have color values ​​from 100 to 175. This number of pixels may represent the background. Pixels with color values ​​between 75 and 100 may represent either the apples 1804 and 1806 or the background. Thus, during RGB pixel extraction, sometimes parts of the apples 1804 and 1806 may not be extracted from the blue channel data 1808. As a result, parts of the apples 1804 and 1806 may be missing, as shown by the apple binary mask 1812. In some implementations, parts of the background may be present in the apple extraction data. Depending on the threshold used, some pixels of the apples 1804 and 1806 may be lost (e.g., when using 75 as the threshold) and / or some pixels of the unwanted background may appear in the extracted data (e.g., when using 100 as the threshold). The computer system may not be able to accurately quantify the color change of the apples 1804 and 1806 when parts of the apples 1804 and 1806 are missing from the apple binary mask 1812 or when parts of the background are present.

[0200]

[0223] Thus, the computer system may also use clustering techniques 1814 to identify and extract the apples 1804 and 1806. Clustering 1814 may provide a more accurate extraction of the apples 1804 and 1806, such that there are no or few, if any, missing portions of the apples 1804 and 1806. As a result, the computer system may more accurately quantify the color variation of the apples 1804 and 1806. The apples 1804 and 1806 may be clustered to generate an apple binary mask 1816. As shown in the apple binary mask 1816, the clustering techniques 1814 may be used to extract the complete apples 1804 and 1806, compared to the gaps and portions of the apples 1804 and 1806 that are missing in the apple binary mask 1812.

[0201]

[0224] The computer system can identify color values ​​from the image data 1802 that are indicative of apples 1804 and 1806 and color values ​​that are indicative of the background. The computer system can then cluster the identified color values. In other words, the computer system can graph all color channels in a 3D space and then identify clusters of different colors. By looking for clusters of colors in a particular 3D space, the computer system can identify which clusters represent apples and which clusters represent background. Using discrimination techniques, the computer system can select the clusters to keep (e.g., clusters that represent apples) and remove the clusters that were not selected (e.g., clusters that represent background).

[0202]

[0225] In the clustering 1814 of the example apples 1804 and 1806 of FIG. 18, four different clusters of colors are identified. Cluster 0 represents the purple channel. Cluster 1 represents the dark blue channel. Cluster 2 represents the light blue channel. Cluster 3 represents the green channel. By way of example, clusters 0, 1, 2, and 3 appear in a 3D space, and the computer system can determine that clusters 1 and 2 represent the apples 1804 and 1806, and clusters 0 and 3 represent the background. The clustering model can generate, for example, a list of cluster centers in RGB space. This information can then be used by the computer system to determine which two clusters are in the background, and therefore discard all pixels in those two clusters. Thus, the computer system can select and extract clusters 1 and 2. Thus, the apple binary mask 1816 shows a cleaner and more defined shape of the apples 1804 and 1806 compared to the apple binary mask 1812, which includes more gaps or missing parts from the apples 1804 and 1806.

[0203]

[0226] The blue channel histogram 1810 can be beneficial for one or more types of produce, such as avocados, limes, and lemons. The blue channel histogram 1810 may not be favorable for other types of produce that may have more color, such as apples and mangoes. The clustering 1814 can be used for any type of produce being imaged.

[0204]

[0227] The techniques described herein can be used in conjunction with hyperspectral images of the apples 1804 and 1806 to detect quality attributes, such as the firmness of the apples 1804 and 1806. Quality attributes, such as firmness, can be a more effective indicator of the age of the apples 1804 and 1806 compared to the color and / or volume of other types of produce, such as avocados, lemons, and limes. The computer system can use one or more machine learning models trained to predict the firmness of the apples 1804 and 1806 based on the hyperspectral images. The models can be trained with training image data that is labeled with hardness measurements captured by a penetrometer. The models can be trained to predict the firmness of the apples 1804 and 1806 based on a combination of the penetrometer hardness measurements and the hyperspectral training image data. The firmness of the apples 1804 and 1806 can then be used by the computer system to identify the age of the apples 1804 and 1806.

[0205]

[0228] 19 is an example shelf life analysis 1900 of apples using the disclosed techniques. A computer system, such as computer system 110, can receive image data 1802 of apples 1804 and 1806. The computer system can also receive image data of apples 1804 and 1806 at one or more different time intervals throughout the time period during which apples 1804 and 1806 are imaged. The computer system can process the image data, such as image data 1802, to identify quality attributes of apples 1804 and 1806 over time. The computer system can identify quality attributes of each of apples 1804 and 1806. Apples with indexes or identifiers from 0 to 7 are untreated apples 1804 and apples with indexes or identifiers from 8 to 15 are treated apples 1806. Based on how the quality attributes change over time, the computer system can identify the shelf life of each of apples 1804 and 1806. The computer system may determine the age of apples 1804 and 1806 based on, for example, color change, volume change, or a combination of both.

[0206]

[0229] Graph 1902 illustrates the shelf life of apples 1804 and 1806 based on color change. A threshold color change can be used to identify when each of apples 1804 and 1806 will reach the end of their shelf life. The threshold color change can be a maximum of 13 color distances. In other words, the threshold color change can be a color distance of 13 from the original color (e.g., statistically scaled color value, as described throughout this disclosure) of apples 1804 and 1806 on the first day of imaging. As shown in graph 1902, untreated apples 1804 (apples 0-7) reach the threshold change 20-60 days after first imaging. Treated apples 1806 (apples 8-15) reach the color change threshold over 100 days after first imaging. As a result, a lower limit can be set for the duration of treated apples. Thus, by treating apples 1806 with the shelf-life extension coating solution, the duration (eg, ripeness, edibility, shelf life, saleability) of apples 1806 is at least 2.4 times that of untreated apples 1804.

[0207]

[0230] Graph 1904 shows the shelf life of apples 1804 and 1806 based on color change over time. As previously mentioned, the threshold color change is a color distance of 13 from the original color (e.g., statistically scaled color value) of apples 1804 and 1806 on the first day of imaging. As shown in graph 1904, the untreated apple 1804 reaches the color change threshold 20-60 days after the first imaging. On the other hand, the treated apple 1806 never crosses the color change threshold beyond 100 days. As a result of treating apple 1806 with the shelf life extension coating solution, the treated apple 1806 lasts longer than the untreated apple 1804.

[0208]

[0231] Graph 1906 shows the volume change over time for apples 1804 and 1806. Apples generally may not change volume as dramatically as other produce, such as avocados and mangoes. Thus, volume change over time may not be a valid quality attribute to use in measuring the duration of an apple. Nevertheless, on average, graph 1906 demonstrates that treated apples 1806 shrink in volume much slower than untreated apples 1804. Treated apples 1806 shrink in volume at a constant rate, starting from a volume of 1.0 to a volume of 0.9 over the course of 100 days. On the other hand, untreated apples 1804 shrink in volume at a random, sporadic rate. Untreated apples 1804 shrink, on average, from a volume of 1.0 to a volume of 0.8 or less over the course of 100 days. Thus, graph 1906 demonstrates that the untreated apples 1804 shrink in volume more appreciably over time than the treated apples 1906. Thus, the duration (e.g., shelf life, edibility, saleability, ripeness) of the treated apples 1806 is longer than that of the untreated apples 1804.

[0209]

[0232] FIG. 20 is a flow chart of a process 2000 for determining quality metrics, machine learning models, and / or quality thresholds to be selected for a type of product during time-lapse imaging. The process 2000 can be performed as part of block 312 of process 300 (see, e.g., FIG. 3A and FIG. 3B) when the computer system applies one or more machine learning models and / or algorithms to the extracted product. The process 2000 may be performed before, or after one or more other blocks of the process 300. For example, the process 2000 can be performed as part of performing object detection to identify the bounding box of each product in the image data (block 304 of process 300). The process 2000 can be performed once for all of the products in the image data. For example, the process 2000 can be performed the first time the product is imaged (e.g., day 1).

[0210]

[0233] Process 2000 may be performed by computer system 110. Process 2000 may also be performed by one or more other computers, computer systems, devices, cloud-based services, and / or networks of computers and / or devices. For example, process 2000 may be performed by an edge computing device. For illustrative purposes, process 2000 is described from the perspective of a computer system.

[0211]

[0234] Referring to process 2000, a computer system may identify a type of product in the image data (2002). The computer system may apply one or more machine learning trained models to the image data, which may be trained to identify the type of product found in the image data. In some implementations, the computer system may receive metadata along with the image data, including some identifier that identifies the product. Additionally, in some implementations, a user may provide information identifying the product as an input to the computer system.

[0212]

[0235] The computer system can then select one or more duration scoring metrics from the plurality of metrics for the identified produce type in 2004. In other words, the computer system can retrieve one or more metrics from the data store that can be used to determine the duration of a particular type of produce. For example, if the produce is identified as an avocado in block 2002, the computer system can use the avocado identifier to select which quality metric is used to determine the duration of the produce. Thus, if the duration of an avocado is typically determined using changes in color and volume of the avocado, the computer system can select color and volume as the quality metrics to be determined in block 314 of process 300 of FIG. 3A and FIG. 3B. As another example, if the produce is identified as an apple in block 2002, the computer system can use the apple identifier to select from the data store the particular quality metric to be used to determine the duration of the apple. In some implementations, the selected quality metric can be color only, firmness only, and / or a combination of one or more other quality attributes described throughout this disclosure.

[0213]

[0236] In 2006, the computer system may select one or more thresholds for one or more selected metrics for the identified produce type. The computer system may use an identifier indicative of the produce type to determine which thresholds are used when identifying the duration of the produce (e.g., shelf life, edibility, saleability, ripeness). For example, if the produce is identified as an avocado and the computer system selects color and volume as the quality metrics, the computer system may select appropriate threshold levels for the avocado based on color and volume. In other words, the computer system may identify a threshold level for the shelf life of the avocado based on color and identify a threshold level for the shelf life of the avocado based on volume. These threshold levels may be identified in advance by the computer system and / or relevant stakeholders. These threshold levels may be stored in a data store or other storage and accessed later by the computer system. As discussed above, threshold levels may be selected for all produce in the image data when the produce is first imaged. The same selected threshold levels may then be used at subsequent time intervals throughout the time period of imaging.

[0214]

[0237] The computer system may also select one or more machine learning models and / or algorithms for the selected one or more metrics of the identified produce type (2008). The computer system may use an identifier indicative of the produce type to determine which model and / or algorithm is used to identify the duration of the produce. For example, if the produce is identified as a mango and the computer system selects the wrinkle quality metric, the computer system may select from the data store one or more models and / or algorithms to use to identify and quantify wrinkles in the mango. The computer system may also select from the data store one or more models and / or algorithms to use to identify the duration of the mango based on the identification and quantification of wrinkles over time.

[0215]

[0238] The computer system may execute blocks 2004, 2006, and 2008 in any order. The computer system may execute blocks 2004, 2006, and 2008 sequentially. In some implementations, the computer system may execute blocks 2004, 2006, and 2008 simultaneously (e.g., in parallel).

[0216]

[0239] The computer system then applies one or more selected machine learning models and / or algorithms to the extracted artifacts in the image data at 2010. The computer system may also apply the models and / or algorithms to the image data to determine a duration scoring metric for the artifacts and identify a duration of the artifacts. Thus, the computer system may return to block 314 of process 300 to determine a duration scoring metric for the extracted artifacts.

[0217]

[0240] 21 is a flow chart of a process 2100 for comparing the shelf life of a treated product to the shelf life of an untreated product to determine one or more modifications to a shelf life extension coating solution. Process 2100 may be performed once a computer system determines a duration score for the products, both treated and untreated products, imaged over a period of time. Process 2100 may be performed automatically and / or semi-automatically by a computer system. In some embodiments, one or more blocks in process 2100 may be performed by an associated stakeholder (e.g., a user) in the supply chain.

[0218]

[0241] Process 2100 may be performed by computer system 110. Process 2100 may also be performed by one or more other computers, computer systems, devices, cloud-based services, and / or networks of computers and / or devices. For example, process 2100 may be performed by an edge computing device. For purposes of illustration, process 2100 is described from the perspective of a computer system.

[0219]

[0242] Referring to process 2100, the computer system may retrieve duration scores for unprocessed and processed products of the same product type at 2102. The duration scores may be stored and accessed in one or more data stores, databases, and / or cloud-based storage. The computer system may retrieve each duration score determined for each product in the image data using process 300 of FIG. 3A and FIG. 3B. In some implementations, the computer system may retrieve an average duration score for all processed products in the image data and an average duration score for all unprocessed products in the image data. Additionally, in some implementations, the computer system may retrieve duration scores for different batches of processed and unprocessed products of the same type. For example, duration scores for a first batch of processed and unprocessed avocados imaged during a first time period may be retrieved. Duration scores for a second batch of processed and unprocessed avocados imaged during a second time period may also be retrieved.

[0220]

[0243] The computer system may then compare the duration scores of the processed products to those of unprocessed products of the same product type (2104). The computer system may compare the duration scores by graphing changes in quality attributes (e.g., duration scoring metrics) of the processed and unprocessed products in one or more graphical charts (e.g., line graphs, histograms, bar graphs, etc.). The computer system may also compare the duration scores of the processed and unprocessed products based on finding differences in the scores and / or identifying how far each duration score deviates from a threshold level / range.

[0221]

[0244] The computer system can identify one or more differences in the shelf life between the untreated product and the treated product (2106). As mentioned above, the computer system can determine how far each shelf life score deviates from a threshold level / range. The computer system can determine how much greater the shelf life score of the treated product is compared to the shelf life score of the untreated product. This difference in shelf life scores can indicate how much the shelf life of the treated product will be extended by applying the shelf life extension coating solution. The difference in shelf life can also be used to determine whether the shelf life extension coating solution is sufficiently effective and / or whether changes can be made to the shelf life extension coating solution to further enhance the solution to the shelf life of the product.

[0222]

[0245] Thus, the computer system can determine one or more modifications to the shelf-life extension coating solution based on the identified one or more differences (2108). For example, using one or more machine learning trained algorithms, the computer system can generate suggestions to modify such solution that may improve the duration of a product treated with the shelf-life extension coating solution. In some embodiments, relevant stakeholders can determine one or more modifications to the shelf-life extension coating solution with or without the assistance of the computer system's models, algorithms, and / or techniques.

[0223]

[0246] One or more different modifications can be determined. For example, modifications can include increasing or decreasing the concentration of the shelf-life extension coating solution formulation. The concentration of the formulation can determine the thickness of the solution. For climacteric fruits such as avocados, there can be an optimal concentration because they ripen by respiration (e.g., consuming oxygen and releasing CO2). The solution extends shelf life by reducing the rate of this respiration process, but if this process is reduced too much, the fruit may not ripen and may even rot. A concentration that is too low may also be undesirable, as it may lead to a shorter shelf-life extension than possible. Each product can also have a different formulation and concentration based on the product's characteristics as part of the development process.

[0224]

[0247] Other modifications can include adding a wetting agent to the concentration of the formulation. Different products can have different surfaces, and depending on the type of surface, the formulation can spread well and evenly, wetting and covering the entire surface (optimal) or the formulation can concentrate in droplets on the surface, thus creating an uneven barrier that can reduce shelf life extension performance. Thus, the addition of a wetting agent can improve the wettability and aid in a more uniform coating of the product.

[0225]

[0248] Another example modification can include modifying the application process of the shelf life extension coating solution. For example, the speed of the brush bed used in application can be increased (e.g., to increase throughput) or decreased (e.g., to ensure better coating coverage). As another example, the temperature of the dryer can be increased to ensure that the coating dries completely on the produce or decreased to ensure that no damage occurs to the produce (this can depend on produce characteristics such as produce size, thickness of the produce skin, etc.).

[0226]

[0249] Optionally, at 2110, the computer system can modify the shelf-life extension coating solution. For example, the computer system can instruct one or more devices or other computing systems to adjust the concentration levels of one or more components or elements that make up the shelf-life extension coating solution. In some embodiments, the computer system can transmit instructions, suggestions, and / or guidance that can be viewed by relevant stakeholders in the supply chain. The relevant stakeholders can then decide whether or not to implement any of the instructions, suggestions, and / or guidance to modify the shelf-life extension coating solution. The relevant stakeholders can implement the determined modifications manually. In some embodiments, the relevant stakeholders can control one or more other computing systems and / or devices to automatically or semi-automatically implement the determined modifications.

[0227]

[0250] Optionally, the computer system may apply the modified shelf-life extension coating solution to a batch of products of the same product type, at 2112. Once the solution is changed, the computer system may, for example, send instructions to another computing system and / or equipment to cause an equipment to automatically coat a subsequent batch of products with the modified solution. In some embodiments, the computer system may send instructions to one or more relevant stakeholder user devices to apply the modified solution to the products. The relevant stakeholder may then manually apply the modified solution to a subsequent batch of products and / or control an equipment to apply the modified solution. In some embodiments, the relevant stakeholder may also approve the changes to the solution, thereby causing one or more computing systems and / or equipment to semi-automatically apply the modified solution to the products.

[0228]

[0251] 22 is a block diagram of system components that can be used to implement a system for assessing the quality of one or more food products. Computing device 2200 is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. Mobile computing device is intended to represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are intended to be exemplary only and are not intended to limit the implementation of the invention described and / or claimed herein.

[0229]

[0252] The computing device 2200 includes a processor 2202, a memory 2204, a storage device 2206, a high-speed interface 2208 that connects to the memory 2204 and multiple high-speed expansion ports 2210, and a low-speed interface 2212 that connects to the low-speed expansion port 2214 and the storage device 2206. Each of the processor 2202, the memory 2204, the storage device 2206, the high-speed interface 2208, the high-speed expansion port 2210, and the low-speed interface 2212 are interconnected using various buses and may be mounted on a common motherboard or otherwise mounted as appropriate. The processor 2202 may process instructions for execution within the computing device 2200, including instructions stored in the memory 2204 or the storage device 2206 for displaying graphical information for a GUI on an external input / output device, such as a display 2216 coupled to the high-speed interface 2208. In other implementations, multiple processors and / or multiple buses may be used as appropriate, along with multiple memories and multiple types of memories. Additionally, multiple computing devices may be connected, each providing a portion of the required operations (eg, as a bank of servers, a collection of blade servers, or a multi-processor system).

[0230]

[0253] The memory 2204 stores information within the computing device 2200. In some implementations, the memory 2204 is one or more volatile memory units. In some implementations, the memory 2204 is one or more non-volatile memory units. The memory 2204 may also be another form of computer-readable medium, such as a magnetic disk or optical disk.

[0231]

[0254] The storage device 2206 can provide mass storage for the computing device 2200. In some implementations, the storage device 2206 can be or include a computer-readable medium, such as a floppy disk device, a hard disk device, an optical disk device or a tape device, a flash memory or other similar solid-state memory device, or an array of devices, including a storage area network or other configuration of devices. The computer program product can be tangibly embodied in an information carrier. The computer program product can also include instructions that, when executed, perform one or more methods, such as those described above. The computer program product can be tangibly embodied in a computer or machine-readable medium, such as the memory 2204, the storage device 2206, or a memory on the processor 2202.

[0232]

[0255] The high-speed interface 2208 manages bandwidth-intensive operations for the computing device 2200, and the low-speed interface 2212 manages less bandwidth-intensive operations. Such an allocation of functions is merely an example. In some implementations, the high-speed interface 2208 is coupled to the memory 2204, the display 2216 (e.g., through a graphics processor or accelerator), and a high-speed expansion port 2210, which can accept various expansion cards (not shown). In this implementation, the low-speed interface 2212 is coupled to the storage device 2206 and the low-speed expansion port 2214. The low-speed expansion port 2214 can include various communication ports (e.g., USB, Bluetooth, Ethernet, wireless Ethernet) and can be coupled, for example, via a network adapter, to one or more input / output devices, such as a keyboard, a pointing device, a scanner, or a networking device, such as a switch or router.

[0233]

[0256] The computing device 2200 can be implemented in a number of different forms, as shown. For example, the computing device 2200 can be implemented as a standard server 2220 or multiple times within a cluster of such servers. In addition, the computing device 2200 can be implemented within a personal computer, such as a laptop computer 2222. The computing device 2200 can also be implemented as part of a rack server system 2224. Alternatively, components from the computing device 2200 can be combined with other components in a mobile device (not shown), such as a mobile computing device 2250. Each such device can include one or more of the computing device 2200 and the mobile computing device 2250, and the entire system can be comprised of multiple computing devices in communication with each other.

[0234]

[0257] The mobile computing device 2250 includes, among other components, a processor 2252, a memory 2264, input / output devices such as a display 2254, a communication interface 2266, and a transceiver 2268. The mobile computing device 2250 may also include a storage device, such as a microdrive or other device, to provide additional storage. Each of the processor 2252, memory 2264, display 2254, communication interface 2266, and transceiver 2268 are interconnected using various buses, and several of these components may be mounted on a common motherboard or in other suitable manners.

[0235]

[0258] The processor 2252 can execute instructions within the mobile computing device 2250, including instructions stored in the memory 2264. The processor 2252 can be implemented as a chipset of chips including multiple separate analog and digital processors. The processor 2252 can provide for the cooperation of other components of the mobile computing device 2250, such as control of a user interface, applications executed by the mobile computing device 2250, and wireless communication by the mobile computing device 2250.

[0236]

[0259] The processor 2252 can communicate with a user through a control interface 2258 and a display interface 2256, which is coupled to a display 2254. The display 2254 can be, for example, a TFT (thin film transistor liquid crystal display) display or an OLED (organic light emitting diode) display or other suitable display technology. The display interface 2256 can include appropriate circuitry to drive the display 2254 to present graphical and other information to a user. The control interface 2258 can receive commands from a user and translate those commands for transmission to the processor 2252. Additionally, an external interface 2262 can provide communication with the processor 2252 to enable short-range communication between the mobile computing device 2250 and other devices. The external interface 2262 can be provided, for example, for wired communication in some implementations or for wireless communication in other implementations, and multiple interfaces can also be used.

[0237]

[0260] The memory 2264 stores information within the mobile computing device 2250. The memory 2264 may be implemented as one or more of one or more computer readable media, one or more volatile memory units, or one or more non-volatile memory units. An expansion memory 2274 may also be provided and may be connected to the mobile computing device 2250 through an expansion interface 2272, which may include, for example, a SIMM (Single In-Line Memory Module) card interface. The expansion memory 2274 may provide extra storage space for the mobile computing device 2250 or may store applications or other information for the mobile computing device 2250. In particular, the expansion memory 2274 may include instructions for implementing or supplementing the processes described above and may also include secure information. Thus, for example, the expansion memory 2274 may provide a security module for the mobile computing device 2250 and may be programmed with instructions that enable secure use of the mobile computing device 2250. Additionally, secure applications can be provided via the SIMM card in a non-hackable manner, along with additional information such as location identification information on the SIMM card.

[0238]

[0261] The memory may include, for example, flash memory and / or NVRAM memory (non-volatile random access memory), as discussed below. In some implementations, the computer program product is tangibly embodied on an information carrier. The computer program product includes instructions that, when executed, perform one or more methods, such as those described above. The computer program product may be a computer or machine readable medium, such as memory 2264, expansion memory 22224, or memory on processor 2252. In some implementations, the computer program product may be received in a propagated signal, for example, via transceiver 2268 or external interface 2262.

[0239]

[0262] The mobile computing device 2250 can communicate wirelessly through a communication interface 2266, which can include digital signal processing circuitry, if necessary. The communication interface 2266 can provide communications under various modes or protocols, such as GSM voice calls (Global System for Mobile Communications), SMS (Short Message Service), EMS (Enhanced Messaging Service), or MMS messaging (Multimedia Messaging Service), CDMA (Code Division Multiple Access), TDMA (Time Division Multiple Access), PDC (Personal Digital Cellular), WCDMA (Wideband Code Division Multiple Access), CDMA2000, or GPRS (General Packet Radio Service), among others. Such communications can be through a transceiver 2268 using, for example, radio frequencies. Additionally, short-range communications can be performed using Bluetooth, Wi-Fi, or other such transceivers (not shown), etc. In addition, a GPS (Global Positioning System) receiver module 22220 can provide additional navigation- and location-related radio data to the mobile computing device 2250, such data can be used as appropriate by applications executing on the mobile computing device 2250.

[0240]

[0263] The mobile computing device 2250 can also communicate audibly using a voice codec 2260, which can receive verbal information from a user and convert it into usable digital information. The voice codec 2260 can also generate audible sounds for the user, such as through a speaker in a handset of the mobile computing device 2250. Such sounds can include sounds from a voice telephone call, can include recorded sounds (e.g., voice messages, music files, etc.), and can also include sounds generated by applications running on the mobile computing device 2250.

[0241]

[0264] The mobile computing device 2250 can be implemented in a number of different forms as shown in the figure. For example, the mobile computing device 2250 can be implemented as a cellular telephone 2280. The mobile computing device 2250 may also be implemented as part of a smartphone 2282, personal digital assistant, or other similar mobile device.

[0242]

[0265] Various implementations of the systems and techniques described herein can be realized in digital electronic circuitry, integrated circuits, specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs executable and / or interpretable on a programmable system including at least one programmable processor, which can be special-purpose or general-purpose, and can be coupled to receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0243]

[0266] These computer programs (also known as programs, software, software applications, or code) include machine instructions for a programmable processor and may be implemented in a high-level procedural and / or object-oriented programming language and / or assembly / machine language. As used herein, the terms machine-readable medium and computer-readable medium refer to any computer program product, apparatus, and / or device (e.g., magnetic disk, optical disk, memory, programmable logic device (PLD)) used to provide machine instructions and / or data to a programmable processor, including a machine-readable medium that receives the machine instructions as a machine-readable signal. The term machine-readable signal refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0244]

[0267] To provide for user interaction, the systems and techniques described herein can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user, and a keyboard and pointing device (e.g., a mouse or trackball) that allows the user to provide input to the computer. Other types of devices can be used to provide user interaction as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback) and can receive any form of input from the user, including acoustic, voice, or tactile input.

[0245]

[0268] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), middleware components (e.g., an application server), front-end components (e.g., a client computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described herein), or any combination of such back-end, middleware, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0246]

[0269] A computing system may include clients and servers. Clients and servers are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

[0247]

[0270] Although the specification contains many specific implementation details, these should not be construed as limitations to the scope of the disclosed technology or limitations to what may be claimed, but rather as descriptions of features that may be specific to certain embodiments of the disclosed technology. Certain features described herein with respect to separate embodiments may be implemented in combination, either partially or in whole, in one embodiment. Conversely, various features described with respect to one embodiment may be implemented separately in multiple embodiments or in any suitable subcombination. Furthermore, although features may be described herein and / or initially claimed as operating in a particular combination, one or more features from the claimed combination may, in some cases, be removed from that combination, and the claimed combination may be directed to a subcombination or a variation of the subcombination. Similarly, although operations may be described in a particular order, this should not be understood as requiring that such operations be performed in that particular order or sequentially, or that all operations be performed, to achieve desired results. Certain embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims.

[0248]

[0271] While the disclosed inventive concepts include those defined in the appended claims, it should be understood that the inventive concepts may also be defined by the following embodiments.

[0249]

[0272] In addition to the embodiments in the accompanying claims and described above, the following numbered embodiments are also innovative.

[0250]

[0273] Embodiment 1 includes a method of identifying duration of a product using image data, the method including: receiving, by a computing system, from an imaging device, image data of a product captured at consistent time intervals during a time period, the image data including one or more processed products coated with a shelf-life extension coating solution and one or more unprocessed products not coated with a shelf-life extension coating solution during the time period, the processed products and the unprocessed products being of the same product type; performing, by the computing system, an object detection technique on the image data to identify a bounding box around each product in the image data; identifying, by the computing system, quality attributes of each of the products within each bounding box at each of the time intervals during the time period; determining, by the computing system, for each of the products in the image data, one or more duration scoring metrics based on comparing the identified quality attributes of the product over the time period; and transmitting, by the computing system, the one or more duration scoring metrics of each of the products to a user device for display on a graphical user interface (GUI).

[0251]

[0274] Example 2 is the method of example 1, further comprising storing, by the computing system, in the data store, for each of the artifacts: (i) a bounding box; (ii) a quality attribute; and (iii) a duration scoring metric.

[0252]

[0275] Example 3 is the method of example 1 or 2, further including identifying, by the computing system, a grid structure in the image data; and assigning, by the computing system, a grid index within the grid structure to each bounding box, the grid index being used to identify a product within the bounding box.

[0253]

[0276] Embodiment 4 is the method of any one of embodiments 1-3, further comprising determining, by the computational system, a duration score for the product in the image data based on identifying an extended shelf-life duration from the one or more duration scoring metrics, the extended shelf-life duration experienced by the treated product compared to an untreated product over a period of time.

[0254]

[0277] Embodiment 5 is the method of embodiment 4, further comprising determining, by the computing device, one or more modifications to make to the shelf-life extension coating solution based at least in part on the one or more duration scoring metrics and duration score of the product.

[0255]

[0278] Example 6 is the method of example 5, further comprising sending, by the computing system, instructions to a controller of the supply chain participant to modify the shelf life extension coating solution based on the determined one or more modifications.

[0256]

[0279] Embodiment 7 is the method of embodiment 5, including instructions that, when the one or more changes are implemented by a supply chain participant, cause at least one of: (i) increasing a concentration of one or more components of the shelf-life extension coating solution; (ii) decreasing a concentration of one or more components of the shelf-life extension coating solution; (iii) applying the shelf-life extension coating solution to a batch of unprocessed product of the same product type; (iv) increasing an amount of the shelf-life extension coating solution applied to a subsequent batch of unprocessed product of the same product type; and (v) decreasing an amount of the shelf-life extension coating solution applied to a subsequent batch of unprocessed product of the same product type.

[0257]

[0280] Embodiment 8 is the method of any one of embodiments 1-7, wherein the quality attributes include at least one of color, volume, hardness, wrinkles, spoilage, and mold.

[0258]

[0281] Example 9 is the method of any one of Examples 1-8, further including: mapping, by a computing system, RGB values ​​from the image data into a three-dimensional (3D) space; identifying, by the computing system, clusters of RGB values ​​in the 3D space; selecting, by the computing system, one or more of the clusters in the 3D space having a threshold median color; and extracting, by the computing system, the selected one or more clusters from the image data, wherein the selected one or more clusters represent processed and unprocessed products in the image data.

[0259]

[0282] Embodiment 10 is the method of any one of embodiments 1 to 9, wherein identifying, by the computing system, a quality attribute of each of the products within each bounding box includes identifying color values ​​of all pixels within the bounding box, and determining, by the computing system, for each product in the image data, one or more duration scoring metrics includes identifying a distance between each of the identified color values ​​and a statistical scale color value of the product in the image data, determining whether the distance between each of the identified color values ​​and the statistical scale color value at each time interval exceeds a threshold level, identifying a duration scoring metric for each product as a time interval during the time period when the distance between the identified color value and the statistical scale color value exceeds the threshold level, and generating an output visually indicating the distance between each of the identified color values ​​and the statistical scale color value of the product at each time interval during the time period.

[0260]

[0283] Example 11 is the method of example 10, in which the statistical scale color value is determined by a calculation system based on receiving image data of the product for a first time interval during the time period from the imaging device, identifying color values ​​of all pixels representing the product in the image data for the first time interval, and calculating a statistical scale color value of the product in the image data based on taking an average of the identified color values.

[0261]

[0284] Embodiment 12 is the method of embodiment 11 or 12, wherein the statistical scale color value represents the average color of the treated product and the untreated product when the products are imaged by the imaging device at the first time interval during the time period.

[0262]

[0285] Embodiment 13 is the method of any one of embodiments 1-12, wherein the one or more duration scoring metrics correspond to at least one of the product's ripe for eating, end of ripe for eating, length of ripe for eating, shelf life, end of shelf life, length of shelf life, edibility, end of edibility, length of edibility, saleability, end of saleability, and length of saleability, for each of the products in the image data.

[0263]

[0286] Embodiment 14 is the method of any one of embodiments 1 to 13, wherein identifying, by the computing system, the quality attributes of each of the products within each bounding box includes: determining an area of ​​the product within the bounding box based on counting the number of pixels representing the product within the bounding box; determining a radius of the product based on the area and a formula for the area of ​​a circle; and determining a volume of the product at each time interval based on the radius and a formula for the volume of a sphere; and determining, by the computing system, one or more duration scoring metrics for each of the products in the image data further includes: determining a change in the volume of the product at each time interval relative to the volume of the product in a first time interval; determining whether the volume change exceeds a threshold level; identifying a duration scoring metric for each product as the time interval during the time period when the volume change exceeds the threshold level; and generating an output visually indicating the volume change of each product at each time period during the time period.

[0264]

[0287] Embodiment 15 is the method of embodiment 14, wherein the threshold level is a 10% volumetric shrinkage between the volume of the product in one of the time intervals and the volume of the product in the first time interval.

[0265]

[0288]

[0023] Embodiment 16 is the method of any one of embodiments 1 to 15, wherein identifying, by the computing system, the quality attributes of each of the products within each bounding box includes identifying a grid structure of the products within the bounding box, retrieving a machine learning trained wrinkle model from the data store, and applying the wrinkle model to each grid cell in the grid structure to identify one or more wrinkle patches; and determining, by the computing system, for each of the products in the image data, includes counting a percentage of grid cells of the bounding box that have wrinkle patches, determining whether the percentage of grid cells that have wrinkle patches exceeds a threshold level, identifying a duration scoring metric for the product as a percentage of wrinkle coverage based on a determination that the percentage of grid cells that have wrinkle patches exceeds the threshold level, and generating an output visually indicative of the percentage of wrinkle coverage of the product.

[0266]

[0289] Example 17 is the method of example 16, wherein the machine learning trained wrinkle model is trained using a set of training image data of other products of the same type, the set of training image data being annotated based on a previous identification of a first portion of the other product as wrinkled and a previous identification of a second portion of the other product as not wrinkled.

[0267]

[0290] Embodiment 18 is the method of any one of embodiments 1 to 17, wherein identifying, by the computing system, a quality attribute of each of the products within each bounding box includes identifying a grid structure of the products within the bounding box, retrieving a machine learning trained quality feature model from the data store, and applying the quality feature model to each grid cell in the grid structure to identify one or more quality features; and determining, by the computing system, one or more duration scoring metrics for each of the products in the image data includes counting a percentage of grid cells that have the quality feature of the bounding box, determining whether the percentage of grid cells that have the quality feature exceeds a threshold level, identifying a duration scoring metric for the product as a percentage of coverage of the quality feature based on a determination that the percentage of grid cells that have the quality feature exceeds the threshold level, and generating an output visually indicating the percentage of coverage of the quality feature of the product.

[0268]

[0291] Example 19 is the method of example 18, in which the machine learning trained quality feature model is trained using a set of training image data of other products of the same type, and the set of training image data is annotated based on a previous identification of a first portion of the other product as having one or more of the quality features and a previous identification of a second portion of the other product as not having the quality features.

[0269]

[0292] Embodiment 20 is the method of embodiment 18 or 19, wherein the quality characteristics include at least one of hardness, internal spoilage, external spoilage, internal infection, external infection, internal spoilage, external spoilage, dry matter content, pH, and sugar content.

[0270]

[0293] Embodiment 21 is the method of any one of embodiments 1-20, wherein the produce in the image data is an avocado, (i) the quality attribute is at least one of color, volume, firmness, and ripeness, and (ii) the one or more duration scoring metrics is at least one of a change in color over the time period, a change in volume over the time period, a change in firmness over the time period, and a change in ripeness over the time period.

[0271]

[0294] Embodiment 22 is any one of the methods of embodiments 1 to 20, wherein the product in the image data is lime, the quality attribute is at least one of color and volume, and the one or more duration scoring metrics is at least one of change in color and change in volume over a period of time.

[0272]

[0295] Embodiment 23 is any one of the methods of embodiments 1 to 20, wherein the produce in the image data is an apple, (i) the quality attribute is at least one of color and firmness, and (ii) the one or more duration scoring metrics is at least one of a change in color over a time period and a change in firmness over a time period.

[0273]

[0296] Embodiment 24 is the method of any one of embodiments 1 to 23, further comprising: identifying, by a computing system, a product type of the product in the image data at a first time interval during the time period; selecting, by the computing system, one or more types of duration scoring metrics from a plurality of duration scoring metrics for the identified product type; selecting, by the computing system, one or more threshold levels for the selected types of duration scoring metrics for the identified product type; identifying, by the computing system, quality attributes corresponding to each of the selected types of duration scoring metrics; and determining, by the computing system, the selected one or more duration scoring metrics based on a comparison of the identified quality attributes with the selected one or more threshold levels.

[0274]

[0297] Example 25 is the method of any one of Examples 1 to 24, further comprising retrieving, by a computing system, duration scoring metrics for the processed and unprocessed products over the time period from the data store, identifying, by the computing system, one or more differences between the duration scoring metrics for the processed products and the duration scoring metrics for the unprocessed products, and generating, by the computing system, an output indicative of the identified one or more differences.

[0275]

[0298] Embodiment 26 is the method of embodiment 25, wherein the output generated includes one or more modifications of the shelf-life extension coating solution, the one or more modifications being determined by the computing system based on the one or more differences identified.

[0276]

[0299] Embodiment 27 is the method of any one of embodiments 1-26, wherein the produce is at least one of avocado, lime, lemon, apple, and mango.

[0277]

[0300] Example 28 is the method of any one of Examples 1 to 27, wherein the image data includes at least one of color data, an RGB color model, hyperspectral image data, time lapse video data, and time lapse hyperspectral image data.

[0278]

[0301]

[0043] Embodiment 29 includes a system for identifying age of a product using image data, the system comprising: a conveyor system configured to transport a product along a path through a facility; one or more imaging devices positioned proximate the conveyor system and configured to capture image data of the product at consistent time intervals during the time period, where the product is of the same product type; and at least one computing system in communication with the one or more imaging devices, the at least one computing system configured to receive from the one or more imaging devices image data of the product captured at consistent time intervals during the time period, the image data representing one or more processed products coated with a shelf-life extending coating solution and one or more uncoated products coated with the shelf-life extending coating solution during the time period. and one or more unprocessed products that are not processed, where the processed product and the unprocessed product are of the same product type; performing object detection techniques on the image data to identify a bounding box around each product in the image data; identifying quality attributes of each of the products within each bounding box at each time interval during the time period, the quality attributes including at least one of color, volume, hardness, wrinkles, spoilage, and mold; determining, for each of the products in the image data, one or more duration scoring metrics based on comparing the identified quality attributes of the products over the time period; and transmitting the one or more duration scoring metrics of each of the products to a user device for display on a graphical user interface (GUI).

Claims

1. 1. A method for determining a duration of a product using image data, comprising: receiving, by a computing system, from an imaging device, image data of a product captured at consistent time intervals during a time period, the image data including one or more processed products that have been coated with a shelf-life extending coating solution and one or more unprocessed products that have not been coated with the shelf-life extending coating solution during the time period, the processed products and the unprocessed products being of the same product type; performing, by the computing system, an object detection technique on the image data to identify a bounding box around each artifact in the image data; identifying, by the computing system, quality attributes of each of the products within each bounding box at each of the time intervals during the time period; determining, by the computing system, for each of the artifacts in the image data, one or more duration scoring metrics based on comparing the identified quality attributes of the artifacts over the time period; transmitting, by the computing system, the one or more duration scoring metrics for each of the products to a user device for display on a graphical user interface (GUI); A method comprising:

2. 2. The method of claim 1, further comprising storing, by the computing system, in a data store, for each of the artifacts: (i) the bounding box, (ii) the quality attributes, and (iii) the duration scoring metric.

3. identifying, by the computing system, a grid structure for the image data; assigning, by the computing system, a grid index within the grid structure to each bounding box, the grid index being used to identify the product within the bounding box; The method of claim 1 further comprising:

4. 2. The method of claim 1, further comprising: determining, by the computing system, a duration score for the product in the image data based on identifying an extension in shelf-life duration from the one or more duration scoring metrics, the extension in shelf-life duration experienced by the treated product compared to the untreated product over the time period.

5. 5. The method of claim 4, further comprising determining, by the computing device, one or more modifications to make to the shelf-life extension coating solution based at least in part on the one or more duration scoring metrics and the duration score of the product.

6. 6. The method of claim 5, further comprising: transmitting, by the computing system, instructions to a supply chain participant controller to modify the shelf life extension coating solution based on the determined one or more modifications.

7. 6. The method of claim 5, wherein the one or more changes, when executed by a supply chain participant, include instructions to cause at least one of: (i) increasing a concentration of one or more components of the shelf-life extension coating solution; (ii) decreasing a concentration of one or more components of the shelf-life extension coating solution; (iii) applying the shelf-life extension coating solution to a batch of unprocessed products of the same product type; (iv) increasing an amount of the shelf-life extension coating solution applied to a subsequent batch of unprocessed products of the same product type; and (v) decreasing an amount of the shelf-life extension coating solution applied to a subsequent batch of unprocessed products of the same product type.

8. The method of claim 1 , wherein the quality attributes include at least one of color, volume, hardness, wrinkles, spoilage, and mold.

9. mapping, by the computing system, RGB values ​​from the image data into three-dimensional (3D) space; identifying, by the computing system, clusters of RGB values ​​in the 3D space; selecting, by the computing system, one or more of the clusters in the 3D space having a threshold median color; extracting, by the computing system, the selected one or more clusters from the image data, the selected one or more clusters representing the processed product and the unprocessed product in the image data; The method of claim 1 further comprising:

10. and identifying, by the computing system, a quality attribute of each of the products within each bounding box, comprising identifying color values ​​of all pixels within the bounding box; Determining, by the computing system, one or more duration scoring metrics for each artifact in the image data includes: determining a distance between each of the identified color values ​​and a statistical scale color value of the product in the image data; determining whether a distance between each of the identified color values ​​and the statistical scale color value in each of the time intervals exceeds a threshold level; identifying a duration scoring metric for each product as a time interval during said time period when a distance between said identified color value and said statistical scale color value exceeds a threshold level; generating an output visually indicative of a distance between each of the identified color values ​​and the statistical scale color value of the product at each time interval during the time period; The method of claim 1 , comprising:

11. The statistical scale color value is calculated by the calculation system as follows: receiving image data of the product from the imaging device for a first time interval during the time period; identifying color values ​​of all pixels representing the product in the image data during the first time interval; calculating the statistical scale color value of the product in the image data based on averaging the identified color values; The method of claim 10, wherein the target is determined based on:

12. 11. The method of claim 10, wherein the statistical scale color value represents an average color of the processed product and the unprocessed product when the products are imaged by the imaging device at a first time interval during the time period.

13. 2. The method of claim 1, wherein the one or more duration scoring metrics correspond, for each of the products in the image data, to at least one of the product's prime time, end of prime time, length of prime time, shelf life, end of shelf life, length of shelf life, edibility, end of edibility, length of edibility, saleability, end of saleability, and length of saleability.

14. Identifying, by the computing system, a quality attribute of each of the products within each bounding box, determining an area of ​​the product within the bounding box based on counting the number of pixels representing the product within the bounding box; determining a radius of the product based on the area and a formula for the area of ​​a circle; determining a volume of the product at each time interval based on the radius and a formula for the volume of a sphere; Including, Determining, by the computing system, one or more duration scoring metrics for each of the artifacts in the image data includes: determining a change in the volume of the product at each time interval relative to a volume of the product at a first time interval; determining whether the volume change exceeds a threshold level; identifying a duration scoring metric for each product as the time interval during said time period when said volume change exceeds said threshold level; generating an output visually illustrating the volume change of each product at each time period during said time period; The method of claim 1 further comprising:

15. 15. The method of claim 14, wherein the threshold level is a 10% volumetric shrinkage between the volume of the product in one of the time intervals and the volume of the product in the first time interval.

16. Identifying, by the computing system, a quality attribute of each of the products within each bounding box, identifying a grid structure of the production within the bounding box; Retrieving a machine learning trained wrinkle model from the data store; and applying the wrinkle model to each grid cell in the grid structure to identify one or more patches of wrinkles; Including, Determining, by the computing system, one or more duration scoring metrics for each of the artifacts in the image data includes: counting the percentage of grid cells that have said patch of wrinkles in said bounding box; determining whether the percentage of grid cells having the patch of wrinkles exceeds a threshold level; identifying a duration scoring metric for the product as a percent coverage of wrinkles based on a determination that the percentage of grid cells having the patch of wrinkles exceeds the threshold level; generating an output visually indicative of the percent coverage of wrinkles on the product; The method of claim 1 , comprising:

17. 17. The method of claim 16, wherein the machine learning trained wrinkle model is trained using a set of training image data of other products of the same type, the set of training image data being annotated based on a previous identification of a first portion of the other product as wrinkled and a previous identification of a second portion of the other product as not wrinkled.

18. Identifying, by the computing system, a quality attribute of each of the products within each bounding box, identifying a grid structure of the production within the bounding box; Retrieving a machine learning trained quality feature model from the data store; applying the quality feature model to each grid cell in the grid structure to identify one or more quality features; Including, Determining, by the computing system, one or more duration scoring metrics for each of the artifacts in the image data includes: counting a percentage of grid cells having the quality feature of the bounding box; determining whether the percentage of grid cells having the quality characteristic exceeds a threshold level; identifying a duration scoring metric for the product as a percent coverage of a quality feature based on a determination that the percentage of grid cells having the quality feature exceeds the threshold level; generating an output that visually indicates the percent coverage of a quality characteristic of the product; The method of claim 1 , comprising:

19. 20. The method of claim 18, wherein the machine learning trained quality feature model is trained using a set of training image data of other products of the same type, the set of training image data being annotated based on a prior identification of a first portion of the other product as having one or more of the quality features and a prior identification of a second portion of the other product as not having the quality features.

20. 20. The method of claim 18, wherein the quality characteristics include at least one of hardness, internal spoilage, external spoilage, internal infection, external infection, internal spoilage, external spoilage, dry matter content, pH, and sugar content.

21. 2. The method of claim 1, wherein the produce in the image data is an avocado, and (i) the quality attribute is at least one of color, volume, firmness, and ripeness, and (ii) the one or more duration scoring metrics is at least one of a change in color over the time period, a change in volume over the time period, a change in firmness over the time period, and a change in ripeness over the time period.

22. 2. The method of claim 1, wherein the product in the image data is lime, the quality attribute is at least one of color and volume, and the one or more duration scoring metrics is at least one of change in color and change in volume over the time period.

23. 2. The method of claim 1 , wherein the produce in the image data are apples, (i) the quality attribute is at least one of color and firmness, and (ii) the one or more duration scoring metrics is at least one of a change in color over the time period and a change in firmness over the time period.

24. identifying, by the computing system, a product type of the product in the image data at a first time interval during the time period; selecting, by the computing system, one or more types of duration scoring metrics from a plurality of duration scoring metrics for the identified product types; selecting, by the computing system, one or more threshold levels for duration scoring metrics of the selected types of the identified product types; identifying, by the computing system, the quality attributes corresponding to each of the selected types of duration scoring metrics; determining, by the computing system, the selected one or more duration scoring metrics based on a comparison of the identified quality attributes to the selected one or more threshold levels; The method of claim 1 further comprising:

25. retrieving, by the computing system, duration scoring metrics for the processed and unprocessed products over the time period from a data store; identifying, by the computing system, one or more differences between the duration scoring metric of the processed product and the duration scoring metric of the unprocessed product; generating, by the computing system, an output indicative of the identified one or more differences; The method of claim 1 further comprising:

26. 26. The method of claim 25, wherein the generated output includes one or more modifications to the shelf-life extension coating solution, the one or more modifications being determined by the computing system based on the identified one or more differences.

27. 10. The method of claim 1, wherein the produce is at least one of an avocado, a lime, a lemon, an apple, and a mango.

28. The method of claim 1 , wherein the image data includes at least one of color data, an RGB color model, hyperspectral image data, time lapse video data, and time lapse hyperspectral image data.

29. 1. A system for identifying a duration of a product using image data, comprising: a conveyor system configured to transport the product along a path through the facility; one or more imaging devices positioned proximate the conveyor system and configured to capture image data of the products at consistent time intervals during a time period, the products being of the same product type; at least one computing system in communication with the one or more imaging devices; Equipped with The at least one computing system receiving, from the one or more imaging devices, image data of the products captured at the consistent time intervals during the time period, the image data including one or more processed products that have been coated with a shelf-life extending coating solution and one or more unprocessed products that have not been coated with the shelf-life extending coating solution during the time period, the processed products and the unprocessed products being of the same product type; performing an object detection technique on the image data to identify a bounding box around each artifact in the image data; identifying quality attributes of each of the produce within each bounding box at each of the time intervals during the time period, the quality attributes including at least one of color, volume, hardness, wrinkles, spoilage, and mold; determining, for each of the artifacts in the image data, one or more duration scoring metrics based on comparing the identified quality attributes of the artifacts over the time period; transmitting the one or more duration scoring metrics for each of the products to a user device for display on a graphical user interface (GUI); A system configured to:

Images