System for configuring orchestration and operations of facility equipment

Abstract

A system and devices for automating the tracking, configuring, controlling, and scheduling of processing of assets, inventory, and / or items at a facility, such as a storage facility, shipping yard, processing plant, warehouse, distribution center, port, rail yard, rail terminal, and the like. In some examples, the system may be configured to track the location, age, and other statuses associated with the items as well as the facility equipment, containers, and the like. The system may generate a proactive plan for the operations of the facilities (e.g., future planned operations) and to generate a reactive plan associated with a current period of time to adjust and / or control the day-to-day operations of the facility.

Description

SYSTEM FOR CONFIGURING ORCHESTRATION AND OPERATIONS OFFACILITY EQUIPMENTCROSS-REFERENCE TO RELATED APPLICATION

[0001] This application claims priority to U.S. Provisional Application No. 63 / 713,169, filed on October 29, 2024 and entitled “System for Configuring Orchestration and Operations of Facility Equipment,'’ the entirety of which is incorporated herein by reference.BACKGROUND

[0002] Facilities, such as shipping yards, processing plants, warehouses, distribution centers, ports, yards, transports, and the like store and process vast quantities of assets, inventory, and items over various periods of time. In some situations, delays, conflicts, and other issues, such as missing or delayed delivery of inventory', may arise as the facility processes (both incoming and outgoing) inventory. In some cases, the operational equipment of the facility , such as vehicles, equipment, conveyors, autonomous systems, robotic equipment, and the like may be idled or otherwise underutilized.BRIEF DESCRIPTION OF THE DRAWINGS

[0003] The detailed description is described with reference to the accompanying figures. In the figures, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. The use of the same reference numbers in different figures indicates similar or identical components or features.

[0004] FIG. 1 is an example block diagram of an orchestration system and platform for configuring equipment and scheduling operations at a facility according to some implementations.

[0005] FIG. 2 is a flow diagram illustrating an example process associated with orchestration system operating at a facility' according to some implementations.

[0006] FIG. 3 is a flow diagram illustrating an example process associated with orchestration system operating at a facility according to some implementations.

[0007] FIG. 4 is a flow diagram illustrating an example process associated with orchestration system operating at a facility’ according to some implementations.

[0008] FIG. 5 is a block diagram of an example facility equipment / system configured to operate in conjunction with an orchestration system implementing proactive and reactive plans, according to some implementations.

[0009] FIG. 6 is an example orchestration system that may implement the techniques described herein according to some implementations.DETAILED DESCRIPTION

[0010] Discussed herein are systems and devices for automating the tracking, configuring, controlling, and scheduling of processing (e.g., packing, loading, unloading, assembling, tagging, and the like) of assets, inventory, and / or items at a facility, such as a storage facility, shipping yard, processing plant, warehouse, distribution center, port, rail yard, rail terminal, and the like. In some examples, an orchestration or scheduling system may be configured to track the location, age, and other statuses associated with the items as well as the facility equipment (e.g., vehicles, conveyors, autonomous systems, robotic systems, and the like), containers, and the like. For instance, the orchestration system may be configured to track assets, items, and / or inventory that are stored at the facility (e.g., at a preparation area, holding area, storage area, on vehicles or containers parked at the facility, and the like) as well as tracking equipment and controlling or configuring equipment at the facility7(such as conveyor speed or throughput, robotic arm processing parameters, combinations of equipment working in unison, and the like).

[0011] The orchestration system may also receive data associated assets, inventory, and items incoming to the facility7(e.g., enroute or planned shipments), at other facilities, outgoing or requested by third-party downstream venues (e.g., downstream processing venues, retail outlets, consumers homes, and the like), partner facilities (e.g., other facilities operating in coordination with the facility, such as multiple distributed facilities at various physical locations), and the like. The orchestration system may also track assets, items, and / or inventory7as the assets, items, and / or inventory7are moved about the facility, loaded and / or unloaded from one or more vehicles and / or containers, and the like. For instance, the system may receive sensor data associated with loading and / or unloading operations (e.g., the sensors may be associated with the loading / unloading areas, transportation vehicles, such as a forklift, and / or attached to facility operators).L.©© & H © ©s

[0012] The orchestration system may also receive sensor data associated with vehicles arriving and departing the facility as well as during the loading and unloading of the vehicles (e.g., via an exit or entry location sensor system). In some cases, the system may also receive location data and / or asset data from one or more vehicles scheduled to make a delivery' to the facility7and / or accept a shipment from the facility7. In some implementations, the system may also receive data associated with the equipment and / or personnel operating at the facility' (such as shift data, configuration data, proximate supplies of items and / or inventory, operational data, such as downtime, and the like).

[0013] In some examples, the orchestration system may also receive data associated with consumption of assets, items, and / or inventory at point of sale or retail locations. For example, the system may receive data representing a supply of a specific item at a retail venue and / or a rate of consumption, estimated rate of consumption or the like of the specific item. In some cases, the consumption data may be utilized to preplan or forecast operational needs into a future period of time. In some cases, the consumption data may be associated with a direct or in-direct customer (e.g., a downstream venue of a direct customer of the facility, or the like). In this manner, the orchestration system may have consumption data associated with assets, items, and / or inventory' at various stages of the items lifecycle as the items are processed sold, resold and the like until reaching the final destination and / or consumer.

[0014] In various examples, the consumption data may be received from one or more downstream facilities including a retail venue and / or one or more intermediate shipping or distribution facilities. In one specific example, the consumption data may be received from a device associated with the end consumer. For example, an application hosted on a mobile device may send the consumption data for individual items that are purchased from a retail venue, such as when the mobile device is used to scan, process, pay checkout or the like the item from the retail venue. As another example, the consumption data may be received from a pay ment processor system (such as a credit card or debit card processing system) that is used to purchase the item from a specific retailer. In this manner, the consumption data may be collected and aggregated from multiple third-party7sources to determine, for example, a rate of consumption at specific retail venues and / or determine an estimate (e.g., estimated period of time, destination - such as a next downstream facility from the current facility,L.©© & H © ©squantity' of items, skew number, item instance or type, and / or the like) for a next order or subsequent order, various machine readable codes, and / or the like.

[0015] Likewise, the incoming asset or item data may be received from one or more upstream facilities and / or transportation vehicles (such as a delivery vehicle), in some cases, the upstream facilities may include customs facilities or organizations, governmental entities, ports, freight forwarders, shipping companies (e.g., facilities and / or vehicles), processing plants, assembly plants, and / or the like, chain of title entities, and / or the like.

[0016] In some cases, the orchestration system may also store or receive assets, items, and / or inventory data and / or scheduling data associated with scheduled shipments and / or deliveries from, for instance, the various third-party systems. For example, the system may receive lists of assets to be shipped from the facility and / or delivered to the facility' . In this manner, the orchestration system may store the location and status of each asset and / or item arriving at the facility implementing the orchestration system, stored at the facility, and departing the facility’. For example, if a vehicle arrives at the facility, the orchestration system may have a list of assets and / or items stored on the vehicle and check in the vehicle via sensor data of the vehicle captured as the vehicle arrives.

[0017] The orchestration system may also receive data associated with the items and / or assets as the vehicle and / or container is unloaded, such as via sensors at the door docks, check in / out gates, within the yard or interior of the facilities, associated yvith operators, mounted on facility’ equipment, and / or the like. In some cases, the orchestration system may track via the sensor data associated yvith the unloading / loading the assets and / or items and may store and / or record a location the assets are stored within the facility and any status data associated with the assets (e.g., damage, age, inventory quantity, container quality, and the like). The orchestration system may also track the assets as the assets are loaded onto a vehicle for shipment via sensor data associated with the loading area, the loading equipment (e.g., a forklift or the like), on personnel, or associated with the vehicle. The orchestration system may then store, update, and / or record a location the assets are leaving the facility and any status data associated yvith the assets as the assets depart the facility.

[0018] In this manner, the orchestration system may have sensor data and location data as well as status data for each asset and / or item on premise, each asset and / item that has departed the facility (e.g., through downstream facilities and in some cases upto the end consumer possession of the item), and / or arriving at the facility. The orchestration system may then be configured to schedule operations, adjust equipment settings and / or parameters, generate operational plans, and / or the like. For example, the orchestration system may generate a proactive plan for the operations of the facilities (e.g., future planned operations) including human operators (e.g., shift schedules, assignments, personnel requirements on each shift, number of shifts, overtime if necessary, and / or the like), equipment settings (operational speeds, tasks, assignments, combination of equipment, and / or like), storage location for assets and / or items (such as moving items to a processing area, loading area, long term storage, short term storage and / or the like to improve overall operational flow and production rates at the facility), assets or items to be processed (e.g., sequence or order for which requests and orders are fulfilled, processed, or the like, manage delays and over productions, and / or the like), place orders on behalf of the facility with upstream venders and / or facilities, schedule delivers of assets and / or items, and the like.

[0019] As an example, the orchestration system may be configured to generate and / or place orders on behalf of the facility and automatically (e.g., without human input and / or authorization) and in response to outputs from one or more machine learned models or networks trained on and configured to receive downstream data associated with assets and items, upstream data associated with assets and items, and inventory data associated assets and / or items at the facility generate and / or update (e.g., modify) the proactive plan for the facility, as discussed herein.

[0020] The orchestration system may also be configured to generate a reactive plan associated with a current (e.g., within a sliding window of time from a present time) to adjust and / or control the day to day operations of the facility in response to real-time and / or concurrent data received from upstream facilities or venues, downstream facilities and / or venues, and facility systems and / or sensors. For example, the orchestration system may be configured to generate the proactive plan for future planning and the reactive plan for adjusting and controlling the day to day operations of the facility.

[0021] As one example, the reactive plan may include sending control signals to adjust operations of autonomous systems or robotic operations in substantially realtime in response to incoming data related to deliveries, orders, injuries, equipment malfunctions, and / or the like. For instance, the orchestration system may be configured to generate and / or adjust settings for the autonomous systems and robotics (e.g., belts,arms, vehicles, and / or the like), orders on behalf of the facility and automatically (e.g., without human input and / or authorization), and in response to outputs from one or more machine learned models or networks trained on and configured to receive downstream data associated with assets and items, upstream data associated with assets and items, and inventory data associated assets and / or items at the facility, generate the proactive plan (e.g., the proactive plan may be an input to generate the reactive plan), as discussed herein.

[0022] As another example, the orchestration system may be configured to generate and / or adjust personnel assignments and / or operational tasks (e.g., which items are being processed) for facility personnel and / or teams, such as by sending instructions to various facility devices and / or personnel devices, on behalf of the facility and automatically (e.g., without human input and / or authorization), and in response to outputs from one or more machine learned models or networks trained on and configured to receive downstream data associated with assets and items, upstream data associated with assets and items, and inventory data associated assets and / or items at the facility, generate the proactive plan (e.g.. the proactive plan may be an input to generate the reactive plan), as discussed herein.

[0023] For instance, the orchestration system may in response to receiving data that a deliver}' is ahead of schedule adjust (e.g., increase) a throughput of an assembly line (e.g., by changing settings on the autonomous systems and sending instructions to the personnel) in substantially real-time, such that the facility is ready to receive the delivery at the new estimated time of arrival. As another example, the orchestration system may reassign a robotic system from a first line to a second line in response to receiving data that another robotic system on the second line has malfunctioned together with data indicating that the items associated with the second line are shipping out prior to the first line. In this manner, the second line suffer only minimal delays and the ripple effects of the malfunction may be minimized over the subsequent time period (e.g., a next process for a next item on the second line may require priority over the productions scheduled on the first line and the like).

[0024] In some examples discussed herein, the sensors may be internet of things (loT) computing devices that may be equipped with various sensor and / or image capture technologies, and configured to capture, parse, and identify vehicle and container information from the exterior of vehicles, containers, pallets, and the like. The vehicle and container information may include shipping documents, such as BOL(Bill of Lading), packing list, container identifiers, chassis identifiers, vehicle identifiers, and the like. The loT computing devices may also capture, parse, and identify driver information in various formats, such a driver licenses, driver’s identification papers, facial features and recognition, and the like.

[0025] As discussed above, the system may include multiple loT devices at various locations as well as cloud-based services, such as cloud-based data processing. One or more loT computing device(s) may be installed at entry and / or exit points of a facility. The loT computing devices may include a smart network video recorder (NVR) or other type of EDGE computing device. Each loT device may also be equipped with sensors and / or image capture devices usable at night or during the day. The sensors may be weather agnostic (e.g., may operate in foggy, rainy, or snowy conditions), such as via infrared image systems, radar based image systems, LIDAR based image systems, SIWIR based image systems, Muon based image systems, radio wave based image systems, and / or the like. The loT computing devices and / or the cloud-based services may also be equipped with models and instructions to capture, parse, identify, and extract information from the vehicles, containers, and / or various documents associated with the logistics and shipping industry. For example, the loT computing devices and / or the cloud-based services may be configured to perform segmentation, classification, attribute detection, recognition, document data extraction, and the like. In some cases, the loT computing devices and / or an associated cloud-based service may utilize machine learning and / or deep learning models to perform the various tasks and operations.

[0026] In some cases, since the sensor data received may be from different sources or types of sensors at different ranges and generalities, the loT computing devices may perform a data normalization using techniques such as threshold-based data normalization and machine learning algorithms to identify7the driver, vehicle, or container. It should be understood that the system may utilize different weighted averages or thresholds based on the data source (e.g., sensor type, location, distance, and position), the current weather (e.g., sunny, rainy, snowy, or foggy), and time of day when performing data normalization. In some cases, machine learning algorithms may also be applied to remove the distortion from images caused by rain, dust, sand, fog, and the like as well as to brighten the sensor and / or images shot in low-light or dark conditions.L.©© & H © ©s

[0027] As described herein, the machine learned models may be generated using various machine learning techniques. For example, the models may be generated using one or more neural network(s). A neural network may be a biologically inspired algorithm or technique which passes input data (e.g., image and sensor data captured by the loT computing devices) through a series of connected layers to produce an output or learned inference. Each layer in a neural network can also comprise another neural network or can comprise any number of layers (whether convolutional or not). As can be understood in the context of this disclosure, a neural network can utilize machine learning, which can refer to a broad class of such techniques in which an output is generated based on learned parameters.

[0028] As an illustrative example, one or more neural network(s) may generate any number of learned inferences or heads from the captured sensor and / or image data. In some cases, the neural network may be a trained network architecture that is end-to- end. In one example, the machine learned models may include segmenting and / or classifying extracted deep convolutional features of the sensor and / or image data into semantic data. In some cases, appropriate truth outputs of the model in the form of semantic per-pixel classifications (e.g., vehicle identifier, container identifier, driver identifier, and the like).

[0029] Although discussed in the context of neural networks, any ty pe of machine learning can be used consistent with this disclosure. For example, machine learning algorithms can include, but are not limited to, regression algorithms (e g., ordinary7least squares regression (OLSR), linear regression, logistic regression, stepwise regression, multivariate adaptive regression splines (MARS), locally estimated scatterplot smoothing (LOESS)), instance-based algorithms (e.g., ridge regression, least absolute shrinkage and selection operator (LASSO), elastic net, least-angle regression (LARS)), decisions tree algorithms (e.g., classification and regression tree (CART), iterative dichotomiser 3 (ID3), Chi-squared automatic interaction detection (CHAID), decision stump, conditional decision trees), Bayesian algorithms (e g., naive Bayes, Gaussian naive Bayes, multinomial naive Bay es, average one-dependence estimators (AODE), Bayesian belief network (BNN), Bayesian networks), clustering algorithms (e.g., k- means, k-medians, expectation maximization (EM), hierarchical clustering), association rule learning algorithms (e.g., perceptron, back-propagation, hopfield network, Radial Basis Function Network (RBFN)), deep learning algorithms (e.g., Deep Boltzmann Machine (DBM), Deep Belief Networks (DBN), ConvolutionalNeural Network (CNN), Stacked Auto-Encoders), Dimensionality Reduction Algorithms (e.g., Principal Component Analysis (PCA). Principal Component Regression (PCR), Partial Least Squares Regression (PLSR), Sammon Mapping, Multidimensional Scaling (MDS), Projection Pursuit, Linear Discriminant Analysis (LDA), Mixture Discriminant Analysis (MDA), Quadratic Discriminant Analysis (QDA), Flexible Discriminant Analy sis (FDA)), Ensemble Algorithms (e.g., Boosting, Bootstrapped Aggregation (Bagging), AdaBoost, Stacked Generalization (blending), Gradient Boosting Machines (GBM), Gradient Boosted Regression Trees (GBRT), Random Forest), SVM (support vector machine), supervised learning, unsupervised learning, semi-supervised learning, etc. Additional examples of architectures include neural networks such as ResNet50, ResNetlOl, VGG. DenseNet, PointNet, and the like. In some cases, the system may also apply Gaussian blurs, Bayes Functions, color analyzing or processing techniques and / or a combination thereof.

[0030] In some implementations, the loT computing devices may also be configured to estimate one or more statuses of the contents of the containers, crates, and the like as the vehicles enter and exit the facility. For example, the loT computing devices may use various types of a sensors (e.g., LIDAR, SIWIR, Radio Wave, Muon, etc.), with capabilities such as but not limited to varying fields of view, along with the camera or image systems and edge computing capabilities to detect various attributes such as container damage, leakage, size, weight, and the like of a vehicle, chassis, and / or container. In this manner, the loT computing devices may operate as part of a network, loT, colocation, Wi-Fi, local-zones, Bluetooth Low Energy, or the like to provide a comprehensive diagnostic of the physical attributes of a vehicle, truck, trailer, chassis, rail car, cargo, ship, and / or container during entry and exit of the facility. In some cases, the loT computing devices and / or the cloud-based services may be used to identify vehicles, chassis, and / or containers that require maintenance prior to further deployment.

[0031] FIG. 1 is an example block diagram 100 of an orchestration system 102 and platform for configuring equipment and scheduling operations at a facility according to some implementations. As discussed herein, the orchestration system 102 may be configured to track the location, age, and other statuses associated with the items as well as assets and equipment (e.g., vehicles, conveyors, autonomous systems, robotic systems, and the like), containers, and the like. For instance, the orchestration system 102 may be configured to track the assets, items, and / or inventory that are stored at thefacility (e.g., at a preparation area, holding area, storage area, on vehicles or containers parked at the facility, and the like) as well as tracking equipment and controlling or configuring equipment at the facility (such as conveyor speed or throughput, robotic arm processing parameters, combinations of equipment working in unison, and the like).

[0032] In the illustrated example, the orchestration system 102 may receive various types of data associated with the items and / or assets from various sources. For instance, the orchestration system 102 may receive inventory data 104 from one or more facility sensor systems 106 associated with the items and / or inventory stored at the facility , entering the facility, exiting the facility, and / or the like. The orchestration system 102 may also receive equipment / asset data 108 from the one or more facility sensor systems 106 associated with the facility equipment and / or assets (e.g., containers, ramps, personnel, and / or the like) operating at the facility', entering the facility, exiting the facility, and / or the like. In this manner, the orchestration system 102 may track incoming items and / or assets, outgoing items and / or assets, and stored items and / or assets.

[0033] The orchestration system 102 may also receive the inventory data 104 and / or the equipment data 108 from facility' systems 110 (e.g., forklifts, autonomous robotic systems, yard mules, tugs, toes, picking systems, inventory' retrial systems, and / or the like). For instance, as the facilities systems 110 process the inventory (e.g., picking, assembly, disassembly, testing, quality’ review, packing, loading, unloading, shipping, and / or the like), the facility systems 110 may generate the equipment / asset data 108 and the inventory data 104 and provide the equipment / asset data 108 and the inventory data 104 in substantially real-time or concurrently with the processing to the orchestration system 102. In this manner, via the facility sensors 106 and the facility systems 110 (including wearable, portable, or other equipment of personnel), the orchestration system 102 may receive substantially real-time data associated with the overall operations of the facility.

[0034] In some cases, the orchestration system 102 may also receive item data 112, order data 114, consumption data 1 16, and / or the like from third-party systems 118. For example, the orchestration system 102 may receive order data 114 from third-party' downstream vendors or end-customers. Likewise, the orchestration system 102 mayreceive item data 112 and / or consumption data 116 from consumers devices, such as hosting downloadable applications, downstream facilities, vehicles, vendors, and / or thelike, upstream facilities, vehicles, and / or the like, shipping or freight forwarding entities, government entities, and / or the like. For instance, the consumption data 116 may be received from the consumer devices, point-of-sale devices at retail venues, payment processing systems, on-line based ordering systems, and / or the like.

[0035] The orchestration system 102 may also receive item data 112 and / or deliverydata 118 (such as location, estimated time of arrival, and the like) from vehicle sensor systems 120, and / or the like from third-party systems 118. For example, as vehicles are enroute to the facility, the vehicle sensor systems 120 may send or otherwise provide their location data as well as speed, velocity, various traffic conditions (which may also be obtained from various cloud-based and / or mapping services), projected routes, estimated time of arrivals, and / or the like. In some specific examples, the orchestration system 102 may receive either from the vehicle sensor systems 120 and / or the third- party systems 126 data related to facility congestion (such as airports, docks, ports of entry, shipping lanes, and the like).

[0036] In some implementations, based at least in part on the inventory data 104, the equipment / asset data 108, the item data 112, the order data 114, the consumption data 11 , the deliver}- data 118, the orchestration system 102 may determine a proactive plan 122 associated with the operations of the facility. The proactive plan 122 may project into the future for a predetermined period of time (a day, a number of days, a week, a month, a quarter, and / or the like) the planned operations of the facility. In some cases, the proactive plan 122 may include data indicating a number of processes to be performed in parallel or in series (such as by multiple production lines, multiple packing areas, multiple dock doors, multiple deliver}- areas, and / or the like), production of individual products, quantity of products, assigned personal and / or equipment for each process, settings and / or configuration for autonomous or semi-autonomous systems, facility storage / packing arrangements (such as locations in the facility to store inventory and / or items, and the like), estimated throughput, operational hours per process, and / or the like.

[0037] As an example, the orchestration system 102 may be configured to generate (and / or update) the proactive plan on behalf of the facility and automatically (e.g., without human input and / or authorization) and in response to outputs from one or more machine learning models or networks trained on and configured to receive as an input the inventory data 104. the equipment / asset data 108, the item data 112, the order data 114, the consumption data 1 16, the deliver}- data 118. For example, the orchestrationsystem 102 may receive the data 104, 108. 114, 116, and 118 in substantially real time and as the data is received input the data into the one or more machine learning models or networks. As an output of the one or more machine learning models or networks the orchestration system 102 may receive the updated proactive plan 122. In this manner, the proactive plan 122 may be updated in substantially real-time, for instance, for a sliding window of time based on the current time.

[0038] In some examples, the one or more machine learning models or networks may be trained on historical data including historical inventory data, equipment / asset data, item data, order data, consumption data, the delivery data, as well as other operational data. In some specific examples, the one or more machine learning models or networks may segment, classify, and / or arrange the incoming data to generate the proactive plan 122 as part of one or more heads of a neural network. In some cases, the one or more machine learning models or networks may be a neural network that is trained end to end to generate the proactive plan 122 for the sliding window' of time.

[0039] In some cases, the orchestration system 102 may also be configured to generate a reactive plan 124 associated with the operations of the facility. For example, the orchestration system 102 may receive the proactive plan 122 as w ell as the inventory data 104, the equipment / asset data 108, the item data 112, the order data 114, the consumption data 116, the delivery data 118 in substantially real-time and utilize the proactive plan 122 as well as the inventor)’ data 104, the equipment / asset data 108. the item data 112, the order data 114, the consumption data 116, the delivery data 118 to generate a reactive plan for the current operations of the facility (such as within a second sliding window' of time that is smaller than the window of time associated w ith the proactive plan 122). For instance, the proactive plan 122 may be associated with the operation of the facility for the upcoming week while the reactive plan 124 may be associated with the operations of the facility for the current day or shift.

[0040] In the various examples, both the proactive plan 122 and the reactive plan 124 may include control signals that are provide to autonomous and semi-autonomous robotic systems throughout the facility and may be used to control the autonomous and semi-autonomous robotic systems without the need for human input. In some cases, the control signals may be adjusted on the fly and in substantially real-time by the orchestration system 102 using the machine learning models (such as large language models and artificial intelligence systems).L.©© & H © ©s

[0041] In some cases, based at least in part on the inventory data 104, the equipment / asset data 108, the item data 112, the order data 114, the consumption data 116, the delivery data 1 18, the orchestration system 102 may determine a proactive plan 122 associated with the operations of the facility. The proactive plan 122 may project into the future for a predetermined period of time (a day, a number of days, a week, a month, a quarter, and / or the like) the planned operations of the facility. In some cases, the proactive plan 122 may include data indicating a number of processes to be performed in parallel or in series (such as by multiple production lines, multiple packing areas, multiple dock doors, multiple delivery areas, and / or the like) production of individual products, quantity of products, assigned personal and / or equipment for each process, settings and / or configuration for autonomous or semi-autonomous systems, facility storage / packing arrangements (such as locations in the facility to store inventory and / or items, and the like), estimated throughput, operational hours per process, and / or the like.

[0042] As an example, the orchestration system 102 may be configured to generate (and / or update) the proactive plan on behalf of the facility and automatically (e.g., without human input and / or authorization) and in response to outputs from one or more machine learning models or networks trained on and configured to receive as an input the inventory data 104, the equipment / asset data 108, the item data 112, the order data 114, the consumption data 116, the delivery’ data 118. For example, the orchestration system 102 may receive the inventory data 104, the equipment / asset data 108, the item data 112, the order data 114, the consumption data 116, and the delivery data 118 in substantially real time and as the data is received input the data into the one or more machine learning models or networks. As an output of the one or more machine learning models or networks the orchestration system 102 may receive the updated proactive plan 122. In this manner, the proactive plan 122 may be updated in substantially realtime, for instance, for a sliding window of time based on the current time.

[0043] In some examples, the one or more machine learning models or networks may be trained on historical data including historical inventory data, equipment / asset data, item data, order data, consumption data, the delivery data, as well as other operational data. In some specific examples, the one or more machine learning models or networks may segment, classify, and / or arrange the incoming data to generate the proactive plan 122 as part of one or more heads of a neural network. In some cases, theL.©© & H © ©sone or more machine learning models or networks may be a neural network that is trained end to end to generate the proactive plan 122 for the sliding window of time.

[0044] In the current example, the data 104, 108, 112, 114, and 118 as well as the proactive plans 122 and the reactive plans 124 as well as other data may be transmitted between various systems using networks, generally indicated by 128-134. The networks 128-134 may be any type of network that facilitates compunction between one or more systems and may include one or more cellular networks, radio, WiFi networks, short- range or near-field networks, infrared signals, local area networks, wide area networks, the internet, and so forth. In the current example, each network 128-134 is show n as a separate netw ork but it should be understood that tw o or more of the networks may be combined or the same.

[0045] FIGS. 2-4 are flow diagrams illustrating example processes associated with the orchestration system discussed herein. The processes are illustrated as a collection of blocks in a logical flow diagram, which represent a sequence of operations, some or all of which can be implemented in hardware, software, or a combination thereof. In the context of software, the blocks represent computer-executable instructions stored on one or more computer-readable media that, which when executed by one or more processor(s), perform the recited operations. Generally, computer-executable instructions include routines, programs, objects, components, encryption, deciphering, compressing, recording, data structures and the like that perform particular functions or implement particular abstract data types.

[0046] The order in which the operations are described should not be construed as a limitation. Any number of the described blocks can be combined in any order and / or in parallel to implement the processes, or alternative processes, and not all of the blocks need be executed. For discussion purposes, the processes herein are described with reference to the frameworks, architectures and environments described in the examples herein, although the processes may be implemented in a wide variety of other frameworks, architectures or environments.

[0047] FIG. 2 is a flow diagram illustrating an example process associated with orchestration system 200 operating at a facility according to some implementations. In the current example, the orchestration system may include a forecasting system 202 to generate a proactive plan 204 for a period of time, as discussed above, and an adjustment system 206 to generate the reactive plan 208 in substantially real-time as operations are being performed at the facility.

[0048] In the current example, the forecasting system 202 may receive consumption data 210, delivery data 212, order data 214, equipment / asset data 216, item data 218, inventory data 220, and the like as discussed above with respect to FIG 1. The forecasting system 202 of the orchestration system 200 may process the incoming data 210-220 as well as any additional inputs such as operational plans, operator inputs, personnel data, or the like provided by an operator of the orchestration system 200 to generate the proactive plan 204 for the facility.

[0049] The proactive plan 204 may include configurations, settings, parameters, and the like for operations and autonomous and / or semi -autonomous systems of the facility in the future. The proactive plan 204 may also include planned operations, expected throughput, operational shifts, personal and / or equipment assignments, product productions, task to area assignments, and the like. In some cases, the proactive plan 204 may project into the future for a first predetermined period of time (a day, a number of days, a week, a month, a quarter, and / or the like) or a first sliding window of time the planned operations of the facility.

[0050] In some cases, the proactive plan 204 may include data indicating a number of processes to be performed in parallel or in series (such as by multiple production lines, multiple packing areas, multiple dock doors, multiple delivery areas, and / or the like) production of individual products, quantity of products, assigned personal and / or equipment for each process, settings and / or configuration for autonomous or semi- autonomous systems, facility storage / packing arrangements (such as locations in the facility to store inventory' and / or items, and the like), estimated throughput, operational hours per process, and / or the like.

[0051] In some cases, the forecasting system 202 may be configured to generate (and / or update) the proactive plan 204 on behalf of the facility and automatically (e.g., without human input and / or authorization) and in response to outputs from one or more machine learning models or networks trained on and configured to receive as an input the data 210-220 and the like. For example, the forecasting system 202 may receive the data 210-220 in substantially real-time and as the data is received input the data into the one or more machine learning models or networks. As an output of the one or more machine learning models or networks the forecasting system 202 may receive the updated proactive plan 204. In this manner, the proactive plan 204 may be updated in substantially real-time, for instance, for the first sliding window of time based on the current time.

[0052] In some examples, the one or more machine learning models or networks may be trained on historical data including historical data associated with the data 210- 220. In some specific examples, the one or more machine learning models or networks may segment, classify, and / or arrange the incoming data to generate the proactive plan 204 as part of one or more heads of a neural network. In some cases, the one or more machine learning models or networks may be a neural network that is trained end to end to generate the proactive plan 204 for the sliding window of time.

[0053] The orchestration system 200 may also include an adjustment system 206 that may receive the data 210-220 and / or the proactive plan 204 as an input. The adjustment system 206 may also receive sensor data 222 associated with the operation of the facility in substantially real-time or concurrently with operations being performed at the facility. The sensor data 222 may include data associated with the types of the data 210-220, such as updated equipment data, updated asset data, and the like. The adjustment system 206 may then output the reactive plan 208 that may be used to control and implement day to day operations of the facility’ based on current conditions and the proactive plan 204. In this manner, the reactive plan 208 may allow the facility to adapt to changing circumstances in substantially real-time or concurrently as the circumstances change. Accordingly, a facility7implementing the orchestration system 200 provides for technical improvements that provide tangible or concrete improvements over conventional facilities by improving controls and operations of autonomous systems, semi-autonomous systems, robotic systems, as well as operations associated with the facility7.

[0054] In the current example, the reactive plan 208 may include configuration data 224 (e.g.. equipment arrangements, equipment set-ups, equipment positioning, and the like), control signals 226 (e.g., signals to cause the equipment to perform tasks either autonomously or semi-autonomously and the like), setting data 228, parameter data 230, as well as other plan data 232 (e.g., operational assignments, personnel assignments, equipment assignments, throughput assignments, product processing assignments, and / or the like). In some cases, the configuration data 224, the control signals 226, the setting data 228, and / or the parameter data 230 may be used to control autonomous and semi-autonomous robotic systems throughout the facility without the need for human input. In some cases, the configuration data 224, the control signals 226, the setting data 228, the parameter data 230 of the reactive plan 208 may be adjusted on the fly and in substantially real-time by the adjustment system 206 of theorchestration system 200 using the machine learning models (such as large language models and artificial intelligence systems).

[0055] As an example, the adjustment system 206 may be configured to generate (and / or update) the reactive plan 208 on behalf of the facility and automatically (e.g., without human input and / or authorization) and in response to outputs from one or more machine learning models or networks trained on and configured to receive as an input the data 210-220, as well as the updates sensor data 222 and the proactive plan 204. For example, the adjustment system 206 may receive the sensor data 222 in substantially real time and, as the data 222 is received, input the data into the one or more machine learning models or networks. As an output of the one or more machine learning models or networks the adjustment system 206 may receive the updated proactive plan 204. In this manner, the proactive plan 204 may be updated in substantially real-time, for instance, based on a current operation at the facility.

[0056] In some cases, the reactive plan 208 and / or the proactive plan 204 may be provided to facility systems 234. such as the autonomous or semi-autonomous systems for controlling the systems. The facility systems 234 may also include vehicles, displays or electronic devices for presenting information to personnel at the facility and the like.

[0057] FIG. 3 is a flow diagram illustrating an example process 300 associated with orchestration system operating at a facility according to some implementations. As discussed above, the orchestration system may determine a proactive plan for the operations of the facility that forecast the expected operations and / or throughput of the facility (e.g., product quantity and / or type produced, shipped, manufactured, and the like) over a given period of time, such as a sliding window of time (e.g., one day in advance, a number of days in advance, one week in advance, a number of weeks in advance, a month in advance, a quarter in advance, a year in advance, and the like).

[0058] At 302, the orchestration system may receive data associated with a facility. As discussed above the data may include consumption data associated with a downstream venue, delivery / shipping data associated with incoming and outgoing orders, order data, equipment / asset data, item data, inventory data, and / or the like. In some cases, the data may be received from downstream or upstream venues, vehicles enroute to or from the facility, equipment or personnel operating at the facility, sensors systems associated with the facility, third-party' systems and / or the like.

[0059] At 304, the orchestration system may determine, based at least in part on the data associated with the faculty, a proactive plan over a sliding window of time for thefacility, the facility including one or more equipment or machinery'. For example, the orchestration system may be configured to input the data into the one or more machine learning models or networks and receive the proactive plan as an output of the one or more machine learning models or networks based at least in part on the sliding window of time. In some cases, the proactive plan may be generated at various intervals and / or in substantially real-time as the data is received. For instance, the data may be updated every hour, number of hours, day, week, and / or the like and the orchestration system may generate, such as via the one or more machine learning models and / or networks, a proactive plan for one or more of sliding window of time moving forward from a current period of time or future selected starting point (e.g., a proactive plan for a subsequent week starting on the next Monday at 8am or the like).

[0060] In some examples, the one or more machine learning models or networks may be trained on historical data including historical data associated with downstream venues, delivery / shipping data associated with incoming and outgoing orders, orders, equipment / asset performance and operations, item data, inventory, and / or the like. In some specific examples, the one or more machine learning models or networks may segment, classify, and / or arrange the incoming data to generate the proactive plan as part of one or more heads of a neural network. In some cases, the one or more machine learning models or networks may be a neural network that is trained end to end to generate the proactive plan for the sliding window of time.

[0061] At 306, the orchestration system may receive updated data associated with the facility upon a predetermined interval elapsing and, at 308, update, based at least in part on the predetermined interval, the sliding window of time. For example, the orchestration system may receive updated data every’ half hour, hour, number of hours, day, week and / or the like. The orchestration system may then advance the sliding window of time based on the elapsed interval of time and then re-generate or generate a new' proactive plan based on the new sliding window' of time and the updated data. In this manner, the proactive plan may be continuously updated based on the facilities operations and real-life events occurring (e.g., a traffic jam or accident delaying an incoming shipment) during the elapsed interval.

[0062] FIG. 4 is a flow' diagram illustrating an example process 400 associated wdth orchestration system operating at a facility according to some implementations. As discussed above, the orchestration system may determine a proactive plan for the operations of the facility that forecast the expected operations and / or throughput of thefacility (e.g., product quantity and / or type produced, shipped, manufactured, and the like) over a given period of time, such as a sliding window of time (e.g., one day in advance, a number of days in advance, one week in advance, a number of weeks in advance, a month in advance, a quarter in advance, a year in advance, and the like). The orchestration system may also generate reactive plans that update the day to day operations of the facility such as in reaction to real-time and real world occurrences, situations, and operations.

[0063] At 402, the orchestration system may receive data associated with a facility for a period of time. As discussed above the data may include consumption data associated with a downstream venue, delivery / shipping data associated with incoming and outgoing orders, order data, equipment / asset data, item data, inventory data, and / or the like. In some cases, the data may be received from downstream or upstream venues, vehicles enroute to or from the facility, equipment or personnel operating at the facility, sensors systems associated with the facility, third-party7systems and / or the like.

[0064] At 404, the orchestration system may determine, based at least in part on the data associated with the faculty, a proactive plan over a sliding window of time for the facility, the facility including one or more equipment or machinery7. For example, the orchestration system may be configured to input the data into the one or more machine learning models or networks and receive the proactive plan as an output of the one or more machine learning models or networks based at least in part on the sliding window of time. In some cases, the proactive plan may be generated at various intervals and / or in substantially7real-time as the data is received. For instance, the data may be updated every hour, number of hours, day, w eek, and / or the like and the orchestration system may generate, such as via the one or more machine learning models and / or networks, a proactive plan for one or more of sliding window of time moving forward from a cunent period of time or future selected starting point (e.g., a proactive plan for a subsequent week starting on the next Monday at 8am or the like).

[0065] In some examples, the one or more machine learning models or networks may be trained on historical data including historical data associated with downstream venues, delivery / shipping data associated with incoming and outgoing orders, orders, equipment / asset performance and operations, item data, inventory, and / or the like. In some specific examples, the one or more machine learning models or networks maysegment, classify, and / or arrange the incoming data to generate the proactive plan as part of one or more heads of a neural network. In some cases, the one or more machinelearning models or networks may be a neural network that is trained end to end to generate the proactive plan for the sliding window of time.

[0066] At 406, the orchestration system may receive additional data associated with the facility. The additional data may include additional data associated with downstream venues, deliveries / shipments, orders, equipment / assets, items, inventory7, and / or the like. In some cases, the additional data may be associated with operations, such as one or more operational lines and the like, that are captured in substantially real-time or concurrently while the inventory' is processed (e.g., either as incoming inventory’ or outgoing shipments).

[0067] At 408, the orchestration system may generate, based at least in part on the proactive plan and the additional data, a reactive plan for operations of the facility, as discussed herein. For example, the reactive plan may be used to control and implement day7to day operations of the facility based on current conditions and the proactive plan. In this manner, the reactive plan may allow7the facility7to adapt to changing circumstances in substantially real-time or concurrently as the circumstances change. Accordingly, a facility implementing the orchestration system provides for technical improvements that provide tangible or concrete improvements over conventional facilities by improving controls and operations of autonomous systems, semi- autonomous systems, robotic systems, as well as operations associated with the facility7.

[0068] In some cases, the reactive plan may include configuration data (e.g., equipment arrangements, equipment set-ups, equipment positioning, and the like), control signals (e.g., signals to cause the equipment to perform tasks either autonomously or semi-autonomously and the like), setting data, parameter data, as w ell as other plan data (e.g., operational assignments, personnel assignments, equipment assignments, throughput assignments, product processing assignments, and / or the like). In some cases, the configuration data, the control signals, the setting data, and / or the parameter data may be used to control autonomous and semi-autonomous robotic systems throughout the facility without the need for human input.

[0069] In some implementations, the orchestration system may be configured to generate (and / or update) the reactive plan on behalf of the facility' and automatically (e.g., without human input and / or authorization) and in response to outputs from one or more machine learning models or netw orks.

[0070] At 410, the orchestration system may cause the reactive plan to become an active plan associated with the operations of the facility7and, at 412, the orchestrationsystem may cause, based at least in part on the reactive plan, one or more faculty equipment or machinery to perform an operation or change an operation associated with an asset. For example, the reactive plan may replace a prior reactive plan and / or the proactive plan to control the operations of the facility.

[0071] In some cases, the orchestration system may be configured to generate reactive plans and / or proactive plans on a predetermined interval or in substantially real-time as the additional data is received. In these cases, the process 400 may return to 406 to regenerate the reactive plan and / or optionally 402 to regenerate the proactive plan based on the additional data received in the intervening time since the prior iteration of each instance of the individual plan(s).

[0072] FIG. 5 is a block diagram 500 of an example facility equipment / system 502 configured to operate in conjunction with an orchestration system implementing proactive and reactive plans, as described herein. In this embodiment, the equipment / system 502 is an autonomous or semi-autonomous system that may include a vehicle, machinery, computing device, or other operational component of a facility. The equipment / system 502 may include one or more computing components 504, one or more sensor systems 506, one or more communication connections 508, and one or more operations systems 510 (e.g., control systems).

[0073] The computing components 504 may include one or more processors 512 (or processing resources) and computer readable media 514 communicatively coupled with the one or more processors 512. In the illustrated example, the computer readable media 514 of the computing components 504 stores planning components 516, perception components 518, prediction components 520, as well as other components 522 associated with autonomous operations. The computer readable media 514 may also store sensor data 524 (e.g., data captured by the sensor systems 506) and reference data 526 (e.g., object data, environment data, and the like). In some implementations, it should be understood that the systems as well as data stored on the computer readable media 514 may additionally, or alternatively, be accessible to the equipment / system 502 (e.g.. stored on, or otherwise accessible by, other computer readable media remote from the equipment / system 502).

[0074] In some implementations, the prediction components 520 may be configured to estimate current and / or predict future characteristics or states of objects (e.g., vehicles, pedestrians, animals, etc.), such as pose, speed, trajectory, velocity, yaw, yaw rate, roll, roll rate, pitch, pitch rate, position, acceleration, or other characteristics.

[0075] The prediction system 520 may be configured to determine a predicted behavior and / or state corresponding to an identified object (e.g., such as an inventory item, component, assembly, storage bin or container, and / or the like). For example, the prediction system 520 may be configured to predict a velocity, position, change in trajectory, or otherwise predict the decisions and movement of the identified objects. For example, the prediction system 520 may include one or more machine learned models that may, based on inputs such as object type or classification and object characteristics, output predicted characteristics of the object at one or more future points in time. In some cases, the predicted behaviors and / or states may be assigned a confidence value, such that the behaviors and / or states may be sorted, ranked, and / or the like.

[0076] The planning system 516 may be configured to determine a movement (e.g., and / or route for a vehicle) for the equipment / system 502 to perform (such as a pick movement of an autonomous robotic arm). For example, the planning system 516 may determine various movements and various levels of detail based at least in part on the identified objects, the predicted behaviors, states and / or characteristics of the object at future times, the confidence value associated with each predicted behavior or state, and a set of safety requirements corresponding to the current scenario (e.g., combination of objects detected and / or environmental conditions). In some cases, a movement may be planned as a sequence of waypoints between two poses (e.g.. between two six degree of freedom poses or the like).

[0077] The equipment / system 502 can also include one or more communication connection(s) 508 that enable communication between the equipment / system 502 and one or more other local or remote computing device(s), such as the orchestration system 530 to receive the proactive and / or reactive plans, configurations, settings, control signals, and / or the like. The communication connection(s) 508 may allow the equipment / system 502 to communicate with other nearby computing device(s) (e.g., other nearby equipment / system, personnel, and the like).

[0078] In at least one example, the sensor system(s) 506 can include lidar sensors, radar sensors, ultrasonic transducers, sonar sensors, location sensors (e.g., GPS, compass, etc.), inertial sensors (e.g., inertial measurement units (IMUs), accelerometers, magnetometers, gyroscopes, etc.), cameras (e.g., RGB, IR, intensity, depth, time of flight, etc.), microphones, wheel encoders, environment sensors (e.g., temperature sensors, humidity sensors, light sensors, pressure sensors, etc.), and one ormore time of flight (ToF) sensors, etc. The sensor system(s) 506 can include multiple instances of each of these or other types of sensors. For instance, the lidar sensors may include individual lidar sensors located at the comers, front, back, sides, and / or top of the equipment / system 502. As another example, the camera sensors can include multiple cameras disposed at various locations about the exterior and / or interior of the equipment / system 502. The sensor system(s) 506 may provide input to the computing components 504. Additionally, or alternatively, the sensor system(s) 506 can send sensor data, via the one or more networks 528, to the one or more computing device(s) 530 at a particular frequency, after a lapse of a predetermined period of time, in near real-time, etc.

[0079] In at least one example, the equipment / system 502 can include one or more operational systems 510 (such as one or more drive motor or the like). In at least one example, the components discussed herein can process sensor data 524, as described above, and may send their respective outputs, over the one or more network(s) 528. In at least one example, the components discussed herein may send their respective outputs to an orchestration system 530 at a particular frequency, after a lapse of a predetermined period of time, in near real-time, and the like.

[0080] In some examples, the equipment / system 502 can send sensor data to the orchestration systems 530 via the network(s) 528 and receive the control signals, configurations, settings, and / or the reactive and / or proactive pans.

[0081] The computing system(s) 530 may include processor(s) 532 and computer readable media 534 storing a forecasting system 536 and / or an adjustment system 538 usable to generate the proactive plans and / or the reactive plans, as discussed herein, as well as other components, sensor data 540 and training data 542 usable by the one or more machine learning models or networks associated with the orchestration system 530.

[0082] The processor(s) 512 ofthe equipment / system 502 and the processor(s) 532 of the orchestration system 530 may be any suitable processor capable of executing instructions to process data and perform operations as described herein. By way of example and not limitation, the processor(s) 512 and 532 can comprise one or more Central Processing Units (CPUs), Graphics Processing Units (GPUs), or any other device or portion of a device that processes electronic data to transform that electronic data into other electronic data that can be stored in registers and / or computer readable media. In some examples, integrated circuits (e.g., ASICs, etc.), gate arrays (e.g.,FPGAs, etc.), and other hardware devices can also be considered processors in so far as they are configured to implement encoded instructions.

[0083] Computer readable media 514 and 534 are examples of non-transitory computer-readable media. The computer readable media 514 and 534 can store an operating system and one or more software applications, instructions, programs, and / or data to implement the methods described herein and the functions attributed to the various systems. In various implementations, the computer readable media can be implemented using any suitable computer readable media technology, such as static random-access memory (SRAM), synchronous dynamic RAM (SDRAM), nonvolatile / Flash-type memory', or any other type of computer readable media capable of storing information. The architectures, systems, and individual elements described herein can include many other logical, programmatic, and physical components, of which those shown in the accompanying figures are merely examples that are related to the discussion herein.

[0084] As can be understood, the components discussed herein are described as divided for illustrative purposes. However, the operations performed by the various components can be combined or performed in any other component.

[0085] FIG. 6 is an example orchestration system 600 that may implement the techniques described herein according to some implementations. The system 600 may include one or more communication interface(s) 602 (also referred to as communication devices and / or modems), one or more processor(s) 604, and one or more computer readable media 606.

[0086] The system 600 can include one or more communication interfaces(s) 602 that enable communication between the system 600 and one or more other local or remote computing device(s) or remote services, such as a sensor system of FIG. 1. For instance, the communication interface(s) 602 can facilitate communication with other central processing systems, a sensor system, or other facility systems. The communications interfaces(s) 402 may enable Wi-Fi-based communication such as via frequencies defined by the IEEE 802. 11 standards, short range wireless frequencies such as Bluetooth, cellular communication (e.g., 2G, 3G, 4G, 4G LTE, 5G, etc.), satellite communication, dedicated short-range communications (DSRC), or any suitable wired or wireless communications protocol that enables the respective computing device to interface with the other computing device(s).L.©© & H © ©s

[0087] The system 600 may include one or more processors 604 and one or more computer-readable media 606. Each of the processors 604 may itself comprise one or more processors or processing cores. The computer-readable media 606 is illustrated as including memory / storage. The computer-readable media 606 may include volatile media (such as random access memory (RAM)) and / or nonvolatile media (such as read only memory (ROM), Flash memory, optical disks, magnetic disks, and so forth). The computer-readable media 606 may include fixed media (e.g., RAM, ROM, a fixed hard drive, and so on) as well as removable media (e g., Flash memory, a removable hard drive, an optical disc, and so forth). The computer-readable media 606 may be configured in a variety of other ways as further described below.

[0001] Several modules such as instructions, data stores, and so forth may be stored within the computer-readable media 606 and configured to execute on the processors 604. For example, as illustrated, the computer-readable media 606 stores asset data collection and processing instructions 608, proactive plan generation instructions 610, reactive plan generation instructions 612. time period selection instructions 614, alert / notification instructions 616, item ordering instruction 618, scheduling instructions 620, as well as other instructions 622, such as an operating system. The computer-readable media 606 may also be configured to store data, such as third-party data 624, machine learning models 626. facility data 828 (e.g., data associated with a downstream venues, data associated with deliveries and / or shipments, order data, equipment / asset data, item data, inventory data, and / or the like), training data 630 for the machine learning models 626, as well as other data.

[0088] The asset data collection and processing instructions 608 may be associated with causing sensor systems to capture facility data 628 and / or process facility data 628 received from other sources (e.g., third-party sources). For example, the asset data collection and processing instructions 608 may sort, rank, order, aggregate, combine, or otherwise process the facility data 628 for use by other instructions, such as the proactive plan generation instructions 610 and / or the reactive plan generation instructions 612.

[0089] The proactive plan generation instructions 610 may be configured to generate one or more proactive plans for the facility as discussed herein.

[0090] The reactive plan generation instructions 612 may be configured to generate one or more reactive plans for the facility as discussed herein.

[0091] The time period selection instructions 614 may be configured to select or assist a user in selecting a period of time, such as intervals for the generation of proactive plans or reactive plans, a sliding window of time for the proactive plans as discussed herein, and / or the like.

[0092] The alert / notification instructions 616 maybe configure to provide alerts and notification associated with the proactive plans and / or the reactive plans to various systems, personnel, individuals, and / or the equipment / machinery. For example, the alert / notification instructions 616 may send control signals to the autonomous equipment to cause the autonomous equipment to perform various tasks and / or operations.

[0093] The item ordering instructions 618 may be configured to order inventory on behalf of the facility based at least in part on one or more proactive plans and / or reactive plans.

[0094] The scheduling instructions 620 may be configured to schedule and update a schedule associated with facility operations as discussed herein based at least in part on one or more proactive plans and / or reactive plans.

[0095] Although the discussion above sets forth example implementations of the described techniques, other architectures may be used to implement the described functionality and are intended to be w ithin the scope of this disclosure. Furthermore, although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as exemplary forms of implementing the claims.EXAMPLE CLAUSES

[0001] A.

[0002] While the example clauses described above are described with respect to one particular implementation, it should be understood that, in the context of this document, the content of the example clauses can also be implemented via a method, device, system, a computer-readable medium, and / or another implementation. Additionally, any of examples A-0 may be implemented alone or in combination with any other one or more of the examples A-O.L.©© & H © ©sCONCLUSION

[0003] While one or more examples of the techniques described herein have been described, various alterations, additions, permutations and equivalents thereof are included within the scope of the techniques described herein. As can be understood, the components discussed herein are described as divided for illustrative purposes. However, the operations performed by the various components can be combined or performed in any other component. It should also be understood that components or steps discussed with respect to one example or implementation may be used in conjunction with components or steps of other examples.|0004| In the description of examples, reference is made to the accompanying drawings that form a part hereof, which show by way of illustration specific examples of the claimed subject matter. It is to be understood that other examples can be used and that changes or alterations, such as structural changes, can be made. Such examples, changes or alterations are not necessarily departures from the scope with respect to the intended claimed subject matter. While the steps herein may be presented in a certain order, in some cases the ordering may be changed so that certain inputs are provided at different times or in a different order without changing the function of the systems and methods described. The disclosed procedures could also be executed in different orders. Additionally, various computations that are herein need not be performed in the order disclosed, and other examples using alternative orderings of the computations could be readily implemented. In addition to being reordered, the computations could also be decomposed into sub-computations with the same results.L.©© &. H © ©s

Claims

CLAIMS1. The method comprising: receiving first sensor data associated with a first physical environment; determining, based at least in part on the first sensor data, a proactive plan associated with equipment of the first physical environment; receiving second sensor data associated with a first physical environment; performing operations associated with the proactive plan; determining, based at least in part on the second sensor data, a reactive plan associated with equipment of the first physical environment; and cease performing operations associated with the proactive plan and commence performing operations associated with the reactive plan.

2. The method of claim 1 , wherein determining the proactive plan is based at least in part on a future sliding window of time, the future sliding window of time being at least a period of time in the future from a current time.

3. The method of claims 1 or 2, wherein the proactive plan is determined based at least in part on personnel associated with the first physical environment.

4. The method of any of claims 1-3, further comprising generating, based at least in part on the proactive plan and the reactive plan, a second proactive plan for a second future sliding window of time, the future sliding window of time being at least a second period of time in the future from the current time.

5. The method of any of claims 1-3, wherein: the first sensor data is equipment and asset data; and determining the proactive plan associated with equipment of the first physical environment is based at least in part on consumption data, delivery data, order data, item data, and inventory data.

6. The method of any of claims 8-11, wherein the reactive plan includes configuration data, control signals, setting data, parameter data, and plan data.L.©© &. H ciy ©S7. The method of any of claims 1-6. wherein generating the first proactive plan includes inputting the first sensor data into a second machine learning model and receiving as an output of the first machine learning model the first proactive plan.

8. The method of any of claims 1-7, wherein the proactive plan and the reactive plan include control signals for controlling the operations of one or more autonomous equipment associated with the first physical environment.

9. The method of any of claims 1-8, wherein generating the second proactive plan includes inputting the first proactive plan and the reactive plan into a second machine learning model and receiving as an output of the second machine learning model the reactive plan.

10. A computer program product comprising coded instructions that, when run on a computer, implement a method as claimed in any of claims 1-9.

11. A system comprising: a forecasting system to generate a first proactive plan based at least in part on first sensor data associated with a first physical environment, the proactive plan to control operations of equipment in a first physical environment for a first period of time; an adjustment system to generate a reactive plan based at least in part on the proactive plan and second sensor data associated with the first physical environment, the second sensor data captured by one or more sensor devices in communication with the adjustment system during the first period of time; and one or more autonomous equipment within the physical environment to perform operations based at least in part on one or more of the proactive plan or the reactive plan.

12. The system of claim 11, wherein the forecasting system is configured to generate a second proactive plan based at least in part on the first proactive plan and the reactive plan, the second proactive plan for a second period of time, the second period of time at least partially non-overlapping with the first period of time.

13. The system of claims 11 or 12, wherein the first sensor data includes consumption data received from a system associated with a first third-party, delivery data received from a system associated with a second third-party, order data received from a system associated with a third third-party, item data, and inventory data, the first third-party7different than the second third-party7and the third third-party7different than the second third-party and the first third-party.

14. The system of any of claims 11 -13, wherein the adjustment system may set the reactive plan as an active plan and generate various iterations of the reactive plan at various intervals during the first period of time, at each interval the adjustment system generate a corresponding reactive plan based at least in part on the active plan for the prior iteration and additional sensor data received during the prior iteration and, upon generation, set the corresponding reactive plan to the active plan.

15. The system of any of claims 11-14, wherein the autonomous equipment includes at least one of a conveyor blet system, robotic arm system, or forklift.L.©© & H © ©s

Images