Methods and apparatus for multi-agent path finding within diagnostic laboratory systems

Abstract

In some embodiments, a method of path finding for sample containers in diagnostic laboratory systems includes receiving a batch of sample containers in a diagnostic laboratory system having diagnostic laboratory equipment; obtaining a grid for the diagnostic laboratory system, the grid having cells that include the diagnostic laboratory equipment and one or more tracks connecting the diagnostic laboratory equipment; assigning an agent to each of the sample carriers; determining, for each agent assigned to a sample carrier holding a sample container, a goal including an earliest arrival time constraint, a deadline constraint, and a goal processing time constraint; and employing a multi-agent path finding (MAPF) trained neural network to determine an action for each agent, the neural network trained with arrival, deadline, and precedence constraints and with non-instantaneous goal processing times. Numerous other aspects are provided.

Description

METHODS AND APPARATUS FOR MULTI-AGENT PATH FINDING WITHIN DIAGNOSTIC LABORATORY SYSTEMSCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims benefit under 35 USC § 119(e) of US Provisional Application No. 63 / 518,565, filed August 10, 2023. The entire contents of the abovereferenced patent application(s) are hereby expressly incorporated herein by reference.FIELD

[0002] The present disclosure relates to diagnostic laboratories and more particularly to methods and apparatus for multi-agent path finding within diagnostic laboratory systems.BACKGROUND

[0003] Diagnostic laboratory systems conduct assays or tests to identify analytes or other constituents in biological samples such as blood serum, blood plasma, urine, interstitial liquid, cerebrospinal liquids, and the like. The biological samples are collected in sample containers, such as test tubes, by medical technicians and are then brought to a diagnostic laboratory system for testing. Each biological sample may have a plurality of test requirements, wherein each test requirement requires performing a different test by the diagnostic laboratory system.

[0004] A diagnostic laboratory system may include a plurality of instruments that are each configured to perform one or more tests. When a biological sample having a single test requirement is received in the diagnostic laboratory system, the biological sample is transferred to an instrument that is configured to perform the test. Likewise, when a biological sample having a plurality of test requirements is received in the diagnostic laboratory system, the biological sample is directed to one or more of the instruments that are collectively configured to perform the plurality of tests.

[0005] A scheduling program creates a schedule for directing sample containers to specific instruments to conduct tests on samples contained therein. However, determining the optimal paths and timing for transporting and processing multiple sample containers, while avoiding collisions and unnecessary delays, is challenging. Accordingly, improved methods and apparatus for sample container route planning within diagnostic laboratory systems are needed.SUMMARY

[0006] In some embodiments, a method of path finding for sample containers in diagnostic laboratory systems is provided that includes receiving a batch of sample containers in a diagnostic laboratory system having diagnostic laboratory equipment, one or more tracks connecting the diagnostic laboratory equipment, and a plurality of sample carriers configured to transport the sample containers within the diagnostic laboratory system, each of the sample containers containing a sample to be processed; obtaining a grid for the diagnostic laboratory system, the grid having cells that include the diagnostic laboratory equipment and the one or more tracks connecting the diagnostic laboratory equipment; assigning an agent to each of the sample carriers; determining, for each agent assigned to a sample carrier holding a sample container, a goal including an earliest arrival time constraint, a deadline constraint, and a goal processing time constraint; and employing a multi-agent path finding (MAPF) trained neural network to determine an action for each agent, the neural network trained with arrival, deadline, and precedence constraints and with non-instantaneous goal processing times.

[0007] In some embodiments, a diagnostic laboratory system is provided that includes diagnostic laboratory equipment; one or more tracks connecting the diagnostic laboratory equipment; a processor; and a memory coupled to the processor. The memory includes computer executable instructions that, when executed by the processor, cause the processor to obtain a grid for the diagnostic laboratory system, the grid having cells that include the diagnostic laboratory equipment and the one or more tracks connecting the diagnostic laboratory equipment; assign an agent to each sample carrier within the diagnostic laboratory system; determine for each agent assigned to a sample carrier holding a sample container, a goal including an earliest arrival time constraint, a deadline constraint, and a goal processing time constraint; and employ a MAPF trained neural network to determine an action for each agent, the neural network trained with arrival, deadline, and precedence constraints and with non-instantaneous goal processing times.

[0008] In some embodiments, a method of path finding for sample containers in diagnostic laboratory systems is provided that includes creating a grid for a diagnostic laboratory system, the grid having cells that include diagnostic laboratory equipment and one or more tracks connecting the diagnostic laboratory equipment; creating a plurality of agents, each agent representative of a sample carrier within the diagnosticlaboratory system; determining, for each agent assigned to a sample carrier holding a sample container, a goal including an earliest arrival time constraint, a deadline constraint, and a goal processing time constraint; creating a neural network having an earliest arrival time constraint, deadline constraint, and goal processing time constraint for an agent as inputs and actions or probabilities of actions for the agent as outputs; training the neural network with arrival, deadline, and precedence constraints and with non-instantaneous goal processing times; and creating a path finding program that employs the trained neural network to generate actions for each agent within the diagnostic laboratory system.

[0009] Still other aspects, features, and advantages of this disclosure may be readily apparent from the following description and illustration of example embodiments, including the best mode contemplated for carrying out the disclosure. This disclosure may also be capable of other and different embodiments, and its several details may be modified in various respects, all without departing from the scope of the disclosure.BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The drawings described below are provided for illustrative purposes and are not necessarily drawn to scale. Accordingly, the drawings and descriptions are to be regarded as illustrative in nature, and not as restrictive. The drawings are not intended to limit the scope of the disclosure in any way.

[0011] FIG. 1 illustrates an embodiment of a diagnostic laboratory system in accordance with embodiments provided herein.

[0012] FIG. 2A illustrates an example embodiment of a computer of FIG. 1 in accordance with some embodiments provided herein.

[0013] FIG. 2B illustrates an example flow diagram depicting the flow of information from a scheduler program to a path finding program, from the path finding program to and from a neural network, and from the path finding program to a sample carrier controller in accordance with some embodiments provided herein.

[0014] FIG. 3A illustrates an example embodiment of another diagnostic laboratory system in accordance with embodiments provided herein.

[0015] FIG. 3B illustrates a portion of the diagnostic laboratory system of FIG. 3A having a first agent and a second agent in accordance with embodiments provided herein.

[0016] FIG. 3C illustrates the portion of the diagnostic laboratory system of FIG. 3B with vertices connected by edges shown for cells to which sample carriers may travel in accordance with embodiments provided herein.

[0017] FIG. 3D illustrates the portion of the diagnostic laboratory of FIG. 3B illustrating vertices and edges without cells for clarity.

[0018] FIG. 3E illustrates a plurality of example masks representing the states of cells within the diagnostic laboratory system of FIG. 3A in accordance with embodiments provided herein.

[0019] FIG. 4 is a flowchart of an example method of path finding for sample containers in diagnostic laboratory systems in accordance with some embodiments provided herein.

[0020] FIG. 5 is a flowchart of another method of path finding for sample containers in diagnostic laboratory systems in accordance with some embodiments provided herein.DETAILED DESCRIPTION

[0021] Independent of the grammatical term usage, individuals with male, female, or other gender identities are included within the term.

[0022] Scheduling and / or routing sample containers to specific instruments in a diagnostic laboratory system is complex because diagnostic laboratory systems have many different types of components and may receive many different types of samples. For example, diagnostic laboratory systems may include numerous instruments each of which may be configured with different testing menus, testing procedures, and testing durations. Further, samples to be analyzed within a diagnostic laboratory system may have priorities such as routine or stat samples that require immediate processing.

[0023] Embodiments provided herein employ multi-agent path finding techniques to determine actions for transporting and processing sample containers within a diagnostic laboratory system. Path finding plays a critical role in many automation domains. Many automation settings involve multiple interacting components that require coordinated actions. The Multi-Agent Path Finding Problem (MAPF) formulates important aspects of the target domains such as collision avoidance and travel time minimization. However, a key assumption present among current MAPF is that task processing is instantaneous. This discrepancy is amplified in domains such lab or factory automation where tasks require non-trivial processing as a part of theautomation pipeline. A machine may take variable times to process different tasks. Without time slack corresponding to machine availability, agent congestion may arise as such approaches fail to capture the precedence constraints and queuing effects induced by non-instantaneous task processing times.

[0024] In accordance with one or more embodiments provided herein, the above MAPF problem is addressed with execution schedules and precedence constraints. For example, a schedule may be enforced by automation pipeline capabilities such as machine operating characteristics. In some embodiments, a MAPF with time constraints (MAPF-TC) approach is provided that may be modelled as a sequential decision-making problem.

[0025] In one or more embodiments, a multi-agent reinforcement learning (MARL) model is employed to solve the multi-agent path finding with time-constraints problem. In some embodiments, a time-constrained version of the MAPF problem is provided that introduces arrival, deadline, and precedence constraints with non-instantaneous goal processing times. Further, in some embodiments, a novel reward and observation space for modeling the temporal aspects and precedence constraints is provided.

[0026] These and other embodiments provided herein are described below with reference to FIGS. 1-5.

[0027] Reference is made to FIG. 1 , which illustrates an embodiment of a diagnostic laboratory system 100 in accordance with embodiments provided herein. Diagnostic laboratory system 100 may include a plurality of instruments 102 (e.g., diagnostic laboratory equipment) configured to process samples stored within sample containers 104 (a few labelled) and to conduct tests (e.g., assays or other tests) on the samples. Performing the tests may include performing one or more operations on the samples. Each operation may include one or more measurements, for example. One or more of the instruments 102 may include a plurality of different modules configured to perform the operations.

[0028] The samples may be various biological specimens collected from individuals, such as patients being evaluated by medical professionals. The samples may be collected in sample containers 104 and delivered to diagnostic laboratory system 100 and thereafter transported via a track 108 throughout diagnostic laboratory system 100, such as to instruments 102. Sample containers 104 may be transported by sample carriers 110 (a few labelled), for example. In the embodiment of FIG. 1 , diagnostic laboratory system 100 has three instruments 102, which include a samplehandler 114, a first analyzer 116, and a second analyzer 118. Diagnostic laboratory system 100 may include fewer or more instruments than shown in FIG. 1 .

[0029] In some embodiments, track 108 may extend proximate or around instruments 102 as shown in FIG. 1 . Portions or modules of instruments 102 may have devices, such as robots (not shown in FIG. 1 ), that transfer sample containers 104 to and from sample carriers 110. Track 108 may include a plurality of segments 120 (a few labelled) that may be interconnected. Sample carriers 110 may move as shown by dashed lines 126 in segments 120. In some embodiments, some of segments 120 may be integral with one or more of instruments 102.

[0030] Diagnostic laboratory systems, such as the laboratory system 100, may have many instruments and may have tracks linked to other laboratory systems. Such laboratory systems, including the diagnostic laboratory system 100, may simultaneously move and process a plurality of sample carriers 110 and their respective sample containers 104. In some embodiments, diagnostic laboratory system 100 may move and process hundreds or thousands of sample carriers 110 and their respective sample containers 104 simultaneously.

[0031] Diagnostic laboratory system 100 may include or be coupled to a computer 130 configured to execute one or more programs and control operation of diagnostic laboratory system 100. Computer 130 may be configured to communicate with instruments 102 and other components of diagnostic laboratory system 100, such as components in a transport system (e.g., track 108 and components that control its operation). The transport system may include some or all components configured to transport samples throughout diagnostic laboratory system 100 (e.g., motors, sensors, power supplies, etc.). Computer 130 may include a processor 132 configured to execute programs including programs other than those described herein. The programs may be implemented in computer code, computer executable instructions, or the like.

[0032] Computer 130 may include or have access to memory 134 that may store one or more programs 136 and / or data. Memory 134 may be any suitable type of memory, such as, but not limited to, one or more of a volatile memory and / or a nonvolatile memory. In one or more embodiments, memory 134 may be a non-transitory memory (e.g., a hard drive, a solid-state drive, a flash drive, another non-transitory computer-readable medium, etc.). Programs 136 may be computer code and / orinstructions executable on or by processor 132. Such programs may control all or a portion of operation of diagnostic laboratory system 100 as described below.

[0033] Computer 130 may be coupled to a workstation 138 that is configured to enable users to interface with diagnostic laboratory system 100. Workstation 138 may include a display 140, a keyboard 142, and other peripherals. Programs within memory 134 may cause display 140 to display information regarding processing of samples within diagnostic laboratory system 100.

[0034] When a sample is received in diagnostic laboratory system 100 for testing, the sample undergoes a complex set of sequential operations or processes in a specific workflow that is defined by specific tests. Each type of test may have a unique sequence of operations. The workflow sequence may start with a sample handling or a sample container handling operation followed by operations of sample and / or reagent aspiration and dispensing into a cuvette. The mixture in the cuvette may undergo other operations required by the test. The workflow sequence may conclude with measurement operations, such as photometric measurements, to determine chemical properties of the sample. Each of the operations in the workflow sequences may be performed using one or more of the instruments 102.

[0035] FIG. 2A illustrates an example embodiment of computer 130 of FIG. 1 in accordance with some embodiments. In the embodiment of FIG. 2A, memory 134 of computer 130 includes a scheduler program 202 coupled to a path finding program 204. As described below, path finding program 204 may receive schedule information from scheduler program 202 and employ a multi-agent path finding (MAPF) trained neural network 206 to determine actions for sample carriers 110 within diagnostic laboratory system 100 (such as carrier movements for each sample carrier 110). These actions may be communicated from path finding program 204 to a sample carrier controller 208 for execution. For example, sample carrier controller 208 may control operation of motors, sensors, power supplies, etc., used to transport sample carriers 110 on track 108. While shown within memory 134 of computer 130, it will be understood that scheduler program 202 and / or sample carrier controller 208 may be located within a different memory and / or executed by a different processor.

[0036] FIG. 2B illustrates an example flow diagram depicting the flow of information from scheduler program 202 to path finding program 204, from path finding program 204 to and from neural network 206 and from path finding program 204 to carrier controller 208 in accordance with some embodiments.

[0037] Scheduler program 202 may be configured to determine a schedule or workflow for processing all sample containers 104 within diagnostic laboratory system 100 (e.g., such as where sample carriers 110 should travel to pick up and transport sample containers 104 for processing). For example, scheduler program 202 may determine that sample containers 104 within sample handler 114 should be retrieved and delivered to one or more of first analyzer 116 and second analyzer 118 for processing. In some embodiments, scheduler program 202 may generate a workflow that indicates the diagnostic laboratory equipment and the order thereof to be visited by each sample container (e.g., based on information from scanned or imaged indicia on a label attached to the sample container regarding one or more analyses to be performed on a sample in the sample container). In one or more embodiments, processor 132 (or another processor) may simulate the workflow of each sample container to estimate workflow completion time information for each.

[0038] Schedule information may be provided to path finding program 204 which in turn may employ MAPF-trained neural network 206 to determine actions for each sample carrier 110 within diagnostic laboratory system 100 so that the schedule determined by scheduler program 202 is carried out efficiently (e.g., collision free delivery of sample containers for processing within any required deadlines). These actions may be fed to sample carrier controller 208 for execution. Accordingly, scheduler program 202, path finding program 204, MAPF-trained neural network 206, and sample carrier controller 208 may be deployed for use within diagnostic laboratory systems to facilitate sample processing.

[0039] In one or more embodiments, the MAPF problem with time constraints may be defined by an undirected graph G=(V,E), a set of m agents {ai ... am}, and a schedule S. The set of vertices V corresponds to locations on the graph and the set of edges E represents motion constraints for each vertex. At each step, an agent can either move or wait at its current position. Each agent is assigned a sequence of goal locations {l'(1), l'(2)... I'(n)} according to the schedule s. The schedule s represents timewindows that include Earliest Arrival Time (At'®), Deadline (Dt'®), and Goal Processing Time (Pt'®) for agent / and goal j. The schedule S and the earliest arrival time, deadline, and goal processing time constraints may be provided by scheduler program 202, for example.

[0040] At'® constrains when an agent can start its goal processing and where an agent may arrive prior to starting, but the goal will not be processed until the scheduledtime. Dti(i)constrains the latest an agent is allowed to reach its goal for processing. Pt'® represents (e.g., and constrains) non-instantaneous goal processing where an agent is not allowed to move. The schedule also enforces precedence constraints for goal locations shared among multiple agents (e.g., a factory machine such as a sample analyzer). While soft violation of the deadlines with a penalty may be allowed, the precedence constraints remain enforced, effectively inducing an agent queue for shared goal locations. In some embodiments, agents are always present on the graph from the beginning and do not disappear after completing their goals. The persistence of all agents on a space-constrained track is a key challenge.

[0041] In one or more embodiments, the agent interaction with the environment may be modeled as a partially observable Markov decision process (POMDP) (S, O, A, P, R, Y) where S is the set of environment states, O is the set of partial observations, A is the set of actions, P is a function representing the transition probabilities, R is the reward function, and Y is the discount factor. In the embodiments described herein, the environments are restricted to 2D grids where each agent is limited to a local field of view observation. In other embodiments, a 3D grid may be employed. A homogenous policy may be learned that can be deployed to different numbers of agents and executed in a decentralized manner.

[0042] In some embodiments, the observation space may include two components. First, a local field of view may represent the state of neighboring vertices relative to an agent’s current location. Second, information about a next goal location may be encoded directly using vectors representing the spatial information and temporal information of the goal with three dimensions as described below.

[0043] FIG. 3A illustrates an example embodiment of another diagnostic laboratory system 300 in accordance with embodiments provided herein. With reference to FIG. 3A, diagnostic laboratory system 300 is represented as a grid 302 which defines a plurality of cells 304 (only a few labelled) within diagnostic laboratory system 300. As indicated in the key in FIG. 3A, cells 304 that have no shading represent sample carrier movement locations such as track sections, track connections, machine padding locations, or the like. Cells 304 with medium shading represent diagnostic laboratory equipment such as sample analyzers, sample handlers, or the like. Cells 304 with dark shading represent areas of diagnostic laboratory system 300 that are empty or unavailable. In the embodiment of FIG. 3A, carrier movements include movements in the north (N), south (S), east (E), and west (W) directions. In some embodiments, cells304 may be sized based on one or more of the sizes of sample carriers 110 (e.g., large enough to accommodate a sample carrier), the movement resolution of the track sections employed, or the like. For example, in some embodiments, each cell 304 may be sized to hold a single sample carrier 110. Other cell sizes may be used.

[0044] FIG. 3B illustrates a portion 300a of diagnostic laboratory system 300 of FIG. 3A having a first agent 308a and a second agent 308b in accordance with embodiments provided herein. A first local FOV 310a is shown for first agent 308a and a second local FOV 310b is shown for second agent 308b, each FOV including nearest neighbor cells 304. Other FOV sizes may be employed. First agent 308a is located on a track section near first diagnostic laboratory equipment 312a (e.g., a first sample analyzer). Second agent 308b is located within second diagnostic laboratory equipment 312b (e.g., a second sample analyzer).

[0045] FIG. 3C illustrates the portion 300a of diagnostic laboratory system 300 with vertices 314 (connected by edges 316) shown for cells 304 to which sample carriers may travel in accordance with embodiments provided herein. FIG. 3D illustrates the portion 300a illustrating vertices 314 and edges 316 without cells 304 for clarity.

[0046] As shown in FIG. 3D, local field of view 310a may represent the state of neighboring vertices 314 relative to the current location of first agent 308a within a radius R. For example, the local field of view (FOV) may become a square around the agent with length 2*R+1 .

[0047] In some embodiments, the state of each cell 304 within a field of view may be represented by a plurality of masks 318a-d as shown in FIG. 3E. While four masks are shown in FIG. 3E, it will be understood that fewer or more masks may be employed, as may larger masks (e.g., for larger fields of view). As an example, each cell state may be represented by one or more of (1 ) a binary mask of obstacles; (2) a binary mask of locations of all agents within the FOV; (3) a binary mask of goal locations of other agents within the FOV; and (4) a binary mask of the goal location of the agent within the cell, projected to the border if it lies outside the FOV. Further, in some embodiments, each cell state of a cell with an agent may also be represented by a normalized distance to the agent’s goal from each vertex in the FOV and / or an order of other agents in a queue if the other agents share the same goal location with the agent.

[0048] As stated, in some embodiments, information about a next goal location of an agent may be encoded directly using vectors representing the spatial informationand temporal information of the goal with three dimensions. For example, a spatial vector Gs may have a first vector component Gs1 representing the difference between the agent’s current and goal x-coordinates. A second vector component Gs2 may represent the difference between the agent’s current and goal y-coordinates. A third vector component Gs3 may represent the magnitude of a vector from the agent’s current location to its goal location, clipped at an absolute value (e.g., 60 in some embodiments).

[0049] A temporal vector Gt may represent the temporal information of the agent’s goal with 3 dimensions. For example, a first vector component Gt1 may represent the time left until the earliest arrival time At if the current time is less than At, otherwise 0.0. A second vector Gt2 may represent the time left until the deadline Dt if the current time is between At and Dt, otherwise 0.0. A third vector Gt3 may be set to 1 if the deadline Dt has passed indicating that the agent is late, otherwise 0.0. Other spatial and / or temporal vectors may be employed.

[0050] Returning to FIG. 2A, in some embodiments, path finding program 204 may include computer executable instructions (stored in memory 134) that, when executed by processor 132, cause processor 132 to obtain a grid for a diagnostic laboratory system (e.g., grid 302 for diagnostic laboratory system 300 in FIG. 3A). The grid defines a plurality of cells that includes cells within diagnostic laboratory equipment and one or more tracks connecting the diagnostic laboratory equipment (e.g., as shown in FIGS. 3A and 3B).

[0051] Path finding program 204 may also assign an agent to each sample carrier within the diagnostic laboratory system. For example, first agent 308a and second agent 308b are assigned to sample carriers 320a and 320b, respectively (FIG. 3B).

[0052] Path finding program 204 may further determine, for each agent assigned to a sample carrier holding a sample container, a goal including an earliest arrival time constraint, a deadline constraint, and a goal processing time constraint. As stated, such information may be provided from scheduler program 202 (FIG. 2A) in some embodiments.

[0053] Path finding program 204 may employ a MAPF-trained neural network (e.g., MAPF-trained neural network 206) to determine an action for each agent, the neural network trained with arrival, deadline, and precedence constraints with non- instantaneous goal processing times. In the example of FIG. 3A, actions for agents(e.g., sample carriers) may include waiting or moving north, south, east, or west. Other actions may be provided.

[0054] In some embodiments, sample carrier controller 208 may be employed to perform one or more of the determined actions (via computer program instructions within memory 134). For example, sample carrier controller 208 may cause the transfer of a sample container for processing within diagnostic laboratory system 300. Training of Neural Network

[0055] MAPF neural network 206 may be any suitable neural network such as a convolutional neural network, a transformer neural network, or the like. In some embodiments, each agent may receive a positive reward when it reaches its goal within the scheduled time-window. A negative penalty may be given for each time step an agent occupies its goal before the earliest arrival time. This approach aims to reduce congestion as agents waiting at their goals may block the passage of other agents. A negative penalty may be given for each time step to any agent that has not reached its goal past the deadline.

[0056] In reinforcement learning (RL) formulations of MAPF, penalties are commonly given when an agent collides eitherwith static obstacles in the environment or with other agents. Embodiments provided herein aim to reduce unnecessary subproblems for more efficient learning. One sub-problem is static obstacle avoidance. As such, invalid actions may be masked (e.g., ignored) where an agent would collide into an obstacle. Another sub-problem is collision-avoidance with other agents. The present inventors have observed that without a collision penalty, neural network policies tend to converge faster on goal-completing behavior while implicitly avoiding collisions. Explicit collision penalties may slow the training as the agents are more likely to collide with one another early in the learning process. Collision avoidance is still enforced through the environment. Additionally, in some embodiments, a penalty is not imposed on agent motion. Further, in one or more embodiments, an agent may need to wait or reroute its path without being late to its deadline.

[0057] Unlike some prior approaches, a global positive reward may be assigned to all agents when any agent completes a goal. Similarly, a global negative reward may be assigned to all agents when any agent is late past its deadline. The idea is to incentivize cooperation among agents, especially for agents with slack until their deadline and that can afford to take longer paths.

[0058] Table 1 provides an example of reward values for different events. Other reward values may be employed.Table 1

[0059] In some embodiments, the local field of view is processed by a convolutional neural network (CNN), such as a 3-layer CNN, with each layer followed by a max pooling operation except for the last layer which is followed by a global averaging to produce a field of view representation vector. Then the location and time-window vectors are fed through fully connected layers to produce a combined goal representation vector. Both the field of view and goal representation vectors are then passed through a recurrent module to incorporate information about past states as a mitigation for partial observability problems. The model is optimized with Proximal Policy Optimization (PPO) as described in Schulman, John, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. “Proximal Policy Optimization Algorithms.” arXiv. http: / / arxiv.org / abs / 1707.06347.

[0060] Selective scenarios may be employed to study the effects of different components on model performance. In some embodiments, time-windows may be parameterized to vary the resulting schedule slack.

[0061] Unlike the warehouse domain, automation plants are restricted by empty space, machine dimensions, and safety constraints such as power and ventilation outlets. To capture these various constraints, a parameterized floorplan generator maybe employed to generate a parameterized layout for a diagnostic laboratory system. Training on a variety of such layouts may generate a model capable of meeting almost 100% of the goals within their respective time-windows. In such layouts, agents may be able to learn traffic-like behaviors such as parallel lanes / corridors for opposite flow. While scaling to a larger number of agents within the same fixed layout is an important aspect of MAPF, it may not hold the same value with time constraints where the throughput of the entire system is limited by the (automation) schedule capacity. Adding more agents may simply result in the agents waiting idly and having sufficient time to navigate the track. As the number of agents increases in MAPF with time constraints, the bottleneck becomes the schedule rather than the path finding.

[0062] In some embodiments, more restrictive grid layouts (e.g., single corridor) may still be capable of completing nearly 100% of agent goals. In well-connected layouts where the track forms a continuous path, agents may learn to circle the track, maintaining a state of flow that does not block other agents until their deadlines are due (at which time they may deviate toward their goals). In the more restrictive layouts with forking dead ends, agents may learn to maneuver through the padded sections surrounding decision points (e.g., locations for sample carriers that are adjacent diagnostic laboratory equipment).

[0063] In some embodiments, a layout design may be created to identify the influence of different factors on performance as well as to establish the limits of the path finding program 204. For example, layouts may be varied across three design dimensions: redundancy, size, and padding. Redundancy may be varied by connecting tracks to remove the forking dead ends. Size may be tested by varying the number of goals / corridors. In some embodiments, this may include producing two variations: small and large with 3 and 6 goals / corridors, respectively. This factor may influence the number of agents (e.g., the number of agents may be set equal to the number of goals plus 2). Finally, different padding may be added around decision joints.

[0064] Schedule distributions may be varied across multiple design dimensions such as single-agent shortest distance (A* factor) which is the minimum distance traveled by an agent assuming there are no other agents in the goal and multi-agent estimated congestion wherein the average of all single agent shortest distances is added as a congestion estimate. These represent scheduling constraints that influence the earliest arrival time for each agent providing minimal necessaryconditions for the schedule to be feasible. Additionally, the size of the time-windows from earliest arrival time to deadline and task processing runtime before completion may be varied.

[0065] An objective of MAPF-trained neural network 206 is to maximize the number of goals completed within their scheduled time-windows. Relative throughput of goals completed within the time-windows may be measured. This normalizes the comparison to understand the effect of different time-window distributions on the reinforcement learning models. Example metrics that may be considered include percentage goals completed within a margin of lateness, highlighting the degree of suboptimality of the model (if any), as well as time to complete a predetermined number of goals (e.g., 100), normalizing across different schedules. Coefficient of variation, defined as the ratio between the mean and standard deviation of goal completion times, may also be employed, capturing the reliability of the model in producing similar performance.

[0066] FIG. 4 is a flowchart of an example method 400 of path finding for sample containers in diagnostic laboratory systems in accordance with some embodiments. In some implementations, one or more process blocks of FIG. 4 may be performed by a path finding program such as path finding program 204 of FIG. 2A. For example, memory 134 may include computer executable instructions that, when executed by processor 132, cause processor 132 to perform one or more of the process steps described with reference to FIG. 4. In at least some embodiments, such computer executable instructions may be stored in a non-transitory memory and / or be included in a non-transitory computer-readable medium.

[0067] As shown in FIG. 4, method 400 may include receiving a batch of sample containers in a diagnostic laboratory system having diagnostic laboratory equipment, one or more tracks connecting the diagnostic laboratory equipment, and a plurality of sample carriers configured to transport the sample containers within the diagnostic laboratory system, each of the sample containers containing a sample to be processed (block 402). For example, a sample handler such as sample handler 1 14 of FIG. 1 may receive a batch of sample containers (e.g., sample containers 104) in a diagnostic laboratory system (e.g., diagnostic laboratory system 100 of FIG. 1 or diagnostic laboratory system 300 of FIG. 3A) having diagnostic laboratory equipment (e.g., sample handlers, sample analyzers, or the like), one or more tracks connecting the diagnostic laboratory equipment, and a plurality of sample carriers configured totransport the sample containers within the diagnostic laboratory system, each of the sample containers containing a sample to be processed, as described above (e.g., sample handler 114, first sample analyzer 116, second analyzer 118, track 108, sample containers 104, and sample carriers 110 of FIG. 1 ). Note that some sample carriers may be empty until they receive sample containers at an input module of a diagnostic laboratory system (e.g., at sample handler 114 of diagnostic laboratory system 100).

[0068] As also shown in FIG. 4, method 400 may include obtaining a grid for the diagnostic laboratory system, the grid having cells that include the diagnostic laboratory equipment and the one or more tracks connecting the diagnostic laboratory equipment (block 404). For example, path finding program 204 of FIG. 2A may obtain grid 302 for diagnostic laboratory system 300 (FIG. 3A), grid 302 having cells 304 that include diagnostic laboratory equipment and one or more tracks connecting the diagnostic laboratory equipment, as described above. In some embodiments, the grid may be stored in memory 134 or generated automatically by scheduler program 202 and / or path finding program 204 and / or another program.

[0069] As further shown in FIG. 4, method 400 may include assigning an agent to each of the sample carriers (block 406). For example, path finding program 204 may assign an agent to each sample carrier 110 within the diagnostic laboratory system 100, as described above.

[0070] As also shown in FIG. 4, method 400 may include determining, for each agent assigned to a sample carrier holding a sample container, a goal including an earliest arrival time constraint, a deadline constraint, and a goal processing time constraint (block 408). For example, path finding program 204 (e.g., via scheduler program 202) may determine, for each agent assigned to a sample carrier 110 holding a sample container 104, a goal including an earliest arrival time constraint, a deadline constraint, and a goal processing time constraint, as described above. In some embodiments, this may include receiving earliest arrival time, deadline, and goal processing time constraints from a program such as scheduler program 202 (FIG. 2A). Further, in some embodiments, path finding program 204 (and / or scheduler program 202) may determine a series of goals for each agent, each goal including an earliest arrival time constraint, a deadline constraint, and a goal processing time constraint.

[0071] As further shown in FIG. 4, method 400 may include employing a MAPF- trained neural network to determine an action for each agent, the neural networktrained with arrival, deadline, and precedence constraints and with non-instantaneous goal processing times (block 410). For example, path finding program 204 may employ MAPF-trained neural network 206 to determine an action for each agent, neural network 206 trained with arrival, deadline, and precedence constraints with non- instantaneous goal processing times, as described above.

[0072] In some embodiments, employing the MAPF-trained neural network to determine an action for each agent may include inputting local field of view information for each agent into the neural network, the local field of view information representing a state of neighboring vertices within a predetermined distance of a current location of the agent. For example, the predetermined distance may be a predetermined radius R that defines a local field of view for each agent as a square with length 2*R+1 as shown in FIG. 3D for local field of view 310a.

[0073] Each cell may have a cell type, and the local field of view for an agent may include a center cell with the agent and a plurality of cells that surround the center cell such as is shown for local field of view 310a and local field of view 310b of FIGS. 3B- 3D.

[0074] In one or more embodiments, employing the MAPF trained neural network to determine an action for each agent may include inputting one or more masks (e.g., masks 318a-d of FIG. 3E) for the local field of view into the neural network, the one or more masks including at least one of a mask of obstacles, a mask of locations of all agents within the local field of view, a mask of goal locations of other agents within the local field of view, and a mask of the agent’s goal location, as described above. Further, employing the MAPF trained neural network to determine an action for each agent may include inputting at least one of distance information to the agent’s goal and an order for other agents to be processed if the other agents share a goal location with the agent.

[0075] Employing the MAPF trained neural network to determine an action for each agent may also include inputting to the neural network next goal location information for each agent. For example, next goal location information for each agent may include information regarding a difference between a current location for the agent and a next goal location for the agent, a time left until an earliest arrival time for an agent, a time left until a goal deadline, and / or whether the goal deadline has passed.

[0076] In some embodiments, neural network 206 may be trained employing at least one of a global positive reward for all agents when any agent completes a goal and a global negative reward for all agents when any agent is past a deadline.

[0077] Although FIG. 4 shows example blocks of method 400, in some implementations, method 400 may include additional blocks, fewer blocks, different blocks, or differently arranged blocks than those depicted in FIG. 4. Additionally, or alternatively, two or more of the blocks of method 400 may be performed in parallel.

[0078] In some embodiments, sample carrier controller 208 (FIG. 2A) of diagnostic laboratory system 100 or 300 may be employed to perform one or more of the actions determined by path finding program 204 and / or neural network 206 including transferring a sample container for processing (e.g., transferring a sample container 104 to sample analyzer 116 or 1 18).

[0079] FIG. 5 is a flowchart of another method 500 of path finding for sample containers in diagnostic laboratory systems in accordance with some embodiments. In some implementations, one or more process blocks of FIG. 5 may be performed by a processor such as processor 132 of FIG. 2A. For example, memory 134 may include computer executable instructions that, when executed by processor 132, cause processor 132 to perform one or more of the process steps described with reference to FIG. 5. In at least some embodiments, such computer executable instructions may be stored in a non-transitory memory and / or be included in a non-transitory computer- readable medium.

[0080] As shown in FIG. 5, method 500 may include creating a grid for a diagnostic laboratory system, the grid having cells that include diagnostic laboratory equipment and one or more tracks connecting the diagnostic laboratory equipment (block 502). For example, processor 132 executing one or more programs may create a grid for a diagnostic laboratory system, the grid having cells that include diagnostic laboratory equipment and the one or more tracks connecting the diagnostic laboratory equipment, as described above.

[0081] As also shown in FIG. 5, method 500 may include creating a plurality of agents, each agent representative of a sample carrier within the diagnostic laboratory system (block 504). For example, processor 132 executing one or more programs may create a plurality of agents, each agent representative of a sample carrier within the diagnostic laboratory system, as described above.

[0082] As further shown in FIG. 5, method 500 may include determining, for each agent assigned to a sample carrier holding a sample container, a goal including an earliest arrival time constraint, a deadline constraint, and a goal processing time constraint (block 506). For example, processor 132 executing scheduler program 202 and / or another program (e.g., path finding program 204) may determine, for each agent assigned to a sample carrier holding a sample container, a goal including an earliest arrival time constraint, a deadline constraint, and a goal processing time constraint, as described above.

[0083] As also shown in FIG. 5, method 500 may include creating a neural network having the earliest arrival time constraint, deadline constraint, and goal processing time constraint for an agent as inputs and actions or probabilities of actions for the agent as outputs (block 508). As described above, in some embodiments, neural network 206 may be any suitable neural network such as a convolutional neural network, a transformer neural network, or the like.

[0084] As further shown in FIG. 5, method 500 may include training the neural network with arrival, deadline, and precedence constraints and with non- instantaneous goal processing times (block 510). For example, processor 132 may train neural network 206 with arrival, deadline, and precedence constraints and with non-instantaneous goal processing times, as described above.

[0085] As also shown in FIG. 5, method 500 may include creating a path finding program that employs the trained neural network to generate actions for each agent within the diagnostic laboratory system (block 212). For example, processor 132 may be employed to create path finding program 204 that employs trained neural network 206 to generate actions for each agent within a diagnostic laboratory system, as described above.

[0086] Although FIG. 5 shows example blocks of method 500, in some implementations, method 500 may include additional blocks, fewer blocks, different blocks, or differently arranged blocks than those depicted in FIG. 5. Additionally, or alternatively, two or more of the blocks of method 500 may be performed in parallel.NON-LIMITING ILLUSTRATIVE EMBODIMENTS

[0087] The following provides a non-limiting list of illustrative embodiments of this disclosure:

[0088] An illustrative method of path finding for sample containers in diagnostic laboratory systems, the method comprising: receiving a batch of sample containers ina diagnostic laboratory system having diagnostic laboratory equipment, one or more tracks connecting the diagnostic laboratory equipment, and a plurality of sample carriers configured to transport the sample containers within the diagnostic laboratory system, each of the sample containers containing a sample to be processed; obtaining a grid for the diagnostic laboratory system, the grid having cells that include the diagnostic laboratory equipment and the one or more tracks connecting the diagnostic laboratory equipment; assigning an agent to each sample carrier within the diagnostic laboratory system; determining, for each agent assigned to a sample carrier holding a sample container, a goal including an earliest arrival time constraint, a deadline constraint, and a goal processing time constraint; and employing a multi-agent path finding (MAPF) trained neural network to determine an action for each agent, the neural network trained with arrival, deadline, and precedence constraints and with non-instantaneous goal processing times.

[0089] The illustrative method of any of the proceeding illustrative embodiments further comprising employing a sample carrier controller of the diagnostic laboratory system to perform one of more of the determined actions including transferring a sample container for processing.

[0090] The illustrative method of any of the proceeding illustrative embodiments wherein determining, for each agent assigned to a sample carrier holding a sample container, a goal including an earliest arrival time constraint, a deadline constraint, and a goal processing time constraint comprises receiving earliest arrival time, deadline, and goal processing time constraints from a scheduler program.

[0091] The illustrative method of any of the proceeding illustrative embodiments further comprising determining a series of goals for each agent, each goal including an earliest arrival time constraint, a deadline constraint, and a goal processing time constraint.

[0092] The illustrative method of any of the proceeding illustrative embodiments wherein employing the MAPF trained neural network to determine an action for each agent includes inputting local field of view information for each agent into the neural network, the local field of view information representing a state of neighboring vertices within a predetermined distance of a current location of the agent.

[0093] The illustrative method of any of the proceeding illustrative embodiments wherein the predetermined distance is a predetermined radius R that defines a local field of view for each agent as a square with length 2*R+1 .

[0094] The illustrative method of any of the proceeding illustrative embodiments wherein each cell has a cell type, and wherein the local field of view for an agent includes a center cell that includes the agent and a plurality of cells that surround the center cell.

[0095] The illustrative method of any of the proceeding illustrative embodiments wherein employing the MAPF trained neural network to determine an action for each agent includes inputting one or more masks for the local field of view into the neural network, the one or more masks including at least one of a mask of obstacles, a mask of locations of all agents within the local field of view, a mask of goal locations of other agents within the local field of view, and a mask of a goal location of the agent.

[0096] The illustrative method of any of the proceeding illustrative embodiments wherein employing the MAPF trained neural network to determine an action for each agent includes inputting at least one of distance information to the agent’s goal and an order for other agents to be processed if the other agents share a goal location with the agent.

[0097] The illustrative method of any of the proceeding illustrative embodiments wherein employing the MAPF trained neural network to determine an action for each agent includes inputting to the neural network next goal location information for each agent.

[0098] The illustrative method of any of the proceeding illustrative embodiments wherein the next goal location information for each agent includes information regarding a difference between a current location for the agent and a next goal location for the agent.

[0099] The illustrative method of any of the proceeding illustrative embodiments wherein the next goal location information for each agent includes at least one of a time left until an earliest arrival time for an agent, a time left until a goal deadline, and whether the goal deadline has passed.

[0100] The illustrative method of any of the proceeding illustrative embodiments wherein the neural network is trained employing at least one of a global positive reward for all agents when any agent completes a goal and a global negative reward for all agents when any agent is past a deadline.

[0101] An illustrative diagnostic laboratory system comprising diagnostic laboratory equipment; one or more tracks connecting the diagnostic laboratory equipment; a processor; and a memory coupled to the processor, the memory including computerexecutable instructions that, when executed by the processor, cause the processor to: obtain a grid for the diagnostic laboratory system, the grid having cells that include the diagnostic laboratory equipment and the one or more tracks connecting the diagnostic laboratory equipment; assign an agent to each sample carrier within the diagnostic laboratory system; determine for each agent assigned to a sample carrier holding a sample container, a goal including an earliest arrival time constraint, a deadline constraint, and a goal processing time constraint; and employ a multi-agent path finding (MAPF) trained neural network to determine an action for each agent, the neural network trained with arrival, deadline, and precedence constraints and with non-instantaneous goal processing times.

[0102] The illustrative diagnostic laboratory system of any of the proceeding illustrative embodiments further comprising a sample carrier controller and wherein the memory includes computer executable instructions that, when executed by the processor, cause the processor to employ the sample carrier controller to perform one of more of the determined actions including transferring a sample container for processing.

[0103] The illustrative diagnostic laboratory system of any of the proceeding illustrative embodiments wherein the memory includes computer executable instructions that, when executed by the processor, cause the processor to determine a series of goals for each agent, each goal including an earliest arrival time constraint, a deadline constraint, and a goal processing time constraint.

[0104] The illustrative diagnostic laboratory system of any of the proceeding illustrative embodiments wherein the memory includes computer executable instructions that, when executed by the processor, cause the processor to input local field of view information for each agent into the neural network, the local field of view information representing a state of neighboring vertices within a predetermined distance of a current location of the agent.

[0105] The illustrative diagnostic laboratory system of any of the proceeding illustrative embodiments wherein the predetermined distance is a predetermined radius R that defines a local field of view for each agent as a square with length 2*R+1 .

[0106] The illustrative diagnostic laboratory system of any of the proceeding illustrative embodiments wherein each cell has a cell type, and wherein the local field of view for an agent includes a center cell that includes the agent and a plurality of cells that surround the center cell.

[0107] The illustrative diagnostic laboratory system of any of the proceeding illustrative embodiments wherein the memory includes computer executable instructions that, when executed by the processor, cause the processor to input one or more masks for the local field of view into the neural network, the one or more masks including at least one of a mask of obstacles, a mask of locations of all agents within the local field of view, a mask of goal locations of other agents within the local field of view, and a mask of a goal location of the agent.

[0108] The illustrative diagnostic laboratory system of any of the proceeding illustrative embodiments wherein the memory includes computer executable instructions that, when executed by the processor, cause the processor to input for each agent distance information to the agents goal, and an order for other agents to be processed if the other agents share a goal location with the agent.

[0109] The illustrative diagnostic laboratory system of any of the proceeding illustrative embodiments wherein the memory includes computer executable instructions that, when executed by the processor, cause the processor to employ the MAPF trained neural network to determine an action for each agent by inputting to the neural network next goal location information for each agent.

[0110] The illustrative diagnostic laboratory system of any of the proceeding illustrative embodiments wherein the next goal location information for each agent includes information regarding a difference between a current location for the agent and a next goal location for the agent.

[0111] The illustrative diagnostic laboratory system of any of the proceeding illustrative embodiments wherein the next goal location information for each agent includes at least one of a time left until an earliest arrival time, a time left until a goal deadline, and whether the goal deadline has passed.

[0112] The illustrative diagnostic laboratory system of any of the proceeding illustrative embodiments wherein the neural network is trained employing at least one of a global positive reward for all agents when any agent completes a goal and a global negative reward for all agents when any agent is past a deadline.

[0113] An illustrative method of path finding for sample containers in diagnostic laboratory systems, the method comprising: creating a grid for a diagnostic laboratory system, the grid having cells that include diagnostic laboratory equipment and one or more tracks connecting the diagnostic laboratory equipment; creating a plurality of agents, each agent representative of a sample carrier within the diagnostic laboratorysystem; determining, for each agent assigned to a sample carrier holding a sample container, a goal including an earliest arrival time constraint, a deadline constraint, and a goal processing time constraint; creating a neural network having an earliest arrival time constraint, deadline constraint, and goal processing time constraint for an agent as inputs and actions or probabilities of actions for the agent as outputs; training the neural network with arrival, deadline, and precedence constraints and with non- instantaneous goal processing times; and creating a path finding program that employs the trained neural network to generate actions for each agent within the diagnostic laboratory system.

[0114] The illustrative method of any of the proceeding illustrative embodiments further comprising deploying the trained neural network and the path finding program for use by the diagnostic laboratory system.

[0115] The illustrative method of any of the proceeding illustrative embodiments further comprising employing the path finding program to determine one or more actions including transferring a sample container for processing.

[0116] The illustrative method of any of the proceeding illustrative embodiments wherein the neural network is configured to input local field of view information for each agent into the neural network, the local field of view information representing a state of neighboring vertices within a predetermined distance of a current location of the agent.

[0117] The illustrative method of any of the proceeding illustrative embodiments wherein the neural network is configured to input one or more masks for the local field of view into the neural network, the one or more masks including at least one of a mask of obstacles, a mask of locations of all agents within the local field of view, a mask of goal locations of other agents within the local field of view, and a mask of a goal location of the agent.

[0118] The illustrative method of any of the proceeding illustrative embodiments wherein the neural network is configured to input for each agent at least one of distance information to the agent’s goal and an order for other agents to be processed if the other agents share a goal location with the agent.

[0119] The illustrative method of any of the proceeding illustrative embodiments wherein the neural network is configured to input next goal location information for each agent.

[0120] The illustrative method of any of the proceeding illustrative embodiments wherein the next goal location information for each agent includes information regarding a difference between a current location for the agent and a next goal location for the agent.

[0121] The illustrative method of any of the proceeding illustrative embodiments wherein the next goal location information for each agent includes at least one of a time left until an earliest arrival time for an agent, a time left until a goal deadline, and whether the goal deadline has passed.

[0122] The illustrative method of any of the proceeding illustrative embodiments further comprising training the neural network with at least one of a global positive reward for all agents when any agent completes a goal and a global negative reward for all agents when any agent is past a deadline.

[0123] While the disclosure is susceptible to various modifications and alternative forms, specific method and apparatus embodiments have been shown by way of example in the drawings and are described in detail herein. It should be understood, however, that the particular methods and apparatus disclosed herein are not intended to limit the disclosure.

Claims

WHAT IS CLAIMED IS:

1. A method of path finding for sample containers in diagnostic laboratory systems, the method comprising: receiving a batch of sample containers in a diagnostic laboratory system having diagnostic laboratory equipment, one or more tracks connecting the diagnostic laboratory equipment, and a plurality of sample carriers configured to transport the sample containers within the diagnostic laboratory system, each of the sample containers containing a sample to be processed; obtaining a grid for the diagnostic laboratory system, the grid having cells that include the diagnostic laboratory equipment and the one or more tracks connecting the diagnostic laboratory equipment; assigning an agent to each of the sample carriers; determining, for each agent assigned to a sample carrier holding a sample container, a goal including an earliest arrival time constraint, a deadline constraint, and a goal processing time constraint; and employing a multi-agent path finding (MAPF) trained neural network to determine an action for each agent, the neural network trained with arrival, deadline, and precedence constraints and with non-instantaneous goal processing times.

2. The method of claim 1 further comprising employing a sample carrier controller of the diagnostic laboratory system to perform one of more of the determined actions including transferring a sample container for processing.

3. The method of claim 1 wherein determining, for each agent assigned to a sample carrier holding a sample container, a goal including an earliest arrival time constraint, a deadline constraint, and a goal processing time constraint comprises receiving earliest arrival time, deadline, and goal processing time constraints from a scheduler program.

4. The method of claim 1 further comprising determining a series of goals for each agent, each goal including an earliest arrival time constraint, a deadline constraint, and a goal processing time constraint.

5. The method of claim 1 wherein employing the MAPF trained neural network to determine an action for each agent includes inputting local field of view information for each agent into the neural network, the local field of view information representing a state of neighboring vertices within a predetermined distance of a current location of the agent.

6. The method of claim 5 wherein the predetermined distance is a predetermined radius R that defines a local field of view for each agent as a square with length 2*R+1 .

7. The method of claim 5 wherein each cell has a cell type, and wherein the local field of view for an agent includes a center cell that includes the agent and a plurality of cells that surround the center cell.

8. The method of claim 7 wherein employing the MAPF trained neural network to determine an action for each agent includes inputting one or more masks for the local field of view into the neural network, the one or more masks including at least one of a mask of obstacles, a mask of locations of all agents within the local field of view, a mask of goal locations of other agents within the local field of view, and a mask of a goal location of the agent.

9. The method of claim 1 wherein employing the MAPF trained neural network to determine an action for each agent includes inputting at least one of distance information to the agent’s goal and an order for other agents to be processed if the other agents share a goal location with the agent.

10. The method of claim 1 wherein employing the MAPF trained neural network to determine an action for each agent includes inputting to the neural network next goal location information for each agent.

11. The method of claim 10 wherein the next goal location information for each agent includes information regarding a difference between a current location for the agent and a next goal location for the agent.

12. The method of claim 10 wherein the next goal location information for each agent includes at least one of a time left until an earliest arrival time for an agent, a time left until a goal deadline, and whether the goal deadline has passed.

13. The method of claim 1 wherein the neural network is trained employing at least one of a global positive reward for all agents when any agent completes a goal and a global negative reward for all agents when any agent is past a deadline.

14. A diagnostic laboratory system comprising: diagnostic laboratory equipment; one or more tracks connecting the diagnostic laboratory equipment; a processor; and a memory coupled to the processor, the memory including computer executable instructions that, when executed by the processor, cause the processor to: obtain a grid for the diagnostic laboratory system, the grid having cells that include the diagnostic laboratory equipment and the one or more tracks connecting the diagnostic laboratory equipment; assign an agent to each sample carrier within the diagnostic laboratory system; determine for each agent assigned to a sample carrier holding a sample container, a goal including an earliest arrival time constraint, a deadline constraint, and a goal processing time constraint; and employ a multi-agent path finding (MAPF) trained neural network to determine an action for each agent, the neural network trained with arrival, deadline, and precedence constraints and with non-instantaneous goal processing times.

15. The diagnostic laboratory system of claim 14 further comprising a sample carrier controller and wherein the memory includes computer executable instructions that, when executed by the processor, cause the processor to employ the sample carrier controller to perform one of more of the determined actions including transferring a sample container for processing.

16. The diagnostic laboratory system of claim 14 wherein the memory includes computer executable instructions that, when executed by the processor, cause theprocessor to determine a series of goals for each agent, each goal including an earliest arrival time constraint, a deadline constraint, and a goal processing time constraint.

17. The diagnostic laboratory system of claim 14 wherein the memory includes computer executable instructions that, when executed by the processor, cause the processor to input local field of view information for each agent into the neural network, the local field of view information representing a state of neighboring vertices within a predetermined distance of a current location of the agent.

18. The diagnostic laboratory system of claim 17 wherein the predetermined distance is a predetermined radius R that defines a local field of view for each agent as a square with length 2*R+1 .

19. The diagnostic laboratory system of claim 17 wherein each cell has a cell type, and wherein a local field of view for an agent includes a center cell that includes the agent and a plurality of cells that surround the center cell.

20. The diagnostic laboratory system of claim 19 wherein the memory includes computer executable instructions that, when executed by the processor, cause the processor to input one or more masks for the local field of view into the neural network, the one or more masks including at least one of a mask of obstacles, a mask of locations of all agents within the local field of view, a mask of goal locations of other agents within the local field of view, and a mask of a goal location of the agent.

21. The diagnostic laboratory system of claim 14 wherein the memory includes computer executable instructions that, when executed by the processor, cause the processor to input for each agent distance information to the agent’s goal, and an order for other agents to be processed if the other agents share a goal location with the agent.

22. The diagnostic laboratory system of claim 14 wherein the memory includes computer executable instructions that, when executed by the processor, cause the processor to employ the MAPF trained neural network to determine an action for each agent by inputting to the neural network next goal location information for each agent.

23. The diagnostic laboratory system of claim 22 wherein the next goal location information for each agent includes information regarding a difference between a current location for the agent and a next goal location for the agent.

24. The diagnostic laboratory system of claim 22 wherein the next goal location information for each agent includes at least one of a time left until an earliest arrival time, a time left until a goal deadline, and whether the goal deadline has passed.

25. The diagnostic laboratory system of claim 14 wherein the neural network is trained employing at least one of a global positive reward for all agents when any agent completes a goal and a global negative reward for all agents when any agent is past a deadline.

26. A method of path finding for sample containers in diagnostic laboratory systems, the method comprising: creating a grid for a diagnostic laboratory system, the grid having cells that include diagnostic laboratory equipment and one or more tracks connecting the diagnostic laboratory equipment; creating a plurality of agents, each agent representative of a sample carrier within the diagnostic laboratory system; determining, for each agent assigned to a sample carrier holding a sample container, a goal including an earliest arrival time constraint, a deadline constraint, and a goal processing time constraint; creating a neural network having an earliest arrival time constraint, deadline constraint, and goal processing time constraint for an agent as inputs and actions or probabilities of actions for the agent as outputs; training the neural network with arrival, deadline, and precedence constraints and with non-instantaneous goal processing times; and creating a path finding program that employs the trained neural network to generate actions for each agent within the diagnostic laboratory system.

27. The method of claim 26 further comprising deploying the trained neural network and the path finding program for use by the diagnostic laboratory system.

28. The method of claim 27 further comprising employing the path finding program to determine one or more actions including transferring a sample container for processing.

29. The method of claim 26 wherein the neural network is configured to input local field of view information for each agent into the neural network, the local field of view information representing a state of neighboring vertices within a predetermined distance of a current location of the agent.

30. The method of claim 29 wherein the neural network is configured to input one or more masks for a local field of view into the neural network, the one or more masks including at least one of a mask of obstacles, a mask of locations of all agents within the local field of view, a mask of goal locations of other agents within the local field of view, and a mask of a goal location of the agent.31 . The method of claim 26 wherein the neural network is configured to input for each agent at least one of distance information to the agent’s goal and an order for other agents to be processed if the other agents share a goal location with the agent.

32. The method of claim 26 wherein the neural network is configured to input next goal location information for each agent.

33. The method of claim 32 wherein the next goal location information for each agent includes information regarding a difference between a current location for the agent and a next goal location for the agent.

34. The method of claim 32 wherein the next goal location information for each agent includes at least one of a time left until an earliest arrival time for an agent, a time left until a goal deadline, and whether the goal deadline has passed.

35. The method of claim 26 further comprising training the neural network with at least one of a global positive reward for all agents when any agent completes a goal and a global negative reward for all agents when any agent is past a deadline.

Images