Scenario generation for path planning

Abstract

A system can generate a database of various scenarios in a work environment that includes plurality of agents capable of moving within the work environment according to respective dynamics of each agent. The database can receive a requirement related to a movement path of at least one of the plurality of agents. Responsive to the requirement, the system can determine whether one or more scenarios defined by the database satisfy the requirement.

Description

SCENARIO GENERATION FOR PATH PLANNINGBACKGROUND

[0001] Autonomous operations, such as robotic grasping and placing, in unknown or dynamic environments present various technical challenges. Furthermore, humans and robots (autonomous cyber physical systems) often share workspace in modern manufacturing environments. For example, in an industrial setting, robotic arms might operate at high speed such that the regions around such robots can be considered hazard zones. These environments are dynamic in nature with many uncertainties such as, for example, the movement of human workers themselves, robots, etc.

[0002] Various issues arise, for example safety-critical issues, because small errors can cause harm to machine or human workers. Therefore, robots, human workers, and other assets in these environments are often monitored with multi-modal sensor technologies such as cameras, RTLS (real-time location systems,) etc. In some cases, runtime monitoring techniques can be employed to enhance safety to human workers. The testing of such autonomous safety critical systems can be of paramount importance. It is recognized herein, however, that current approaches to testing a such a system in an environment with dynamic objects lack capabilities due to various technical challenges.BRIEF SUMMARY

[0003] Embodiments of the invention address and overcome one or more of the described- herein shortcomings or technical problems by providing methods, systems, and apparatuses for enhancing operational safety and functionality of various systems that operate within dynamic environments. For example, a system can generate various scenarios and test data to satisfy various conditions in requirements in dynamic environments.

[0004] In an example aspect, a computing system the system can generate a model of a system defining a work environment and a plurality of agents capable of moving within the work environment according to respective dynamics of each agent. Based on the model, the system can generate a rapidly exploring random tree (RRT) for each agent of the plurality of agents, so as to define a plurality of trees, each RRT of the plurality of trees defining a set of trajectories of the respective agent within the work environment consistent with the respective dynamics.Each RRT can define multiple levels having a plurality of nodes at each of the multiple levels. The system can generate a database by combining nodes at each level, so as to define a product tree representative of the plurality of agents in the database. The database can receive a requirement related to a movement path of at least one of the plurality of agents. Responsive to the requirement, the system can determine a plurality of scenarios defined by the database that satisfy the requirement. The system can extract the plurality of scenarios from the database by performing a plurality of model checking operations. In some cases, the system can receive the requirement that is specified in discrete-time signal temporal logic (DT-STL), and can extract the plurality of scenarios based on the DT-STL. In another example aspect, the requirement stipulates two or more agents of the plurality of agents are in a specific location relative to a hazard within the environment, within a specific range of time.BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

[0005] The foregoing and other aspects of the present invention are best understood from the following detailed description when read in connection with the accompanying drawings. For the purpose of illustrating the invention, there is shown in the drawings embodiments that are presently preferred, it being understood, however, that the invention is not limited to the specific instrumentalities disclosed. Included in the drawings are the following Figures:

[0006] FIG. 1 shows example inputs and output for a computing system to construct a rapidly exploring random tree (RRT) in accordance with an example embodiment.

[0007] FIG. 2 illustrates example operations that the computing system can perform to construct an RRT, in accordance with an example embodiment.

[0008] FIG. 3 illustrates example RRTs for specific dynamic agents within an environment, and a product tree that combines the RRTs for the respective agents, in accordance with an example embodiment.

[0009] FIG. 4 illustrates how model checking can be performed on the product tree to generate various scenarios, in accordance with an example embodiment.

[0010] FIG. 5 illustrates example operations that the computing system can perform to perform model checking, in accordance with an example embodiment.

[0011] FIG. 6A illustrates example RRTs that the system can generate, in accordance with an example use case.

[0012] FIG. 6B illustrates trajectories of agents that satisfy an example requirement, in accordance with an example embodiment.

[0013] FIG. 6C illustrates example scenarios that satisfy the example requirement of FIGs. 6A and 6B.

[0014] FIG. 7 illustrates a computing environment within which embodiments of the disclosure may be implemented.DETAILED DESCRIPTION

[0015] As an initial matter, it is recognized herein that safety- critical autonomous systems often operate in highly uncertain environments that can consist of various agents, some of which are designed (e.g., robots) and some represent the uncertain aspects of the environment (e.g., humans). Testing autonomous systems can require the generation of diverse scenarios. For example, test methodologies in dynamic environments often require scenarios and test data that are difficult to obtain, as dynamic environments involve one or more agents (e.g., humans, robots, etc.) that can move freely and in an unpredictable manner within the environment. It is further recognized herein that potential scenarios can be astronomically large in number and extracting interesting or useful scenarios is a technical challenge. By way of example, an example scenario might include that ten human workers walk on a factory floor and at some point in time, two of them enter a particular hazard zone. Examples described herein are often described in the context of industrial manufacturing environments, though it will be understood that embodiments can be implemented in alternative or additional environments (e.g., roadways with autonomous driving), and all such environments are contemplated as being within the scope of this disclosure.

[0016] By way of background, it is recognized herein that robot programming is typically explicit. For example, a programmer might use code to command a robot to move to various locations. The positions of the robot can be specified using numeric values and data structures representing joint positions and paths. Some robots are programmed explicitly using teach points that represent recorded robotic poses that are stored in a robot application. Such teach points can be referenced by issued commands. The poses may be generated by using the manual interface of the robot to put it in a particular pose, or it may be “demonstrated” by putting the robot’s motors into a resistive state, grabbing the robot manually, and physicallymoving it into the desired pose. The teach point may also be demonstrated virtually by using a digital representation of the robot and putting that representation into the desired pose.

[0017] For a robot to operate autonomously, it can infer important attributes of its environment and act accordingly. In some cases, robots operate within a range from completely programmed and command driven to completely autonomous and sensor driven. In more autonomous scenarios, the path that the robot follows is often not predetermined. When the robot moves in new ways or along new paths, it is important that the robot does not collide with unintended objects, such as the work surface, humans, fixtures, other robots, and the like. In order to avoid collisions, in some cases, the application should ascertain the other physical objects that may interfere with the robot’s movement, and then calculate a path that avoids the obstacles. Additionally, or alternatively, the system can use sensors, such as a 3D camera, that can survey an area around the robot and update a dynamic world model as the application progresses.

[0018] To test the safety of autonomous systems, in some cases, a digital twin is created. Simulations can be generated to test the system’s behavior in nominal and edge case scenarios. By way of further background, OpenScenario uses an XML format typically used for modeling autonomous vehicle scenarios, where the motion of an agent is characterized through actions and triggers. In OpenScenario, it is recognized herein that different programs need to be written to observe various behaviors. In contrast, various embodiments described herein perform sampling-based planning, which results in an advantage, among others, of being able to extract various types of scenarios from the same trees. For example, a tree (dataset) might be generated once and different scenarios can be extracted from that dataset. By way of further background, Scenic is a probabilistic programming language capable of handling dynamic scenarios through actions specified by simulators. A Scenic program defines a probability distribution on all the objects and the properties of the scenes. Scenic samples scenes from this distribution based on rejection sampling (i.e., it repeatedly samples scenes until the scene that satisfies the requirement is found). It is recognized herein, however, that if the requirements have a low probability of being satisfied, it may take many iterations to find a valid scene. Thus, in the worst case, if the requirements cannot be satisfied, rejection sampling can run forever. In contrast to this approach, embodiments described herein focus on a requirement- driven approach. It is further recognized herein that data-driven scenario generation utilizeslarge databases of test cases, but these simulation-based datasets have a major drawback that they lack certain scenarios that can lead to unsafe situations.

[0019] Referring now to FIG. 1, a computing or simulation system can generate various rapidly exploring random trees, for instance a rapidly exploring random tree (RRT) 102 that can be used to generate possible trajectories for agents within an environment that is consistent with a given agent’s dynamics. The RRT described herein can generate possible paths for the agents that give coverage over the state space. A given autonomous system can define dynamic agents (e.g., human workers) and an environment (e.g., factory floor with hazard regions). A scenario can define a set of trajectories for each agent that satisfy a specification or requirement. Various examples described herein describe incremental sampling-based path planning to generate a very large set of scenarios consistent with a given scenario description. Examples described herein are based on rapidly exploring random trees (RRT) mechanisms and variants thereof, but will be understood that embodiments can implement alternative mechanisms, such as probabilistic roadmaps and variants thereof, and all such mechanisms are contemplated as being within the scope of this disclosure. In various examples, scenario requirements are defined in discrete-time Signal Temporal Logic (STL). Extracting scenarios satisfying a given discrete-time STL requirement can be performed by an adaptation of CTL model checking operations, described herein.

[0020] Still referring to FIG. 1, the RRT 102 can be generated based on agent dynamics 104 and an environment specification 106. The environment specification 106 can define an abstract representation of the physical world containing (mobile) agents, static objects, sensors, obstacles, and the like, and thus can define an environment E. The environment E can be represented as a tuple L, A, H, Z where L represents a finite or infinite set of locations within the environment, A represents a set of dynamic autonomous agents, H represents a set of hazard regions (where Hk£ )anc[ z represents a set of sensors that can observe the agents. By way of example, in a warehouse environment, there might be multiple humans (agents) working in the factory around robot manipulators. The area around the robots can be considered hazard regions, as those regions might not be safe for humans to enter the while the robots are operating.

[0021] The agent dynamics 104 can define a model. In some examples, the agents in the environment can be modeled using the dynamics of the Dubins cars. The environment can have a finite set of agents. A given agent A can be defined as tuple S, U, T, n where S represents a setof states, such that at any time t, the agent can be in state stE S. U represents a set of actions the agent can take, such that an action at time t can be represented as ut. T can represent a function that maps a state in S and an action in U to a next (in S), and n can represent a stochastic policy of the agents that represents a distribution over the set U conditioned by the current state. In various examples, a given agent can have several independent state variables, and the state space of the agent can be represented as the Cartesian product of the domains of each of the state variables. The state variables include the location I of the agent in the environment and the orientation 0 of the agent in the environment, where 0= {0} (the angle between the agent’s heading and the X-axis) for 2-D environments, and 0= {< >, 0, <p} (the angle between the heading of the agent with X, Y, Z axis, respectively, for 3-D environments. A hazard H can be represented as a tuple that indicates the set of contiguous locations that constitute the hazard H, and a state that indicates the state of the hazard. For example, the environment can be equipped with sensors (e.g., RFID, lasers, etc.) that observe the environment. At any time t, the environment can change and the states of agents can be updated accordingly. Thus, the configuration of the environment at time t can be defined by the states of each of the agents A and the hazards H.

[0022] Referring also to FIG. 2, the computing or simulation system can perform operations 101 to generate the RRT 102, based on the agent dynamics 104 and the environment specification 106. In particular, for example, the system can split the environment that the environment E is represented as n x m blocks to systematically cover the working area of the agents. A block B can be defined as the block located at row i and column j of the environment E such that Bij c L. To ensure the coverage of the environment by the agents, the system can sample goal points from each block of the environment. As shown in operations 101, the coverage based-RRT 102 Tj = (F E,) can be constructed for each agent in the environment. Referring also to FIG. 3, the system can generate a first RRT 102a corresponding to a first agent, and the system can generate a second RRT 102b corresponding to a second agent, though it will be understood that additional trees can be constructed for additional agents. The operations 101 can take as input a set of initial configurations and a maximum number of iterations. The initial state of a given agent can be sampled from a set of initial configurations. For example, the system can initialize Vt=and Et= < >. The initial state xinitcan be assigned as the root node of the RRT of an agent At. At each time t, a random configuration xsampiecan be sampled from the state space S. Next, the system can search thenode xnearE V( in the tree Tj that is closest to xsampieusing Euclidean distance. An input u can be determined to move the system from xnearto xsample. In various examples, velocity v is an input that can selected from the minimum and maximum velocity, and the steering angle 0 E [0,2TT]. The input u can be applied to identify the new node xnewusing the dynamics of the system f x, u). The path between xnearand xnewcan be checked for collisions before adding it to the tree T;. When the path is collision free, the system can add xnewto the vertex set V(, and the edge (xnear, xnew) can be added to the edge set Et. As shown in operations 101, the tree 102 can be iteratively built until there are stepsmaxin the tree T;or all of the goals have been reached. At each node, the system can store the state s, inputs u, and time t to reach xnewfrom the root. Thus, the timestamp t at each node can define the depth of that node in the tree.

[0023] Thus, the RRT 102 can define an RRT variant that gives coverage over the environment, and can be used to generate possible trajectories for an agent that is consistent with the agent’s dynamics. Each human agent can maintain a tree of possible waypoints (trajectory). Each node in the RRT 102 can tree represent a hybrid state: (x, y, velocity, 0, timestamp) where x, y, z represent the 3D co-ordinates, 6 is the orientation, and v is the velocity in which an agent moves from one state to the next state. The RRT tree 102 grows randomly, thereby interesting or useful nodes are also generated (such as the nodes that reach hazard zones or other interesting regions). In various examples, operations 101 are repeated for each agent until we have trees for each agent.

[0024] Referring to FIG. 3, the generated RRT for each agent, for instance the first and second RRT 102a and 102, can be combined in a multi-agent or product RRT, for instance a multiagent product RRT 300. The RRT of a given agentcan be represented as Tj = (Vj, £)), where Vtis the set of nodes and Etis the set of edges in the tree. Each node can be associated with state information x, actions a, and a timestamp t. The agent trees (e.g., 102a andl02b) can be combined to generate a multi-agent tree (e.g., tree 300). The tree 300 can be constructed by taking the Cartesian product of all nodes at time t in each agent tree. At t = 0, the root of the tree is initialized, and at each time t, nodes are extracted from all trees that can be reached in time t from the root nodes of each agent tree. Thus, the product RRT 300 can be generated by combining nodes at each level, in particular, by computing the Cartesian product of all nodes at time t, and connecting them to their combined parent nodes.

[0025] Referring to FIG. 4, after constructing the combined tree (e.g., tree 300), scenarios can be extracted that satisfy a requirement <p, for instance a requirement defined in a scenario specification 402. In particular, the scenario specification 402 or requirement can be specified in the Discrete-time Signal Temporal Logic (DT-STL). Referring also to FIG. 5, the system can perform model checking operations 401 to extract interesting or useful scenarios, for instance a first scenario 404a and a second scenario 404b, which satisfy certain conditions or requirements from the scenario specification 402. Operations 401 can include a CTL model checking algorithm. The model checking operations 401 can be performed using a recursive procedure that computes the satisfaction sets of all subformulae of requirements, and finally evaluates if any of the initial states are present in satisfaction (Sat) sets of <p. The recursive computation of Sat(<p) can be based on the bottom-up traversal of the parse tree of the requirement formula. For each node in the parse tree (i.e., for each subformula \| / of <p)the Sat(y) can be computed such that: Sat(\p) = {v G T | v |= \| / }. This computation can start from the leaf nodes and progress toward the root of the parse tree. At an intermediate node, in various examples, the satisfaction sets of its children are used and combined appropriately to establish the states of its associated subformula. The type of computation at such node can depend on the operator at the “top-level” of the subformula. Once the system reaches the root of the parse tree, satisfaction set of the requirement Sat(<p) is obtained. Thus, the system can identify if there are any scenarios that satisfy <p if T G Sat(<p). Scenarios can be extracted by tracing the tree T to obtain the paths in the T that satisfy <p.

[0026] By way of example, and without limitation, a first example specification can be represented as F[0, T]{(distance (x, y)i ), (( ,y)2) < s)}, which requires that a multi-agent RRT produces an event which, between the time steps 0 to T, the distance between the agents is less than epsilon. A second example specification can be represented asF[0, T](((x, y)i) in hazard) && ((%, y)2) in hazard')'), which requires two people to be in the hazard zone at some time t between 0 to T. By way of further example, a scenario can be defined as the finite sequence of configurations generated by the moves within the environment. A scenario can be represented as length L G N as {c1(c2cL}. Another example scenario in the industrial warehouse domain is that all agents should walk near a robot to deliver some items. In various examples, DT-STL is the formal language that represents such scenario requirements.

[0027] By way of example, and without limitation, embodiments described herein are implemented in an example use case involving an industrial warehouse. In many industrial warehouses, mobile robots and fixed robotic manipulators are employed to perform various tasks like delivering packages to stations, assembling manufacturing parts, etc. Such robots are mostly operated in fenced areas because of the safety hazards they could cause to the humans around the factory. If the humans navigate close to the robots, the robots are turned off. This hampers the productivity in the warehouse. Monitoring the human activities near the working region of the robots could help avoid accidents and ensure the safety of humans working in warehouses. In an example, embodiments are implemented in an industrial warehouse where human operators navigate on the factory floor in the presence of several robots. These robots are fixed at specific positions and perform manipulation tasks in their work area. The work areas around these robots are considered hazard zones. In the example, a Unity-based simulator is built to synthesize these settings and visualize the scenarios.

[0028] The agents in the example environment are modeled in accordance with the dynamics of Dubin cars and can represented as:In the example, the environment defines an M*M warehouse with n agents that maneuver in a square area in the presence of k hazardous regions. In one example, two agents are specified in the warehouse environment with two manipulator robots. The robots are surrounded by hazard regions. There are four doors to the warehouse, and the agents can enter only through these entrances into the warehouse.

[0029] Continuing with the example, the system can evaluate various scenarios in the warehouse. For example, the system can receive a requirement that stipulates scenarios where all the agents walk only in safe regions. Such scenarios can be represented by <pi:

[0030] FIG. 6A shows complete for both the example agents, which can be generated by performing operations 101. FIG. 6B shows example scenarios that satisfy the requirement (pi. The scenarios can be extracted from the multi-agent RRT using the model-checking operations 401. FIG. 6C shows an example of five scenarios generated by the system that satisfy <pi. As shown, the trajectories for both agents do not intersect with the hazard regions.

[0031] Thus, the system described herein can generate a model of a system defining a work environment and a plurality of agents capable of moving within the work environment according to respective dynamics of each agent. Based on the model, the system can generate a rapidly exploring random tree (RRT) for each agent of the plurality of agents, so as to define a plurality of trees, each RRT of the plurality of trees defining a set of trajectories of the respective agent within the work environment consistent with the respective dynamics. Each RRT can define multiple levels having a plurality of nodes at each of the multiple levels. The system can generate a database by combining nodes at each level, so as to define a product tree representative of the plurality of agents in the database. The database can receive a requirement related to a movement path of at least one of the plurality of agents. Responsive to the requirement, the system can determine a plurality of scenarios defined by the database that satisfy the requirement. The system can extract the plurality of scenarios from the database by performing a plurality of model checking operations. In some cases, the system can receive the requirement that is specified in discrete-time signal temporal logic (DT-STL), and extract the plurality of scenarios based on the DT-STL. In another example aspect, the requirement stipulates two or more agents of the plurality of agents are in a specific location relative to a hazard within the environment, within a specific range of time.

[0032] Without being bound by theory, it is recognized herein that scenic programs might allow the specification of temporal logic requirements using LTL (Linear Temporal Logic) that constrain the set of scenes that will be generated. The overall approach of scenario generation in Scenic is based on rejection sampling. The requirements are used to constrain the generation of test scenarios to only interesting scenarios; for example, “at some time, two workers are within some 8 distance of each other.” However, the set of scenarios satisfying the requirement may have a very small measure in the overall distribution, making a rejection sampling-based approach inefficient as it is not requirement-driven. In contrast, our embodiments described herein define a requirement-driven scenario generation with the use of incremental samplingbased path planning operations to generate a very large set of scenarios consistent with a given scenario description. The extraction of satisfying a given Discrete-time STL requirement is performed using the model checking operations 401.

[0033] EIG. 7 illustrates an example of a computing environment that can include the simulation system within which embodiments of the present disclosure may be implemented. A computing environment 700 includes a computer system 710 that may include a communicationmechanism such as a system bus 721 or other communication mechanism for communicating information within the computer system 710. The computer system 710 further includes one or more processors 720 coupled with the system bus 721 for processing the information.

[0034] The processors 720 may include one or more central processing units (CPUs), graphical processing units (GPUs), or any other processor known in the art. More generally, a processor as described herein is a device for executing machine-readable instructions stored on a computer readable medium, for performing tasks and may comprise any one or combination of, hardware and firmware. A processor may also comprise memory storing machine-readable instructions executable for performing tasks. A processor acts upon information by manipulating, analyzing, modifying, converting or transmitting information for use by an executable procedure or an information device, and / or by routing the information to an output device. A processor may use or comprise the capabilities of a computer, controller or microprocessor, for example, and be conditioned using executable instructions to perform special purpose functions not performed by a general purpose computer. A processor may include any type of suitable processing unit including, but not limited to, a central processing unit, a microprocessor, a Reduced Instruction Set Computer (RISC) microprocessor, a Complex Instruction Set Computer (CISC) microprocessor, a microcontroller, an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), a System-on-a-Chip (SoC), a digital signal processor (DSP), and so forth. Further, the processor(s) 720 may have any suitable microarchitecture design that includes any number of constituent components such as, for example, registers, multiplexers, arithmetic logic units, cache controllers for controlling read / write operations to cache memory, branch predictors, or the like. The microarchitecture design of the processor may be capable of supporting any of a variety of instruction sets. A processor may be coupled (electrically and / or as comprising executable components) with any other processor enabling interaction and / or communication there-between. A user interface processor or generator is a known element comprising electronic circuitry or software or a combination of both for generating display images or portions thereof. A user interface comprises one or more display images enabling user interaction with a processor or other device.

[0035] The system bus 721 may include at least one of a system bus, a memory bus, an address bus, or a message bus, and may permit exchange of information (e.g., data (including computer-executable code), signaling, etc.) between various components of the computersystem 710. The system bus 721 may include, without limitation, a memory bus or a memory controller, a peripheral bus, an accelerated graphics port, and so forth. The system bus 721 may be associated with any suitable bus architecture including, without limitation, an Industry Standard Architecture (ISA), a Micro Channel Architecture (MCA), an Enhanced ISA (EISA), a Video Electronics Standards Association (VESA) architecture, an Accelerated Graphics Port (AGP) architecture, a Peripheral Component Interconnects (PCI) architecture, a PCI -Express architecture, a Personal Computer Memory Card International Association (PCMCIA) architecture, a Universal Serial Bus (USB) architecture, and so forth.

[0036] Continuing with reference to FIG. 7, the computer system 710 may also include a system memory 730 coupled to the system bus 721 for storing information and instructions to be executed by processors 720. The system memory 730 may include computer readable storage media in the form of volatile and / or nonvolatile memory, such as read only memory (ROM) 731 and / or random access memory (RAM) 732. The RAM 732 may include other dynamic storage device(s) (e.g., dynamic RAM, static RAM, and synchronous DRAM). The ROM 731 may include other static storage device(s) (e.g., programmable ROM, erasable PROM, and electrically erasable PROM). In addition, the system memory 730 may be used for storing temporary variables or other intermediate information during the execution of instructions by the processors 720. A basic input / output system 733 (BIOS) containing the basic routines that help to transfer information between elements within computer system 710, such as during start-up, may be stored in the ROM 731. RAM 732 may contain data and / or program modules that are immediately accessible to and / or presently being operated on by the processors 720. System memory 730 may additionally include, for example, operating system 734, application programs 735, and other program modules 736. Application programs 735 may also include a user portal for development of the application program, allowing input parameters to be entered and modified as necessary.

[0037] The operating system 734 may be loaded into the memory 730 and may provide an interface between other application software executing on the computer system 710 and hardware resources of the computer system 710. More specifically, the operating system 734 may include a set of computer-executable instructions for managing hardware resources of the computer system 710 and for providing common services to other application programs (e.g., managing memory allocation among various application programs). In certain example embodiments, the operating system 734 may control execution of one or more of the programmodules depicted as being stored in the data storage 740. The operating system 734 may include any operating system now known or which may be developed in the future including, but not limited to, any server operating system, any mainframe operating system, or any other proprietary or non-proprietary operating system.

[0038] The computer system 710 may also include a disk / media controller 743 coupled to the system bus 721 to control one or more storage devices for storing information and instructions, such as a magnetic hard disk 741 and / or a removable media drive 742 (e.g., floppy disk drive, compact disc drive, tape drive, flash drive, and / or solid state drive). Storage devices 740 may be added to the computer system 710 using an appropriate device interface (e.g., a small computer system interface (SCSI), integrated device electronics (IDE), Universal Serial Bus (USB), or FireWire). Storage devices 741 , 742 may be external to the computer system 710.

[0039] The computer system 710 may perform a portion or all of the processing steps of embodiments of the invention in response to the processors 720 executing one or more sequences of one or more instructions contained in a memory, such as the system memory 730. Such instructions may be read into the system memory 730 from another computer readable medium of storage 740, such as the magnetic hard disk 741 or the removable media drive 742. The magnetic hard disk 741 (or solid state drive) and / or removable media drive 742 may contain one or more data stores and data files used by embodiments of the present disclosure. The data store 740 may include, but are not limited to, databases (e.g., relational, object-oriented, etc.), file systems, flat files, distributed data stores in which data is stored on more than one node of a computer network, peer-to-peer network data stores, or the like. The data stores may store various types of data such as, for example, skill data, sensor data, or any other data generated in accordance with the embodiments of the disclosure. Data store contents and data files may be encrypted to improve security. The processors 720 may also be employed in a multi-processing arrangement to execute the one or more sequences of instructions contained in system memory 730. In alternative embodiments, hard-wired circuitry may be used in place of or in combination with software instructions. Thus, embodiments are not limited to any specific combination of hardware circuitry and software.

[0040] As stated above, the computer system 710 may include at least one computer readable medium or memory for holding instructions programmed according to embodiments of the invention and for containing data structures, tables, records, or other data described herein. The term “computer readable medium” as used herein refers to any medium that participates inproviding instructions to the processors 720 for execution. A computer readable medium may take many forms including, but not limited to, non-transitory, non-volatile media, volatile media, and transmission media. Non-limiting examples of non-volatile media include optical disks, solid state drives, magnetic disks, and magneto-optical disks, such as magnetic hard disk 741 or removable media drive 742. Non-limiting examples of volatile media include dynamic memory, such as system memory 730. Non-limiting examples of transmission media include coaxial cables, copper wire, and fiber optics, including the wires that make up the system bus 721. Transmission media may also take the form of acoustic or light waves, such as those generated during radio wave and infrared data communications.

[0041] Computer readable medium instructions for carrying out operations of the present disclosure may be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, statesetting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) may execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present disclosure.

[0042] Aspects of the present disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchartillustrations and / or block diagrams, may be implemented by computer readable medium instructions.

[0043] The computing environment 700 may further include the computer system 710 operating in a networked environment using logical connections to one or more remote computers, such as remote computing device 780. The network interface 770 may enable communication, for example, with other remote devices 780 or systems and / or the storage devices 741, 742 via the network 771. Remote computing device 780 may be a personal computer (laptop or desktop), a mobile device, a server, a router, a network PC, a peer device or other common network node, and typically includes many or all of the elements described above relative to computer system 710. When used in a networking environment, computer system 710 may include modem 772 for establishing communications over a network 771, such as the Internet. Modem 772 may be connected to system bus 721 via user network interface 770, or via another appropriate mechanism.

[0044] Network 771 may be any network or system generally known in the art, including the Internet, an intranet, a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a direct connection or series of connections, a cellular telephone network, or any other network or medium capable of facilitating communication between computer system 710 and other computers (e.g., remote computing device 780). The network 771 may be wired, wireless or a combination thereof. Wired connections may be implemented using Ethernet, Universal Serial Bus (USB), RJ-6, or any other wired connection generally known in the art. Wireless connections may be implemented using Wi-Fi, WiMAX, and Bluetooth, infrared, cellular networks, satellite or any other wireless connection methodology generally known in the art. Additionally, several networks may work alone or in communication with each other to facilitate communication in the network 771.

[0045] It should be appreciated that the program modules, applications, computer-executable instructions, code, or the like depicted in FIG. 7 as being stored in the system memory 730 are merely illustrative and not exhaustive and that processing described as being supported by any particular module may alternatively be distributed across multiple modules or performed by a different module. In addition, various program module(s), script(s), plug-in(s), Application Programming Interface(s) (API(s)), or any other suitable computer-executable code hosted locally on the computer system 710, the remote device 780, and / or hosted on other computing device(s) accessible via one or more of the network(s) 771, may be provided to supportfunctionality provided by the program modules, applications, or computer-executable code depicted in FIG. 7 and / or additional or alternate functionality. Further, functionality may be modularized differently such that processing described as being supported collectively by the collection of program modules depicted in FIG. 7 may be performed by a fewer or greater number of modules, or functionality described as being supported by any particular module may be supported, at least in part, by another module. In addition, program modules that support the functionality described herein may form part of one or more applications executable across any number of systems or devices in accordance with any suitable computing model such as, for example, a client-server model, a peer-to-peer model, and so forth. In addition, any of the functionality described as being supported by any of the program modules depicted in FIG. 5 may be implemented, at least partially, in hardware and / or firmware across any number of devices.

[0046] It should further be appreciated that the computer system 710 may include alternate and / or additional hardware, software, or firmware components beyond those described or depicted without departing from the scope of the disclosure. More particularly, it should be appreciated that software, firmware, or hardware components depicted as forming part of the computer system 710 are merely illustrative and that some components may not be present or additional components may be provided in various embodiments. While various illustrative program modules have been depicted and described as software modules stored in system memory 730, it should be appreciated that functionality described as being supported by the program modules may be enabled by any combination of hardware, software, and / or firmware. It should further be appreciated that each of the above-mentioned modules may, in various embodiments, represent a logical partitioning of supported functionality. This logical partitioning is depicted for ease of explanation of the functionality and may not be representative of the structure of software, hardware, and / or firmware for implementing the functionality. Accordingly, it should be appreciated that functionality described as being provided by a particular module may, in various embodiments, be provided at least in part by one or more other modules. Further, one or more depicted modules may not be present in certain embodiments, while in other embodiments, additional modules not depicted may be present and may support at least a portion of the described functionality and / or additional functionality. Moreover, while certain modules may be depicted and described as sub-modulesof another module, in certain embodiments, such modules may be provided as independent modules or as sub-modules of other modules.

[0047] Although specific embodiments of the disclosure have been described, one of ordinary skill in the art will recognize that numerous other modifications and alternative embodiments are within the scope of the disclosure. For example, any of the functionality and / or processing capabilities described with respect to a particular device or component may be performed by any other device or component. Further, while various illustrative implementations and architectures have been described in accordance with embodiments of the disclosure, one of ordinary skill in the art will appreciate that numerous other modifications to the illustrative implementations and architectures described herein are also within the scope of this disclosure. In addition, it should be appreciated that any operation, element, component, data, or the like described herein as being based on another operation, element, component, data, or the like can be additionally based on one or more other operations, elements, components, data, or the like. Accordingly, the phrase “based on,” or variants thereof, should be interpreted as “based at least in part on.”

[0048] Although embodiments have been described in language specific to structural features and / or methodological acts, it is to be understood that the disclosure is not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as illustrative forms of implementing the embodiments. Conditional language, such as, among others, “can,” “could,” “might,” or “may,” unless specifically stated otherwise, or otherwise understood within the context as used, is generally intended to convey that certain embodiments could include, while other embodiments do not include, certain features, elements, and / or steps. Thus, such conditional language is not generally intended to imply that features, elements, and / or steps are in any way required for one or more embodiments or that one or more embodiments necessarily include logic for deciding, with or without user input or prompting, whether these features, elements, and / or steps are included or are to be performed in any particular embodiment.

Claims

CLAIMSWhat is claimed is:

1. A computer-implemented method, the method comprising: generating a model of a system defining a work environment and a plurality of agents capable of moving within the work environment according to respective dynamics of each agent; based on the model, generating a rapidly exploring random tree (RRT) for each agent of the plurality of agents, so as to define a plurality of trees, each RRT of the plurality of trees defining a set of trajectories of the respective agent within the work environment consistent with the respective dynamics, and each RRT defining multiple levels having a plurality of nodes at each of the multiple levels; and generating a database by combining nodes at each level, so as to define a product tree representative of the plurality of agents in the database.

2. The computer- implemented method as recited in claim 1, the method further comprising: receiving, at the database, a requirement related to a movement path of at least one of the plurality of agents; and responsive to the requirement, determining a plurality of scenarios defined by the database that satisfy the requirement.

3. The computer-implemented method as recited in claim 2, the method further comprising: extracting the plurality of scenarios from the database by performing a plurality of model checking operations.

4. The computer-implemented method as recited in claim 2, the method further comprising: receiving the requirement that is specified in discrete-time signal temporal logic (DT-STL); and extracting the plurality of scenarios based on the DT-STL.

5. The computer-implemented method as recited in claim 2, wherein the requirement stipulates two or more agents of the plurality of agents are in a specific location relative to a hazard within the environment, within a specific range of time.

6. A computing system, the computing system comprising: a processor; and a memory storing instructions that, when executed by the processor, cause the computing system to: generate a model of a system defining a work environment and a plurality of agents capable of moving within the work environment according to respective dynamics of each agent; based on the model, generate a rapidly exploring random tree (RRT) for each agent of the plurality of agents, so as to define a plurality of trees, each RRT of the plurality of trees defining a set of trajectories of the respective agent within the work environment consistent with the respective dynamics, and each RRT defining multiple levels having a plurality of nodes at each of the multiple levels; and generate a database by combining nodes at each level, so as to define a product tree representative of the plurality of agents in the database.

7. The computing system as recited in claim 6, the memory further storing instructions that, when executed by the processor, further cause the computing system to: receive, at the database, a requirement related to a movement path of at least one of the plurality of agents; and responsive to the requirement, determine a plurality of scenarios defined by the database that satisfy the requirement.

8. The computing system as recited in claim 7, the memory further storing instructions that, when executed by the processor, further cause the computing system to: extract the plurality of scenarios from the database by performing a plurality of model checking operations.

9. The computing system as recited in claim 7, the memory further storing instructions that, when executed by the processor, further cause the computing system to: receive the requirement that is specified in discrete-time signal temporal logic (DT- STL); and extract the plurality of scenarios based on the DT-STL.

10. The computing system as recited in claim 7, wherein the requirement stipulates two or more agents of the plurality of agents are in a specific location relative to a hazard within the environment, within a specific range of time.

11. A non-transitory computer-readable medium encoded with executable instructions that, when executed, cause one or more computer systems to perform a process as in any one claims 1 to 5.

Images