Generate planning hints for collision-free motion

Abstract

The present invention relates to generating planning cues for collision-free motion. A system can define planning cues that indicate where a robot can move or position. Planning cues can be used to generate collision-free paths for a robot.

Description

Background Art

[0001] Autonomous operations such as robotic gripping and placement in unknown or dynamic environments present various technical challenges. When developing automation applications, robots are often required to move within dynamic environments that may contain multiple machines or objects. In such environments, preprogramming robot motions may result in collisions or other unintended robot poses or paths. Furthermore, it is recognized in this article that current computer-aided design (CAD) tools for motion planning often result in inefficient or undesirable robot motions. Summary of the invention

[0002] Embodiments of the present invention address and overcome one or more of the disadvantages described herein by providing methods, systems, and apparatus for enhancing user interface tools for various autonomous systems. For example, the system can define planning cues that represent or indicate poses and positions within a work environment in which a robot can move.

[0003] In one example aspect, an automation computing system is capable of generating a model of an automation system, the automation system including a work environment and a robot configured to move within the work environment. The system is capable of locating a first planning hint for robot motion planning within the work environment. The planning hint defines a first posture of the robot that can be used when moving the robot from an initial posture to a target posture. The system is also capable of displaying a graphical representation of the robot positioned in the first posture. Using the planning hint, the system is capable of generating a path for the robot to move from the initial posture to the target posture without colliding with any object. In one example, the path includes the robot moving to a first posture between the initial posture and the target posture. In another example, the system is capable of locating multiple planning hints within the work environment. Each planning hint can define a corresponding posture and position of the robot within the work environment. Using multiple planning hints, the system is capable of generating multiple paths for the robot to move from the initial posture to the target posture. In one example, at least one of the multiple paths defines a collision-free path on which the robot avoids collision with any object. In some cases, at least one path includes at least one of a posture and a position defined by at least one planning hint in the planning hint. In another example, at least one path includes no more than two of a posture and a position defined by no more than two planning hints in the planning hint. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] The foregoing and other aspects of the present invention are best understood from the following detailed description when read in conjunction with the accompanying drawings. For the purpose of illustrating the present invention, presently preferred embodiments are shown in the accompanying drawings, however, it should be understood that the present invention is not limited to the specific means disclosed. The accompanying drawings include the following illustrations:

[0005] Figure 1 An example user interface (UI) is shown that defines two planning cues for a mobile robot that indicate locations within a working environment in accordance with an example embodiment.

[0006] Figure 2A Another example UI is shown that defines a new gesture element or six degrees of freedom according to an example embodiment.

[0007] Figure 2B A property editor of a UI for setting properties of a new gesture element is illustrated according to an example embodiment.

[0008] Figure 3A A UI is shown with a graphical representation of a robot moving to a position and pose corresponding to a pose element.

[0009] Figure 3B According to an example embodiment, Figure 2B The Property Editor where pose elements are assigned to planning cues.

[0010] Figure 4 A UI with a graphical representation of six degrees of freedom gesture elements is shown.

[0011] Figure 5A is a graph illustrating an example movement strategy for moving a robot from an initial pose to a target pose.

[0012] Figure 5B is a diagram illustrating an example movement strategy for moving a robot from an initial pose to a target pose via planning hint positions according to an example embodiment.

[0013] Figure 6 is a diagram illustrating an example movement strategy for moving a robot from an initial pose to a target pose via multiple planning hint positions according to an example embodiment.

[0014] Figure 7 A computing environment is illustrated within which embodiments of the present disclosure can be implemented. DETAILED DESCRIPTION

[0015] According to various embodiments, various interfaces for 3D editing are described herein. Such interfaces can be used as a front end for a robotic motion planning service in a robotic library for a programmable logic controller (PLC).

[0016] First, the goal of collision-free motion planning services is to generate a path that moves a robot from its current position to another target position without hitting another object along the way. Even if the target pose of the robot is known, there may still be various paths to reach the target pose. In order to determine the best path, in some cases, the search takes into account changes in the robot's joint limitations and overall configuration, which may pass through singular positions where the joint configuration is discontinuous. Therefore, it is recognized in this article that the movement of a given robot is often not obvious, and simple and efficient movements are often preferred. However, various potential paths may be blocked by objects that need to be avoided. It is also recognized in this article that technical challenges need to be addressed so that effective paths without collisions can be effectively searched and generated with minimal user interaction.

[0017] By way of background, it is recognized in this article that robot programming is typically explicit. For example, a programmer can use code to command a robot to move to various positions. The position of the robot can be specified using numerical values ​​and data structures representing joint positions and paths. Some robots are explicitly programmed using teach points, which represent recorded robot postures stored in the robot application. Such teach points can be referenced by issued commands. Postures can be generated by placing the robot in a specific posture using the robot's manual interface, or can be "demonstrated" by placing the robot's motors in a resistive state, manually grabbing the robot and physically moving it to the desired posture. Teach points can also be virtually demonstrated using a digital representation of the robot and placing the representation in the desired posture.

[0018] For a robot to operate autonomously, it is able to infer important properties of its environment and act accordingly. In some cases, the range of robot operation ranges from fully programmed and command-driven to fully autonomous and sensor-driven. In more autonomous scenarios, the path followed by the robot is often not predetermined. When the robot moves on or along a new path, it is important that the robot does not collide with unexpected objects, such as work surfaces, fixtures, other robots, etc. In order to avoid collisions, in some cases, the application needs to determine other physical objects that may interfere with the movement of the robot, and then calculate a path to avoid obstacles. According to various embodiments described herein, a path to avoid obstacles is calculated. In some examples, in order to find obstacles, the system uses a world model, in which the expected composition of a given work unit is prepared in advance and described in sufficient detail. Additionally or alternatively, the system can use sensors such as 3D cameras that can survey the area around the robot and update the dynamic world model when the application is in progress.

[0019] In order to calculate a path for a given pose of the robot, in various examples, it is determined whether various positions will result in collisions with unexpected obstacles. For example, the system can search a given workspace of all robot poses to find a collision-free path. There are a variety of algorithms for finding collisions between three-dimensional (3D) shapes. Typically, the shapes of the robot and obstacles are simplified to polyhedra with fewer facets, concave faces, or simplified geometry to make the calculation of shape intersections easier and more efficient.

[0020] However, it is recognized in this article that there are various technical challenges when searching the robot's posture space. For example, the dimensionality of the robot's posture may make a simple search strategy impractical. In addition, it is also undesirable to create paths that are too complex and / or produce undesirable movements, although feasible. Current methods tend to be based on random values, such as probabilistic roadmaps (PRMs) and rapidly exploring random trees (RRT) algorithms. Both algorithms create random robot postures to create a search space of possible paths. Without filtering the results, it is recognized in this article that both methods tend to create unique robot motions that tend to move away from the target as much as towards the target. Even in the case of filtering, it is also recognized in this article that these motions still tend to contain undesirable artifacts, such as extra motion on unnecessary joints.

[0021] It is recognized herein that a goal of various industrial processes is to move materials along unnecessarily complex paths by robots. However, it is also recognized herein that there may be various technical challenges for a robot to automatically find an uncomplicated path. For example, in some cases, the elements processed by the robot appear in hundreds of locations, where the changing paths and destinations can only be determined at processing time. Conversely, for example, if the robot only serves one path, even if the path is complex, the path can be programmed once, and the robot can then repeat the path. However, many industrial tasks are unknown before the operation is performed, such as picking scattered objects from a bin or loading a cargo hold.

[0022] Generating paths in the presence of obstacles can be challenging because, while there may be many possible simple paths, it can be difficult to find a strategy that works for any possible situation. For example, in a pick and place scenario, a strategy that moves the robot's end effector on an arc path up to the open space above the material being picked up to travel sideways may work well. But in another example scenario where an object is placed vertically rather than horizontally, the same arc motion may fail. In such scenarios, general path finding schemes such as RRT can treat the topology of the site as a puzzle, and it is recognized in this article that the resulting solutions tend to be overly complex.

[0023] Therefore, the embodiments described herein can define a balance between general path finding solutions and user guidance instructions in some cases. For example, an application developer can provide one or more planning hints to indicate key positions that a robot may move to in a given workspace. Such key positions can define spaces that are generally open to allow further movement and away from obstacles that may block the robot. In various examples, planning hints do not define waypoints, because the robot may not be commanded to move to a position or position corresponding to a planning hint at any time. However, in some examples, when it is calculated that other options are not available, the robot can automatically choose to move to the planning hint position. In addition, the path generation algorithm can use specific planning hints, or a given planning hint may be ignored by the path generation operation, for example, depending on the current robot position and destination. In one example, developers can distribute planning hint sets in a star shape around the application work area at key positions. In particular, for example, hints can be placed in bottleneck areas where there are more obstacles in the space or in areas where the robot needs to be transferred to a specific configuration to move through.

[0024] Reference now Figure 1 , the computing automation system can define various user interfaces (UIs), such as a UI 100 that allows a user (e.g., an application developer) to generate a planning hint object (e.g., a first planning hint 102 and a second planning hint 104). In one example, the planning hint object is generated as an element of a three-dimensional (3D) modeling editor. The 3D editor can be used to assemble a model of a work cell or environment for a given robotic application. When calculating collision-free motion, the model can define a knowledge base of collision surface locations. The 3D editor can also serve as a testing rig for verifying whether the motion of the robot when moving between various positions will act as expected.

[0025] In various examples, an application developer (user) is able to create a model of a given work unit by adding components such as, for example, robots, fixtures, work materials, robot accessories (e.g., wire harnesses, grippers, cameras, etc.). The user is also able to introduce semantic components (e.g., semantic tags) into the model, which represent various instructions such as robot position, how to grab materials, where to place materials, etc. Semantic tags can represent various behaviors in various automation systems or applications. For example, a computing automation system can define a low-code programming environment that can be used to specify an automation application using a 3D graphical model of the work materials managed or modified by the components of the automation application and the automation application. In particular, for example, a 3D graphical analog of a component can include a six-degree-of-freedom (DOF) posture component (or posture element). The posture element can represent the joint configuration of a particular robot reaching to a given position.

[0026] Also refer to Figure 2A , the example UI 200 can define a new pose element or six degrees of freedom component 202. For example, the UI 100 can define a palette that presents labeled objects or elements, such as pose elements 202. In one example, a user can drag an instance of the element 202 from the palette to a location in the 3D model space 204 to add the new pose element 202 to the model. The new pose element 202 can begin in an unassigned state, in which it is not yet attached to the robot 206. In various examples, the new pose element 202 can define a graphical appearance (e.g., color, shape, etc.) that indicates that the pose element 202 is unassigned. Also refer to Figure 2B , a user can assign a gesture element 202 to a robot, such as robot 206, using a property editor 208 defined by a UI. When a gesture element 202 is assigned to a robot 206, the gesture element 202 can graphically represent how the robot 206 reaches the location of the element 202. If the robot 206 cannot reach the location, the graphical representation can indicate this. By way of example and not limitation, the graphical representation can define a different color, icon, or other user interface technique to indicate that the robot 206 cannot reach the location defined by the gesture element 202.

[0027] Special References Figure 2B , the pose element 202 can be associated with various other parameters 210 that can be included in the property editor 208. In various examples, a user can enter values ​​associated with the parameters 210 to adjust the pose of the robot (e.g., angles or linear joint values ​​of the robot axes), the configuration for which the orientation joint values ​​should be calculated, or the tool center point (TCP) indication. The TCP indication can define the location of the point on the robot 206 (or a component attached to the robot 206) used as the point for calculating the pose. For example, referring to Figure 3A , the robot 206 can define a flange 212 with a flange TCP. When the TCP indication corresponds to the flange 212, the system can present a graphical representation 300 of placing the flange TCP at the position defined by the pose element 202. In various examples, the original joint positions of the robot object are not changed. In addition, in some cases, the position of the robot can be changed to match the position of the pose element 202. Thus, in various examples, a user can create multiple pose elements for a given robot and quickly and easily switch between the multiple pose elements.

[0028] The six degrees of freedom posture component 202 can be used for different purposes. In one example, the six degrees of freedom posture component 202 can define waypoints in traditional robot programming. In another example, the six degrees of freedom posture component 202 can illustrate or record the axis positions of the actual connected robot. According to various embodiments described herein, the six degrees of freedom component or posture element, in particular the posture element 202, defines a planning hint. For example, also refer to Figure 3B , a user or application developer can designate a gesture element 202 as a planning hint by setting a planner hint parameter or attribute 302 of parameter 210 to “true”.

[0029] Reference now Figure 4 , the posture element 202 is set as a planning prompt 402 via the property editor 210. Compared with the posture element that is not a planning prompt, the posture element 202 set as the planning prompt 402 can define an alternative graphical appearance. For example, but not limitation, the planning prompt 402 can be illustrated with a colored circular mark at the center of the crosshairs defined by the posture element 202. It should be understood that the planning prompt can be presented on the UI alternatively so that they are visually distinguishable from other posture elements that are not set as planning prompts, and all such alternative presentations are considered to be within the scope of the present disclosure. In addition, although a posture element 202 is illustrated as a planning prompt 402, it should be understood that the user can assign multiple posture elements as planning prompts spaced apart from each other. One or more planning prompts can provide the system with instructions about where the robot 206 can move to reach a given destination. Therefore, in various examples, the planning prompt does not define the entire path, but defines one or more locations that are desired or can be included in the path generated by the system.

[0030] In one example, after setting a planning hint (e.g., planning hint 402), the system can use the planning hint to generate a collision-free path. In some examples, a command language is used to create instructions that move the robot 206 in different ways to generate a path. In some cases, the instructions can represent a set of primitive moves, such as incrementing the value of an axis joint in two-degree steps, or translating the position of the robot's TCP along the X-axis or first direction. The commands can incorporate variables that encode key information about the desired movement, such as the current robot state, the target robot state, various reference frames (e.g., the global frame), and other variables that access application data, such as, but not limited to, the positions of obstacles, other robots, and user-provided markers (e.g., planning hints).

[0031] Reference now Figure 5A , the computing automation system can generate various paths from an initial position or pose 502 defined by the robot 206 to an intended target destination or pose 504 of the robot 206. The system can use command primitives to determine one or more paths for moving the robot from the initial pose 502 to the target pose 504. The system can determine the paths based on various movement strategies, such as a first movement strategy 506a, a second movement strategy 506b, and a third strategy 506c. Although Figure 5A , it should be understood that the system can select alternative or additional movement strategies as needed, and all such movement strategies are considered to be within the scope of the present disclosure. For example, a first movement strategy 506a can include the robot 206 moving its TCP in Cartesian coordinates. A second movement strategy 506b can include the robot 206 moving in small increments at a time between the initial pose 502 and the target pose 504. A third movement strategy 506c can include the robot 206 moving in its joint space. The movement strategies 506a-506c can specify corresponding movements between any two robot poses, such as the movement between the initial pose 502 and the target pose 504. Reference Figure 5A, the lines at 506a-506c between the initial pose 502 and the target pose 504 can represent the corresponding strategy for searching for a collision-free path, and therefore do not represent the physical movement that the robot 206 may actually take. The actual movement may be curved or straight, depending on the commands taken by the strategy. When the robot can move to the target position 504 without colliding with other objects, a valid path is defined. One, more than one, or none of the strategies 506a-506c can produce a collision-free and valid path. When the strategies 506a-506c produce valid multiple paths, the system can select one of the valid paths based on various criteria. For example, the first path found can be selected, or the path with the least joint movement can be selected. Figure 5A 5 shows example paths generated using strategies 506a - 506c that do not include any planning hints.

[0032] refer to Figure 5B , the user can provide a planning hint 508 that can define an intermediate point along the path between the initial pose 502 and the target pose 504. Thus, for a pose-to-pose transition between the initial pose 502 and the target pose 504, the system can generate one or more paths or movement selections that include or exclude the location of the planning hint 508. For each movement path, such as the first path 512, the second path 514, and the third path 516, a plurality of strategies can be employed, such as Cartesian, incremental movement, and joint movement strategies 506a-506c. Based on the planning hint, the system can generate a search command that incorporates the planning hint location as part of the search path. If there is a single planning hint pose, such as Figure 5B If the planning hint 508 is provided, the system can use strategies 506a-506c to generate a path based on a first path 512, in which the robot travels directly from the initial pose 502 to the target pose 504. Additionally, the system can use any of the strategies 506a-506c to generate another path including a second path 514 and a third path 516, in which the robot travels from the initial pose 502 to the pose defined by the planning hint 508, and then also using strategies 506a-506c, travels from the planning hint pose to the target pose 504. Thus, multiple strategies can be implemented for searching for collision-free paths, and these strategies can be used in Figure 5B merged to form a corresponding single dashed line. For the portion of the path traveled, the effective strategy is the strategy that produces collision-free motion. In various examples, any one of the effective strategies in the multi-strategy line can make Figure 5B504. The entire line in becomes valid. To be a completely valid path, there is a valid strategy on the continuous line from the initial posture 502 to the target posture 504. Therefore, the valid path can be defined by the selected strategy for 512 as moving directly from the initial posture 502 to the target posture 504, or by the strategy definition independently selected for 514 and 516, where the robot moves from the initial posture 502 to the target posture 504 via the planning prompt posture 508. In some cases, the search strategy for finding a complete valid path can include performing a depth-first search, which starts with the path with the least number of intermediate postures. Each strategy in each path segment can be tested, and if any strategy is valid, the system can start checking the path in the next connection strategy set.

[0033] Reference now Figure 6 , when there are multiple planning hints, such as the first planning hint 602, the second planning hint 604, and the third planning hint 606, the system can limit the number of search paths by limiting the number of hints that can be used as intermediate steps. In one example, the number of steps can be limited to two, such as the first step 608 and the second step 610, but it should be understood that the system can be limited to an alternative number of steps, such as the user can request that additional steps be included, and all such numbers of steps are considered to be within the scope of the present disclosure. In some cases, using more than two planning hints as steps may cause the hint positions to be executed as waypoints because the system may search for different sequences of possible pose positions defined by the planning hints.

[0034] Still refer to Figure 6 , when there are two or more planning cues (e.g., planning cues 602, 604, and 606), the system is able to generate a search pattern using up to two-step paths (e.g., steps 608 and 610). For example, the system is able to generate a path that defines a direct move between an initial position and a target position 504, as well as a path that defines a direct move from the first planning cue step 608 to the target position 504, and other paths. In various examples, the system omits a strategy for traveling to itself from a given planning cue because the robot is already at that location at the given planning cue. Similarly, in systems that define more than two steps, paths that backtrack to previously visited cues may also be excluded for the same reasons. Therefore, reference is still made to Figure 6, the system may search for a valid path from the initial pose 502 to the goal pose 504 with no hints, one intermediate hint (e.g., planning hint 602, 604, or 606), or two intermediate hints. In some examples, the system can prioritize paths with the fewest number of steps or the least overall robot motion. The system can also perform further refinement on the paths, such as simplifying the motion by removing sharp turns or zigzag moves. In various examples, after path selection and refinement, the final motion path is checked to verify that it is collision-free. Alternatively, for example, if the search was Figure 6 If the entire graph is not shown, and there is no valid path from the initial pose 502 to the goal pose 504, the system can present an indication to the user that no path was found. For error handling, in some cases the system can indicate a valid search path to indicate how far the system can move the robot before no collision-free path is found.

[0035] Thus, as described herein, planning hints or positions that do not define waypoints can be provided. For example, the actual path used by the robot may use the planned positions in its path, or may rely on other positions that are not defined by the planning hints. The systems described herein can incorporate planning hint positions non-deterministically, and in some cases, apply planning hints only when necessary. For example, when a direct path between two locations is blocked, the robot can choose to use an intermediate pose corresponding to a planning hint. The system can also optimize movement by choosing between multiple intermediate points and choosing the path that maximizes some criteria (such as, for example, the least joint movement).

[0036] As further described herein, a computing automation system is capable of generating a model of an automation system including a work environment and a robot configured to move within the work environment, the automation system. The system is capable of locating a first planning hint for robot motion planning within the work environment. The planning hint can define a first posture of the robot that can be used when moving the robot from an initial posture to a target posture. The system is also capable of displaying a graphical representation of the robot positioned in the first posture. Using the planning hint, the system is capable of generating a path for the robot to move from the initial posture to the target posture without colliding with any object. In one example, the path includes the robot moving to a first posture between the initial posture and the target posture. In another example, the system is capable of locating multiple planning hints within the work environment. Each planning hint can define a corresponding posture and position of the robot within the work environment. Using multiple planning hints, the system is capable of generating multiple paths for the robot to move from the initial posture to the target posture. In one example, at least one of the multiple paths defines a collision-free path on which the robot avoids collision with any object. In some cases, at least one path includes at least one of a posture and a position defined by at least one planning hint in the planning hint. In another example, at least one path includes no more than two of a posture and a position defined by no more than two planning hints in the planning hint.

[0037] Figure 7 An example of a computing environment that can include a simulation system within which embodiments of the present disclosure can be implemented is illustrated. Computing environment 700 includes a computer system 710 that can include a communication mechanism such as a system bus 721 or other communication mechanism for transmitting information within the computer system 710. The computer system 710 also includes one or more processors 720 coupled to the system bus 721 for processing information.

[0038] Processor 720 can include one or more central processing units (CPUs), graphics processing units (GPUs), or any other processors known in the art. More generally, a processor as described herein is a device for executing machine-readable instructions stored on a computer-readable medium, for performing tasks, and can include any one or a combination of hardware and firmware. The processor can also include a memory that stores machine-readable instructions that can be executed for performing tasks. The processor takes action based on information by manipulating, analyzing, modifying, converting, or transmitting information for use by an executable program or information device and / or by routing the information to an output device. The processor can use or include the capabilities of, for example, a computer, a controller, or a microprocessor, and uses executable instructions to adjust to perform special functions that cannot be performed by a general-purpose computer. The processor can include any type of suitable processing unit, including but not limited to a central processing unit, a microprocessor, a reduced instruction set computer (RISC) microprocessor, a complex instruction set computer (CISC) microprocessor, a microcontroller, an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a system on a chip (SoC), a digital signal processor (DSP), and the like. In addition, the processor(s) 720 can have any suitable micro-architecture design including any number of components, such as, for example, registers, multiplexers, arithmetic logic units, a cache controller for controlling read / write operations to cache memory, a branch predictor, etc. The micro-architecture design of the processor can support any of a variety of instruction sets. The processor can be coupled (electrically coupled and / or include executable components) to any other processor to enable interaction and / or communication therebetween. A user interface processor or generator is a known element that includes electronic circuitry or software or a combination of both for generating a display image or a portion thereof. The user interface includes one or more display images, thereby enabling a user to interact with the processor or other device.

[0039] The system bus 721 can include at least one of a system bus, a memory bus, an address bus, or a message bus, and can permit information (e.g., data (including computer executable code), signaling, etc.) to be exchanged between various components of the computer system 710. The system bus 721 can include, but is not limited to, a memory bus or a memory controller, a peripheral bus, an accelerated graphics port, and the like. The system bus 721 can be associated with any suitable bus architecture, including, but not limited to, an Industry Standard Architecture (ISA), a Micro Channel Architecture (MCA), an Enhanced ISA (EISA), a Video Electronics Standards Association (VESA) architecture, an Accelerated Graphics Port (AGP) architecture, a Peripheral Component Interconnect (PCI) architecture, a PCI-Express architecture, a Personal Computer Memory Card International Association (PCMCIA) architecture, a Universal Serial Bus (USB) architecture, and the like.

[0040] Continue to refer Figure 7 , the computer system 710 can also include a system memory 730, which is coupled to the system bus 721 for storing information and instructions to be executed by the processor 720. The system memory 730 can include computer-readable storage media in the form of volatile and / or non-volatile memory, such as read-only memory (ROM) 731 and / or random access memory (RAM) 732. RAM 732 can include (one or more) other dynamic storage devices (e.g., dynamic RAM, static RAM, and synchronous DRAM). ROM 731 can include (one or more) other static storage devices (e.g., programmable ROM, erasable PROM, and electrically erasable PROM). In addition, the system memory 730 can be used to store temporary variables or other intermediate information during the execution of instructions by the processor 720. The basic input / output system 433 (BIOS) contains basic routines that help transfer information between elements within the computer system 710, such as at startup, and can be stored in ROM 731. RAM 732 can contain data and / or program modules that are immediately accessible to and / or currently being operated on by processor 720. System memory 730 can additionally include, for example, an operating system 734, application programs 735, and other program modules 736. Application programs 735 can also include a user portal for developing applications, allowing input parameters to be entered and modified as desired.

[0041] The operating system 734 can be loaded into the memory 730 and can provide an interface between other application software executed on the computer system 710 and the computer system 710 and hardware resources. More specifically, the operating system 734 can include a set of computer executable instructions for managing the hardware resources of the computer system 710 and for providing general services to other applications (e.g., managing memory allocation between various application programs). In certain example embodiments, the operating system 734 can control the execution of one or more program modules depicted as being stored in the data storage 740. The operating system 734 can include any operating system now known or that may be developed in the future, including but not limited to any server operating system, any mainframe operating system, or any other proprietary or non-proprietary operating system.

[0042] The computer system 710 also includes a disk / media controller 743 coupled to the system bus 721 to control one or more storage devices for storing information and instructions, such as a magnetic hard disk 741 and / or a removable media drive 742 (e.g., a floppy disk drive, a compact disk drive, a tape drive, a flash drive, and / or a solid-state drive). The storage device 740 can be added to the computer system 710 using an appropriate device interface (e.g., a small computer system interface (SCSI), an integrated device electronics (IDE), a universal serial bus (USB), or a firewire). The storage devices 741, 742 can be external to the computer system 710.

[0043] The computer system 710 can perform some or all of the processing steps of an embodiment of the present invention in response to the processor 720 executing one or more sequences of one or more instructions contained in a memory such as a system memory 730. Such instructions can be read into the system memory 730 from another computer-readable medium (such as a magnetic hard disk 741 or a removable media drive 742) of the storage 740. The magnetic hard disk 741 (or solid-state drive) and / or the removable media drive 742 can contain one or more data storage areas and data files used by embodiments of the present disclosure. The data storage area 740 can include, but is not limited to, a database (e.g., relational, object-oriented, etc.), a file system, a flat file, a distributed data storage area in which data is stored on more than one node of a computer network, a peer-to-peer network data storage area, etc. The data storage area can store various types of data, such as, for example, skill data, sensor data, or any other data generated according to embodiments of the present disclosure. The data storage area content and data files can be encrypted to increase security. The processor 720 can also be employed in a multi-processing arrangement to execute one or more sequences of instructions contained in the system memory 730. In alternative embodiments, hard-wired circuitry can be used in place of or in combination with software instructions.Thus, embodiments are not limited to any specific combination of hardware circuitry and software.

[0044] As described above, the computer system 710 can include at least one computer-readable medium or memory for storing instructions programmed according to embodiments of the present invention and for containing data structures, tables, records, or other data described herein. As used herein, the term "computer-readable medium" refers to any medium that participates in providing instructions to the processor 720 for execution. Computer-readable media can take many forms, including but not limited to non-temporary media, non-volatile media, volatile media, and transmission media. Non-limiting examples of non-volatile media include optical disks, solid-state drives, magnetic disks, and magneto-optical disks, such as magnetic hard disks 741 or removable media drives 742. Non-limiting examples of volatile media include dynamic memory, such as system memory 730. Non-limiting examples of transmission media include coaxial cables, copper wires, and optical fibers, including wires that constitute the system bus 721. Transmission media can also take the form of sound waves or light waves, such as those generated during radio wave and infrared data communications.

[0045] The computer-readable medium instructions for implementing the operation of the present disclosure can be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​(such as Smalltalk, C++, etc.) and conventional procedural programming languages ​​(such as "C" programming language or similar programming languages). The computer-readable program instructions can be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or completely on a remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network (including a local area network (LAN) or a wide area network (WAN)), or a connection to an external computer can be made (for example, through the Internet using an Internet service provider). In some embodiments, electronic circuits including, for example, programmable logic circuits, field programmable gate arrays (FPGAs), or programmable logic arrays (PLAs) can execute computer-readable program instructions by utilizing the state information of the computer-readable program instructions to personalize the electronic circuits so as to perform various aspects of the present disclosure.

[0046] Various aspects of the present disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present disclosure. It should be understood that each block of the flowchart illustration and / or block diagram and the combination of blocks in the flowchart illustration and / or block diagram can be implemented by computer-readable medium instructions.

[0047] The computing environment 700 can also include a computer system 710 that operates using a logical connection to one or more remote computers (such as a remote computing device 780) in a networked environment. The network interface 770 can enable, for example, communication with other remote devices 780 or systems and / or storage devices 741, 742 via a network 771. The remote computing device 780 can be a personal computer (laptop or desktop), a mobile device, a server, a router, a network PC, a peer device, or other common network node, and typically includes many or all of the elements described above relative to the computer system 710. When used in a networked environment, the computer system 710 can include a modem 772 for establishing communication on a network 771 (such as the Internet). The modem 772 can be connected to the system bus 721 via the user network interface 770 or via another appropriate mechanism.

[0048] The network 771 can be any network or system generally known in the art, including the Internet, an intranet, a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a direct connection or a series of connections, a cellular telephone network, or any other network or medium that can facilitate the communication between the computer system 710 and other computers (e.g., a remote computing device 780). The network 771 can be wired, wireless, or a combination thereof. A wired connection can be implemented using Ethernet, a universal serial bus (USB), RJ-6, or any other wired connection generally known in the art. A wireless connection can be implemented using Wi-Fi, WiMAX, and Bluetooth, infrared, a cellular network, a satellite, or any other wireless connection method generally known in the art. Additionally, several networks can work alone or communicate with each other to facilitate communication in the network 771.

[0049] It should be understood that Figure 7 The program modules, applications, computer executable instructions, codes, etc. depicted as being stored in system memory 730 are merely illustrative and non-exhaustive, and the processing described as being supported by any particular module can alternatively be distributed across multiple modules or performed by different modules. In addition, various (one or more) program modules, (one or more) scripts, (one or more) plug-ins, (one or more) application programming interfaces (APIs), or any other suitable computer executable code can be provided, which are hosted locally on the computer system 710, the remote device 780, and / or hosted on other (one or more) computing devices accessible via one or more of the (one or more) networks 771, to support the processing by Figure 7 The functionality and / or additional or alternative functionality provided by the program modules, applications or computer executable code depicted in the description. In addition, the functionality can be modularized differently so that the functionality described as being comprised of Figure 7The processing collectively supported by the collection of program modules depicted in the description can be performed by a fewer or greater number of modules, or the functionality described as supported by any particular module can be supported at least in part by another module. In addition, the program modules supporting the functionality described herein can form part of one or more applications that are executable on any number of systems or devices according to any suitable computing model (such as, for example, a client-server model, a peer-to-peer model, etc.). In addition, the processing collectively supported by the collection of program modules depicted in the description can be performed by a fewer or greater number of modules, or the functionality described as supported by any particular module can be supported at least in part by another module. In addition, the program modules supporting the functionality described herein can form part of one or more applications that are executable on any number of systems or devices according to any suitable computing model (such as, for example, a client-server model, a peer-to-peer model, etc.). Figure 7 Any of the functionality supported by any of the program modules depicted in the can be implemented at least partially in hardware and / or firmware on any number of devices.

[0050] It should also be understood that, without departing from the scope of the present disclosure, the computer system 710 can include replacement and / or additional hardware, software or firmware components beyond the hardware, software or firmware components described or depicted. More particularly, it should be understood that the software, firmware or hardware components depicted as forming a part of the computer system 710 are merely illustrative, and some components may not exist or additional components can be provided in various embodiments. Although various illustrative program modules have been depicted and described as software modules stored in the system memory 730, it should be understood that the functionality described as supported by the program modules can be implemented by any combination of hardware, software and / or firmware. It should be further understood that each of the modules mentioned above can represent a logical partition of the supported functionality in various embodiments. For ease of explanation of functionality, the logical partition is depicted, and it does not represent the structure of software, hardware and / or firmware for implementing functionality. Therefore, it should be understood that the functionality described as provided by a specific module can be provided at least in part by one or more other modules in various embodiments. Furthermore, one or more of the depicted modules may not be present in certain embodiments, while in other embodiments, additional modules not depicted may be present and capable of supporting at least some of the described functionality and / or additional functionality. Furthermore, while certain modules can be depicted and described as submodules of another module, in certain embodiments, such modules can be provided as standalone modules or as submodules of other modules.

[0051] Although the specific embodiments of the present disclosure have been described, it will be appreciated by those of ordinary skill in the art that many other modifications and alternative embodiments are within the scope of the present disclosure. For example, any functionality and / or processing capability described with respect to a particular device or component can be performed by any other device or component. In addition, although various illustrative implementations and architectures have been described according to the embodiments of the present disclosure, it will be appreciated by those of ordinary skill in the art that many other modifications to the illustrative implementations and architectures described herein are also within the scope of the present disclosure. In addition, it should be understood that any operation, element, component, data, etc. described herein as being based on another operation, element, component, data, etc. can be additionally based on one or more other operations, elements, components, data, etc. Therefore, the phrase "based on" or variations thereof should be interpreted as "based at least in part on".

[0052] Although embodiments have been described with language specific to structural features and / or method actions, it should be understood that the disclosure is not necessarily limited to the specific features or actions described. Rather, specific features and actions are disclosed as illustrative forms for implementing embodiments. Conditional language, such as "can", "may", "perhaps" or "likely" and others, unless otherwise specifically stated, or otherwise understood in the context of use, is generally intended to convey that certain embodiments can include certain features, elements and / or steps, while other embodiments do not include. Therefore, such conditional language is generally not intended to mean that one or more embodiments require features, elements and / or steps in any way, or one or more embodiments must include logic for determining whether these features, elements and / or steps are included in any particular embodiment or will be executed with or without user input or prompts.

Claims

1. A computer-implemented method, the method comprising: generating a model of an automated system comprising a work environment and a robot configured to move within the work environment; positioning a first planning cue within the work environment, the planning cue defining a first pose of a robot, the first pose being operable to move the robot from an initial pose to a target pose; as well as A graphical representation of the robot positioned in the first pose is displayed.

2. The method according to claim 1, further comprising: Using the planning hint, a path is generated for the robot to move from the initial pose to the target pose without colliding with any object.

3. The method according to claim 2, wherein: The path includes the robot moving to the first pose between the initial pose and the target pose.

4. The method of claim 1, locating a plurality of planning cues within the work environment, each planning cue defining a respective pose and position of the robot within the work environment.

5. The method according to claim 4, further comprising: Using the plurality of planning cues, a plurality of paths are generated for the robot to move from the initial pose to the target pose.

6. The method according to claim 5, wherein: At least one path of the plurality of paths defines a collision-free path in which the robot avoids collision with any object.

7. The method according to claim 6, wherein: The at least one path includes at least one of a pose and a position defined by at least one of the planning hints.

8. A computing automation system, the computing automation system comprising: processor; as well as a memory storing instructions that, when executed by the processor, cause the computing system to: generating a model of an automated system including a work environment and a robot configured to move within the work environment; positioning a first planning cue within the work environment, the planning cue defining a first pose of a robot, the first pose being operable to move the robot from an initial pose to a target pose; as well as A graphical representation of the robot positioned in the first pose is displayed.

9. The system of claim 8, wherein the memory further stores instructions that, when executed by the processor, further cause the computing system to: The planning cues are used to generate a path for the robot to move from the initial pose to the target pose without colliding with any objects.

10. The system according to claim 9, wherein: The path includes the robot moving to the first pose between the initial pose and the target pose.

Images