Digital representation of a robot operating environment useful in motion planning of a robot

Abstract

filtering the super-large representation of at least a portion of the robot (e.g., voxels are set to unoccupied for any objects that are entirely within the super-large representation) from the representation of the operating environment, which provides a digital model of the operating environment that can be used, for example, for motion planning of the robot. The super-large representation exceeds the physical dimensions of at least a portion of the robot (e.g., an attachment) to advantageously account for cables and other features that are attached and extend beyond the external dimensions of the robot. The specific dimensions of the super-large representation can be based on a variety of factors, such as the geometry of the cable that can be modeled, the orientation or position of the robot attachment, the orientation or position of the cable relative to the robot attachment, the speed of the attachment, the slack of the cable, etc.

Description

Technical Field

[0001] This disclosure generally relates to a digital representation of an operating environment in which one or more robots operate, and to robot motion planning employing a digital representation of the operating environment, such as systems and methods for performing collision detection using a digital representation generated from perception data collected from sensors to generate motion plans for driving robots, etc. Background Technology

[0002] Description of related technologies

[0003] Motion planning is a fundamental problem in robot control and robotics. A motion plan precisely specifies the path a robot can follow from an initial state to a target state, typically in a three-dimensional operating environment, without colliding with any obstacles in the operating environment, or with a reduced probability of collision. Challenges in motion planning include the ability to execute motion plans quickly, even as the characteristics of the 3D operating environment change. For example, characteristics such as the position or shape of one or more obstacles in a 3D operating environment may change over time.

[0004] Typically, one or more sensors capture information about a three-dimensional operating environment in which one or more robots can operate. For example, the three-dimensional operating environment can take the form of work cells in which one or more robots operate. For instance, each robot may have a corresponding movable robotic attachment with an end effector or end-of-arm tool and may interact with one or more workpieces. The captured sensor information is used to generate a digital representation or model of the three-dimensional operating environment, where various parts of the three-dimensional environment are represented as unoccupied or occupied by one or more objects that may be located within the three-dimensional operating environment. Objects may take the form of obstacles to be avoided or targets that the robot wants to interact with. The digital representation can be used to perform motion planning to generate motion plans for driving the robot while avoiding collisions with various obstacles in the three-dimensional operating environment. Summary of the Invention

[0005] Robots can take many forms and typically include a base, attachments, and an end effector or arm-end tool located at the distal end of the attachment. The base can be fixed or movable. The attachment is movable relative to the base and can include one or more links coupled via one or more joints, where various actuators (e.g., electric motors, stepper motors, solenoids, electromagnets, pistons, and cylinders with associated valves and pressurized fluid reservoirs) are coupled and operated to drive the links to rotate about the joints. The end effector can take any of a variety of forms, such as a gripper, a pair of opposing fingers, a rotary drill bit, a screwdriver or bolt driver, a welding head, a sensor, etc.

[0006] Often, structures may extend outward from a part of the robot. For example, the robot may be physically coupled to one or more cables, or one or more cables may be attached to various parts of the robot. For example, one or more cables may extend between the robot's base and attachments. Additionally or alternatively, one or more cables may extend between the links of an attachment or between the attachment and an end effector. Cables can take many forms. For example, one or more cables may be used to supply power or pressurized fluid (e.g., hydraulic, pneumatic) to one or more actuators. As another example, one or more cables may be used to route communication signals, such as those from one or more sensors mounted on the robot (e.g., cameras, position or rotary encoders, proximity sensors). Cables may be attached to the robot at various points or locations along the robot, such as at several points along an attachment or along the links of an attachment, typically extending outward relative to the edges of the attachment, links, or other parts of the robot. In some cases, one or more portions of the cable may droop, sink, or hang from a part of the robot at at least some locations and orientations. In some cases, one or more sections of a cable can change their relative position or orientation with respect to that section of the robot as a part of the robot moves, for example, when inertial forces act on the cable or a section thereof. Other structures (e.g., sensors, triaxial accelerometers) can also be attached to one or more sections of the robot, extending outward relative to the edges of attachments, links, or other parts of the robot.

[0007] When generating a digital representation of a 3D operating environment based on sensor or perception data, it may be advantageous to "filter" the robot itself out of the digital representation to prevent a given robot from hindering itself during motion planning. Therefore, it may be advantageous to display a portion of the 3D operating environment occupied by the robot as unoccupied in the digital representation of the 3D operating environment.

[0008] For example, the volume occupied by a robot can be specified by a digital representation or model of the robot, taking into account the position of one or more joints of the robot at any given time. This digital representation or model can represent the external dimensions of the robot very accurately, including attachments and end effectors. However, this digital representation or model fails to account for the various structures that may be attached to different parts of the robot, such as one or more cables attached to one or more parts of the robot and moving with one or more parts of the robot.

[0009] As described herein, an oversized or “expanded” representation of at least a portion of a robot is filtered from a representation of a three-dimensional operating environment. The resulting “filtered” representation provides a digital model of the three-dimensional operating environment, which can be used, for example, for motion planning of a given robot. The oversized representation exceeds one or more physical dimensions of at least a portion of the robot (e.g., attachments) to advantageously account for cables and other features attached to the robot and potentially extending beyond the robot’s external dimensions. The specific dimensions of the oversized representation can be based on a variety of factors, such as cable geometry, the orientation or position of robot attachments, the orientation or position of cables relative to robot attachments, the velocity of attachments, and the slack, droop, or tension of cables. For example, filtering a robot from the representation may include setting the occupancy value of one or more voxels to unoccupied for any object that is entirely within the oversized representation of the robot or a portion thereof. In this way, each robot in the three-dimensional operating environment and structures attached to it (e.g., cables) are filtered out from the digital representation of the three-dimensional operating environment, which can be used for motion planning of the same robot. Other robots in the three-dimensional operating environment, if any, constitute obstacles during the motion planning of a given robot and are therefore not filtered out from the digital representation used for motion planning of the given robot.

[0010] Therefore, an oversized or "extended" representation of at least a portion of the robot can be determined, which is selected or generated to be large enough to encompass the robot and related structures (e.g., cables) extending from or from the robot or its parts. For example, the amount of oversized representation can be based on a set of heuristics that can be determined during pre-run modeling.

[0011] In practice, any object representation that is entirely within the region corresponding to the oversized or "expanded" representation can be indicated as unoccupied, for example, by changing the occupancy value of the relevant voxel from occupied to unoccupied. Any object representation that is entirely outside or crosses the region can be indicated as occupied, for example, by leaving the occupancy value of the relevant voxel as occupied. Attached Figure Description

[0012] In the accompanying drawings, the same reference numerals identify similar elements or actions. The size and relative position of the elements in the drawings are not necessarily drawn to scale. For example, the shapes and angles of various elements are not drawn to scale, and some of these elements are arbitrarily enlarged and positioned to improve the readability of the drawing. Furthermore, the specific shapes of the depicted elements are not intended to convey any information about the actual shape of the particular element, and are chosen solely for ease of identification in the accompanying drawings.

[0013] Figure 1 It is a schematic diagram of an operating environment in which one or more robots operate, including an environment modeling system with various sensors and an environment modeling computer system, as well as an optional motion planner and an optional robot control system, according to a schematic implementation.

[0014] Figure 2 It is implemented according to a diagram. Figure 1 Functional block diagram of the environment modeling system.

[0015] Figure 3 This illustrates an implementation method based on a diagram. Figure 1 A flowchart of advanced operation methods for an environment modeling system.

[0016] Figure 4 This illustrates an implementation method based on a diagram. Figure 1 A flowchart of the low-level operation method of the environment modeling system.

[0017] Figure 5 This illustrates an implementation method based on a diagram. Figure 1 A flowchart of the low-level operation method of the environment modeling system.

[0018] Figure 6 This illustrates an implementation method based on a diagram. Figure 1 A flowchart of the low-level operation method of the environment modeling system.

[0019] Figure 7 This is a flowchart illustrating a low-level operation method of a processor-based system implemented according to a diagram.

[0020] Figure 8 This is a flowchart illustrating a low-level operation method of a processor-based system according to an illustrated implementation. Detailed Implementation

[0021] In the following description, certain specific details are set forth in order to provide a thorough understanding of the various disclosed embodiments. However, those skilled in the art will recognize that the embodiments can be practiced without one or more of these specific details or using other methods, components, materials, etc. In other instances, well-known structures associated with computer systems, actuator systems, and / or communication networks have not been shown or described in detail to avoid unnecessarily obscuring the description of the embodiments. In other instances, well-known computer vision methods and techniques for generating perceptual data and volumetric representations of one or more objects, etc., have not been described in detail to avoid unnecessarily obscuring the description of the embodiments.

[0022] Unless the context otherwise requires, throughout the specification and appended claims, the word “comprise” and its variations such as “comprises” and “comprising” shall be interpreted in an open, inclusive sense, meaning “including but not limited to”.

[0023] Throughout this specification, references to "an implementation," "an embodiment," or "an example" or "an example" mean that a particular feature, structure, or characteristic described in connection with that embodiment is included in at least one implementation or at least one example. Therefore, the phrases "an implementation," "an embodiment," "in one example," or "in an example" appearing in various places throughout this specification do not necessarily all refer to the same implementation or example. Furthermore, a particular feature, structure, or characteristic may be combined in any suitable manner in one or more implementations or examples.

[0024] As used in this specification and the appended claims, unless otherwise expressly stated, the singular forms “a,” “an,” and “the” include plural indicators. It should also be noted that, unless otherwise expressly stated, the term “or” is generally used to mean “and / or.”

[0025] As used in this specification and the appended claims, the terms determine, perform determine, and be determined, when used in the context of whether a collision will occur or result, mean to assess or predict whether a given posture or movement between two postures via multiple intermediate postures will result in a collision between a part of the robot and an object (e.g., another part of the robot, a permanent obstacle, a temporary obstacle, or some object other than the current target object).

[0026] The titles and summaries of the disclosures provided herein are for convenience only and do not explain the scope or meaning of the embodiments.

[0027] In summary, filtering out at least a portion of the robot's hyperscale representation from the representation of the operating environment (e.g., setting voxels to unoccupied for any object entirely within the hyperscale representation) provides a digital model of the operating environment that can be used, for example, for robot motion planning. The hyperscale representation exceeds the physical dimensions of at least a portion of the robot (e.g., attachments) to advantageously account for external dimensions attached to the robot, as well as structures extending beyond the robot's external dimensions (e.g., cables, cable ties, sensors, other features). The specific dimensions of the hyperscale representation can be based on a variety of factors, such as the geometry of the attachments, the orientation or position of the robot attachments, the orientation or position of the attachments relative to the robot attachments, the velocity of the attachments, the slackness of the attachments, etc., all of which can be modeled.

[0028] Figure 1 A three-dimensional operating environment 100 is shown in which one or more robots 102a, 102b, 102c (three are shown, collectively referred to as 102) can operate, according to an illustrated embodiment. For brevity, the three-dimensional operating environment 100 is referred to herein as environment 100. Environment 100 represents the three-dimensional space in which the robot 102 can operate and move. Note that environment 100 is different from the “configuration space” or “C-space” of any given robot 102.

[0029] Environment 100 may include obstacles 106a to 106e (collectively referred to as 106) representing areas of potential collision. The terms one obstacle and multiple obstacles 106 are used to indicate objects that represent a collision risk for a given robot 102 (e.g., inanimate objects including other robots, and living objects including humans and other animals).

[0030] Some of these obstacles 106a, 106b, and 106c may take the form of static obstacles or objects (i.e., obstacles or objects that do not move (i.e., translate, rotate) or change shape during the relevant time period or running time (e.g., buildings, trees, rocks, furniture, bases, supports, shelves). Some of these obstacles may take the form of dynamic obstacles or objects 106d and 106e in environment 100 (i.e., obstacles or objects that move (i.e., translate, rotate) or change shape during at least a portion of the relevant time period or running time, such as other robots, containers, vehicles, or robots, people, animals, rolling or moving objects). For example, a set of bases, walls, and support columns may be fixed or may not move or change shape during the relevant time period (e.g., running time) and are therefore considered static obstacles. Also, for example, a set of containers, workpieces, and another robot 102 may move or change shape (e.g., tilt) during the relevant time period (e.g., running time) and are therefore considered dynamic obstacles.

[0031] Some obstacles 106a to 106c occupy areas or volumes that do not change over time, for example, remaining fixed or unchanged during the robot's operation or movement. Such obstacles can therefore be referred to as static or persistent obstacles. The area or volume occupied by static or persistent obstacles (e.g., obstacles 106a to 106c) can be known at the time when the model is determined or the computational circuit is configured (referred to as the configuration time or pre-run time).

[0032] For other obstacles 106d and 106e, the corresponding area or volume occupied by the obstacle changes or is expected to change over time, for example, during robot operation or movement. Such obstacles can therefore be referred to as dynamic or temporary obstacles. The area or volume occupied by dynamic or temporary obstacles (such as obstacles 106d and 106e) is typically unknown during configuration time, but is determined during robot operation or runtime.

[0033] Environment 100 may optionally include one or more target objects 108a, 108b (two shown, collectively referred to as 108), with robot 102 intended to interact with one or more target objects 108a, 108b, for example by grasping, moving, or otherwise engaging target objects 108, to perform some defined task or operation. Target objects 108 are generally not considered obstacles, but in some implementations they may constitute obstacles, for example, in the presence of multiple target objects 108 that one robot 102a will sequentially engage, or when considered relative to another robot 102b, 102c that does not target a particular target object 108. Some implementations may not include any target objects, with robot 102 moving between various postures without interacting with or engaging any object.

[0034] Figure 1 The illustration depicts a representative environment 100 with a finite number of obstacles 106 and target objects 108. Typical environments may include numerous additional obstacles 106 and target objects 108, including objects that are other robots and various other natural or artificial static and dynamic obstacles 106 or target objects 108. Some environments 100 may omit the target objects 108 entirely, or even the dynamic obstacles 106d, 106e. The concepts taught herein can be applied in a similar manner to denser environments than those shown.

[0035] Robot 102 can be any type of robot, including but not limited to: Cartesian robots, selectively compliant arms for robotic assembly (SCARA) robots, cylindrical robots, delta robots, and polar and vertically articulated robots. Robot 102 can also be mobile, for example, in the form of a car, airplane, drone, or any other vehicle that can operate autonomously or semi-autonomously (i.e., at least partially autonomously) and move in the space represented by environment 100.

[0036] In the illustrated implementation, robot 102 includes a base 110 and an attachment 111 (only one is shown) formed by a set of links 112a, 112b (only two are shown, collectively referred to as 112) and a set of joints 114 (only one is shown), each joint 114 physically coupling a corresponding link pair 112. Robot 102 may also include one or more actuators 116 (only one is shown), which drive one link 112a to move relative to another link 112b or relative to the base 110. The actuators 116 can take any of a variety of forms, such as electric motors, stepper motors, solenoids, electromagnets, hydraulic pistons and cylinders, pneumatic pistons and cylinders, hydraulic valves, pneumatic valves, pumps or compressors for vacuum systems, hydraulic systems, and pneumatic systems including hydraulic and / or pneumatic reservoirs. Robot 102 may also include an end effector or end-of-arm tool 118, such as a gripper with an opposing finger, hook, or vacuum port, to physically engage a target object 108 in environment 100.

[0037] It is worth noting that robot 102 may have one or more structures attached to it, and these one or more structures may extend outward from robot 102. In the illustrated example, robot 102 includes a first cable 119a and a second cable 119b (only two cables are shown, collectively referred to as 119). For example, one or more cables 119 may be attached to various parts of robot 102. For example, one or more cables 119 may extend between the base 110 of robot 102 and attachment 111 (e.g., an attachment formed by links 112 and joints 114). Additionally or alternatively, one or more cables 119 may extend between the various links 112 of attachment 111 or between attachment 111 and end effector 118.

[0038] Cables 119 can take many forms. For example, one or more cables 119 can be used to supply power or pressurized fluid (e.g., hydraulic, pneumatic) to one or more actuators 116. For example, one or more cables 119 can be used to route communication signals from one or more sensors mounted on the robot (e.g., cameras, position or rotary encoders, proximity sensors, inertial sensors) or to one or more actuators 116.

[0039] Cable 119 may be attached to robot 102 at various points or locations along robot 102, such as at several points along attachment 111, and typically extends outward relative to the edge of attachment 111 or other parts of robot 102. In some cases, one or more portions of cable 119 may dangle, hang, or suspend from a portion of robot 102 at at least some locations and orientations of robot 102. In some cases, one or more portions of cable 119 may change their relative position or orientation to a portion of robot 102 as a portion of robot 102 moves, for example, when inertial forces act on cable 119 or a portion thereof. Other structures (e.g., sensors, inertial sensors, such as triaxial accelerometers) may also be attached to one or more portions of robot 102, typically extending outward from robot 102.

[0040] The environmental modeling system 120 may include one or more environmental sensors 122a, 122b, 122c, 122d (four are shown, collectively referred to as 122) and an environmental modeling computer system 124.

[0041] The environmental sensor 122 can take any of a variety of forms or types, such as one or more digital cameras 122a, 122b (e.g., time-of-flight digital cameras, 3D cameras), one or more motion sensors (e.g., passive-infrared motion sensors) or radar 122c, one or more LiDAR sensors 122d, one or more microphones (not shown), one or more weight sensors or weighing sensors (not shown), one or more photoelectric sensors (e.g., passive infrared (IR) sensors, including IR light sources and IR sensors) (not shown), one or more encoders (e.g., position encoders, rotary encoders, reed switches) (not shown), one or more temperature sensors (not shown), humidity sensors (not shown), and / or one or more pressure sensors (not shown), etc. The sensor 122 detects characteristics of the environment 100, including characteristics of obstacles, target objects, robots, and / or other objects in the environment 100 (e.g., position, orientation, shape, occupation, movement, speed). Sensor 122 can provide signals directly or indirectly to environment modeling computer system 124 via a processor-based system, either wired, optically, or wirelessly. The processor-based system collects and optionally preprocesses the collected sensor information. At least some of these signals can optionally be encoded or otherwise represented as sensed data.

[0042] The environmental modeling computer system 124 may include circuitry, such as one or more processors and / or one or more non-transitory processor-readable media (e.g., non-volatile memory, volatile memory, rotating memory), and may execute, for example, a set of one or more processor-executable instructions stored in the non-transitory processor-readable media. The environmental modeling computer system 124 may be communicatively coupled (e.g., wired, optical, wireless) to one or more sensors 122 to receive sensed information, such as perception data, directly or indirectly. The environmental modeling computer system 124 may optionally be communicatively coupled (e.g., wired, optical, wireless) to receive one or more models of the robot 102, such as one or more kinematic models 130 of the robot 102. The kinematic model 130 may, for example, take the form of a hierarchical data structure. For example, the hierarchical data structure may take the form of one or more types of trees. For example, suitable hierarchical data structures may include octrees, axis-aligned bounding boxes (AABB) trees, oriented (non-axis-aligned) bounding box trees, sphere trees, and / or other tree-like data structures. The kinematic model 130 can take the form of a non-hierarchical data structure (e.g., Euclidean distance field).

[0043] In some implementations, one or more processors executing processor-executable instructions can cause the environment modeling computer system 124 to process or preprocess some or all of the received sensor information. As described herein, the environment modeling computer system 124 can generate one or more digital representations or models 132 of a three-dimensional operating environment 100, which include any obstacles present in the environment 100 during a given time period of runtime. The digital representation or model 132 of the operating environment 100 can advantageously take the form of a “filtered” digital representation or model 132. The digital representation or model 132 of the operating environment 100 can advantageously be used for motion planning of a given robot, and in which areas occupied by the oversized representations 131a, 131b, 131c (three shown, collectively referred to as 131) of the given robot are indicated as unoccupied, while other robots (if any) remain unfiltered from the digital representation or model 132 of the operating environment 100 generated for motion planning of the given robot. The digital representation or model 132 can take any of a variety of forms, such as an occupied mesh.

[0044] One or more motion planners or motion planning systems 126 (only one shown) may be communicatively coupled (e.g., wired, optical, wireless) to an environment modeling computer system 124 to receive information from it and generate motion plans based at least in part on the received information. The information may include a digital representation or model 132 of the operating environment 100 (e.g., a “filtered” digital representation or model 132), which includes any obstacles present in the operating environment 100 (including other robots 102). The input to the motion planner or motion planning system 126 may also include a set of tasks, objectives, or goals 133 to be performed by each robot 102. Task performance typically employs motion planning, which in turn employs collision detection.

[0045] The motion planner or motion planning system 126 may be an integral part of the robot 102, separate from and distinct from the robot 102, or one or more parts may be on the robot 102 while one or more other parts may be separate from the robot 102 (i.e., outside the robot 102). The motion planner or motion planning system 126 may include circuitry, such as one or more processors and / or one or more non-transitory processor-readable media (e.g., non-volatile memory, volatile memory, rotating memory), and may execute, for example, a set of one or more processor-executable instructions stored by the non-transitory processor-readable media. The motion planner or motion planning system 126 may generate a motion plan 127 for causing the robot 102 to perform a specific task, such as moving between a series of consecutive postures, preferably without collision with or with a reduced probability of collision with obstacle 106. The motion planner or motion planning system 126 may be communicatively coupled (e.g., wired, optical, wireless) to one or more robot control systems 138 to provide instructions to which a particular robot 102 follows or executes the motion plan.

[0046] The motion planner or motion planning system 126 may include or access the collision detection system 140. The collision detection system 140 may include circuitry, such as one or more processors and / or one or more non-transitory processor-readable media (e.g., non-volatile memory, volatile memory, rotating memory), and may execute, for example, a set of one or more processor-executable instructions stored by the non-transitory processor-readable media. The collision detection system 140 advantageously employs a “filtered” digital representation or model 132 and optionally a kinematic model 130 of the robot (e.g., a data structure representation of the kinematic model 130) to determine, detect, or evaluate the probability of the robot colliding with obstacles 106 in the environment 100 as it moves in various postures or between postures. Those obstacles 106 may include other robots in the environment 100. The motion planner or motion planning system 126 and / or the collision detection system 140 may, for example, take the form of the motion planning system and collision detection system described in International (PCT) Patent Application PCT / US2019 / 045270, filed August 6, 2019.

[0047] The robot control system 138 may include several components, which are typically different but may be combined in some implementations in a common circuit board, processor, or other circuitry. For example, a set of drivers may include circuitry communicatively coupled to actuator 116 to drive actuator 116 to cause robot 102 to adopt or move into a defined posture. For example, the drivers may include motor controllers and similar circuitry that drive any one or more of an electric motor, stepper motor, solenoid, electromagnet, hydraulic piston, pneumatic piston, hydraulic valve, pneumatic valve, vacuum system, hydraulic system, and / or pneumatic system pump or compressor.

[0048] Figure 2 A system 200 is illustrated according to at least one illustrated implementation. The system 200 may include or be implemented in... Figure 1 Shown and about Figure 1 The various components or structures described.

[0049] System 200 may include one or more sensors 202, one or more environment modeling computer systems 204, one or more motion planners 206, and one or more robots 208. One or more environment modeling computer systems 204 may be communicatively coupled to one or more sensors 202 to receive perceived information or data from them. One or more environment modeling computer systems 204 may be communicatively coupled to provide a digital representation or model of the three-dimensional operating environment. The digital representation or model of the three-dimensional operating environment may advantageously be filtered by a large representation 131 of the given robot to include structures (e.g., cables) extending outward from the robot. One or more motion planners 206 may be communicatively coupled to provide motion plans 127 to one or more robots 208 via one or more motion controllers 210. Figure 1 ).

[0050] As previously described, each robot 208 may include an attachment formed by a set of links and joints, wherein an end-effector or end effector is provided at the end of the attachment, and / or each robot 208 may include one or more actuators 211a, 211b, 211c capable of operating to move the links about the joints. Figure 2 Three are shown in the diagram, collectively referred to as 211. Each robot 208 may include one or more motion controllers (e.g., motor controllers) 210 (only one is shown), which receive control signals, for example, from a motion planner or motion planning system 206 and provide drive signals to drive actuators 211. Motion controllers 210 may be dedicated to controlling a specific actuator in actuators 211.

[0051] For illustrative purposes, an exemplary environment modeling computer system 204 will be described in detail. Those skilled in the art will recognize that this description is exemplary and that changes may be made to the described and illustrated environment modeling computer system 204.

[0052] The environment modeling computer system 204 may include one or more processors 222 and one or more associated non-transitory computer- or processor-readable storage media, such as system memory 224a, disk drive 224b, and / or memory or registers of processor 222 (not shown). The non-transitory computer- or processor-readable storage media 224a, 224b are communicatively coupled to processor 222a via one or more communication channels, such as system bus 229. System bus 229 may employ any known bus structure or architecture, including memory bus with memory controller, peripheral bus, and / or local bus. One or more of such components may also be or alternatively connected via one or more other communication channels (e.g., one or more parallel cables, serial cables, or wireless network channels capable of high-speed communication, such as Universal Serial Bus (“USB”) 3.0, Peripheral Component Interconnect Express (PCIe)) or via... They communicate with each other.

[0053] The environmental modeling computer system 204 can also be communicatively coupled to one or more remote computer systems 212, such as server computers, desktop computers, laptop computers, ultra-portable computers, tablet computers, smartphones, wearable computers, and / or sensors. Figure 2 (Not shown in the diagram), one or more remote computer systems 212 are directly or indirectly coupled to various components of the environment modeling computer system 204, for example, via a network interface (not shown). The remote computing system (e.g., a server computer) can be used to program, configure, control, or otherwise communicate with the environment modeling computer system 204 via an interface or provide input data (e.g., robot models) to the environment modeling computer system 204. Such connections can be made via one or more communication channels 214, such as one or more wide area networks (WANs) such as Ethernet or the Internet using the Internet Protocol. In some implementations, pre-runtime calculations or configuration-time calculations (e.g., cable modeling) can be performed by a system separate from the environment modeling computer system 204 (e.g., computer system 212). Runtime calculations can be performed by one or more environment modeling computer systems 204 and / or motion planner 206.

[0054] As described, the environment modeling computer system 204 may include one or more processors 222 (i.e., circuitry), non-transitory storage media 224a, 224b, and a system bus 229 coupling various system components. The processor 222 may be any logic processing unit, such as one or more central processing units (CPUs), digital signal processors (DSPs), graphics processing units (GPUs), field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), programmable logic controllers (PLCs), etc. Non-limiting examples of commercial computer systems include, but are not limited to, those in the United States. The company provides Celeron, Core, Core 2, Itanium, and Xeon series microprocessors; Advanced Micro Devices provides K8, K10, Bulldozer, and Bobcat series microprocessors; Apple Computer provides A5, A6, and A7 series microprocessors; Qualcomm provides Snapdragon series microprocessors; and Oracle provides SPARC series microprocessors. Figure 2The construction and operation of the various structures shown can be implemented or employ structures, techniques, and algorithms described in or similar to those described in the following patent applications: International Patent Application No. PCT / US2017 / 036880, filed June 9, 2017, entitled “MOTION PLANNING FORAUTONOMOUS VEHICLES AND RECONFIGURABLE MOTION PLANNING PROCESSORS”; International Patent Application Publication No. WO2016 / 122840, filed January 5, 2016, entitled “SPECIALIZED ROBOT MOTIONPLANNING HARDWARE AND METHODS OF MAKING AND USING SAME”; and APPARATUS, METHOD AND ARTICLE TOFACILITATE MOTION PLANNING OF AN AUTONOMOUS VEHICLE IN AN ENVIRONMENT HAVINGDYNAMIC, filed January 12, 2018. The U.S. Patent Application No. 62 / 616,783 entitled “OBJECTS”; U.S. Patent Application Serial No. 62 / 865,431 entitled “MOTIONPLANNING FOR MULTIPLE ROBOTS IN SHARED WORKSPACE” filed on June 24, 2019; and / or the international (PCT) patent application PCT / US2019 / 045270 filed on August 6, 2019.

[0055] System memory 224a may include read-only memory (“ROM”) 226, random access memory (“RAM”) 228, flash memory 230, and EEPROM (not shown). A basic input / output system (“BIOS”) 232, which may form part of ROM 226, contains basic routines that help transfer information between elements within the environment modeling computer system 204, for example, during startup.

[0056] Drive 224b may be, for example, a hard disk drive for reading from and writing to a disk, a solid-state (e.g., flash memory) drive for reading from and writing to a solid-state memory, and / or an optical disc drive for reading from and writing to a removable optical disc. The environment modeling computer system 204 may also include any combination of such drives in various different embodiments. Drive 224b may communicate with processor 222 via system bus 229. Drive 224b may include an interface or controller (not shown) coupled between such a drive and system bus 229, as known to those skilled in the art. Drive 224b and associated computer-readable media provide the environment modeling computer system 204 with non-volatile storage of computer- or processor-readable and / or executable instructions, data structures, program modules, and other data. Those skilled in the art will understand that other types of computer-readable media capable of storing computer-accessible data may be employed, such as WORM drives, RAID drives, magnetic tape, digital video disks (“DVDs”), Bernoulli cassette tapes, RAM, ROM, smart cards, etc.

[0057] Executable instructions and data can be stored in system memory 224a, such as operating system 236, one or more applications 238, other programs or modules 240, and program data 242. Application 238 may include processor-executable instructions that cause processor 222 to execute one or more of the following: collecting or receiving sensor or perception data; receiving or generating a representation or model of a three-dimensional environment; receiving or generating a large representation 131 of the robot. Figure 1 The system identifies objects within the region corresponding to the robot's oversized representation 131 in the representation of the 3D environment, sets the occupancy value of the region occupied by these objects to unoccupied, and provides the resulting "filtered" representation or model for further operations, such as motion planning. This operation can be performed as described herein (e.g., see reference 131). Figure 3 and Figure 8 Application 238 may include one or more machine-readable and machine-executable instructions that cause processor 222 to perform other operations, such as optionally processing sensed data (captured via sensors). Processor-executable instructions cause processor 222 to construct a “filtered” representation or model based on the sensed data, wherein volumes not only containing areas occupied by a given robot are indicated as unoccupied. Application 238 may additionally include one or more machine-executable instructions that cause processor 222 to perform various other methods described herein and by reference to the references incorporated herein.

[0058] Although Figure 2 The system is shown to be stored in system memory 224a, but the operating system 236, application program 238, other applications, program / module 240, and program data 242 may be stored on other non-transitory computer or processor readable media such as drive 224b.

[0059] Although not strictly necessary, many implementations will be described in the general context of computer-executable instructions, such as program application modules, objects, or macros stored on a computer or processor-readable medium and executed by one or more computers or processors. In various implementations, operations may be performed entirely in hardware circuitry or as software stored in a storage device such as system memory 224a and executed by one or more hardware processors 222, such as one or more microprocessors, digital signal processors (DSPs), field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), graphics processing unit (GPU) processors, programmable logic controllers (PLCs), electrically programmable read-only memory (EEPROMs), or as a combination of hardware circuitry and software stored in a storage device.

[0060] The environment modeling computer system 204 may optionally include one or more input / output components, such as a monitor or touch screen display 244, a keypad or keyboard 246, and / or a pointing device such as a computer mouse 248.

[0061] Those skilled in the art will understand that the illustrated and other implementations can be practiced with other system architectures and arrangements and / or other computing system architectures and arrangements, including those of robots, handheld devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, personal computers (“PCs”), networked PCs, minicomputers, mainframes, etc. Implementations or embodiments, or portions thereof (e.g., at configuration time and runtime), can be practiced in a distributed computing environment where tasks or modules are executed by remote processing devices linked via a communication network. In a distributed computing environment, program modules can reside on both local and remote storage devices or media. However, the location and manner in which certain types of information are stored can be important for helping to improve robot configuration.

[0062] The motion planner or motion planning system 206 may include one or more processors 250, and one or more associated non-transitory computer- or processor-readable storage media, such as system memory 252, disk drives (not shown), and / or registers of processor 250 (not shown). The non-transitory computer- or processor-readable storage media (e.g., system memory 252) is communicatively coupled to processor 250 via one or more communication channels, such as system bus 254. System bus 254 may employ any known bus structure or architecture, including memory buses with memory controllers, peripheral buses, and / or local buses. One or more such components may also, or alternatively, be connected via one or more other communication channels (e.g., one or more parallel cables, serial cables, or wireless network channels capable of high-speed communication, such as Universal Serial Bus (“USB”) 3.0, Peripheral Component Interconnect Standard (PCIe)) or via... Communication between them. One or more communication ports (not shown) may provide communication between the motion planner or motion planning system 206 and the environment modeling computer system 204 and / or motion controller 210. The motion planner or motion planning system 206 may optionally be communicatively coupled (e.g., wired, optical, or wireless) to a computer or terminal (not shown) to allow user input (e.g., indications of motion subdivision granularity values ​​or specifications) and to provide user output.

[0063] The motion planner or motion planning system 206 can receive a filtered representation or model of the environment, wherein a given hyperscale representation 131 of the robot is provided. Figure 1 The position has been indicated as unoccupied. The motion planner or motion planning system 206 may also receive the robot model, task, objective, limit on the total number of robots, limit on the task of each robot, limit or constraint on variables or other parameters, and / or limit on iteration.

[0064] The motion planner or motion planning system 206 executes processor-executable instructions (application 256) that cause the motion planner or motion planning system 206 to perform motion planning, typically relying on collision evaluation to generate a motion plan for the robot to perform a task. The motion planner or motion planning system 206 may, for example, construct a motion plan by performing collision detection or evaluation, update the cost of edges in the motion planning graph based on collision detection or evaluation, and perform path search or evaluation. The motion planner or motion planning system 206 may, for example, generate a group of candidate solutions, model the candidate solutions, generate or determine the costs associated with the respective candidate solutions based at least in part on the modeling, perform optimization on the group of candidate solutions through an optimization engine that collaboratively optimizes two or more of the following across two or more non-homogeneous sets of parameters: the robot's corresponding base position and orientation, task assignment to the corresponding robot in the robot, and the robot's corresponding target sequence; and / or provide outputs that can be used to locate and orient the robot in a multi-robot operating environment and enable the robot to perform a task.

[0065] Figure 3 This illustrates an implementation method based on a diagram. Figure 1 A high-level operation method 300 for an environment modeling system. Method 300 can be executed by one or more processor-based systems, for example, by executing instructions stored on one or more non-transitory processor-readable media.

[0066] Regarding two or more robots 102a to 102c ( Figure 1 ), 208 Figure 2 The operating environment 100 in which it operates Figure 1 Method 300 is described using the method described above. Therefore, method 300 is shown as employing an iterative loop that generates a representation or model of a three-dimensional environment, which allows for sequential execution of motion planning for each of the robots, with a total number specified by an integer value N. Although shown as being executed sequentially for robots 102a to 102c, the specific order can be changed, and various implementations can operate on robot 102 in any desired order, including repeating the process multiple times for a single robot 102a before execution for different robots in robots 102b, 102c. In some implementations, one or more threads or instances of method 300 can operate in parallel, for example, separate threads or instances operating simultaneously for each of robots 102a to 102c in operating environment 100.

[0067] Method 300 begins at 302, for example in response to a call via a calling routine or module, or in response to receiving a signal or detecting an event that could affect the computer system, such as environment modeling computer system 124. Figure 1 ), 204 Figure 2 Apply electricity.

[0068] At 304, one or more sensors 122, 202 capture a three-dimensional operating environment 100 characterizing the operation of one or more robots 102, 208 therein. Figure 1 Sensor data or perceived data. Sensors 122, 202 can take many forms, such as one or more of the following: digital camera, LIDAR sensor, microphone, weight sensor or load cell, photoelectric sensor (e.g., IR light source and IR sensor), encoder (e.g., position encoder, rotary encoder, reed switch), temperature sensor, humidity sensor and / or pressure sensor, etc. Sensor or perceived data can be captured in any of these multiple forms and can also be represented in any of these multiple forms, such as point cloud or occupied grid.

[0069] For example, sensors 122, 202 can provide sensor or sensing data and / or other sensing information to one or more processors. The sensor or sensing data can be provided as a stream representing which voxels or sub-volumes (e.g., boxes) are occupied in the environment at the current time (e.g., pre-configuration time, configuration time, runtime). This sensor or sensing data can be provided to one or more processors in the form of an occupation grid. Specifically, objects in environment 100 (e.g., robot 102, obstacle 106, target object 108) can be represented by representing the corresponding surfaces of the objects in environment 100 (e.g., robot 102, obstacle 106, target object 108) as a mesh of voxels (3D pixels) or polygons (typically triangles). Each discrete spatial region is called a "voxel," equivalent to a 3D (volume) pixel.

[0070] At position 306, processor-based systems, such as environmental modeling computer systems 124, are used. Figure 1 ), 204 Figure 2 Access and / or generate robot 102 ( Figure 1 A three-dimensional environment in which operations are performed (100) Figure 1 A digital representation or model of the sensor 122. For example, environmental modeling computer systems 124, 204 can access digital representations or models collected or assembled by a processor-based dedicated system communicatively located at sensor 122. Figure 1 The environment modeling computer system 124 is located between or between the sensor 122 and the environmental modeling computer system 124, 204. Alternatively, the environmental modeling computer system 124 can directly receive sensor or sensing information or data from the sensor 122 and generate a three-dimensional representation or model 132 of the operating environment. Figure 1 ).

[0071] Therefore, at least one sensor 122, 202 can capture one or more physical characteristics of the three-dimensional operating environment 100 during operation. At least one processor-based device generates a digital representation of the three-dimensional operating environment 100 based on the captured physical characteristics of the three-dimensional operating environment sensed by at least one sensor 122, 202. For example, the processor-based device (e.g., a computer system) can generate one or more of a point cloud, voxel mesh, surface map, grid, occupancy mesh, or other types of data structure.

[0072] For example, a 3D operating environment 100 can be represented as a plurality of voxels, each voxel including at least one corresponding occupancy value that indicates the occupancy status or state of the 3D operating environment as either occupied or unoccupied. In some implementations, the occupancy value is a Boolean value that indicates only two states: occupied or unoccupied. In other implementations, the occupancy value can represent three or more states, such as occupied, unoccupied, and unknown. In other implementations, the occupancy value can be, for example, an integer or real value representing, the probability of occupancy at a corresponding location (e.g., a voxel).

[0073] Alternatively or concurrently, in at least some implementations, the environment modeling computer systems 124 and 204 can transform the outputs of sensors 122 and 202. For example, the environment modeling computer systems 124 and 204 can transform the outputs of sensors 122 and 202, can combine the outputs of multiple sensors 122, and / or can use coarser voxels to represent the environment 100. In some cases, representing objects as boxes (rectangular prisms) may be advantageous. Due to the fact that the shape of an object is not randomly formed, the organization of voxels can have a great deal of structure. Therefore, representing an object as a box may require far fewer bits than a voxel-based representation (i.e., perhaps only the x, y, z Cartesian coordinates of the two opposite corners of the box are needed). Furthermore, performing cross-testing on boxes is comparable in complexity to performing cross-testing on voxels.

[0074] At 308, a processor-based system, such as environment modeling computer systems 124 and 204, selects robot I, for which a "filtered" representation or model of the operating environment 100 is generated, for example, for motion planning of robot I. In some implementations, a single robot may be present. In other implementations, two or more robots that at least partially share or operate in the operating environment 100 may be included. Various heuristics can be used in selecting robot I. For example, environment modeling computer systems 124 and 204 may select a robot based on a corresponding priority of a specific task assigned to the respective robot to perform. Alternatively, environment modeling computer systems 124 and 204 may iterate sequentially over the robots (e.g., robots 102a to 102c). As explained herein, this method can be repeated for the various robots 102a to 102c in any desired order.

[0075] At 310, a processor-based system, such as environment modeling computer systems 124, 204, identifies one or more objects in the three-dimensional environment 100, as represented by a representation or model of the operating environment 100 based on sensor or perceived information or data. Objects may represent robots (e.g., robot 102a), for which the three-dimensional representation or model will be used for motion planning; objects may represent one or more obstacles in the operating environment, including other robots (e.g., 102b, 102c); and / or may represent one or more targets 108 in the operating environment 100. For example, the environment modeling computer systems 124, 204 identify, in a digital representation of the three-dimensional operating environment 100, one or more elements representing one or more physical objects located in the three-dimensional operating environment 100.

[0076] At 312, processor-based systems such as environment modeling computer systems 124, 204 access and / or generate a 3D hyperscale representation or “expanded” model 131 of the current robot I. Figure 1 ).

[0077] At 314, a processor-based system, such as environment modeling computer systems 124 and 204, determines which objects (if any) are entirely located within the three-dimensional region corresponding to the expanded model or hyperscale representation 131 of robot I. For example, environment modeling computer systems 124 and 204 determine which physical objects (if any) represented in the digital representation of the three-dimensional operating environment 100 are entirely located within the three-dimensional representation of the hyperscale volume 131, which includes at least attachment 111 of the first robot (e.g., 102a). Figure 1), wherein at least a portion of the oversized volume 131 extends beyond the corresponding peripheral dimensions of the attachment 111 of at least the first robot (e.g., 102a) to include one or more cables 119 physically coupled to the attachment 111 of the first robot. Figure 1 For example, the environment modeling computer systems 124 and 204 can identify one or more groups of voxels in the digital representation of the 3D operating environment 100, wherein the group of voxels is contiguous and each has a corresponding occupancy value representing an occupancy status. For example, an algorithm called Connected Component can be executed to group connected voxels together. As another example, the environment modeling computer systems 124 and 204 can then determine, for each group of voxels in the digital representation of the 3D operating environment that has been identified as representing physical objects, whether that group of voxels crosses the boundary of a large-volume 3D representation.

[0078] At 316, for any object completely located within the 3D region corresponding to the expanded model or oversized representation 131 of the current robot I, a processor-based system, such as environment modeling computer systems 124, 204, sets an occupancy value to represent the region as unoccupied, in order to provide a filtered discrete representation 132 of the operating environment 100. Figure 1 In this discrete representation, the current robot I has been removed as an object. For example, for any physical object determined to be entirely within the 3D representation of the hypervolume 131, the environment modeling computer systems 124, 204 set one or more occupancy values ​​in the digital representation of the 3D operating environment 100 to represent the volume corresponding to the corresponding physical object as unoccupied, thereby providing a filtered representation 132 of the 3D operating environment 100 in which the hypervolume 131, containing at least robot attachment 111 and at least one cable 119, is not indicated or represented as an obstacle. For example, for each voxel in the digital representation of the 3D operating environment 100 corresponding to any object represented in the digital representation of the 3D operating environment 100 that is entirely within the hypervolume 131, the environment modeling computer system 124 may set the corresponding occupancy value of the corresponding voxel to represent unoccupied.

[0079] At 318, a processor-based system, such as motion planner 126, performs motion planning on the current robot I using a filtered discrete representation 132 of the three-dimensional operating environment 100.

[0080] Various algorithms can be used to perform motion planning. Each of these algorithms typically needs to be able to determine whether a given pose of the robot, or a movement from one pose to another, will result in a collision with the robot itself or with obstacles in the environment.

[0081] By referring to or incorporating herein the collision detection hardware, software, and / or firmware, it is possible to determine or calculate whether a single robot pose or robot motion from one pose (e.g., a start pose) to another pose (e.g., an end pose) results in a collision between the robot and itself or with any obstacle in the current environment in which the robot operates. The environment may include obstacles, i.e., objects with which there is a risk of collision (e.g., inanimate objects, living objects including humans and other animals). The environment may or may not include one or more target objects, i.e., objects with which the robot intends to engage. For some obstacles, the corresponding volume occupied by the obstacle is known during the time the motion planner's model or computational circuitry is configured (referred to as the configuration time) and is expected to remain fixed or unchanged during the robot's operation (referred to as the runtime time). These obstacles are called permanent obstacles because the volume occupied by the obstacle is known during configuration and is expected to remain fixed or unchanged during runtime. For other obstacles, the corresponding volume occupied by the obstacle is unknown during configuration time and is determined only during runtime. These obstacles are called temporary obstacles because the volume occupied by the obstacle is unknown during configuration time. The corresponding volume occupied by one or more of these temporary obstacles may be fixed or not move or change over time and are referred to as static obstacles. The corresponding volume occupied by one or more of these temporary obstacles may move or change over time and are referred to as dynamic obstacles.

[0082] The collision detection hardware, software, and / or firmware described herein can be implemented as subroutines or functions that can be called or enabled by various motion planning algorithms such as Probabilistic Route Graph (PRM), Rapid Exploratory Random Tree (RRT), RRT*, Bidirectional RRT, etc. The collision detection hardware, software, and / or firmware described herein can also be used to accelerate grasping planning by rapidly evaluating many candidate grasping poses.

[0083] The various implementations described in this paper typically employ two or three configuration-time inputs: i) a kinematic model of the robot; ii) a representation of permanent obstacles in the environment, the volume occupied by which the permanent obstacles in the environment is known at configuration time; and optionally iii) motion subdivision granularity values ​​or specifications. The robot's kinematic model includes constraints on any one of the robot's multiple joints (e.g., minimum and maximum angles of the elbow joint), transformations from each link of the robot to the corresponding parent link of the robot, the axes of each joint of the robot, and the geometric specifications of each link of the robot.

[0084] The various implementations described herein typically employ two or three runtime inputs: a) a starting pose; b) an ending pose, optionally selected if motion is being evaluated; and c) a representation of temporary obstacles in the environment, the volume of which is known during runtime but unknown during configuration time. Temporary obstacles can be static (i.e., fixed or unmoved or unchanged in shape during the relevant or runtime period) or dynamic (i.e., moved or changed in shape during at least a portion of the relevant or runtime period). For a robot with D degrees of freedom, the pose can be specified as a D-tuple, where each element of the tuple specifies the position or rotation of that degree of freedom (joint).

[0085] Efficient collision detection hardware can be achieved by carefully selecting the data structure to represent objects, poses, and / or motions. Choosing a good data structure can advantageously reduce the amount of memory required for storage, the amount of hardware needed for collision detection, and the latency and power consumption of performing collision detection.

[0086] At 320, a processor-based system, such as environmental modeling computer systems 124 and 204, determines whether the robot's motion planning is complete. If the robot's motion planning is incomplete, control is passed to 308, where environmental modeling computer systems 124 and 204 select a robot again. In some cases, environmental modeling computer systems 124 and 204 may select the same robot as selected in the most recent iteration. In other cases, environmental modeling computer systems 124 and 204 may select a different robot than selected in the most recent iteration.

[0087] If the robot's motion planning is complete, control is passed to 322, where method 300 terminates, for example, until it is called or enabled again. Alternatively, method 300 may be executed continuously, or, for example, as multiple threads on a multi-threaded processor or on the corresponding core of the processor.

[0088] Figure 4 This illustrates an implementation method based on a diagram. Figure 1 The low-level operation method 400 of the environment modeling system is described. Method 400 can be executed as part of the execution of method 300. Although voxels and voxel space have been described, similar methods can be performed using other representations of the 3D environment and / or other representations of objects in the 3D environment. Such other representations can be used to identify object representations and determine whether these object representations are entirely within the defined 3D region, or whether these object representations are outside or cross the defined 3D region.

[0089] At 402, processor-based systems, such as environmental modeling computer systems 124, are used. Figure 1 ), 204 Figure 2 This involves identifying one or more sets of continuous voxels in a discrete representation of a 3D operating environment 100, where each continuous voxel includes a corresponding occupancy value that represents the corresponding location in the 3D operating environment 100 as an occupied object among one or more objects. For example, an algorithm called connected component analysis can be executed by a processor to combine connected occupied voxels together. Many other algorithms and / or processor-executable instruction sets can be used to identify sets of proximal neighbor voxels.

[0090] At 404, a processor-based system, such as an environment modeling computer system 124, 204, determines which (if any) of one or more sets of continuous voxels in the voxels lie entirely within the extended model or hyperscale representation of the robot 131. Figure 1 The three-dimensional region of the voxel in the discrete representation of the corresponding three-dimensional operating environment 100.

[0091] At 406, for each voxel in the discrete representation of the three-dimensional operating environment 100—which corresponds to any one of one or more sets of consecutive voxels within a three-dimensional region of voxels completely located within the discrete representation of the operating environment (which corresponds to the expanded model or oversized representation 131 of the current robot I (e.g., the first robot 102a))—a processor-based system, such as environment modeling computer systems 124, 204, sets the occupancy value of the corresponding voxel to indicate that it is unoccupied. In practice, any object representation completely located within the region corresponding to the oversized or “expanded” representation 131 of the robot (e.g., the first robot 102) can be indicated as unoccupied, for example, by changing the occupancy value or state of the relevant pixel from occupied to unoccupied. Any object representation completely located outside or across the region can be indicated as occupied, for example, by maintaining the occupancy value or state of the relevant voxel as occupied.

[0092] Figure 5 This illustrates an implementation method based on a diagram. Figure 1 The low-level operation method 500 of the environment modeling system. Method 500 can be used to determine the oversized or "expanded" representation 131 of robot 102. Method 500 can be executed as part of the execution of method 300.

[0093] Method 500 can enter iterative loop 502 to process each part J of the current robot I in turn, where J = 1 to M, and M is an integer representing the total number of parts (e.g., links, joints) of the current robot I that will be modeled or evaluated.

[0094] At position 504, processor-based systems, such as environment modeling computer systems 124, are used. Figure 1), 204 Figure 2 The corresponding super-large representation of the corresponding part J of the current robot I is determined. The corresponding super-large representation of the corresponding part J of the current robot I contains a three-dimensional volume larger than the three-dimensional volume of the corresponding physical part of the current robot I. For example, a processor-based system can modify Annex 111 of the first robot (e.g., 102a). Figure 1 The three-dimensional representation of a component 111 of a first robot (e.g., 102a) can be modified to increase at least one dimension of the component at least at or along a location thereon. For example, a processor-based system can modify the three-dimensional representation of a component 111 of a first robot (e.g., 102a) to increase at least one dimension along at least a portion of the component 111 (e.g., at least one cable 119). Figure 1 The processor-based system may, either intentionally or not, increase any dimension of the attachment 111 along a portion of its extension. For any portion of the attachment 111 of the first robot (e.g., 102a) along which at least one cable 119 does not extend, the processor-based system may, intentionally or not, increase any dimension of the three-dimensional representation of the attachment 111.

[0095] Figure 6 This illustrates an implementation method based on a diagram. Figure 1 A low-level operation method 600 for an environmental modeling system. Method 600 can be used to determine the robot 102 ( Figure 1 The super-large or "expansion" of 131 indicates 131. Figure 1 Method 600 can be executed as part of the execution of method 300 and / or method 500.

[0096] Method 600 can enter an iterative loop at 602 to process each part J of the current robot I in turn, where J = 1 to M, and M is an integer representing the total number of parts of the current robot I to be modeled or evaluated.

[0097] At position 604, processor-based systems, such as environmental modeling computer systems 124, are used. Figure 1 ), 204 Figure 2 Determine whether a given portion J of the current robot I (e.g., the first robot 102a) has a cable extending along it.

[0098] If a given part J of the current robot I does not have a cable extending along it, the environment modeling computer system 124, 204 uses the actual physical dimensions of part J of the current robot I at 606 for representation.

[0099] If a given portion J of the current robot I does indeed have cables extending along it, the environment modeling computer systems 124, 204 determine an oversized representation of portion J of the current robot I at 608 for use in the oversized representation. The oversized representation of portion J of the current robot I has dimensions that include the actual physical dimensions of portion J and extends beyond the actual physical dimensions of portion J at least at one location along portion J of the current robot I. The size of the oversized representation should be large enough to include any attachment structures (e.g., cables) extending outward from portion J of the current robot I.

[0100] Figure 7 A low-level operation method 700 for a processor-based system, implemented in one illustrative manner, is shown. Method 700 can be used to determine the operation of robot 102 (…). Figure 1 The super-large or "expansion" of 131 indicates 131. Figure 1 Method 700 may be executed during the configuration or pre-run process, or alternatively during runtime as part of the execution of methods 300 and / or 500.

[0101] Processor-based systems rely on one or more of the following to process one or more cables 119 ( Figure 1 The model occupies the corresponding volume: the geometry of cable 119, the current set of joint positions of the current robot I (e.g., the first robot 102), the position and orientation of cable 119 relative to a portion of the current robot I, the effect of gravity on cable 119, the slack, sagging, sinking or tension of cable 119 and / or the current velocity of at least a portion of the current robot I or a portion of cable 119 (e.g., considering inertial effects on cable 119).

[0102] For example, a processor-based system can generate a three-dimensional representation of a hypervolume based at least in part on the position of at least a portion of the attachment 111 of the current robot I, taking into account gravity—particularly considering that the cable 119 may sag, sink, hang, or swing. For example, a processor-based system can generate a three-dimensional representation of a hypervolume based at least in part on the orientation of at least a portion of the attachment 111 of the current robot I, taking into account gravity. For example, a processor-based system can generate a three-dimensional representation of a hypervolume based at least in part on the current set of joint positions of the current robot I. For example, a processor-based system can generate a three-dimensional representation of a hypervolume including the base 110 of the current robot I (…). Figure 1 At least two links 112a and 112b of attachment 111 of the current robot I. Figure 1The system can generate a three-dimensional representation of the oversized volume of the end effector of the current robot I and at least two cables 119 coupled to move with one or more attachments. For example, a processor-based system can generate the three-dimensional representation of the oversized volume based at least in part on the geometry of at least one cable 119. For example, a processor-based system can generate the three-dimensional representation of the oversized volume based at least in part on at least a portion of an attachment 111 of the first robot 102 or the velocity of a cable 119 attached to the robot 102.

[0103] In some implementations, processor-based systems can generate a 3D representation of a large volume in bounding box form based on a set of sizes of attachments to the first robot and a set of boarder buffer specifications that define offsets from the robot's perimeter.

[0104] Figure 8 This illustrates an implementation method based on a diagram. Figure 1 The low-level operational method 800 of the environmental modeling system. In some implementations, the processor-based system can model cables to facilitate the determination of a very large representation of the robot or its parts. Method 800 can be executed during configuration or pre-running processes, or alternatively, during runtime as part of the execution of methods 300 and / or 500.

[0105] At 802, during the pre-run period, the processor-based system performs multiple operations.

[0106] For example, at 804, the processor-based system repeatedly moves at least a portion of the robot having at least one cable.

[0107] For example, at 806, a processor-based system captures data from one or more sensors representing the volume occupied by at least one cable during each movement of said part of the robot.

[0108] For example, a processor-based system generates a digital model of at least one of the cables at 808.

[0109] By analyzing cable 119 ( Figure 1 The system can model or generate a specified robot 102. Figure 1 Annex 111 () Figure 1A precise representation of the region surrounding each segment of the cable 119. A very precise mechanical model can specify almost exactly where the cable 119 is, based on the geometry of the cable 119, the stiffness of the cable 119, the attachment point of the cable 119 to the robot 102, the current set of joint positions of the robot 102, the sag and / or tension of the cable 119, and / or the velocity or acceleration of the cable 119 and / or attachment 111. The cable 119 can be modeled using any kind of computer-aided technique or representation (e.g., nonrational uniform b-splines (NRUBS)). This model can be learned during the training period when simulating robot operation (e.g., through neural networks or other machine learning processes), and a model representing how the cable 119 behaves can be developed.

[0110] Example

[0111] Example 1: A system comprising:

[0112] At least one processor;

[0113] At least one non-transitory processor-readable medium communicatively coupled to the at least one processor and storing processor-executable instructions that, when executed by the at least one processor, cause the at least one processor to:

[0114] Identify one or more elements representing one or more physical objects located in the three-dimensional operating environment in a digital representation of the three-dimensional operating environment;

[0115] Determine which physical objects—if any—are represented in the digital representation of the three-dimensional operating environment entirely within a 3D representation of a hypervolume, the hypervolume comprising at least an attachment of the first robot, at least a portion of the hypervolume extending beyond the corresponding peripheral dimensions of at least the attachment of the first robot to include one or more cables physically coupled to the attachment of the first robot; and

[0116] For any physical object determined to be entirely within the three-dimensional representation of the oversized volume, one or more occupancy values ​​are set in the digital representation of the three-dimensional operating environment to represent the volume corresponding to the physical object as unoccupied, thereby providing a filtered representation of the three-dimensional operating environment in which the oversized volume containing at least the robot attachment of the first robot and at least one cable is not indicated as an obstacle.

[0117] Example 2. The system according to Example 1, wherein the instruction, when executed by the at least one processor, causes the at least one processor to also:

[0118] The motion planning of the first robot is performed using a filtered representation of the three-dimensional operating environment.

[0119] Example 3. The system according to Example 1, wherein the instruction, when executed by the at least one processor, causes the at least one processor to also:

[0120] Generate a three-dimensional representation of the hypervolume, the hypervolume comprising at least the attachment of the first robot, at least a portion of the hypervolume extending beyond the corresponding peripheral dimensions of the attachment of the first robot to include one or more cables physically coupled to the attachment of the first robot.

[0121] Example 4. The system according to Example 3, wherein, in order to generate a three-dimensional representation of a very large volume, the instructions, when executed by the at least one processor, cause the at least one processor to:

[0122] The three-dimensional representation of the attachment of the first robot is modified to increase at least one dimension of the attachment at at least one location on the attachment.

[0123] Example 5. The system according to Example 3, wherein, in order to generate a three-dimensional representation of a very large volume, the instructions, when executed by the at least one processor, cause the at least one processor to:

[0124] The three-dimensional representation of the attachment of the first robot is modified to increase at least one dimension of the attachment along a portion of the attachment, and at least one cable extends along said portion of the attachment.

[0125] Example 6. The system according to Example 5, wherein, in order to generate a three-dimensional representation of a very large volume, the instructions, when executed by the at least one processor, cause the at least one processor to:

[0126] No increase in any dimension of the attachment of the first robot in the three-dimensional representation of the attachment along which at least one cable does not extend.

[0127] Example 7. The system according to Example 3, wherein, in order to generate a three-dimensional representation of a very large volume, the instructions, when executed by the at least one processor, cause the at least one processor to:

[0128] The ultra-large volume three-dimensional representation is generated at least in part based on the position of at least one cable relative to at least a portion of the attachment of the first robot, taking gravity into account.

[0129] Example 8. The system according to Example 3, wherein, in order to generate a three-dimensional representation of a very large volume, the instructions, when executed by the at least one processor, cause the at least one processor to:

[0130] The three-dimensional representation of the ultra-large volume is generated based at least in part on the orientation of at least a portion of the attachment of the first robot, taking into account gravity, and the slack of at least one cable.

[0131] Example 9. The system according to Example 3, wherein, in order to generate a three-dimensional representation of a very large volume, the instructions, when executed by the at least one processor, cause the at least one processor to:

[0132] A bounding box representation is generated based on the size set of the first robot's attachments and the boundary buffer specification set.

[0133] Example 10. The system according to Example 3, wherein, in order to generate a three-dimensional representation of a very large volume, the instructions, when executed by the at least one processor, cause the at least one processor to:

[0134] The ultra-large volume three-dimensional representation is generated at least in part based on the current set of joint positions of the first robot.

[0135] Example 11. According to the system described in Example 3, in order to generate a three-dimensional representation of a very large volume, the instructions, when executed by the at least one processor, cause the at least one processor to:

[0136] Generate a three-dimensional representation of a massive volume comprising the base of the first robot, at least two attachments of the first robot, the end effector of the first robot, and at least two cables coupled to move with one or more of the attachments.

[0137] Example 12. A system according to any one of Examples 3 to 11, wherein, in order to generate a three-dimensional representation of a very large volume, the instructions, when executed by the at least one processor, cause the at least one processor to:

[0138] The three-dimensional representation of the ultra-large volume is generated based at least in part on the geometry of at least one cable.

[0139] Example 13. The system according to Example 12, wherein, in order to generate a three-dimensional representation of a very large volume, the instructions, when executed by the at least one processor, cause the at least one processor to:

[0140] The ultra-large volume three-dimensional representation is generated based at least in part on the velocity of at least a portion of the attachment of the first robot.

[0141] Example 14. The system according to Example 1 further includes:

[0142] At least one sensor that captures one or more physical characteristics of the three-dimensional operating environment during operation.

[0143] Example 15. The system according to Example 14, wherein the instructions, when executed by the at least one processor, cause the at least one processor to also:

[0144] A digital representation of the three-dimensional operating environment is generated based on one or more physical characteristics of the three-dimensional operating environment captured by the at least one sensor.

[0145] Example 16. The system according to Example 14, wherein, in order to generate a digital representation of the three-dimensional operating environment, the instructions, when executed by the at least one processor, cause the at least one processor to:

[0146] Generate at least one of the following: point cloud, voxel mesh, surface map, grid, occupied mesh, k-ary tree, Euclidean distance field representation, hierarchical data structure, or non-hierarchical data structure.

[0147] Example 17. The system according to Example 14, wherein, in order to generate a digital representation of the three-dimensional operating environment, the instructions, when executed by the at least one processor, cause the at least one processor to:

[0148] The three-dimensional operating environment is represented as a plurality of voxels, wherein each voxel includes at least one corresponding occupancy value, the at least one corresponding occupancy value representing the occupancy status of the three-dimensional operating environment as at least one of occupied or unoccupied.

[0149] Example 18. The system according to Example 1, wherein, in order to identify one or more elements representing one or more physical objects located in the three-dimensional operating environment in a digital representation of the three-dimensional operating environment, the instructions, when executed by the at least one processor, cause the at least one processor to:

[0150] In the digital representation of the three-dimensional operating environment, one or more groups of voxels are identified, which are consecutive and each has a corresponding occupancy value representing the occupancy status.

[0151] Example 19. The system according to Example 18, wherein, in order to determine which physical objects—if any—are entirely located within the massive 3D representation of the digital representation of the 3D operating environment, the instructions, when executed by the at least one processor, cause the at least one processor to:

[0152] For each of the group or groups of voxels in the digital representation of the three-dimensional operating environment that has been identified as representing a physical object, determine whether the group of voxels crosses the boundary of the three-dimensional representation of the hypervolume.

[0153] Example 20. The system according to Example 18, wherein, in order to set one or more occupancy values ​​in the digital representation of the three-dimensional operating environment to represent the volume corresponding to the respective physical object as unoccupied, the instruction, when executed by the at least one processor, causes the at least one processor to:

[0154] For each voxel in the digital representation of the three-dimensional operating environment that corresponds to any object that is completely located within the three-dimensional representation of the hypervolume, the corresponding voxel's occupancy value is set to indicate that it is not occupied.

[0155] Example 21. The system according to Example 1, wherein the instructions, when executed by the at least one processor, cause the at least one processor to:

[0156] During the pre-run period

[0157] Repeatedly move at least a portion of the first robot having at least one cable;

[0158] Capture data representing the volume occupied by the at least one cable during each movement of said portion of the first robot; and

[0159] Generate a digital model of at least one of the cables.

[0160] Example 22, the system according to Example 1, wherein the instructions, when executed by the at least one processor, cause the at least one processor to:

[0161] Determine which physical objects—if any—are represented in the digital representation of the three-dimensional operating environment entirely within a 3D representation of a hypervolume, the hypervolume comprising at least an attachment of the second robot, at least a portion of the hypervolume extending beyond the corresponding peripheral dimensions of at least the attachment of the second robot to include one or more cables physically coupled to the attachment of the second robot; and

[0162] For any physical object determined to be entirely within the three-dimensional representation of the oversized volume, one or more occupancy values ​​are set in the digital representation of the three-dimensional operating environment to represent the volume corresponding to the physical object as unoccupied, thereby providing a filtered representation of the three-dimensional operating environment in which the oversized volume containing at least the robot attachment of the second robot and at least one cable is not indicated as an obstacle.

[0163] Example 23. A method for operating an operating system, the system comprising at least one processor and at least one non-transitory processor-readable medium, the at least one non-transitory processor-readable medium being communicatively coupled to the at least one processor and storing processor-executable instructions, the method comprising:

[0164] Identify one or more elements representing one or more physical objects located in the three-dimensional operating environment in a digital representation of the three-dimensional operating environment;

[0165] Determine which physical objects—if any—are represented in the digital representation of the three-dimensional operating environment entirely within a 3D representation of a hypervolume, the hypervolume comprising at least an attachment of the first robot, at least a portion of the hypervolume extending beyond the corresponding peripheral dimensions of at least the attachment of the first robot to include one or more cables physically coupled to the attachment of the first robot; and

[0166] For any physical object determined to be entirely within the three-dimensional representation of the oversized volume, one or more occupancy values ​​are set in the digital representation of the three-dimensional operating environment to represent the volume corresponding to the physical object as unoccupied, thereby providing a filtered representation of the three-dimensional operating environment in which the oversized volume containing at least the robot attachment of the first robot and at least one cable is not indicated as an obstacle.

[0167] Example 24, the method described in Example 23 further includes:

[0168] The motion planning of the first robot is performed using a filtered representation of the three-dimensional operating environment.

[0169] Example 25, the method described in Example 23 further includes:

[0170] Generate a three-dimensional representation of the hypervolume, the hypervolume comprising at least the attachment of the first robot, at least a portion of the hypervolume extending beyond the corresponding peripheral dimensions of the attachment of the first robot to include one or more cables physically coupled to the attachment of the first robot.

[0171] Example 26: According to the method described in Example 25, generating a three-dimensional representation of a very large volume includes:

[0172] The three-dimensional representation of the attachment of the first robot is modified to increase at least one dimension of the attachment at at least one location on the attachment.

[0173] Example 27: According to the method described in Example 25, generating a three-dimensional representation of a very large volume includes:

[0174] The three-dimensional representation of the attachment of the first robot is modified to increase at least one dimension of the attachment along a portion of the attachment, and at least one cable extends along said portion of the attachment.

[0175] Example 28, according to the method described in Example 27, wherein generating a three-dimensional representation of a very large volume includes:

[0176] No increase in any dimension of the attachment of the first robot in the three-dimensional representation of the attachment along which at least one cable does not extend.

[0177] Example 29, according to the method described in Example 25, wherein generating a three-dimensional representation of a very large volume includes:

[0178] The ultra-large volume three-dimensional representation is generated at least in part based on the position of at least one cable relative to at least a portion of the attachment of the first robot, taking gravity into account.

[0179] Example 30: According to the method described in Example 25, generating a three-dimensional representation of a very large volume includes:

[0180] The three-dimensional representation of the ultra-large volume is generated based at least in part on the orientation of at least a portion of the attachment of the first robot, taking into account gravity, and the slack of at least one cable.

[0181] Example 31, according to the method described in Example 25, wherein generating a three-dimensional representation of a very large volume includes:

[0182] A bounding box representation is generated based on the size set of the first robot's attachments and the boundary buffer specification set.

[0183] Example 32, according to the method described in Example 25, wherein generating a three-dimensional representation of a very large volume includes:

[0184] The ultra-large volume three-dimensional representation is generated at least in part based on the current set of joint positions of the first robot.

[0185] Example 33, according to the method described in Example 25, wherein generating a three-dimensional representation of a very large volume includes:

[0186] Generate a three-dimensional representation of a massive volume comprising the base of the first robot, at least two attachments of the first robot, the end effector of the first robot, and at least two cables coupled to move with one or more of the attachments.

[0187] Example 34, the method according to any one of Examples 25 to 33, wherein generating a three-dimensional representation of a very large volume includes:

[0188] The three-dimensional representation of the ultra-large volume is generated based at least in part on the geometry of at least one cable.

[0189] Example 35, according to the method described in Example 34, wherein generating a three-dimensional representation of a very large volume includes:

[0190] The ultra-large volume three-dimensional representation is generated based at least in part on the velocity of at least a portion of the attachment of the first robot.

[0191] Example 36. The method described in Example 23 further includes:

[0192] One or more physical characteristics of the three-dimensional operating environment are captured via at least one sensor.

[0193] Example 37. The method described in Example 36 further includes:

[0194] A digital representation of the three-dimensional operating environment is generated based on one or more physical characteristics of the three-dimensional operating environment captured by the at least one sensor.

[0195] Example 38, according to the method described in Example 36, wherein generating the digital representation of the three-dimensional operating environment includes:

[0196] Generate at least one of the following: point cloud, voxel mesh, surface map, grid, occupied mesh, k-ary tree, Euclidean distance field representation, hierarchical data structure, or non-hierarchical data structure.

[0197] Example 39. According to the method described in Example 36, generating the digital representation of the three-dimensional operating environment includes:

[0198] The three-dimensional operating environment is represented as a plurality of voxels, wherein each voxel includes at least one corresponding occupancy value, the at least one corresponding occupancy value representing the occupancy status of the three-dimensional operating environment as at least one of occupied or unoccupied.

[0199] Example 40, the method according to Example 23, wherein identifying one or more elements representing one or more physical objects located in the three-dimensional operating environment in the digital representation of the three-dimensional operating environment includes:

[0200] In the digital representation of the three-dimensional operating environment, one or more groups of voxels are identified, which are consecutive and each has a corresponding occupancy value representing the occupancy status.

[0201] Example 41, according to the method of Example 40, wherein determining which physical objects—if any—are entirely located within the large-volume three-dimensional representation of the digital representation of the three-dimensional operating environment comprises:

[0202] For each of the group or groups of voxels in the digital representation of the three-dimensional operating environment that has been identified as representing a physical object, determine whether the group of voxels crosses the boundary of the three-dimensional representation of the hypervolume.

[0203] Example 42, according to the method of Example 40, wherein setting one or more occupancy values ​​in the digital representation of the three-dimensional operating environment to represent the volume corresponding to the corresponding physical object as unoccupied includes:

[0204] For each voxel in the digital representation of the three-dimensional operating environment that corresponds to any object that is completely located within the three-dimensional representation of the hypervolume, the corresponding occupancy value of the corresponding voxel is set to indicate that it is not occupied.

[0205] Example 43, the method described in Example 23 further includes:

[0206] During the pre-run period

[0207] Repeatedly move at least a portion of the first robot having at least one cable;

[0208] Capture data representing the volume occupied by the at least one cable during each movement of said portion of the first robot; and

[0209] Generate a digital model of at least one of the cables.

[0210] Example 44, the method described in Example 23 further includes:

[0211] Determine which physical objects—if any—are represented in the digital representation of the three-dimensional operating environment entirely within a 3D representation of a hypervolume, the hypervolume comprising at least an attachment of the second robot, at least a portion of the hypervolume extending beyond the corresponding peripheral dimensions of at least the attachment of the second robot to include one or more cables physically coupled to the attachment of the second robot; and

[0212] For any physical object determined to be entirely within the three-dimensional representation of the oversized volume, one or more occupancy values ​​are set in the digital representation of the three-dimensional operating environment to represent the volume corresponding to the physical object as unoccupied, thereby providing a filtered representation of the three-dimensional operating environment in which the oversized volume containing at least the robot attachment of the second robot and at least one cable is not indicated as an obstacle.

[0213] Example 45. A system comprising:

[0214] At least one processor;

[0215] At least one non-transitory processor-readable medium communicatively coupled to the at least one processor and storing processor-executable instructions that, when executed by the at least one processor, cause the at least one processor to:

[0216] Identify one or more elements representing one or more physical objects located in the three-dimensional operating environment in a digital representation of the three-dimensional operating environment;

[0217] Determine which physical objects—if any—are represented in the digital representation of the three-dimensional operating environment entirely within a three-dimensional representation of a hypervolume, the hypervolume comprising at least the attachments of the first robot, at least a portion of the hypervolume extending beyond the corresponding peripheral dimensions of at least the attachments of the first robot to include one or more structures physically coupled to the attachments of the first robot; and

[0218] For any physical object determined to be entirely within the three-dimensional representation of the oversized volume, one or more occupancy values ​​are set in the digital representation of the three-dimensional operating environment to represent the volume corresponding to the physical object as unoccupied, thereby providing a filtered representation of the three-dimensional operating environment in which the oversized volume containing at least the robot attachment of the first robot and at least one structure is not indicated as an obstacle.

[0219] Example 46, the system according to Example 45, wherein the one or more structures are cables.

[0220] Example 47, the system according to Example 45, wherein the one or more structures are sensors or cable attachments.

[0221] in conclusion

[0222] The foregoing detailed description has illustrated various embodiments of the device and / or process using block diagrams, schematic diagrams, and examples. Where such block diagrams, schematic diagrams, and examples contain one or more functions and / or operations, those skilled in the art will understand that each function and / or operation in such block diagrams, flowcharts, or examples can be implemented individually and / or collectively by various hardware, software, firmware, or virtually any combination thereof. In one embodiment, this subject matter can be implemented by Boolean circuits, application-specific integrated circuits (ASICs), and / or FPGAs. However, those skilled in the art will recognize that all or part of the embodiments disclosed herein can be implemented in standard integrated circuits in various different ways as one or more computer programs running on one or more computers (e.g., as one or more programs running on one or more computer systems), as one or more programs running on one or more controllers (e.g., microcontrollers), as one or more programs running on one or more processors (e.g., microprocessors), as firmware, or as any combination thereof, and that designing circuits and / or writing software and / or firmware code in accordance with this disclosure will be within the skill of those skilled in the art.

[0223] Those skilled in the art will recognize that many of the methods or algorithms described herein may employ additional actions, omit some actions, and / or perform actions in a different order than specified.

[0224] Furthermore, those skilled in the art will understand that the mechanisms taught herein can be implemented in hardware, such as with one or more FPGAs or ASICs.

[0225] While typically described in terms of voxel space and voxels, various other forms can be used to represent robots, obstacles, and target objects. For example, octrees, box sets, or Euclidean distance fields (EDFs) can be used to efficiently represent a wide range of geometries.

[0226] An octree is a hierarchical data structure used to store voxel occupancy data. The tree structure allows collision detection to descend only as needed into the tree's inherent hierarchical structure, thus improving the computational efficiency of the collision detection process. One advantage of octrees is their inherent hierarchical structure. Of course, other hierarchical data structures may also exist that are suitable.

[0227] Boundary sets can take various forms, such as axis-aligned bounding box (AABB) trees, oriented (non-axis-aligned) bounding box trees, or sphere trees. It's worth noting that the leaves of any of these tree-like data structures can have a different shape than the other nodes in the data structure; for example, all nodes can be AABBs, except for the root node, which can take the form of a triangular mesh. The choice of boundary volume representation involves a trade-off between the latency required to construct the corresponding data structure and the latency of computational conflicts, including, for example, how many tree traversals are needed before collision detection can be completed. For example, using spheres as boundary volumes facilitates fast comparisons (i.e., determining whether spheres overlap is computationally easy). For example, as discussed elsewhere in this paper, using a k-ary sphere tree to represent each link of the robot may be preferred. In some implementations, voxels representing the environment can be grouped together into AABB sets. This can significantly simplify the voxel data, making it computationally faster and more memory-efficient in some cases than using octrees.

[0228] EDF discretizes the workspace into a 3D mesh of voxels, and the value of each voxel encodes the distance to the nearest obstacle in that workspace. The workspace contains all the space that the robot can reach in any pose.

[0229] A robot can be modeled as a tree of links connected by joints. For a single robot attachment or arm, this "tree" is typically unary, but can be more general, for example, having links with two or more child links.

[0230] For obstacles that will occupy a consistent or constant volume in the environment throughout the entire runtime, and where the volume to be occupied at the configuration time is known, it is more preferable to represent those permanent obstacles using an Euclidean distance field.

[0231] The various embodiments described above can be combined to provide other embodiments. All commonly assigned U.S. patent applications, U.S. patent applications, foreign patents, and foreign patent applications mentioned in and / or listed in the application data sheet are incorporated herein by reference in their entirety, including but not limited to: International Patent Application No. PCT / US2017 / 036880, filed June 9, 2017, entitled “MOTION PLANNING FOR AUTONOMOUS VEHICLES AND RECONFIGURABLE MOTION PLANNING PROCESSORS”; International Patent Application Publication No. WO2016 / 122840, filed January 5, 2016, entitled “SPECIALIZED ROBOT MOTION PLANNING HARDWARE AND METHODS OF MAKING AND USING SAME”; and International Patent Application No. WO2016 / 122840, filed January 12, 2018, entitled “APPARATUS, METHOD AND ARTICLE TOFACILITATE MOTION PLANNING OF AN AUTONOMOUS VEHICLE IN AN ENVIRONMENT HAVINGDYNAMIC”. U.S. Patent Application No. 62 / 616,783 entitled “OBJECTS”; U.S. Patent Application No. 62 / 626,939 entitled “MOTION PLANNING OF A ROBOT STORING A DISCRETIZED ENVIRONMENT ON ONE OR MORE PROCESSORS AND IMPROVED OPERATION OF SAME”, filed February 6, 2018; U.S. Patent Application No. 62 / 722,067 entitled “COLLISION DETECTION USEFUL IN MOTION PLANNING FOR ROBOTICS”, filed August 23, 2018; International Patent Application No. PCT / US2019 / 045270 entitled “COLLISION DETECTION USEFUL IN MOTION PLANNING FOR ROBOTICS”, filed August 6, 2019; and International Patent Application No. PCT / US2019 / 045270 entitled “CONFIGURATION OF ROBOTS IN”, filed January 22, 2020. U.S. Patent Application No. 62 / 964,405, “MULTI-ROBOTOPERATIONAL ENVIRONMENT”;And U.S. Patent Application No. 62 / 991,487, filed March 18, 2020, entitled "DIGITAL REPRESENTATIONS OF ROBOT OPERATIONAL ENVIRONMENT, USEVUL INMOTION PLANNING FOR ROBOTS". These and other changes can be made to the embodiments based on the above detailed description.

[0232] Generally, the terminology used in the appended claims should not be construed as limiting the claims to the specific embodiments disclosed in the specification and claims, but should be interpreted to include all possible embodiments and the full scope of equivalents conferred by such claims. Therefore, the claims are not limited by this disclosure.

Claims

1. A system for robot motion planning, comprising: At least one processor; At least one non-transitory processor-readable medium communicatively coupled to the at least one processor and storing processor-executable instructions that, when executed by the at least one processor, cause the at least one processor to: Identify one or more elements representing one or more physical objects located in the three-dimensional operating environment in a digital representation of the three-dimensional operating environment; Determine which physical objects represented in the digital representation of the three-dimensional operating environment are entirely located within a three-dimensional representation of a hypervolume, the hypervolume comprising attachments of a first robot and one or more cables physically coupled to the attachments of the first robot, at least a portion of the hypervolume extending beyond at least the corresponding peripheral dimension of the attachments of the first robot to include the one or more cables physically coupled to the attachments of the first robot, wherein the attachments are formed by an assembly of links and joints of the first robot; as well as For any physical object determined to be entirely within the three-dimensional representation of the oversized volume, one or more occupancy values ​​are set in the digital representation of the three-dimensional operating environment to represent the volume corresponding to the physical object as unoccupied, thereby providing a filtered representation of the three-dimensional operating environment in which the oversized volume containing the robot attachment of the first robot and at least one cable is not indicated as an obstacle.

2. The system according to claim 1, wherein, When the instruction is executed by the at least one processor, it causes the at least one processor to also: The motion planning of the first robot is performed using a filtered representation of the three-dimensional operating environment.

3. The system according to claim 1, wherein, When the instruction is executed by the at least one processor, it causes the at least one processor to also: Generate a three-dimensional representation of the ultra-large volume.

4. The system according to claim 3, wherein, To generate a three-dimensional representation of a very large volume, the instructions, when executed by the at least one processor, cause the at least one processor to: The three-dimensional representation of the attachment of the first robot is modified to increase at least one dimension of the attachment at at least one location on the attachment.

5. The system according to claim 3, wherein, To generate a three-dimensional representation of a very large volume, the instructions, when executed by the at least one processor, cause the at least one processor to: The three-dimensional representation of the attachment of the first robot is modified to increase at least one dimension of the attachment along a portion of the attachment, the at least one cable extending along said portion of the attachment.

6. The system according to claim 5, wherein, To generate a three-dimensional representation of a very large volume, the instructions, when executed by the at least one processor, cause the at least one processor to: No increase in any dimension of the attachment of the first robot in the three-dimensional representation of the at least one cable not extending along it.

7. The system according to claim 3, wherein, To generate a three-dimensional representation of a very large volume, the instructions, when executed by the at least one processor, cause the at least one processor to: A three-dimensional representation of the ultra-large volume is generated based at least on the position of the at least one cable relative to at least a portion of the attachment of the first robot, taking gravity into account.

8. The system according to claim 3, wherein, To generate a three-dimensional representation of a very large volume, the instructions, when executed by the at least one processor, cause the at least one processor to: A three-dimensional representation of the ultra-large volume is generated based at least on the orientation of at least a portion of the attachment of the first robot, taking gravity into account, and the slack of at least one cable.

9. The system according to claim 3, wherein, To generate a three-dimensional representation of a very large volume, the instructions, when executed by the at least one processor, cause the at least one processor to: A bounding box representation is generated based on a set of dimensions of the attachments of the first robot and a set of boundary buffer specifications defined by offset from the periphery of the first robot.

10. The system according to claim 3, wherein, To generate a three-dimensional representation of a very large volume, the instructions, when executed by the at least one processor, cause the at least one processor to: A three-dimensional representation of a very large volume is generated based at least on the current set of joint positions of the first robot.

11. The system according to claim 3, wherein, To generate a three-dimensional representation of a very large volume, the instructions, when executed by the at least one processor, cause the at least one processor to: Generate a three-dimensional representation of a massive volume comprising the base of the first robot, at least two attachments of the first robot, the end effector of the first robot, and at least two cables coupled to move with one or more of the attachments.

12. The system according to any one of claims 3 to 11, wherein, To generate a three-dimensional representation of a very large volume, the instructions, when executed by the at least one processor, cause the at least one processor to: A three-dimensional representation of the ultra-large volume is generated based at least on the geometry of the at least one cable.

13. The system according to claim 12, wherein, To generate a three-dimensional representation of a very large volume, the instructions, when executed by the at least one processor, cause the at least one processor to: A large-volume 3D representation is generated based on at least a portion of the velocity of the attachment of the first robot.

14. The system according to claim 1, further comprising: At least one sensor that captures one or more physical characteristics of the three-dimensional operating environment during operation.

15. The system according to claim 14, wherein, When the instruction is executed by the at least one processor, it causes the at least one processor to also: A digital representation of the three-dimensional operating environment is generated based on one or more physical characteristics of the three-dimensional operating environment captured by the at least one sensor.

16. The system according to claim 14, wherein, In order to generate a digital representation of the three-dimensional operating environment, the instructions, when executed by the at least one processor, cause the at least one processor to: Generate at least one of a hierarchical data structure or a non-hierarchical data structure.

17. The system according to claim 16, wherein, The hierarchical data structure is a k-ary tree.

18. The system according to claim 16, wherein, The non-hierarchical data structure is an Euclidean distance field.

19. The system according to claim 14, wherein, In order to generate a digital representation of the three-dimensional operating environment, the instructions, when executed by the at least one processor, cause the at least one processor to: Generate point clouds.

20. The system according to claim 14, wherein, In order to generate a digital representation of the three-dimensional operating environment, the instructions, when executed by the at least one processor, cause the at least one processor to: Generate a voxel mesh.

21. The system according to claim 14, wherein, In order to generate a digital representation of the three-dimensional operating environment, the instructions, when executed by the at least one processor, cause the at least one processor to: Generate a surface map.

22. The system according to claim 14, wherein, In order to generate a digital representation of the three-dimensional operating environment, the instructions, when executed by the at least one processor, cause the at least one processor to: Generate an occupied grid.

23. The system according to claim 14, wherein, In order to generate a digital representation of the three-dimensional operating environment, the instructions, when executed by the at least one processor, cause the at least one processor to: The three-dimensional operating environment is represented as a plurality of voxels, wherein each voxel includes at least one corresponding occupancy value, the at least one corresponding occupancy value representing the occupancy status of the three-dimensional operating environment as occupied or unoccupied.

24. The system according to claim 1, wherein, In order to identify one or more elements representing one or more physical objects located in the three-dimensional operating environment in a digital representation of the three-dimensional operating environment, the instructions, when executed by the at least one processor, cause the at least one processor to: In the digital representation of the three-dimensional operating environment, one or more sets of voxels are identified, the set of voxels being continuous, wherein the set of voxels has a corresponding occupancy value representing an occupied occupancy status, or each of the more than one set of voxels has a corresponding occupancy value representing an occupied occupancy status.

25. The system according to claim 24, wherein, In order to determine which physical objects, represented in the digital representation of the three-dimensional operating environment, are entirely located within the massive three-dimensional representation, the instructions, when executed by the at least one processor, cause the at least one processor to: For each set of voxels in the digital representation of the three-dimensional operating environment that has been identified as representing a physical object, or for each of the more than one set of voxels in the digital representation of the three-dimensional operating environment that has been identified as representing a physical object, determine whether the set of voxels crosses the boundary of the three-dimensional representation of the hypervolume.

26. The system according to claim 24, wherein, In order to set one or more occupancy values ​​in the digital representation of the three-dimensional operating environment to represent the volume corresponding to the corresponding physical object as unoccupied, the instruction, when executed by the at least one processor, causes the at least one processor to: For each voxel in the digital representation of the three-dimensional operating environment and corresponding to any object that is entirely located within the three-dimensional representation of the hypervolume, the corresponding voxel's occupancy value is set to indicate that it is not occupied.

27. The system according to claim 1, wherein, When the instruction is executed by the at least one processor, the at least one processor causes the at least one processor to: During the pre-run period Repeatedly move at least a portion of the first robot having at least one cable; Capture data representing the volume occupied by the at least one cable during each movement of the portion of the first robot; as well as Generate a digital model of the at least one cable.

28. The system according to claim 1, wherein, When the instruction is executed by the at least one processor, the at least one processor causes the at least one processor to: Determine which physical objects represented in the digital representation of the three-dimensional operating environment are entirely located within the three-dimensional representation of a hypervolume, the hypervolume containing at least an attachment of the second robot, at least a portion of the hypervolume extending beyond the corresponding peripheral dimensions of at least the attachment of the second robot to include one or more cables physically coupled to the attachment of the second robot. as well as For any physical object determined to be entirely within the three-dimensional representation of the oversized volume, one or more occupancy values ​​are set in the digital representation of the three-dimensional operating environment to represent the volume corresponding to the respective physical object as unoccupied, thereby providing a filtered representation of the three-dimensional operating environment in which the oversized volume containing at least the robot attachment of the second robot and the at least one cable is not indicated as an obstacle.

29. A method of operating a system for robot motion planning, the system comprising at least one processor and at least one non-transitory processor-readable medium, the at least one non-transitory processor-readable medium being communicatively coupled to the at least one processor and storing processor-executable instructions, the method comprising: Identify one or more elements representing one or more physical objects located in the three-dimensional operating environment in a digital representation of the three-dimensional operating environment; Determine which physical objects represented in the digital representation of the three-dimensional operating environment are entirely located within a three-dimensional representation of a hypervolume, the hypervolume comprising attachments of a first robot and one or more cables physically coupled to the attachments of the first robot, at least a portion of the hypervolume extending beyond at least the corresponding peripheral dimension of the attachments of the first robot to include the one or more cables physically coupled to the attachments of the first robot, wherein the attachments are formed by an assembly of links and joints of the first robot; as well as For any physical object determined to be entirely within the three-dimensional representation of the oversized volume, one or more occupancy values ​​are set in the digital representation of the three-dimensional operating environment to represent the volume corresponding to the physical object as unoccupied, thereby providing a filtered representation of the three-dimensional operating environment in which the oversized volume containing the robot attachment of the first robot and at least one cable is not indicated as an obstacle.

30. The method of claim 29, further comprising: The motion planning of the first robot is performed using a filtered representation of the three-dimensional operating environment.

31. The method of claim 29, further comprising: Generate a three-dimensional representation of the ultra-large volume.

32. The method according to claim 31, wherein, Generating 3D representations of extremely large volumes includes: The three-dimensional representation of the attachment of the first robot is modified to increase at least one dimension of the attachment at at least one location on the attachment.

33. The method according to claim 31, wherein, Generating 3D representations of extremely large volumes includes: The three-dimensional representation of the attachment of the first robot is modified to increase at least one dimension of the attachment along a portion of the attachment, the at least one cable extending along said portion of the attachment.

34. The method according to claim 33, wherein, Generating 3D representations of extremely large volumes includes: No increase in any dimension of the attachment of the first robot in the three-dimensional representation of the at least one cable not extending along it.

35. The method according to claim 31, wherein, Generating 3D representations of extremely large volumes includes: A three-dimensional representation of the ultra-large volume is generated based at least on the position of the at least one cable relative to at least a portion of the attachment of the first robot, taking gravity into account.

36. The method according to claim 31, wherein, Generating 3D representations of extremely large volumes includes: A three-dimensional representation of the ultra-large volume is generated based at least on the orientation of at least a portion of the attachment of the first robot, taking gravity into account, and the slack of at least one cable.

37. The method according to claim 31, wherein, Generating 3D representations of extremely large volumes includes: A bounding box representation is generated based on a set of dimensions of the attachments of the first robot and a set of boundary buffer specifications defined by offset from the periphery of the first robot.

38. The method according to claim 31, wherein, Generating 3D representations of extremely large volumes includes: A three-dimensional representation of a very large volume is generated based at least on the current set of joint positions of the first robot.

39. The method according to claim 31, wherein, Generating 3D representations of extremely large volumes includes: Generate a three-dimensional representation of a massive volume comprising the base of the first robot, at least two attachments of the first robot, the end effector of the first robot, and at least two cables coupled to move with one or more of the attachments.

40. The method according to any one of claims 31 to 39, wherein, Generating 3D representations of extremely large volumes includes: A three-dimensional representation of the ultra-large volume is generated based at least on the geometry of the at least one cable.

41. The method according to claim 40, wherein, Generating 3D representations of extremely large volumes includes: A large-volume 3D representation is generated based on at least a portion of the velocity of the attachment of the first robot.

42. The method of claim 29, further comprising: One or more physical characteristics of the three-dimensional operating environment are captured via at least one sensor.

43. The method of claim 42, further comprising: A digital representation of the three-dimensional operating environment is generated based on one or more physical characteristics of the three-dimensional operating environment captured by the at least one sensor.

44. The method according to claim 42, wherein, The digital representation of the three-dimensional operating environment includes: Generate at least one of a hierarchical data structure or a non-hierarchical data structure.

45. The method according to claim 44, wherein, The hierarchical data structure is a k-ary tree.

46. ​​The method of claim 44, wherein, The non-hierarchical data structure is an Euclidean distance field.

47. The method according to claim 42, wherein, The digital representation of the three-dimensional operating environment includes: Generate point clouds.

48. The method according to claim 42, wherein, The digital representation of the three-dimensional operating environment includes: Generate a voxel mesh.

49. The method according to claim 42, wherein, The digital representation of the three-dimensional operating environment includes: Generate a surface map.

50. The method according to claim 42, wherein, The digital representation of the three-dimensional operating environment includes: Generate an occupied grid.

51. The method according to claim 42, wherein, The digital representation of the three-dimensional operating environment includes: The three-dimensional operating environment is represented as a plurality of voxels, wherein each voxel includes at least one corresponding occupancy value, the at least one corresponding occupancy value representing the occupancy status of the three-dimensional operating environment as occupied or unoccupied.

52. The method according to claim 29, wherein, Identifying one or more elements representing one or more physical objects located within the three-dimensional operating environment in the digital representation of the three-dimensional operating environment includes: In the digital representation of the three-dimensional operating environment, one or more sets of voxels are identified, the set of voxels being continuous, wherein the set of voxels has a corresponding occupancy value representing an occupied occupancy status, or each of the more than one set of voxels has a corresponding occupancy value representing an occupied occupancy status.

53. The method according to claim 52, wherein, Determining which physical objects in the digital representation of the three-dimensional operating environment are entirely located within the ultra-large volume three-dimensional representation includes: For each set of voxels in the digital representation of the three-dimensional operating environment that has been identified as representing a physical object, or for each of the more than one set of voxels in the digital representation of the three-dimensional operating environment that has been identified as representing a physical object, determine whether the set of voxels crosses the boundary of the three-dimensional representation of the hypervolume.

54. The method according to claim 52, wherein, Setting one or more occupancy values ​​in the digital representation of the three-dimensional operating environment to represent the volume corresponding to the corresponding physical object as unoccupied includes: For each voxel in the digital representation of the three-dimensional operating environment and corresponding to any object that is entirely located within the three-dimensional representation of the hypervolume represented in the digital representation of the three-dimensional operating environment, the corresponding occupancy value of the corresponding voxel is set to indicate that it is not occupied.

55. The method of claim 29, further comprising: During the pre-run period Repeatedly move at least a portion of the first robot having at least one cable; Capture data representing the volume occupied by the at least one cable during each movement of the portion of the first robot; as well as Generate a digital model of the at least one cable.

56. The method of claim 29, further comprising: Determine which physical objects represented in the digital representation of the three-dimensional operating environment are entirely located within the three-dimensional representation of a hypervolume, the hypervolume containing at least an attachment of the second robot, at least a portion of the hypervolume extending beyond the corresponding peripheral dimensions of at least the attachment of the second robot to include one or more cables physically coupled to the attachment of the second robot. as well as For any physical object determined to be entirely within the three-dimensional representation of the oversized volume, one or more occupancy values ​​are set in the digital representation of the three-dimensional operating environment to represent the volume corresponding to the physical object as unoccupied, thereby providing a filtered representation of the three-dimensional operating environment in which the oversized volume containing at least the robot attachment of the second robot and at least one cable is not indicated as an obstacle.

57. A system for robot motion planning, comprising: At least one processor; At least one non-transitory processor-readable medium communicatively coupled to the at least one processor and storing processor-executable instructions that, when executed by the at least one processor, cause the at least one processor to: Identify one or more elements representing one or more physical objects located in the three-dimensional operating environment in a digital representation of the three-dimensional operating environment; Determine which physical objects represented in the digital representation of the three-dimensional operating environment are entirely located within the three-dimensional representation of a hypervolume, the hypervolume comprising attachments of a first robot and one or more structures physically coupled to the attachments of the first robot, at least a portion of the hypervolume extending beyond at least the corresponding peripheral dimensions of the attachments of the first robot to include one or more structures physically coupled to the attachments of the first robot, wherein the attachments are formed by an assembly of links and joints of the first robot; as well as For any physical object determined to be entirely within the three-dimensional representation of the oversized volume, one or more occupancy values ​​are set in the digital representation of the three-dimensional operating environment to represent the volume corresponding to the physical object as unoccupied, thereby providing a filtered representation of the three-dimensional operating environment in which the oversized volume containing the robot attachments of the first robot and at least one structure is not indicated as an obstacle.

58. The system according to claim 57, wherein, The one or more structures are cables.

59. The system according to claim 57, wherein, The one or more structures are sensors or cable attachments.

Images