Systems and methods for the design, testing, and verification of robotic systems

Abstract

A system and method for designing, testing, and verifying a space robotic system includes a robotic manipulator, a dynamic system emulator that simulates a motion behavior response of the first space robotic system based on forces and moments measured by the first robotic manipulator during physical interaction between the first and second robotic manipulators, a second robotic manipulator that emulates the motion behavior of the second space robotic system, an arm controller that provides the robotic manipulator with a manipulator tip reference trajectory command generated based on the motion behavior response, and an arm mechanism that causes the robotic manipulator to emulate the motion behavior of the first space robotic system by tracking a trajectory based on the manipulator tip reference trajectory command.

Description

Technical Field

[0004] , , ,

[0005] , ,

[0001] The following generally relates to robotics and, more particularly, to systems and methods for designing, testing, and validating a robotic system for space that includes an emulation of robotic interaction with a freely flying spacecraft.

Background Art

[0002] With the advancement of automation and artificial intelligence, robotic systems are expected to become an essential part of future space development concepts, such as lunar gateway and on-orbit satellite servicing. There are many challenges in the design, testing, and validation of robotic systems for space, but one of the most difficult is to optimize the control system for zero-gravity (0G) performance and to evaluate the interaction of such a system with a freely flying payload or various work-site interfaces.

[0003] Therefore, improved systems and methods for designing, testing, and validating a robotic system for space that overcome at least some of the drawbacks of existing systems and methods are desired.

Summary of the Invention

Means for Solving the Problems

[0004] Systems and methods are provided for any one or more of the design, testing, and validation of a robotic system.

[0005] A system is provided for ground testing and operational planning of space robotic systems. The system includes a first robot test system comprising: a first robotic manipulator; a first dynamic system emulator configured to simulate the motion behavior response of a zero-gravity (0G) servicer robotic arm based on forces and moments measured by the first robotic manipulator during a physical interaction between the first robotic manipulator and a second robotic manipulator that emulates the motion behavior of a client space robotic system; a first arm controller configured to generate a first manipulator tip reference trajectory command based on the motion behavior response simulated by the first dynamic system emulator and to provide the first manipulator tip reference trajectory command to the first robotic manipulator; and a first arm mechanism within the first robotic manipulator configured to emulate the motion behavior of the 0G servicer robotic arm by tracking a first trajectory based on the first manipulator tip reference trajectory command.The system further includes a second robot test system comprising: a second robot manipulator; a second dynamic system emulator configured to simulate the motion behavior response of the client space robot system based on forces and moments measured by the second robot manipulator during a physical interaction between the second robot manipulator and the first robot manipulator which emulates the motion behavior of the 0G servicer robot arm; a second arm controller configured to generate a second manipulator tip reference trajectory command based on the motion behavior response simulated by the second dynamic system emulator and to provide the second manipulator tip reference trajectory command to the second robot manipulator; and a second arm mechanism within the second robot manipulator configured to emulate the motion behavior of the client space robot system by tracking a second trajectory based on the second manipulator tip reference trajectory command.

[0006] The system may further include a mixed reality (MR) system that is communicatively connected to the first robot test system and the second robot test system, and is configured to overlay a first virtual model of the 0G servicer robot arm onto the first robot manipulator and a second virtual model of the client space robot system onto the second robot manipulator.

[0007] The client space robot system emulated by the second robot test system may be a free-floating spacecraft.

[0008] A robot test system is provided for ground testing the performance of a space robot system. The robot test system includes a robot manipulator; a dynamic system emulator configured to simulate the motion behavior response of a first space robot system based on forces and moments measured by the first robot manipulator during a physical interaction between the first robot manipulator and a second robot manipulator that emulates the motion behavior of a second space robot system; an arm controller configured to generate a manipulator tip reference trajectory command based on the motion behavior response simulated by the dynamic system emulator and to provide the manipulator tip reference trajectory command to the robot manipulator; and an arm mechanism within the robot manipulator configured to emulate the motion behavior of the first space robot system by tracking a trajectory based on the manipulator tip reference trajectory command.

[0009] The first space robot system may also be a 0G servicer robot arm.

[0010] The first space robot system may be a client space robot system serviced by a servicer robot arm.

[0011] The aforementioned client space robot system may be a free-floating artificial satellite.

[0012] A method is provided for ground testing and verification of robotic interaction between two independent space robotic systems. The method includes the steps of: emulating the motion behavior of a zero-gravity (0G) servicer robotic arm with a first robotic arm using a first dynamic system emulator in a first robotic arm configured to capture the dynamics of the manipulator tip of the zero-gravity (0G) servicer robotic arm; emulating the motion behavior of a free-floating satellite with a second robotic arm using a second dynamic system emulator in a second robotic arm configured to capture the dynamics of a free-floating satellite; causing the first robotic arm and the second robotic arm to physically interact while the motion behavior is being emulated; and determining the forces and moments experienced by the first robotic arm based on the physical interaction via a force moment sensor (FMS) in the first robotic arm. The steps include: measuring; measuring the force and moment experienced by the second robot arm based on the physical interaction via an FMS in the second robot arm; simulating the response of the first robot arm to the measured force and moment using the first dynamic system emulator; simulating the response of the second robot arm to the measured force and moment using the second dynamic system emulator; tracking the trajectory with the first robot arm based on the simulated response output by the first dynamic system emulator; and tracking the trajectory with the second robot arm based on the simulated response output by the second dynamic system emulator.

[0013] The method may further include the step of providing initial conditions to the first dynamic system emulator and the second dynamic system emulator to determine the initial state of the emulated 0G servicer robot arm and the free-floating satellite.

[0014] The initial conditions provided to the second dynamic system emulator may include the initial drift of the emulated free-floating satellite.

[0015] The method may further include the steps of overlaying a virtual model of the emulated 0G servicer robot arm onto the first robot arm, and overlaying a virtual model of the emulated free-floating satellite onto the second robot arm.

[0016] The method may further include the step of gravity offloading the first robot arm to 0G via a passive offloading mechanism.

[0017] The method may further include the step of gravity offloading the first robot arm to 0G via an active offloading mechanism comprising a third robot manipulator.

[0018] The method may further include the step of adjusting the mass, inertia, stiffness, or damping characteristics of the emulated free-floating satellite using the dynamic system emulator to emulate events of change in the mass, inertia, stiffness, or damping characteristics of the emulated spacecraft (or collectively, “dynamics change events”).

[0019] Other aspects and features will become apparent to those skilled in the art by considering the following description of some exemplary embodiments. [Brief explanation of the drawing]

[0020] The drawings contained herein are for illustrating various examples of the articles, methods, and apparatus described herein.

[0021] [Figure 1]FIG. 1 is a block diagram of a robot test system for emulating a space-based robot system to enable any one or more of the design, testing, and verification of a space-based robot system on the ground according to an embodiment.

[0022] [Figure 2] FIG. 2 is a block diagram of the robot test system of FIG. 1 configured to emulate a typical space robot manipulator according to an embodiment.

[0023] [Figure 3] FIG. 3 is a block diagram of the robot test system of FIG. 1 configured to emulate a free-floating spacecraft according to an embodiment.

[0024] [Figure 4] FIG. 4 is a block diagram of a system for designing, testing, or verifying a robot system and mission operation on the ground for use in space, the system including first and second examples of the robot system of FIG. 1 configured to emulate different dynamic systems according to an embodiment.

[0025] [Figure 5] FIG. 5 is a flowchart of a method for ground testing and verification of a space robot system including first and second interacting dynamic systems according to an embodiment.

[0026] [Figure 6] FIG. 6 is a block diagram of a system for designing, testing, and verifying a robot system for use in space according to an embodiment.

[0027] [Figure 7]Figure 7 is a block diagram of the system in Figure 6, further illustrating the interaction and system time delay of Mission Analysis Robot Kit I ("MARK I") and Mission Analysis Robot Kit II ("MARK II") according to one embodiment.

[0028] [Figure 8] Figure 8 is a block diagram illustrating the emulation layers in the MARK I and MARK II systems of the systems shown in Figures 6 and 7, according to one embodiment. [Modes for carrying out the invention]

[0029] The following describes various apparatuses or processes to illustrate embodiments of the claims. The embodiments described below are not intended to limit the embodiments of the claims, and the embodiments of the claims may cover processes or apparatuses other than those described below. The embodiments of the claims are not limited to apparatuses or processes having all the features of any one apparatus or process described below, nor are they limited to features common to multiple or all of the apparatuses described below.

[0030] One or more systems described herein may be implemented in a computer program running on a programmable computer, each having at least one processor, a data storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device. For example, but not limited to, the programmable computer may be a programmable logic unit, a mainframe computer, a server, and a personal computer, a cloud-based program or system, a laptop, a personal data assistance device, a mobile phone, a smartphone, or a tablet device.

[0031] Each program is preferably implemented in a high-level procedural or object-oriented programming and / or scripting language for communicating with a computer system. However, the program may be implemented in assembly language or machine code, if necessary. In any case, the language may be a compiled language or an interpreted language. Each such computer program is preferably stored in a storage medium or device readable by a general-purpose or purpose-specific programmable computer for configuring and operating the computer when the storage medium or device is read by the computer to perform the procedures described herein.

[0032] The description of embodiments having several components that communicate with each other does not mean that all such components are necessary. Rather, various optional components are described in order to illustrate the wide variety of possible embodiments of the present invention.

[0033] Furthermore, process steps, method steps, algorithms, etc., may be described in a sequential order (within this disclosure and / or claims), but such processes, methods, and algorithms may be configured to operate in an alternative order. In other words, any sequence or order of steps that may be described does not necessarily require the steps to be performed in that order. The steps of the processes described herein may be performed in any practical order. Furthermore, some steps may be performed simultaneously.

[0034] Where a single device or article is described herein, it will be readily apparent that multiple devices or articles (whether they work together or not) may be used in place of the single device / article. Similarly, where multiple devices or articles are described herein (whether they work together or not), it will be readily apparent that a single device / article may be used in place of multiple devices or articles.

[0035] The following relates to systems and methods for designing, testing, and verifying robotic systems for use and operation in space, including, in general, robotics, and more specifically, the design, testing, and verification of robotic systems for use and operation in space, including the emulation of robotic interactions with free-flying spacecraft.

[0036] Referring to Figure 1, what is shown there is a robot test system 100 according to one embodiment.

[0037] The robot test system 100 may be used to emulate a servicer robot system or a client robot system as part of a larger system for designing, testing, or verifying a robot system that includes two independent robot systems having independent control systems for performing collaborative tasks. Thus, the robot system 100 can be used, for example, as part of a ground test process to evaluate the performance and interaction of a robot system used in space.

[0038] For example, the robot test system 100 can be used as part of a ground test process to emulate a servicer robot system such as a servicer 0G robot manipulator.

[0039] The robot system 100 includes a robot manipulator (also called a robot arm in the case of a serial robot manipulator) 102 and a robot manipulator control subsystem 104.

[0040] Normally, during operation, the robot manipulator 102 and the robot manipulator control subsystem 104 communicate with each other.

[0041] The robot manipulator 102 includes an arm mechanism 106, a force moment sensor (FMS, or force torque sensor) 108, and an interfacing component 110.

[0042] The arm mechanism 106 includes an assembly of links and joints. Links are the rigid parts of the arm mechanism 106, and joints are connections between two connected links that allow for limited relative motion between the two links. The number of links and joints may vary depending on the application.

[0043] The arm mechanism 106 is configured to track a trajectory based on commands received from the arm controller (arm controller 112, described later). The arm mechanism 106 may also provide joint angle measurements to the robot manipulator control subsystem 104 (i.e., the arm controller) for processing.

[0044] The FMS108 is configured to measure the contact force and moment experienced by the robot manipulator 102. The FMS108 provides the force moment sensor measurements to the robot manipulator control subsystem 104.

[0045] The interfacing component 110 is typically a component located at or near the tip or free end of the robot manipulator 102 and is involved in the physical interface or engagement interaction with a complementary interfacing component on the second robot manipulator of the second independent robot system. Thus, the interfacing component 110 is used to physically engage the robot manipulator 102 with the second robot manipulator in a manner similar to how it would be performed in actual operation (i.e., between two emulated robot systems outside of the test situation).

[0046] In one example, the interfacing component 110 is half of the robot interface that participates in the capture and hardening operation, thereby allowing a first robotic manipulator 102 to harden a second robotic manipulator that emulates a second dynamic system (the robotic manipulator 102 may either capture or perform the capture). The capture and hardening operation may be intended to mimic the capture and hardening of a free-floating or free-flying spacecraft (e.g., a satellite) by a space robotic manipulator.

[0047] The robot manipulator control subsystem 104 includes control software 111 which includes an arm controller 112 and a dynamic system emulator 114.

[0048] The arm controller 112 provides the arm mechanism 106 with level commands for the arm and joints (e.g., command 122, described below). The level commands for the arm and joints may be provided based on one or more of the following: inputs from external sensors, outputs from other controllers, and automated commands. The arm controller 112 may be configured to perform force control by adjusting the contact force between the robot manipulator 102 and a second robot system / manipulator with which the robot manipulator 102 is interacting. The arm controller 112 may be configured to perform visual servo control by maintaining a constant standoff distance and matching the drift rate with a target satellite emulated by the second robot test system with which the robot test system 100 is interacting.

[0049] The dynamic system emulator 114 comprises a simulator that can be configured to capture the dynamics of a desired electromechanical system. The dynamic system emulator 114 may be incorporated into software that controls the reference trajectory of the arm mechanism 106. The dynamic system emulator 114 enables the tip (or a point of some resolution near the free end) of the robot arm 102 to reproduce the behavior of a desired dynamical system. This may include, for example, emulating the tip dynamics of a target manipulator that is different from (e.g., shorter / longer, lighter / heavier) the robot manipulator 102, or emulating the levitation dynamics of a free-floating spacecraft.

[0050] In one embodiment, the dynamics / kinematics of the emulated system (e.g., a space manipulator, a satellite) can be coded in the dynamic system emulator 114 through Simulink or a similar programming environment.

[0051] In one embodiment, the dynamic system emulator 114 may be configured to emulate a 0G robot arm configured to perform one or more service tasks ("servicer robot arm"). In another embodiment, the dynamic system emulator 114 may be configured to emulate a client space robot system, such as a robotic work site interface or a free-floating satellite. Generally, a client space robot system is a robotic system that is serviced by the servicer robot arm via robotic system interaction.

[0052] The dynamic system emulator 114 is configured to run a dynamic system model. The dynamic system model may represent a mathematical model of the target behavior of the tip of the robot manipulator 102.

[0053] In some cases, the dynamic system emulator 114 may be configured such that initial conditions can be provided to the dynamic system emulator 114. The initial conditions may be provided, for example, through user input data entered via a user interface. The initial conditions may then be used by the dynamic system emulator 114 to determine the initial state of the dynamic system to be simulated. For example, if the dynamic system emulator 114 is configured to emulate the dynamics of a free-floating satellite, the initial conditions may be provided to the dynamic system emulator 114 to assign an initial target drift rate to the robot manipulator 102.

[0054] Generally, during the operation of system 100, when the robot manipulator 102 interacts with the environment, the robot manipulator 102 experiences forces (contact forces) and moments 116 measured by FMS 108. The FMS measurements 118 are provided to the dynamic system emulator 114. The FMS measurements 118 are used to drive an internal simulation by the dynamic system emulator 114. The FMS measurements 118 may also be provided as disturbances to the dynamic system model simulation. The dynamic system emulator 114 uses the dynamic system model to simulate the system's response to forces and moments. The output of the internal simulation performed by the dynamic system emulator 114 is then tracked by the arm mechanism 106 of the robot manipulator 102. The system response determined by the dynamic system emulator 114 may be used to generate a reference trajectory for the arm mechanism 106 to track. For example, the dynamic system emulator 114 may generate the system response as an end-point reference command for the manipulator 102, and this command 120 may be transmitted to the arm controller 112. Subsequently, the reference command signal is tracked through the control operations of the arm and joint via arm and joint commands 122 (based on the output of the dynamic system emulator 114) transmitted from the arm controller 112 to the arm mechanism 106.

[0055] Referring now to Figure 2, which shows a robot test system 200 for use in a system for ground testing and verification of space-based robotic system interactions, according to one embodiment.

[0056] The robot test system 200 is an example of the robot test system 100 shown in Figure 1, and the dynamic electromechanical system being emulated is a servicer space robotic arm. Therefore, any of the following examples can be applied to system 100, and any of the above examples can be applied to system 200 with reference to system 100.

[0057] The robot system 200 can be used to replicate the dynamic performance of different manipulator designs and concepts (i.e., one or more "target manipulators") and can interact with a versatile dynamic emulator system (e.g., system 300 in Figure 3) that emulates a wide variety of free-flight payloads and work site interfaces.

[0058] In particular, the robot test system 200 is configured to emulate the tip dynamics of different manipulators. These different manipulators may be, for example, larger or smaller robot arms, or lighter or heavier robot arms. The different manipulators may also be 0G arms.

[0059] The robot manipulator 102 includes an end effector 202 connected to the arm mechanism 106. The end effector 202 is a peripheral device or tool connected to the free end of the arm mechanism 106 and is configured to interact with the environment (for example, other robotic systems such as the robotic test system shown in Figure 3, which will be described later).

[0060] The robot manipulator 102 further includes a probe 204, which is an interfacing component 110 of system 200. The probe 204 is configured to engage with the interfacing component 110 on a second robot test system (e.g., system 300 in Figure 3) to establish a physical connection between robot test system 200 and the second robot test system. The probe 204 may be further configured to perform visual tracking of a target on the second robot test system. The operation of the probe 204 is controlled by a probe controller 206 in the robot manipulator control subsystem 104.

[0061] The robot test system 200 further includes a gravity offload mechanism 208. The gravity offload is configured to gravity offload the robot manipulator 102 so that the robot manipulator 102 functions as a 0G robot manipulator.

[0062] The gravity offload mechanism 208 may be a passive gravity offload mechanism or an active gravity offload mechanism.

[0063] The dynamic system emulator 114 of the robot test system 200 is configured to capture and emulate the tip dynamics of the "target manipulator" (e.g., a 0G manipulator). The dynamic system emulator 114 is configured to capture the dynamics and control scheme of the target manipulator.

[0064] The robotic manipulator 102 may be a 7-degree-of-freedom (DOF) manipulator. The robotic manipulator 102 may be configured to be kinematically equivalent to the robotic arm of a known space manipulator system.

[0065] The robot manipulator 102 may be configured to use a roll-yaw-pitch-pitch-pitch-yaw-roll topology.

[0066] A gravity offload mechanism 208 may be used to counterbalance the entire weight of the arm 102. Passive counterbalancing of the arm 102 may be provided by redistributing the arm weight so that the center of gravity of each link falls on its joint drive axis, and balancing it with the weight of the end effector 202 in a cable pulley system.

[0067] The gravity offload mechanism 208 can ensure that the arm 102 operates in a near-weightless environment over a limited workspace. This behavior may be restricted to effectively make the arm a 6-degree-of-freedom arm by locking the first yaw joint of the arm 102, thereby suppressing rotation in the pitch plane.

[0068] The arm joint of the arm mechanism 106 may be configured to represent flight in terms of gear ratio, gearbox rigidity, and output torque.

[0069] The robot manipulator 102 may include one or more end effectors 202.

[0070] The robot test system 200 may offer superior payload handling capabilities compared to other ground-based robots, provided that the weight of the arm 102 is balanced by the gravity offload mechanism 208.

[0071] For example, the robotic test system 200 can be used to emulate a 0G manipulator configured to perform one or more of the following tasks using the configured dynamic system emulator 114: capturing incoming visitor transports and payloads, docking them (e.g., to the ISS), performing (complex) station maintenance and repair tasks, assembly and maintenance tasks, capturing (unmanned or manned) visitor transports and docking them to the station, repositioning modules on the station, moving along the station's outer structure to reach a desired location, performing maintenance and repair tasks, and satellite service tasks (i.e., space missions aimed at rendezvousing with existing spacecraft in need of support to provide life-extending services such as refueling, maintenance, and repair). Thus, the performance of such tasks can be tested on the ground using the robotic test system 200 (or other systems described herein).

[0072] Referring now to Figure 3, what is shown there is a robot test system 300 according to one embodiment.

[0073] Robot test system 300 is an example of robot test system 100 in Figure 1, where the dynamic electromechanical system to be emulated is a free-floating spacecraft (e.g., a satellite). Therefore, any of the following examples may be applied to system 100, and any of the above examples may be applied to system 300 with reference to system 100.

[0074] The dynamic system emulator 114 of system 300 is configured to emulate the dynamics of a free-floating satellite. The satellites emulated by the dynamic system emulator 114 may include controlled and uncontrolled satellites with various inertia and geometries.

[0075] The robot manipulator 102 may be configured to operate as a conventional 1G-compatible arm.

[0076] The robotic manipulator 102 includes a target 302, which is an interfacing component 110 of system 300. The target 302 is configured to interface with the interfacing component 110 on a second robotic test system (e.g., probe 204 of system 200 in Figure 2) and establish a physical connection between the robotic test system 300 and the second robotic test system. The target 302 may be the passive side or half of the robotic interface, with its active half provided by the second robotic test system interacting with system 300. The target 302 may include a visual target component to enable visual tracking by the second robotic test system.

[0077] The robot manipulator control subsystem 104 further includes an attitude control system 304. The attitude control system 304 is configured to function as an attitude control system for a satellite emulated by the robot test system 300. In this embodiment, the robot manipulator control subsystem 104 can emulate the internal control system of a client spacecraft and effectively test the interaction between the servicer robot and the actively controlled and operating client spacecraft.

[0078] In one embodiment, the robotic manipulator 102 may use a dynamic system emulator 114 to enable hardware-in-the-loop testing of satellite attitude control hardware (actuators and sensors) and software (flight code). In this scenario, the robotic manipulator 102 emulates the passive dynamics of the target satellite's dynamics as described herein. Instead of including virtualized actuator and sensor models, actual actuators (e.g., cold gas thrusters, control moment gyros, or reaction wheels) and sensors (e.g., accelerometers or attitude sensors) may be attached to the end of the arm 102 via a mechanical interface. Control forces and torques are read by the arm's force-torque sensor 108 and used to update the arm response to emulate the target satellite. This setup allows testing of the actual software and hardware of the satellite's attitude and position control subsystem.

[0079] Referring now to Figure 4, which shows a system 400 for testing and verifying space-based robotic system interactions on the ground between a servicer robotic system (e.g., a servicer robotic arm) and a client robotic system (e.g., a free-floating satellite, a robotic work site interface), according to one embodiment.

[0080] For example, system 400 may be used to evaluate and investigate the performance of a space robotic arm when it interacts with a dynamical system whose behavior cannot be easily reproduced by conventional methods.

[0081] System 400 includes a first robot test system 402 and a second robot test system 404. System 400 may further include a mixed reality system 406.

[0082] The first and second robot test systems 402 and 404 are different configuration examples of the robot test system 100 in Figure 1. Therefore, any of the following examples may be applied to system 100, and any of the above examples with reference to system 100 may be applied to system 400. For clarity, the components of the first and second robot test systems 402 are given the same reference numbers as in Figure 1, with the addition of "a" and "b" respectively.

[0083] In particular, the first robot test system 402 is an example of the robot test system 200 shown in Figure 2, and the second robot test system 404 is an example of the robot test system 300 shown in Figure 3.

[0084] The first robot test system 402 is configured to emulate a 0G servicer robot arm, and the second robot test system 404 is configured to emulate a client robot system in the form of a free-floating spacecraft (satellite).

[0085] In other embodiments, the second robot test system 404 may be configured to emulate different client-space robot systems, such as a robot work site interface.

[0086] In one embodiment, the first robot manipulator 102a may be a 7-degree-of-freedom, gravity-compensated (0G) robot arm, and the second robot manipulator 102b may be a 7-degree-of-freedom, 1G robot arm for dynamic system evaluation.

[0087] Generally, the first and second robotic test systems 402 and 404 are configured to physically interact with each other by emulating a servicer robotic arm and a free-floating spacecraft, respectively, so that the interaction between the two space-based robotic systems being emulated can be evaluated in a ground-based test environment.

[0088] In particular, the dynamic system emulator 114 executed by the first robot manipulator control subsystem 104a of the first robot test system 402 is configured so that the tip of the first robot manipulator 102a emulates the motion behavior of a 0G servicer robot arm. The dynamic system emulator 114 executed by the second robot manipulator control subsystem 104b of the second robot test system 404 is configured so that the tip of the second robot manipulator 102b emulates the motion behavior of a free-floating artificial satellite.

[0089] The first and second robot test systems 402 and 404 can implement the same arm and joint control system 112.

[0090] The first and second robotic test systems 402 and 404 are operated such that the first robotic manipulator 102a visually tracks the second robotic manipulator 102b to establish a physical connection (indicated by arrow 406). The physical connection may be a free-flyer capture emulation. When the first and second systems 402 and 404 interact, contact forces and moments 408 and 410 are experienced by the first and second robotic manipulators 102a and 102b, respectively. The forces and moments are measured by systems 402 and 404 as described above and used to drive internal simulations performed by the dynamic system emulator 114.

[0091] The MR system 406 communicates with the control subsystems 104a and 104b of the first and second robot test systems 402 and 404. The data transmitted from the control subsystems 104a and 104b to the MR system 406 may include positioning data (position and orientation). Positioning data may include, for example, positioning data for points with a resolution on the end effectors of robots 102a and 102b. Generally, generalized trajectory information may be shared between the control subsystems 104a and 104b and the MR system 406. Generalized trajectory information may include, for example, generalized position (x, y, z, yaw, pitch, roll) and its derivatives: generalized velocity, acceleration. This is used by the MR software of the MR system 406 to project visual emulations of the subject of interest, such as actual servicer arm geometry and actual client spacecraft geometry, or abstracted data (see, for example, later).

[0092] The data transmitted from the MR system 406 to the control subsystems 104a and 104b includes commands and control data. This allows the operator to send scripted commands to robots 402 and 406 (as described later, for example).

[0093] The MR system 406 may be configured to allow an operator to remotely control the first and second robotic arms 102a and 102b.

[0094] The MR system 406 may be configured to overlay virtual models of the two robotic systems being emulated onto the robotic manipulators 102a and 102b. The MR system 406 may also be configured to fix the virtual models onto the arms 102a and 102b. Overlaying a virtual model of the satellite on top of what robot 404 is emulating makes it easy to understand the maneuvers / tests being performed. Overlaying a virtual model of the robotic manipulator emulated by system 402 provides a quick visualization of things like arm responses.

[0095] The MR system 406 may be configured to render telemetry. The rendered telemetry may include actual telemetry of the robot arms 102a and 102b. The rendered telemetry may also include virtual telemetry generated by the dynamic system emulators 114 (not shown in Figure 4) of systems 402 and 404, respectively.

[0096] The MR system 406 is configured to define models and constraints on those models, create / enhance common reference frames, and route telemetry for rendering.

[0097] The MR system 406 may be configured to script the entire testing of robot systems 402 and 404. For example, the MR system 406 may be configured to allow an operator to load and execute scripted behaviors of the arm.

[0098] The MR system 406 may be configured to allow the user to design, plan, and execute maneuvers using the first and second robotic arm systems 402, 404 to support mission planning tasks. The MR system 406 may provide real-time visualization of the telemetry streams of arms 102a, 102b from any position within the test environment.

[0099] The MR system 406 may be configured to allow, for example, the reconfiguration of arms 102a and 102b and the selection of commands to issue, via a user interface executed by the MR system 406.

[0100] System 400 may offer a variety of functionalities and capabilities and may have a variety of test applications. These test applications may include designing and testing navigation and control algorithms, performing control parameter tuning for actual arm operations, rapidly prototyping novel control algorithms for future robot missions and updates to existing systems, performing performance evaluations of newly tuned first robot arm control system parameters (controller tuning), mission planning, trial operation procedures, evaluation of robot arm performance in free-space maneuvers and when the arm is in contact with a structure, and testing of robot interfaces and satellite capture.

[0101] Referring now to Figure 5, which shows a method 600 for testing and verifying robotic interaction between two independent space robotic systems on the ground, according to one embodiment.

[0102] Method 600 may be performed using the system 400 shown in Figure 4.

[0103] In 602, Method 600 includes emulating the motion behavior of a 0G servicer robot arm using a first robot arm (e.g., a robot test system 402). The emulation is performed using a first dynamic system emulator (e.g., a dynamic system emulator 114 in Figure 1) configured to capture the dynamics of the 0G servicer robot arm. For example, the dynamic system emulator may be configured to capture the tip dynamics of a target manipulator (0G servicer robot arm) and control the scheme in the dynamic system emulator 114.

[0104] In 604, Method 600 includes emulating the motion behavior of a free-floating satellite using a second robotic arm (e.g., robotic test system 404). The emulation is performed using a second dynamic system emulator (e.g., dynamic system emulator 114 in Figure 1) configured to capture dynamics such as the levitation dynamics of a free-floating satellite.

[0105] Emulating the dynamics of a free-flying satellite may involve virtually incorporating the satellite's attitude determination and control into a dynamic systems emulator, or emulating the satellite's attitude determination and control using actual attitude determination and control hardware (such as gyroscopes and thrusters) within a second robotic arm.

[0106] Optionally, prior to 602 and 604, method 600 may include providing initial conditions (e.g., dynamic system conditions) to each of the first and second dynamic system emulators in order to determine the initial states of the 0G servicer robot arm and the free-floating satellite. The initial conditions may be provided by an operator via a user interface. In one example, providing initial conditions may include assigning an initial drift to the second robot arm when emulating the dynamics of the free-floating satellite.

[0107] In 606, Method 600 includes a physical interaction between a first robotic arm (emulating a 0G servicer robotic arm) and a second robotic arm (emulating a free-floating satellite). The physical interaction may include replicating a physical interaction between two emulated robotic systems in space, such as the performance of a task or operation by the servicer robotic arm on a free-floating satellite. In one example, the physical interaction may include a capture and hardening process through which the first robotic arm captures and hardens the second robotic arm, mimicking a free-floating capture operation in space.

[0108] In 608, method 600 includes measuring the forces and moments experienced by the first robot arm based on the physical interactions in 606. The forces and moments are measured by an FMS (e.g., FMS108 in Figure 1) in the first robot arm.

[0109] In 610, method 600 includes measuring the forces and moments experienced by the second robot arm based on the physical interactions in 606. The forces and moments are measured by an FMS (e.g., FMS 108 in Figure 1) in the second robot arm.

[0110] In 612, method 600 includes simulating the response to forces and moments measured by the first robot arm using a first dynamic system emulator in the first robot arm, wherein the first dynamic system emulator simulates the response of the 0G servicer robot arm.

[0111] In 614, method 600 includes simulating the response to forces and moments measured by the second robot arm using a second dynamic system emulator in the second robot arm, the second dynamic system emulator simulating the response of a free-floating satellite.

[0112] In 616, method 600 includes tracking a trajectory using a first robotic arm based on a simulated response output by a first dynamic system emulator in 612. In this process, the first robotic arm emulates the motion behavior of a 0G servicer robotic arm based on physical interaction with a free-floating satellite emulated by a second robotic arm. This may include providing the output of the first dynamic system emulator to the arm controller of the first robotic arm to generate commands for the arm and joints, which can be transmitted from the arm controller to the arm mechanism of the first robotic arm.

[0113] In 618, method 600 includes tracking a trajectory using a second robotic arm based on a simulated response output by a second dynamic system emulator in 614. In this process, the second robotic arm emulates the motion behavior of a free-floating satellite based on physical interaction with a 0G servicer robotic arm emulated by a first robotic arm. This may include providing the output of the second dynamic system emulator to the arm controller of the second robotic arm to generate commands for the arm and joints, which can be transmitted from the arm controller to the arm mechanism of the second robotic arm.

[0114] Optionally, during the execution of Method 600, Method 600 may further include overlaying a virtual model of an emulated 0G servicer robotic arm onto a first robotic arm and overlaying a virtual model of an emulated free-floating satellite onto a second robotic arm. The overlay of the virtual models may be performed by a mixed reality system, such as the MR system 406 in Figure 4. The overlay may include, for example, registering or superimposing the virtual models onto actual physical system hardware (i.e., the first and second robotic arms). Any suitable registration technique may be used. The overlay may be performed during any part of Method 600.

[0115] Method 600 may also include gravity offloading the first robotic arm to 0G via a passive offloading mechanism or an active offloading mechanism (e.g., offloading mechanism 208 in Figure 2). A passive offloading mechanism may include, for example, a pulley system having an offset weight and a pendulum cable offloader, where the end of the first robotic arm is suspended using a pendulum-type cable carried by a gantry system. A passive offloading mechanism may include, for example, a passive pulley-driven offloading system at the tip of the first robotic arm and a counterbalancing function for the joints of the manipulator (e.g., wrist and elbow). An active offloading mechanism may include a robotic manipulator (e.g., a ground-based robotic arm) configured to provide gravity offloading to the first robotic arm.

[0116] Method 600 may also include adjusting the mass, inertia, stiffness, or damping characteristics of the spacecraft emulated by the dynamic system emulator. For example, the dynamic model of the dynamic system emulator can be modified at the controller update rate. This means that changes in mass due to orbital separation, deployment (e.g., deployment of solar arrays, radio apertures, or antennas, which may correspond to adjustments of stiffness or damping characteristics), and thrust use can be emulated (in real time) by the dynamic system emulator.

[0117] Specific embodiments of the systems and methods for testing and verifying space robotic systems of the present disclosure are described here with reference to Figures 6 to 8. Embodiments of this system may be referred to as DREAMR Lab. Embodiments include a first robotic test system referred to as Mission Analysis Robotic Kit I ("MARK I") and a second robotic test system referred to as Mission Analysis Robotic Kit II ("MARK II"). MARK I and MARK II are embodiments of the robotic test system 100 of Figure 1. In particular, MARK I may be an embodiment of the robotic test system 200 of Figure 2, and MARK II may be an embodiment of the robotic test system 300 of Figure 3. MARK I and MARK II may be used together as part of a system for testing and verifying space robotic systems, which may correspond to system 400 of Figure 4.

[0118] The MARK I and MARK II systems feature a set of independent control systems that work together to enable the collaborative operation of two independent robotic systems.

[0119] Referring now to Figures 6 and 7, which show a system 700 for testing and verifying a robotic system for space use according to one embodiment. Figure 6 shows the system 700 as a high-level block diagram of its components. Figure 7 shows further details regarding the interaction between MARK I, MARK II and the system time delay in the system 700 shown in Figure 6. The relevant system time delay may include the round-trip delay from sensing the signal to be processed to the command.

[0120] System 700 includes MARK I702 and MARK II704. System 700 further includes a mixed reality system 706 (or an augmented reality or virtual reality system, referred to herein as DREAMR World), which is communicatively connected to MARK I702 and MARK II704.

[0121] MARK I702 includes a robotic manipulator 708 and a robotic manipulator control subsystem 710 for controlling the operation of the robotic manipulator 708.

[0122] The robotic manipulator 708 includes an arm mechanism 712 and an end effector 714 positioned at the tip of the arm mechanism 712. The robotic manipulator 708 further includes a force moment sensor 716 and a probe 718 (for establishing a physical connection, e.g., engagement, with MARK II).

[0123] The MARK I702 further includes a gantry offloader 720. The gantry offloader is connected to an end effector 714 and is configured to passively offload gravity onto the robot manipulator 708 so that the robot manipulator functions as a 0G robot arm.

[0124] The robot manipulator control subsystem 710 includes an arm controller 722 for controlling the movement of the arm mechanism 712, an emulation controller 724, a force moment accommodation (FMA) and visual servo (VS) unit 726, and a probe controller 728 for controlling the movement of the probe 718. The force moment accommodation (FMA) includes software configured to interpret and utilize FMS data by the manipulator. Similarly, the visual servo uses data from the vision system to provide modifications to the robot's movement. For example, the arm can use data from the vision system to change its trajectory to track a target. The robot manipulator control subsystem 710 further includes an automation and safety controller 730. The automation and safety controller 730 may be configured to detect fault conditions (e.g., experienced load, unexpected current) and trigger safety responses (e.g., brake events, sudden stops). To trigger safety responses, the automation and safety controller 730 may communicate with any one or more of the controllers 722, 724, and 728.

[0125] The MARK II 704 includes a robotic manipulator 732 and a robotic manipulator control subsystem 734 for controlling the operation of the robotic manipulator 732.

[0126] The robotic manipulator 732 includes an arm mechanism 736, a force moment sensor 738, and a target 718 for visual tracking and physical contact with a probe 718.

[0127] The robot manipulator control subsystem 734 includes an arm controller 742 for controlling the operation of the arm mechanism 736, an emulation controller 744, and a posture control system 746. The robot manipulator control subsystem 734 further includes an automation and safety controller 748. The automation and safety controller 748 may be configured to detect fault conditions (e.g., experienced load, unexpected current) and trigger safety responses (e.g., brake events, sudden stops). To trigger safety responses, the automation and safety controller 748 may communicate with any one or more of the controllers 742, 744, and 746.

[0128] Each of the robotic manipulators 708 and 732 of MARK I702 and MARK II704 performs tracking 750 based on the outputs of their respective emulation controllers 724 and 744. Tracking 750 is used as input to the mixed reality system 706.

[0129] The arm mechanisms 712 and 736 of the MARK I robot manipulator 708 and the MARK II robot manipulator 732 generate joint angle data 760a and 760b and transmit them to their respective arm controllers 722 and 742. The communication of the joint angle data 760a and 760b includes associated time delays 762a and 762b.

[0130] The FMS 716 and 738 of the MARK I robot manipulator 708 and the MARK II robot manipulator 732 generate sensor measurements and transmit the sensor measurements 764a and 764b to their respective controller subsystems 710 and 734. The communication of the detected force data 764a and 764b includes associated time delays 766a and 766b. The FMS measurements 764a and 764b are provided as inputs to the emulation controllers 724 and 744.

[0131] The robot manipulator control subsystems 710 and 734 generate commands 768a and 768b and transmit them to their respective robot manipulators 708 and 732. Commands 768a and 768b include arm and joint-level commands (e.g., manipulator tip reference commands) that are tracked through the movement of the arms and joints of the arm mechanisms 712 and 736. The communication of commands 768a and 768b includes associated time delays 770a and 770b.

[0132] System 700 further includes time delays 772a, 772b, 774a, and 774b related to the communication of sensor and command signals between the controller subsystems 710 and 734 of MARK I 702 and MARK II 704 and the MR system 706. Time delays 772a and 772b refer to the signal delay from each robot platform 710 and 734 to the DREAMR system 706 via the tracking system 750. In other words, the external system measures the fiducials of each robot 702 and 704 and relays their position and orientation. The measurement, processing, and transmission of these signals result in time delays. Similarly, robot systems 710 and 734 directly transmit data regarding the state of their sensors and joints to the DREAMR system 706. These signals are also delayed (typically by a smaller amount). This time delay is indicated in 774a and 774b.

[0133] The computing and software running the MARK I and MARK II systems 702, 704 (e.g., emulation controllers 724, 744, emulation layers 902a, 902b) can be optimized to minimize the time delays 762–774 in Figure 7. Minimizing the time delays can be achieved by minimizing the load on the software and data buses, for example by using direct connections with fewer resources using a given data bus, real-time polling or interrupts, and / or implementing higher data rates. Minimizing the time delays 762–774 can reduce the potential for controller performance degradation.

[0134] System 700 is configured to emulate a free-flyer capture operation and to evaluate the control system interactions of the robotic systems (MARK I and MARK II) participating in the free-flyer capture operation.

[0135] To perform the task, the MARK II704 emulation package (e.g., a software package executed by the emulation controller 744, such as the emulation layer 902 in Figure 9) is configured to cause the MARK II704 to emulate the dynamics of a real satellite (i.e., a free-flying spacecraft). The task then involves instructing the MARK I702's capture mechanism (e.g., probe 718) to bring the MARK I702 to a standoff distance from its target 740 and secure the MARK II704's target 730. Performing this free-flyer capture task in a laboratory setting is particularly difficult: in addition to the obvious challenges associated with completing such a free-flyer capture task, doing so in a laboratory requires cooperation between the MARK I702 and MARK II704 once the MARK II704's target 740 (e.g., a liquid apogee motor "LAM" nozzle) is captured and hardened by the MARK I702's capture mechanism. When the two robot systems 702 and 704 are hardened together, they cooperate to avoid inducing unstable movements. In this type of operation, two control system scenarios are often considered. In the first scenario (Scenario 1, passive capture), only the servicer control system (spacecraft and robot servicer arm control system) emulated by the MARK I 702 emulation controller 724 is active, while the client attitude and trajectory control system emulated by the MARK II emulation controller 744 is deactivated, so that the servicer (emulated in system 700 by MARK I 702) and the client trajectory control system do not compete with each other during and after the capture operation. In the second scenario (Scenario 2, active capture), both the servicer (emulated in system 700 by MARK I 702) and the client (emulated in system 700 by MARK II 704) control systems are active.This second scenario is often considered when attempting to capture a freely flying spacecraft without perturbing the pointing of the client satellite.

[0136] In both the first and second scenarios described above, the MARK II704 system actively emulates the dynamics of the client spacecraft (via the emulation controller 744) in system 700 (i.e., the test environment). Therefore, regardless of the scenario being tested, whether it is the first scenario (passive capture) or the second scenario (active capture), the active control systems of MARK I702 and MARK II704 interact after capture. If the respective arm control systems (arm controllers 722 and 746) are not designed to account for this interaction, the respective arm control systems 722 and 746 will ultimately compete, resulting in a significant decrease in emulation fidelity in the best-case scenario and the introduction of instability in the control system in the worst-case scenario. To maintain the fidelity of the task emulation, this behavior is obtained under the constraints that (i) there is no communication / exchange of information between the two robot systems 702 and 704, and (ii) MARK II 704 continues to accurately emulate the behavior of the satellite (via the emulation controller 744) throughout the entire operation (unstable behavior would cause the emulation to diverge from the behavior of the actual system).

[0137] To successfully complete such tasks, the overall system design ensures that the two systems interact passively, i.e., the interaction between the two systems 702 and 704 does not inject energy into the systems themselves. This is a particularly difficult task to achieve, given that after curing, there are several independent active control systems that interact with each other. Specifically, the following control systems are often active during the free-flyer capture operation performed by system 700.

[0138] On MARK I702: i) Arm controller 722: Provides level commands for the arm and joints based on inputs from external sensors, outputs from other controllers, and automated commands: (a) Force control: The contact force between the two robot systems 702 and 704 can be adjusted. (b) Visual servo control: Maintain a predetermined standoff distance and adjust the drift rate to match the target satellite emulated by the MARK II704. ii) Emulation controller 724: Provides the option to match the dynamic response of the MARK I702 tip of the robot manipulator 708 to that of a desired system. iii) Probe controller 728: In coordination with the arm controller 722, it controls the probing mechanism (extension / contraction). iv) High-level automation / safety controller 730: Provides system safety and activates appropriate control mechanisms. This may include behavior-monitoring sensors, such as monitoring for overloads of force / moment sensors at the endpoints of the manipulator 708, tracking of the endpoint trajectory of the manipulator 708, overloads of local manipulator 708 joints, and collisions of the manipulator 708 with itself or its environment. This monitoring is performed using both intrinsic and extrinsic sensing. Intrinsic MARK I702 sensing may include joint position measurement, joint current or torque sensing of the manipulator 708, and the relative position and orientation of other objects obtained from the MARK I702's vision system. Extrinsic sensing may include the use of optical tracking devices to track the position and orientation of a point of resolution on the MARK I702 and other points of resolution in the workspace.

[0139] On MARK II704: i) Arm controller 742: Provides level commands for the arm and joints based on inputs from external sensors, outputs from other controllers, and automated commands. ii) Emulation Controller 744: Emulates the dynamics of a floating passive spacecraft based on measured forces / moments and configured body properties. The internal emulation package performed by the Emulation Controller 744 can also simulate the operation of active attitude and / or orbit control systems, such as the attitude control system 746, and adds the simulated control systems to the list. iii) High-level automation / safety controller 748: Provides safety to the system and activates appropriate control mechanisms. This safety verification can be achieved using methods and sensors such as those used in the MARK I system 702 (as described above).

[0140] All of these control systems on the two robot systems 702 and 704 can interact with each other without direct information about what the other robot system is doing during contact between the MARK I and MARK II systems 702 and 704. In such a situation, any sensor noise and time delay affecting the system can destabilize the interaction. To prevent this from happening, several steps can be taken: (a) Carefully tuning the control system to guarantee passive interaction (however, being too conservative will lead to a significant performance degradation in both the performance of MARK I 702 and the emulation fidelity of MARK II 704); (b) Smoothing sensor data to avoid instability problems caused by the two systems 702 and 704 reacting to sensor noise; (c) Optimizing the computing and software running the two systems 702 and 704 to minimize time delays 762-774 (see Figure 7) that degrade controller performance. (d) To address time delays (such as time delays 762-774 in Figure 7), adopt a more natural and robust emulation / control strategy.

[0141] Figure 7 shows system 700, illustrating the interaction of various control systems on two robot systems 702, 704. It is important to note how the only connection between the two MARK robots 702, 704 comes from "visual tracking" 752 (referring to the MARK I702's ability to receive and process relative pose information of the two systems 702, 704 from an independent vision system, emulating the presence of a machine vision algorithm that processes the camera view) and "physical connection" 754 between the two systems 702, 704 (i.e., between probe 718 and target 740). The forces and moments 756, 758 resulting from the physical contact between robot systems 702, 704 (experienced by MARK II704 and MARK I706, respectively) are always measured independently on the two robots 702, 704.

[0142] Here, we will explain the details of the dynamic emulation layer of system 700 with reference to Figure 8.

[0143] Referring to Figure 8, the block diagram shown illustrates an emulation operation 900 of system 700 according to one embodiment.

[0144] Both MARK I702 and MARK II704 have internal emulation layers (dynamic emulation layers) 902a and 902b, collectively referred to as the emulation layer 902 and commonly called the emulation layer 902. The emulation layers 902a and 902b may also be the dynamic system emulator 114 shown in Figure 1.

[0145] The internal emulation layer 902 is implemented by the emulation controllers 724 and 744 of the respective systems 702 and 704. The internal emulation layer 902 may include one or more software components executed by the respective emulation controllers 724 and 744. The internal emulation layers 902 and 904 enable the tips (or points with resolution near the free ends) of the arms 708 and 732 to reproduce the behavior of a desired dynamical system (e.g., a shorter / longer arm, or a lighter / heavier arm).

[0146] The core emulation layer 902 is a multi-body dynamic system simulator that runs in the overall control systems 710 and 734 of the MARK I and MARK II systems 702 and 704, respectively. This internal simulation 902 provides a reference trajectory for the arms 712 and 736 to track, based on three core elements: a dynamical system model, sensed forces and moments, and initial conditions.

[0147] The dynamic system model represents a mathematical model of the target behavior at the tips of arms 708 and 732. The dynamic system model is implemented by the emulation layer 902, which is executed by emulation controllers 724 and 744. By changing the simulated dynamic system model, dramatically different motion behaviors at the tips of arms 708 and 732 can be emulated. In one embodiment, the emulation software 902 allows the operator to construct their own dynamic system model. This includes rigid and flexible many-body dynamics that simulate single (spacecraft) or many-body systems (multi-joint robotic manipulators), electromechanical models that emulate actively driven articulations (e.g., robotic manipulator joints, attitude control devices such as reaction wheels, or deployment actuators for satellite structures), and computing and software representing various control systems that control the single or many-body dynamics. The emulation package 902 can be configured to emulate a variety of system behaviors. The system behaviors that can be emulated by the emulation layer 902 may include, for example, the dynamics of a floating satellite or the tip dynamics of a large arm, thereby enabling the MARK I702 to accurately emulate the tip dynamics of systems that may not be physically testable in the laboratory.

[0148] The internal simulation 902 further provides a reference trajectory for the robot arm to track, based on the detected forces and moments. The internal simulation 902 is driven by the measured forces and moments 904a, 904b experienced by the arms 708, 732 during actual physical interactions in the test environment (e.g., arm 708 of MARK I 702 contacts arm 732 of MARK II 704, or arm 708 of MARK I 702 contacts a stationary object). The contact forces and moments 904a, 904b are measured by arm FMS (force and moment sensors) 714, 738 in arms 708, 732 of MARK I and MARK II, respectively, and are given as disturbances to the respective dynamic system model simulations 902a, 902b. Subsequently, based on the system mathematical models implemented by the emulation layers 902a and 902b, the responses of systems 702 and 704 to their forces and moments 904a and 904b can be simulated, and the results can be used to generate reference trajectories for arms 708 and 732 to track.

[0149] The FMS measurements of force and moment 904a and 904b are output from FMS 714 and 738 to emulation layers 902a and 902b, with associated time delays 914b and 914e.

[0150] FMS measurements are received as input by emulation layers 902a and 902b, which simulate the system's response to those forces and moments 904a and 904b. Emulator layers 902a and 902b generate simulated responses, including position and orientation tracking for the robot system to track, in order to emulate the dynamics of the emulated system. The simulated response outputs from emulator layers 902a and 902b are output from emulator layers 902a and 902b to error determination units 910a and 910b with associated time delays 914c and 914f.

[0151] MARK I hardware (HW) 708 and MARK II hardware (HW) 732 output measurements 906a and 906b. Measurements 906a and 906b may include joint and motor position / rate measurements used by the local arm and joint controllers. Measurements 906a and 906b may further include vision system estimates and / or force moment measurements. Measurements 906a and 906b, along with associated time delays 914a and 914d, are provided as inputs to error determination units 910a and 910b.

[0152] Each of the robot manipulator control systems 710 and 734 includes error determination units 910a and 910b. The error determination units 910a and 910b receive input from the emulation layers 902a and 902b and the MARK I and MARK II HW708 and 732, determine error measurements 912a and 912b, and provide these error measurements 912a and 912b to the arm and joint controllers 722 and 742. The error measurements 912a and 912b are errors in the control system. The error measurements 912a and 912b are the difference between the target value (determined by emulation) and the measured position and / or velocity of the joint. The controllers 722 and 742 generate command signals to send to the joint to correct the error (error measurements 912a and 912b) between the measured joint position / velocity and the target value.

[0153] The arm and joint controllers 722 and 742 generate arm and joint commands 908a and 908b that describe reference trajectories for the arm mechanisms of MARK I and MARK II HW708 and 732 to track, and provide the commands 908a and 908b to MARK I and MARK II HW708 and 732.

[0154] The internal simulation 902 also generates and provides a reference trajectory for the robotic arm to track, based on initial conditions. Initial conditions can be provided to the emulation package 902 to determine the initial state of the simulated dynamical system. This allows, for example, assigning an initial target drift rate to the arm 732 when emulating the dynamics of a satellite.

[0155] In one embodiment, the reference trajectory may be based not only on the response to disturbances but also on the desired motion of the MARK I's arm 708.

[0156] Based on the elements described above, the emulation package 902 generates the desired system response as manipulator tip reference commands 908a and 908b. The reference signals are then tracked through arm and joint control movements.

[0157] The emulation packages 902a and 902b are implemented for arms 708 and 732 of both MARK I and MARK II. This means that since either system 702 or 704 can enable the emulation function, the operator can use the two arms 708 and 732 to emulate the dynamics and interactions of completely different systems.

[0158] Referring again to Figures 6 and 7, System 700 includes a mixed reality (MR) system 706 that communicates with the MARK I 702 and MARK II 704 systems.

[0159] The use of MR with the MR system 706 allows users to view the system state and telemetry of system 700 in a single worldview. Multiple robots or systems (e.g., systems 702, 704) are displayed. The displayed systems may be scaled down on the hardware of systems 702, 704 or superimposed at full scale. The MR system 706 can then be used to configure a virtual setup of interest to the user. This can be done by a user in the laboratory or a user in a remote location. For example, a user can interact with the MR system 706 via a user interface and reconfigure system 700 (and its components) via the user interface. Similarly, the user can configure the physical elements of the MR world by actually moving the end effects of physical manipulators 708, 732 through user interface commands to the hardware or by using the hand-enabled compliant control features of the MARK I and MARK II arm control systems 710, 734, respectively.

[0160] Next, we will describe various additional details and features of the system 700 as shown in Figures 7 to 9.

[0161] System 700 can function as a microgravity space robotics facility for the design, development, and verification of robotic systems and mission operations used or performed in space. System 700 may also be for on-orbit robotics. Robotics has played a vital role in enabling human space exploration. In particular, space manipulators were essential to the success of the Space Shuttle and International Space Station programs (e.g., Canadarm, Canadarm 2, Dexter, etc.). Robotics will play an even more important role in future space development and missions (e.g., Canadarm 3, satellite maintenance, etc.). Space robots are expected to complete increasingly complex tasks while constantly increasing their level of autonomy.

[0162] System 700 may be used to perform verification of guidance, navigation, and control (GNC) algorithms. System 700 may also verify and estimate the performance of GNC algorithms. Such verification and estimation may be used to meet the increasing demands for the performance of such algorithms.

[0163] The fundamental problem and challenge addressed by System 700 (and other systems described herein) is that the performance of a zero-gravity robotic system differs significantly from that of the same system on Earth. As a result, laboratory testing of a complete robotic system is often impossible. The performance of GNC algorithms is often verified through a combination of high-fidelity simulations and ground tests: single-joint testbeds, floating plane manipulators, 1G arms, etc.

[0164] The MARK I702 can include a 6-degree-of-freedom manipulator 708 that is gravity-offloaded via a passive pulley-driven offload system at the tip of the manipulator 708 (including the gantry offloader 720), as well as counterbalancing capabilities for the manipulator 708's wrist and elbow. These features allow the MARK I702 to very faithfully emulate the performance of a typical space manipulator in a weightless environment.

[0165] MARK I702 may be a 6-degree-of-freedom manipulator that is gravity-offloaded via a passive mechanism. This feature allows MARK I702 to reliably emulate the behavior of a space manipulator. Each joint of MARK I's arm mechanism 712 may be equipped with a set of LEDs (indicator lights) whose color changes depending on the control mode currently in which arm 708 is located. This feature allows a robot operator to always know the current state of the system from any position in the laboratory and to know whether it is safe to directly interact with manipulator 708.

[0166] System 700 may include arm control modes, including safety, arm initialization, standby, passive brimp, arm movement, and arm bored, with color indicators.

[0167] System 700 may include interchangeable end effectors. The end effector 714 of MARK I702 may be interchangeable to allow testing of various different operating and arming mechanisms.

[0168] The MARK I702 can be controlled in various ways. Control may include any one or more of the following: a hand controller (with a computer user interface), a robot operator, and a set of end-effector movement and rotation controls to emulate remote control of the system.

[0169] The control may include a joystick that allows an operator to control the arm 708 while standing nearby to inspect the system during its operation.

[0170] System 700 may include a computer user interface to enable engineers / users to script a complex set of commands that the arm 708 can execute (using the arm controller 722).

[0171] The MARK II704 includes a 7-degree-of-freedom, 1-gravity (1G) arm 732, which is generally used for dynamic system emulation and for performing cooperative tasks with the MARK I702.

[0172] The 7-degree-of-freedom manipulator 732 can be used for dynamic system manipulation and to test cooperative tasks that require MARK I702 to interact with another active system (e.g., emulation of a braked Canada Arm 2). This facilitates the ability to test the performance and / or interactions of large dynamic systems, such as the 17-meter-long Canada Arm 2, which would otherwise be impossible in a laboratory setting.

[0173] System 700 may include an internal module that can emulate the responses of various dynamic systems using the manipulator tip (for example, emulating the behavior of a typical mass spring damper system). Leveraging this functionality provides users with unique capabilities that actively emulate various behaviors in the work environment while also offering the flexibility to modify mechanical properties on the fly.

[0174] System 700 includes an internal emulation package 902 that can be configured to emulate the behavior of free-floating satellites of various inertia. The emulated satellite can also activate its own emulated attitude control system 746 to recover its original attitude after an external perturbation is introduced. The attitude of the system does not change even after the operator perturbs the system.

[0175] In one embodiment, the MR system 706 uses Microsoft® HoloLens 2 technology (or similar technology) to provide an operator with a highly immersive mixed reality environment for training and collaborative design.

[0176] The HoloLens 2 technology can enhance the understanding of the tests being performed by the robotics system 700. The MR system 706 may also be configured to overlay a graphic satellite model onto the MARK II 704, allowing the observer to see not only the tip of the manipulator 732 emulating its movement, but also the movement of the satellite driven by the movement of the tip of the manipulator 732 on the MARK II. This may also include an internal view of the operation, such as the engagement of dynamic parts by internal contact of the capture probe 718 within the liquid apogee nozzle (target 740) of the client spacecraft, or other "peg-in-hole" type engagement operations, which may only be possible through the introduction of mixed reality.

[0177] The applications of System 700 (tests and operations performed) include mission planning, robot interface testing, and free-flyer capture operations.

[0178] The mission plan includes laboratory-assisted operations in the design of complex operations involving robotic systems. The operator may use the MR system 706 to directly control and reposition the MARK II 704 and MARK I 702 to support the design of the free-flyer capture operation. The operator can virtually change the arm configuration and have the robotic systems 702 and 704 perform the desired maneuvers.

[0179] System 700 may enable testing of the robot interface by operating MARK I702 to set up a mockup of the robot interface, with the remaining half of the interface held by MARK II throughout the entire operation. This setup allows for testing of the robot compatibility of a given interface and also allows for estimating the predicted loads experienced by arm 708 and the other side of the interface (on manipulator 732 of MARK II704) throughout the entire operation.

[0180] The free-flyer capture operation involves not only the ability to test the capture of a free-flyer, but also operations requiring cooperation between multiple robotic systems 702 and 704 and MARK I702, which attempts to capture a target floating satellite whose behavior is emulated by MARK II704. After the capture is complete, the two systems 702 and 704 drift together, the captured satellite (emulated by MARK II704) initiates an attitude correction maneuver, and MARK II704 can follow the attitude correction maneuver. The two systems 702 and 704 interact with each other in a stable manner even after the systems have been perturbed by the operators.

[0181] While the above description provides examples of one or more apparatuses, methods, or systems, other apparatuses, methods, or systems are also included in the scope of the claims, as will be understood by those skilled in the art.

Claims

1. This is a system for ground testing the performance of space robot systems. The first robotic manipulator is configured to offload gravity so that it functions as a zero-gravity (0G) robotic manipulator, A first dynamic system emulator configured to simulate the motion behavior response of the first space robot system based on the forces and moments measured by the first robot manipulator during the physical interaction between the first robot manipulator and the second robot manipulator which emulates the motion behavior of the second space robot system, A first arm controller is configured to generate a first manipulator tip reference trajectory command based on the motion behavior response simulated by the first dynamic system emulator, and to provide the first manipulator tip reference trajectory command to the first robot manipulator, A first arm mechanism within the first robot manipulator is configured to track a first trajectory based on a first manipulator tip reference trajectory command configured to emulate the motion behavior of the first robot manipulator, A first robot test system equipped with, The aforementioned second robotic manipulator, A second dynamic system emulator configured to simulate the motion behavior response of the second space robot system based on the forces and moments measured by the second robot manipulator during the physical interaction between the second robot manipulator and the first robot manipulator which emulates the motion behavior of the first robot manipulator, A second arm controller is configured to generate a second manipulator tip reference trajectory command based on the motion behavior response simulated by the second dynamic system emulator, and to provide the second manipulator tip reference trajectory command to the second robot manipulator, A second arm mechanism within the second robot manipulator is configured to track a second trajectory based on a second manipulator tip reference trajectory command configured to emulate the motion behavior of the second robot manipulator, A second robot test system equipped with, A system equipped with these features.

2. A mixed reality system that is communicatively connected to the first robot test system and the second robot test system, and configured to overlay a first virtual model of the first robot manipulator onto the first robot manipulator, and to overlay a second virtual model of the second space robot system onto the second robot manipulator, The system according to claim 1, further comprising:

3. The system according to claim 1, wherein the second space robot system emulated by the second robot test system is a free-floating spacecraft.

4. A robot test system for ground testing the performance of space robot systems, The first robotic manipulator is configured to offload gravity so that it functions as a zero-gravity (0G) robotic manipulator, A dynamic system emulator configured to simulate the motion response of the first space robot system based on the forces and moments measured by the first robot manipulator during the physical interaction between the first robot manipulator and the second robot manipulator that emulates the motion behavior of the second space robot system, An arm controller configured to generate a manipulator tip reference trajectory command based on the motion behavior response simulated by the dynamic system emulator, and to provide the manipulator tip reference trajectory command to the robot manipulator, The robot manipulator is configured to emulate the motion behavior of the first space robot system by tracking a trajectory based on the manipulator tip reference trajectory command, and the arm mechanism within the robot manipulator is configured to do so. A robotic testing system equipped with the following features.

5. The robot testing system according to claim 4, wherein the first space robot system is a 0G servicer robot arm.

6. The robot test system according to claim 4, wherein the first space robot system is a space robot system serviced by a servicer robot arm.

7. The robot test system according to claim 6, wherein the first space robot system is a free-floating artificial satellite.

8. A method for ground testing and verifying robotic interaction between two independent space robotic systems, A step of emulating the motion behavior of the 0G servicer robot arm in the first robot arm using a first dynamic system emulator in the first robot arm configured to capture the dynamics of the manipulator tip of the zero-gravity (0G) servicer robot arm, wherein the first robot arm is gravity offloaded to function as a 0G robot arm, The steps include: emulating the motion behavior of a free-floating satellite with a second robotic arm using a second dynamic system emulator in the second robotic arm configured to capture the dynamics of the free-floating satellite; The steps include: while the aforementioned motion behavior is being emulated, the first robot arm and the second robot arm are made to interact physically; The steps include measuring the force and moment experienced by the first robot arm based on the aforementioned physical interaction via a force moment sensor within the first robot arm, The steps include measuring the force and moment experienced by the second robot arm based on the aforementioned physical interaction via a force moment sensor within the second robot arm, The steps include simulating the response to forces and moments measured by the first robot arm using the first dynamic system emulator, The steps include simulating the response to forces and moments measured by the second robot arm using the second dynamic system emulator, The steps include tracking the trajectory with the first robot arm based on the simulated response output by the first dynamic system emulator, The steps include tracking the trajectory with the second robot arm based on the simulated response output by the second dynamic system emulator, A method that includes this.

9. The method according to claim 8, further comprising the step of providing initial conditions to each of the first and second dynamic system emulators to determine the initial state of the 0G servicer robot arm and the free-floating satellite to be emulated.

10. The method according to claim 9, wherein the initial conditions provided to the second dynamic system emulator include the initial drift of the emulated free-floating satellite.

11. The method according to claim 8, further comprising the steps of overlaying a virtual model of the emulated 0G servicer robot arm onto the first robot arm and overlaying a virtual model of the emulated free-floating satellite onto the second robot arm.

12. The method according to claim 8, wherein the first robotic arm is gravity-offloaded via a passive offloading mechanism.

13. The method according to claim 8, wherein the first robot arm is gravity-offloaded via an active offload mechanism comprising a third robot manipulator.

14. The method according to claim 8, further comprising the step of adjusting the mass, inertia, stiffness, or damping characteristics of the emulated free-floating satellite using the dynamic system emulator to emulate a mass or inertia change event of the emulated spacecraft.

Images