Robot camera control via motion capture

Abstract

To provide robotic camera control via preferred motion capture. A motion capture device records the movements of an operator, and a control engine then converts those movements into control signals for controlling the robot. The control engine can directly convert the operator's movements into similar movements to be performed by the robot, or the control engine can calculate robot dynamics that cause parts of the robot to mimic corresponding parts of the operator.

Description

[Technical Field]

[0001] (Citation of Related Application) This application claims the benefit of U.S. Patent Application No. 15 / 216,583, filed July 21, 2016, which is incorporated herein by reference. FIELD OF THE INVENTION FIELD OF THE INVENTION Embodiments of the present invention relate generally to robotics, and more particularly to robotic camera control via motion capture. [Background technology]

[0002] When shooting a motion picture, a camera operator controls the position, orientation, and movement of the camera to film shots, capture sequences, and transition between camera angles, among other film procedures. However, operating a camera can be difficult and / or dangerous in certain situations. For example, when shooting a close-up sequence, a camera operator may be required to assume an awkward position for an extended period of time while physically supporting heavy camera equipment. Or, when shooting an action sequence, a camera operator may be required to risk injury to achieve a necessary proximity to the action being filmed. In either case, the camera operator must typically perform highly complex camera movements and often repeat these motions across multiple runs.

[0003] As the foregoing discussion illustrates, what is needed in the art is a more efficient approach to filming motion picture sequences. Summary of the Invention [Means for solving the problem]

[0004] Various embodiments of the present invention describe a computer-implemented method for controlling a robot, the computer-implemented method including generating motion capture data based on one or more movements of an operator, processing the motion capture data to generate control signals for controlling the robot, and transmitting the control signals to the robot to cause the robot to mimic the one or more movements of the operator.

[0005] At least one advantage of the approach discussed herein is that a human camera operator need not be exposed to discomfort or injury when shooting footage. The present invention further provides, for example, the following: (Item 1) 1. A computer-implemented method for controlling a robot, the method comprising: generating motion capture data based on one or more movements of an operator; processing the motion capture data to generate control signals for controlling the robot; transmitting the control signal to the robot, causing the robot to mimic the one or more movements of the operator; and 11. A computer-implemented method comprising: (Item 2) Item 10. The computer-implemented method of item 1, wherein generating the motion capture data includes tracking, via a motion capture device, at least one of head, torso, and limb movements associated with the operator. (Item 3) Processing the motion capture data and generating the control signal includes: modeling a first set of dynamics of the operator corresponding to the one or more movements; determining a second set of motions of the robot corresponding to the one or more movements; generating the control signal based on the second set of behaviors; and Item 1. The computer-implemented method of item 1, comprising: (Item 4) Item 1. The computer-implemented method of item 1, wherein causing the robot to mimic the one or more movements of the operator includes actuating a set of joints included in the robot to perform a joint sequence associated with the one or more movements. (Item 5) Item 5. The computer-implemented method of item 4, wherein the robot comprises a robot arm including the set of joints. (Item 6) Item 10. The computer-implemented method of item 1, wherein causing the robot to mimic the one or more movements includes adjusting a speed of at least one motor coupled to a propeller. (Item 7) Item 7. The computer-implemented method of item 6, wherein the robot comprises an aerial drone. (Item 8) Item 10. The computer-implemented method of item 1, wherein causing the robot to imitate the one or more movements includes causing only a first portion of the robot to imitate only a portion of the one or more movements. (Item 9) Item 10. The computer-implemented method of item 1, further comprising capturing sensor data while the robot mimics the one or more movements. (Item 10) Item 10. The computer-implemented method of item 1, wherein the one or more movements comprise a target trajectory for a sensor array coupled to the robot. (Item 11) A non-transitory computer-readable medium that, when executed by a processor, generating motion capture data based on one or more movements of an operator; processing the motion capture data to generate control signals for controlling a robot; transmitting the control signal to the robot, causing the robot to mimic the one or more movements of the operator; a non-transitory computer readable medium for causing the processor to control the robot by performing (Item 12) Item 12. The non-transitory computer-readable medium of item 11, wherein generating the motion capture data includes tracking, via a motion capture device, at least one of head, torso, and limb movements associated with the operator. (Item 13) Processing the motion capture data and generating the control signal includes: modeling a first set of dynamics of the operator corresponding to the one or more movements; determining a second set of motions of the robot corresponding to the one or more movements; generating the control signal based on the second set of behaviors; and Item 12. The non-transitory computer-readable medium of item 11, comprising: (Item 14) Item 12. The non-transitory computer-readable medium of item 11, wherein causing the robot to mimic the one or more movements of the operator includes actuating a set of joints included in the robot to perform a joint sequence associated with the one or more movements. (Item 15) Item 12. The non-transitory computer-readable medium of item 11, wherein causing the robot to mimic the one or more movements includes adjusting a speed of at least one motor coupled to a propeller. (Item 16) Item 12. The non-transitory computer-readable medium of item 11, wherein causing the robot to imitate the one or more movements includes causing only a first portion of the robot to imitate only a portion of the one or more movements. (Item 17) Item 12. The non-transitory computer-readable medium of item 11, further comprising capturing sensor data while the robot mimics the one or more movements. (Item 18) Item 12. The non-transitory computer-readable medium of item 11, wherein the one or more movements comprise a target trajectory for a sensor array coupled to the robot. (Item 19) 1. A system for controlling a robot, the system comprising: a memory storing a control application; a processor coupled to the memory; Equipped with The processor: generating motion capture data based on one or more movements of an operator; processing the motion capture data to generate control signals for controlling the robot; transmitting the control signal to the robot, causing the robot to mimic the one or more movements of the operator; and A system that is configured to: (Item 20) the processor executes the control application; generating the motion capture data; processing the motion capture data to generate the control signal; transmitting the control signal to the robot; 20. The system of item 19, configured to: [Brief explanation of the drawings]

[0006] A more particular description of the invention briefly summarized above in such a manner that the above-recited features of the invention may be understood in detail may be had by reference to the embodiments, some of which are illustrated in the accompanying drawings. It should be noted, however, that since the invention is susceptible of other equally effective embodiments, the appended drawings only illustrate typical embodiments of the invention and are therefore not to be considered as limiting its scope. [Figure 1] FIG. 1 illustrates a system configured to implement one or more aspects of the present invention. [Figure 2] FIG. 2 is a more detailed diagram of the control engine of FIG. 1 in accordance with various embodiments of the present invention. [Figure 3] FIG. 3 illustrates how the articulation of a robotic arm according to various embodiments of the present invention can be controlled via a motion capture device. [Figure 4] FIG. 4 illustrates how the position and orientation of a robotic drone according to various embodiments of the present invention can be controlled via motion capture equipment. [Figure 5] FIG. 5 is a flow diagram of method steps for converting motion capture data into control signals for controlling a robot according to various embodiments of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0007] In the following description, numerous specific details are set forth to provide a more thorough understanding of the present invention. However, it will be apparent to one skilled in the art that the present invention may be practiced without one or more of these specific details.

[0008] (System Overview) 1 illustrates a system configured to implement one or more aspects of the present invention. As shown, system 100 includes a motion capture device 110, a computer 120, and a robot 140. Motion capture device 110 is coupled to computer 120, which is coupled to robot 140.

[0009] The motion capture device 110 includes sensors 112(0) and 112(1) configured to capture movements associated with an operator 114. The motion capture device 110 may include any number of different sensors, but generally, the motion capture device 110 includes at least two sensors 112 and captures binocular data. The motion capture device 110 outputs motion capture data 150 to a computer 120 for processing.

[0010] Computer 120 includes a processor 122, an input / output (I / O) utility 124, and a memory 126 coupled together. Processor 122 may be any technically feasible form of processing device configured to process data and execute program code. Processor 122 may be, for example, a central processing unit (CPU), a graphics processing unit (GPU), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), any technically feasible combination of such units, etc. I / O utility 124 may include devices configured to receive input, including, for example, a keyboard, a mouse, etc. I / O utility 124 may also include devices configured to provide output, including, for example, a display device, a speaker, etc. I / O utility 124 may further include devices configured to both receive and provide input and output, respectively, including, for example, a touchscreen, a universal serial bus (USB) port, etc.

[0011] The memory 126 may be any technically feasible storage medium configured to store data and software applications. The memory 126 may be, for example, a hard disk, a random access memory (RAM) module, a read-only memory (ROM), etc. The memory 126 includes a control engine 128 and a database 130. The control engine 128 is a software application that, when executed by the processor 122, causes the processor 122 to interact with the robot 140.

[0012] Robot 140 includes an actuator 142 coupled to sensor array 144. Actuator 142 can be any technically feasible type of mechanism configured to induce any kind of physical movement, including a linear or rotary motor, a hydraulic or pneumatic pump, etc. In one embodiment (described by way of example in conjunction with FIG. 3 ), actuator 142 includes a rotary motor configured to articulate a robotic arm of robot 140. In another embodiment (described by way of example in conjunction with FIG. 4 ), actuator 142 includes a rotary motor configured to drive a set of propellers that propel robot 140 through the air.

[0013] Sensor array 144 may include any technologically feasible collection of sensors. For example, sensor array 144 may include optical sensors, acoustic sensors, and / or other types of sensors configured to measure physical quantities. Generally, sensor array 144 is configured to record multimedia data. In practice, sensor array 144 includes a video camera configured to capture frames 146. By capturing a series of such frames, sensor array 144 may record a movie.

[0014] In operation, the motion capture device 110 captures movements associated with the operator 114 and then transmits a motion capture signal 150 to the control engine 128. The control engine 128 processes the motion capture signal 150 and generates a control signal 152. The control engine 128 transmits the control signal 152 to the robot 140, which controls the movement of the robot 140 via the actuators 140. Generally, the resulting movement of the robot 140 mimics and / or is derived from the movement of the operator 114. The robot 140 receives sensor data 154 via the sensor array 146 and transmits the sensor data to the control engine 128. The control engine 128 processes the received data for storage in the database 130.

[0015] In this manner, operator 114 may control robot 140 and capture video sequences without being required to physically and / or directly operate any camera equipment. One advantage of this approach is that operator 114 does not need to be exposed to difficult and / or dangerous working environments when filming. Control engine 128 is described in further detail below in conjunction with FIG. 2.

[0016] 2 is a more detailed diagram of the control engine of FIG. 1, in accordance with various embodiments of the present invention. As shown, the control engine 128 includes a motion capture analyzer 200, a motion extractor 210, a motion converter 220, and a multimedia capture module 230.

[0017] The motion capture analyzer 200 is configured to receive motion capture signals 150 from the motion capture device 110, process those signals, and generate raw data 202. The motion capture signals 150 generally include video data captured by the sensors 112. The motion capture analyzer 200 processes this video data to identify the position and orientation of some or all of the operator 114 over a period of time. In one embodiment, the operator 114 wears clothing with reflective markers that can be tracked by the motion capture analyzer 200. In some embodiments, the motion capture analyzer 200 may be included within the motion capture device 110. The motion capture analyzer 200 generates raw data 202, which includes a set of quantities that describe the position and orientation of any tracked parts of the operator 114 over the period of time.

[0018] The motion extractor 210 receives the raw data 202 and then processes the data to generate processed data 212. In doing so, the motion extractor 210 models the motion of the operator 114 based on the raw data 202 over that period of time. The motion extractor 210 may determine the joint movements of the limbs of the operator 114, as well as the trajectory of any part of the operator 114 and / or various rotations and / or translations of parts of the operator 114, including any objects associated with the operator 114. The motion extractor 220 provides the processed data 212 to a motion converter 220.

[0019] The dynamics converter 220 is configured to convert the modeled dynamics contained within the processed data 212 into control signals 152 for controlling the robot 140 to obtain dynamics derived from the modeled dynamics. The dynamics converter 220 may generate some control signals 152 that cause the robot 140 to directly mimic the dynamics and movements of the operator 114, or may generate other control signals 152 that cause only a portion of the robot 140 to mimic the dynamics and movements of the operator 114.

[0020] For example, the dynamics converter 220 may generate control signals 152 that cause the actuators 142 in the robotic arm of the robot 140 to mimic the articulation of the joints in the arm of the operator 114. This particular example is described in more detail below in conjunction with FIG. 3. Alternatively, the dynamics converter 220 may generate control signals 152 that cause the actuators 142 to perform any technically feasible set of actuations and cause the sensor array 144 to follow a path similar to that of an object held by the operator 114. This example is described in more detail below in conjunction with FIG. 4. In one embodiment, when generating the control signals 152, the dynamics converter 220 may amplify the movements of the operator 114 so that the dynamics of the robot 140 represent exaggerated versions of the dynamics of the operator 114.

[0021] In response to control signals 152, actuators 142 within robot 140 actuate and move sensor array 144 (potentially robot 140 as a whole). Sensor array 144 captures sensor data 154 and transmits this data to multimedia capture module 230 within control engine 128. Multimedia capture application 230 generally manages the operation of the sensor array and processes input sensor signals, such as sensor signal 154. Based on these signals, multimedia capture module 230 generates multimedia data 232 for storage in database 130. Multimedia data 232 may include any technically feasible type of data, but in practice multimedia data 232 includes frames of video captured by sensor array 144 and possibly audio data as well.

[0022] As a general matter, the control engine 128 is configured to translate movements performed by the operator 114 into movements performed by the robot 140. In this manner, the operator 114 can control the robot 140 by performing a set of desired movements. This approach may be particularly applicable to cinematography, where a camera operator may desire to control the robot 140 in a particular manner to shoot a cinematic sequence. However, those skilled in the art will understand that the techniques disclosed herein for controlling a robot may be applicable to other fields besides cinematography.

[0023] (Example robot control via motion capture) 3 illustrates how the articulation of a robotic arm according to various embodiments of the present invention may be controlled via a motion capture device. The scenario depicted in FIG. 3 is provided for illustrative purposes and illustrates only one possible robot and one possible set of movements. The techniques described herein are applicable to any technically feasible robot and any technically feasible set of movements.

[0024] As shown, operator 114 moves arm 300 upward from position 310(A) to assume position 312(A). Operator 114 may optionally hold motion capture object 302. Motion capture device 110 is configured to track the movements of operator 114 generally, and specifically, to track the articulation of joints in operator's 114's arm 300. In one embodiment, motion capture device 110 relies on markers that indicate the positions of joints in arm 300. Motion capture device 110 may also track the position and location of motion capture object 302. Motion capture device 110 transmits motion capture signals 150 representing any and all captured data to control engine 128.

[0025] Control engine 128 then generates control signals 152 to cause robot 140 to move from position 310(B) to position 312(B). Positions 310(A) and 310(B) are generally similar and have similar articulation movements. Similarly, positions 312(A) and 312(B) are similar and have similar articulation movements. Additionally, any intermediate positions are also generally similar. Robot 140 can effect these similar articulation movements by actuating actuators 142(0), 142(1), 142(2), and 142(3) to perform joint rotations similar to those performed by operator 114 using arm 300. In this manner, robot 140 performs movements substantially identical to those performed by operator 114. In some embodiments, the movements of robot 140 represent exaggerated versions of the movements of operator 114. For example, the control engine 128 may scale the joint rotations associated with the arm 300 by a preset amount when generating the generated signal 152 to the actuator 142 or perform a smoothing process to attenuate disturbances and / or other perturbations in the simulated movement.

[0026] During operation, the sensor array 144 captures data that is transmitted to the control engine 128 as sensor data 154. This sensor data is collected from a series of locations indicated by the operator 114 by performing corresponding movements to those locations, as will be described. However, in some cases, the robot 140 need not implement similar dynamics as the operator 114. In particular, when the operator 114 moves the motion-controlled object 302 along a path, the robot 140 may perform any technically feasible combination of movements to cause the sensor array 144 to follow a similar path. This approach is described in further detail below in conjunction with FIG. 4.

[0027] Figure 4 illustrates how the position of a robotic drone can be controlled via motion capture devices, according to various embodiments of the present invention. As with Figure 3, the scenario depicted in Figure 4 is provided for illustrative purposes and illustrates only one possible robot and one possible set of movements. The techniques described herein are applicable to any technically feasible robot and any technically feasible set of movements.

[0028] In Figure 4, the operator 114 moves the motion capture object 302 along a trajectory 400(A) from an initial position 410(A) to a final position 412(A). In doing so, the operator 114 articulates the arm 300 in a similar manner as described above in conjunction with Figure 3. However, in the exemplary scenario shown in Figure 4, the motion capture device 110 does not need to capture the specific articulation or movement of the arm 300.

[0029] Instead, motion capture device 110 tracks the trajectory of motion capture object 302 and generates motion capture signals 150. Based on motion capture signals 150, control engine 128 determines specific dynamics for robot 140 that cause sensor array 144 to traverse a trajectory similar to trajectory 400(A). Control engine 128 generates control signals 152 representing these dynamics and then transmits control signals 152 to robot 140 for execution. In response, robot 140 moves sensor array 144 from position 410(A) to position 410(B) along trajectory 400(B).

[0030] Depending on the type of robot implemented, control engine 128 may generate different control signals 152. In the example shown, robot 140 is a quadcopter drone. Thus, control engine 128 generates control signals 152 that modulate the thrust of one or more propellers, causing the drone to move sensor array 144 along trajectory 400(B). In another example, robot 140 may include an arm with several joints similar to robot 140 shown in FIG. 3. In this case, control engine 128 may determine a set of joint movements that will cause robot 140 to move sensor array 144 along trajectory 400(B). Again, those joint movements need not be similar to the joint movements of arm 300.

[0031] Using the approach described herein, the operator 114 indicates, via the motion capture object 302, a trajectory that the sensor array 144 should follow. The control engine 128 then causes the robot to follow a substantially similar trajectory using the sensor array 144. During operation, the sensor array 144 captures sensor data 154 for transmission to the control engine 128.

[0032] Generally, with reference to FIGS. 3-4 , those skilled in the art will understand that the techniques described above represent only two exemplary approaches for generating robot control signals based on motion capture data. Other approaches are within the scope of the present invention. For example, the techniques described in conjunction with FIGS. 2 and 3 may be combined. According to this combined technique, the control engine 128 may command the robot 140 to mimic the movements of the operator 114 in some situations and the trajectory of the motion capture object 302 in other situations. Alternatively, the control engine 128 may interpret the movements of the operator 114 according to a trained, gesture-based language. For example, the operator 114 may perform specific hand movements and indicate camera movements such as “zoom in” or “roll the camera left.” Using this approach, the operator 114 may have fine-grained control over both the movements of the robot 140 and more specific cinematography movements. In one embodiment, the control engine 128 records any and all movements performed by the operator 114 during real-time control of the robot 140 for later playback to the robot 140. In this manner, the robot 140 can be made to repeat a set of movements across multiple tasks without requiring the operator 114 to repeat the movements associated with them.

[0033] (Procedure for robot control via motion capture) 5 is a flow diagram of method steps for converting motion capture data into control signals for controlling a robot, according to various embodiments of the present invention. Although the method steps are described in conjunction with the systems of FIGS. 1-4, one skilled in the art will understand that any system configured to perform the method steps, in any order, is within the scope of the present invention.

[0034] As shown, method 500 begins at step 502, where motion capture device 110 receives motion capture data associated with operator 114. Motion capture device 110 may implement computer vision techniques to track the head, torso, and limbs of operator 114, or may rely on marking tracking techniques to track the movements of operator 114. The motion capture device transmits motion capture signals 150 to control engine 128 for processing.

[0035] In step 504, the control engine 128 processes the motion capture signals 150 and determines an articulation sequence to be performed by the robot 140 and / or an end effector path to be followed by the robot 140. An example of an articulation sequence is discussed above in conjunction with Figure 3. An example of an end effector path is described above in conjunction with Figure 4.

[0036] In step 506, the control engine 128 generates control signals 152 based on the determined articulation sequence and / or end effector path. The control signals 152 may vary depending on the type of robot 140 implemented. For example, to control a robot arm of a robot, the control signals 152 may include joint position commands. Alternatively, to control a robotic drone, the control signals 152 may include motor speed commands to modulate the thrust produced by a propeller.

[0037] In step 508, the control engine 128 transmits the control signals 152 to the actuators 142 of the robot 140. The control engine 128 may transmit the control signals 152 via one or more physical cables coupled to the robot 140, or may transmit the control signals 152 wirelessly.

[0038] In step 510, the control engine 128 captures multimedia data from a sensor array 144 coupled to the robot 140. The sensor array 144 may include any technically feasible type of sensor configured to measure physical quantities, including optical sensors, acoustic sensors, vibration sensors, etc. In practice, the sensor array 144 generally includes a video camera and possibly an audio capture device as well. The multimedia capture module 230 processes the sensor data from the sensor array 144 and generates multimedia data.

[0039] Those skilled in the art will appreciate that while the techniques described herein are described for various cinematographic operations, the techniques also apply generally to robotic control. For example, the motion capture device 110 and the control engine 128 may cooperate with each other to control a robot 140 for any technical purpose other than cinematography.

[0040] In essence, the motion capture device records the movements of the operator, and the control engine then converts those movements into control signals for controlling the robot. The control engine may directly convert the operator movements into similar movements to be performed by the robot, or the control engine may calculate robot dynamics that cause parts of the robot to mimic corresponding parts of the operator.

[0041] At least one advantage of the techniques described herein is that a human camera operator need not be exposed to discomfort or injury when shooting footage. Instead of holding the camera equipment and being physically present to shoot, the camera operator can simply operate a robot that then operates the camera equipment, potentially from a remote location.

[0042] The descriptions of various embodiments are provided for purposes of illustration and are not intended to be exhaustive or limiting of the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments.

[0043] Aspects of the present embodiments may be embodied as a system, method, or computer program product. Thus, aspects of the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, microcode, etc.), or an embodiment combining software and hardware aspects, all of which may generally be referred to herein as a "module" or "system." Furthermore, aspects of the present disclosure may take the form of a computer program product embodied in one or more computer-readable medium(s) having computer-readable program code embodied thereon.

[0044] Any combination of one or more computer-readable media may be utilized. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media would include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable ROM (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer-readable storage medium may be any tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device.

[0045] Aspects of the present disclosure are described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that the instructions, executing via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / acts defined in the blocks of the flowchart illustrations and / or block diagrams. Such a processor may be, but is not limited to, a general-purpose processor, a special-purpose processor, an application-specific processor, or a field-programmable processor or gate array.

[0046] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion that includes one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions described in the blocks may occur out of the order described in the figures. For example, two blocks shown in succession may be executed substantially concurrently, or may sometimes be executed in the reverse order, depending on the functionality involved. It should also be noted that each block of the block diagrams and / or flowchart diagrams, and combinations of blocks in the block diagrams and / or flowchart diagrams, may be implemented by a special-purpose hardware-based system that performs the specified functions or actions, or a combination of special-purpose hardware and computer instructions.

[0047] While the foregoing is directed to embodiments of the present disclosure, other and further embodiments of the present disclosure may be devised without departing from the basic scope thereof, which scope is determined by the claims that follow.

Claims

1. 1. A computer-implemented method for controlling a robot, the method comprising: generating motion capture data by tracking one or more movements of an operator; processing the motion capture data to generate control signals for controlling the robot based on the tracking to mimic the one or more movements of the operator, wherein one or more sets of sensors are mounted on the robot; transmitting the control signal to the robot to cause the robot to mimic the one or more movements of the operator; capturing sensor data using the set of one or more sensors while the robot is mimicking the one or more movements; generating multimedia content associated with the one or more movements of the operator based on data captured by the set of one or more sensors for storage in a database; Including, generating the control signal by processing the motion capture data, modeling a first set of dynamics of the operator corresponding to the one or more movements; determining a second set of motions of the robot corresponding to the one or more movements; generating the control signal based on the second set of behaviors; and Including, the first set of motions includes rotations of a set of joints of the operator; A computer-implemented method, wherein determining the second set of kinematics of the robot includes scaling the rotations of the set of joints of the operator by a predetermined amount.

2. 10. The computer-implemented method of claim 1, wherein tracking the one or more movements of the operator comprises tracking at least one of head, torso, and limb movements associated with the operator via motion capture equipment.

3. 2. The computer-implemented method of claim 1, wherein causing the robot to mimic the one or more movements of the operator comprises actuating a set of joints included in the robot to perform an articulation sequence associated with the one or more movements.

4. The computer-implemented method of claim 3 , wherein the robot comprises a robot arm that includes the set of joints.

5. The computer-implemented method of claim 1 , wherein causing the robot to mimic the one or more movements comprises causing only a first portion of the robot to mimic only a portion of the one or more movements.

6. The computer-implemented method of claim 1 , wherein the one or more movements comprise a target trajectory for the set of one or more sensors coupled to the robot.

7. A non-transitory computer-readable medium, the non-transitory computer-readable medium, when executed by a processor, generating motion capture data by tracking one or more movements of an operator; processing the motion capture data to generate control signals for controlling a robot based on the tracking to mimic the one or more movements of the operator, wherein one or more sets of sensors are mounted on the robot; transmitting the control signal to the robot to cause the robot to mimic the one or more movements of the operator; capturing sensor data using the set of one or more sensors while the robot is mimicking the one or more movements; generating multimedia content associated with the one or more movements of the operator based on data captured by the set of one or more sensors for storage in a database; causing the processor to control the robot by performing generating the control signal by processing the motion capture data, modeling a first set of dynamics of the operator corresponding to the one or more movements; determining a second set of motions of the robot corresponding to the one or more movements; generating the control signal based on the second set of behaviors; and Including, the first set of motions includes rotations of a set of joints of the operator; a non-transitory computer-readable medium, wherein determining the second set of kinematics of the robot includes scaling the rotations of the set of joints of the operator by a predetermined amount;

8. 1. A system for controlling a robot, the system comprising: a memory storing a control application; a processor coupled to the memory; Equipped with The processor: generating motion capture data by tracking one or more movements of an operator; processing the motion capture data to generate control signals for controlling the robot based on the tracking to mimic the one or more movements of the operator, wherein one or more sets of sensors are mounted on the robot; transmitting the control signal to the robot to cause the robot to mimic the one or more movements of the operator; capturing sensor data using the set of one or more sensors while the robot is mimicking the one or more movements; generating multimedia content associated with the one or more movements of the operator based on data captured by the set of one or more sensors for storage in a database; and generating the control signal by processing the motion capture data, modeling a first set of dynamics of the operator corresponding to the one or more movements; determining a second set of motions of the robot corresponding to the one or more movements; generating the control signal based on the second set of behaviors; and Including, the first set of motions includes rotations of a set of joints of the operator; determining the second set of kinematics of the robot includes scaling the rotations of the set of joints of the operator by a predetermined amount.

Images