Methods & systems for remote robot control

Abstract

A first method comprises: Receiving, by client device 110, instructions to control robot 100, wherein the instructions comprise a first predetermined datatype corresponding to a robot modality, the ro

Claims

1. A computer-implemented method for remotely controlling a robot, the method comprising, at a client device:connecting to a robot remote from the client device via a data connection over a network, the robot comprising a plurality of sensors and / or actuators, at least two of the plurality of sensors and / or actuators corresponding to a single robot modality of one or more robot modalities;receiving, by the client device, instructions to control operation of the robot, wherein the instructions comprise a first predetermined datatype corresponding to the single robot modality, the first predetermined datatype encoding data corresponding to the at least two of the plurality of sensors and / or actuators;transmitting, by the client device, the instructions to the robot via the data connection to control operation of the robot in accordance with the instructions.

2. The method of claim 1, wherein the instructions are encoded, by the client device, into a serialised data format for transmission over the data connection.

3. The method of any claim 1 or 2, wherein the network is a wide area network.

4. The method of claim 1,2 or 3, wherein the data connection is a peer-to-peer dataconnection.

5. The method of any preceding claim, wherein the first predetermined datatype is one of a plurality of predetermined datatypes, the plurality of predetermined datatypes comprising at least one predetermined datatype for encoding data and corresponding to a visual modality of the one or more robot modalities.

6. The method of claim 5, wherein the plurality of predetermined datatypes comprises at least one predetermined datatype for encoding data and corresponding to a motor modality of the one or more robot modalities.

7. The method of any preceding claim, wherein the plurality of sensors and / or actuators comprise at least one camera and at least one motor.

8. The method of claim 7, wherein the robot is a first robot of a plurality of robots and wherein the first robot has a different arrangement of sensors and / or actuators to a second robot of the plurality of robots.

9. The method of any preceding claim, further comprising:establishing the connection in accordance with a set of WebRTC protocols.

10. The method of claim 2 or any claim dependent thereon, wherein the serialised data format is a Protobuf data format.

11. The method of any preceding claim, wherein the robot is located in a different geographical location to the client device.

12. The method of any preceding claim, further comprising:receiving, from the robot over the connection, data from at least two actuators and / or sensors encoded using the first predetermined datatype.

13. The method of any preceding claim, wherein each one of the one or more robot modalities indicates a mode in which the robot can interact with or experience an environment.

14. A computer-implemented method for controlling a robot, comprising:identifying a first plurality of sensors capable of receiving an input from an environment, wherein each sensor of the first plurality of sensors are different ones of the first plurality of sensors each corresponding to a first robot modality;receiving, by the robot, a first input data stream from a first sensor of the plurality of sensors;receiving, by the robot, a second input data stream from a second sensor of the plurality of sensors;mapping the first input data stream and the second input data stream to a robot coordinate system of the robot;generating a robot modality input by combining the first input data stream and the second input data stream; andtransmitting data to a client device from the robot, the data comprising the robot modality input.

15. The method of claim 14, further comprising receiving, by the robot from the client device, an instruction configured to actuate the robot, the instruction defined with reference to the robot coordinate system; andcausing the robot to be controlled in accordance with the instruction.

16. The method of claim 14 or 15, wherein each one of the plurality of sensors is a camera sensor, a LIDAR sensor, a sonar sensor, a microphone, or a radar sensor.

17. The method of any one of claims 14 to 16, further comprising:receiving a plurality of input data streams from the plurality of sensors, wherein each one of the plurality of input data streams corresponds to a different one of the plurality of sensors;generating a plurality of robot modality inputs, each corresponding to a respective robot modality, comprising:generating, for each of the plurality of robot modalities, a robot modality input by combining two or more of the plurality of input data streams that are associated with the robot modality.

18. The method of any one of claims 14 to 17, wherein the coordinate system is based upon a point of origin defined with reference to the robot.

19. A computer readable storage medium comprising computer readable instructions, the computer readable instructions, when executed by a processor, cause the processor to carry out the method of any preceding claim.

20. An apparatus comprising:a processor;a memory comprising computer readable instructions, that when executed by the processor, cause the processor to carry out the method of any of claims 1 to 18.

Images