Method and system for enhancing robot capability

Abstract

A method for controlling operation of a robot (105) in an environment (110) is provided. The robot (105) is configured to exchange data with the remote server (150) by establishing a robot-remote server communication (RL0) via the communication network (155). The method comprises:-causing the robot to be requested to perform a task in the environment (100); -generating (206) a robotic process RP adapted to be executed by the robot (105) to perform the task, said robotic process RP comprising a set of corresponding processes Pj, each process corresponding to one or more corresponding software applications SP comprising instructions for respectively controlling the operation of the robot (105) to perform the corresponding process Pj; -selecting (206) a set of at least one device (Di), each device (Di) of the set having a processing unit adapted to execute at least one of said software applications SP, each device (Di) of the set being located in an environment (110); -causing the remote server (150) to cause (208) each device (Di) of the set of devices to download and install at least one of the software applications SP; -causing the robot (105) to perform (210) the robot process RP at least partially under the control of a command sent by at least one device (Di) of the set, said command being generated by at least one device (Di) of the set by an instruction to cause a processing unit of the at least one device (Di) of the set to execute the installed at least one software application SP.

Description

Technical Field

[0001] The present invention generally relates to the field of robotics. More specifically, the present invention relates to a method and system for enhancing the processing capabilities of a robot. Background Art

[0002] The term robot herein is intended to refer to any kind of machine that is capable of performing tasks such as, for example, opening a closed door, grasping an object and transporting it to another location autonomously, ie by making decisions autonomously and without direct intervention by a human user.

[0003] The robot is equipped with at least one processing device, such as a processing unit, adapted to process data and autonomously make decisions to control actuator devices of the robot accordingly in order to perform one or more tasks.

[0004] Since the tasks to be performed by the robot are usually complex, the processing equipment on board the robot should have sufficient processing power to be able to manage very high computational loads. However, processing equipment capable of managing very high computational loads is expensive and / or involves high power consumption that cannot be efficiently handled by the robot's power source, such as a battery, without endangering the correct operation of the robot. In fact, equipping the robot with large batteries disadvantageously hinders the robot's mobility due to the excessive size and mass of the batteries.

[0005] A known solution to this shortcoming is to provide the robot with processing equipment that is less expensive and / or has reduced power consumption (which can be more easily managed by the robot's battery), and to compensate for the reduced processing power of the onboard processing equipment by borrowing processing power from a cloud computing system.

[0006] However, communication between the robot and a remote server of a cloud computing system (e.g., based on the Internet) may be subject to latency issues, which may jeopardize the correct operation of the robot if the task to be performed by the robot requires a quick response.

[0007] US20160182639 discloses an Internet of Things network system, and more specifically, discloses an Internet of Things network system in which the installation and version management of an application for operating a virtual machine of a detection server device is performed through a fog computing network in response to a request from an administrator, so that an application dedicated to the detection server device can be easily installed or managed through the fog computing network.

[0008] US10788836 discloses a method comprising capturing an image of a workspace by an image sensor placed on a robot; acquiring the captured image by a processor of the robot or via a cloud; comparing at least one object from the captured image with objects in an object dictionary by the processor of the robot or via a cloud; identifying a category to which at least one object belongs using an object classification unit by the processor of the robot or via a cloud; and instructing the robot by the processor of the robot to perform at least one action based on the identified object category.

[0009] The paper "Multi-level Resource Sharing Framework usingCollaborative Fog Environment for Smart Cities" published by Tariq Qayyum, Zouheir Trabelsi, Asad Waqar Malik and Kadhim Hayawi in IEEE Access, Volume 4, 2016, relates to a simulation framework for fog devices that can use end devices to handle peak computing loads to provide better quality of service (QoS). Regional fog nodes are deployed at the edge of the network, and they are used as intelligent agents to handle computing requests by scheduling them on local servers, cloud data centers, or underutilized end-user devices. The device-to-device resource sharing model relies on ant colony optimization (ACO) and earliest due first (EDF) algorithms to provide better quality of service using available devices at multi-level designs. The concept of using IoT devices as fog nodes improves the performance of traditional fog-based systems. Summary of the invention

[0010] None of the solutions known in the art is able to improve the operation of a robot equipped with an on-board processing device having low processing power.

[0011] The solution disclosed in US20160182639 is simply aimed at updating IoT devices.

[0012] According to the solution disclosed in US10788836, the robot is still forced to perform the task by utilizing its own processing capabilities.

[0013] The solution disclosed in the paper “Multi-level Resource Sharing Framework using Collaborative FogEnvironment for Smart Cities” only involves the distribution of computational load among fog nodes according to computational load peaks detected by smart nodes.

[0014] In view of the above, the Applicant has devised a solution to improve the operation of a robot equipped with an on-board processing device having low processing power.

[0015] One or more aspects of the invention are set out in the independent claims, and advantageous features of the same invention are indicated in the dependent claims, the wording of which is hereby incorporated verbatim by reference (and any advantageous features provided with respect to a specific aspect of the invention apply mutatis mutandis to any other aspect of the invention).

[0016] Aspects of the invention relate to a method for controlling operation of a robot in an environment.

[0017] The robot is configured to exchange data with the remote server by establishing robot-remote server communication via the communication network.

[0018] The method involves having a robot being asked to perform a task in an environment.

[0019] The method includes generating a robotic procedure adapted to be executed by a robot to perform the task.

[0020] The robot process includes a set of corresponding processes, each process corresponding to one or more corresponding software applications, which include instructions for controlling the operation of the robot accordingly to perform the corresponding process.

[0021] The method includes selecting a set of at least one device.

[0022] Each device in the set has a processing unit adapted to execute at least one of said software applications.

[0023] Each device in a collection is located in an environment.

[0024] The method includes causing each device in the set of devices to download and install at least one of the software applications.

[0025] The method includes causing the robot to perform a robotic process at least partially under control of commands sent by at least one device in the collection.

[0026] The command is generated by at least one device in the set by causing a processing unit of at least one device in the set to execute instructions of at least one installed software application.

[0027] In this way, the computational load required to allow the robot to correctly execute the various processes to perform the task is (at least partially) handled by the processing unit of the (one or more) selected devices, rather than entirely by (the processing unit of) the robot.

[0028] According to an embodiment of the invention, each device in the set has a hardware specification suitable for allowing the device to execute at least one of the software applications.

[0029] According to an embodiment of the present invention, the hardware specification of a device is intended to refer to at least one of the CPU, GPU, RAM and storage unit of the device, the processing capabilities of its CPU and GPU, and its data communication capabilities.

[0030] According to an embodiment of the present invention, the robot comprises at least one of the following robot modules:

[0031] - an actuator module configured to allow the robot to interact with the environment;

[0032] - a motion module configured to allow the robot to move in an environment;

[0033] - A sensor module configured to collect data from an environment.

[0034] According to an embodiment of the present invention, the command sent by the devices in the set is a command for controlling the operation of at least one of the robot modules so as to perform at least one corresponding process.

[0035] According to an embodiment of the present invention, each device in a set located in an environment is able to exchange data with the robot by establishing a corresponding local device-robot communication link without involving a communication network.

[0036] According to an embodiment of the invention, said command is sent by said at least one device in the set to the robot via a corresponding local device-robot communication link.

[0037] According to an embodiment of the present invention, the transmission delay of each local device-robot communication link is lower than the transmission delay of the robot-remote server communication between the robot and the remote server.

[0038] In this way, the robot is advantageously allowed to borrow processing power from other entities (devices) without incurring latency issues which could jeopardize the proper operation of the robot if the task to be performed by the robot requires a rapid response.

[0039] According to an embodiment of the present invention, the communication between the local device and the robot is less affected by the jitter problem than the communication between the robot and the remote server.

[0040] According to an embodiment of the present invention, the communication between the local device and the robot is less affected by the packet loss problem than the communication between the robot and the remote server.

[0041] According to an embodiment of the present invention, the method further comprises having the robot inspect the environment to identify equipment in the environment.

[0042] According to an embodiment of the present invention, the method further comprises having the robot collect device hardware specifications of each of the identified devices.

[0043] According to an embodiment of the present invention, said selecting a set of at least one device comprises selecting said set of at least one device among the identified devices.

[0044] In this way, when operating in an environment, the robot is advantageously provided with a potentially large number of different devices that can be utilized to support performance of the task.

[0045] According to an embodiment of the present invention, selecting the set of at least one device includes letting a remote server select the set of at least one device among the identified devices based on the collected device hardware specifications.

[0046] In this way, only devices with processing capabilities suitable for supporting the robot are advantageously selected.

[0047] According to an embodiment of the present invention, the method further includes having the robot send a generated robot process request for the robot process to a remote server.

[0048] According to an embodiment of the present invention, the robot process request further comprises a request parameter comprising at least one of the collected device hardware specifications, a task description describing a task to be performed by the robot in the environment, and the hardware specifications of the robot.

[0049] According to an embodiment of the present invention, the method further comprises having a remote server generate the robot process based on the request parameters.

[0050] In this way, the remote server is advantageously allowed to create a robotic process that conforms to the actual task to be performed, and / or select equipment with specifications suitable for supporting the robot in carrying out the specific task.

[0051] According to an embodiment of the present invention, the method further comprises having the robot collect data from the environment.

[0052] According to an embodiment of the present invention, the method further comprises letting the robot send the collected data to a remote server.

[0053] According to an embodiment of the present invention, the method further comprises having a remote server generate the task description based on the received collected data.

[0054] According to an embodiment of the present invention, the method further comprises letting the remote server send the generated task description to the robot.

[0055] In this way, the description of the tasks used by the remote server for the generation of the robot process is generated from the data collected by the robot in the environment.

[0056] According to an embodiment of the present invention, the method further comprises allowing a user of the remote server to generate the task description, and allowing the remote server to send the generated task description to the robot.

[0057] In this way, the description of the generated tasks used by the remote server for the robot process is generated from data provided by an entity other than the robot.

[0058] According to an embodiment of the present invention, the method further comprises having the remote server cause each device in the set of devices to uninstall at least one of the software applications installed in the device itself after the robot process has been executed.

[0059] According to an embodiment of the invention, each local device-robot communication link is one of the following:

[0060] -WiFi direct link;

[0061] -WLAN link;

[0062] - A communication link established across a local cell of a mobile network operating in core-in-a-box mode.

[0063] According to an embodiment of the present invention, the communication network includes the Internet.

[0064] Another aspect of the present invention relates to a system.

[0065] The system includes a robot located in an environment.

[0066] The system includes a remote server.

[0067] The robot is configured to exchange data with the remote server by establishing robot-remote server communication via the communication network.

[0068] The system is configured to generate a robotic process adapted to be executed by a robot to perform a task in an environment.

[0069] The robot process includes a set of corresponding processes, each process corresponding to one or more corresponding software applications, which include instructions for controlling the operation of the robot accordingly to perform the corresponding process.

[0070] The system is configured to select a set of at least one device.

[0071] Each device in the set has a processing unit adapted to execute at least one of said software applications.

[0072] Each device in the collection is located in the environment and is able to exchange data with the robot.

[0073] The system is configured to cause each device in the set of devices to download and install at least one of the software applications.

[0074] The robot is configured to perform a robotic process at least in part under the control of commands sent by at least one device in the set.

[0075] The command is generated by at least one device in the set by causing a processing unit of at least one device in the set to execute instructions of at least one installed software application.

[0076] According to an embodiment of the invention, the remote server is configured to generate said robotic process, which is adapted to be executed by the robot to perform the task in the environment.

[0077] According to an embodiment of the present invention, the remote server is configured to select said set of at least one device.

[0078] According to an embodiment of the invention, the remote server is configured to cause each device in the set of devices to download and install at least one of the software applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] These and other features and advantages of the present invention will become more apparent by reading the detailed description of the following exemplary and non-limiting embodiments of the present invention. For its better understandability, the following description should be read with reference to the attached drawings, in which:

[0080] Figure 1AThe following diagram illustrates a scenario of application of an embodiment of the invention;

[0081] Figure 1B The diagram shows an embodiment of the present invention. Figure 1A Some modules of the robot operating in the scene;

[0082] Figure 2 is a diagram illustrating an embodiment of the present invention. Figure 1A Used for operation in the scene Figure 1B A flow chart of the main operations of the robotic method;

[0083] Figure 3A and Figure 3B is a timing diagram showing how to generate a task description DTSK according to two different embodiments of the invention;

[0084] Figure 4 According to an embodiment of the present invention Figure 2 A timing diagram of the main operations of the flowchart;

[0085] Figure 5 Schematically illustrates a method according to an embodiment of the present invention. Figure 1A The robots and devices of the scene share the ROS environment. DETAILED DESCRIPTION

[0086] With reference to the accompanying drawings, Figure 1A A scenario of application of an embodiment of the present invention is illustrated, in which a robot 105 located in an environment 110 (eg, a house, a factory, a warehouse) is configured to perform tasks that require interaction with the environment 110 .

[0087] Non-limiting examples of robot 105 may include an inspection robot configured to perform a patrol mission to patrol an area such as a warehouse or industrial plant to assess potential abnormal situations, or a delivery robot configured to perform a delivery mission to transport goods from one location to another.

[0088] like Figure 1B As schematically illustrated in , according to an embodiment of the present invention, the robot 105 may include several modules, the types and functions of which depend on the specific application of the robot 105 itself. It should be noted that the terms "unit", "system", "module" are intended to include, but are not limited to, hardware, firmware, a combination of hardware and software, and software in this document. Examples of possible modules included in the robot 105 are shown below.

[0089] According to an embodiment of the present invention, the robot 105 includes one or more processing modules 121 (e.g., one or more processor units), which are adapted to autonomously control the operation of one or more other modules of the robot 105 so as to allow the robot 105 to perform one or more tasks.

[0090] According to an embodiment of the present invention, the robot 105 includes one or more sensor modules 122 adapted to collect data from the environment in which the robot 105 operates. Examples of the sensor module 122 may include one or more of a camera, a proximity sensor, a temperature sensor.

[0091] According to an embodiment of the present invention, the robot 105 includes one or more motion modules 124 adapted to allow movement of the robot 105 across the environment 110 in which the robot 105 operates. Examples of the motion modules 124 may include a track or wheel system for allowing terrestrial movement of the robot 105, or a rotary rotor system for allowing aerial movement of the robot 105.

[0092] According to an embodiment of the invention, the robot 105 includes one or more actuator modules 125 adapted to allow the robot 105 to interact with the environment 110 in which the robot 105 operates. Examples of actuator modules 125 may include a robotic arm or a gripper.

[0093] According to an embodiment of the present invention, the robot 105 includes one or more communication modules 126 (eg, including a radio transceiver coupled to an antenna) adapted to wirelessly transmit / receive data to / from an external device.

[0094] According to an embodiment of the present invention, the robot 105 includes a power module 128 , for example including one or more batteries, and the power module is used to supply power to other modules of the robot 105 .

[0095] According to an embodiment of the present invention, in order to describe the robot 105 from the perspective of its ability to perform tasks, the robot 105 stores a robot capability list RC in, for example, a memory unit 130 coupled to the processing module 121, the robot capability list listing at least one of the following items:

[0096] - the processing capabilities of its processing module 121;

[0097] - specifications of its sensor modules 122 and description of their interfaces;

[0098] - Hardware specifications of its motion modules 124 and description of their interfaces;

[0099] - Hardware specifications of its actuator modules 125 and description of their interfaces.

[0100] The "processing capability" of a processing module is intended herein to refer to the ability of the processing module to perform computer operations, such as in at least one of the following:

[0101] - the time required to perform the operation;

[0102] - the number of operations that can be managed in parallel;

[0103] -The complexity of the operations that can be performed.

[0104] Return to Figure 1A According to an embodiment of the present invention, the robot 105 is configured to exchange data with one or more remote servers 150 (only one is shown) of the cloud computing system. For this purpose, according to an embodiment of the present invention, the communication module 126 of the robot 105 is configured to establish a corresponding communication RLO with the remote server 150 through a communication network 155 such as the Internet. For example, the communication module 126 can be connected to a router device (not shown in the figure) that provides access to the communication network 155, or if the communication module 126 is equipped with a SIM / eSIM card / module, the communication network 155 can be accessed through a mobile communication network (not shown in the figure).

[0105] According to an embodiment of the present invention, the robot 105 (the communication module 126 ) is also configured to establish a local communication link LLi with an electronic device Di (i=1, 2, . . . ) located in the environment 110 and close to the actual location of the robot 105 .

[0106] According to an embodiment of the present invention, the device Di may be any kind of electronic device located near the robot 105, which is capable of establishing a local communication link LLi with (the communication module 126 of) the robot 105, and which is equipped with at least a processing module (e.g., a microprocessor). Non-limiting examples of this kind of device Di may include a smartphone, a personal computer, a smart TV, a game console, a router device, other robots, and a workstation.

[0107] According to an embodiment of the present invention, the local communication link LLi between the robot 105 and the device Di is configured to allow data exchange without involving the communication network 155. According to an embodiment of the present invention, the local communication link LLi can be implemented with any known wireless communication technology for local / direct / short-range data communication such as, for example, WiFi Direct, WLAN communication technology. The local communication link LLi can also be established across a local cell of a mobile network operating in a "core in a box" mode, i.e., utilizing mobile network-based communications managed entirely within the local cell without involving the core portion of the mobile network.

[0108] According to an embodiment of the present invention, the device Di is also configured to establish communication RLi with the communication network 155 for exchanging data with the remote server 150. For example, the device Di is configured to interface with a router device (not shown in the figure) that provides access to the communication network 155, and / or is equipped with a SIM / eSIM card / module that allows access to the communication network 155 through a mobile communication network.

[0109] The delay affecting data exchange through the local communication link LLi between the robot 105 and the device Di is significantly lower than the delay affecting data exchange using the communication RL0 between the robot 105 and the remote server 150. In fact, the data packets transported by the method of communication RL0 between the robot 105 and the remote server 150 must travel a very long distance across many network nodes of the communication network 155 (e.g., physical network nodes including host computers), while the data packets transported along the local communication link LLi between the robot 105 and the device Di must travel a very short distance (the device Di and the robot 105 are located near each other) and do not have to involve other network nodes. Therefore, when the bandwidth is the same, the direct communication between the robot 105 and the device Di near the robot 105 benefits from higher throughput compared to the communication between the robot 105 and the remote server 150.

[0110] Furthermore, data exchange via the local communication link LLi between the robot 105 and the device Di is less susceptible to jitter problems than data exchange via the communication RL0 between the robot 105 and the remote server 150 .

[0111] Similarly, data exchanged via the local communication link LLi between the robot 105 and the device Di is less susceptible to packet loss problems than data exchanged via the communication RL0 between the robot 105 and the remote server 150.

[0112] Generally speaking, the solution according to an embodiment of the present invention provides for the robot 105 to temporarily borrow processing power from one or more selected devices Di among the devices Di in the vicinity of the robot 105 when the robot 105 has to perform a task.

[0113] According to an embodiment of the present invention, the selection of the device Di for this purpose is performed by taking into account the specific task to be performed by the robot 105, the robot capability list RC of the robot 105, and / or the hardware specifications SDi of the device Di. The hardware specifications SDi of the device Di are herein intended to refer to at least one of the identification of the CPU, GPU, RAM and storage units of the device Di, the processing power of its CPU and GPU, and the latency of the local communication link LLi between the device Di and the robot 105.

[0114] Figure 2 is a flowchart illustrating main operations of a method for operating the robot 105 according to an embodiment of the present invention.

[0115] According to an embodiment of the invention, the robot 105 checks the environment 110 to identify the presence of devices Di in its vicinity and collects their specifications SDi (box 202). For this purpose, according to an embodiment of the invention, the robot 105 scans its surrounding area with its communication module 126, for example using a long polling transmission scan with a corresponding configurable polling time period T, and sends a request message GET_SPEC() to each identified device Di to obtain the specifications SDi of the device Di. The specifications SDi sent by the various devices Di are collected by the robot 105, for example in the form of a device list DL stored at the robot 105, for example in a memory unit 130. According to an embodiment of the invention, when a device Di is no longer available, for example because the actual distance between the robot 105 and the device Di is too great for establishing a local communication link LLi or because the device Di has been switched off, its specifications SDi are removed from the device list DL.

[0116] According to an embodiment of the invention, the robot 105 retrieves a task description DTSK describing a task TSK to be performed by the robot 105 itself (block 204 ).

[0117] According to the diagram in Figure 3A In the embodiment of the present invention illustrated in the timing diagram in, the task description DTSK is generated in the following manner.

[0118] The robot 105 uses one or more of its sensor modules 122 to collect data CD from the environment 110 (action 302). For example, if the robot 105 is in front of a closed door, the data CD may correspond to a picture of the closed door taken from the robot 105 onboard camera. Next, the robot 105 sends the collected data CD to the remote server 150 (action 304) using the communication RL0 (see FIG. 1) established between the robot 105 and the remote server 150. After the remote server 150 receives the data CD, the data CD is processed to generate a task description DTSK (action 308). According to an embodiment of the present invention, the remote server 150 generates the task description DTSK by processing the received data CD using an artificial intelligence (AI) algorithm such as, for example, an object recognition algorithm. By considering the above example, where the data CD may correspond to a picture of a closed door, the task description DTSK may therefore correspond to the recognition of "to open the closed door". Next, the task description DTSK is sent (action 310) to the robot 105 using the communication RL0 (see FIG. 1) established between the robot 105 and the remote server 150.

[0119] According to the diagram in Figure 3B In another embodiment of the present invention illustrated in the timing diagram in FIG. 3 , the task description DTSK may be directly generated by a user of the remote server 150 (action 312) and then sent (action 314) to the robot 105 using the communication RLO established between the robot 105 and the remote server 150. For example, the task description DTSK may be provided in the form of a natural language (e.g., in the form of a string like "open the closed door in front of you").

[0120] It is noted that the concept of the present invention can be applied to instances in which the task description DTSK is generated in different ways. For example, the task description DTSK can be provided with processing and transmission capabilities by an object located in the environment 110. Referring to the previous example, in which the robot 105 is in front of a closed door, the door itself can be provided with hardware that is configured to broadcast data including information describing the door itself. The data is collected by the robot 105 and sent to the remote server 150, so that the remote server 150 can use the data to generate the corresponding task description DTSK.

[0121] Return to Figure 2According to an embodiment of the present invention, the remote server 150 generates a robot process RP adapted to be executed by the robot 105 to implement the task TSK described by the task description DTSK (block 206). According to an embodiment of the present invention, the robot process RP is generated by considering the capabilities of the robot 105 (i.e., the robot capability list RC) and the hardware specifications SDi of the devices Di near the robot 105 (i.e., the device list DL) in addition to the specific task TSK to be implemented (i.e., the task description DTSK). According to different embodiments of the present invention, the robot process RP may be generated, for example, by the robot 105 or by one of the devices Di near the robot 105.

[0122] By reference Figure 4 , the generation of the robot process RP according to the embodiment of the present invention is carried out in the following manner.

[0123] According to an embodiment of the present invention, the robot 105 sends (action 402) a request GET_RP to obtain the robot process RP to the remote server 150 using the communication RL0 established between the robot 105 and the remote server 150. According to an embodiment of the present invention, the request GET_RP includes (for example, in the form of request parameters) the task description DTSK, the robot capability list ...

[0124] RC and the device list DL. In this way, the remote server 150 is provided with the robot

[0125] A description of the task TSK that the robot 105 must perform, the capabilities of the robot 105 itself, and a list of devices Di (and their specifications) in the vicinity of the robot 105 that can be used to support the robot 105 in performing the task TSK.

[0126] According to an embodiment of the present invention, the remote server 150 processes (action 404) the received request GET_RP to generate:

[0127] a robot process RP suitable for being executed by the robot 105 to carry out the task TSK;

[0128] - A supported device list SUD, which specifies one or more devices Di (selected from the devices Di in the device list DL), which are used to support the robot 105 when executing the robot process RP with the processing power of their GPU and / or CPU, their RAM, their storage unit and / or their connection capabilities.

[0129] According to an embodiment of the present invention, the robot process RP includes a set of corresponding processes Pj (j=1, 2, 3, ...) to be executed by the robot 105 to implement the task TSK. According to an embodiment of the present invention, the processes Pj in the set are arranged in an ordered sequence, which the robot 105 must follow to correctly implement the corresponding task TSK. According to an embodiment of the present invention, each process Pj corresponds to one or more corresponding software applications SP, which include instructions that can be executed by a processor device to control the selected modules of the robot 105 accordingly so as to execute the process Pj.

[0130] According to an embodiment of the invention, the remote server 150 generates the robot process RP by processing the task description DTSK using an AI algorithm capable of translating the task description DTSK into a sequence of (elementary) processes Pj.

[0131] By referring to the previously considered example, where the robot 105 is in front of a closed door, and the task description DTSK may correspond to "to open the closed door" or "open the closed door in front of you", the AI ​​algorithm may generate a robot process RP comprising the following exemplary and non-limiting sequence of processes Pj:

[0132] P1: Identify the size and position of the door in front of you through the onboard camera;

[0133] P2: bend the onboard robotic arm to reach the door handle;

[0134] P3: Grasp the handle with an onboard gripper attached to the robotic arm;

[0135] P4: Move the robotic arm to open the door;

[0136] P5: Identify the door opening status through the onboard camera;

[0137] P6: The handle is released by opening the gripper when the door opening is detected.

[0138] According to an embodiment of the present invention, each process Pj is selected by the AI ​​algorithm running at the remote server 150 by considering (through the robot capability list RC) the capabilities of the robot 105. By referring to the above example, the process P1 can be selected by considering the hardware specifications of the camera of the robot 105, the process P2 can be selected by considering the hardware specifications of the robot 105's mechanical arm, and so on.

[0139] As will be described in more detail below, according to an embodiment of the present invention, instructions of the software application SP corresponding to the process Pj are advantageously executed by a CPU / GPU of at least one of the devices Di supporting the device list SUD in addition to or instead of being executed by the processing module 121 of the robot 105. In this way, the processing capabilities of the processing module 121 of the robot 105 can be supported or replaced by the processing capabilities of the CPU / GPU of at least one of the devices Di.

[0140] According to an embodiment of the present invention, the generation of the support device list SUD is performed by the remote server 150 (or by another entity of the system, such as, for example, by the robot 105 or by one of the devices Di near the robot 105) by selecting (one or more) devices Di from the devices Di near the robot 105, the hardware specifications SDi of the (one or more) selected devices Di (in particular, the processing power of their CPU / GPU) being sufficient to allow the (one or more) selected devices Di to execute the instructions of the software application SP of the process Pj on their CPU / GPU without causing delays that may endanger the correct execution of the process Pj. According to an embodiment of the present invention, the generation of the support device list SUD may also be performed by an AI algorithm running at the remote server 150.

[0141] According to a non-limiting embodiment of the present invention, the user of the device Di can agree / refuse to use his / her device to support the operation of the robot 105. For example, if the user allows the supported device list SUD to include his / her device, he / she can receive compensation.

[0142] In the example illustrated in the figure, the supported device list SUD generated by the remote server 150 includes a single device Di, namely D2.

[0143] According to an embodiment of the present invention, the generated robot process RP and the support device list SUD are then sent (action 408 ) to the robot 105 using the communication RL0 established between the robot 105 and the remote server 150 .

[0144] Return to reference Figure 2 According to an embodiment of the present invention, the remote server 150 causes the software application SP corresponding to the process Pj of the generated robot process RP to be installed into the (one or more) devices Di listed in the support device list SUD (box 208). According to different embodiments of the present invention, the installation of the software application SP corresponding to the process Pj into the (one or more) devices Di listed in the support device list SUD can be caused by another entity of the system, such as, for example, by the robot 105 or by one of the devices Di near the robot 105.

[0145] By referring again Figure 4 , according to an embodiment of the present invention, this installation is carried out in the following manner.

[0146] According to an embodiment of the present invention, the robot 105 (using the communication RL0 established between the robot 105 and the remote server 150) sends (action 410) a request ALLOC_(RP,SUD) to the remote server 150 to install the software application SP of the process Pj corresponding to the generated robot process RP to the (one or more) selected devices Di listed in the list of supported devices SUD (in the example considered, the device D2).

[0147] According to an embodiment of the invention, the remote server 150 then sends (action 420) to the selected device(s) Di a command LOAD_START(RP, RC) for activating the selected device(s) Di, so that the selected device(s) Di downloads and installs the software application SP of the process Pj corresponding to the generated robot process RP, and configures the installed software application SP according to the capabilities of the robot 105 (i.e., according to the capability list RC). According to an embodiment of the invention, the command LOAD_START(RP, RC) is sent to the selected device(s) Di by utilizing the communication RLi established between the remote server 150 and each of the selected device(s) Di. In the example considered, the selected device Di is the device D2, and the command LOAD_START(RP, RC) is sent using the communication RL2. According to an embodiment of the present invention, the command LOAD_START(RP,RC) includes, or is sent together with, an indication of the software application SP and / or a way to obtain them of the process Pj corresponding to the robot process RP, such as, for example, a list of download links DL (for example, in the form of URLs) for downloading the software application SP.

[0148] According to an embodiment of the invention, the selected device Di (in the example in question, the device D2) activates, downloads the software applications SP through the download link DL, installs them and then configures them according to the capability list RC (action 422).

[0149] Return to Figure 2According to an embodiment of the present invention, when the software application SP corresponding to the process Pj of the generated robot process RP has been installed in the (one or more) selected devices Di listed in the supported device list SUD and has been configured according to the capabilities of the robot 105, the robot 105, with the support of the (one or more) selected devices Di, uses the CPU / GPU of the (one or more) selected devices to execute the process Pj of the robot process RP (box 210), and the CPU / GPU executes the instructions of the software application SP installed thereon for controlling the module of the robot 105.

[0150] By referring again Figure 4 , according to an embodiment of the present invention, the robot process RP is executed in the following manner.

[0151] The robot 105 and the selected device(s) Di establish a data connection via a corresponding local communication link LLi (in the example considered, the device D2 exchanges data with the robot 105 via the local communication link LL2). Through this data connection, the robot 105 and the selected device(s) Di exchange commands and messages, allowing the robot 105 to execute a process Pj of the robot process RP (action 450) by having its modules (e.g., sensor modules 122 and actuator modules 125) controlled by commands sent by the selected device(s) Di and generated by the CPU / GPU of the selected device(s) Di in response to the execution of instructions of the software application SP installed thereon and in response to feedback generated by the robot's sensor module(s) 122 during the execution of the process Pj.

[0152] In this way, according to an embodiment of the present invention, the computational load required to allow the robot 105 to correctly execute each process Pj is (at least partially) processed by the CPU / GPU of (one or more) selected devices, rather than being processed entirely by the processing module 121 of the robot 105.

[0153] In other words, according to an embodiment of the present invention, the robot 105 is allowed to temporarily borrow processing power from (the CPU / GPU of) its nearby device Di for executing the robot process RP.

[0154] Because according to an embodiment of the present invention, the robot 105 and (one or more) selected devices Di exchange data through a local communication link LLi with low latency, the commands sent by (one or more) selected devices Di to the robot 105 for controlling the operation of the modules of the robot 105, as well as the messages generally exchanged between the robot 105 and (one or more) selected devices Di, are not affected by delays that could endanger the correct execution of the robot process RP and therefore the correct execution of the task TSK.

[0155] On the contrary, if the robot 105 has to borrow processing power from the remote server 150, as in known solutions based entirely on cloud computing systems, the messages and commands exchanged between the robot 105 and the remote server 150 may be subject to delays due to the high latency of the communication RL0 between the robot 105 and the remote server 150, thereby potentially jeopardizing the correct execution of the robot process RP and, therefore, the correct execution of the task TSK.

[0156] according to Figure 5 In the exemplary and non-limiting embodiment of the present invention illustrated in the figure, in order to execute the robot process RP, the exchange of commands and messages between the robot 105 and (one or more) selected devices Di listed in the support device list SUD through the local communication link LLi is implemented using the known Robot Operating System (ROS) framework, where the software application SP corresponding to the general process Pj is an application called ROS node N.

[0157] As known to those skilled in the art, ROS nodes N can interact and exchange data with each other through a message interface using publish-subscribe and request-response paradigms in the following three ways:

[0158] - Topics: publisher applications publish messages on topics, and subscriber applications subscribe to said topics to receive published messages (synchronous mechanism);

[0159] - Service: The server application processes the request message and returns a reply message to the client application (synchronous mechanism);

[0160] - Action: The action server application processes the target message and returns feedback messages to the action client application during the processing, and returns a result message to the action client application when the processing is finished (hybrid asynchronous-synchronous mechanism).

[0161] Each selected device Di listed in the supported device list SUD and connected to the robot 105 via a corresponding communication link LLi is configured to share the ROS environment of the robot 105, which is configured in Figure 5500. The ROS interface exposed by the ROS node N is shared between the robot 105 and the device Di. The ROS node N is configured to control the operation of (modules of) the robot 105 to perform the process Pj through a suitable ROS interface.

[0162] Return to Figure 2 According to an embodiment of the present invention, when all processes Pj of the robot process RP have been executed, the remote server 150 causes the software application SP to be uninstalled from the device(s) Di listed in the supported device list SUD (block 212).

[0163] By referring again Figure 4 , according to an embodiment of the present invention, the unloading is performed in the following manner.

[0164] According to an embodiment of the present invention, when the last process Pj of the robot process RP has been executed, which means that the robot 105 has completed the task TSK, the robot 105 (using the communication RL0 established between the robot 105 and the remote server 150) sends (action 460) a request DEALLOC_(RP,SUD) to the remote server 150 to uninstall the software application SP corresponding to the process Pj from (one or more) devices Di (in the example considered, device D2) included in the supported device list SUD.

[0165] For example, by referring to the example discussed, in which the task TSK to be performed involves the opening of a closed door, the robot 105 can send a request DEALLOC_(RP, SUD) when the door is actually opened.

[0166] By referring to the example discussed, where the software application SP corresponding to the process Pj is a ROS node N, a "closed door" event can be generated by publishing a corresponding ROS topic conveying a message reporting the state of the door. This message can be appropriately subscribed by the ROS node N, which after the message is processed causes the request DEALLOC_(RP, SUD) to be sent to the remote server 150.

[0167] According to an embodiment of the present invention, the remote server 150 then sends (action 470) a STOP_UNLOAD (RP) command to the (one or more) devices Di listed in the supported device list SUD to terminate the operation of the software application SP installed on the (one or more) devices Di, so that the software application SP is uninstalled from the (one or more) devices Di, and the (one or more) devices Di may be shut down (or put into an inactive (passive) state).

[0168] Naturally, in order to meet local and specific needs, those skilled in the art may apply many logical and / or physical modifications and changes to the invention described above. More specifically, although the present invention has been described with reference to its preferred embodiments to a certain degree of specificity, it should be understood that omissions, substitutions and changes in form and detail and other embodiments are possible. Specifically, different embodiments of the invention may be implemented even without the specific details set forth in the foregoing description for providing a more thorough understanding of the invention, whereas well-known details may have been omitted or simplified so as not to obstruct the description with unnecessary details. In addition, it is clearly contemplated that the specific elements and / or method steps described in conjunction with any disclosed embodiment of the invention may be incorporated into any other embodiment.

[0169] More specifically, the invention can be implemented by equivalent methods (by using similar steps, removing some unnecessary steps, or adding further optional steps); moreover, the steps may be performed in a different order, simultaneously or in a (at least partially) staggered manner.

Claims

1. A method for controlling operation of a robot (105) in an environment (110), the robot (105) being configured to exchange data with a remote server (150) by establishing robot-remote server communication (RLO) via a communication network (155), the method comprising: - allowing the robot to be required to perform a task in the environment (100); - generating (206) a robot process adapted to be executed by the robot (105) to perform the task, the robot process comprising a set of corresponding processes, each process corresponding to one or more corresponding software applications, the software applications comprising instructions for controlling the operation of the robot (105) accordingly to perform the corresponding process, - selecting (206) a set of at least one device (Di), each device (Di) of the set having a processing unit suitable for executing at least one of the software applications, each device (Di) of the set being located in the environment (110); - causing (208) each device (Di) of said set of devices to download and install at least one of said software applications; - causing the robot (105) to perform (210) the robot process at least partially under the control of a command sent by at least one device (Di) in the set, the command being generated by the at least one device (Di) in the set by causing the processing unit of the at least one device (Di) in the set to execute the instructions of at least one installed software application.

2. The method according to claim 1, wherein the robot (105) comprises at least one of the following robot modules: - an actuator module (125) configured to allow the robot (105) to interact with the environment (110); - a motion module (124) configured to allow the robot (105) to move in the environment (110); - a sensor module (122) configured to collect data from the environment (110), Said commands sent by the devices (Di) in said set are commands for controlling the operation of at least one of said robotic modules so as to perform at least one corresponding process.

3. A method according to any one of the preceding claims, wherein: - each device (Di) of the set located in the environment (110) is capable of exchanging data with the robot (105) without involving the communication network (155) by establishing a corresponding local device-robot communication link (LLi), the command being sent by the at least one device (Di) of the set to the robot (105) via the corresponding local device-robot communication link (LLi), - The transmission delay of each local device-robot communication link (LLi) is lower than the transmission delay of the robot-remote server communication (RL0) between the robot and the remote server (105).

4. The method according to any one of the preceding claims, further comprising: - having the robot (105) inspect (202) the environment (110) to identify a device (Di) in the environment; - having the robot (105) collect device hardware specifications of each of the identified devices (Di), wherein: - said selecting (206) a set of at least one device (Di) comprises selecting (206) said set of at least one device (Di) among said identified devices (Di).

5. The method according to claim 4, wherein the selecting (206) the set of at least one device (Di) comprises letting the remote server (150) select (206) the set of at least one device (Di) among the identified devices (Di) based on the collected device hardware specifications.

6. The method according to claim 4 or 5, further comprising: - having the robot (105) send (206) to the remote server (150) a generated robot process request for the robot process, the robot process request further comprising request parameters, the request parameters comprising at least one of: - the collected device hardware specifications, - a task description describing the task to be performed by the robot (105) in the environment (110), and - hardware specifications of the robot (105), - having the remote server (150) generate (206) the robot process based on the request parameters.

7. The method according to claim 6, further comprising: - allowing the robot (105) to collect data from the environment (110); - having the robot (105) send the collected data to the remote server (150); - having the remote server (150) generate the task description based on the received collected data; - Having the remote server (150) send the generated task description to the robot (105).

8. The method according to claim 6, further comprising: - allowing a user of the remote server (150) to generate the task description; - Having the remote server (150) send the generated task description to the robot (105).

9. The method according to any of the preceding claims further includes having the remote server (150) cause each device (Di) of the set of devices (212) to uninstall at least one of the software applications installed in the device itself after the robotic process has been executed.

10. A system comprising: - a robot (105), said robot being located in an environment (110); a remote server (150), the robot (105) being configured to exchange data with the remote server (150) by establishing a robot-remote server communication (RLO) via a communication network (155), in: the system is configured to generate a robotic process adapted to be executed by the robot to perform a task in the environment, the robotic process comprising a set of corresponding processes, each process corresponding to one or more corresponding software applications, the software applications comprising instructions for controlling the operation of the robot accordingly to perform the corresponding process, - the system is configured to select a set of at least one device (Di), each device (Di) of the set having a processing unit suitable for executing at least one of the software applications, each device (Di) of the set being located in the environment (110) and being able to exchange data with the robot (105), - said system being configured to cause each device (Di) of said set of devices to download and install at least one of said software applications, and in: -The robot (105) is configured to perform (210) the robot process at least partially under the control of a command sent by at least one device (Di) in the set, the command being generated by the at least one device (Di) in the set by having the processing unit of the at least one device (Di) in the set execute the instructions of at least one installed software application.

11. The system of claim 10, wherein: - the remote server (150) is configured to generate the robotic process, the robotic process being adapted to be executed by the robot to perform a task in the environment; - said remote server (150) is configured to select said set of at least one device (Di); - said remote server (150) is configured to cause each device (Di) of said set of devices to download and install at least one of said software applications.

Images