System for assessing the skills of a robot for carrying out parallel missions

The system for humanoid robots optimally manages parallel missions by using an interpretation and supervision module with task planners and low-level runes to ensure real-time adaptation and deterministic task execution, addressing the challenges of evolving constraints and resource limitations.

WO2026233795A1PCT designated stage Publication Date: 2026-11-12ENCHANTED TOOLS
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
PCT/EP2026/062217
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2025-05-07
Filing Date
2026-04-28
Publication Date
2026-11-12

AI Technical Summary

Technical Problem

Existing technologies fail to optimally manage and execute parallel missions for robots, particularly humanoid robots, due to the probabilistic nature of foundation models and the lack of real-time consideration of evolving constraints and limited resources.

Method used

A system comprising an interpretation and supervision module, coupled with a task planner, processes information from perception, skills, and world state to determine an ordered list of missions with priority levels, using foundation models like large language models, and incorporates low-level runes for deterministic control, ensuring real-time adaptation and resource management.

Benefits of technology

Enables efficient execution of parallel missions by humanoid robots, accounting for evolving constraints and resource limitations, ensuring deterministic and prioritized task execution.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure EP2026062217_12112026_PF_FP_ABST
    Figure EP2026062217_12112026_PF_FP_ABST
Patent Text Reader

Abstract

The present invention relates to a system for assessing the skills of a robot for carrying out a set of missions, the robot having a set of skills, a set of perception means and a set of available hardware resources, the system comprising an interpretation and supervision module coupled to a task scheduler that is configured to manage the missions and to determine an ordered list of missions based on information received from the set of skills, the set of perception means and the set of available hardware resources, each mission having a priority level, the task scheduler defining a list of tasks to be accomplished by the robot, multiple tasks in the list of tasks being able to be implemented in parallel by the robot where appropriate.
Need to check novelty before this filing date? Find Prior Art

Description

A robot's skill control system for performing parallel missions

[0001] The present invention relates to the field of robots, particularly humanoid robots. Such robots possess diverse capabilities. A PDDL (Planning Domain Definition Language) interface can manage instruction sets, that is, a sequence of assigned tasks, enabling a robot to accomplish a mission or a series of missions, executed using the robot's capabilities. The invention specifically aims to optimally manage the execution of a mission assigned to the robot, or even situations in which several missions are assigned to the robot in parallel. STATE OF THE ART

[0002] The principle of foundation models, including large language models or large visual models, is known for interpreting received instructions and setting a list of missions assigned to a robot.

[0003] The principle of definition language interfaces is also well-established. Such definition language interfaces are known as "PDDL interfaces," for "Planning Domain Definition Language." As is known, in the field of artificial intelligence, they are used to format the input data of a task scheduler. In the context of this invention, a modified PDDL interface will be referred to as an interpretation and monitoring module.

[0004] The present invention relates to the control of parallel missions assigned to a robot, equipped with skills and material resources, within an evolving surrounding context, also referred to as the "state of the world." In other words, the invention lies at the intersection of an interpretation and supervision module, the robot's skills, and the implementation of a foundational model, for the purpose of accomplishing a set of missions assigned to the robot.

[0005] A "competency" is defined as a robot's ability to perform a task. In other words, a competency is essentially a partial task, such as "turn 45 degrees," "turn towards the target," "animate the arm to simulate an emotion," "give a thumbs-up," "move forward one meter," and so on. Competencies are configurable, for example, in terms of "degrees," "distance in meters," etc. Some competencies are considered "high-level." High-level competencies are those coupled with constraints. Competencies thus vary in complexity, particularly in that they require the control of more or fewer of the robot's hardware sub-assemblies: for example, two base motors for turning, or more actuators and components to animate the arms, display a specific dynamic projected face, and so forth.

[0006] The robot's skills are designed to be combined to perform tasks and accomplish missions.

[0007] The determination of how the robot's skills should be combined to perform tasks and accomplish missions is carried out by the interpretation and supervision module, notably through the implementation of a foundation model.

[0008] The general problem is to propose a deterministic operation of an interpretation and supervision module in a context where the missions assigned to the robot are determined by a foundation model, in particular a grand language model or a visual language model, whose operation is probabilistic.

[0009] In the state of the art, a PDDL interface defines, for a robot, an objective, a main mission. The robot then carries out successive missions, without any real-time questioning of the relevance of the way in which the different missions are ordered, including when the state of the world changes.

[0010] With this invention, we hope to manage several parallel missions.

[0011] In other words, the technical problem can be expressed as follows: how to use the robot's skills while taking into account the constraints of reality, which evolve over time, and the limited material resources of the robot, particularly in the context of managing parallel missions.

[0012] The main parameters are the robot's available hardware resources, the constraints of the surrounding real world, and mission prioritization. PRESENTATION OF THE INVENTION

[0013] More specifically, the invention relates to a system for controlling the skills of a robot for the performance of a set of missions, the set of missions being assigned to the robot, the robot having a set of skills enabling the performance of tasks, a set of perception means and a set of available material resources, the system comprising an interpretation and supervision module coupled to a task planner, the interpretation and supervision module being configured to ensure the management of the missions of the set of missions, in parallel, and to determine, based on information, including textual information, received from the set of skills, the set of perception means and the set of available material resources, an ordered list of missions, each mission having a priority level, the task planner defining a list of tasks to be performed by the robot,Each task in the task list is associated with information, including text, relating to the mission to which that task is dedicated; several tasks in the task list can, if necessary, be implemented in parallel by the robot.

[0014] The robot can notably be a humanoid robot, specifically configured to move on a single spherical wheel.

[0015] The interpretation and supervision module notably implements a foundation model.

[0016] In addition, the foundation model is adapted to process together the information received from the perception set, the skills set, and the ordered list of missions, as well as a world state set, a set of the task(s) in progress associated with the mission(s) to which it / they is / are devoted, and a set of the task(s) to be executed; each of said received information being described by information, including textual information, and processed by the foundation model of the interpretation and supervision module which updates the task planner in real time.

[0017] According to one embodiment, the foundation model is adapted to process together, in addition, a history of past information received from the perception set, the skills set, and / or the ordered list of missions, and / or the state of the world set, and / or the set of the task(s) in progress associated with the mission(s) to which it is / are devoted, and / or the set of the task(s) to be executed; said past information being each described by information, including textual information, and processed by the foundation model of the interpretation and supervision module which updates the task planner in real time.

[0018] According to one embodiment, the task(s) being executed leads to the activation of robot actuators.

[0019] Advantageously, the robot skill control system according to the invention may include means for inhibiting the activation of robot actuators.

[0020] According to one embodiment, the robot's skill set is configured to be enriched by one or more learned skills.

[0021] According to one embodiment, at least one learned skill is acquired by the robot through behavioral programming via electronic tags.

[0022] According to one embodiment, at least part of the tasks in the task list is assigned by instructions given in natural language interpreted by the interpretation and supervision module, by means of a foundation model, in particular a large language model or a visual language model.

[0023] According to one embodiment, at least some of the missions in the mission list are assigned to the robot through behavioral programming via electronic tags.

[0024] The present invention also relates to a robot comprising a system for controlling the skills of a robot as briefly described above.

[0025] In particular, such a robot can be configured to move on a single spherical wheel. PRESENTATION OF THE FIGURES

[0026] The invention will be better understood upon reading the following description, given solely by way of example, and referring to the attached drawings, given by way of non-limiting example, and on which:

[0027] laest a schematic representation showing the operation of the system according to the invention;

[0028] laest a schematic representation showing the operation of the system according to the invention, taking into account the history.

[0029] It should be noted that the figures set out the invention in detail to enable implementation of the invention; although not limiting, said figures serve in particular to better define the invention where appropriate. DETAILED DESCRIPTION OF THE INVENTION

[0030] The invention relates to a system for determining and controlling a mission or a set of parallel missions assigned to a robot.

[0031] The system includes, in particular, an interpretation and supervision module. The interpretation and supervision module is coupled with a set of robot skills, a perception set, a set of surrounding context, also referred to as the world state set, and a task planner.

[0032] The interpretation and supervision module determines, from the set of parallel missions assigned to the robot, an ordered list of missions to be accomplished by the robot, each mission being associated with a level of priority, for example "low priority" or "medium priority" or "high priority".

[0033] It is noted in parallel that, in the field of robots, particularly humanoid robots, in order to overcome the structural constraints of the robot and in particular to avoid a "specialization" of robots, the applicant has developed, as described in the patent application whose registration number is FR2212216, a robotic network architecture comprising a set of electronic labels, designated "tags" or "runes", and at least one robot, in particular including at least one predefined gripper.

[0034] In such a robotic network architecture, runes are configured to be linked to a robotic network element and contain information identifying the element, as well as information enabling the element's location and / or other information allowing the robot to determine its behavior toward the element. Thanks to the information contained in the rune, the robot can determine, in particular, whether it has the ability, capacity, and / or right to grasp, manipulate, and / or move the element, especially using the predefined gripper. The robot is configured to interact with any type of element containing at least one rune, and the robot includes a module for interpreting this identification information, as well as any other information that may be contained in the rune.

[0035] Within this general context, generative artificial intelligence algorithms based on foundation models (LxM), also known as deep learning models trained on very large datasets, are implemented. Specifically, such foundation models can be large language models (LLMs). Other foundation models can be implemented in this context, such as visual large models (VLMs), which are visual language models that simultaneously interpret images from the robot's cameras.

[0036] These foundational models are developed to allow the robot to interact with its environment, particularly its "social" environment. The robot is thus equipped with perception capabilities, including listening capabilities via microphones, and a module for interpreting instructions given to it in "natural language".

[0037] Instructions can also be given to the robot via runes, as described above. In this case, a behavioral programming process for the robot is implemented, using physical or virtual runes, within the robotic network architecture. The robotic network architecture includes the robot and at least one element: the runes, which belong to one of the identification and action types. The identification type includes subtypes such as location, object, and person. The behavioral programming process then comprises the following steps:

[0038] the association of at least one identification-type rune with at least one element of the robotic network;

[0039] the coupling of at least one identification type rune with at least one action type rune;

[0040] the activation of at least one rune to generate the execution of a task by the robot with respect to the element associated with said identification type rune coupled with said action type rune.

[0041] The coupling step corresponds to the definition or modification of a rule configured so that the robot performs an interpretation of the activation of at least one rune to perform a predefined task with respect to the element associated with said rune of identification type coupled with said rune of action type, said interpretation being carried out by means of the implementation of the interpretation module, in particular based on a foundation model, for example a large language model or a visual language model, the task to be carried out by the robot corresponding to an output of the interpretation module, the task planner having the function of ordering the succession of tasks to be carried out by the robot.

[0042] Within the framework of the present invention, the system includes an interpretation and supervision module, with expanded capabilities compared to the interpretation module described above.

[0043] Thus, the foundational models implemented in the context of the present invention allow the interpretation and supervision module to determine an ordered list of tasks to be performed by the robot. These tasks are associated with priority levels and are translated, using the task planner, into a list of tasks to be carried out by the robot in the real world. These tasks involve, for example, the activation of the robot's actuators.

[0044] The robot's abilities may include: moving to a place, object or person equipped with a rune, moving to a location, grasping an object equipped with a rune, smiling, speaking.

[0045] Preferably, the interpretation and supervision module is also coupled with a set of tasks that are currently running or whose execution is planned and in the future.

[0046] The perception system may include: one or more optical sensors, including cameras, one or more touch sensors, one or more Bluetooth® antennas, etc.

[0047] The world state set, in other words the representation of the world as taken into consideration by the interpretation and supervision module, can include textual information about the robot's location or its state: it is holding something, etc. This corresponds to the state of the world as it can be perceived at time t.

[0048] Each set can be enriched and / or evolve over time, including in parallel with the execution of tasks by the robot.

[0049] Each set is associated with information, including textual information, to serve as input to the foundation model.

[0050] In particular, each mission is described by textual information and presents a level of priority.

[0051] In particular, each task is described by textual information and has a priority level. Each task is dedicated to a mission.

[0052] Tasks can trigger the activation of the robot's actuators. These activations can be inhibited by low-level runes.

[0053] Indeed, in this context, to set real-world limits on what the robot can do, low-level runes interact with the robot "without the robot's knowledge," that is, by "bypassing" the interpretation module implementing the generative artificial intelligence algorithms. For example, a verification module processes all output from ongoing tasks to account for the existence of instructions from low-level runes that impose "physical" limits on the robot. Low-level runes are therefore runes with an active "low-level flag," also known as a "low-level field."

[0054] The principle is that the implementation of these low-level runes is deterministic: the robot is configured to implement the instructions contained in the low-level runes in an unavoidable way. Therefore, we know with certainty that the rules and prohibitions imposed by these low-level runes, via the instructions they contain, will be implemented by the robot.

[0055] The instructions contained in low-level runes are interpreted by the robot independently of the interpretation module.

[0056] The instructions contained in the low-level runes are configured to allow the robot to override functions. If the robot receives instructions and physical limits from the low-level runes, including after instructions from the interpretation module, these limits are implemented directly and without delay by the robot.

[0057] Low-level runes can thus be used to inhibit the activation of actuators necessary for the implementation of a task assigned to the robot.

[0058] Furthermore, the robot can acquire new skills through behavioral programming, notably via runes.

[0059] Missions are determined following instructions given in natural language or through behavioral programming, notably via runes.

[0060] The missions are processed in parallel under the control of the interpretation and supervision module.

[0061] The interpretation and supervision module takes into account a set of available hardware resources of the robot, in particular to update the ordered list of missions or to directly modify the list of tasks defined by the task planner.

[0062] The interpretation and supervision module acts in a way as a "primordial brain" of the robot.

[0063] In particular, the foundation model of the interpretation and supervision module is adapted to process together the information received from the perception set, the skills set, and the ordered list of missions, as well as a set of world state, a set of the task(s) in progress associated with the mission(s) to which it / they is / are devoted, and a set of the task(s) to be executed; each of said received information being described by information, including textual information, and processed by the foundation model of the interpretation and supervision module, which updates the task planner in real time.

[0064] In the example, the robot's skill set includes: "go to a rune"; "go to a location (paired with a rune)"; "grab (an object equipped with) a rune"; "smile"; "say"; etc.

[0065] The state of the world includes, among other things, the following information: "the robot is at the entrance"; "the robot is carrying flowers"; etc.

[0066] The robot's perception system allows the robot to have the following information: one or more vision sensors (cameras); one or more touch sensors; one or more rune detectors; one or more ASR sensors; etc.

[0067] Parallel missions are assigned to the robot, in particular the following missions: "guide customers", with a high degree of priority; "provide information to Jerome", with a medium degree of priority; "take flowers at the entrance", with a low degree of priority; "express a joyful mood", with a low degree of priority; etc.

[0068] Therefore, the LxM foundation model is implemented.

[0069] The ordered list of tasks is processed by the task planner to generate robot tasks to be performed sequentially or in parallel, these tasks being associated with the missions in the ordered list of missions assigned to the robot, including:

[0070] "Tell the customers: 'follow me'", associated with the mission "guide the customers"; then "go to a place (where Jerome is)", associated with the mission "provide information to Jerome"; then "tell Jerome", associated with the mission "provide information to Jerome"; then "go to a rune (paired with the flowers)", associated with the mission "take the flowers at the entrance"; "grab the flowers (paired with a rune) with your left hand"; and, in parallel:

[0071] “smile”, associated with the mission “express a joyful mood”.

[0072] Some tasks require the activation of actuators. Where appropriate, the verification module may include means to inhibit these tasks by preventing the activation of actuators.

[0073] These robot tasks may require the activation of robot actuators.

[0074] The interpretation and supervision module forms a kind of "primordial brain" of the robot. Specifically, the interpretation and supervision module is linked to the perception set, the skills set, and the ordered list of missions. The interpretation and supervision module is also linked to the world state set, the set of tasks currently being executed associated with the mission(s) to which they are dedicated, and the set of tasks to be executed. Each of these received pieces of information is described by a specific piece of information, often textual, and processed by the LxM foundation model of the interpretation and supervision module, which updates the task scheduler in real time.Preferably, the history of this information is also described by information, including textual information, and processed by the LxM foundation model of the interpretation and supervision module which updates the task scheduler in real time.

[0075] The interpretation and monitoring module is implemented primarily using hardware resources (computer, memory, etc.) embedded in the robot. Similarly, the history of information processed by the interpretation and monitoring module can be stored in the robot's memory. Alternatively, the interpretation and monitoring module can be implemented using remote hardware resources (computer, memory, etc.) installed on a remote server. Likewise, the history of information processed by the interpretation and monitoring module can be stored in the memory of a remote server.

[0076] It should be noted that the invention is not limited to the embodiments described above. Indeed, it will become apparent to a person skilled in the art that various modifications can be made to the embodiments described above, in light of the instruction that has just been disclosed to them.

[0077] In the detailed presentation of the invention given above, the terms used shall not be interpreted as limiting the invention to the embodiment set forth in this description, but shall be interpreted to include all equivalents which can be foreseen by a person skilled in the art by applying their general knowledge to the implementation of the teaching which has just been disclosed to them.

Claims

A robot skills control system for performing a set of missions, the set of missions being assigned to the robot, the robot having a set of skills enabling the performance of tasks, a set of perception means and a set of available material resources, the system comprising an interpretation and supervision module coupled to a task planner, the interpretation and supervision module being configured to manage the missions of the set of missions in parallel, and to determine, based on information, including textual information, received from the set of skills, the set of perception means and the set of available material resources, an ordered list of missions, each mission having a priority level, the task planner defining a list of tasks to be performed by the robot,Each task in the task list is associated with information, including text, relating to the mission to which that task is dedicated. Several tasks in the task list can, if necessary, be implemented in parallel by the robot. The interpretation and supervision module implements a foundation model (LxM) adapted to process together the information received from the perception set, the skills set, and the ordered list of missions, as well as a set of world states, a set of the task(s) currently being executed associated with the mission(s) to which it / they are dedicated, and a set of the task(s) to be executed. Each of the information received is described by information, including text, and processed by the foundation model (LxM) of the interpretation and supervision module, which updates the task planner in real time. A robot skills control system according to claim 1, wherein the foundation model (LxM) is adapted to process together, in addition, a history of past information received from the perception set, the skills set, and / or the ordered list of missions, and / or the world state set, and / or the set of the task(s) in progress associated with the mission(s) to which it is / are devoted, and / or the set of the task(s) to be executed; said past information each being described by information, including textual information, and processed by the foundation model (LxM) of the interpretation and supervision module which updates the task planner in real time. A robot skills control system according to any one of the preceding claims, wherein the task(s) being performed results in the activation of robot actuators. A robot skill control system according to claim 3, comprising means for inhibiting the activation of robot actuators. A robot skill control system according to any one of the preceding claims, wherein the robot's skill set is configured to be enriched by one or more learned skills. A robot skill control system according to claim 5, at least one learned skill being acquired by the robot through behavioral programming via electronic tags. A robot skills control system according to any one of the preceding claims, wherein at least a part of the missions in the mission list is assigned by instructions given in natural language interpreted by the interpretation and supervision module, by means of a foundation model (LxM), in particular a large language model or a visual language model. A robot skills control system according to any one of the preceding claims, wherein at least some of the missions in the mission list are assigned to the robot through behavioral programming via electronic tags. Robot, comprising a system for controlling the skills of a robot according to one of the preceding claims. Robot according to the preceding claim, configured to move on a single spherical wheel.