Natural-Language Robot Control for Autonomous Task Planning

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current robot control systems lack the ability to autonomously understand and execute complex tasks in dynamic environments without extensive human intervention, as they rely on predefined instructions and lack the capability to adapt to new objectives or environments.

Innovation Solution

The integration of a large language model (LLM) into robot systems that captures sensor data, generates natural language descriptions of the environment, and uses these descriptions to query the LLM for task objectives and plans, allowing the robot to autonomously execute tasks by interpreting and executing the received plans.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If predefined instructions are used for robot control, then the control system is simple and reliable, but the robot cannot adapt to new objectives or environments

Engineering Contradiction:

Improveadaptability to new objectivesVSAvoidcontrol system complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

A language model is introduced as an intermediary component between the robot's sensor inputs and its action outputs. The language model processes natural language descriptions of environments and objectives, translating them into actionable task plans without requiring complex reprogramming. This mediator enables adaptability while keeping the underlying control system architecture relatively simple.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent replaces traditional mechanical control systems (based on predefined instructions and hard-coded logic) with a cognitive system based on language processing. Instead of using complex mechanical control architectures to handle new objectives, the system uses natural language understanding and generation capabilities to adapt to new tasks, substituting mechanical complexity with information processing.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Productivity

If extensive human intervention is provided for task execution, then task accuracy is high, but productivity decreases

Engineering Contradiction:

Improveautonomous task executionVSAvoidtask execution accuracy

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The robot system is designed to serve itself by autonomously generating task plans from natural language objectives. The language model enables the robot to understand its own objectives and generate its own execution plans without continuous human guidance. This self-service capability increases productivity while maintaining reliability through the structured task planning process.

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The system incorporates feedback loops where the robot monitors its environment through sensors, compares current state with planned actions, and adjusts its task execution accordingly. This feedback mechanism ensures that autonomous operation maintains high accuracy by continuously verifying that actions align with objectives and environmental conditions.

Inventive Principle:

Principle #23Feedback

3Extent of automation

If complex task planning is implemented autonomously, then human intervention is reduced, but the system requires sophisticated processing capabilities

Engineering Contradiction:

Improveautonomous task planningVSAvoidprocessing capability requirements

Core Design Contradiction:

Extent of automationVSDevice complexity

Solution Approach 1:

The patent replaces sophisticated autonomous processing requirements with a language-based approach. Instead of implementing complex autonomous reasoning and planning algorithms, the system uses natural language processing capabilities to achieve the same effect. The language model handles the complexity of task planning by translating objectives into structured plans, reducing the need for sophisticated autonomous processing while maintaining high automation.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentUS12162153B2Robot systems, methods, control modules, and computer program products that leverage large language models

Publication Date: 2024.12.10 SANCTUARY COGNITIVE SYST CORP

AI summary

Robot control systems, methods, control modules and computer program products that leverage one or more large language model(s) (LLMs) in order to achieve at least some degree of autonomy are described. Robot control parameters, environment details, and/or instructions may advantageously be specified in natural language (NL) and communicated with the LLM via an NL prompt or query. The NL query may include a request for one or more work objectives from the LLM, such as “What can I do here?”, thereby establishing a form of agency by which the robot system may identify activities to perform without operator intervention. The LLM may also be queried to convert each work objective into a task plan providing a sequence of steps that the robot system may execute to complete the work objective. Optionally, the robot system may communicate with an operator to determine whether or not to execute a task plan.