Scene Graph Task and Motion Planning for Flexible Robot Tasks

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing robotic control systems face challenges in automating complex task and motion planning tasks due to the need for numerous intermediate steps and limitations in flexibility and perception, particularly when dealing with complex objects, which hinders their ability to adapt to new tasks without specifying domain files.

Innovation Solution

A novel method utilizing 3D scene graphs for task and motion planning, integrating complex object perception and enabling long-horizon task planning without requiring specific domain files, by generating geometric and symbolic scene graphs to facilitate both task and motion planning directly on a unified representation.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Extent of automation

If traditional robotic control systems are used for task and motion planning, then automation of difficult and dangerous tasks is achieved, but flexibility and adaptability to new tasks without domain files are limited

Engineering Contradiction:

Improveautomation of task and motion planningVSAvoidflexibility to adapt to new tasks

Core Design Contradiction:

Extent of automationVSAdaptability or versatility

Solution Approach 1:

The patent replaces traditional mechanical control systems with a machine learning model (neural network) that processes scene graphs and generates task plans. This substitution enables the system to learn from demonstrations and adapt to new tasks without requiring explicit programming or domain files, thereby improving both automation extent and adaptability simultaneously

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The system changes the representation parameters from traditional state-space formulations to scene graphs with geometric and symbolic representations. This parameter transformation allows the machine learning model to process environmental information more effectively and generalize to new tasks, resolving the contradiction between automation and adaptability

Inventive Principle:

Principle #35Parameter changes

2Adaptability or versatility

If complex objects are perceived and manipulated, then task capability is improved, but system complexity increases

Engineering Contradiction:

Improvecapability to handle complex objectsVSAvoidsystem complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent segments the complex perception and planning system into distinct components: scene graph generation, geometric graph processing, symbolic graph processing, and task planning. This segmentation allows each component to handle specific aspects of complex object manipulation independently, improving overall capability while managing system complexity through modular architecture

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces scene graphs as an intermediary representation between raw sensor data and task planning. This intermediary structure simplifies the processing of complex objects by providing a unified geometric and symbolic representation, thereby improving task capability without proportionally increasing system complexity

Inventive Principle:

Principle #24Intermediary (Mediator)

3Reliability

If numerous intermediate steps are required for task completion, then task success rate is improved, but execution time increases

Engineering Contradiction:

Improvetask completion success rateVSAvoidexecution time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The patent performs preliminary action by pre-processing sensor data into scene graphs and pre-planning task sequences using the machine learning model. This allows the system to prepare multiple intermediate steps in advance, ensuring task success while reducing actual execution time by having the plan ready before physical execution begins

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system maintains continuity of useful action by continuously updating scene graphs and adjusting task plans based on real-time feedback during execution. This continuous adaptation ensures that all necessary intermediate steps are executed in the correct sequence for task success, while minimizing idle time through seamless transitions between steps

Inventive Principle:

Principle #20Continuity of useful action

Data Source

PatentUS20260091496A1Machine learning model for task and motion planning

Publication Date: 2026.04.02 NVIDIA CORP

AI summary

Apparatuses, systems, and techniques are described that solve task and motion planning problems. In at least one embodiment, a task and motion planning problem is modeled using a geometric scene graph that records positions and orientations of objects within a playfield, and a symbolic scene graph that represents states of objects within context of a task to be solved. In at least one embodiment, task planning is performed using symbolic scene graph, and motion planning is performed using a geometric scene graph.