Autonomous Machine Control Using Blackboard Context Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing control systems for autonomous machinery struggle to effectively process rich context sensor inputs, such as real-time audio/video, to generate control commands in dynamic environments.
Innovation Solution
A computer-implemented method that acquires and processes sensor data from multiple sensors calibrated to a common reference frame, using perception modules to extract information, a context awareness module to fuse this information into a blackboard image, and an action evaluation module to determine control actions based on a learned mapping of context to action.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If rich context sensor inputs (audio/video) are used as inputs, then the control system capability is improved, but the processing complexity and computational requirements increase
Solution Approach 1:
The patent introduces an intermediary representation called 'blackboard image' that mediates between rich sensor inputs and control decisions. Multiple perception modules process sensor data independently and write to a shared blackboard data structure, which is then processed by a single control policy. This intermediary structure reduces processing complexity by organizing information flow and enabling modular processing while maintaining the ability to handle rich context inputs.
Solution Approach 2:
The control system is segmented into multiple independent perception modules (e.g., audio perception, video perception, depth perception) that each process specific types of sensor data independently. These modules write their findings to a shared blackboard, allowing parallel processing of different sensor modalities without requiring a single complex processing pipeline, thus reducing overall processing complexity.
2Loss of information
If multiple perception modules process sensor data in real time, then the context awareness is improved, but the computational load increases
Solution Approach 1:
The patent merges the outputs of multiple independent perception modules into a single blackboard data structure. Instead of having each module independently process and generate control actions (which would multiply computational load), their perceptions are combined into a unified context representation that a single control policy then processes, reducing redundant computation while maintaining comprehensive context awareness.
3Measurement precision
If a learned mapping of context to control action is used, then the decision-making accuracy is improved, but the training data requirements and system complexity increase
Solution Approach 1:
The patent uses a learned mapping (control policy) that has been trained on training data to copy successful decision-making patterns from training scenarios to real-time operation. The control policy learns from training data how to map blackboard images to control actions, achieving high decision-making accuracy by replicating effective behaviors without requiring complex real-time reasoning during operation.
Data Source
AI summary
A computer-implemented method for controlling an autonomous machine includes processing sensor data streamed via a plurality of calibrated sensors by a plurality of perception modules to extract perception information from the sensor data in real time. The extracted real time perception information from the plurality of perception modules is fused by a context awareness module to create a blackboard image, which is a representation of an operating environment of the autonomous machine derived from fusion of the extracted perception information using a controlled semantic, defining a context of the autonomous machine. A stream of blackboard images, representing a time evolving context of the autonomous machine, is processed by an action evaluation module, using a control policy, to output a control action to be executed by the autonomous machine. The control policy includes a learned mapping of context to control action represented by blackboard images created using the controlled semantic.