Multi-Sensor RNN Classification for Temporal Object Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural network systems for autonomous vehicles face challenges in accurately classifying objects from sensor data without human intervention, as they require manual determination of feature representation combinations from different sensing channels, which can lead to suboptimal classification results.
Innovation Solution
A neural network system comprising multiple encoder recurrent neural networks (RNNs) and a classification neural network subsystem that processes sensor data from various sensing subsystems, such as LIDAR, RADAR, and cameras, to generate feature representations and target classifications, automatically learning optimal parameter combinations for accurate object classification.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual determination of feature representation combinations is used, then human intervention is required, but classification accuracy deteriorates due to suboptimal combinations
Solution Approach 1:
The neural network system automatically determines optimal feature representation combinations without human intervention. The system self-adjusts by processing sensor data through multiple encoder RNNs that automatically learn and combine feature representations from different sensing channels (LIDAR, RADAR, cameras) to achieve accurate object classification
Solution Approach 2:
The patent replaces manual human determination (mechanical/intellectual process) with an automated neural network system. The encoder RNNs and classification subsystem automatically perform feature extraction, combination, and classification tasks that previously required human expert intervention, substituting human cognitive processes with computational algorithms
2Reliability
If multiple sensing subsystems are used, then more data is available, but system complexity increases due to combining feature representations
Solution Approach 1:
The system segments the complex task of multi-sensor data fusion into distinct modular components: separate encoder RNNs for each sensing subsystem (LIDAR, RADAR, cameras), and a dedicated classification subsystem. Each encoder RNN independently processes features from its corresponding sensor, dividing the overall complexity into manageable, specialized modules that can be developed and maintained independently
Solution Approach 2:
The encoder RNN architecture serves as a universal processing component that can handle multiple types of sensor data. The same encoder RNN structure is applied across different sensing subsystems, providing a unified approach to feature extraction and combination that reduces overall system complexity while maintaining the ability to process diverse sensor inputs
3Loss of information
If feature representations from different time steps are combined, then temporal context is improved, but processing complexity increases
Solution Approach 1:
The recurrent neural network architecture enables continuous processing of sensor data across time steps. The encoder RNNs maintain hidden states that carry temporal context forward, allowing the system to continuously process and integrate information from multiple time steps without breaking the temporal continuity of the data stream
Solution Approach 2:
The system dynamically adjusts feature representation combinations based on temporal context. The RNNs automatically learn which temporal patterns are important and adaptively combine features from different time steps, making the processing complexity dynamic rather than static, allowing the system to focus computational resources on relevant temporal relationships
Data Source
AI summary
Disclosed herein are neural networks for generating target classifications for an object from a set of input sequences. Each input sequence includes a respective input at each of multiple time steps, and each input sequence corresponds to a different sensing subsystem of multiple sensing subsystems. For each time step in the multiple time steps and for each input sequence in the set of input sequences, a respective feature representation is generated for the input sequence by processing the respective input from the input sequence at the time step using a respective encoder recurrent neural network (RNN) subsystem for the sensing subsystem that corresponds to the input sequence. For each time step in at least a subset of the multiple time steps, the respective feature representations are processed using a classification neural network subsystem to select a respective target classification for the object at the time step.