Intermediate ML Input for Efficient Object Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current techniques for processing sensor data to determine object classifications in autonomous vehicles face challenges in resource-constrained environments, particularly due to the substantial input data requirements and processing restraints.
Innovation Solution
A machine-learned model is employed to process multi-channel image data by separating it into spatial and non-spatial features, using a backbone neural network for spatial features and a dense neural network for non-spatial features, allowing parallel processing of different ranges with varying resolutions to improve efficiency and reduce resource demands.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If substantial input data is processed to determine object classifications, then classification accuracy is improved, but resource consumption and processing time increase
Solution Approach 1:
The patent segments the processing pipeline into two distinct portions: a first portion that processes multi-channel image data to extract spatial features, and a second portion that processes the extracted features to determine object classifications. This segmentation allows each portion to be optimized independently, with the first portion handling raw data efficiently and the second portion focusing on classification accuracy, thereby resolving the contradiction between processing efficiency and classification accuracy.
Solution Approach 2:
The patent extracts intermediate features from the first processing portion and uses them as input for the second processing portion. By taking out and reusing these intermediate results, the system avoids redundant processing while maintaining high classification accuracy, thus improving overall processing efficiency without sacrificing measurement precision.
2Measurement precision
If substantial input data is processed to determine object classifications, then classification accuracy is improved, but resource constraints are exceeded
Solution Approach 1:
The processing system is divided into two portions that operate sequentially, with the first portion processing image data to extract features and the second portion using those features for classification. This segmentation reduces the memory footprint at any given time compared to processing all data simultaneously, making the system feasible for resource-constrained autonomous vehicle environments while maintaining high classification accuracy.
3Loss of time
If processing time is reduced for quick detection, then safety outcomes are improved, but processing accuracy may deteriorate
Solution Approach 1:
The first processing portion performs preliminary feature extraction from multi-channel image data before the second portion conducts detailed classification. By preparing and organizing intermediate features in advance, the system enables the second portion to quickly process and classify objects with high accuracy, thus reducing overall detection time without sacrificing measurement precision.
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
Techniques for determining a classification probability of an object in an environment are discussed herein. Techniques may include analyzing sensor data associated with an environment from a perspective, such as a top-down perspective, using multi-channel data. From this perspective, techniques may determine channels of multi-channel input data and additional feature data. Channels corresponding to spatial features may be included in the multi-channel input data and data corresponding to non-spatial features may be included in the additional feature data. The multi-channel input data may be input to a first portion of a machine-learned (ML) model, and the additional feature data may be concatenated with intermediate output data from the first portion of the ML model, and input into a second portion of the ML model for subsequent processing and to determine the classification probabilities. Additionally, techniques may be performed on a multi-resolution voxel space representing the environment.