Transformer Attention for Adaptive Point Cloud Object Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing transformer-based object detection methods are limited to image data and have not been effectively applied to large point clouds for autonomous driving, leading to inefficiencies in resource utilization and accuracy due to the need for anchor boxes in sparsely populated point clouds.
Innovation Solution
A method using a transformer with attention model that employs a backbone to calculate feature vectors from point cloud data, refining anchor positions and transforming result feature vectors to improve object detection accuracy by aligning them with actual objects, thereby reducing computational resources and enhancing detection precision.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If transformer-based object detection is applied to large point clouds for autonomous driving, then detection capability is extended to 3D space, but computational resources are wasted on locating positions that actually have data in sparsely populated point clouds
Solution Approach 1:
The patent applies preliminary action by using a backbone network to pre-process point cloud data and generate feature vectors before feeding them to the transformer. This preliminary processing filters and organizes the data, allowing the transformer to focus only on relevant regions rather than wasting computational resources on empty spaces in sparsely populated point clouds.
Solution Approach 2:
The backbone network serves as an intermediary between the raw point cloud data and the transformer model. It processes the data first, creating a refined representation that reduces the computational burden on the transformer while maintaining detection accuracy in 3D space.
2Ease of manufacture
If anchor boxes are used in sparsely populated point clouds, then object detection framework is established, but detection accuracy is reduced due to mismatch between anchor positions and actual objects
Solution Approach 1:
The patent applies dynamics by making the anchor positions adaptive rather than fixed. The anchor positions are dynamically adjusted based on the actual object locations detected by the transformer, allowing the detection framework to maintain accuracy even in sparsely populated point clouds where fixed anchors would mismatch with actual objects.
Solution Approach 2:
The patent implements feedback by using the transformer's detection results to refine and update the anchor positions. This feedback loop ensures that the anchor boxes are continuously adjusted to match the actual object locations, improving detection accuracy while maintaining the framework's structural integrity.
3Device complexity
If encoder is used to calculate feature vectors from point cloud data, then transformer structure is complete, but input sequence for self-attention is limited and grid-based backbone requires large cell size
Solution Approach 1:
The patent applies the extraction principle by removing the traditional encoder component from the transformer structure and replacing it with a backbone network. This extraction allows for greater flexibility in processing point cloud data, as the backbone can adaptively extract features without being constrained by the encoder's fixed architecture, enabling smaller cell sizes in grid-based backbones.
Data Source
AI summary
The invention relates to a method for detecting multiple objects (O1, O2) from point cloud data using a transformer with attention model, wherein the state of the tracked objects (O1, O2) is stored in a feature space. The following steps are carried out:a. calculating feature vectors from the point cloud data by means of a backbone (2), wherein the feature vectors serve as key vectors (ki) and value vectors (νi) for the transformer;b. calculating first anchor positions (ρi(0)) for a first layer (s0) of the transformer from the point cloud data using a sampling method (4);c. ascertaining feature vectors from the first anchor positions (ρi(0)) using an encoding (5), wherein the feature vectors serve as object queries (γi(0)) for the first layer (s0) of the transformer;d. ascertaining result feature vectors (z1(0)) in the first layer (s0) of the transformer from the object queries (γi(0)) and the key vectors (ki) and value vectors (νi) using the first layer (s0) of a decoder (6) of the transformer;e. calculating (7) box parameters (b|z<sub2>i</sub2><sup2>(0)</sup2>|) for the result feature vectors (zi(0)) of the first layer (s0) of the transformer;f. adapting (40, 140) the anchor positions (ρi(s)) for at least one further layer (s) of the transformer by adding the position differences of the box parameters(b[zi(0)])to the first anchor positions (ρi(0));g. ascertaining feature vectors from the adapted anchor positions (ρi(s)) using an encoding (50, 150), wherein the feature vectors serve as object queries (γi(s)) for the at least one further layer (s) of the transformer;h. transforming (90) the result feature vectors (zi(0)) of the first layer with respect to the adapted anchor positions (ρi(l)), wherein the transformed result feature vectors ({tilde over (z)}i(0)) serve as object queries for the at least one further layer (s) of the transformer;i. ascertaining result feature vectors (zi(s)) in the at least one further layer (s) of the transformer from the transformed result feature vectors ({tilde over (z)}i(0)) of the previous layer (s0), the calculated object queries γi(s) of the current layer (s), and the key vectors (ki) and value vectors (νi) using the decoder (60) of the transformer.