Visual Sensor Data Splitting for Autonomous Driving
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for annotating visual sensor data for autonomous driving systems are inefficient due to varying work package durations, differing labeler interpretations, and the trade-off between labeling speed and quality, necessitating improved methods for splitting batches of images.
Innovation Solution
A method for automatically splitting visual sensor data by assigning scene numbers to images based on optical flow, GPS coordinates, or IMU acceleration data, and determining accumulated effort using neural networks for object detection, with packages created unless the effort surpasses a threshold, allowing for consistent and efficient labeling.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is split into fixed sized chunks of frames, then throughput is improved, but labeling quality becomes uneven across different work packages
Solution Approach 1:
The patent segments the video data into work packages based on scene changes detected through optical flow analysis, rather than fixed frame counts. This segmentation ensures that each work package contains a complete scene context, leading to more uniform labeling quality across all packages while maintaining high throughput by enabling parallel processing of independent scene-based packages.
Solution Approach 2:
The system performs preliminary analysis of optical flow and scene structure before dividing the data into work packages. By pre-identifying scene boundaries and complexity characteristics, the system can create balanced work packages with similar labeling effort requirements, ensuring quality uniformity while optimizing the number of packages for high throughput.
2Productivity
If more labelers are employed to speed up labeling, then throughput increases, but annotation quality decreases due to more interpretations
Solution Approach 1:
By segmenting data into scene-based work packages with clear boundaries, the patent ensures that each labeler works on a self-contained scene context. This reduces interpretation variability within each package and enables consistent quality across multiple labelers, as each package is complete and unambiguous.
Solution Approach 2:
The system assigns work packages to labelers based on their expertise matching the scene type, and ensures each package has consistent internal quality characteristics. This local quality approach allows different labelers to maintain high consistent quality on their respective packages without requiring uniform interpretation across all labelers.
3Productivity
If work packages contain diverse scenes, then throughput is improved, but quality checks become more time-intensive
Solution Approach 1:
The patent segments work packages by scene boundaries, ensuring that each package contains a homogeneous scene type. This segmentation allows quality checks to be performed more efficiently, as checkers can develop scene-specific quality criteria and apply them consistently within each package type, reducing the time required for diverse scene verification.
Solution Approach 2:
The system changes the organizational parameter from fixed frame counts to scene-based segmentation, which fundamentally alters how quality checks are performed. Instead of checking arbitrary frame boundaries, quality checks can focus on scene coherence and completeness, reducing time expenditure while maintaining high standards across diverse content.
4Adaptability or versatility
If work packages have different turnover durations, then flexibility is improved, but process complexity increases
Solution Approach 1:
By segmenting work packages based on scene complexity and expected labeling effort rather than fixed time durations, the system achieves flexibility in accommodating different scene types while maintaining manageable package sizes. The scene-based segmentation naturally creates packages with comparable processing characteristics, reducing management complexity.
Data Source
AI summary
A method for automatically splitting visual sensor data comprising consecutive images, the method being executed by at least one processor of a host computer, the method comprising: a) assigning a scene number to each image, wherein a scene comprises a plurality of images taken in a single environment, wherein assigning a scene number to each image is performed based on a comparison between consecutive images; b) determining an accumulated effort for the images in each scene, wherein the accumulated effort is determined based on the number of objects in the images of the scene, wherein the number of objects is determined using one or more neural networks for object detection; and c) creating packages of images, wherein the images with the same scene number are assigned to the same package unless the accumulated effort of the images in the package surpasses a package threshold.