Sensor Data Segmentation Using Image-Derived Training Labels

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Manual segmentation of sensor data from non-image capture devices is time-consuming and costly, making it unsuitable for many applications, especially in autonomous vehicle operations where efficient data processing is crucial.

Innovation Solution

A system that captures image data and sensor data concurrently, uses image data to aid in segmenting sensor data by projecting sensor data onto segmented images to create a training dataset, which is then used to train a sensor data segmentation model for automatic segmentation, enabling the use of sensor data in applications like autonomous vehicle navigation.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If manual segmentation is used for sensor data, then segmentation accuracy can be achieved, but time consumption and cost increase significantly

Engineering Contradiction:

Improvesegmentation accuracyVSAvoidsegmentation time

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The system performs preliminary action by using image segmentation results to create training data that guides subsequent sensor data segmentation. The image data is segmented first using established computer vision techniques, and these segmentation results are then used to train a model that automatically segments sensor data, eliminating the need for manual segmentation while maintaining accuracy.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

Image data serves as an intermediary between manual segmentation and sensor data segmentation. The system segments image data (which is easier to segment automatically), uses these results to create training datasets, and then applies machine learning models to segment sensor data. This intermediary approach enables automatic segmentation of difficult-to-segment sensor data by leveraging the easier-to-segment image data.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Measurement precision

If manual segmentation is used for sensor data, then accurate object identification can be achieved, but cost increases prohibitively

Engineering Contradiction:

Improveobject identification accuracyVSAvoidsegmentation cost

Core Design Contradiction:

Measurement precisionVSEase of manufacture

Solution Approach 1:

The system creates a copy of the segmentation process by using image segmentation results as training data for sensor data segmentation. Instead of manually segmenting sensor data directly, the system copies the segmentation approach from image data to sensor data through machine learning, significantly reducing cost while maintaining accuracy.

Inventive Principle:

Principle #26Copying

Solution Approach 2:

Image data acts as an intermediary that bridges the gap between manual segmentation and automatic sensor data segmentation. By segmenting image data first and using these results to train automatic segmentation models, the system achieves accurate object identification in sensor data without the prohibitive costs of manual segmentation.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Productivity

If automatic segmentation is applied to image data, then efficient processing can be achieved, but this approach does not directly apply to other sensor types

Engineering Contradiction:

Improvedata processing efficiencyVSAvoidsensor type applicability

Core Design Contradiction:

ProductivityVSAdaptability or versatility

Solution Approach 1:

The system achieves universality by creating a multi-functional segmentation framework. Image segmentation techniques are used not only for image data but also as a foundation for segmenting other sensor types through training data generation. The machine learning model trained on image-derived training data can then process multiple sensor types including LIDAR, radar, and camera data, making the system universally applicable across different sensor modalities.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

Image data serves as an intermediary that enables the transfer of segmentation capabilities from images to other sensor types. The system segments image data using established automatic techniques, uses these results to create training datasets, and then applies machine learning to extend this segmentation capability to other sensor types that previously could not be automatically segmented.

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS11475573B2Sensor data segmentation

Publication Date: 2022.10.18 ZOOX INC

AI summary

A system may include one or more processors configured to receive a plurality of images representing an environment. The images may include image data generated by an image capture device. The processors may also be configured to transmit the image data to an image segmentation network configured to segment the images. The processors may also be configured to receive sensor data associated with the environment including sensor data generated by a sensor of a type different than an image capture device. The processors may be configured to associate the sensor data with segmented images to create a training dataset. The processors may be configured to transmit the training dataset to a machine learning network configured to run a sensor data segmentation model, and train the sensor data segmentation model using the training dataset, such that the sensor data segmentation model is configured to segment sensor data.