Contrastive Learning for Robust Instance Segmentation Training

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Training instance segmentation neural networks using conventional methods is challenging due to the low reliability of sensor images, especially under varying environmental conditions, and requires extensive manually labeled data, which is time-consuming and expensive.

Innovation Solution

Employing contrastive learning to generate embeddings for sensor images, using positive and negative pairs of embeddings associated with the same or different object instances, and incorporating optical flow to leverage different sensors and improve training efficiency without full labeling.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If conventional training methods are used for instance segmentation neural networks, then training can be performed with standard datasets, but the reliability of the network under varying environmental conditions deteriorates

Engineering Contradiction:

Improvereliability of instance segmentation under varying environmental conditionsVSAvoidadaptability to different environmental conditions

Core Design Contradiction:

ReliabilityVSAdaptability or versatility

Solution Approach 1:

The patent applies dynamic contrastive learning that adapts to varying environmental conditions by dynamically adjusting the contrastive loss function based on environmental context. The system dynamically selects and weights different training strategies (supervised, unsupervised, semi-supervised) according to environmental conditions, enabling the network to maintain high reliability across diverse conditions without requiring exhaustive manual labeling for each scenario.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The patent changes the training parameters by introducing contrastive learning objectives that modify the loss function to include both classification accuracy and instance discrimination terms. This parameter change enables the network to learn more robust features that generalize better across environmental conditions, improving reliability without requiring extensive retraining for each environment.

Inventive Principle:

Principle #35Parameter changes

2Measurement precision

If extensive manually labeled data is used for training, then training accuracy improves, but training time and cost increase

Engineering Contradiction:

Improvetraining accuracy of instance segmentationVSAvoidtime required for manual labeling

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The patent implements self-service training through unsupervised contrastive learning components that enable the network to learn instance discrimination from unlabeled data automatically. The system uses contrastive losses that leverage temporal consistency and spatial relationships to self-supervise the learning process, achieving high training accuracy without requiring extensive manual labeling for every training sample.

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The patent applies partial labeling strategies where only a subset of training data requires manual labels, while the majority of data is processed through contrastive learning with partial supervision. This partial action approach maintains high training accuracy by combining supervised signals from labeled data with unsupervised contrastive learning from unlabeled data, significantly reducing the time and cost of manual labeling.

Inventive Principle:

Principle #16Partial or excessive action

3Ease of manufacture

If sensor images with low reliability are used for training, then data collection is easier, but the quality of instance segmentation output deteriorates

Engineering Contradiction:

Improveease of data collectionVSAvoidquality of instance segmentation output

Core Design Contradiction:

Ease of manufactureVSManufacturing precision

Solution Approach 1:

The patent introduces contrastive learning objectives as an intermediary mechanism that bridges the gap between low-reliability sensor images and high-quality instance segmentation output. The contrastive loss function acts as a mediator that extracts reliable instance discrimination signals even from noisy or low-quality sensor data, enabling accurate segmentation without requiring high-quality labeled training data.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent replaces traditional supervised learning mechanisms that rely on high-quality labeled data with contrastive learning mechanisms that can extract useful signals from lower quality data. By substituting the training mechanism to use contrastive objectives based on temporal and spatial consistency, the system achieves high segmentation quality while maintaining ease of data collection from various sensor sources.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentUS12437527B2Training instance segmentation neural networks through contrastive learning

Publication Date: 2025.10.07 WAYMO LLC

AI summary

Methods, systems, and apparatus for processing inputs that include video frames using neural networks. In one aspect, a system comprises one or more computers configured to obtain a set of one or more training images and, for each training image, ground truth instance data that identifies, for each of one or more object instances, a corresponding region of the training image that depicts the object instance. For each training image in the set, the one or more computers process the training image using an instance segmentation neural network to generate an embedding output comprising a respective embedding for each of a plurality of output pixels. The one or more computers then train the instance segmentation neural network to minimize a loss function.