Self-Supervised Representation Learning for Vision-Based Reinforcement Learning

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Conventional vision-based reinforcement learning models are hindered by visual distractions, such as clouds and shadows, which are not considered during training, leading to deteriorated performance in real-world applications like robotics and autonomous driving.

Innovation Solution

A self-supervised representation learning method using a generator and discriminator, with augmentation algorithms to create distorted images, trains the model to minimize cost functions based on latent states, improving robustness to visual distractions.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If conventional vision-based reinforcement learning is used without considering visual distractions during training, then the model can be trained simpler and faster, but the performance deteriorates significantly when visual distractions appear in real-world applications

Engineering Contradiction:
Improveperformance robustness to visual distractionsVSAvoidtraining complexity
Core Design Contradiction:
ReliabilityVSDevice complexity

Solution Approach 1:

The patent applies preliminary action by pre-augmenting training images with various visual distractions (clouds, shadows, light variations) before the model sees them. This prepares the model in advance for real-world conditions, so when actual visual distractions appear during deployment, the model remains robust. The data augmentation process adds distracting elements to training images beforehand, allowing the model to learn invariant features despite the added complexity in training.

Inventive Principle:
Principle #10Preliminary action

2Reliability

If data augmentation with visual distractions is applied during training, then the model becomes robust to visual distractions, but the training process becomes more complex

Engineering Contradiction:
Improverobustness to visual distractionsVSAvoidtraining process complexity
Core Design Contradiction:
ReliabilityVSDevice complexity

Solution Approach 1:

The patent uses copying by creating augmented copies of original training images with added visual distractions. Instead of modifying the original images or creating entirely new datasets, the system generates multiple distorted versions (copies) of each training image with different distraction levels and types. These copied images with distractions serve as additional training samples, enabling the model to learn robustness without requiring fundamentally new training approaches.

Inventive Principle:
Principle #26Copying

3Adaptability or versatility

If the model is trained to handle various visual distractions, then the model generalizes better to real-world conditions, but the training data requires more processing and augmentation

Engineering Contradiction:
Improvegeneralization to real-world conditionsVSAvoiddata processing complexity
Core Design Contradiction:
Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent applies parameter changes by systematically varying parameters of visual distractions in training data, such as opacity levels, spatial positions, sizes, and types of distractions (clouds, shadows, light). By changing these parameters across different training samples, the model learns to generalize across diverse real-world conditions. The augmentation process modifies image parameters to simulate different environmental conditions, enabling the model to adapt to varying real-world scenarios.

Inventive Principle:
Principle #35Parameter changes

Data Source

PatentUS20240273792A1Method for self-supervised representation learning for vision-based reinforcement learning robust to visual distraction, data inference apparatus using self-supervised learning model, and storage medium storing instructions to perform method for self-supervised representation learning
Publication Date: 2024.08.15 RES & BUSINESS FOUND SUNGKYUNKWAN UNIV
  • US20240273792A1 patent drawing
  • US20240273792A1 patent drawing
  • US20240273792A1 patent drawing

AI summary

There is provided a self-supervised representation learning training method. The method comprises receiving an image to generate a first augmented image and a second augmented image using at least one augmentation algorithm; inputting the first augmented image and the second augmented image to the generator to output a first latent state from the first augmented image and output a second latent state from the second augmented image; and training the generator and the discriminator to minimize at least one cost function with respect to the first latent state and the second latent state.