Grasp Neural Network Score Transformation for Runtime Shift Correction

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current grasping neural networks face performance degradation due to covariate shift, which occurs when the distribution of input data changes from training to deployment, leading to sub-optimal or incorrect grasp location predictions in robotic grasping operations.

Innovation Solution

The output of the grasping neural network is transformed to improve the accuracy of grasp location identification by filtering out incorrectly labeled scores and adjusting grasp points to avoid edges and ensure sufficient distance between grasp locations, thereby enhancing the robustness of robotic grasping operations.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If grasping neural networks are trained on synthetic data, then training efficiency is improved, but performance degrades when deployed with real camera data due to covariate shift

Engineering Contradiction:

Improvetraining efficiencyVSAvoidgrasp prediction accuracy

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

Instead of transforming the input data to match training distribution (traditional covariate shift correction), this patent inverts the approach by transforming the neural network output scores. The method applies a cumulative distribution function (CDF) transformation to the grasp scores, mapping them from the test data distribution back to the training data distribution, thereby correcting the covariate shift at the output stage rather than the input stage

Inventive Principle:

Principle #13The other way round (Inversion)

Solution Approach 2:

The patent changes the parameter distribution of the neural network output by applying a CDF transformation. This mathematical transformation modifies the score distribution to match the training distribution, effectively adjusting the parameter characteristics of the output without retraining the network or changing the input data

Inventive Principle:

Principle #35Parameter changes

2Speed

If neural network output is used directly without transformation, then processing speed is maintained, but grasp location accuracy decreases due to covariate shift

Engineering Contradiction:

Improveprocessing speedVSAvoidgrasp location accuracy

Core Design Contradiction:

SpeedVSMeasurement precision

Solution Approach 1:

The patent applies a preliminary CDF transformation to the neural network output scores before using them for grasp location selection. This preliminary action corrects the covariate shift in advance, ensuring that the transformed scores accurately reflect grasp quality without requiring subsequent complex corrections or retraining

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The CDF transformation acts as an intermediary between the neural network output and the grasp location selection process. It mediates the distribution mismatch by transforming the scores into a common distribution space, enabling accurate grasp prediction without modifying the original neural network or input data

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS20240198515A1Transformation for covariate shift of grasp neural networks

Publication Date: 2024.06.20 SIEMENS AG

AI summary

A covariate shift generally refers to the change of the distribution of the input data (e.g., noise distribution) between the training and inference regimes. Such covariate shifts can degrade the performance grasping neural networks, and thus robotic grasping operations. As described herein, an output of a grasp neural network can be transformed, so as to determine appropriate locations on a given object for a robot or autonomous machine to grasp.