Robot Vision Object Recognition via Variance Image Masking

Solution Overview

Problem

Existing robot systems face inefficiencies in recognizing and classifying objects in unstructured environments due to background clutter, requiring objects to be moved to a uniform background for inspection, which consumes time and energy.

Innovation Solution

A method involving a robot arm with a gripper and image sensor that captures multiple images of the object, computes an average and variance image, forms a filtering image, and masks the average image to isolate the object, allowing for object classification without moving it to a clean background.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If objects are moved to a uniform background for inspection, then object recognition quality is improved, but time consumption and energy consumption increase due to frequent robot arm movements

Engineering Contradiction:

Improveobject recognition qualityVSAvoidtime consumption

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The patent extracts the object from the complex background environment by computing a difference image that removes background elements. The system captures multiple images with the robot arm at different positions, computes their average to represent the background, and then subtracts this average from individual images to isolate the object of interest. This allows recognition without physically moving the object to a clean background.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent creates a virtual copy of the background by computing an average image from multiple captured frames. This average image serves as a representation of the background that can be subtracted from individual frames, effectively copying the background characteristics without requiring physical relocation of objects or the robot arm to a standardized inspection position.

Inventive Principle:

Principle #26Copying

2Measurement precision

If objects are moved to a uniform background for inspection, then object recognition quality is improved, but energy consumption increases due to frequent robot arm movements

Engineering Contradiction:

Improveobject recognition qualityVSAvoidenergy consumption

Core Design Contradiction:

Measurement precisionVSUse of energy by moving object

Solution Approach 1:

The patent extracts the object from the complex background environment by computing a difference image that removes background elements. The system captures multiple images with the robot arm at different positions, computes their average to represent the background, and then subtracts this average from individual images to isolate the object of interest. This allows recognition without physically moving the object to a clean background.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent creates a virtual copy of the background by computing an average image from multiple captured frames. This average image serves as a representation of the background that can be subtracted from individual frames, effectively copying the background characteristics without requiring physical relocation of objects or the robot arm to a standardized inspection position.

Inventive Principle:

Principle #26Copying

3Adaptability or versatility

If pattern recognition algorithms are used on images with background clutter, then object recognition in unstructured environments is enabled, but recognition accuracy decreases

Engineering Contradiction:

Improveability to recognize objects in unstructured environmentsVSAvoidrecognition accuracy

Core Design Contradiction:

Adaptability or versatilityVSMeasurement precision

Solution Approach 1:

The patent extracts the object from the complex background environment by computing a difference image that removes background elements. The system captures multiple images with the robot arm at different positions, computes their average to represent the background, and then subtracts this average from individual images to isolate the object of interest. This allows recognition without physically moving the object to a clean background.

Inventive Principle:

Principle #2Taking out (Extraction)

Data Source

PatentEP2629939B1Method for the filtering of target object images in a robot system

Publication Date: 2019.03.13 ZENROBOTICS OY

Figure 1
Figure 2A
Figure 2B

AI summary

The invention relates to a method and system for recognizing physical objects (103-105). In the method an object (105) is gripped with a gripper (1 12), which is attached to a robot arm (1 16) or mounted separately. Using an image sensor (1 14), a plurality of source images of an area comprising the object (105) is captured while the object (105) is moved with the robot arm (1 16). The camera (1 14) is configured to move along the gripper (1 12), attached to the gripper (1 12) or otherwise able to monitor the movement of the gripper (1 12). Moving image elements are extracted from the plurality of source images by computing a variance image from the source images and forming a filtering image from the variance image. A result image is obtained by using the filtering image as a bitmask. The result image is used for classifying the gripped object (105).