Electron Microscope Image Augmentation for Boundary-Preserving ML

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional data augmentation techniques are ineffective for electron microscope images, particularly TEM images, due to minimal shape and size variations, boundary information loss, and texture randomness, which degrade metrology accuracy in automated metrology algorithms.

Innovation Solution

A data augmentation method that converts the interior and exterior regions of interest in electron microscope images to single colors and modifies pixel positions within the interior region, preserving the object's shape and size, to enhance training data for machine learning models.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Quantity of substance

If conventional data augmentation techniques (rotation, symmetry, cropping, resizing) are applied to TEM images, then the quantity of training data is increased, but the boundary information is distorted and metrology accuracy is degraded

Engineering Contradiction:

Improvequantity of training dataVSAvoidmetrology accuracy

Core Design Contradiction:

Quantity of substanceVSMeasurement precision

Solution Approach 1:

The patent applies different transformation rules to different regions of the image: the object region of interest undergoes only non-distorting transformations (translation, reflection, rotation around object center), while the background region can undergo additional transformations. This local differentiation preserves boundary information in the critical object region while still generating data diversity.

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

The patent changes the parameters of transformation operations to suit TEM image characteristics. Specifically, it uses translation instead of cropping, reflection across arbitrary axes instead of only horizontal/vertical flips, and rotation around the object center instead of image center. These parameter adjustments prevent boundary distortion while maintaining data augmentation effectiveness.

Inventive Principle:

Principle #35Parameter changes

2Adaptability or versatility

If standard data augmentation transformations are applied to TEM images, then dataset diversity is enhanced, but the shape and size consistency of objects is altered

Engineering Contradiction:

Improvedataset diversityVSAvoidobject shape consistency

Core Design Contradiction:

Adaptability or versatilityVSShape

Solution Approach 1:

The patent segments the image into the object region of interest and the background region, then applies different transformation sets to each segment. The object region undergoes only shape-preserving transformations (translation, reflection, rotation), while the background can undergo additional transformations, thus maintaining object shape consistency while achieving dataset diversity.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Instead of applying the full set of conventional augmentations to the entire image, the patent inverts the approach by applying a restricted set of transformations only to the object region, and a different set to the background region. This inversion of the traditional augmentation strategy preserves object geometry while still creating diverse training samples.

Inventive Principle:

Principle #13The other way round (Inversion)

3Adaptability or versatility

If data augmentation is applied to TEM images with randomized texture regions, then the texture variations are amplified, but the boundary information is lost or distorted

Engineering Contradiction:

Improvetexture variation coverageVSAvoidboundary information loss

Core Design Contradiction:

Adaptability or versatilityVSLoss of information

Solution Approach 1:

The patent applies different transformation rules to different regions: the object region undergoes only boundary-preserving transformations (translation, reflection, rotation around object center), while the background region with randomized texture can undergo additional transformations. This local differentiation ensures boundary information is preserved in the object region while still achieving texture variation coverage through background transformations.

Inventive Principle:

Principle #3Local quality

Data Source

PatentUS20250272897A1Data augmentation method and apparatus for machine learning and applications thereof

Publication Date: 2025.08.28 SK HYNIX INC

AI summary

Data augmentation methods and apparatus for machine learning, and utilization thereof, are disclosed. A computer-implemented method for data augmentation in electron microscope imaging, the method comprising: receiving, using a processor, an input image captured by an electron microscope; processing, using a processor, the input image to generate an augmented image dataset, the processing comprising: generating a first transformed image by converting an interior region of an object region of interest in the input image to a single color; generating a second transformed image by converting a region other than the object region of interest in the input image to a single color; and generating a third transformed image by modifying pixel positions exclusively within the interior region of the object region of interest in the input image; and outputting the augmented image dataset to train a machine learning model.