Automated Image Alignment via Triangular Mesh Refinement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image alignment systems for pathological diagnosis, particularly for tissue samples stained with different methods like H&E and IHC, require manual user intervention due to significant differences in image appearance and local deformations, making it tedious and impractical, especially for large images.
Innovation Solution
An automated method using keypoint generation, cross-matching, and Delaunay triangulation to create and refine triangular meshes for affine transformation estimation, allowing for synchronized viewing across images with local deformations, and storing aligned coordinates for accurate alignment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If manual alignment methods are used for tissue sample images, then users can locate corresponding areas on virtual slides, but the process becomes tedious and impractical for large images requiring repeated manual intervention
Solution Approach 1:
The system performs automatic image alignment without requiring user intervention. The computer system automatically identifies corresponding areas between H&E and IHC images through algorithmic processing, eliminating the need for manual locating and alignment operations by pathologists
Solution Approach 2:
The patent replaces manual mechanical alignment operations with automated computational methods. Instead of users manually navigating and aligning images, the system uses computer vision algorithms and image processing techniques to automatically determine and apply transformations
2Measurement precision
If manual alignment is performed at high resolution, then local details can be examined, but the appearance between corresponding regions diverges rapidly making it difficult to find matching points
Solution Approach 1:
The system performs preliminary alignment at lower resolutions to establish initial correspondence between images. This preliminary transformation is then applied to guide high-resolution examination, making it easier to locate corresponding regions when detailed viewing is required
Solution Approach 2:
The patent implements a multi-resolution approach, transitioning between different scales of image examination. The system first establishes alignment relationships at coarser resolutions where matching is easier, then applies these relationships to facilitate accurate correspondence identification at higher resolutions
3Adaptability or versatility
If tissue slices are stained by different methods (H&E and IHC), then comprehensive diagnostic information can be obtained, but significant differences in image appearance and local deformations make alignment challenging
Solution Approach 1:
The system applies different transformation models to different regions of the images. Instead of assuming uniform transformation across the entire image, the patent allows local areas to have different transformation characteristics, accommodating the local deformations that occur during tissue processing and staining
Solution Approach 2:
The patent transforms images from different staining methods into a common coordinate system through parameter transformations. The system adjusts geometric parameters and transformation models to account for differences in appearance between H&E and IHC stained images, enabling accurate alignment despite staining method variations
4Area of stationary object
If large images (e.g., 100k x 100k) are manually aligned, then complete coverage can be examined, but the process becomes excessively time-consuming and impractical
Solution Approach 1:
The system divides large images into smaller manageable regions or tiles for processing. By segmenting the alignment task into smaller units, the system can efficiently process large areas without requiring excessive manual intervention time, while maintaining overall alignment accuracy across the complete image
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method for automatically aligning images includes (a) determining initial correspondent point features between a first image and a second image, (b) creating a triangular meshes for the images from the initial correspondent point features within the images, and (c) refining point correspondence between the first and second images based on affine transformation estimation using the triangular meshes. The method may also include (d) creating refined triangular meshes for the images from the point correspondence refined in (c), and (e) determining coordinate alignment within the areas of pairs of correspondent triangles in the refined triangular meshes through interpolation of affine transformation on the pairs of correspondent triangles.