CNN Image Stitching With Small Overlap Displacement Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image stitching techniques face challenges in achieving accurate and reliable results due to local maxima or minima in similarity measures, leading to inaccurate stitching or failure, particularly in X-ray imaging of large anatomies, which also increases patient dose.
Innovation Solution
A method for training a convolutional neural network (CNN) to determine optimal image displacement for stitching by optimizing cost functions, using a data-driven approach to learn a representation that reduces overlapping areas and minimizes patient exposure, while avoiding false global minima.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If large overlapping areas are used in image stitching, then the stitching reliability is improved, but the patient dose increases
Solution Approach 1:
The patent transforms the stitching problem from direct image space to a learned feature space using a neural network. By changing the parameter space from raw pixel values to learned representations, the system can achieve reliable stitching with smaller overlapping areas, thereby reducing patient dose while maintaining stitching accuracy.
Solution Approach 2:
The patent replaces traditional mechanical/image-processing-based similarity measures with a data-driven neural network approach. This substitution allows the system to learn robust displacement patterns from training data, enabling accurate stitching even with minimal overlapping areas, thus resolving the contradiction between reliability and patient dose.
2Object-affected harmful factors
If small overlapping areas are used in image stitching, then the patient dose is reduced, but the stitching reliability deteriorates
Solution Approach 1:
The patent performs preliminary training of the neural network on large datasets with known displacements before actual stitching. This preliminary action embeds robust displacement estimation capabilities into the network, enabling it to achieve high stitching reliability even when presented with small overlapping areas during deployment, thus allowing dose reduction without sacrificing reliability.
Solution Approach 2:
By replacing traditional similarity measure mechanisms with a pre-trained neural network, the system gains the ability to extract meaningful displacement information from minimal overlapping regions. The neural network's learned features provide robustness that compensates for the reduced overlap, maintaining reliability while enabling smaller overlap areas and lower patient dose.
3Device complexity
If classical similarity measures are used for displacement estimation, then the process is simple, but the accuracy deteriorates due to local maxima or minima
Solution Approach 1:
The patent substitutes classical similarity measure algorithms with a neural network-based displacement estimation system. Although this increases computational complexity, it eliminates the local maxima/minima problem inherent in traditional methods by learning from training data, thereby significantly improving measurement precision while accepting a reasonable increase in process complexity.
Solution Approach 2:
The neural network acts as an intermediary between the input images and the displacement estimation. Instead of directly computing similarity measures that are prone to local extrema, the network processes images through learned feature representations, serving as a mediator that transforms the problem into a more robust solution space where accurate displacement estimation can be achieved.
Data Source
AI summary
The invention concerns an image processing system configured for carrying out a computer-implemented method for generating a stitched image with a convolutional neural network (208), a computer-implemented method for generating a stitched image with a convolutional neural network, and a method of training a convolutional neural network, for determining an image representation for image stitching. The training of the convolutional neural network comprises receiving image data, wherein the image data comprises at least two images (101, 102), overlapping each other in an overlapping area (103, 203) in which the images overlapping each other in a target anatomy (104), and wherein the overlapping area comprises an optimum displacement between the two images such that the two images can be correctly combined for image stitching, determining a cost function (105, 205) of the displacement of the two images in the overlapping area, determining a deviation of the cost function from a reference cost function, and optimizing the CNN based on the deviation.