Pathological Image Extraction Using Local and Non-Local Feature Vectors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for pathological image analysis using digitized atlases face challenges in extracting relevant reference images, especially when the medical case is difficult to interpret or is a first-time case, making it hard to set appropriate queries and find suitable references.
Innovation Solution
An image extraction device and system that calculates local and non-local feature amounts of a pathological image using pre-trained convolutional neural networks, allowing for the extraction of appropriate reference images from a database based on similarity, and includes features for determining gene mutations and displaying relevant information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If text query is used to search pathological images in a database, then the search process is simple and fast, but it is difficult to extract appropriate reference images for difficult-to-interpret or first-time medical cases
Solution Approach 1:
The patent replaces the mechanical text-based query system with an image-based query system using neural networks. Instead of manually entering text descriptions to search for reference images, the system automatically processes the input pathological image through convolutional neural networks to extract features and retrieve similar images from the database, eliminating the need for manual text formulation while improving accuracy
Solution Approach 2:
The patent changes the search parameter from text description to image feature vectors. By transforming the query from a text-based approach (requiring manual description of medical case characteristics) to an image-based approach (using automated feature extraction through neural networks), the system maintains search efficiency while dramatically improving the ability to find appropriate reference images for complex cases
2Productivity
If local feature amount only is calculated for pathological image, then the calculation process is simple and fast, but the ability to capture overall image characteristics and relationships is insufficient
Solution Approach 1:
The patent segments the feature extraction process into two distinct components: local feature extraction (using convolutional neural networks to capture detailed local characteristics) and non-local feature extraction (using self-attention mechanisms to capture global relationships). By dividing the feature extraction into these segments, the system achieves both computational efficiency and comprehensive feature representation
Solution Approach 2:
The patent merges local features and non-local features into a unified feature representation. The convolutional neural network extracts local features from image regions, while the self-attention mechanism captures non-local relationships between different regions. These two types of features are combined to create a comprehensive feature vector that preserves both detailed local information and global contextual relationships, improving the overall accuracy of reference image retrieval
3Measurement precision
If non-local feature amount is calculated using multiple hidden layers and multiple neural networks, then the feature representation becomes more accurate and comprehensive, but the device complexity and computational load increase
Solution Approach 1:
The patent uses pre-trained convolutional neural networks for feature extraction, eliminating the need to train complex models from scratch. The pre-trained networks have already learned robust feature representations during their training phase, so they can be directly applied to extract features from pathological images. This preliminary preparation significantly reduces the computational burden and device complexity during the actual inference phase while maintaining high feature representation accuracy
Solution Approach 2:
The patent introduces a self-attention mechanism as an intermediary between the convolutional neural network and the reference image retrieval process. The self-attention mechanism efficiently captures non-local relationships by computing attention scores between different image regions, serving as a bridge that connects local features to global contextual information. This intermediary approach achieves comprehensive feature representation with relatively simple computational operations, avoiding the need for extremely complex multi-layer architectures
Data Source
AI summary
An image extraction device, an image extraction system, an image extraction method, and an image extraction program are capable of extracting an appropriate reference image from a database by using a pathological image as a query. The image extraction device includes: an acquisition unit which acquires a pathological image; a first calculation unit which calculates a local feature amount of the pathological image; a second calculation unit which calculates a non-local feature amount based on a correlation with the local feature amount; a third calculation unit which calculates a degree of similarity between the non-local feature amount and a feature amount of each of a plurality of reference images stored in a database; and an extraction unit which extracts one or more reference images based on the degree of similarity.