Medical Image Recognition With Two-Stage Spatial-Temporal Transformers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing object recognition methods in medical imaging lack accuracy in characterizing spatial and temporal changes in medical images, leading to suboptimal recognition results.
Innovation Solution
The method employs a first transformer network to extract spatial features from multiple medical images of the same object at different times, fuses these features, and then uses a second transformer network to derive a spatial-temporal feature, considering the temporal relationship between the images, to improve recognition accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional image processing methods are used for object recognition in medical imaging, then the recognition process is simple and fast, but the recognition accuracy is insufficient due to inability to characterize spatial and temporal changes
Solution Approach 1:
The patent segments the feature extraction process into two distinct transformer networks: the first transformer network extracts spatial features from individual medical images, while the second transformer network extracts temporal features from the sequence of spatial features. This segmentation allows each network to specialize in specific aspects of the data, improving overall recognition accuracy while maintaining manageable system complexity through modular design.
Solution Approach 2:
The patent transitions from analyzing only spatial dimensions of medical images to incorporating the temporal dimension by processing sequences of medical images taken at different time points. The second transformer network specifically captures temporal relationships and changes across time, enabling the system to characterize both spatial and temporal variations in medical images for more accurate object recognition.
2Measurement precision
If multiple medical images at different times are processed to capture temporal changes, then the recognition accuracy improves, but the computational complexity and processing time increase
Solution Approach 1:
The first transformer network performs preliminary extraction of spatial features from each medical image in the sequence before the second transformer network processes the temporal relationships. This preliminary action organizes the data in advance, allowing the second network to focus specifically on temporal patterns without reprocessing the entire image data, thus reducing overall processing time while maintaining accurate temporal change characterization.
Solution Approach 2:
The patent extracts and separates spatial features from temporal features using two distinct transformer networks. The first network extracts spatial features from individual images, which are then fed as input to the second network that extracts temporal features from the sequence. This extraction and separation approach allows the system to process spatial and temporal information independently and efficiently, reducing computational redundancy and processing time.
3Loss of information
If spatial features of multiple medical images are fused and temporal relationships are considered, then the representation of spatial-temporal information becomes more accurate, but the model complexity increases
Solution Approach 1:
The first transformer network acts as an intermediary that processes individual medical images to extract spatial features, which then serve as input to the second transformer network. This intermediary structure allows the system to progressively transform raw image data into meaningful spatial representations, and then into temporal patterns, improving information representation accuracy while managing model complexity through staged processing.
Solution Approach 2:
The patent segments the complex task of spatial-temporal feature extraction into two specialized transformer networks. The first network handles spatial feature extraction from individual images, while the second network handles temporal feature extraction from sequences of spatial features. This segmentation divides the complex model into manageable modules, each optimized for specific functions, thereby improving information representation while controlling overall model complexity.
Data Source
AI summary
An object recognition method includes extracting, by a first transformer network, spatial features of a plurality of medical images respectively, the plurality of medical images being images of a same object at different times, and fusing the extracted plurality of spatial features, to obtain a first fusion spatial feature of the object. The method further includes extracting, by a second transformer network, a spatial-temporal feature of the object based on the first fusion spatial feature. The spatial-temporal feature indicates a change in the spatial features of the plurality of medical images at the different times. The method further includes recognizing a state of the object based on the spatial-temporal feature, to obtain a recognition result of the object.