Masked Image Model Training for Intra-Image Association Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing self-supervised learning methods fail to effectively learn the association relationship between regions inside an image, leading to a poor effect of model training.
Innovation Solution
A model training method that involves acquiring a first sample image with uncovered and covered image blocks, processing the image to obtain features, reconstructing the covered blocks, determining fusion prediction features, and updating model parameters based on target image features and loss functions to learn both intra-image and inter-image associations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If self-supervised learning is implemented through comparative learning method comparing current input image with other images, then the model can learn relationships between images, but the model fails to learn association relationships between respective regions inside the image, resulting in poor training effect
Solution Approach 1:
The image is divided into multiple image blocks (first image block, second image block, third image block) that can be independently processed. The model learns associations between these segmented blocks through reconstruction tasks, where covered blocks are reconstructed from uncovered blocks, enabling the model to capture intra-image regional relationships while maintaining overall image understanding.
Solution Approach 2:
Image features extracted from uncovered image blocks serve as intermediaries to reconstruct covered image blocks. The fusion prediction feature combines information from multiple blocks to predict the content of covered regions, enabling the model to learn associations between regions through these feature intermediaries without requiring labeled data.
2Device complexity
If the model only learns inter-image relationships through comparative learning, then the training process is simple, but the model cannot capture the association relationship between regions inside the image
Solution Approach 1:
The model performs preliminary feature extraction from uncovered image blocks before reconstruction. By pre-processing the visible blocks to extract their features, the model prepares the necessary information in advance to reconstruct covered blocks, simplifying the overall training process while enabling region association learning through the structured feature-based reconstruction approach.
Data Source
AI summary
The present disclosure provides a model training method and apparatus, and an electronic device; and the method includes: acquiring a first sample image, which includes a first image block which is uncovered and an second image block which is covered; processing the first sample image through a first model, to obtain a first image feature corresponding to the first image block; reconstructing the second image block according to the first image feature, to obtain a first image, and determining a fusion prediction feature of the first image block and the second image block, according to the first image feature; acquiring a target image feature in a target image, the target image being an image after preprocessing of the first sample image; and updating a model parameter of the first model, according to the first image, the second image block, the fusion prediction feature and the target image feature.