Masked Image Reconstruction Using Multi-Depth Encoder Features
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing self-supervised visual representation learning method based on mask-reconstruction is slow due to relying solely on the final output of the encoder's final output for image reconstruction, which does not utilize lower-layer features, leading to inefficient learning and model training.
Innovation Solution
Implement a model with an encoding network and multiple decoding networks with different depths, utilizing features with varying scales for image reconstruction, and incorporating random occlusion to generate diverse supervision information, enhancing learning efficiency and precision.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If only the final output feature of the encoder is used for image reconstruction, then the model structure is simple, but the learning efficiency and training speed are slow
Solution Approach 1:
The patent divides the encoding network into multiple network blocks at different depths, and introduces multiple decoding networks corresponding to different network blocks. This segmentation allows features from different depths to be utilized for image reconstruction, improving learning efficiency without creating an overly complex structure.
Solution Approach 2:
The patent adds a depth dimension to the feature utilization by introducing multiple decoding networks at different network depths. Instead of using only the final output feature, features from intermediate network blocks are also utilized, creating a multi-dimensional feature extraction approach that enhances learning efficiency.
2Measurement precision
If only the final output feature is used for reconstruction, then the model structure is simple, but the feature extraction detail and reconstruction accuracy are insufficient
Solution Approach 1:
The patent segments the feature extraction process into multiple stages by introducing decoding networks at different network block depths. Each decoding network processes features at its corresponding depth level, enabling detailed feature extraction and improving reconstruction accuracy through multi-level feature utilization.
Solution Approach 2:
Different decoding networks are assigned to process features at different depths, with each decoding network optimized for its specific depth level. This local quality approach ensures that features at each depth are processed appropriately, improving overall reconstruction accuracy without requiring a single overly complex decoding network.
3Productivity
If mask-reconstruction method is used, then self-supervised learning can be performed, but the training process is slow due to single-scale feature utilization
Solution Approach 1:
The patent introduces a depth dimension to feature utilization by connecting multiple decoding networks to different network blocks. This creates multi-scale feature extraction, where features at different depths provide diverse scale information, accelerating training through more comprehensive feature utilization.
Solution Approach 2:
The patent segments the feature utilization process into multiple depth levels, with each decoding network handling features at its corresponding level. This segmentation enables parallel processing of multi-scale features, improving training speed while maintaining the adaptability of self-supervised mask-reconstruction learning.
Data Source
Figure 1~2
Figure 3~4
Figure 5
AI summary
A method and an apparatus for training a model, an electronic device, and a storage medium are provided. The method includes: dividing a training image into blocks, to obtain a plurality of first image blocks (301); performing occlusion on the plurality of first image blocks, to obtain a plurality of second image blocks, where the plurality of second image blocks are unoccluded first image blocks in the plurality of first image blocks (302); inputting a feature vector of each second image block into an encoding network to perform encoding, to obtain a plurality of encoding features corresponding to a plurality of network blocks, where the plurality of network blocks are in a one-to-one correspondence with the plurality of encoding features (303); inputting each encoding feature into a decoding network corresponding to each encoding feature to perform image reconstruction, to obtain a reconstructed image corresponding to each decoding network (304); and training the model based on the reconstructed image corresponding to each decoding network and supervision information corresponding to each decoding network (305). This helps improve model training efficiency.