Encoder Model Training With Temporal Offset Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing encoder models struggle to effectively extract features from sequences of frames, particularly in video data, as they often lose important temporal information due to the lack of robustness in handling temporal shifts, which is crucial for tasks like fine-grained human action recognition.
Innovation Solution
A computer-implemented method for training an encoder model by obtaining subsequences from an input signal, encoding them, and offsetting based on shifts between the sequence and subsequence, allowing the model to learn features that are aware of the succession dimension, thereby improving generalization and accuracy in downstream tasks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Stability of the object's composition
If feature persistence is enforced over the sequence, then temporal invariance is improved, but temporal order information is lost
Solution Approach 1:
The patent segments the sequence into multiple subsequences with different temporal offsets. Each subsequence is encoded separately, allowing the model to capture both temporal invariance (through shared encoding) and temporal order information (through offset differentiation). This segmentation resolves the contradiction by processing the sequence in differentiated temporal segments rather than as a single persistent unit.
Solution Approach 2:
The patent applies local quality by assigning different temporal offsets to different subsequences. Instead of treating the entire sequence uniformly, each subsequence receives a customized offset that preserves its temporal position information. This allows feature persistence to be maintained at the subsequence level while temporal order information is preserved through the offset assignments.
2Reliability
If random spatial cropping is used for data augmentation, then training robustness is improved, but temporal information is insufficient
Solution Approach 1:
The patent extends data augmentation from purely spatial operations to spatio-temporal operations. By introducing temporal offsets as an additional dimension of transformation, the model learns to handle both spatial variations (through cropping) and temporal variations (through offsets). This dimensional extension resolves the contradiction by adding temporal awareness to the augmentation process.
Solution Approach 2:
The patent changes the augmentation parameters from purely spatial (crop size, crop position) to spatio-temporal (crop size, crop position, temporal offset). This parameter extension allows the model to generate training samples that preserve temporal information while maintaining robustness through varied spatial and temporal transformations.
Data Source
Figure 1~2
Figure 3
Figure 4(a)~4(b)
AI summary
A computer-implemented method for training an encoder model (f) to extract features from an input signal comprising a sequence of frames, the computer-implemented method comprising: - obtaining at least one subsequence (20) from at least one such input signal; - encoding (22) the at least one subsequence with the encoder model (f); - offsetting the encoded at least one subsequence (24) based on a shift between the sequence of the at least one input signal and the at least one subsequence; - training (30) the encoder model (f) based on the offset encoded at least one subsequence (26).