Neural Network Keyframe Scheduling for Video Semantic Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video semantic segmentation technologies face challenges in efficiently scheduling key frames, which is crucial for applications like automatic driving and video monitoring, as they struggle to rapidly and accurately adapt to changes in low-level features between frames.
Innovation Solution
A key frame scheduling method that involves performing feature extraction on current frames using a neural network with two layers of different depths, determining scheduling probabilities based on low-level features of adjacent key frames, and selectively performing high-level feature extraction on scheduled key frames to improve scheduling efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If full high-level feature extraction is performed on every frame, then segmentation accuracy is improved, but computational complexity and processing time increase significantly
Solution Approach 1:
The patent segments the video processing task into two parts: low-level feature extraction performed on every frame, and high-level feature extraction performed only on key frames. This segmentation allows the system to maintain segmentation accuracy on important frames while reducing overall computational complexity by avoiding redundant high-level extraction on non-key frames.
Solution Approach 2:
The patent applies partial action by performing complete high-level feature extraction only on key frames (a subset of frames) rather than on all frames. The scheduling probability mechanism determines which frames receive the full processing action, optimizing the balance between accuracy and computational load.
2Speed
If key frame scheduling is performed using traditional methods, then processing speed is improved, but adaptability to feature changes between frames deteriorates
Solution Approach 1:
The patent introduces a feedback mechanism where the scheduling probability of a frame is determined by comparing its low-level features with those of the previous key frame. This feedback loop allows the system to adapt to feature changes between frames dynamically, improving both processing speed and adaptability by intelligently selecting key frames based on actual feature variations.
Solution Approach 2:
The patent performs preliminary low-level feature extraction on all frames before determining scheduling probabilities. This preliminary action enables the system to quickly assess feature changes and make informed decisions about which frames require full processing, thereby improving both speed and adaptability.
3Measurement precision
If low-level feature extraction is performed on all frames, then feature change detection accuracy is improved, but energy consumption increases
Solution Approach 1:
The patent segments the energy-intensive processing into two levels: low-level feature extraction (energy-efficient) performed on all frames for change detection, and high-level feature extraction (energy-intensive) performed only on key frames. This segmentation maintains feature change detection accuracy while significantly reducing overall energy consumption.
Solution Approach 2:
The patent applies partial action by limiting full high-level processing to only those frames identified as key frames through the scheduling probability mechanism. This reduces energy consumption by avoiding unnecessary full processing on frames that do not require it, while maintaining detection accuracy through comprehensive low-level analysis of all frames.
Data Source
AI summary
A key frame scheduling method and apparatus include: performing feature extraction on a current frame through a first network layer of a neural network to obtain low-layer features of the current frame acquiring a scheduling probability of the current frame according to low-level features of a previous key frame adjacent to the current frame and the low-level features of the current frame; determining whether the current frame is scheduled as a key frame according to the scheduling probability value of the current frame; and when determining that the current frame is scheduled as a key frame, performing feature extraction on low-level features of a current key frame via a second network layer of the neural network to obtain high-level features of the current key frame, where the network depth of the first network layer is less than the network depth of the second network layer.