Neural Network Frame Classification with Early Termination
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current deep learning technologies for image and video classification are inefficient in determining relevant frames for class identification, as they often generate final feature vectors for all frames, leading to increased processing time and costs without distinguishing between necessary and unnecessary frames for class determination.
Innovation Solution
A processor-implemented method using a neural network with multiple layers to generate final feature vectors for frames, determine frame classes, and calculate similarity scores between reference and non-final feature frames, allowing for early termination of processing sequences for non-relevant frames, thereby optimizing throughput and resource usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If final feature vectors are generated for all frames, then class determination accuracy is maintained, but processing time and computational costs increase
Solution Approach 1:
The system performs preliminary actions by generating feature vectors at intermediate stages before completing the full sequence processing. This allows the system to evaluate whether a frame can be terminated early based on accumulated feature information, avoiding unnecessary completion of processing for frames that are already sufficiently classified
Solution Approach 2:
The processing sequence is made dynamic by allowing early termination based on real-time evaluation of feature vector similarity and class determination confidence. The system adjusts the processing depth for each frame individually, transitioning from a static fixed-depth processing model to a dynamic variable-depth model that adapts to the specific characteristics of each frame
2Productivity
If processing sequences are terminated early for non-relevant frames, then processing time is reduced, but the risk of missing relevant frame information increases
Solution Approach 1:
The system implements feedback mechanisms where intermediate feature vectors are continuously evaluated against reference feature vectors and class criteria. This feedback loop allows the system to make informed decisions about whether to terminate processing early or continue to the final stage, ensuring that only frames with sufficient confidence are terminated
Solution Approach 2:
The system applies partial action by processing frames to the extent necessary for reliable classification rather than always processing to completion. For frames that can be confidently classified at intermediate stages, the system stops processing early (partial action), while frames requiring more analysis continue through the full sequence
3Measurement precision
If feature vectors are generated for all frames, then comprehensive frame analysis is achieved, but computational resource consumption increases
Solution Approach 1:
The system extracts only the necessary computational processing for frames that require it. By evaluating feature vector similarity and class determination criteria at intermediate stages, the system identifies frames that can be terminated early, extracting only the essential computational resources needed for those frames rather than uniformly processing all frames through the complete sequence
Data Source
AI summary
A processor-implemented method includes generating respective final feature vectors of a plurality of frames of time-series data, while sequentially processing the plurality of frames by using a neural network comprising a plurality of layers, determining a class of the time-series data based on at least one final feature vector of the respective final feature vectors, generating a reference feature vector based on the at least one final feature vector, calculating a similarity score between the reference feature vector and a feature vector of at least one second frame, wherein the second frame includes a non-final feature frame where the final feature vector is not generated, and determining the at least one second frame to be the frame corresponding to the class, based on a result of comparing the similarity score and a threshold value.