GPU Inference Control via Frame Similarity and Queue Constraints
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Inference processing systems that execute on all frames of a video can lead to GPU occupation by frames with small content changes, reducing throughput and increasing waiting times due to unnecessary processing of similar frames.
Innovation Solution
An information processing apparatus that calculates the similarity between frames using a GPU, determines whether to execute inference processing based on similarity thresholds, and prioritizes frames with significant changes, using a constraint mechanism to manage queue lengths and optimize GPU usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If inference processing is executed on all frames, then complete analysis is achieved, but GPU is occupied by frames with small content changes reducing throughput
Solution Approach 1:
The patent extracts and identifies frames with significant content changes using similarity comparison, separating them from frames with minor changes. By taking out only the necessary frames for inference processing, the system maintains analysis completeness for important frames while reducing unnecessary processing on similar frames, thereby improving GPU throughput without sacrificing reliability.
Solution Approach 2:
Instead of executing inference processing on all frames (excessive action), the patent applies partial action by selectively processing only frames that meet the similarity threshold criterion. This partial processing approach reduces GPU occupation time while ensuring that frames with meaningful content changes are still thoroughly analyzed, resolving the contradiction between completeness and throughput.
2Reliability
If inference processing is executed on all frames, then no frames are missed, but waiting time increases due to unnecessary processing
Solution Approach 1:
The patent performs preliminary action by calculating similarity between consecutive frames before deciding whether to execute inference processing. This preliminary similarity check allows the system to identify and skip frames with minimal changes, reducing waiting time while ensuring that frames with significant changes are processed. The preliminary action maintains reliability by not skipping potentially important frames.
3Productivity
If similarity calculation is performed to reduce processing frames, then throughput improves, but system complexity increases
Solution Approach 1:
The patent introduces an intermediary similarity calculation mechanism that acts as a mediator between frame input and inference processing. This intermediary step compares frames and determines which ones require full inference processing, enabling throughput improvement without directly increasing the complexity of the inference engine itself. The similarity calculation serves as a filtering intermediary that manages the flow of frames to the GPU.
4Productivity
If multiple applications share GPU resources, then resource utilization improves, but queue accumulation occurs when throughput exceeds processing capacity
Solution Approach 1:
The patent applies dynamics by making the inference processing execution conditional and adaptive based on real-time similarity assessment. When multiple applications share GPU resources, the system dynamically determines which frames require processing by comparing them with previous frames. This dynamic approach prevents queue accumulation by avoiding the submission of redundant frames to the GPU, thereby maintaining high resource utilization without overwhelming the processing capacity.
Data Source
AI summary
An information processing apparatus includes processors configured to execute inference processing using a GPU on frames of a moving image through each application for each of a plurality of applications, acquire a first degree of similarity between a target frame and a frame at a time of execution of previous inference processing, acquire a second degree of similarity between the target frame and an immediately preceding frame that is executed after the previous inference processing, set a constraint on execution of inference processing for the target frame having a throughput larger than a certain value when a number of frames accumulated in a queue that is used for waiting for the execution of the inference processing and accumulates frames exceeds a certain number, and determine whether to execute the inference processing for the target frame based on the first degree, the second degree, and whether the constraint presents.