Video Camera Selective Frame Buffering for Cascaded Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge of managing large image buffer sizes in video analytics systems due to latency between image capture and metadata delivery, especially in video cameras, where a cascade of evaluation functions requires buffering raw image data for subsequent analysis, leading to memory constraints.
Innovation Solution
A video camera system that buffers only image frames under evaluation by a first function, allowing the second function to access these frames when ready, while deleting unneeded frames, thereby reducing memory usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If image data is buffered to compensate for latency between capture and metadata delivery, then the second evaluation function can access required image frames, but the image buffer size becomes excessively large
Solution Approach 1:
The system performs preliminary actions by having the first evaluation function mark image frames as 'under evaluation' before the second evaluation function needs them. This allows the buffer function to proactively retain only those frames that will be needed, rather than buffering all frames in advance. The marking mechanism enables early identification of required frames, reducing buffer size while ensuring availability when needed.
Solution Approach 2:
The buffer function serves itself by using metadata from the first evaluation function to automatically determine which frames to retain. Instead of requiring external control or buffering all frames, the system uses the evaluation metadata to self-regulate the buffer contents, retaining only frames marked as 'under evaluation' and deleting others automatically.
2Measurement precision
If the first evaluation function has evaluation time larger than 1/fps, then comprehensive analysis can be performed, but image frames may be lost by the time evaluation completes
Solution Approach 1:
The system performs preliminary actions by marking frames as 'under evaluation' as soon as the first evaluation function identifies them, well before the evaluation completes. This preliminary marking ensures that the buffer function retains these frames throughout the extended evaluation period, preventing frame loss even when evaluation time exceeds the frame interval (1/fps).
Solution Approach 2:
The buffer function provides a cushioning mechanism by retaining marked frames in memory during the extended evaluation period. This creates a protective buffer that prevents frame loss or unavailability, ensuring that when the comprehensive evaluation finally completes, the required frames are still available for the second evaluation function.
3Adaptability or versatility
If all image frames are buffered initially, then the second evaluation function can access any frame, but memory usage increases significantly
Solution Approach 1:
The system extracts only the necessary information from the first evaluation function's metadata - specifically which frames are marked as 'under evaluation' - and uses this extracted information to control buffer contents. Instead of buffering all frames, only the extracted subset of marked frames is retained, dramatically reducing memory usage while maintaining accessibility to required frames.
Solution Approach 2:
The buffer function applies local quality by treating different frames differently based on their evaluation status. Frames marked as 'under evaluation' receive special treatment (retention in buffer), while unmarked frames are deleted. This selective quality approach ensures that memory resources are allocated only to frames that need them, optimizing both accessibility and memory efficiency.
Data Source
AI summary
A video camera comprises an image sensor for capturing image data of image frames of a video stream; an image buffer for buffering image; and circuitry. The circuitry executes: a first evaluation function to evaluate a subset of image frames in the video stream by evaluating data pertaining to the subset of image frames and to output a corresponding first evaluation result, wherein the evaluation of an image frame has an evaluation time larger than 1/fps for the video stream; a buffer function to, for each image frame being, mark the image frame as being under evaluation and indicate to the image buffer to buffer image data of the marked image frame; and a second evaluation function to access the image buffer to evaluate image data of each of the subset of image frames based on the corresponding first evaluation result and output a corresponding second evaluation result.