Partial Content Signature Inspection for Buffer Segments
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data inspection engines face challenges in efficiently processing large content items for multiple users, leading to high memory usage and response time issues, and often result in false positives due to pattern identification in file streams, which increases unnecessary content inspection.
Innovation Solution
The method involves receiving content item portions in a buffer divided into segments, computing partial signatures, and comparing them against known trustworthy signatures to allow transmission only if a match is found, thereby reducing unnecessary inspections and improving response time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If full buffering of content item is performed to compute signature, then signature matching accuracy is improved, but memory resource consumption increases and response time increases
Solution Approach 1:
The content item is divided into multiple segments, and signatures are computed incrementally for each segment as it is received in the buffer, rather than waiting for the entire content item. This allows partial signature matching to occur earlier, reducing response time while maintaining matching accuracy through cumulative signature computation.
Solution Approach 2:
The system pre-computes and stores signatures of known trustworthy content items in advance. When a new content item arrives, the system performs preliminary partial signature matching against the pre-computed signatures, allowing early identification of matching content without requiring full buffering of the incoming content item.
2Productivity
If winnowing process is used to compute signatures for fixed length segments, then partial content inspection is enabled, but signature computation time increases significantly
Solution Approach 1:
Instead of computing signatures for every fixed length segment as in winnowing, the system computes signatures only for received portions that fill buffer segments, and performs partial matching. This reduces the total number of signature computations required while still enabling partial content inspection and early matching identification.
3Productivity
If winnowing identifies patterns in file stream, then partial matching is achieved, but false positives increase due to common patterns in malware and non-malware files
Solution Approach 1:
The system pre-computes signatures only for content items that have been determined to be trustworthy through complete inspection. These pre-computed signatures serve as a reference for partial matching, ensuring that matches are against known good content rather than generic patterns, thereby reducing false positives while maintaining partial matching capability.
4Duration of action of stationary object
If multiple streams are associated with scanning states for content inspection, then continuous inspection is maintained, but system complexity increases
Solution Approach 1:
The system merges the signature computation and matching functions into the buffer management process. As content is received and buffered in segments, signatures are computed incrementally and matched against pre-computed trustworthy signatures. This integration eliminates the need for separate scanning states for multiple streams, reducing system complexity while maintaining continuous inspection capability.
Data Source
AI summary
Systems, methods and apparatus for a content item inspection. A plurality of portions of a content item are received in a buffer, the buffer divided into a plurality of segments. A partial signature of the content item is computed using the received portions of the content item in a most recently received segment and a partial signature computed for a preceding segment. The computed partial signature is compared against a plurality of partial signatures associated with trustworthy content items. If a matching partial signature associated with a trustworthy content item is found for the computed partial signature, the most recently received segment is allowed to be transmitted to a device that requested the content item.