Delta Data Collection for Enterprise Security Management
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Enterprise security management systems face significant burdens in collecting and analyzing large amounts of machine characteristics data, such as Windows registry data, due to the repeated uploading of full data sets, which strains network resources and is inefficient, especially when data has not substantially changed between collection cycles.
Innovation Solution
Implementing a delta data collection technique where client machines upload only the changes (deltas) from a baseline, with the machine assessment service applying these patches to maintain a virtual representation of the machine, reducing the amount of data transferred and utilizing text encoding for easier comparison and filtering.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If full copy of machine characteristics data is uploaded repeatedly from client machines, then the enterprise security management system can perform thorough analysis and detect issues quickly, but the network and resources experience significant burden
Solution Approach 1:
The patent extracts only the changed portions (deltas) of machine characteristics data from the full data set. Instead of uploading complete registry hives repeatedly, the system identifies and uploads only the specific registry keys, values, or subkeys that have changed since the last collection cycle, significantly reducing network bandwidth consumption while maintaining detection accuracy.
Solution Approach 2:
The patent segments the machine characteristics data into hierarchical components (registry hives, keys, subkeys, values) and tracks changes at each level. This segmentation allows the system to identify and transmit only the specific segments that have changed, rather than treating the entire data set as a single unit that must be reuploaded completely.
2Loss of information
If full copy of machine characteristics data is uploaded from each client machine for each collection cycle, then complete data is available for analysis, but the upload process creates excessive strain on company network and resources
Solution Approach 1:
The system extracts and transmits only the delta portions of data that have changed between collection cycles. By comparing the current state of machine characteristics against the previous state, the system identifies and uploads only the differences, maintaining data completeness for analysis while dramatically improving collection efficiency and reducing network strain.
Solution Approach 2:
The patent merges the delta data with the previously stored baseline data on the server side to reconstruct the complete current state of machine characteristics. This approach allows the system to maintain full data completeness for security analysis while only transmitting the minimal necessary changes over the network, thereby improving productivity.
3Reliability
If repeated full data uploads are performed, then the enterprise security management system maintains accurate machine representations, but network bandwidth and client resources are excessively consumed
Solution Approach 1:
The system extracts only the specific data elements that have changed (registry key modifications, value changes, new keys, deleted keys) and transmits these deltas to the server. This extraction approach ensures data accuracy is maintained for security analysis while reducing data transfer volume from hundreds of megabytes to potentially kilobytes or small megabytes per collection cycle.
Solution Approach 2:
The patent performs preliminary comparison and delta identification on the client side before upload. By pre-processing the data to identify changes against the stored baseline, the system prepares only the necessary delta portions for transmission, ensuring data accuracy is captured while minimizing the quantity of data transferred over the network.
Data Source
AI summary
Systems and methods are disclosed to implement a delta data collection technique for collecting machine characteristics data from client machines. In embodiments, the collected data is used by a machine assessment service to maintain a virtual representation of the client machine for assessments. To initialize the collection process, the client uploads an initial copy of the data in full. Subsequently, the client determines periodic deltas between a current baseline of the data and a last reported baseline, and the deltas are uploaded as patches. The machine assessment service then applies these patches to update the virtual representation of the client machine. In embodiments, to facilitate the generation or uploading of the patches, the client may generate the baselines in a different encoding format as used by the data. For example, baselines in the new encoding format may be more easily compared and manipulated during the patch generation process.