Artifact Store for Network File Extraction and Reconstruction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for extracting and reconstructing files from network data are inefficient, leading to excessive disk operations, file management challenges, and cumbersome reconstruction processes due to repeated deep packet inspection (DPI) and unnecessary data extraction, which negatively impact performance and require improved efficiency and capability.
Innovation Solution
A method and system that utilize an artifact store to linearly store and manage artifact chunks from network data, employing unique artifact locator identifiers (ALI) and artifact chunk information (ACI) to efficiently locate and reconstruct files by storing and pointing to artifact chunks across multiple files, allowing for forward and reverse sequential access within and between files.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all network data is extracted and stored to disk, then complete file capture is achieved, but disk write operations increase significantly
Solution Approach 1:
The patent extracts only the necessary file content portions (artifacts) from network data using deep packet inspection, rather than capturing and storing all network data. This selective extraction reduces the volume of data written to disk while maintaining complete capture of relevant files through the artifact chunk storage mechanism.
2Manufacturing precision
If deep packet inspection is invoked twice, then file extraction capability is improved, but processing time increases
Solution Approach 1:
The patent performs deep packet inspection once during data capture to extract artifact chunks and store them in the artifact store with associated metadata. This preliminary action prepares the data structure for efficient later retrieval and reconstruction, eliminating the need for a second DPI invocation and reducing overall processing time.
3Quantity of substance
If millions of files are created, then comprehensive data collection is achieved, but file management complexity increases
Solution Approach 1:
The patent segments files into smaller artifact chunks and stores them in a structured artifact store with metadata linking chunks to their parent files. This segmentation approach maintains comprehensive data collection while simplifying management through chunk-level organization and efficient reconstruction capabilities.
4Reliability
If all network data is extracted, then complete analysis capability is achieved, but unnecessary data processing occurs
Solution Approach 1:
The patent extracts only relevant artifact chunks from network data based on analysis requirements, rather than processing all captured data. The artifact store structure enables selective retrieval of needed files and chunks, eliminating unnecessary data processing while maintaining complete analysis capability for relevant threats.
Data Source
AI summary
A method is provided of extracting file content from a live stream of network data streaming multiple files. The method includes receiving artifact chunks each including a portion of file content of a file of the multiple files as identified by DPI at first capture of the network data, storing the artifact chunks in an artifact chunk section of an artifact store for linearly storing the artifact chunks in an order as received, storing artifact chunk information (ACI) for each artifact chunk in a chunk table section of the artifact store, the ACI identifying the artifact chunk, identifying another artifact chunk of the file, and identifying a location of the artifact chunk in the artifact chunk section, and storing a unique artifact locator identifier (ALI) that uniquely identifies a storage location within the artifact store of an artifact chunk associated with file, wherein the ALI and ACI associated with the artifact chunks are configured to locate each artifact chunk associated with the file that is stored in the artifact store for future reconstruction of the file.