Efficient resolution method and system for results of a taint analysis

Abstract

The present application belongs to the technical field of program analysis, and relates to a high-efficiency analysis method and system for a result of a pollution analysis. The method comprises the following steps: receiving and verifying an input propagation process file and a node relationship file; constructing a mapping relationship from a source node to a target node for the node relationship file and extracting instruction numbers, generating a source node-target node mapping file and an index file according to the mapping relationship from the source node to the target node, and generating an instruction number file according to the instruction numbers; performing segmented processing on the propagation process file to obtain a segmented file and a segmented information file; and locating and querying pollution propagation information of any instruction number or index position by using the files. The present application realizes high-efficiency processing and fast querying of a large-scale pollution analysis result file, solves the problems of excessively large file size, low querying efficiency and excessive memory occupation in the prior art, and significantly improves the analysis and querying capacity for pollution analysis result data.

Description

Technical Field

[0001] This invention belongs to the field of program analysis technology, specifically relating to an efficient parsing method and system for taint analysis results. Background Technology

[0002] Taint analysis is a dynamic program analysis technique used to track the flow of tainted data (taint sources) during program execution. This technique marks specific input data as taint sources during execution and dynamically tracks the propagation trajectory of this tainted data during instruction execution, memory read / write operations, register operations, etc., recording the propagation relationship of tainted data from the source instruction to the target instruction, as well as the execution state information such as memory addresses and registers affected by the tainted data. By constructing instruction-level propagation graphs and byte-level propagation process records, this technique can accurately locate the propagation path of tainted data in a program, thus enabling applications in security research scenarios such as vulnerability detection, security analysis, and data flow tracing.

[0003] The taint propagation file is one of the result files generated by taint analysis technology, used to record the byte-level propagation trajectory of taint data during program execution. This file records the propagation time of each taint line by line. Each line includes fields such as the target node instruction number, thread number, associated taint source offset and length, tainted memory address, source node instruction number, target node instruction address, and instruction content. These fields are organized using specific delimiters. Parsing this file allows tracing the propagation path of taint data from the source node to the target node, identifying which memory addresses are tainted, the specific location of the taint source, and the instruction information involved in the propagation process. Because it records all taint propagation events during program execution, this file is typically quite large, potentially exceeding 10GB, thus requiring efficient parsing and storage strategies.

[0004] The node relationship file is one of the result files generated by taint analysis, used to record the taint propagation relationships between instructions during program execution. This file is stored in binary format, with each line a fixed 16 bytes. The first 8 bytes represent the target node instruction number, and the last 8 bytes represent the source node instruction number. This compact storage method records the taint propagation process from the source node instruction to the target node instruction. By establishing a directed propagation graph between instructions, this file supports quick querying of all target nodes corresponding to a specified source node, thereby achieving efficient tracing and analysis of taint propagation paths.

[0005] In existing technologies, methods for parsing taint analysis results are primarily used to process large-scale propagation process files and node relationship files generated by dynamic taint analysis tools, supporting applications such as taint analysis path queries, instruction statistics, and function analysis. These methods support subsequent query and analysis operations by parsing and indexing taint analysis data. Existing technologies typically rely on sequential scanning or simple indexing mechanisms, but they have significant limitations when handling extremely large files or high-frequency query scenarios. Typical existing implementations include: 1) Memory parsing method based on full file loading. This method loads the entire taint propagation result file into memory for parsing and index construction. By mapping the file to the virtual address space, the operating system's page scheduling mechanism is used to achieve on-demand loading. Taking advantage of the efficiency of memory access, random access and fast query are supported. In the implementation process, the propagation process file and node relationship file are first fully loaded into memory to build a hash table or tree index structure. This method has obvious defects: (1) huge memory consumption. When the file size reaches more than 10GB, even if memory mapping is used, it will lead to excessive virtual memory consumption, which may trigger insufficient system memory or frequent page swapping, seriously affecting system performance; (2) long index construction time. For large-scale files, it is necessary to traverse all records to build the index, and the initialization time is too long; (3) it cannot effectively handle the situation where the file size exceeds the available memory, which limits the application scope of this method.

[0006] 2) Sequential scanning-based streaming parsing method. This method uses streaming processing to read file content line by line or block by block, without loading the entire file into memory. By sequentially reading the file content, key information is extracted using regular expressions or string matching, and the data structure being processed is maintained in memory. This method has a small memory footprint and can handle files of any size, completing statistical or simple query operations in a single traversal. This method has obvious drawbacks: (1) Low query efficiency, as each query requires rescanning the entire file, which cannot meet the real-time requirements for high-frequency query scenarios; (2) Inability to support random access, as when querying information with a specific instruction number, the scan must start from the beginning until the target record is found; (3) High overhead of repeated parsing, as multiple queries on the same file require repeated parsing operations, wasting computational resources.

[0007] 3) Query method based on simple hash index. This method constructs a simple hash index during the initial processing, mapping the instruction number to the file offset. During the query, the corresponding position in the file is quickly located through the index. The index structure is constructed through a single traversal, and the index is persisted to disk. During the query, the index is loaded and the target position is located. This method can improve query efficiency to a certain extent. This method has obvious defects: (1) The index file itself may be very large, and loading the index still takes a long time; (2) It cannot effectively handle node relationship queries. When it is necessary to query all target nodes corresponding to a specified source node, the simple hash index cannot efficiently support one-to-many relationship queries, and additional traversal operations are required; (3) It lacks a segmentation processing mechanism for the file during the propagation process. When reading and parsing the target record, a large amount of irrelevant data still needs to be processed, resulting in low efficiency.

[0008] 4) Database-based parsing method. This method imports taint analysis results into a relational database and uses the database's indexing and query optimization capabilities for data retrieval. The original file is converted into database records through the ETL process, and the database's B+ tree index or hash index is used to support efficient queries. This method utilizes mature database optimization techniques to support complex query operations and transaction processing. This method has obvious drawbacks: (1) Long data import time, importing large-scale files into the database takes several hours, which cannot meet the needs of rapid deployment; (2) High storage space overhead, the database storage format usually has 2-3 times the storage space compared to the original binary format, which further increases storage costs; (3) High query latency, even with indexes, database queries still need to go through multiple steps such as SQL parsing, query optimization, and execution plan generation; (4) Lack of support for fast conversion between instruction numbers and indexes, which cannot efficiently support query methods based on index positions.

[0009] In summary, current methods for taint analysis parsing, whether based on full file loading in memory, sequential scanning in streaming, simple hash indexing, or database storage, can only process taint analysis result files to a certain extent. They suffer from high memory consumption, low query efficiency, and inability to effectively handle extremely large files or high-frequency queries. These problems hinder the efficient and real-time processing of large-scale taint analysis results, rendering taint analysis tools less applicable in real-world security analysis scenarios. This invention aims to address these practical problems by providing an efficient parsing method for taint analysis results. Summary of the Invention

[0010] The technical problem to be solved by this invention is as follows: 1) Efficient processing and memory optimization of large-scale files: (1) How to construct a parsing method that can efficiently process ultra-large-scale taint analysis result files based on memory mapping and segmentation processing mechanism, avoiding the problems of excessive memory occupation and system performance degradation caused by traditional full file loading method; (2) How to control the computation and I / O overhead in the file reading process within an acceptable range to support high-frequency query operations on large-scale propagation process files and node relationship files; 2) Performance optimization and index construction for high-frequency queries: (1) How to accurately construct a fast query index from the source node to the target node from the node relationship file and integrate it with the segmented information of the propagation process file to avoid query delays caused by missing indexes or low query efficiency; (2) How to handle the overhead of repeated parsing and repeated scanning in the query process to ensure that the query results at the specified instruction number or index position have fast response capability and determinism, and meet the real-time requirements in high-frequency query scenarios; 3) Intelligent segmentation and selective parsing support: (1) How to support intelligent segmentation of the transmission process file, and achieve efficient positioning based on the range of instruction numbers while ensuring that records with the same instruction number are not split, thus avoiding performance bottlenecks caused by the query process; (2) How to perform efficient selective parsing during the segmentation process, and only perform regularized parsing on key lines when necessary to extract instruction numbers, so as to ensure that the segmentation accuracy can meet the data integrity requirements without introducing excessive parsing time overhead; 4) Unified support for dual query methods: (1) How to support unified processing of two query methods, namely by instruction number and by instruction index, and realize the fast conversion between number and index through the instruction number file generated by preprocessing, so as to avoid the complexity and inconsistency of the query interface; (2) How to efficiently integrate the segment information of the propagation process file and the index information of the node relationship file during the query process, so as to ensure the integrity and accuracy of the query results, while maintaining the efficiency of the query operation.

[0011] The technical solution adopted in this invention is as follows: An efficient method for parsing stain analysis results includes the following steps: Receive and verify the input propagation process file and node relationship file; Construct a mapping relationship from source nodes to target nodes in the node relationship file and extract instruction numbers. Generate a source node-target node mapping file and an index file based on the mapping relationship from source nodes to target nodes, and generate an instruction number file based on the instruction numbers. The transmission process file is segmented to obtain segmented files and segmented information files; Using source-target node mapping files, index files, instruction number files, segment files, and segment information files, locate and query taint propagation information at any instruction number or index position.

[0012] Furthermore, the verification includes verifying the format and integrity of the propagation process file and the node relationship file.

[0013] Furthermore, the source node-target node mapping file includes source node IDs, the number of target nodes, and a list of target nodes; the index file records the correspondence between source node IDs and file offsets, and sorts them by source node IDs, supporting quick location of the target node list through binary search; the segment information file records the range of instruction numbers contained in the segmented file; the instruction number file supports obtaining the corresponding instruction number based on the index position.

[0014] Furthermore, the segmentation of the propagation process file includes: The propagation process file is segmented according to the user-specified checkpoint-count parameter. The file is read line by line starting from the beginning. When the checkpoint-count line is reached, the instruction number of the current line is extracted using a regular expression, and the instruction number of the next line is checked. If the instruction number of the next line is the same as the current line, continue writing the subsequent lines to the current segment file; if they are different or there is no next line, close the current segment file and create a new segment file; record the range of instruction numbers contained in each segment file in the segment information file.

[0015] Furthermore, the query includes: Parse the query parameters and determine the query method. If the query is performed by index, the corresponding instruction number is directly read from the instruction number file according to the index position. If the query is performed by instruction number, binary search is used to locate the corresponding index position in the instruction number file, realizing bidirectional conversion between instruction number and index position. Load the segment information file, quickly locate the segment file containing the target instruction based on the range of the target instruction number, load the relevant segment file, parse the records in the segment file line by line, determine whether the current record belongs to the query target, and extract the taint propagation information; Based on the source node information, the target node list corresponding to the source node is located using a binary search through the index file, and the target node information is loaded to realize node relationship query.

[0016] Furthermore, the above method employs a multi-level caching mechanism to reduce redundant loading, which includes an instruction number list cache and a segmentation information cache.

[0017] A high-efficiency parsing system for stain analysis results, comprising: The file input module is used to receive and verify the input propagation process file and node relationship file; The preprocessing module is used to construct the mapping relationship from source nodes to target nodes in the node relationship file and extract the instruction number. Based on the mapping relationship from source nodes to target nodes, it generates a source node-target node mapping file and an index file, and generates an instruction number file based on the instruction number. The propagation process file is segmented to obtain a segment file and a segment information file. The query processing module is used to locate and query taint propagation information at any instruction number or index position by utilizing the source node-target node mapping file, index file, instruction number file, segment file, and segment information file. The results output module is used to output the query results.

[0018] Furthermore, the system also includes a cache optimization module for reducing redundant loading by employing a multi-level caching mechanism, which includes an instruction number list cache and a segmentation information cache.

[0019] This invention achieves efficient processing and rapid querying of large-scale taint analysis result files through intelligent segmented storage, dual index construction, and on-demand loading mechanisms, solving the problems of excessively large file sizes, low query efficiency, and excessive memory consumption in existing technologies. This method significantly improves the ability to parse and query taint analysis result data.

[0020] The following section elaborates on the beneficial effects of the present invention, starting from the key aspects of the technical solution: Key Point 1: A propagation process file organization strategy based on intelligent segmentation was designed and implemented. By segmenting the propagation process file according to the instruction number range and establishing a segment information index, combined with a selective parsing mechanism, the data location speed and response efficiency during queries are significantly improved. Traditional methods require a full file scan or loading the entire file into memory when processing propagation process files larger than 10GB, resulting in excessively long query times or high memory consumption. This invention uses an intelligent segmentation mechanism to divide the propagation process file into multiple smaller segment files according to the instruction number range, with each segment file recording the instruction number range it contains. During queries, the system can quickly locate the segment file containing the target instruction based on the range information of the target instruction number, requiring only the loading and parsing of the relevant segment file, avoiding the overhead of a full file scan. At the same time, the system adopts a selective parsing strategy, extracting instruction numbers only at segment boundaries and when necessary, avoiding full regular expression parsing of every line, further reducing the time overhead in the preprocessing and querying processes. When processing 10GB-sized propagation process files, this method reduces query response time from tens of minutes to a few seconds, improving query efficiency by two orders of magnitude.

[0021] Key Point 2: A dual index construction mechanism was designed and implemented. By establishing a mapping index from source nodes to target nodes and an instruction number file, the problem of low efficiency in node relationship queries and difficulty in converting instruction numbers to index positions was effectively solved. Traditional methods require traversing the entire node relationship file or building a simple hash index when processing node relationship queries, which cannot efficiently support one-to-many relationship queries. This invention constructs a mapping file from source nodes to target nodes in the preprocessing stage and generates an index file (source node ID + file offset, sorted by source node ID), which supports fast location of all target nodes corresponding to a specific source node through binary search, achieving a query operation with O(log n) time complexity. At the same time, the system extracts all unique instruction numbers from the node relationship file and generates an instruction number file (each instruction number occupies 8 bytes and is arranged sequentially), which supports fast retrieval of the corresponding instruction number based on the index position, achieving bidirectional conversion between instruction number and index position with O(1) or O(log n) time complexity. When processing a 500MB node relationship file, this method reduces the node relationship query time from several seconds in traditional methods to the millisecond level, significantly improving query performance.

[0022] Key Point 3: A complete preprocessing and query separation architecture was designed and implemented. The preprocessing stage builds the index and segments the data for storage, while the query stage utilizes the preprocessing results for rapid location, achieving a highly efficient model of "preprocess once, query multiple times." Traditional methods require reprocessing the original file for each query, failing to utilize historical processing results and leading to low query efficiency. This invention, through its preprocessing and query separation architecture, completes the index building of node relationship files and the segmentation processing of propagation process files in one go during the preprocessing stage, generating reusable index and segment files. The query stage directly utilizes the preprocessing results, avoiding repeated file parsing and index building operations. In scenarios involving multiple queries on large-scale files, this method keeps preprocessing time to the minute level, with each subsequent query taking only a few seconds, achieving a highly efficient model of "preprocess once, query multiple times," significantly improving the overall system efficiency. Attached Figure Description

[0023] Figure 1 This is a flowchart of the steps of the efficient analysis method for stain analysis results of the present invention.

[0024] Figure 2 This is a schematic diagram of the structure of the efficient analysis system for stain analysis results of the present invention. Detailed Implementation

[0025] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to specific embodiments and accompanying drawings.

[0026] This invention discloses an efficient parsing method for taint analysis results. The scheme revolves around a final path of "preprocessing index construction + intelligent segmented storage + on-demand loading query," achieving fast and low-overhead parsing and querying of ultra-large-scale taint analysis result files. The scheme first preprocesses the node relationship file and propagation process file to construct an efficient index structure and segmented storage mechanism. Then, it performs fast queries based on memory mapping and on-demand loading mechanisms, supporting both query methods by instruction number and by index. The innovation of this scheme lies in: 1) A "smart segmentation" mechanism for processing propagation process files was designed and implemented. Instruction number information is directly extracted from the propagation process files to form a segmented storage structure based on the instruction number range. Each segment file records the range of instruction numbers it contains. During segmentation, it ensures that records with the same instruction number are not separated into different files, guaranteeing data integrity. Initial segmentation is performed according to the checkpoint-count parameter (the number of records per segment). Instruction number changes are checked at segment boundaries, and regular expression parsing is performed only when necessary to extract the instruction number, avoiding a complete parsing operation for every single line.

[0027] 2) A node relationship query structure with a 'dual index' was designed and implemented. A mapping relationship from source nodes to target nodes is constructed by analyzing the node relationship file, and binary format data of the source node ID, the number of target nodes, and the target node list is stored. First, a mapping file from source nodes to target nodes is generated, and an index file is constructed to record the correspondence between the source node ID and the file offset. The index file is sorted by the source node ID, supporting fast location of the target node list through binary search.

[0028] 3) A fast conversion mechanism between instruction number and index was designed and implemented. During the preprocessing stage, all unique instruction numbers are extracted from the node relationship file to generate an instruction number file (each instruction number occupies 8 bytes and is arranged sequentially), supporting fast retrieval of the corresponding instruction number based on the index position. During queries, binary search is used to achieve fast conversion between instruction number and index position, ensuring unified processing for both instruction number-based and index-based queries, avoiding complexity and inconsistency in the query interface.

[0029] 4) An efficient I / O path with on-demand loading and memory mapping was designed and implemented. The segmented information of the propagation process file is used to construct a JSON-formatted checkpoint information file and cached. During querying, only the segmented file containing the target instruction number is loaded, reducing the I / O overhead of large-scale data. mmap memory mapping technology is used to directly access the file content, avoiding loading the entire file into memory. Meanwhile, unordered_map and unordered_set are used to achieve fast lookup with O(1) time complexity, improving query throughput and response efficiency.

[0030] 5) The system is designed and implemented for fast deterministic lookup of arbitrary instruction numbers. During the query, the corresponding segment file is quickly located based on the instruction range, and only records in the relevant segment file are parsed and loaded, avoiding full file scanning. The system supports real-time response requirements in high-frequency scenarios through a three-level optimization mechanism of segment location, index lookup, and memory-mapped reading.

[0031] This invention proposes an efficient parsing method for taint analysis results. This method combines preprocessing index construction with intelligent segmented storage, enabling rapid retrieval of detailed taint analysis information at specified instruction numbers or index positions, supporting efficient processing of large-scale taint analysis result files. The main steps of this method are as follows: Figure 1 As shown, it includes the following steps: Step S1: Receive and verify the input propagation process file and node relationship file.

[0032] The verification process includes verifying the format and integrity of the propagation process files and node relationship files.

[0033] Step S2: Construct a mapping relationship from source nodes to target nodes in the node relationship file and extract instruction numbers. Generate a source node-target node mapping file and an index file based on the mapping relationship from source nodes to target nodes, and generate an instruction number file based on the instruction numbers.

[0034] The source-target node mapping file contains the source node ID, the number of target nodes, and a list of target nodes. The index file records the correspondence between source node IDs and file offsets, sorted by source node ID, and supports quick location of the target node list using binary search. The segment information file records the range of instruction numbers contained in the segment file. The instruction number file supports retrieving the corresponding instruction number based on the index position.

[0035] Step S3: Segment the propagation process file to obtain segmented files and segmented information files.

[0036] The steps for segmenting the transmission process files include: The propagation process file is segmented according to the user-specified checkpoint-count parameter. The file is read line by line starting from the beginning. When the checkpoint-count line is reached, the instruction number of the current line is extracted using a regular expression, and the instruction number of the next line is checked. If the instruction number of the next line is the same as the current line, continue writing the subsequent lines to the current segment file; if they are different or there is no next line, close the current segment file and create a new segment file; record the range of instruction numbers contained in each segment file in the segment information file.

[0037] Step S4: Using the source node-target node mapping file, index file, instruction number file, segment file, and segment information file, locate and query taint propagation information for any instruction number or index position.

[0038] The specific query process for this step includes: Parse the query parameters and determine the query method. If the query is performed by index, the corresponding instruction number is directly read from the instruction number file according to the index position. If the query is performed by instruction number, binary search is used to locate the corresponding index position in the instruction number file, realizing bidirectional conversion between instruction number and index position. Load the segment information file, quickly locate the segment file containing the target instruction based on the range of the target instruction number, load the relevant segment file, parse the records in the segment file line by line, determine whether the current record belongs to the query target, and extract the taint propagation information; Based on the source node information, the target node list corresponding to the source node is located using a binary search through the index file, and the target node information is loaded to realize node relationship query.

[0039] One embodiment of the present invention provides an efficient method for analyzing stain analysis results, the specific processing flow of which includes the following steps: Step 1: File Input and Validation. Receive the input propagation process file and node relationship file, and verify the file format and integrity. For the node relationship file, check if the file size is a multiple of 16 bytes. For the propagation process file, verify if the file format meets expectations.

[0040] Step 2: Node Relationship File Preprocessing. The node relationship file is read using mmap memory mapping technology, avoiding loading the entire file into memory. All records are scanned to construct the mapping relationship from source nodes to target nodes, while extracting all unique instruction numbers. The mapping relationship is written to a binary format source-target node mapping file (source node ID + number of target nodes + list of target nodes), generating an index file (source node ID + file offset, sorted by source node ID), supporting fast location via binary search. The file offset is used to find the list of target nodes in the source-target node mapping file based on the source node ID. The extracted unique instruction numbers are written to an instruction number file (each instruction number occupies 8 bytes and is arranged sequentially).

[0041] Step 3: Intelligent Segmentation of the Propagation Process File. The propagation process file is segmented according to the user-specified `checkpoint-count` parameter. The file is read line by line from the beginning. At each `checkpoint-count` line, a regular expression is used to extract the instruction number of the current line, and the instruction number of the next line is checked. If the instruction number of the next line is the same as the current line, the subsequent lines are written to the current segment file, and the counter is reset; if they are different or there is no next line, the current segment file is closed, and a new segment file is created. Instruction number parsing is performed only at segment boundaries and when necessary, avoiding full regular expression parsing for every line. For each segment file, the range of instruction numbers it contains is recorded, and the instruction number range of each segment file is written to a segment information file (segment information JSON file).

[0042] Step 4: Query Parameter Parsing and Conversion. When querying detailed information for a specific instruction, the query parameters are first parsed to determine the query method. If querying by index, the corresponding instruction number is directly read from the instruction number file based on the index position. If querying by instruction number, binary search is used to locate the corresponding index position in the instruction number file, achieving bidirectional conversion between instruction number and index position.

[0043] Step 5: Segment Location and Data Extraction. Load the segment information JSON file. Based on the range information of the target instruction number, quickly locate the segment file containing the target instruction. Use mmap memory mapping technology to load the relevant segment files, avoiding full file scanning. Parse the records in the segment files line by line to quickly determine whether the current record belongs to the query target and extract taint propagation information.

[0044] Step Six: Node Relationship Query and Result Integration. Based on the extracted source node information, use the index file to locate the target node list corresponding to the source node through binary search, and load the target node information. Integrate the propagation process data and node relationship data to generate complete query results. The propagation process data refers to the propagation process results within a specified range obtained by searching segmented propagation process files when a query is needed.

[0045] Through the above steps, this method achieves efficient preprocessing and query processing when handling large-scale taint analysis result files, which is significantly better than the traditional full file scanning method, greatly reducing query time and improving system response performance.

[0046] Another embodiment of the present invention provides a high-efficiency parsing system for taint analysis results, which efficiently processes and queries the result files generated by taint analysis. The system is as follows: Figure 2 As shown, the system 20 mainly includes the following modules: File input module 21 is used to receive and verify the input propagation process file and node relationship file; The preprocessing module 22 is used to construct the mapping relationship from source nodes to target nodes in the node relationship file and extract the instruction number; generate the source node-target node mapping file and index file according to the mapping relationship from source nodes to target nodes; generate the instruction number file according to the instruction number; and perform segmentation processing on the propagation process file to obtain the segment file and segment information file. The query processing module 23 is used to locate and query taint propagation information at any instruction number or index position by utilizing the source node-target node mapping file, index file, instruction number file, segment file and segment information file; Result output module 24 is used to output query results.

[0047] In another embodiment of the present invention, the system further includes a cache optimization module 25, which is used to reduce repeated loading by adopting a multi-level caching mechanism, including an instruction number list cache and a segment information cache.

[0048] Another embodiment of the present invention provides a high-efficiency analysis system for stain analysis results, and the processing procedures of each module are described below: The file input module is responsible for receiving and verifying the input taint analysis result files, including the propagation process file (each line represents the taint propagation process at the byte level) and the node relationship file (each line represents the taint propagation relationship between instructions), and supports input processing of large-scale files.

[0049] The preprocessing module comprises two core functions: node relationship preprocessing and propagation process file segmentation, referred to as the node relationship file preprocessing submodule and the propagation process file segmentation submodule, respectively. The node relationship file preprocessing submodule parses the node relationship file, establishes the mapping relationship from source nodes to target nodes, constructs a binary source node-target node mapping file and an index file, and extracts all unique instruction numbers to generate an instruction number file. The propagation process file segmentation submodule, based on the user-specified `checkpoint-count` parameter, intelligently segments the propagation process file, ensuring that records with the same instruction number are not split into different files, generating segment files and corresponding segment information files. The segment information file records the range of instruction numbers contained in the segment files. The preprocessing module also includes an index building submodule, responsible for generating and managing all index structures, supporting fast query and location.

[0050] The query processing module quickly locates and extracts relevant information from segmented files based on the instruction number or index position of the target instruction. The query parameter parsing submodule supports both instruction number and index-based queries, enabling bidirectional conversion between instruction numbers and index positions. The segment location and data extraction submodule quickly locates the relevant segmented files based on the instruction number range, loads and parses the propagation process records using memory mapping technology. The node relationship query submodule quickly queries all target nodes corresponding to the source node through the index file. Finally, it integrates propagation process data and node relationship data to generate complete query results.

[0051] Cache optimization module: Employs a multi-level caching mechanism to reduce redundant loading, including instruction number list caching and segment information caching. The system checks file modification time and only reloads when files are updated, avoiding redundant I / O operations and improving query performance and response efficiency.

[0052] The results output module is responsible for outputting the query results in JSON format, supporting subsequent analysis and use.

[0053] The overall architecture of the above system adopts a layered design. The input is the propagation process file and the node relationship file. After processing by the preprocessing module, an efficient index structure and segmented storage are formed. The query processing module uses these indexes and segmented data to quickly locate and query taint propagation information for any instruction number or index position.

[0054] Other embodiments of the present invention: 1) Regarding the segmentation organization of files in the propagation process, not only can a fixed segmentation strategy based on the instruction number range be used for partitioning and indexing, but also a fixed segmentation strategy based on file size or a dynamic partitioning strategy based on instruction type can be adopted. These alternative methods, while maintaining the structured representation of the propagation process data, may provide a simpler implementation in specific scenarios (such as when the instruction numbers are unevenly distributed).

[0055] 2) For the bidirectional conversion mechanism between instruction numbers and index positions, it can be implemented not only through an instruction number file (each instruction number occupies 8 bytes arranged sequentially), but also through a hash table-based mapping structure or a B+ tree-based index structure. These alternative methods, while maintaining the function of converting between numbers and indexes, may provide a more flexible query interface in specific scenarios (such as when range queries are required).

[0056] 3) For selective parsing mechanisms, this can be achieved not only by extracting instruction numbers at segment boundaries and when necessary, but also by performing complete parsing on each line or by batch parsing based on pre-compiled regular expressions. These alternative methods, while maintaining parsing functionality, may provide more complete data extraction capabilities in specific scenarios (such as when all field information needs to be extracted).

[0057] It should be understood that the methods and systems disclosed in the above embodiments of the present invention can be implemented in other ways. For example, the above module division can be implemented in other ways, multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. Each step and module in the present invention can be implemented in the form of software functional units and can be stored in a computer-readable storage medium, including several instructions to cause a computer device to execute some or all of the steps of the method described in the present invention. For example, one embodiment of the present invention provides a computer device (computer, server, etc.) including a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for executing each step of the method of the present invention. For example, another embodiment of the present invention provides a computer-readable storage medium (such as ROM / RAM, disk, optical disk, etc.) storing a computer program, which, when executed by a computer, implements each step of the method of the present invention. For example, another embodiment of the present invention provides a computer program product including a computer program, which, when executed by a computer, implements the steps of the method of the present invention.

[0058] The specific embodiments of the present invention disclosed above are intended to help understand the content of the present invention and to implement it accordingly. Those skilled in the art will understand that various substitutions, changes, and modifications are possible without departing from the spirit and scope of the present invention. The present invention should not be limited to the content disclosed in the embodiments of this specification; the scope of protection of the present invention is defined by the claims.

Claims

1. A highly efficient method for analyzing stain analysis results, characterized in that, Includes the following steps: Receive and verify the input node relationship file and propagation process file; Construct a mapping relationship from source nodes to target nodes in the node relationship file and extract instruction numbers. Generate a source node-target node mapping file and an index file based on the mapping relationship from source nodes to target nodes, and generate an instruction number file based on the instruction numbers. The transmission process file is segmented to obtain segmented files and segmented information files; Using source-target node mapping files, index files, instruction number files, segment files, and segment information files, locate and query taint propagation information at any instruction number or index position.

2. The method according to claim 1, characterized in that, The verification includes verifying the format and integrity of the propagation process files and node relationship files.

3. The method according to claim 1, characterized in that, The source node-target node mapping file contains the source node ID, the number of target nodes, and a list of target nodes; the index file records the correspondence between the source node ID and the file offset, and is sorted by the source node ID, supporting quick location of the target node list through binary search; the segment information file records the range of instruction numbers contained in the segment file; the instruction number file supports obtaining the corresponding instruction number based on the index position.

4. The method according to claim 1, characterized in that, The segmentation of the transmission process file includes: The propagation process file is segmented according to the user-specified checkpoint-count parameter. The file is read line by line starting from the beginning. When the checkpoint-count line is reached, the instruction number of the current line is extracted using a regular expression, and the instruction number of the next line is checked. If the instruction number of the next line is the same as the current line, continue writing the subsequent lines to the current segment file; if they are different or there is no next line, close the current segment file and create a new segment file, recording the range of instruction numbers contained in each segment file in the segment information file.

5. The method according to claim 1, characterized in that, The query includes: Parse the query parameters and determine the query method. If the query is performed by index, the corresponding instruction number is directly read from the instruction number file according to the index position. If the query is performed by instruction number, binary search is used to locate the corresponding index position in the instruction number file, realizing bidirectional conversion between instruction number and index position. Load the segment information file, quickly locate the segment file containing the target instruction based on the range of the target instruction number, load the segment file and parse the records in the segment file line by line, determine whether the record belongs to the query target, and extract the taint propagation information; Based on the source node information, the target node list corresponding to the source node is located using the index file, and the target node information is loaded to realize node relationship query.

6. The method according to claim 1, characterized in that, A multi-level caching mechanism is adopted to reduce repeated loading. The multi-level caching mechanism includes an instruction number list cache and a segment information cache.

7. A high-efficiency analysis system for stain analysis results, characterized in that, include: The file input module is used to receive and verify the input propagation process file and node relationship file; The preprocessing module is used to construct the mapping relationship from source nodes to target nodes in the node relationship file and extract the instruction number. Based on the mapping relationship from source nodes to target nodes, it generates a source node-target node mapping file and an index file. Based on the instruction number, it generates an instruction number file. It also segments the propagation process file to obtain a segment file and a segment information file. The query processing module is used to locate and query taint propagation information at any instruction number or index position by utilizing the source node-target node mapping file, index file, instruction number file, segment file, and segment information file. The results output module is used to output the query results.

8. The system according to claim 7, characterized in that, It also includes a cache optimization module, which uses a multi-level caching mechanism to reduce repeated loading. The multi-level caching mechanism includes an instruction number list cache and a segment information cache.

9. A computer device, characterized in that, It includes a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the method of any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a computer, implements the method according to any one of claims 1 to 7.

Images