Code blood relationship tracing method and device based on multi-modal features

Abstract

The application belongs to the technical field of data analysis, and discloses a code blood relationship tracing method and device based on multi-modal features, which comprises the following steps: obtaining open source code data and extracting the data to obtain structured features, semantic features and statistical features and establish a multi-dimensional feature library; obtaining the code of a to-be-detected object, performing feature extraction based on the multi-dimensional feature library and the code to obtain code feature data of the to-be-detected object; analyzing the code feature data of the to-be-detected object according to a neural network to obtain a blood relationship heterogeneous graph; matching the multi-dimensional feature library, the code feature data of the to-be-detected object and the blood relationship heterogeneous graph to obtain target code tracing matching relationship data set; and analyzing the target code tracing matching relationship data set to obtain a blood relationship multi-dimensional feature graph. The application can realize high-precision identification of obfuscated code and deep analysis of nested dependency relationships.

Description

Technical Field

[0001] This application relates to the field of data analysis technology, and in particular to a method and apparatus for tracing code lineage based on multimodal features. Background Technology

[0002] With the increasing application of open source software / components in modern software system development, code reuse efficiency has been significantly improved, and open source software has become an important foundation for modern software system development.

[0003] However, the widespread use of open-source software has also introduced complex security risks. Furthermore, existing open-source software tracing technologies have many shortcomings, making it difficult to meet practical application needs. Specifically, on the one hand, the multi-layered nested dependencies of open-source components make dependency relationships difficult to trace, and techniques such as code obfuscation and hardening render traditional tracing methods ineffective; existing technologies struggle to accurately identify component versions and code snippet-level reference relationships. On the other hand, existing open-source software tracing technologies mainly fall into three categories, all of which have significant shortcomings: benchmark-based comparison methods are costly to maintain and struggle to handle obfuscated code; signature-based tracing methods, while possessing strong anti-obfuscation capabilities, are highly dependent on programming languages ​​and compilation environments; and machine learning-based methods lack generalization ability and struggle to handle complex nested reference relationships. In addition, existing technologies generally lack an immutable evidence storage mechanism for tracing results, making it difficult to meet the compliance requirements of software supply chain auditing.

[0004] Therefore, how to achieve high-precision identification of obfuscated code and deep analysis of nested dependencies has become a problem to be solved. Summary of the Invention

[0005] This application provides a method and apparatus for tracing code lineage based on multimodal features. It can acquire open-source code data and construct a multi-dimensional feature library to achieve comprehensive integration of open-source code features, providing rich feature support for code tracing and solving the problems of single feature dimensions and insufficient supporting evidence in traditional tracing methods. After acquiring the code of the object to be detected, feature extraction is performed based on the multi-dimensional feature library, which can accurately capture the core feature information of the code of the object to be detected, ensuring that the extracted code feature data is highly compatible with the tracing requirements and improving the targeting of subsequent tracing analysis. A neural network is used to analyze the code feature data to obtain a lineage heterogeneous graph, which can clearly present the complex lineage relationship between the code of the object to be detected and the open-source code, breaking the limitation of traditional analysis methods that cannot intuitively display complex lineage relationships. Combining the code feature data of the multi-dimensional feature library and the lineage heterogeneous graph for matching yields a target code tracing matching relationship dataset, realizing multi-dimensional tracing matching, significantly improving the accuracy of code tracing matching, reducing tracing errors, and ensuring that the matching results can truly reflect the lineage relationship between codes. Analyzing the target code tracing and matching relationship dataset yields a multi-dimensional feature map of lineage relationships. This map integrates code lineage relationships with features of various dimensions, not only visually demonstrating the distribution patterns of code lineage relationships but also clearly reflecting the impact of different features on lineage relationships. This provides strong support for the verification, analysis, and subsequent code optimization of code tracing results, further reducing the difficulty and cost of code tracing.

[0006] In a first aspect, embodiments of this application provide a code lineage tracing method based on multimodal features, the method comprising: Obtain open-source code data and extract it to obtain structured features, semantic features, and statistical features, and build a multi-dimensional feature library; The code of the object to be detected is obtained, and feature extraction is performed based on a multi-dimensional feature library and the code to obtain the code feature data of the object to be detected. Based on the analysis of the code feature data of the object to be detected by the neural network, a bloodline heterogeneity map is obtained; Matching is performed based on a multi-dimensional feature library, code feature data of the object to be detected, and a lineage heterogeneity graph to obtain a target code source tracing matching relationship dataset. The target code source matching relationship dataset was analyzed to obtain a multidimensional feature map of blood relations.

[0007] Furthermore, the structural features include abstract syntax tree features, control flow graph features, and function call graph features; the semantic features include natural semantic features and program behavior semantic features based on the understanding of the large code model; and the statistical features include code metric features and constant fingerprint features.

[0008] Furthermore, open-source code data is acquired and extracted to obtain structured features, including: Obtain open-source code data and perform language parsing to obtain the first data; The second data is obtained by extracting the node type sequence based on the first data; The third data is obtained by extracting the structural path based on the second data; The fourth data is obtained by performing subtree pattern mining based on the third data. Vector encoding is performed based on the fourth data to obtain abstract syntax tree features as structured features.

[0009] Furthermore, open-source code data is acquired and extracted to obtain semantic features, including: Obtain open-source code data and perform symbolic initialization to obtain the first parameter; The second parameter is obtained by extracting path constraints based on the first parameter. The third parameter is obtained by simplifying the operation based on the second parameter. Based on the third parameter, a behavior summarization operation is performed to obtain semantic features of program behavior.

[0010] Furthermore, feature extraction is performed based on a multi-dimensional feature library and code to obtain code feature data of the object to be detected, including: Basic block recognition is performed based on a multi-dimensional feature library and code to obtain the first field; Based on the first field, construct the inter-block control flow edge to obtain the second field; The third field is obtained by analyzing the block's termination instruction type based on the second field. Based on the third field, a graph structure is constructed and graph features are calculated to obtain the code feature data of the object to be detected.

[0011] Furthermore, lineage heterogeneous graphs include package-level graphs, module-level graphs, class-level graphs, and function-level graphs.

[0012] Furthermore, based on a multi-dimensional feature library, the code feature data of the target object, and a lineage heterogeneity graph, a target code source tracing matching dataset is obtained, including: Lightweight statistical fingerprinting is used to quickly match the first candidate set based on a multi-dimensional feature library, code feature data of the object to be detected, and heterogeneous lineage graph. Based on the first candidate set, a Bloom filter is used for pre-screening to obtain the second candidate set; Cache hit optimization is performed based on the second candidate set to obtain the target code source matching relationship dataset.

[0013] Secondly, embodiments of this application provide a code lineage tracing device based on multimodal features, the device comprising: The benchmark feature library construction module is used to acquire and extract open-source code data to obtain structured features, semantic features, and statistical features, and to build a multi-dimensional feature library. The code feature extraction module is used to obtain the code of the object to be detected. Based on a multi-dimensional feature library and the code, feature extraction is performed to obtain the code feature data of the object to be detected. The neural network analysis module is used to analyze the code feature data of the object to be detected based on the neural network to obtain a lineage heterogeneity graph; The source matching module is used to match based on a multi-dimensional feature library, code feature data of the object to be detected, and a lineage heterogeneity graph to obtain a target code source matching relationship dataset; The graph generation module is used to analyze the target code source matching relationship dataset to obtain a multi-dimensional feature map of blood relations.

[0014] Thirdly, embodiments of this application provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it performs the steps of a code lineage tracing method based on multimodal features as described in any of the above embodiments.

[0015] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of a code lineage tracing method based on multimodal features as described in any of the above embodiments.

[0016] In summary, compared with the prior art, the beneficial effects of the technical solution provided in this application include at least the following: This application provides a code lineage tracing method based on multimodal features. By acquiring open-source code data and constructing a multi-dimensional feature library, it achieves comprehensive integration of open-source code features, providing rich feature support for code tracing and solving the problems of single feature dimensions and insufficient supporting evidence in traditional tracing methods. After acquiring the code of the object to be detected, feature extraction is performed based on the multi-dimensional feature library, accurately capturing the core feature information of the code and ensuring that the extracted code feature data is highly compatible with tracing requirements, thus improving the targeting of subsequent tracing analysis. A neural network is used to analyze the code feature data to obtain a lineage heterogeneous graph, which clearly presents the complex lineage relationship between the code of the object to be detected and the open-source code, breaking the limitation of traditional analysis methods that cannot intuitively display complex lineage relationships. Combining the multi-dimensional feature library code feature data and the lineage heterogeneous graph for matching yields a target code tracing matching relationship dataset, realizing multi-dimensional tracing matching, significantly improving the accuracy of code tracing matching, reducing tracing errors, and ensuring that the matching results truly reflect the lineage relationship between codes. Analyzing the target code tracing and matching relationship dataset yields a multi-dimensional feature map of lineage relationships. This map integrates code lineage relationships with features of various dimensions, not only visually demonstrating the distribution patterns of code lineage relationships but also clearly reflecting the impact of different features on lineage relationships. This provides strong support for the verification, analysis, and subsequent code optimization of code tracing results, further reducing the difficulty and cost of code tracing. Attached Figure Description

[0017] Figure 1 A flowchart illustrating a code lineage tracing method based on multimodal features, provided as an exemplary embodiment of this application.

[0018] Figure 2 This is a structural diagram of a code lineage tracing device based on multimodal features, provided as an exemplary embodiment of this application. Detailed Implementation

[0019] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0020] Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] Please see Figure 1 This application provides a code lineage tracing method based on multimodal features, which specifically includes the following steps: Step S1: Obtain open-source code data and extract it to obtain structured features, semantic features, and statistical features, and establish a multi-dimensional feature library.

[0022] In some embodiments, structural features include abstract syntax tree features, control flow graph features, and function call graph features; semantic features include natural semantic features and program behavior semantic features based on the understanding of the large code model; statistical features include code metric features and constant fingerprint features.

[0023] In some embodiments, open-source code data is acquired and extracted to obtain structured features, including: Obtain open-source code data and perform language parsing to obtain the first data; The second data is obtained by extracting the node type sequence based on the first data; The third data is obtained by extracting the structural path based on the second data; The fourth data is obtained by performing subtree pattern mining based on the third data. Vector encoding is performed based on the fourth data to obtain abstract syntax tree features as structured features.

[0024] In some embodiments, open-source code data is acquired and extracted to obtain semantic features, including: Obtain open-source code data and perform symbolic initialization to obtain the first parameter; The second parameter is obtained by extracting path constraints based on the first parameter. The third parameter is obtained by simplifying the operation based on the second parameter. Based on the third parameter, a behavior summarization operation is performed to obtain semantic features of program behavior.

[0025] In one feasible implementation, the structured features include abstract syntax tree features, control flow graph features, and function call graph features; the semantic features include natural semantic features and program behavior semantic features based on the understanding of the large code model; and the statistical features include code metric features and constant fingerprint features. Specifically, the input open-source code repository, complete source code project, and compiled source code products are processed by structured feature extraction, and the output includes abstract syntax tree features, control flow graph features, and function call graph feature data; after semantic feature extraction, the output includes natural semantic features and program behavior semantic feature data; and after statistical feature extraction, the output includes code metric features and constant fingerprint features, etc., as code baseline feature data.

[0026] In one feasible implementation, the extraction and construction process of the abstract syntax tree (AST) features in the structured features is as follows: First, language-specific parsing of the source code of the code repository; second, extraction of node type sequences; third, extraction of structural paths; fourth, subtree pattern mining; and fifth, vector encoding. After the aforementioned steps, the abstract syntax tree feature data is output, including basic syntax structure feature data (node ​​sequence data such as node type, node depth, and node position), syntax tree structure feature data (syntactic structure data such as total number of nodes, node depth, average depth, number of leaf nodes, and branch nodes), execution path feature data (root-to-leaf path, node pair path, and jump branch path data), and code structure feature data (control structure features, function / method feature data, etc.). The key innovation of abstract syntax tree feature extraction is the use of AST normalization processing that resists syntax mutation. For differences in code style (different naming conventions, comment differences), an AST normalization algorithm is used, specifically including identifier anonymization to eliminate naming differences; literal generalization to eliminate constant differences; type standardization to unify type representation; and comment removal to eliminate interference noise.

[0027] In one feasible implementation, the control flow graph feature extraction and construction process in structured features is as follows: First, basic block partitioning, dividing the code into sequentially executed basic blocks; second, edge feature extraction, recording the transition conditions between code blocks (unconditional jumps, conditional true / false branches, exception handling, etc.); third, path-sensitive encoding, using symbolic execution to extract path constraints; fourth, generating CFG feature fingerprints. After the aforementioned steps, control flow graph feature data is output, including statement block features, statement block entry and exit features, edge features, nested loop branch structure features, exception control features, etc. The key innovation of control flow graph feature extraction is generating feature fingerprints by analyzing control flow complexity, loop structure patterns (nesting depth, loop type), exception handling structure, etc.

[0028] In one feasible implementation, the function call graph features in the structured features are extracted using a cross-process analysis strategy. The construction process is as follows: First, static analysis, constructing an approximate function call graph through class hierarchy analysis and fast type analysis; second, dynamic analysis, collecting actual call relationships based on runtime code instrumentation; third, hybrid refinement: refining the virtual call edges from the static analysis using dynamic results. After the aforementioned steps, function call graph feature data is output, including function identifier attribute features, function size (function code lines, cyclomatic complexity, fan-in, fan-out, and number of parameters, etc.), function call relationship features, and function functional attribute features. The core of function call graph feature extraction lies in using a construction method that combines static and dynamic analysis, while further performing hybrid refinement processing to improve the accuracy of function call graph feature representation.

[0029] In one feasible implementation, the semantic features are natural semantic features based on the understanding of a large code model. The semantic feature extraction process is as follows: First, general code representation learning is performed using a two-stage pre-training-fine-tuning strategy on open-source code. Second, source tracing task fine-tuning is conducted to construct comparative learning training data. The training data includes positive sample data, negative sample data, and specific sample data. Positive sample data consists of different versions / variants of the same open-source component, with the goal of minimizing the vector distance. Negative sample data consists of components with similar functions but different origins, with the goal of maximizing the vector distance. Specific sample data consists of similar components (easily confused) within the same ecosystem, used to clarify boundary distance constraints.

[0030] In one feasible implementation, the program behavior semantic features in the semantic features are mainly extracted through symbolic execution. The extraction and construction process is as follows: First, initialize the symbolic execution engine; second, extract path constraints; third, simplify constraints, retaining key API call patterns; fourth, generate a behavior summary. After the aforementioned steps, the program behavior semantic features are output, including input sources, output sinks, data flow graphs, and relational information encoded as behavior vectors.

[0031] In one feasible implementation, the code metric feature in the statistical features is a combined metric of Halstead complexity and McCabe cyclomatic complexity. Its measurement dimensions include code size measurement, complexity measurement, Halstead metric, and coupling measurement. Specifically, the size measurement is based on AST statistics of effective lines of code and comment rate, used to quickly filter components of similar size; the complexity measurement uses CFG analysis of cyclomatic complexity to identify algorithm similarity; the Halstead metric uses lexical analysis of operator / operand types and frequencies to estimate computational difficulty and workload; and the coupling measurement uses call graph analysis of fan-in / fan-out and class coupling to determine architectural similarity.

[0032] In one feasible implementation, the constant fingerprint in the statistical features is obtained by extracting string constants, numerical constants, and array initialization values ​​from the code. String constants are stored using hashing; for privacy protection, only the constant length, prefix, and hash value are retained, and the original constant value is not stored. Numerical constants retain only their numerical precision features but are discretized to a certain numerical range. The array initialization fingerprint is used to identify the hard-coded table, generate a Bloom Filter fingerprint, and support fast similarity queries.

[0033] In one feasible implementation, the aforementioned multimodal features are fused and uniformly represented through steps such as multimodal feature alignment, adaptive feature weighting, and weight interpretability analysis. Due to the significant differences in the dimensions of different modal features, cross-modal alignment is necessary. For example, AST structure vectors, semantic embedding vectors, CFG graph vectors, statistical feature vectors, and constant fingerprints need to be aligned uniformly. Adaptive feature weighting uses an attention mechanism to dynamically adjust the weight strategy for each modality. The steps include adding modality type embeddings, injecting modality priors, cross-modal attention fusion, weighted aggregation, and returning the fused features and weights (for interpretability). Weight interpretability analysis is mainly applied to obfuscated code scenarios, automatically increasing the weights of AST structure features and CFG features to make the syntactic structure more stable; increasing the weights of semantic features based on similar functional code to capture functional similarity; and increasing the weights of import dependency patterns based on code from the same ecosystem to capture ecosystem-specific habits.

[0034] Furthermore, the multi-dimensional feature library can be maintained daily and continuously updated through an automated data acquisition pipeline. Its main steps include: triggering startup → repository discovery → scheduled incremental crawler scanning → version identification (parsing version tags, commit history, and Release information) → source code acquisition (cloning the repository, downloading Source Jar / Wheel) → dependency resolution (e.g., resolving dependencies via pom.xml / package.json / requirements.txt) → parallel feature extraction (AST parsing queue, semantic encoding queue, graph construction queue, statistical calculation queue) → quality verification (feature integrity check, outlier detection) → incremental indexing (updating the feature vector database, graph database, and relational database) → consistency verification (data consistency verification and alignment verification).

[0035] This approach effectively overcomes the limitations of single feature extraction methods by comprehensively extracting structured semantic statistical multi-dimensional features from open-source code and constructing a feature library, thereby improving the accuracy of open-source code feature representation. Structured feature extraction employs multiple optimization strategies; semantic features are combined with large-scale code model understanding and symbolic execution techniques; statistical features utilize appropriate metrics and fingerprint processing; and various features are fused to achieve a unified representation, enhancing the reliability of feature applications. An automated data acquisition pipeline enables daily maintenance and continuous updates of the feature library, reducing manual operation costs, ensuring the integrity of the feature library data, and providing strong support for open-source code-related analysis and identification.

[0036] Step S2: Obtain the code of the object to be detected, and perform feature extraction based on the multi-dimensional feature library and the code to obtain the code feature data of the object to be detected.

[0037] In some embodiments, feature extraction is performed based on a multi-dimensional feature library and code to obtain code feature data of the object to be detected, including: Basic block recognition is performed based on a multi-dimensional feature library and code to obtain the first field; Based on the first field, construct the inter-block control flow edge to obtain the second field; The third field is obtained by analyzing the block's termination instruction type based on the second field. Based on the third field, a graph structure is constructed and graph features are calculated to obtain the code feature data of the object to be detected.

[0038] In one feasible implementation, this application primarily employs a multimodal code feature extraction method, including source code-level feature extraction, deep semantic feature extraction, engineered statistical feature extraction, binary-level feature extraction, and code obfuscation-resistant feature extraction, to analyze and extract the target software code features. Finally, feature fusion and unified representation are performed. The input data for this step can be the source code of the object to be detected, or the compiled intermediate code and binary code. After multimodal code feature extraction, the output is the code feature data of the object to be detected.

[0039] In one feasible implementation, the multimodal code feature extraction method employs a multi-channel parallel pipeline design, with concurrent processing channels for different code forms, including a source code channel, a bytecode channel, a binary channel, an obfuscation adversarial channel, and a hybrid channel. The source code channel takes the original source code as input and outputs AST, CFG, and semantic embedding information through a syntax parser and semantic analyzer. The bytecode channel takes compiled bytecode as input and outputs intermediate code representation and call relationship information through bytecode decompilation and control flow reconstruction techniques. The binary channel targets binary bytecode, employing a disassembler and symbolic execution techniques to output assembly instruction flow and function boundary information. The obfuscation adversarial channel targets obfuscated / hardened code, employing symbolic execution, taint analysis, and deobfuscation to output the equivalent features of the restored code. The hybrid channel targets multi-language mixtures, using language boundary detection and code block processing to uniformly represent the code feature set.

[0040] In one feasible implementation, the source code-level feature extraction method mainly employs methods such as deep parsing of Abstract Syntax Trees (ASTs), construction and feature extraction of Control Flow Graphs (CFGs), and construction and feature extraction of Call Graphs. The deep parsing of ASTs uses a multi-language parsing engine with Tree-sitter as the basic parsing framework, supporting unified parsing of multiple programming languages. The main steps are: loading a pre-trained language-specific parser → unified parsing interface: input source code, return general AST nodes → converting to a language-independent unified AST representation → converting language-specific AST nodes to a unified format. The AST feature encoding strategy uses hierarchical encoding to capture multi-granularity structural information. By encoding the AST from bottom to top, a root node representation is generated. For leaf nodes, position information and initial state are added; for non-leaf nodes, child nodes are encoded first, child node information is aggregated, and the current node type is determined, then the entire tree is updated. AST path feature extraction uses the root-to-leaf path as a structural fingerprint to extract key paths in the AST for similarity comparison (e.g., adding the current node to the path → reaching the leaf node, recording the path → continuing depth traversal → filtering paths with high information content). The control flow graph (CFG) construction and feature extraction method mainly adopts a multi-level CFG construction approach at the function, class, and module levels. The process is as follows: First, identify basic blocks; second, construct control flow edges between blocks; third, analyze the terminal instruction types of blocks (such as true condition branches, false condition branches, loop body edges, loop exit edges, etc.); fourth, construct the graph structure; fifth, calculate graph features. Finally, CFG feature vectorization is performed for similarity matching (basic graph features, path features, loop structure features, control flow patterns, etc.). Call graph construction and feature extraction combine static and dynamic analysis to improve accuracy. Static analysis uses methods such as class hierarchy analysis, fast type analysis, and variable type analysis to analyze results, while dynamic analysis uses runtime call acquisition based on instrumentation (automatic instrumentation, execution of test cases (or production traffic replay), and construction of call graphs from execution trajectories) analysis methods.

[0041] In one feasible implementation, the semantic feature deep extraction method mainly adopts the natural semantic feature and program behavior semantic feature extraction method based on the understanding of the code big model, which is consistent with the semantic feature method described in step S1, and will not be repeated here.

[0042] In one feasible implementation, the statistical feature engineering extraction method mainly adopts statistical feature extraction methods such as code metrics and constant fingerprint features, which are consistent with the statistical feature methods described in step S1, and will not be repeated here.

[0043] In one feasible implementation, the binary-level feature extraction method uses a binary file parser to analyze binary files and extract multi-level features of the binary files, including instruction-level features, control flow features, data flow features, call features, compiler fingerprints, etc. The extracted content includes binary file format, extracted metadata, control flow, function identification and boundaries, jumps, exception handling and other information.

[0044] In one feasible implementation, the code obfuscation resistance feature extraction method primarily targets the features extracted after code obfuscation. First, code obfuscation resistance and feature recovery are performed, followed by feature extraction. Multi-dimensional deobfuscation techniques are used to detect the obfuscation type, and corresponding recovery strategies are executed to restore the original code features (such as system call sequences, API call graphs, data reference patterns, string / constant layouts, exception handling structures, resources and metadata, statistical invariance, etc.). Simultaneously, opaque predicates (constantly true / false conditions) are analyzed and eliminated, suspicious conditional branches are identified, and constantly true / false conditions are replaced with unconditional jumps. Then, based on dynamic analysis of symbolic execution, the recovered features are extracted.

[0045] In one feasible implementation, feature fusion and unified representation align the aforementioned cross-modal features of the code, standardize the feature space, and project the multimodal features into a unified space to achieve alignment between the cross-modal features and the benchmark feature library before they can be matched with the features in the benchmark feature library.

[0046] In acquiring code feature data of the target object, a multimodal code feature extraction method and a multi-channel parallel pipeline design were used to efficiently process code of different forms, achieving comprehensive feature extraction of source code, bytecode, binary code, and obfuscated code. Obfuscation-resistant feature extraction technology was used to recover the features of obfuscated code. Combined with feature fusion and unified representation, the completeness of the extracted code feature data was ensured, providing reliable support for subsequent feature matching. This further addresses the problems of incomplete and inaccurate feature extraction under various code forms and obfuscation scenarios, improving the efficiency and quality of code feature extraction.

[0047] Step S3: Analyze the code feature data of the object to be detected based on the neural network to obtain the bloodline heterogeneity map.

[0048] In some embodiments, the lineage heterogeneous graph includes a package-level graph, a module-level graph, a class-level graph, and a function-level graph.

[0049] In one feasible implementation, the heterogeneous lineage graph includes package-level, module-level, class-level, and function-level graphs. Simultaneously, it constructs mapping relationships between levels (hierarchical inclusion relationships, runtime dependency call relationships, object-oriented inheritance / implementation relationships, module dependency reference / usage relationships, different version evolution relationships, defect and vulnerability impact relationships, license compatibility relationships, etc.), ultimately forming a unified heterogeneous graph. The input data can be multimodal code feature data of the target code of the object to be detected, which, after analysis and processing, generates and outputs multi-level heterogeneous relationship graphs such as package-level, module-level, class-level, and function-level graphs, and constructs the relevant mapping relationships between each level.

[0050] In one feasible implementation, the package-level graph is the coarsest granularity, with the target software project itself as the root node. Explicit and implicit dependencies are discovered and resolved through code analysis, with different evolved versions as relational edges (connecting different versions of the same package), and vulnerabilities as impact edges.

[0051] In one feasible implementation, the function-level graph is the finest granularity. All function definitions are extracted by parsing the AST, and internal and external calls (across files / packets) are analyzed. Call edges are refined based on symbolic execution (such as resolving virtual calls / dynamic dispatch) to form a function-level call graph.

[0052] This method utilizes neural networks to analyze code feature data and construct a heterogeneous lineage graph, encompassing package-level, module-level, class-level, and function-level graphs. It establishes various mapping relationships between these levels, overcoming the limitations of single-granularity in traditional code analysis. The package-level graph comprehensively captures explicit and implicit dependencies of software packages, as well as version evolution and vulnerability impacts. The function-level graph, through AST parsing and symbolic execution refining of call edges, achieves precise parsing of function call relationships. The synergistic effect of these graphs and mapping relationships clearly presents the interconnected logic of each level of the software code, aiding in software vulnerability detection, version evolution tracing, and quality control. This improves code analysis efficiency and reduces software maintenance and testing costs.

[0053] Step S4: Match the target code source matching relationship dataset based on the multi-dimensional feature library, the code feature data of the object to be detected, and the lineage heterogeneity graph.

[0054] In some embodiments, matching is performed based on a multi-dimensional feature library, code feature data of the object to be detected, and a lineage heterogeneity graph to obtain a target code source tracing matching relationship dataset, including: Lightweight statistical fingerprinting is used to quickly match the first candidate set based on a multi-dimensional feature library, code feature data of the object to be detected, and heterogeneous lineage graph. Based on the first candidate set, a Bloom filter is used for pre-screening to obtain the second candidate set; Cache hit optimization is performed based on the second candidate set to obtain the target code source matching relationship dataset.

[0055] In one feasible implementation, a hierarchical matching strategy is used to achieve accurate source tracing from the package level to the code snippet level. The hierarchical matching strategy includes L0-rapid filtering layer, L1-coarse-grained matching layer, L2-fine-grained verification layer, L3-code snippet-level source tracing, and L4-version inference layer. The input data can be baseline feature data, multimodal code feature data of the target code to be detected, and generated multi-level heterogeneous relationship graph data (i.e., lineage heterogeneous graph). After strategy matching between the feature data of the target code to be detected and the baseline feature data, a target code source tracing matching relationship dataset is generated and output.

[0056] The hierarchical matching strategy employs a lightweight statistical fingerprinting layer for fast matching, a Bloom filter for pre-screening, and cache hit optimization to output a Top N candidate set (e.g., N=10). The hierarchical Bloom filter architecture supports progressive filtering, quickly selecting the candidate set and then refining it layer by layer to generate a query fingerprint. The L1-coarse-grained matching layer in the hierarchical matching strategy outputs a Top N candidate set (e.g., N=100) through feature vector similarity calculation, approximate nearest neighbor search, and parallel retrieval. Similarity calculations based on AST trees can be performed by setting the weights and confidence levels of each modality. The L2-fine-grained verification layer in the hierarchical matching strategy uses graph structure isomorphism verification, call graph subgraph matching, and semantic embedding alignment verification to output a Top N candidate set (e.g., N=1000). The L3-code fragment-level source tracing layer in the hierarchical matching strategy uses AST subtree isomorphism matching, function-level code clone detection, and precise code line number location to output a Top N candidate set (e.g., N=10000). AST subtree precise matching is based on syntax tree subtree matching to accurately locate code snippets. It indexes all subtrees in the source code repository. The basic process involves parsing the AST, extracting all function-level and statement block-level subtrees, and generating function-level code structure fingerprints. The L4-version inference layer in the hierarchical matching strategy outputs precise version numbers and confidence scores through version evolution graph analysis, probabilistic graphical model version inference, and API compatibility matrix matching. Version evolution graph analysis is based on precise version inference using a temporal graph neural network to construct the version evolution graph of components.

[0057] This application employs a multi-dimensional feature library, combining code feature data of the target object with a heterogeneous lineage graph for matching, along with a hierarchical matching strategy to achieve code source tracing matching, thus improving the accuracy and efficiency of source tracing. The hierarchical matching strategy refines the process layer by layer, from rapid screening to version inference. Lightweight statistical fingerprint matching, Bloom filter pre-screening, and cache hit optimization quickly narrow down the candidate range and reduce redundant computation. Subsequent levels utilize feature vector similarity calculation, graph structure isomorphism verification, and AST subtree matching to achieve accurate source tracing from the package level to the code fragment level, while simultaneously completing accurate version inference and confidence output. This application can quickly obtain accurate target code source tracing matching relationship datasets, providing strong support for code source tracing, vulnerability location, and version tracking, and reducing source tracing costs.

[0058] Step S5: Analyze the target code source matching relationship dataset to obtain a multi-dimensional feature map of blood relations.

[0059] This can generate multi-dimensional feature maps of lineage relationships, including component attributes, dependency types, and reference locations. These multi-dimensional feature maps allow for the visualization, analysis, and utilization of code lineage relationships. Specifically, the input data can be a dataset of source matching relationships between the feature data of the target code of the object to be detected and the baseline feature data. After analysis and processing, multi-dimensional feature maps of lineage relationships are output, such as entity mapping relationship maps, attribute relationship maps, version evolution maps, and multi-perspective relationship maps for code dependencies / function calls / vulnerability risks / compliance, etc.

[0060] In one feasible implementation, data input can include matching results obtained through a hierarchical matching strategy, generated node / edge vector graph neural networks, component attribute relationships, dependency relationships, vulnerability and risk data, license and compliance information, etc. A knowledge graph engine is used to construct entity mapping relationships, attribute graph relationships, version evolution modeling, and multi-perspective relationships such as dependency / call / risk / compliance views.

[0061] The ontology of a kinship multidimensional feature graph can include entities such as software artifacts, open-source components, code entities, classes / modules, functions / methods, security vulnerabilities, licenses, and tracing events. The core definitions of each entity are as follows: Software artifacts include unique identifiers, names, versions, types, main programming languages, creation time, content hashes, and byte sizes; open-source components include source code repositories, licenses, license texts, stars, forks, last commits, maintainer lists, and health scores; code entities include package names, classes, functions, files, and packages / namespaces (fully qualified names, file paths, and documentation comments); classes / modules include parent classes, implemented interfaces, number of methods, number of fields, number of lines of code, and cyclomatic complexity; functions / methods include full signatures, access permissions, parameter lists, return types, AST structure hashes, semantic embeddings, start line numbers, and end line numbers; security vulnerabilities include CVE numbers, CVSS scores, security levels, descriptions, release times, patch versions, presence of POCs, and exploit maturity; licenses include SPDX identifiers, names, OSI certifications, permitted behaviors, and restrictions; and tracing events include basic attributes such as build / test / release / deployment timestamps, executors, tools, input artifacts, output artifacts, environment information, and digital signatures. Lineage relationships mainly include dependency relationships, inclusion relationships, calling relationships, inheritance relationships, referencing relationships, affected vulnerabilities, license statements, version evolution, and source tracing proofs.

[0062] This method analyzes the target code source code matching dataset to generate a multi-dimensional feature map of lineage relationships. It integrates multiple entities and various lineage relationship types to achieve visualized analysis and efficient utilization of code lineage relationships. The multi-dimensional feature map of lineage relationships covers relationships from multiple perspectives. Its ontology includes rich entities such as software artifacts, open-source components, and code entities. Each entity has a complete core definition and can comprehensively present various attributes and related information related to the code. This feature map clearly shows the lineage relationships between codes, such as dependencies, calls, and inheritance, helping relevant personnel intuitively grasp the code's relational logic. This facilitates vulnerability investigation, compliance checks, version tracing, and code maintenance, improving code management efficiency, providing comprehensive support for software quality control and security assurance, and reducing software management and maintenance costs.

[0063] The code lineage tracing method based on multimodal features provided in the above embodiments can achieve comprehensive integration of open-source code features by acquiring open-source code data and constructing a multi-dimensional feature library, providing rich feature support for code tracing and solving the problems of single feature dimensions and insufficient supporting basis in traditional tracing methods. After acquiring the code of the object to be detected, feature extraction is performed based on the multi-dimensional feature library, which can accurately capture the core feature information of the code of the object to be detected, ensuring that the extracted code feature data is highly adapted to the tracing requirements and improving the targeting of subsequent tracing analysis. The analysis of code feature data using neural networks yields a lineage heterogeneous graph, which can clearly present the complex lineage relationship between the code of the object to be detected and the open-source code, breaking the limitation of traditional analysis methods that are difficult to intuitively display complex lineage relationships. By combining the code feature data of the multi-dimensional feature library and the lineage heterogeneous graph for matching, a target code tracing matching relationship dataset is obtained, realizing multi-dimensional tracing matching, greatly improving the accuracy of code tracing matching, reducing tracing errors, and ensuring that the matching results can truly reflect the lineage relationship between codes. Analyzing the target code tracing and matching relationship dataset yields a multi-dimensional feature map of lineage relationships. This map integrates code lineage relationships with features of various dimensions, not only visually demonstrating the distribution patterns of code lineage relationships but also clearly reflecting the impact of different features on lineage relationships. This provides strong support for the verification, analysis, and subsequent code optimization of code tracing results, further reducing the difficulty and cost of code tracing.

[0064] In some embodiments, blockchain notation is also used to ensure the credibility and traceability of the results. The input data can be extracted feature data, a dataset of traceability matching relationships between the feature data of the target code of the object to be detected and the baseline feature data, a multi-dimensional feature relationship graph, etc. Blockchain notation of the code feature data facilitates subsequent sharing and utilization of feature data.

[0065] Blockchain-based evidence storage involves taking the results obtained from the above steps, including information such as blood relations, original evidence data, identity authentication information, and environmental context, and performing data standardization, sensitive information desensitization (privacy protection processing), evidence integrity calculation, and digital signature generation for evidence storage preprocessing. Then, the processed traceability results are stored in the blockchain management system, which performs data storage. The purpose of introducing the blockchain evidence storage mechanism is to ensure the immutability and auditability of the traceability results and to enable the trusted use of the stored evidence data on-chain or across-chain.

[0066] The system incorporates a blockchain-based evidence storage mechanism to process and store code feature data, source tracing matching datasets, and multi-dimensional feature relationship graphs. Prior to storage, preprocessing steps such as data standardization, sensitive information de-identification, evidence integrity calculation, and digital signature generation are performed, protecting data privacy and ensuring evidence integrity. The blockchain management system stores the processed source tracing results, ensuring their immutability and auditability. This enables trusted on-chain and cross-chain use of the stored data and facilitates subsequent sharing and use of feature data. This mechanism significantly enhances the credibility of code source tracing results, provides strong guarantees for compliance and traceability, reduces disputes over source tracing results, lowers related management costs, and further improves the code source tracing system.

[0067] Please see Figure 2 Another embodiment of this application provides a code lineage tracing device based on multimodal features, the device comprising: The benchmark feature library construction module 101 is used to acquire open source code data and extract it to obtain structured features, semantic features and statistical features, and to build a multi-dimensional feature library.

[0068] The code feature extraction module 102 is used to obtain the code of the object to be detected, and to extract features based on the multi-dimensional feature library and the code to obtain the code feature data of the object to be detected.

[0069] The neural network analysis module 103 is used to analyze the code feature data of the object to be detected based on the neural network to obtain a bloodline heterogeneity map.

[0070] The source matching module 104 is used to perform matching based on the multi-dimensional feature library, the code feature data of the object to be detected, and the lineage heterogeneity graph to obtain the target code source matching relationship dataset.

[0071] The graph generation module 105 is used to analyze the target code source matching relationship dataset to obtain a multi-dimensional feature map of blood relations.

[0072] The specific limitations of the code lineage tracing device based on multimodal features provided in this embodiment can be found in the embodiment of the code lineage tracing method based on multimodal features described above, and will not be repeated here. Each module in the above-described code lineage tracing device based on multimodal features can be implemented entirely or partially through software, hardware, or a combination thereof. Each module can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0073] This application provides a computer device that may include a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it causes the processor to perform the steps of a code lineage tracing method based on multimodal features, as described in any of the above embodiments.

[0074] The working process, working details and technical effects of the computer device provided in this embodiment can be found in the embodiment of a code lineage tracing method based on multimodal features mentioned above, and will not be repeated here.

[0075] This application provides a computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the steps of a code lineage tracing method based on multimodal features as described in any of the above embodiments. The computer-readable storage medium refers to a data storage medium, which may include, but is not limited to, floppy disks, optical disks, hard disks, flash memory, USB flash drives, and / or Memory Sticks. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0076] The working process, working details, and technical effects of the computer-readable storage medium provided in this embodiment can be found in the embodiment of a code lineage tracing method based on multimodal features described above, and will not be repeated here.

[0077] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0078] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0079] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A code lineage tracing method based on multimodal features, characterized in that, The method includes: Obtain open-source code data and extract it to obtain structured features, semantic features, and statistical features, and build a multi-dimensional feature library; The code of the object to be detected is obtained, and feature extraction is performed based on the multi-dimensional feature library and the code to obtain the code feature data of the object to be detected. Based on the analysis of the code feature data of the object to be detected by the neural network, a bloodline heterogeneity map is obtained; Matching is performed based on a multi-dimensional feature library, code feature data of the object to be detected, and a lineage heterogeneity graph to obtain a target code source tracing matching relationship dataset. The target code source matching relationship dataset was analyzed to obtain a multi-dimensional feature map of blood relations.

2. The code lineage tracing method based on multimodal features according to claim 1, characterized in that, The structured features include abstract syntax tree features, control flow graph features, and function call graph features; the semantic features include natural semantic features and program behavior semantic features based on the understanding of the large code model; the statistical features include code metric features and constant fingerprint features.

3. The code lineage tracing method based on multimodal features according to claim 2, characterized in that, The process of acquiring and extracting open-source code data to obtain structured features includes: Obtain open-source code data and perform language parsing to obtain the first data; Based on the first data, the node type sequence is extracted to obtain the second data; Based on the second data, structural path extraction is performed to obtain the third data; Based on the third data, subtree pattern mining is performed to obtain the fourth data; Vector encoding is performed based on the fourth data to obtain abstract syntax tree features as structured features.

4. The code lineage tracing method based on multimodal features according to claim 2, characterized in that, The process of acquiring and extracting open-source code data to obtain semantic features includes: Obtain open-source code data and perform symbolic initialization to obtain the first parameter; Based on the first parameter, path constraints are extracted to obtain the second parameter; The third parameter is obtained by simplifying the second parameter. Based on the third parameter, a behavior summarization operation is performed to obtain the semantic features of program behavior.

5. The code lineage tracing method based on multimodal features according to claim 1, characterized in that, The step of extracting features based on the multi-dimensional feature library and the code to obtain code feature data of the object to be detected includes: Based on the multi-dimensional feature library and the code, basic block recognition is performed to obtain the first field; Based on the first field, construct the inter-block control flow edge to obtain the second field; The third field is obtained by analyzing the block termination instruction type based on the second field. Based on the third field, a graph structure is constructed and graph features are calculated to obtain the code feature data of the object to be detected.

6. The code lineage tracing method based on multimodal features according to claim 1, characterized in that, The lineage heterogeneous graph includes package-level graphs, module-level graphs, class-level graphs, and function-level graphs.

7. The code lineage tracing method based on multimodal features according to claim 1, characterized in that, The matching process, based on a multi-dimensional feature library, code feature data of the target object, and a lineage heterogeneity graph, yields a target code tracing and matching relationship dataset, including: Lightweight statistical fingerprinting is used to quickly match the first candidate set based on a multi-dimensional feature library, code feature data of the object to be detected, and heterogeneous lineage graph. Based on the first candidate set, a Bloom filter is used for pre-screening to obtain a second candidate set; Based on the second candidate set, cache hit optimization is performed to obtain the target code source tracing matching relationship dataset.

8. A code lineage tracing device based on multimodal features, characterized in that, The device includes: The benchmark feature library construction module is used to acquire and extract open-source code data to obtain structured features, semantic features, and statistical features, and to build a multi-dimensional feature library. The code feature extraction module is used to acquire the code of the object to be detected, and to perform feature extraction based on the multi-dimensional feature library and the code to obtain the code feature data of the object to be detected. The neural network analysis module is used to analyze the code feature data of the object to be detected based on the neural network to obtain a bloodline heterogeneity graph; The source matching module is used to match based on a multi-dimensional feature library, code feature data of the object to be detected, and a lineage heterogeneity graph to obtain a target code source matching relationship dataset; The graph generation module is used to analyze the target code source matching relationship dataset to obtain a multi-dimensional feature map of blood relations.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the code lineage tracing method based on multimodal features as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the code lineage tracing method based on multimodal features as described in any one of claims 1 to 7.

Images