Intelligent tracing and automatic detection method and system for vulnerability of power information system based on LLM

Abstract

The application discloses a LLM-based power information system vulnerability intelligent tracing and automatic detection method and system, and belongs to the technical field of artificial intelligence. Step 1, a software project in a power information system is acquired, and the software project comprises source code, a package management file and a developer document; step 2, the source code and the developer document are subjected to structured analysis, and corresponding code structure nodes and edges and document structure nodes and edges are obtained; step 3, the code structure nodes and edges and the document structure nodes and edges are fused, and a node set and an edge set at a project level are obtained; and step 4, the package management file is parsed, and a dependent node set and a dependent edge set are obtained. The natural language description in the document can be used to assist in understanding the code intention, and the third-party dependency relationship is combined, so that the missing report problem caused by the context loss in a complex power information system is effectively solved, and the accuracy of vulnerability positioning is greatly improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and more specifically, to a method and system for intelligent tracing and automated detection of vulnerabilities in power information systems based on LLM. Background Technology

[0002] With the continuous advancement of the construction of new power systems, power information systems have become a key carrier supporting power grid production scheduling, operation management, marketing services and other businesses. Their safe and stable operation is directly related to the safety of energy infrastructure and the stability of social life. Vulnerability detection and vulnerability tracing are key aspects of power information system network security protection. Currently, the mainstream detection and tracing solutions in the industry are still supported by Static Application Security Testing (SAST) and Dynamic Application Security Testing (DAST) technologies.

[0003] Existing traditional technical solutions have many insurmountable technical shortcomings when facing large-scale, highly complex power software projects. First, traditional code auditing and vulnerability detection tools, based on basic technologies such as rule matching and control flow analysis, generally lack the ability to understand the deep semantics of the code. These tools can only perform isolated syntactic analysis of the source code and cannot accurately capture the complex logical relationships and data flow relationships hidden between variables, functions, and modules. This directly leads to extremely high false positive and false negative rates in the detection results. This not only significantly increases the time and manpower costs for security personnel to manually review the data, but also makes it difficult to identify deep-seated logical vulnerabilities hidden in power business scenarios and cannot meet the high-level, high-reliability security protection requirements of power systems.

[0004] Secondly, existing detection solutions suffer from insufficient utilization of multi-source security information, making it difficult to construct a complete project security knowledge graph. Traditional detection methods mostly focus only on the source code itself, generally neglecting key auxiliary information in developer documentation and package management files. On the one hand, they cannot utilize the natural language descriptions such as function definitions, interface specifications, and business logic descriptions in developer documentation to assist in semantic alignment and deep understanding of the code's true intent. On the other hand, they lack the ability to conduct automated risk analysis of third-party dependency packages, making it impossible to effectively identify potential security vulnerabilities at the software supply chain level. The lack of the aforementioned key information directly leads to the fragmentation of project context information during the source tracing analysis process, making it impossible to achieve accurate source tracing of vulnerabilities across the entire chain.

[0005] Traditional solutions rely heavily on security experts to manually write test scripts and set up dedicated test environments to verify vulnerabilities after a suspected vulnerability is discovered. This is not only inefficient, but the verification process is also difficult to standardize and is easily affected by differences in personnel's technical capabilities. While existing automated fuzz testing technologies can generate test cases in batches, they lack the specificity and intelligent planning capabilities for power business scenarios. They are unable to automatically construct effective attack paths in complex power business logic, and they cannot dynamically optimize and adjust test strategies based on test results, making it difficult to achieve closed-loop verification and in-depth vulnerability discovery. Summary of the Invention

[0006] To address the problems existing in the prior art, the purpose of this invention is to provide an intelligent source tracing and automated detection method and system for vulnerabilities in power information systems based on LLM. This method can utilize natural language descriptions in documents to assist in understanding the code intent and, combined with third-party dependencies, effectively solve the problem of missed detections caused by missing context in complex power information systems, greatly improving the accuracy of vulnerability location.

[0007] To solve the above problems, the present invention adopts the following technical solution:

[0008] The first aspect is a method for intelligent tracing and automated detection of vulnerabilities in power information systems based on LLM, including:

[0009] Step 1: Obtain the software project in the power information system. The software project includes source code, package management files, and developer documentation.

[0010] Step 2: Perform structured analysis on the source code and developer documentation to obtain the corresponding code structure nodes and edges, as well as document structure nodes and edges. Code structure nodes include at least function nodes, class nodes, and module nodes, and code structure edges include at least edges representing function call relationships, class-function inclusion relationships, and module-function definition relationships. Document structure nodes include document fragment text, format, and associated code entity identifiers, and document structure edges represent the descriptive relationship between code and documentation.

[0011] Step 3: Merge the code structure nodes and edges with the document structure nodes and edges to obtain a project-level set of nodes and edges;

[0012] Step 4: Parse the package management file to obtain the set of dependency nodes and the set of dependency edges; the attributes of the dependency nodes include the package name, version number and source address, and the dependency edges represent the relationship between the code calling the interface of the external dependency package;

[0013] Step 5: Integrate the dependency node set and dependency edge set into the project-level node set and edge set to construct a knowledge graph, and perform semantic embedding on the nodes in the knowledge graph to obtain a graph-enhanced retrieval library;

[0014] Step 6: Obtain vulnerability information through the first intelligent agent, extract key information from the vulnerability information using a large language model, and perform subgraph retrieval based on the key information and the graph-enhanced retrieval library to obtain the corresponding subgraph.

[0015] Step 7: The second agent sends the subgraph and feedback information from the third agent to the large language model to plan the test scheme;

[0016] Step 8: Run the test plan in the sandbox through a third intelligent agent, locate the vulnerable code fragment, and choose to output vulnerability information or provide feedback information to the second intelligent agent to regenerate the test plan based on the running results.

[0017] Further, step 2 includes:

[0018] Step 21: Parse the source code file for functions, extract function nodes, and set node attributes;

[0019] Step 22: Perform class parsing on the source code file, extract class nodes, and set node attributes;

[0020] Step 23: Perform module parsing on the source code file, extract module nodes, and set node attributes;

[0021] Step 24: Based on the extracted function nodes, class nodes, and module nodes, perform code structure analysis to obtain an edge set containing function call relationships, class-function inclusion relationships, and module-function definition relationships.

[0022] Furthermore, step 2 also includes:

[0023] Step 25: Parse the developer documentation, extract the document nodes related to the source code file, and set the node attributes, including the document fragment text, format, and associated code entity identifiers.

[0024] Step 26: Analyze the extracted document nodes and code structure nodes to obtain a set of edges representing the descriptive relationship between the code and the document.

[0025] Further, step 3 includes:

[0026] Step 31: Merge the set of code structure nodes and the set of document nodes corresponding to a single source code file to obtain the merged node set of the file;

[0027] Step 32: Merge the code structure edge set and document edge set corresponding to a single source code file to obtain the fused edge set of that file;

[0028] Step 33: Merge the set of nodes for all files to obtain the set of nodes for the entire project;

[0029] Step 34: Merge the edge sets of all files to obtain the edge set of the entire project.

[0030] Further, step 4 includes:

[0031] Step 41: Parse the package management file, extract the dependency package nodes, and set the package name, version number, source address, and license attributes for each dependency package node;

[0032] Step 42: Perform structural analysis on the package management file, dependency package nodes, and project code nodes to obtain a set of edges representing the dependency relationships of code calling external interfaces.

[0033] Further, step 5 includes:

[0034] Step 51: Merge the dependency node set with the project-level node set to obtain the final knowledge graph node set;

[0035] Step 52: Merge the dependency edge set with the project-level edge set to obtain the final knowledge graph edge set;

[0036] Step 53: For each node in the knowledge graph, generate a semantic embedding vector based on its code content and documentation, and store it in a vector database to build a graph-enhanced retrieval library.

[0037] Further, step 6 includes:

[0038] Step 61: Obtain vulnerability information from the vulnerability release platform through the first intelligent agent, and extract key vulnerability information using a large language model;

[0039] Step 62: Semantically embed the obtained vulnerability information to obtain a vulnerability semantic vector;

[0040] Step 63: Match the key information and semantic vector of the vulnerability with the nodes in the knowledge graph, and select the nodes with the highest matching degree as seed nodes.

[0041] Step 64: Starting from the seed node, perform graph traversal in the knowledge graph to expand and obtain subgraphs related to vulnerability information.

[0042] Further, step 7 includes:

[0043] Step 71: In the second agent, the retrieved subgraph is deserialized into a natural language context.

[0044] Step 72: Input the key vulnerability information, natural language context, and feedback information from the third agent into the large language model to generate a test plan.

[0045] Further, step 8 includes:

[0046] Step 81: In the third intelligent agent, build a sandbox environment for the software project that includes a virtual system environment and testing tools;

[0047] Step 82: Run the generated test plan in a sandbox environment and obtain the results;

[0048] Step 83: Evaluate whether the running result triggers the same vulnerability as the vulnerability report. If so, locate and output the code snippet containing the vulnerability. If not, and the maximum number of retries has not been reached, generate feedback information and send it to the second agent to trigger the regeneration of the test plan.

[0049] Secondly, this invention also provides an LLM-based intelligent vulnerability tracing and automated detection system for power information systems, comprising:

[0050] The knowledge graph fusion building module is used to analyze and fuse source code, developer documentation, and package management files to build a knowledge graph with semantic embedding.

[0051] The vulnerability information acquisition module is used to acquire vulnerability information and extract key information;

[0052] The vulnerability retrieval module is used to retrieve relevant subgraphs in the knowledge graph based on vulnerability information;

[0053] The test plan planning module is used to generate test plans by calling the large language model based on the retrieved subgraphs and feedback information.

[0054] The test execution and feedback module is used to execute test plans in a sandbox environment, evaluate results, and generate feedback information.

[0055] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0056] (1) This invention constructs a unified knowledge graph containing function calls, module dependencies, and document descriptions by performing structured analysis and semantic embedding on source code, package management files, and developer documentation. This method can not only analyze the logic of the code itself, but also use the natural language descriptions in the documentation to help understand the code intent. Combined with third-party dependency relationships, it effectively solves the problem of missed detections caused by missing context in complex power information systems, and greatly improves the accuracy of vulnerability location.

[0057] (2) This invention utilizes a large language model to extract and vectorize key vulnerability information, and constructs a contextual environment by retrieving highly relevant subgraph information through GraphRAG. Compared with traditional keyword matching or regular expression matching, this invention can understand the deep semantic features of vulnerabilities, thereby discovering logical vulnerabilities and hidden vulnerabilities across files and modules that are difficult to identify by traditional tools.

[0058] (3) This invention designs multiple intelligent agents to work collaboratively and run test cases in an isolated sandbox environment. The system has self-correction capabilities. When the test does not trigger a vulnerability, it can readjust and generate a new test plan based on the feedback information from the sandbox. This closed-loop mechanism replaces the tedious process of manually writing proof of concept in the traditional way, realizes complete automation from vulnerability discovery to verification, and confirms the vulnerability through the actual running results of the sandbox, eliminating the interference of false alarms common in static analysis.

[0059] (4) When performing structured analysis and edge set extraction on the source code, this invention utilizes the efficient indexing mechanism of LSP itself to accelerate the parsing speed of the code syntax tree and reference relationship, avoids repeated scanning of the original code in LLM, and reduces the computational overhead of the system and the token overhead of LLM in building GraphRAG. Attached Figure Description

[0060] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0061] Figure 1 This is a flowchart of the LLM-based intelligent tracing and automated detection method for vulnerabilities in power information systems according to the present invention.

[0062] Figure 2 This is a schematic diagram of the LLM-based intelligent tracing and automated detection system for vulnerabilities in power information systems according to the present invention.

[0063] Figure 3 This is a data flow diagram between various modules in the LLM-based intelligent tracing and automated detection system for power information system vulnerabilities of the present invention. Detailed Implementation

[0064] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0065] Example 1:

[0066] Please see Figure 1 A method for intelligent tracing and automated detection of vulnerabilities in power information systems based on LLM, comprising the following:

[0067] Step 1: Obtain the software project from the power information system. The software project includes source code, package management files, and developer documentation. The specific steps are as follows:

[0068] Given a software project in a power information system ,in Includes source code collection Package Management Files With developer documentation ,in, This indicates the number of files in project P, and also the number of blocks into which the developer documentation is divided, with each block corresponding to a relevant section of the source code file.

[0069] In a preferred embodiment of the present invention, step 2 is further included: performing structured analysis on the source code and developer documentation to obtain corresponding code structure nodes and edges, as well as document structure nodes and edges; the code structure nodes include at least function nodes, class nodes, and module nodes, and the code structure edges include at least edges representing function call relationships, class-function inclusion relationships, and module-function definition relationships; the attributes of the document structure nodes include document fragment text, format, and associated code entity identifiers, and the document structure edges represent the descriptive relationship between code and document, as detailed below:

[0070] For the project Each code file in the source code collection and its corresponding document section Perform structured analysis to obtain the corresponding node set. , Sum of edges , Each node and edge has its own attributes.

[0071] Step 2 also includes the following steps:

[0072] Step 21: Parse the source code file for functions, extract function nodes, and set node attributes. The specific operations are as follows:

[0073] For source code files Analysis and extraction of function node set For each of the nodes Set its properties;

[0074] ;

[0075] ;

[0076] ;

[0077] in, This represents a analytic function. This represents the property setting function. This represents a function definition in the source code. Indicates the function in the file The number, This represents the function name, and its value is set to... , This represents the function signature, and its value is set to... , Includes the path to the file containing the function and the line number of the function; its value is set to... , This represents the original code corresponding to the function, and its value is set to... , Indicates whether a function is an external function; its value is set to... .

[0078] Step 22: Perform class parsing on the source code file, extract class nodes, and set node attributes. The specific operations are as follows:

[0079] For source code files Analyze and extract the class node set For each node Set its properties;

[0080] ;

[0081] ;

[0082] ;

[0083] in, Represents a class resolution function. This represents the property setting function. This represents the definition of a class or struct in the source code. Indicates the class in the file The number, This represents the name of the class, and its value is set to... , This represents the list of method names contained in the class, and its value is set to... , This represents the original code corresponding to the class, and its value is set to... , The value of the parent class name inherited by the class is set to... .

[0084] Step 23: Perform module parsing on the source code file, extract module nodes, and set node attributes. The specific operations are as follows:

[0085] For source code files Analysis and extraction module node set For each node Set its properties;

[0086] ;

[0087]

[0088] ;

[0089] in, This represents the module parsing function. This represents the property setting function. This represents a module unit in the source code. Indicates the module in the file The number, This represents the original code corresponding to the module, and its value is set to... , This represents the path to the module, and its value is set to... , Indicates the programming language used by the module, its value is set to , This indicates information about header files or other modules imported by this module; its value is set to... .

[0090] Step 24: Based on the extracted function nodes, class nodes, and module nodes, perform code structure analysis to obtain a set of edges containing function call relationships, class-function inclusion relationships, and module-function definition relationships. The specific operations are as follows:

[0091] For source code files With the node set extracted above Structural analysis yields the corresponding edge set. ;

[0092] ;

[0093] ;

[0094] ;

[0095] ;

[0096] ;

[0097] in, This represents a code structure analysis function; LSP stands for Language Server Protocol, which is used through... It can speed up the acquisition of relevant information. This represents the edge set result obtained from the code structure analysis function, containing... , and , This represents a set of function call edges, where each edge contains an attribute `call` to indicate the function call relationship. For example, for one of them... It can be represented as , meaning function Calling functions , This represents a collection of edges containing methods within a class. Each edge contains the attribute `contain` to indicate the containment relationship between the class and the function. For example, for one of these edges... It can be represented as The meaning is class Include functions , This represents the set of edges that define functions within a module. Each edge contains a `define` attribute to indicate the relationship between the module and the function definition. For example, for one of these edges... It can be represented as , meaning module Define function .

[0098] LSP-based code structure parsing implementation:

[0099] (1) Select the corresponding LSP implementation according to the project language, such as using pylsp for Python, eclipse-jdt-ls for Java, and typescript-language-server for JavaScript / TypeScript.

[0100] (2) Start the LSP server process and initialize the workspace;

[0101] For each source code file Send a textDocument / documentSymbol request to retrieve symbol information such as functions, classes, and modules from the file;

[0102] For each function / method, send a textDocument / definition to obtain its definition location;

[0103] For each function call, send textDocument / references to obtain the call relationship;

[0104] The structured data (JSON format) returned by LSP can be directly used to build... , , Edge set.

[0105] (3) Compared with static AST parsing, LSP provides cross-file reference and definition parsing capabilities, avoiding manual handling of import relationships; it supports incremental updates, making it easy to reuse parsing results during project iteration; the parsing results are highly structured and can be directly mapped to graph nodes and edges.

[0106] Step 25: Parse the developer documentation, extract document nodes related to the source code file, and set node attributes. Node attributes include document fragment text, format, and associated code entity identifiers. The specific operations are as follows:

[0107] For source code files Corresponding documentation Parse and extract document node set For each node Set its properties;

[0108] ;

[0109] ;

[0110] ;

[0111] ;

[0112] in, Extracting code from documents Related content , This represents a document content processing function that uses a large language model for assistance. This represents the property setting function. express Document fragments in Indicates the number of a document segment. This represents the original text content of the document fragment, and its value is set to... , This indicates the document's format, and its value is set to... , This represents the code entity ID associated with the document fragment, and its value is set to... .

[0113] The specific implementation is as follows:

[0114] (1) Perform paragraph segmentation on the developer documentation (Doc) to retain the structural information of headings, code blocks, and natural language paragraphs;

[0115] (2) For each source code file Extract its file name, top-level module name, exported function name, and class name as a set of code entity identifiers. ;

[0116] (3) Using a large language model, such as GPT-4 or locally deployed CodeLlama, perform semantic matching on each document paragraph para, with the prompt word template as follows:

[0117] Please determine whether the following document paragraph is related to any code entity (function, class, module) in the code file "{file_name}". If it is related, output the name of the corresponding code entity; otherwise, output "None".

[0118] Document paragraph: {para_text};

[0119] List of code entities: {E_i};

[0120] Output format: JSON, such as {"related":true,"entities":["func_name1", "ClassName"]} or {"related":false};

[0121] (4) For each successfully matched paragraph, create a document node v_d, where the linked_code_id attribute records the identifier of the associated code entity;

[0122] (5) For unmatched paragraphs, if the chapter they belong to has a file name or path inclusion relationship with the code file (such as the "src / auth / " chapter in README.md), the association is also established, and linked_code_id records the corresponding directory path.

[0123] Step 26: Analyze the extracted document nodes and code structure nodes to obtain a set of edges representing the descriptive relationship between the code and the document. The specific operations are as follows:

[0124] right With the node set extracted above and Structural analysis yields the corresponding edge set. ;

[0125] ;

[0126] in, This represents a document structure parsing function. This represents the set of edges obtained from the document structure parsing function, which includes a series of descriptive relationships between functions, classes, or modules in the code and the document.

[0127] In a preferred embodiment of the present invention, step 3 is further included, which involves fusing the code structure nodes and edges with the document structure nodes and edges to obtain a project-level node set and edge set. The specific operation is as follows:

[0128] Node set , Sum of edges , The fusion process yields a set of fused nodes for each file. and fusion edge set Then, the node set and edge set of each file are merged again to obtain the node set of the entire project. Sum of edges .

[0129] Step 3 also includes the following operations:

[0130] Step 31: Merge the set of code structure nodes and the set of document nodes corresponding to a single source code file to obtain the merged node set for that file. The specific operations are as follows:

[0131] The node set obtained above and The set of fusion nodes is obtained by using the description information on the fusion nodes. ;

[0132] ;

[0133] in, Source code file Fusion Documents The resulting set of nodes Source code file The resulting set of nodes was analyzed. Document The resulting set of nodes was analyzed. Elements with the same information can be merged.

[0134] Step 32: Merge the code structure edge set and document edge set corresponding to a single source code file to obtain the fused edge set for that file. The specific operation is as follows:

[0135] The edge set obtained above and The set of fused edges is obtained by using the descriptive information on the fused edges. ;

[0136] ;

[0137] in, Source code file Fusion Documents The resulting edge set Source code file The set of edges obtained from the analysis, Document The set of edges obtained from the analysis, Elements with the same information can be merged.

[0138] Step 33: Merge the set of nodes for all files to obtain the node set for the entire project. The specific steps are as follows:

[0139] The fusion node set of all the files obtained above The entire project's node set is obtained by merging the description information on the nodes. ;

[0140] ;

[0141] in, Source code file Fusion Documents The resulting set of fused nodes Source code file Fusion Documents The resulting set of fused nodes Source code file Fusion Documents The resulting set of fused nodes This indicates the number of files in project P. Elements with the same information can be merged.

[0142] Step 34: Merge the edge sets of all files to obtain the edge set of the entire project. The specific operation is as follows:

[0143] The fusion edge set of all the files obtained above The edge set of the entire project is obtained by merging the descriptive information on the edges. ;

[0144] ;

[0145] in, Source code file Fusion Documents The resulting set of merged edges Source code file Fusion Documents The resulting set of merged edges Source code file Fusion Documents The resulting set of merged edges This indicates the number of files in project P. Elements with the same information can be merged.

[0146] In a preferred embodiment of the present invention, step 4 is further included: parsing the package management file to obtain a set of dependency nodes and a set of dependency edges; the attributes of the dependency nodes include package name, version number, and source address, and the dependency edges represent the relationship between code calling external dependency package interfaces. The specific operations are as follows:

[0147] For the project The package management file M is parsed to obtain the corresponding set of nodes. Sum of edges And set the properties for each node.

[0148] Step 4 also includes the following steps:

[0149] Step 41: Parse the package management file, extract the dependency package nodes, and set the package name, version number, source address, and license attributes for each dependency package node. The specific operations are as follows:

[0150] Parse the package management file M to extract the set of package nodes. For each node Set properties , , as well as ;

[0151] ;

[0152] ;

[0153] ;

[0154] in, This represents the function for parsing package management files. This represents the property setting function. This refers to a package in the package management file. Indicates the package number. This represents the name of the package, and its value is set to... , This represents the package version number, and its value is set to... , This represents the packet source address, and its value is set to... , The package's license value is set to... .

[0155] Step 42 involves performing structural analysis on the package management file, dependency package nodes, and project code nodes to obtain a set of edges representing the dependency relationships between code calls to external interfaces. The specific operations are as follows:

[0156] Package management files With the node set extracted above and Structural analysis yields the corresponding edge set. ;

[0157] ;

[0158] in, This represents the function for parsing the package structure. This represents the edge set obtained from the packet structure parsing function, which includes a series of code calls to external interfaces and their dependencies.

[0159] In a preferred embodiment of the present invention, step 5 is further included: integrating the dependency node set and dependency edge set into the project-level node set and edge set to construct a knowledge graph, and performing semantic embedding on the nodes in the knowledge graph to obtain a graph-enhanced retrieval library. The specific operations are as follows:

[0160] The collection of nodes generated from the package management file Sum of edges Integration and Get the final set of nodes set of edges To obtain the knowledge graph Next, semantic embedding is performed on all nodes and stored in a vector database to obtain the corresponding GraphRAG.

[0161] Step 5 also includes the following steps:

[0162] Step 51: Merge the dependency node set with the project-level node set to obtain the final knowledge graph node set. The specific operations are as follows:

[0163] Package management file Extracted node set and The final set of nodes is obtained by merging them. ;

[0164] ;

[0165] in, This represents the set of nodes obtained from analyzing the package management file. This represents the set of nodes for the entire project. This indicates a fusion symbol.

[0166] Step 52: Merge the dependency edge set with the project-level edge set to obtain the final knowledge graph edge set. The specific operations are as follows:

[0167] The edge set obtained above and The final set of edges is obtained by merging them. ;

[0168] ;

[0169] in, This represents the set of edges obtained from analyzing the package management file. Represents the set of edges for the entire project. This indicates a fusion symbol.

[0170] Step 53: For each node in the knowledge graph, generate a semantic embedding vector based on its code content and documentation, and store it in a vector database to build a graph-enhanced retrieval library. The specific operations are as follows:

[0171] For a certain node The content is processed by an embedding function to obtain the corresponding semantic embedding vector, and finally the constructed GraphRAG is obtained.

[0172] ;

[0173] ;

[0174] in, This represents a semantic embedding function, and the result is enclosed in a d-dimensional vector. Indicates an embedded function. This represents the content of a node, including the code corresponding to the node. and related documentation .

[0175] Specific implementation details of semantic embedding models and vector storage:

[0176] (1) CodeBERT or GraphCodeBERT is used as the basic embedding model. This model supports joint embedding of code and natural language. For code nodes, the input is code snippets; for document nodes, the input is document snippet text; for dependency package nodes, the input is "package name: version number".

[0177] (2) The context of each node v is constructed as follows:

[0178] Code node: Retrieves the complete code text of a function / class / module. If the length exceeds 512 tokens, it is truncated, retaining 256 tokens at the beginning and end.

[0179] Document node: Take the text fragment of the document. If the length exceeds 512 tokens, split it by a period and take the first 512 tokens.

[0180] Dependency package node: Construct the string "package: {name} version: {version}".

[0181] (3) Load the pre-trained model using the sentence-transformers library to generate 768-dimensional or 1024-dimensional vectors, normalize them, and store them in the Milvus or FAISS vector database.

[0182] (4) In the vector database, each record contains:

[0183] node_id: A unique identifier for the node;

[0184] embedding: semantic vector;

[0185] node_type: Node type (function / class / module / document / dependency);

[0186] neighbors: Stores a list of one-hop neighbor node IDs for subsequent subgraph expansion.

[0187] In a preferred embodiment of the present invention, step 6 is further included: obtaining vulnerability information through a first intelligent agent, extracting key information from the vulnerability information using a large language model, and performing subgraph retrieval based on the key information and a graph-enhanced retrieval library to obtain the corresponding subgraph. The specific operations are as follows:

[0188] By retrieving the agent (i.e., the first agent). The system acquires vulnerability information and extracts key vulnerability information. It then performs semantic embedding representation on the acquired vulnerability information and performs subgraph retrieval based on the semantic information and the extracted key information to obtain the corresponding subgraph.

[0189] Step 6 also includes the following steps:

[0190] Step 61: Obtain vulnerability information from the vulnerability release platform through the first intelligent agent, and extract key vulnerability information using a large language model. The specific operations are as follows:

[0191] exist Obtaining vulnerability information through vulnerability acquisition methods Next, the vulnerability information is extracted using a large language model to extract its key information;

[0192] ;

[0193] ;

[0194] in, This represents a function for identifying vulnerabilities. This represents a collection of websites that publish vulnerability information. This is a function for extracting key information.

[0195] Step 62: Semantic embedding is performed on the acquired vulnerability information to obtain the vulnerability semantic vector. The specific operations are as follows:

[0196] To obtain vulnerability information Semantic embedding is used to obtain vectors ;

[0197] ;

[0198] in, This represents a semantic embedding function. This represents a semantic vector of dimension d obtained after embedding. This represents a vector of dimension d.

[0199] Step 63: Match the key information and semantic vector of the vulnerability with the nodes in the knowledge graph, and select the nodes with the highest matching degree as seed nodes. The specific operation is as follows:

[0200] Key information about the vulnerability and semantic embedding vectors Compare with all nodes in the previously constructed knowledge graph and select the top-K nodes as seeds;

[0201] ;

[0202] ;

[0203] in, This represents the seed set of the obtained nodes. This represents a node in a knowledge graph. This is the semantic embedding for the corresponding node. This means taking the top-k elements. This represents the relevance score between the node and the query. For matching functions, Represents the cosine similarity function. , These are weight parameters;

[0204] ;

[0205] ;

[0206] in, This is a keyword matching function; it returns 1 if two elements are equal, and 0 otherwise. This represents a set of key attributes, including function names, version numbers, and other information closely related to the vulnerability. A function to get the length of a set;

[0207] ;

[0208] in, This represents the dot product of two vectors. and Representing vectors and The L2 norm.

[0209] Step 64: Starting from the seed node, perform graph traversal in the knowledge graph to expand and obtain subgraphs related to vulnerability information. The specific operations are as follows:

[0210] The obtained seeds are then used to obtain the corresponding query subgraphs using a graph traversal algorithm. ;

[0211] ;

[0212] in, Represents the set of nodes in a subgraph. Represents the set of edges of a subgraph. For functions that extend the subgraph, This represents the seed set of the obtained nodes. The knowledge graph constructed for S5 Indicates the maximum number of rows to control the expansion depth. This represents a set of relations that can be traversed. The path decay factor, Represents the set of positive integers;

[0213] ;

[0214] ;

[0215] in, Indicates the currently visited node. Indicates the process The set of nodes that can be reached after jumping. Indicates path relevance score, To set a threshold so that only highly relevant nodes are retained, This indicates that node u is connected to node v through relation r;

[0216] ;

[0217] in, Represents seed node The initial score, From arrive One of the paths, This indicates the number of edges contained in the path. This causes the correlation to decay exponentially as the path length increases, where e represents the path length. One of the edges in, The relation is The weight of the edges can be set accordingly; for example, if the relationship described in the document is more important, then a larger weight should be set.

[0218] In a preferred embodiment of the present invention, step 7 is further included: the second agent sends the subgraph and feedback information from the third agent to the large language model to plan a test scheme. The specific operation is as follows:

[0219] pass After processing the retrieved subgraph and Feedback information is sent to the large language model Plan the corresponding test scheme.

[0220] Regarding the parameter configuration and relation weights for subgraph expansion:

[0221] (1) The set of relation types includes:

[0222] call: Function call relationship, weight θ=1.0;

[0223] contain: Class contains method relationship, weight θ=1.2;

[0224] define: The module defines the functional relationship, with a weight θ = 1.0;

[0225] describe: Document describes the relationship between codes, with a weight θ=1.5;

[0226] import: Module import dependencies, weight θ=0.8;

[0227] invoke_external: Code calls to third-party package interfaces, weight θ=0.9;

[0228] (2) h, which is the maximum number of extended hops, is set to 3 by default to balance relevance and query performance;

[0229] A, or path attenuation factor, is set to 0.85 by default;

[0230] τ_min, the path relevance threshold, is set to 0.4 by default, retaining only nodes with relevance scores higher than this threshold.

[0231] (3) Example of path relevance score calculation: If the seed node score is 1.0, after 2 hops of seed→call→function→describe→doc, the edge weight product is 1.0 × 1.2 × 1.5 = 1.8, the path decay factor is 0.85^2=0.7225, and the final path score is 1.0×0.7225×1.8=1.3005. If the score is higher than the threshold, the target node is included in the subgraph.

[0232] (4) If the same node is reachable by multiple paths, the highest score is retained; the maximum number of nodes in the subgraph is set to 500, and if it exceeds this, it is truncated according to the score.

[0233] Step 7 also includes the following steps:

[0234] Step 71: In the second agent, the retrieved subgraph is deserialized into natural language context. The specific operations are as follows:

[0235] exist In the process, based on the query subgraph obtained in step 6... Serialization yields natural language context. ;

[0236] ;

[0237] in, This is a deserialization function that can transform a query subgraph into a natural language context.

[0238] Step 72: Input the key vulnerability information, natural language context, and feedback information from the third-party agent into the large language model to generate a test plan. The specific operations are as follows:

[0239] Key information about the vulnerability The natural language context obtained as well as Feedback information Inputting into a large language model, the corresponding test plan is planned and obtained. ;

[0240] ;

[0241] in, For large language models, This indicates its parameters.

[0242] When the second agent calls the large language model to generate the test plan, the following prompt word template is used:

[0243] You are a cybersecurity testing expert. Based on the following vulnerability information, code context, and test feedback, please generate an executable test plan.

[0244] [Vulnerability Information]

[0245] {keyInfo}

[0246] [Code Context (Subgraph Deserialization Result)]

[0247] {C_q}

[0248] [Feedback from the last test]

[0249] {feedback} # If this is the first time it is generated, the content will be "none".

[0250] Require:

[0251] 1. The test plan should include: test objectives, test steps, expected results, and test data (if any).

[0252] 2. The test plan should be specific to executable commands, API requests, or script code (such as Python or curl).

[0253] 3. If feedback is received, please focus on correcting the failures pointed out in the feedback.

[0254] Output format: JSON, containing the following fields:

[0255] {

[0256] " ":"Test Name",

[0257] " Static analysis / dynamic execution / fuzz testing

[0258] " ":["Step 1","Step 2"],

[0259] " ":"Example of test input data or request",

[0260] " ":"Expected Result Description";

[0261] };

[0262] Large language models are selected that have code generation capabilities, such as GPT-4, DeepSeek-Coder, or CodeLlama-34B. The generated test plans are then sent to the sandbox for execution after structured parsing.

[0263] In a preferred embodiment of the present invention, step 8 is further included: running the test plan in a sandbox using a third intelligent agent, locating vulnerable code segments, and selecting to output vulnerability information or provide feedback information to a second intelligent agent to regenerate the test plan based on the running results. The specific operations are as follows:

[0264] pass For the project Run the generated test case in the sandbox, identify code snippets containing vulnerabilities, and evaluate the results. If the code snippet meets the criteria, output the code snippet with appropriate explanations; otherwise, provide feedback. Regenerate the test plan.

[0265] Step 8 also includes the following steps:

[0266] Step 81: In the third intelligent agent, build a sandbox environment for the software project, including a virtual system environment and testing tools. The specific operations are as follows:

[0267] exist First, a sandbox environment is built. ;

[0268] ;

[0269] in, For sandbox construction methods, Indicates project code. Indicates a virtual system environment. This indicates the set of tools used.

[0270] Step 82: Run the generated test plan in a sandbox environment to obtain the results. The specific steps are as follows:

[0271] The test plan set obtained from S7 In the constructed sandbox environment Running in the middle;

[0272] ;

[0273] in, This represents the set of results obtained from the test plan. This represents a test plan in the set of test plans.

[0274] Step S83: Evaluate whether the execution result triggers the same vulnerability as the vulnerability report. If so, locate and output the code snippet containing the vulnerability; if not, and the retry limit has not been reached, generate feedback information and send it to the second agent to trigger the regeneration of the test plan. The specific operations are as follows:

[0275] right Each result is evaluated to determine if it triggers the same vulnerability reported in the vulnerability report. If not, it is determined whether the maximum number of retries has been reached. If not, feedback is provided. Regenerate the scheme; if all are triggered, end the process and output the code snippet along with appropriate explanations.

[0276]

[0277] ;

[0278] in, To evaluate the function, assess whether it triggers the same vulnerability as reported in the vulnerability report. The test results for a specific test plan;

[0279] ;

[0280] ;

[0281] in, The number of times the scheme is regenerated. This indicates the maximum number of times a regeneration scheme can be generated. For feedback information, This is the function for generating feedback information. Indicates the results of the sandbox operation. Quantity;

[0282] ;

[0283] in, This indicates the function for locating the vulnerable code snippet. The code snippet contains the corresponding vulnerability. This represents the test result for the j-th test plan.

[0284] The logic and format for generating feedback information are as follows:

[0285] (1) Evaluation function Specific implementation:

[0286] If a test plan results in a crash, abnormal exit, privilege escalation, or return of unexpected data, it is considered to be "potentially triggering a vulnerability".

[0287] Compare the execution results with the vulnerability characteristics described in the vulnerability report (such as error type, stack information, and scope of impact), and use LLM to determine whether there is a match;

[0288] If the match is greater than or equal to 0.7 (the confidence score is output by LLM), then Assess = 1; otherwise, Assess = 0.

[0289] (2) Logic for generating feedback information:

[0290] Collect test results for all vulnerabilities that were not triggered. ;

[0291] For each failed result, extract:

[0292] Test plan content;

[0293] Execution log (only retaining error or exception sections);

[0294] Missing conditions (such as environment variables not set, missing dependencies, insufficient permissions, etc.);

[0295] Use LLM to generate structured feedback; the template is as follows:

[0296] The following test scenario failed to trigger the vulnerability. Please analyze the reason and make adjustments:

[0297] Test plan: {testCase};

[0298] Execution result summary: {execution_summary};

[0299] Possible reason: {likely_reason};

[0300] Suggested direction for revision: {suggestion};

[0301] The feedback from multiple failures is merged into a single feedback message and returned to the second agent.

[0302] (3) The maximum number of retries, retry_max, is set to 3 by default to avoid infinite loops.

[0303] Example 2:

[0304] Please see Figure 2 and Figure 3 Based on Example 1, this example provides an intelligent tracing and automated detection system for vulnerabilities in power information systems based on LLM, including:

[0305] The knowledge graph fusion building module is used to analyze and fuse source code, developer documentation, and package management files to build a knowledge graph with semantic embedding.

[0306] The vulnerability information acquisition module is used to acquire vulnerability information and extract key information;

[0307] The vulnerability retrieval module is used to retrieve relevant subgraphs in the knowledge graph based on vulnerability information;

[0308] The test plan planning module is used to generate test plans by calling the large language model based on the retrieved subgraphs and feedback information.

[0309] The test execution and feedback module is used to execute test plans in a sandbox environment, evaluate results, and generate feedback information.

[0310] The above description is merely a preferred embodiment of the present invention; however, the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and its improved concepts, should be covered within the scope of protection of the present invention.

Claims

1. A method for intelligent tracing and automated detection of vulnerabilities in power information systems based on LLM, characterized in that, include: Step 1: Obtain the software project in the power information system. The software project includes source code, package management files, and developer documentation. Step 2: Perform structured analysis on the source code and developer documentation to obtain the corresponding code structure nodes and edges, as well as document structure nodes and edges. Code structure nodes include at least function nodes, class nodes, and module nodes, and code structure edges include at least edges representing function call relationships, class-function inclusion relationships, and module-function definition relationships. Document structure nodes include document fragment text, format, and associated code entity identifiers, and document structure edges represent the descriptive relationship between code and documentation. Step 3: Merge the code structure nodes and edges with the document structure nodes and edges to obtain a project-level set of nodes and edges; Step 4: Parse the package management file to obtain the set of dependency nodes and the set of dependency edges; the attributes of the dependency nodes include the package name, version number and source address, and the dependency edges represent the relationship between the code calling the interface of the external dependency package; Step 5: Integrate the dependency node set and dependency edge set into the project-level node set and edge set to construct a knowledge graph, and perform semantic embedding on the nodes in the knowledge graph to obtain a graph-enhanced retrieval library; Step 6: Obtain vulnerability information through the first intelligent agent, extract key information from the vulnerability information using a large language model, and perform subgraph retrieval based on the key information and the graph-enhanced retrieval library to obtain the corresponding subgraph. Step 7: The second agent sends the subgraph and feedback information from the third agent to the large language model to plan the test scheme; Step 8: Run the test plan in the sandbox through a third intelligent agent, locate the vulnerable code fragment, and choose to output vulnerability information or provide feedback information to the second intelligent agent to regenerate the test plan based on the running results.

2. The method for intelligent tracing and automated detection of vulnerabilities in power information systems based on LLM as described in claim 1, characterized in that, Step 2 includes: Step 21: Parse the source code file for functions, extract function nodes, and set node attributes; Step 22: Perform class parsing on the source code file, extract class nodes, and set node attributes; Step 23: Perform module parsing on the source code file, extract module nodes, and set node attributes; Step 24: Based on the extracted function nodes, class nodes, and module nodes, perform code structure analysis to obtain an edge set containing function call relationships, class-function inclusion relationships, and module-function definition relationships.

3. The method for intelligent tracing and automated detection of vulnerabilities in power information systems based on LLM as described in claim 2, characterized in that, Step 2 also includes: Step 25: Parse the developer documentation, extract the document nodes related to the source code file, and set the node attributes, including the document fragment text, format, and associated code entity identifiers. Step 26: Analyze the extracted document nodes and code structure nodes to obtain a set of edges representing the descriptive relationship between the code and the document.

4. The method for intelligent tracing and automated detection of vulnerabilities in power information systems based on LLM as described in claim 1, characterized in that, Step 3 includes: Step 31: Merge the set of code structure nodes and the set of document nodes corresponding to a single source code file to obtain the merged node set of the file; Step 32: Merge the code structure edge set and document edge set corresponding to a single source code file to obtain the fused edge set of that file; Step 33: Merge the set of nodes for all files to obtain the set of nodes for the entire project; Step 34: Merge the edge sets of all files to obtain the edge set of the entire project.

5. The method for intelligent tracing and automated detection of vulnerabilities in power information systems based on LLM as described in claim 1, characterized in that, Step 4 includes: Step 41: Parse the package management file, extract the dependency package nodes, and set the package name, version number, source address, and license attributes for each dependency package node; Step 42: Perform structural analysis on the package management file, dependency package nodes, and project code nodes to obtain a set of edges representing the dependency relationships of code calling external interfaces.

6. The method for intelligent tracing and automated detection of vulnerabilities in power information systems based on LLM as described in claim 1, characterized in that, Step 5 includes: Step 51: Merge the dependency node set with the project-level node set to obtain the final knowledge graph node set; Step 52: Merge the dependency edge set with the project-level edge set to obtain the final knowledge graph edge set; Step 53: For each node in the knowledge graph, generate a semantic embedding vector based on its code content and documentation, and store it in a vector database to build a graph-enhanced retrieval library.

7. The method for intelligent tracing and automated detection of vulnerabilities in power information systems based on LLM as described in claim 1, characterized in that, Step 6 includes: Step 61: Obtain vulnerability information from the vulnerability release platform through the first intelligent agent, and extract key vulnerability information using a large language model; Step 62: Semantically embed the obtained vulnerability information to obtain a vulnerability semantic vector; Step 63: Match the key information and semantic vector of the vulnerability with the nodes in the knowledge graph, and select the nodes with the highest matching degree as seed nodes. Step 64: Starting from the seed node, perform graph traversal in the knowledge graph to expand and obtain subgraphs related to vulnerability information.

8. The method for intelligent tracing and automated detection of vulnerabilities in power information systems based on LLM as described in claim 1, characterized in that, Step 7 includes: Step 71: In the second agent, the retrieved subgraph is deserialized into a natural language context. Step 72: Input the key vulnerability information, natural language context, and feedback information from the third agent into the large language model to generate a test plan.

9. The method for intelligent tracing and automated detection of vulnerabilities in power information systems based on LLM as described in claim 1, characterized in that, Step 8 includes: Step 81: In the third intelligent agent, build a sandbox environment for the software project that includes a virtual system environment and testing tools; Step 82: Run the generated test plan in a sandbox environment and obtain the results; Step 83: Evaluate whether the running result triggers the same vulnerability as the vulnerability report. If so, locate and output the code snippet containing the vulnerability. If not, and the maximum number of retries has not been reached, generate feedback information and send it to the second agent to trigger the regeneration of the test plan.

10. An LLM-based intelligent vulnerability tracing and automated detection system for power information systems, used to implement the LLM-based intelligent vulnerability tracing and automated detection method for power information systems as described in any one of claims 1 to 9, characterized in that, include: The knowledge graph fusion building module is used to analyze and fuse source code, developer documentation, and package management files to build a knowledge graph with semantic embedding. The vulnerability information acquisition module is used to acquire vulnerability information and extract key information; The vulnerability retrieval module is used to retrieve relevant subgraphs in the knowledge graph based on vulnerability information; The test plan planning module is used to generate test plans by calling the large language model based on the retrieved subgraphs and feedback information. The test execution and feedback module is used to execute test plans in a sandbox environment, evaluate results, and generate feedback information.

Images