A graph-driven self-attention compressible memory management method

Abstract

This application discloses a graph-driven self-attention compressible memory management method, belonging to the field of penetration testing technology. This application constructs an attention graph and uses edge weights to accurately characterize the semantic dependencies and attention flows between multiple agent nodes, improving the relevance and accessibility of information. In terms of filtering and loading, this application filters key memory fragments based on the edge weights of the attention graph and task-related features, loading only useful information, effectively eliminating redundancy, reducing the amount of data processed by the system, and improving system operating efficiency. When the memory content exceeds the input context window limit of a large model, the semantic compression module can perform summary processing on historical information based on multiple factors. While ensuring that key semantic information is not lost, the summary length is controlled within the window limit, ensuring that the model can process information normally, and improving the system's adaptability and stability to different data volumes.

Description

Technical Field

[0001] This invention belongs to the field of penetration testing technology, specifically relating to a graph-driven self-attention compressible memory management method. Background Technology

[0002] In penetration testing, multi-agent collaboration requires processing a large amount of historical interaction information, including dialogue logs and tool call traces. These historical memory fragments are of significant reference value for the execution of the current task.

[0003] However, traditional memory management methods have the following problems:

[0004] 1. When constructing the memory structure, it is impossible to effectively characterize the complex semantic dependencies and attention flows between multiple agent nodes, resulting in poor information correlation and difficulty in accurately retrieving key information.

[0005] 2. When faced with massive amounts of memory fragments, the lack of an efficient filtering and loading mechanism will introduce a large amount of irrelevant and redundant information into the processing flow, which will not only increase the system burden, but may also interfere with the execution of the current task and reduce efficiency.

[0006] 3. When the loaded memory content exceeds the input context window limit of the large model, the existing processing method often adopts the method of directly truncating the content, which will cause information loss or a significant drop in model performance, and cannot effectively compress the content while ensuring information integrity. Summary of the Invention

[0007] To address the problems mentioned in the background, this application provides a graph-driven self-attention compressible memory management method to solve the problems of poor information correlation, difficulty in accurately retrieving key information, lack of efficient filtering and loading mechanisms, and information loss or significant degradation of model performance caused by simple compression in traditional penetration testing tasks.

[0008] To achieve the above objectives, the present invention provides the following technical solution:

[0009] A graph-driven self-attention compressible memory management method includes the following steps:

[0010] S1: Construct an attention graph; Using each round of messages from multi-agent interactions in a penetration testing task as nodes, construct an attention graph with a dynamic directed graph structure. The messages include dialogue records and tool call data. The edge weights of the attention graph represent the attention dependencies between different nodes. The edge weights are determined by analyzing the semantic correlation and interaction logic flow between nodes, and are used to characterize the semantic dependencies and attention flow between multi-agent nodes. The messages corresponding to the nodes constitute historical memory fragments.

[0011] S2: Targeted retrieval and selective loading; Based on the edge weights of the attention graph, historical memory fragments strongly related to the current penetration testing task are selected from the attention graph, namely key memory fragments. The strong correlation is matched according to the vulnerability type, attack stage and tool characteristics of the current task. The key memory fragments are loaded and irrelevant redundant information is removed. If the total length of the loaded memory fragment content exceeds the preset message length threshold of the large model, then S3 is executed. If it does not exceed the threshold, then S4 is executed directly.

[0012] S3: Adaptive handling of context window overflow; Enable semantic compression module, based on core elements including task intent, tool calls and historical response content, to perform summary processing on key memory fragments loaded by S2, extract key semantic information and generate compressed summaries;

[0013] S4: Multi-scenario adaptation optimization; Based on the memory content loaded in S2 or the compressed summary generated in S3, select to perform multi-parameter tool call optimization and multi-Agent collaborative optimization. Specifically, multi-Agent collaborative optimization involves passing relevant information of agents responsible for different tasks through an attention graph, ensuring that each agent shares the necessary context and performs its work based on the shared information.

[0014] Preferably, in S1, when a new interactive message is added to the attention graph as a node, the edge weights are updated by calculating the semantic similarity between the new node and the existing nodes. The semantic similarity calculation adopts the vector cosine similarity algorithm based on the pre-trained language model. The edge weights are dynamically adjusted according to the new semantic association, and the adjustment range is positively correlated with the semantic similarity.

[0015] Preferably, in S2, the tool features include tool type, tool function, and tool call history parameters. The tool type includes vulnerability scanning tools, vulnerability exploitation tools, and privilege escalation tools. The tool function is divided into information gathering, vulnerability detection, and attack implementation categories according to its role in the penetration testing process. The tool call history parameters include parameter name, parameter value range, and parameter combination mode.

[0016] Preferably, in S3, the semantic compression module performs semantic parsing and content refinement of historical information through a natural language processing model.

[0017] Preferably, in the multi-parameter tool call optimization in S4, tool parameter descriptions are added to the memory content loaded in S2 or the compressed summary generated in S3, thereby enhancing the model's understanding of tool usage.

[0018] Preferably, in the multi-agent collaborative optimization of S4, the relevant information includes task progress, tool call results and permission status. The task progress is presented as a percentage to show the current task completion status and the progress status of each subtask. The tool call results include whether the call was successful, the returned data and error codes. The permission status records the currently acquired system permission level and permission scope.

[0019] Preferably, in S1, the node is uniquely identified. The identification information includes the timestamp of the message generation, the number of the sending Agent, and the type of the message. The number of the sending Agent is a unique identifier assigned by the system. The node's attribute information also includes the message type, message length, and associated tool identifier.

[0020] Preferably, in S3, the generated compressed digest includes index information, which includes the unique identifier of the original node, the storage path, and the generation time.

[0021] Preferably, in S4, when multiple agents are working together for optimization, an information synchronization mechanism is established between agents, and message distribution and reception are uniformly processed through a publish-subscribe pattern.

[0022] Compared with the prior art, the beneficial effects of the present invention are:

[0023] This application constructs an attention graph and uses edge weights to accurately characterize the semantic dependencies and attention flows between multiple agent nodes. This enables better organization and understanding of historical memory fragments, providing a solid data foundation for subsequent retrieval and applications, and improving the relevance and accessibility of information. In terms of filtering and loading, this application filters key memory fragments based on the edge weights of the attention graph and task-related features, loading only useful information, effectively eliminating redundancy, reducing the amount of data processed by the system, improving system efficiency, and avoiding interference from irrelevant information to the current task, thus improving the accuracy of task execution. When the memory content exceeds the input context window limit of a large model, the semantic compression module can perform summary processing on historical information based on multiple factors. While ensuring that key semantic information is not lost, the summary length is controlled within the window limit, ensuring that the model can process information normally, and improving the system's adaptability and stability to different data volumes. Attached Figure Description

[0024] Figure 1 This is a flowchart illustrating the process of this application. Detailed Implementation

[0025] To facilitate understanding of the technical content of this invention by those skilled in the art, the invention will be further described in detail below with reference to the accompanying drawings and specific examples. It should be understood that the specific examples described herein are merely illustrative and not intended to limit the scope of the invention.

[0026] Example 1

[0027] A graph-driven self-attention compressible memory management method, such as Figure 1 As shown, it includes the following steps:

[0028] S1: Construct an attention graph; Using each round of messages from multi-agent interactions in a penetration testing task as nodes, construct an attention graph with a dynamic directed graph structure. The messages include dialogue records and tool call data. The edge weights of the attention graph represent the attention dependencies between different nodes. The edge weights are determined by analyzing the semantic correlation and interaction logic flow between nodes, and are used to characterize the semantic dependencies and attention flow between multi-agent nodes. Nodes and their corresponding messages together constitute historical memory fragments.

[0029] S2: Targeted retrieval and selective loading; Based on the edge weights of the attention graph, historical memory fragments strongly related to the current penetration testing task are selected from the attention graph, namely key memory fragments. The strong correlation is matched according to the vulnerability type, attack stage and tool characteristics of the current task. The key memory fragments are loaded and irrelevant redundant information is removed. If the total length of the loaded memory fragment content exceeds the input context window limit of the large model, then S3 is executed. If it does not exceed the limit, then S4 is executed directly.

[0030] S3: Adaptive handling of context window overflow; Enable semantic compression module, based on core elements including task intent, tool calls and historical response content, to perform summary processing on key memory fragments loaded by S2, extract key semantic information and generate compressed summaries;

[0031] S4: Multi-scenario adaptation optimization; Based on the memory content loaded in S2 or the compressed summary generated in S3, select to perform multi-parameter tool call optimization and multi-Agent collaborative optimization. Specifically, multi-parameter tool call optimization reuses historical parameter call records and combines them with the parameter recommendation and adaptive combination supported by the current test environment. Specifically, multi-Agent collaborative optimization transmits relevant information of agents responsible for different tasks through attention graphs to ensure that each agent shares the necessary context and performs its work based on the shared information.

[0032] In this embodiment, this application constructs an attention graph and uses edge weights to accurately characterize the semantic dependencies and attention flows between multiple agent nodes. This enables better organization and understanding of historical memory fragments, providing a solid data foundation for subsequent retrieval and application, and improving the relevance and accessibility of information. In terms of filtering and loading, this application filters key memory fragments based on the edge weights of the attention graph and task-related features, loading only useful information, effectively eliminating redundancy, reducing the amount of data processed by the system, improving system operating efficiency, and avoiding interference from irrelevant information to the current task, thus improving the accuracy of task execution. When the memory content exceeds the input context window limit of the large model, the semantic compression module can perform summary processing on historical information based on multiple factors. While ensuring that key semantic information is not lost, the summary length is controlled within the window limit, ensuring that the model can process information normally, and improving the system's adaptability and stability to different data volumes.

[0033] Example 2

[0034] The difference between this embodiment and embodiment 1 is that in S1, when a new interactive message is added to the attention graph as a node, the edge weights are updated by calculating the semantic similarity between the new node and the existing nodes. The semantic similarity calculation adopts the vector cosine similarity algorithm based on the pre-trained language model. The edge weights are dynamically adjusted according to the new semantic association, and the adjustment range is positively correlated with the semantic similarity.

[0035] In this embodiment, the pre-trained language model uses the BERT-base model, which converts the node message content into a multi-dimensional vector representation. The vector cosine similarity is calculated through matrix operations, with the result accurate to four decimal places. The initial edge weight is set to 0.5. When a new node is added, after calculating the semantic similarity with all existing nodes, the edge weights of the new node and all other nodes are updated using the following formula:

[0036] New edge weight = old edge weight + (semantic similarity - 0.5) × 0.2;

[0037] Ensure that the adjustment range is positively correlated with semantic similarity and that the edge weights always remain within the range of 0-1.

[0038] Example 3

[0039] The difference between this embodiment and embodiment 1 is that in S2, the tool features include tool type, tool function and tool call history parameters. The tool type includes vulnerability scanning tools, vulnerability exploitation tools and privilege escalation tools. The tool function is divided into information collection, vulnerability detection and attack implementation categories according to its role in the penetration testing process. The tool call history parameters include parameter name, parameter value range and parameter combination mode.

[0040] Example 4

[0041] The difference between this embodiment and embodiment 1 is that in S3, the semantic compression module performs semantic parsing on historical information using a natural language processing model.

[0042] In this embodiment, the natural language processing model adopts the spaCy framework. Entity extraction includes hostnames, IP addresses, vulnerability numbers, etc., while relation extraction covers associations such as "vulnerability exists" and "invoking tools." Key action extraction includes behavioral verbs such as "scan" and "attack." When filtering core elements based on task intent, a combination of keyword matching and semantic relevance ranking is used. The core execution logic is retained in the form of a step-by-step list. The length of the compressed summary is controlled by token counting. The preset semantic similarity threshold is set to 0.8, and the ROUGE-L algorithm is used to calculate similarity.

[0043] Example 5

[0044] The difference between this embodiment and Embodiment 1 lies in that, in the multi-parameter tool call optimization in S4, the recommended tool parameter descriptions are based on the representation vector of the current penetration context information. Cosine similarity is calculated with other message nodes, historical messages with high similarity are retrieved, and neighboring messages with high similarity are recursively retrieved. The penetration context information includes the target system's environmental information, the execution information of the previous tool instruction, the penetration target information output by the Agent at the current penetration time step, and the global final target information. The recursion depth is determined by the configuration file. This method dynamically retrieves different historical context information based on the current penetration environment to assist the Agent in decision-making.

[0045] Example 6

[0046] The difference between this embodiment and embodiment 1 is that in the multi-Agent collaborative optimization of S4, the relevant information includes task progress, tool call results and permission status. The task progress is presented as a percentage to show the current task completion status and the progress status of each subtask. The tool call results include whether the call was successful, the returned data and the error code. The permission status records the currently acquired system permission level and permission range.

[0047] Example 7

[0048] The difference between this embodiment and embodiment 1 is that in S1, nodes are uniquely identified. The identification information includes the timestamp of message generation, the number of the sending Agent, and the type of message. For example, a tool expert can generate tool instruction messages, tool result messages, and tool expert Base reply messages. The number of the sending Agent is a unique identifier assigned by the system. The attribute information of the node also includes message type, message length, and associated tool identifier.

[0049] Example 8

[0050] The difference between this embodiment and embodiment 7 is that in S3, the generated compressed digest includes index information, which includes the unique identifier of the original node, the storage path, and the generation time.

[0051] Example 9

[0052] The difference between this embodiment and embodiment 1 is that in S4, when multiple agents are working together for optimization, an information synchronization mechanism is established between agents. Information is transmitted through a subscription-publishing model. Agents subscribe to relevant information topics according to their own responsibilities and automatically receive the information when it is updated.

[0053] In this embodiment, the publish-subscribe pattern is implemented based on the MQTT protocol. The information topics are divided into a hierarchical structure of "task type / information category", such as "penetration testing / permission status". The correspondence between the agent's responsibilities and the information topics is set through a configuration file and can be dynamically modified.

[0054] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A graph-driven self-attention compressible memory management method, characterized in that, Includes the following steps: S1: Construct an attention graph; Using each round of messages from multi-agent interactions in a penetration testing task as nodes, construct an attention graph with a dynamic directed graph structure. The messages include dialogue records and tool call data. The edge weights of the attention graph represent the attention dependencies between different nodes. The edge weights are determined by analyzing the semantic correlation and interaction logic flow between nodes, and are used to characterize the semantic dependencies and attention flow between multi-agent nodes. The messages corresponding to the nodes constitute historical memory fragments. S2: Targeted retrieval and selective loading; Based on the edge weights of the attention graph, historical memory fragments strongly related to the current penetration testing task are selected from the attention graph, namely key memory fragments. The strong correlation is matched according to the vulnerability type, attack stage and tool characteristics of the current task. The key memory fragments are loaded and irrelevant redundant information is removed. If the total length of the loaded memory fragment content exceeds the preset message length threshold of the large model, then S3 is executed. If it does not exceed the threshold, then S4 is executed directly. Tool characteristics include tool type, tool function, and tool call history parameters; S3: Adaptive handling of context window overflow; Enable semantic compression module, based on core elements including task intent, tool calls and historical response content, to perform summary processing on key memory fragments loaded by S2, extract key semantic information and generate compressed summaries; S4: Multi-scenario adaptation optimization; Based on the memory content loaded in S2 or the compressed summary generated in S3, select to perform multi-parameter tool call optimization and multi-Agent collaborative optimization. Specifically, multi-Agent collaborative optimization involves passing relevant information of agents responsible for different tasks through an attention graph, ensuring that each agent shares the necessary context and performs its work based on the shared information.

2. The graph-driven self-attention compressible memory management method according to claim 1, characterized in that, In S1, when a new interactive message is added to the attention graph as a node, the edge weights are updated by calculating the semantic similarity between the new node and the existing nodes. The semantic similarity calculation adopts the vector cosine similarity algorithm based on the pre-trained language model. The edge weights are dynamically adjusted according to the new semantic association, and the adjustment magnitude is positively correlated with the semantic similarity.

3. The graph-driven self-attention compressible memory management method according to claim 1, characterized in that, In S2, tool types include vulnerability scanning tools, vulnerability exploitation tools, and privilege escalation tools. Tool functions are categorized into information gathering, vulnerability detection, and attack execution based on their role in the penetration testing process. Tool call history parameters include parameter names, parameter value ranges, and parameter combination patterns.

4. The graph-driven self-attention compressible memory management method according to claim 1, characterized in that, In S3, the semantic compression module performs semantic parsing and content refinement of historical information using a natural language processing model.

5. The graph-driven self-attention compressible memory management method according to claim 1, characterized in that, In the multi-parameter tool call optimization of S4, tool parameter descriptions are added to the memory content loaded in S2 or the compressed summary generated in S3, thereby enhancing the model's understanding of tool usage.

6. The graph-driven self-attention compressible memory management method according to claim 1, characterized in that, In S4's multi-agent collaborative optimization, relevant information includes task progress, tool call results, and permission status. Task progress is presented as a percentage to show the current task completion status and the progress status of each subtask. Tool call results include whether the call was successful, the returned data, and the error code. Permission status records the currently acquired system permission level and permission scope.

7. The graph-driven self-attention compressible memory management method according to claim 1, characterized in that, In S1, each node is uniquely identified. The identification information includes the timestamp of the message generation, the ID of the sending Agent, and the message type. The ID of the sending Agent is a unique identifier assigned by the system. The node's attribute information also includes the message type, message length, and associated tool identifier.

8. The graph-driven self-attention compressible memory management method according to claim 7, characterized in that, In S3, the generated compressed digest includes index information, which includes the unique identifier of the original node, the storage path, and the generation time.

9. The graph-driven self-attention compressible memory management method according to claim 1, characterized in that, In S4, when multiple agents collaborate for optimization, an information synchronization mechanism is established between agents, and message distribution and reception are uniformly processed through a publish-subscribe pattern.

Images