A threat hunting method based on large language model and reinforcement learning

Abstract

The present invention proposes a threat hunting method based on a large language model and reinforcement learning, which belongs to the field of network security technology. First, the initial threat entity is extracted from the threat intelligence to form an initial threat node set, and a vector-text retrieval database is constructed. Then, the optimal threat node is selected from the current threat node set through a reinforcement learning algorithm. In combination with the large language model and the vector-text retrieval database, the optimal threat hunting method is selected according to the cold start strategy or the upper confidence bound variant strategy, solving the problem of knowledge scarcity in the cold start phase of threat hunting. Finally, by recursively updating the node quality and updating the method quality based on similarity, the sparse reward problem existing in the threat node selection and threat hunting method selection process is solved. The present invention solves the problems of slow convergence speed and cold start difficulty of the existing technology in threat hunting scenarios, and realizes intelligent and efficient threat hunting.

Description

Technical Field

[0001] The present invention relates to the field of network security technology, and in particular to a threat hunting method based on a large language model and reinforcement learning. Background Art

[0002] With the rapid development and widespread adoption of network technology, the security threats facing global networks are also increasing. In today's complex and volatile cybersecurity environment, cyberattack methods are becoming increasingly diverse and sophisticated, with sophisticated attack methods such as advanced persistent threats emerging in an endless stream. Traditional passive defense measures are no longer effective against these sophisticated attacks. Against this backdrop, threat hunting, as a proactive defense strategy, has gradually become a key research direction in the cybersecurity field. Threat hunting proactively searches for and analyzes anomalous activity and indicators within a network, enabling timely detection and response to threats before they cause significant impact. Threat hunting can uncover unknown threat entities from limited known threat information. Threat hunting methods based on threat intelligence use natural language processing to identify unknown threats, but they struggle to address the timeliness of threat intelligence text.

[0003] To address these issues, reinforcement learning, as an adaptive decision-making method, has begun to be applied to threat hunting. By learning optimal strategies through interaction with the environment, the efficiency of threat hunting is improved. Reinforcement learning-based threat hunting methods can address the lack of timeliness of threat intelligence in traditional threat intelligence-based threat hunting methods. However, because such methods require the introduction of a large number of external data sources and different data acquisition methods, their query costs are high and their query efficiency is low. Current reinforcement learning-based threat hunting methods still face the following challenges in practical applications:

[0004] 1) Slow convergence: In threat hunting, there are scenarios where positive feedback is relatively rare, resulting in inefficient training of reinforcement learning models and slow convergence, leading to low threat hunting efficiency.

[0005] 2) Cold Start Difficulty: When using reinforcement learning-based threat hunting algorithms, the model needs a long time to interact with the environment before learning effective strategies when there is insufficient initial experience and data. This cold start problem leads to low efficiency in the initial stages of threat hunting. Summary of the Invention

[0006] To address the above issues, the present invention aims to provide a threat hunting method based on a large language model and reinforcement learning. By leveraging the knowledge base and zero-shot learning capabilities of the large language model, combined with the strategy optimization and adaptive learning characteristics of reinforcement learning, this method addresses the slow convergence and low cold start efficiency issues of existing technologies, achieving highly efficient threat hunting. The technical solution is as follows:

[0007] A threat hunting method based on a large language model and reinforcement learning includes the following steps:

[0008] Step 1: Context Building

[0009] Extract initial threat entities from threat intelligence to form an initial threat hunting node set, convert threat intelligence text into high-dimensional vectors, and establish a vector-text retrieval database;

[0010] Step 2: Threat Hunting Node and Method Selection

[0011] Based on the current threat hunting node set, the optimal threat node is selected through the reinforcement learning algorithm, and then the threat hunting method corresponding to the optimal threat node is selected to obtain the current optimal threat hunting method; if the method selection process of the current node is in the cold start phase, the external knowledge provided by the large language model is used to assist the reinforcement learning algorithm for method selection; if the method selection process of the current node is not in the cold start phase, the upper confidence bound variant strategy is used for method selection; finally, the selected threat hunting method is executed to generate a new child node set;

[0012] Step 3: Update node quality and method quality

[0013] By recursively updating the node quality and using similarity to update the method quality, the sparse reward problem is solved, and finally the next threat hunting node set is generated.

[0014] Furthermore, in step 1, the initial threat hunting node set is N0; in the initial stage, let the current threat hunting node set, that is, the Tth threat hunting node set N t is the initial threat hunting node set, that is, N t =N0;

[0015] Preprocess the threat intelligence text and segment it into original text segments T′ by sentence i , each original text segment T′ i Use word embedding models to convert text into high-dimensional vectors and represent them as V i ;Build vector-text retrieval database DB Context ={ <V i ,T′ i >}.

[0016] Furthermore, the step 2 is specifically as follows:

[0017] Step 2.1: Based on the T-th threat hunting node set N t The expansion is carried out, and the threat hunting node selection is performed according to the following formula to obtain the current optimal threat node n i :

[0018]

[0019] Where Q t (n) represents the node quality of the corresponding threat node n. When initialized, the node quality Q corresponding to all threat nodes t (n) is 0; P(n) represents the parent node of node n; N(n) represents the number of times the corresponding threat node n is selected as the optimal threat node in the historical threat hunting process; c is a hyperparameter for balancing exploration and exploitation;

[0020] Step 2.2: Select the threat hunting method and obtain the current optimal threat hunting method a j ;

[0021] If n i ∈N0, then the current optimal threat node n i The method selection is set to the cold start stage, otherwise it is the non-cold start stage;

[0022] If the current optimal threat node n i The method selection process is in the cold start stage, so the cold start method selection strategy is used to introduce the external knowledge provided by the large language model to assist the reinforcement learning algorithm to select the method, and the current optimal threat hunting method a is obtained. j :

[0023]

[0024] Where k i is the current optimal threat node n i Node type, such as hash, IP, domain name, etc. i is the current optimal threat node n i A collection of optional threat hunting methods; is the method quality matrix W t The elements in the node type k i The quality of using method a; the method quality matrix W t is an R×M matrix, where R represents the number of threat node types and M represents the total number of threat hunting methods; W t At initialization, all elements are set to 0; N(a) is the number of times method a is selected, LLMScore(a,n i ) is the current optimal threat node n calculated by the large language model i Applicability score with method a;

[0025] If the current optimal threat node n i The method selection process is a non-cold start stage, and the upper confidence bound variant strategy is used for method selection, as shown in the following formula:

[0026]

[0027] Where, N(A i ) is the current optimal threat node n i The total number of visits to the method collection;

[0028] Step 2.3: Execute the selected optimal threat hunting method a through reinforcement learning algorithm j , generate a new threat sub-node set C t+1 .

[0029] Furthermore, in step 2.2, the large language model is used to calculate the current optimal threat node n i The applicability score with method a is as follows:

[0030] LLMScore(a,n i )=LLM(Context(n i ),Description(a))

[0031] In the formula, context (n i ) represents the retrieval of the vector-text database DB Context The node that is found in the current optimal threat node n i Relevant contextual information and the corresponding vector representation; Description(a) is the description of method a; LLM(·) represents the reasoning process of the large language model, and outputs method a at the current optimal threat node n i Ratings on

[0032] Extract Context(n i ) is implemented as follows: the vector database FAISS is used for retrieval, cosine similarity is used as the similarity metric, and the Top-K strategy is adopted to select the most relevant content from the top K context fragments with the highest similarity. The default value of K is 3 and can be adjusted according to the algorithm configuration to achieve accurate matching and rapid extraction of context fragments related to threat nodes.

[0033] Furthermore, step 3 is as follows:

[0034] Step 3.1: Update the threat node set N t+1 =N t ∪C t+1 , and record the parent nodes of all newly generated threat nodes n′ in turn, and the current optimal threat node n i Update to the parent node of all newly generated threat nodes n′:

[0035]

[0036] Step 3.2: For each newly generated node n′, calculate the incentive ΔQ based on the feedback information t (n′), and traverse all newly generated nodes n′, and update the relevant node quality Q t (n):

[0037]

[0038] Where ΔQ t (n′) is the incentive calculated based on the feedback information, Ancestors(n′) represents the set of all ancestor nodes of the newly generated node n′;

[0039] Step 3.3: Update the method quality:

[0040] Using the method similarity function Sim method , the method quality matrix W t To update:

[0041]

[0042] Where A represents the set of all threat hunting methods; a j is the current optimal threat node n i The current best threat hunting method selected on Sim method (a,a j ) is method a and method a j Similarity measure between .

[0043] Furthermore, the node quality Q t In the update of (n), the incentive ΔQ t The strategy of (n′) is: for any newly generated node n′∈C t+1 , if n′ does not belong to the current threat hunting node set N t , then the incentive is 1, otherwise the incentive is 0.

[0044] Furthermore, the method similarity function Sim method (a,a j ) is calculated as follows: for any two methods a, a j If they share the same data source, the similarity is increased by 0.1; if they share the same operation, the similarity is increased by 0.5.

[0045] The beneficial effects of the present invention are:

[0046] 1) Based on a reinforcement learning framework, this invention leverages the knowledge base and zero-shot learning capabilities of a large language model, combined with the strategy optimization and adaptive learning characteristics of reinforcement learning, to improve the efficiency of threat hunting.

[0047] 2) The present invention can solve the problem of sparse rewards by introducing recursive incentives and similarity-based quality updates in scenarios with slow convergence. In cold start scenarios, it uses a large language model to introduce external knowledge to improve cold start efficiency, thereby realizing an efficient threat hunting method. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 This is a flow chart of the threat hunting method of the present invention.

[0049] Figure 2 A flow diagram is constructed for the context of the present invention.

[0050] Figure 3 Schematic diagram of the threat hunting node and method selection process of the present invention.

[0051] Figure 4 Schematic diagram of the node quality and method quality update process of the present invention.

[0052] Figure 5 This is a diagram showing the experimental evaluation effect of the present invention. DETAILED DESCRIPTION

[0053] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0054] This invention proposes a threat hunting method based on a large language model and reinforcement learning. Figure 1 As shown, the present invention mainly includes three steps: context construction, threat hunting node and method selection, and node quality and method quality update, as well as the T-th threat hunting node set and the T+1-th threat hunting node set as outputs of different steps.

[0055] 1. Context Building

[0056] The context building steps are as follows Figure 2 As shown, first, the initial threat entity is extracted from the threat intelligence to form the initial threat node set N0. At the initial stage, let the current threat node set N t is the initial threat node set, that is, N t = N0. Preprocess the threat intelligence text T0 and segment it into original text segments T′ by sentence. i , each T′ i Use word embedding models to convert text into high-dimensional vectors represented as V i , build vector-text retrieval database DB Context ={ <V i ,T′ i >}.

[0057] Preferably, BERT is used as the word embedding model.

[0058] 2. Threat Hunting Node and Method Selection

[0059] Threat hunting node and method selection steps are as follows Figure 3 As shown. Based on the T-th threat hunting node set N t The expansion is carried out, and the threat hunting node selection is performed according to the following formula to obtain the current optimal threat node n i :

[0060]

[0061] Where Q t (n) represents the node quality of the corresponding threat node n. When initialized, all threat nodes correspond to Q t (n) is 0; P(n) represents the parent node of node n; N(n) represents the number of times the corresponding threat node n is selected as the optimal threat node in the historical threat hunting process; c is a hyperparameter for balancing exploration and exploitation.

[0062] Then, the threat hunting method is selected to obtain the current optimal threat hunting method a j If n i ∈N0, then the current node n i The method selection is set to the cold start stage, otherwise it is the non-cold start stage.

[0063] If the current node n i The method selection process is the cold start stage, so the cold start method selection strategy is used to introduce the external knowledge provided by the large language model to assist the reinforcement learning algorithm to select the current optimal threat hunting method a j :

[0064]

[0065] Where k i For node n i Node type, such as hash, IP, domain name, etc. i For the current threat node n i In terms of the set of optional threat hunting methods, the method quality matrix W t is an R×M matrix, where R represents the number of threat node types and M represents the total number of threat hunting methods; W t At initialization, all elements are set to 0; the elements of the matrix Indicates the quality of using method a on node type k. N(a) is the number of times threat hunting method a is selected in historical calls. LLMScore(a,n i ) is the node n calculated by the large language modeli Applicability score with method a.

[0066] Use the large language model to calculate the current selected node n i Applicability score with method a:

[0067] LLMScore(a,n i )=LLM(Context(n i ),Description(a))

[0068] In the formula, context (n i ) represents the retrieval of the vector-text database DB Context The node that is found in the current optimal threat node n i Relevant contextual information and the corresponding vector representation; Description(a) is the description of method a; LLM(·) represents the reasoning process of the large language model, and outputs method a at the current optimal threat node n i Ratings on

[0069] Extract Context(n i The implementation process of ) is as follows: using the vector database FAISS for retrieval, using cosine similarity as the similarity metric, and adopting a Top-K strategy to select the most relevant content from the top K context segments with the highest similarity. The default value of K is 3 and can be adjusted according to the algorithm configuration to achieve accurate matching and rapid extraction of context segments related to threat nodes. The formula for calculating cosine similarity is as follows:

[0070]

[0071] Where A and B represent the query vector and index vector, respectively, · represents the dot product of the vectors, and ∥A∥ and ∥B∥ represent the Euclidean norm of the vectors. By calculating cosine similarity, the system can assess the similarity between the query vector and the index vector.

[0072] Preferably, LLM(·) uses ChatGPT4.0 as the inference model.

[0073] If the current node n i The method selection process is a non-cold start phase, and the upper confidence bound variant strategy is used for method selection:

[0074]

[0075] Where, N(A i ) is node n i The total number of visits to the method set. Then, the algorithm executes the corresponding selected method aj , generate a new child node set C t+1 .

[0076] 3. Update of node quality and method quality

[0077] The node quality and method quality update steps are as follows Figure 4 First, update the threat node set N t+1 =N t ∪C t+1 , and record the parent nodes of all newly generated nodes in turn, and the current optimal threat node n i Update to the parent node of all newly generated threat nodes n′.

[0078]

[0079] According to the expansion results, in order to effectively amplify the incentive, traverse all n′’s ancestor nodes and update the relevant node quality Q t (n):

[0080]

[0081] Where ΔQ t (n′) is the incentive calculated based on the feedback information, and Ancestors(n′) represents the set of all ancestor nodes of the newly generated node n′.

[0082] Preferably, ΔQ t The strategy of (n′) is: for any node If n′ is found not to belong to N t , then the incentive is 1, otherwise the incentive is 0.

[0083] Then, the method quality is updated. Considering the similarity between different threat hunting methods, the method similarity function Sim is used to update the method quality. method , the method quality matrix W t To update:

[0084]

[0085] Where A represents the set of all threat hunting methods; a j is the current optimal threat node n i The current best threat hunting method selected on Sim method (a,a j ) is method a and method a j Similarity measure between .

[0086] Preferably, for any two methods a, a jThe default similarity is 0. If they share the same data source, the similarity is increased by 0.1; if they share the same operation, the similarity is increased by 0.5.

[0087] Then, the T+1th threat hunting node set is generated.

[0088] Optionally, the Tth threat hunting node set may be assigned as the T+1th threat hunting node set, and the threat hunting node and method selection steps may be re-run to start a new round of threat hunting.

[0089] This method, based on a reinforcement learning framework, leverages the knowledge base and zero-shot learning capabilities of a large language model, combined with the strategy optimization and adaptive learning properties of reinforcement learning, to improve the efficiency of threat hunting. This method addresses sparse rewards in scenarios with slow convergence by introducing recursive incentive updates and method similarity quality. In cold-start scenarios, it uses a large language model to introduce external knowledge, enabling efficient threat hunting.

[0090] In order to verify the technical effect of the technical solution proposed by the present invention, the following experimental evaluation was carried out.

[0091] To simulate real-world threat hunting scenarios, the experimental evaluation process of this invention constructed a simulated threat hunting dataset consisting of 186 threat intelligence reports. To emulate the expert threat hunting process, an initial set of threat entities, including IP addresses, domain names, URLs, and file hashes, was extracted from these reports. The data sources were local, VirusTotal, RapidDNS, and ThreatBook, and 21 automated query methods were implemented. During dataset creation, the initial set of threat entities was expanded to simulate the expert query process. Data analysis of the dataset revealed that each report contained an average of 11.77 initial threat entity nodes. After expansion, the average number of entity nodes increased to 415.80, and the average number of valid queries was 925.47. All data sources contributed, with VirusTotal accounting for 34.7%, local query data for 34.0%, RapidDNS for 23.3%, and ThreatBook for 8.1%. This dataset effectively simulates the threat hunting response process for 186 threat events and effectively evaluates the effectiveness of the proposed method in threat hunting response to threat events.

[0092] The experimental evaluation indicators of the present invention include average recall rate and average number of steps to reach the recall rate.

[0093] Average Recall: This measures the proportion of threat nodes correctly identified within a fixed number of search steps. A higher recall indicates a stronger algorithm's ability to identify threat nodes within a given number of steps.

[0094]

[0095] Where TP represents the number of correctly identified threat nodes, and FN represents the number of actual threat nodes that are not identified.

[0096] The average number of steps to reach recall (S R ) is used to measure the average number of steps required to achieve a specific recall rate, reflecting the efficiency of the algorithm. The fewer steps, the more efficient the algorithm is in the threat hunting process.

[0097]

[0098] Where, is the minimum number of steps required for the i-th threat event to reach the recall threshold, and N is the total number of events.

[0099] The performance evaluation process is designed to perform a threat hunting process for each threat event in the dataset, starting from the context building step, with an upper limit of 15,000 rounds. Each event is independent of each other in the experiment, and no information is shared during the hunting process to ensure that the results are not affected by other events. The algorithms to be evaluated include: the RE-HUNTER algorithm implemented by this invention, the MCTS algorithm using the MCTS-UCB strategy to implement node and method selection, the SA-UCB algorithm implemented using the SA strategy in MABAT (Dekel et al. 2021), and the SA-UCB-1 algorithm, which uses the framework of this invention to optimize the SA-UCB algorithm and split the decision-making process into node and method selection.

[0100] like Figure 5 As shown, over the 0-2000 step interval, compared to various baseline methods, our proposed algorithm achieved an 18.24% improvement in average recall over MCTS, a 33.10% improvement over SA-UCB-1, and a 351% improvement over SA-UCB. From the 0-250 step cold start phase, our proposed algorithm achieved an 86.28% improvement over MCTS, a 143.70% improvement over SA-UCB-1, and a 1375% improvement over SA-UCB. Therefore, based on experimental data, our algorithm can achieve a higher recall than the competing methods for the same number of threat hunting steps, resulting in superior performance.

[0101] Table 1. Number of threat hunting decision steps required to achieve different average recall rates

[0102]

[0103] As shown in Table 1, the average number of operation steps required by the present invention is significantly less than that of other methods at the same recall rate, and the overall performance can be divided into three stages for analysis. Specifically, in the initial stage (10%, 20%, and 30% recall rates), the present invention only requires 15 steps to achieve the 10% recall rate target, and at 20% and 30% recall rates, the number of operation steps is reduced by 53.85% and 16.95%, respectively, significantly improving efficiency. Entering the mid-term stage (40%, 50%, and 60% recall rates), the method of the present invention continues to maintain a low step requirement, reducing the number of steps by 9.51%, 15.14%, and 12.26%, respectively. In the later stages (70%, 80%, and 90% recall rates), the present invention still shows obvious advantages at each recall rate level, with the step reduction rate reaching 20.85%, 26.24%, and 35.60%, respectively. These results demonstrate that our method not only quickly initiates the threat hunting process but also remains efficient when higher-coverage threat intelligence is required, effectively reducing search costs. Specifically, achieving a 90% recall rate on the evaluation dataset, our method reduces an average of 1,047 decision and query steps per threat hunting session.

[0104] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the invention.

Claims

1. A threat hunting method based on a large language model and reinforcement learning, characterized in that: The following steps are involved: Step 1: Context Building Extract initial threat entities from threat intelligence to form an initial threat hunting node set, convert threat intelligence text into high-dimensional vectors, and establish a vector-text retrieval database; Step 2: Threat Hunting Node and Method Selection Based on the current threat hunting node set, the optimal threat node is selected through the reinforcement learning algorithm, and then the threat hunting method corresponding to the optimal threat node is selected to obtain the current optimal threat hunting method; if the method selection process of the current node is in the cold start phase, the external knowledge provided by the large language model is used to assist the reinforcement learning algorithm for method selection; if the method selection process of the current node is not in the cold start phase, the upper confidence bound variant strategy is used for method selection; finally, the selected threat hunting method is executed to generate a new child node set; Step 3: Update node quality and method quality By recursively updating the node quality and using similarity to update the method quality, the sparse reward problem is solved, and finally the next threat hunting node set is generated.

2. The threat hunting method based on large language model and reinforcement learning according to claim 1 is characterized in that: In step 1, the initial threat hunting node set is N0; in the initial stage, let the current threat hunting node set, that is, the Tth threat hunting node set N t is the initial threat hunting node set, that is, N t =N0; Preprocess the threat intelligence text and segment it into original text segments T′ by sentence i , each original text segment T′ i Use word embedding models to convert text into high-dimensional vectors and represent them as V i ;Build vector-text retrieval database DB Context ={<V i ,T′ i >}。 3. The threat hunting method based on large language model and reinforcement learning according to claim 2 is characterized in that: The step 2 is specifically as follows: Step 2.1: Based on the T-th threat hunting node set N t The expansion is carried out, and the threat hunting node selection is performed according to the following formula to obtain the current optimal threat node n i : Where Q t (n) represents the node quality of the corresponding threat node n. When initialized, the node quality Q corresponding to all threat nodes t (n) is 0; P(n) represents the parent node of threat node n; N(n) represents the number of times the corresponding threat node n is selected as the optimal threat node in the historical threat hunting process; c is a hyperparameter for balancing exploration and exploitation; Step 2.2: Select the threat hunting method and obtain the current optimal threat hunting method a j ; If n i ∈N0, then the current optimal threat node n i The method selection is set to the cold start stage, otherwise it is the non-cold start stage; If the current optimal threat node n i The method selection process is in the cold start stage, so the cold start method selection strategy is used to introduce the external knowledge provided by the large language model to assist the reinforcement learning algorithm to select the method, and the current optimal threat hunting method a is obtained. j : Where k i is the current optimal threat node n i Node type; A i is the current optimal threat node n i A collection of optional threat hunting methods; ki,a is the method quality matrix W t The elements in the node type k i The quality of using method a; the method quality matrix W t is an R×M matrix, where R represents the number of threat node types and M represents the total number of threat hunting methods; W t At initialization, all elements are set to 0; N(a) is the number of times method a is selected, LLMScore(a,n i ) is the current optimal threat node n calculated by the large language model i Applicability score with method a; If the current optimal threat node n i The method selection process is a non-cold start stage, and the upper confidence bound variant strategy is used for method selection, as shown in the following formula: Where, N(A i ) is the current optimal threat node n i The total number of visits to the method collection; Step 2.3: Execute the selected optimal threat hunting method a through reinforcement learning algorithm j , generate a new threat sub-node set C t+1 .

4. The threat hunting method based on large language model and reinforcement learning according to claim 3 is characterized in that: In step 2.2, the large language model is used to calculate the current optimal threat node n i The applicability score with method a is as follows: LLMScore(a,n i )=LLM(Context(n i ),Description(a)) In the formula, context (n i ) represents the retrieval of the vector-text database DB Context The node that is found in the current optimal threat node n i Relevant contextual information and the corresponding vector representation; Description(a) is the description of method a; LLM(·) represents the reasoning process of the large language model, and outputs method a at the current optimal threat node n i Ratings on Extract Context(n i ) is implemented as follows: the vector database FAISS is used for retrieval, cosine similarity is used as the similarity metric, and the Top-K strategy is adopted to select the most relevant content from the top K context segments with the highest similarity, so as to achieve accurate matching and rapid extraction of context segments related to threat nodes.

5. The threat hunting method based on large language model and reinforcement learning according to claim 3 is characterized in that: Step 3 is as follows: Step 3.1: Update the threat node set N t+1 =N t ∪C t+1 , and record the parent nodes of all newly generated threat nodes n′ in turn, and the current optimal threat node n i Update to the parent node of all newly generated threat nodes n′: Step 3.2: For each newly generated node n′, calculate the incentive ΔQ based on the feedback information t (n′), and traverse all newly generated nodes n′, and update the relevant node quality Q t (n): Where ΔQ t (n′) is the incentive calculated based on the feedback information, Ancestors(n′) represents the set of all ancestor nodes of the newly generated node n′; Step 3.3: Update the method quality: Using the method similarity function Sim method , the method quality matrix W t To update: Where A represents the set of all threat hunting methods; a j is the current optimal threat node n i The current best threat hunting method selected on Sim method (a,a j ) is method a and method a j Similarity measure between .

6. The threat hunting method based on large language model and reinforcement learning according to claim 5 is characterized in that: The node quality Q t In the update of (n), the incentive ΔQ t The strategy of (n′) is: for any newly generated node n′∈C t+1 , if n′ does not belong to the current threat hunting node set N t , then the incentive is 1, otherwise the incentive is 0.

7. The threat hunting method based on large language model and reinforcement learning according to claim 5 is characterized in that: The method similarity function Sim method (a,a j ) is calculated as follows: for any two methods a, a j If they share the same data source, the similarity is increased by 0.1; if they share the same operation, the similarity is increased by 0.5.

Images