Vulnerability processing method, device and system, electronic equipment, storage medium and product

Abstract

The invention relates to the technical field of communication, and particularly provides a vulnerability processing method, device and system, electronic equipment, a storage medium and a product. The method comprises the steps of obtaining network information; performing vulnerability prediction based on the network information to obtain vulnerability prediction information; processing the vulnerability prediction information by using a first model to obtain attack information; the first model is obtained by training based on a reinforcement learning principle; and attacking the target vulnerability based on the attack information. The first model automatically generates the attack information through the vulnerability prediction information, the first model is obtained by training based on the reinforcement learning principle, and the obtained attack information can be continuously adjusted according to feedback, so that the efficiency and accuracy of the attack information output by the first model are improved, and the user experience is improved. Omission or misjudgment of vulnerabilities is avoided to a certain extent, so that potential safety hazards are reduced, and the security of a network system is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of communication technology, and in particular to a vulnerability processing method and device, system, electronic device, storage medium and product. Background Art

[0002] With the rapid development of network technology, the scale and complexity of large network systems are increasing. At present, traditional manual vulnerability processing methods can no longer meet the needs of modern network security. For example, manual vulnerability surveys rely on manual surveys. For another example, manual vulnerability verification requires staff to analyze relevant information about the vulnerability by themselves and handle it according to the analysis results. However, the above operations mainly rely on the experience of the staff, which is easy to miss vulnerabilities and has low processing efficiency. It can be seen that manual vulnerability processing is not only time-consuming and labor-intensive, but also easy to miss potential security risks. Summary of the invention

[0003] In view of the above problems, the present disclosure is proposed. The present disclosure provides a vulnerability processing method and device, system, electronic device, storage medium and product.

[0004] According to one aspect of the present disclosure, a vulnerability processing method is provided, including: obtaining network information; predicting vulnerabilities based on the network information to obtain vulnerability prediction information; processing the vulnerability prediction information using a first model to obtain attack information; the first model is trained based on the principle of reinforcement learning; and attacking the target vulnerability based on the attack information.

[0005] According to another aspect of the present disclosure, the reinforcement learning principle includes: artificial feedback reinforcement learning RLHF.

[0006] According to another aspect of the present disclosure, the first model includes: a language model and a reward model; wherein the language model is used to learn the relationship between the vulnerability prediction information and the attack information; and the reward model is used to evaluate the reward value of the attack information.

[0007] According to another aspect of the present disclosure, the method also includes: obtaining historical network information; using the language model to process the historical network information multiple times to obtain multiple output results; obtaining manual evaluation information of the multiple output results; using the historical network information, the multiple output results and the manual evaluation information as training samples to train the first model.

[0008] According to another aspect of the present disclosure, the method also includes: obtaining first update data; the first update data at least includes vulnerability prediction information and attack information within a first time interval; obtaining a reward value corresponding to the attack information; and using the first update data and the reward value as training samples to perform supervised adjustment training on the first model.

[0009] According to another aspect of the present disclosure, performing vulnerability prediction based on the network information to obtain vulnerability prediction information includes: processing the network information using a second model to obtain the vulnerability prediction information; wherein the second model is a neural network model.

[0010] According to another aspect of the present disclosure, the vulnerability prediction information includes at least: the predicted vulnerability; the vulnerability prediction information also includes: the relative probability of the predicted vulnerability occurring.

[0011] According to another aspect of the present disclosure, the method further includes: obtaining second update data from a vulnerability library; wherein the vulnerability library maintains external vulnerability data; and performing secondary training on the second model using the second update data.

[0012] According to another aspect of the present disclosure, the method further includes: obtaining an attack result; and when the attack result indicates that the attack has failed, redetermining the attack information using the first model.

[0013] According to another aspect of the present disclosure, the method also includes: when the attack result indicates that a first condition is met, writing relevant information of the attack into an attack suggestion knowledge base; wherein the first condition includes at least one of the following: the target vulnerability does not exist, the attack is successful, and the attack reaches the maximum number of times.

[0014] According to another aspect of the present disclosure, the attacking the target vulnerability based on the attack information includes: using a penetration tool to attack the target vulnerability.

[0015] According to another aspect of the present disclosure, the network information includes at least one of the following: the node's operating system, IP address, enabled services and versions, enabled ports, host behavior, vulnerabilities, and patches.

[0016] According to another aspect of the present disclosure, a vulnerability processing device is provided, including: an information acquisition module for acquiring network information; a vulnerability prediction module for performing vulnerability prediction based on the network information to obtain vulnerability prediction information; an information processing module for processing the vulnerability prediction information using a first model to obtain attack information; the first model is trained based on the reinforcement training principle; and a vulnerability attack module for attacking a target vulnerability based on the attack information.

[0017] According to another aspect of the present disclosure, a vulnerability processing system is provided, including: a first model; a second model; a penetration tool; a vulnerability library; and an attack suggestion knowledge base.

[0018] According to another aspect of the present disclosure, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the method described in any one of the above embodiments.

[0019] According to another aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the method described in any of the above embodiments is implemented.

[0020] According to another aspect of the present disclosure, a computer program product is provided, including a computer program / instruction, which implements the method described in any of the above embodiments when the computer program / instruction is executed by a processor.

[0021] As will be described in detail below, according to an embodiment of the present disclosure, a vulnerability processing method and device, system, electronic device, storage medium and product. The present disclosure uses network information to obtain vulnerability prediction information, and uses a first model to process the vulnerability prediction information to obtain attack information, thereby realizing automatic processing of vulnerabilities. There is no need for staff to manually analyze vulnerability prediction information based on personal experience as in the prior art, which improves the efficiency of obtaining attack information. In addition, the first model involved in the present disclosure automatically generates attack information through vulnerability prediction information. The first model is obtained by training based on the principle of reinforcement learning, and the obtained attack information can be continuously adjusted according to feedback, thereby improving the efficiency and accuracy of the attack information output by the first model, avoiding omissions or misjudgments of vulnerabilities to a certain extent, thereby reducing security risks and improving the security of network systems.

[0022] It is to be understood that both the foregoing general description and the following detailed description are exemplary, and are intended to provide further explanation of the technology as claimed. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The above and other purposes, features and advantages of the present disclosure will become more apparent by describing the embodiments of the present disclosure in more detail in conjunction with the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and do not constitute a limitation of the present disclosure. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0024] Figure 1 It is a flowchart illustrating a vulnerability handling method according to an embodiment of the present disclosure.

[0025] Figure 2 is a schematic diagram illustrating language model processing according to an embodiment of the present disclosure.

[0026] Figure 3 Detailed description of the invention is a flowchart illustrating another vulnerability handling method according to an embodiment of the present invention.

[0027] Figure 4 is a block diagram illustrating a vulnerability processing apparatus according to an embodiment of the present disclosure.

[0028] Figure 5 is a hardware block diagram illustrating an electronic device according to an embodiment of the present disclosure.

[0029] Figure 6 is a schematic diagram illustrating a computer-readable storage medium according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the present disclosure more obvious, the exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described here.

[0031] With the rapid development of network technology, the scale and complexity of large network systems are increasing, and traditional manual vulnerability mining methods can no longer meet the needs of modern network security. Manual detection is not only time-consuming and laborious, but also easy to miss potential security risks.

[0032] In order to solve the above problems, the present disclosure provides a vulnerability processing method. The present disclosure utilizes a first model that has undergone reinforcement learning, so that the first model can obtain attack information through vulnerability prediction information, and then adjust the first model according to feedback, thereby improving the accuracy of the attack information obtained by the first model, thereby improving the vulnerability processing efficiency.

[0033] For the vulnerability handling process, please refer to Figure 1 , Figure 1 is a flowchart illustrating a vulnerability handling method according to an embodiment of the present disclosure. Figure 1 The method specifically comprises:

[0034] S101, obtaining network information.

[0035] Specifically, the network information may include relevant information of each physical network node (hereinafter collectively referred to as a node) in the network system, and the relevant information of the node in the network system is the network information. After the network information is obtained in this step, the vulnerability prediction information can be obtained according to the network information (ie, step S102).

[0036] The present disclosure has no specific limitation on the network system and the node types included in the network system. The node types may include but are not limited to at least one of the following: a computer, a router, a switch, and a printer.

[0037] In addition, the present disclosure does not specifically limit the type of information included in the network information, and the network information may include but is not limited to at least one of the following: the node's operating system, IP address, enabled services and versions, enabled ports, host behavior, vulnerabilities, patches, and network environment information. The network environment information includes but is not limited to at least one of the following: network topology information, log data of network devices and servers, and network traffic data.

[0038] Among them, the vulnerabilities carried in the network information can be the vulnerabilities discovered by the node or node agent. Due to the limited prediction ability of the node or node agent, it is impossible to predict vulnerabilities based on other network information. Compared with the vulnerabilities predicted based on other types of network information, the vulnerabilities carried in the network information are generally real-time vulnerabilities. This type of vulnerability can be used as a supplement to the subsequent vulnerability prediction information, thereby improving the comprehensiveness and accuracy of vulnerability discovery.

[0039] S102, performing vulnerability prediction based on the network information to obtain vulnerability prediction information.

[0040] In this step, the information included in the network information can be used to identify possible vulnerabilities, that is, vulnerability prediction information.

[0041] The present disclosure does not specifically limit the vulnerability prediction method, which includes but is not limited to at least one of the following: algorithm prediction, model prediction. Furthermore, the prediction results can be fine-tuned in combination with the staff's experience. For example, vulnerability prediction information can be obtained by processing vulnerability prediction information based on the second model. The specific processing method of the second model will be described in detail later and will not be repeated here.

[0042] Among them, vulnerability prediction information refers to the relevant information of the predicted vulnerability. Vulnerability prediction information includes at least: the predicted vulnerability. Furthermore, vulnerability prediction information may also include but is not limited to at least one of the following: the relative probability of the predicted vulnerability, vulnerability identification, vulnerability type, and impact range, which are not exhaustive here. Specifically, the obtained vulnerability prediction information can better generate attack information to deal with the vulnerability (described in detail later). Taking the vulnerability prediction information also including the vulnerability type as an example, the vulnerability characteristics can be obtained according to the vulnerability type, so that the subsequent verification and processing can be more targeted according to the vulnerability characteristics.

[0043] S103, using a first model to process the vulnerability prediction information to obtain attack information; wherein the first model is trained based on the reinforcement learning principle.

[0044] Specifically, the attack information may include one or more processing methods for the vulnerability, and the processing methods here include but are not limited to at least one of the following: an attack method for the vulnerability and a vulnerability repair method. Therefore, after the attack information is determined in this step, the target vulnerability is processed based on the attack information (i.e., step S104).

[0045] The present disclosure does not specifically limit the type of reinforcement learning principle used when training the first model. Reinforcement learning principles include, but are not limited to, at least one of the following: Hierarchical Reinforcement Learning (HRL), Reinforcement Learning with human feedback (RLHF), and Inverse Reinforcement Learning (IRL), which are not exhaustively listed here. In addition, the present disclosure can adjust the first model based on the reinforcement learning principle and the attack results of the target vulnerability, thereby optimizing the first model, so that the attack information obtained is more in line with the actual application scenario, and the accuracy and efficiency of the obtained attack suggestions are improved.

[0046] Taking RLHF as an example, the staff can score the attack results obtained from the attack information and provide feedback to the first model, thereby optimizing the attack information output by it. Alternatively, the staff can provide feedback to the first model during its training process, and the trained first model can score the attack results by itself and adjust the first model according to the score.

[0047] S104: Attack the target vulnerability based on the attack information.

[0048] In practical applications, this step can also be called the verification process of the target vulnerability, that is, verifying the existence of the vulnerability predicted by the vulnerability prediction information.

[0049] The present disclosure does not specifically limit the module for attacking the target vulnerability. The module can be a specific function deployed in the system, or it can be a third-party tool. Furthermore, the present disclosure does not limit the type of method for attacking vulnerabilities, and the methods for attacking vulnerabilities include but are not limited to at least one of the following: penetration attack, code review, and red and blue team testing. Among them, different attack methods can be implemented by different tools. For example, a penetration attack can be implemented by a penetration tool. That is to say, the first model can hand over the attack information to the penetration tool, and the penetration tool attacks the target vulnerability based on the attack information. Compared with the prior art, in which the staff needs to use the penetration tool to attack the vulnerability based on their own experience, the present disclosure can directly use the attack tool to attack the vulnerability, thereby improving the efficiency of vulnerability verification.

[0050] The present disclosure can specifically attack at least some of the vulnerabilities included in the vulnerability prediction information, and the attacked vulnerabilities are target vulnerabilities. The present disclosure does not specifically limit the method for selecting the target vulnerabilities, including but not limited to at least one of the following: selecting N vulnerabilities with the highest relative probability as target vulnerabilities, selecting vulnerabilities with high risk levels as target vulnerabilities, and the list is not exhaustive.

[0051] In summary, the present disclosure uses network information to obtain vulnerability prediction information, and uses the first model to process the vulnerability prediction information to obtain attack information, thereby realizing automated processing of vulnerabilities. In other words, the first model involved in the present disclosure automatically generates attack information through vulnerability prediction information, and does not need to manually analyze the vulnerability prediction information based on experience by staff as in the prior art, thereby improving the efficiency of obtaining attack information. In addition, the first model is obtained through training based on the principle of reinforcement learning, and the obtained attack information can be continuously adjusted according to feedback, thereby improving the efficiency and accuracy of the attack information output by the first model, avoiding omissions or misjudgments of vulnerabilities to a certain extent, thereby reducing security risks and improving the security of the network system.

[0052] On the basis of the above-mentioned embodiments, the present disclosure also fully considers the structure of the first model and provides different training methods corresponding to first models with different structures.

[0053] In an illustrative embodiment, the first model may include a language model and a reward model. The language model is used to learn the relationship between the vulnerability prediction information and the attack information; the reward model is used to evaluate the reward value of the attack information. Specifically, the reward model can evaluate the output of the language model, and the first model can obtain feedback to adjust it. In addition, the present disclosure does not specifically limit the types of language models and reward models. The language model includes but is not limited to at least one of the following: a generative model, a discriminative model, and an analytical model. The types of reward models include but are not limited to at least one of the following: a deep neural network, a convolutional neural network, a recurrent neural network, opt-350M, etc., which are not exhaustive here.

[0054] Optionally, when the first model includes a language model, the computing power required for vulnerability mining on the network can be calculated based on the network environment information, thereby selecting a suitable language model.

[0055] In addition, in the present disclosure, the adjustment process of the first model is divided into: A. adjustment before application; B. adjustment during application. It should be understood that before application, the first model needs to be trained to adapt to the application environment and generate attack information, and the purpose of adjustment during application is to enable the first model to generate attack information more accurately based on the attack information generated by the first model, that is, the adjustment during application is to further improve the performance of the first model.

[0056] In embodiment A, the adjustment of the first model before application specifically includes:

[0057] S103-a1, obtaining historical network information.

[0058] Specifically, the types of information included in the historical network information at least include the information types of the aforementioned network information.

[0059] S103-a2, using the language model to process the historical network information multiple times to obtain multiple output results.

[0060] Specifically, in order to enable the language model to better cover different outputs and expressions, the language model needs to generate multiple output results and select the optimal output result from the multiple output results (ie, step S103-a3) to provide feedback for the language model, thereby improving the accuracy of the language model output.

[0061] S103-a3, obtaining manual evaluation information of the plurality of output results.

[0062] This step means that the present disclosure can evaluate the output of the language model in combination with the experience of the staff. For example, please refer to Figure 2 , Figure 2 is a schematic diagram illustrating language model processing according to an embodiment of the present disclosure, Figure 2 The interface provides the input of the language model: prompt (i.e., the input information of the language model, i.e., network information, which can be historical network information in actual scenarios). The language model generates multiple completions (i.e., output results) based on the prompt, and the staff evaluates the performance of the completions to obtain a rank (i.e., manual evaluation information).

[0063] It should be noted that the present disclosure can evaluate the output results in combination with manual evaluation information of multiple staff members. That is to say, the present disclosure can adjust the language model by comprehensively considering the experience of different staff members, so as to obtain output results that are more in line with reality.

[0064] S103-a4, using the historical network information, the multiple output results and the manual evaluation information as training samples to train the first model.

[0065] Specifically, based on the above-mentioned manual evaluation information, output results and historical network information, for example, Figure 2 In the example above, (prompt, completions, rank) is used as a training sample, and the present disclosure can train the first model.

[0066] In this way, after the first model is selected and trained, the language model that can be implemented can obtain attack information, thereby automating the acquisition of attack information of the vulnerability, reducing human errors, and improving the efficiency of attack information generation.

[0067] In embodiment B, the adjustment of the first model in the application specifically includes:

[0068] S103-b1, obtaining first update data; the first update data at least includes vulnerability prediction information and attack information within a first time interval.

[0069] In this step, the first model is mainly adjusted according to the attack results in actual applications. That is, the first model is mainly adjusted using vulnerability prediction information, attack information, and the reward value corresponding to the attack information (i.e., step S103-b3). The reward value here can be obtained by evaluating the attack results using the trained reward model.

[0070] The present disclosure does not limit the specific range of the first time interval. Optionally, the first time interval may be the latest update period, for example, the last week, the last day, 6 hours before a certain time, etc., without exhausting the list.

[0071] S103-b2, obtaining a reward value corresponding to the attack information.

[0072] In this step, compared with the aforementioned embodiment A which uses manual evaluation information as feedback, this embodiment pays more attention to the scoring of the attack information by the reward model, that is, the reward value. In other words, the present disclosure uses the reward value as feedback for the application scenario, thereby adjusting the first model.

[0073] S103-b3, using the first updated data and the reward value as training samples, performing supervised adjustment training on the first model.

[0074] Specifically, when adjusting the first model, the first update data can be used as the input of the first model, and the reward value can be used as the feedback of the first model, and the first model can be supervised and adjusted according to the reward value. In this way, the performance of the first model can be optimized during the application process, and the efficiency of vulnerability verification can be improved.

[0075] Among them, supervised fine-tuning is the process of adjusting the first model that is about to be trained (for example, the first model trained in Example A) according to the labeled training samples. The present disclosure does not specifically limit the form of the labeling, for example, the above-mentioned reward value can be considered as a labeling. In this way, the performance of the first model in obtaining attack information can be improved.

[0076] Exemplarily, regardless of whether the attack is successful or not, according to the prompt and completion in the process of generating the current round of attack information, the attack result (for example, the attack result includes attack failure and attack success), the reward for a successful attack is set to 1, and the reward for a failed attack is set to 0, and the data is stored in the form of {prompt, completion, reward}. This allows the first model to be fine-tuned in a supervised manner to improve the ability of the first model to provide attack information.

[0077] It should be noted that the above two embodiments (i.e., Embodiment A and Embodiment B) respectively provide a first model update process before application and a first model update process during application. In order to ensure the real-time performance of the first model, the first model can take into account the above two training processes, i.e., training using the method in Embodiment A before the first model is applied, and supervised fine-tuning using the method in Embodiment 2 during the application process.

[0078] Optionally, in actual application, the attack result can be directly given by the attack subject (such as a penetration tool), or it can be determined by the first model based on the feedback information of the attack subject. The specific method is not limited. Different types of attack methods have different methods for determining attack results, which will not be elaborated here.

[0079] When the attack result indicates that the attack has failed, the first model may be used to re-determine the attack information. The first model may determine the attack information using at least one of feedback information and vulnerability prediction information.

[0080] Optionally, when the first model determines that the attack result indicates that the first condition is met, the current round of attack ends, and the relevant information of the current attack can be written into the attack suggestion knowledge base for subsequent application, for example, the first model can be updated twice based on this (see the above embodiment B). The first condition includes but is not limited to at least one of the following: the target vulnerability does not exist, the attack is successful, and the attack reaches the maximum number of times.

[0081] The present disclosure does not specifically limit the specific content of the relevant information of the attack. The relevant information includes but is not limited to at least one of the following: vulnerability prediction information, relevant information of the target vulnerability, and the attack result of at least one attack. Among them, the relevant information of the target vulnerability includes but is not limited to at least one of the following: the cause of the target vulnerability and the attack information of at least one attack, which are not exhaustive.

[0082] In this embodiment, the attack suggestion knowledge base can share attack-related information with multiple modules. That is, the first update data obtained by the first model in step S103-b1 and the attack result obtained by the second model later can be obtained from the attack suggestion knowledge base, so that the first model and / or the second model can obtain the latest environmental feedback and make adjustments.

[0083] In a specific implementation of the present disclosure, in step S102, vulnerability prediction information can also be obtained using a second model, wherein the second model is trained based on historical network information. The type of the second model may include but is not limited to at least one of the following: a deep neural network, a convolutional neural network, a recurrent neural network, and opt-350M.

[0084] In addition to training the second model before application, the second model can also be trained again using other data during the application process. For example, the second update data in the vulnerability library can be read to perform secondary training on the second model. The specific training can refer to the first model and will not be elaborated here. It should be noted that the present disclosure does not specifically limit the data types of the dimensions in the vulnerability library. External vulnerability data can be maintained in the vulnerability library. Furthermore, the first update data within the network system can also be maintained in the vulnerability library. Specifically, the second model can not only use the local first update data, but also update data outside the system, so as to comprehensively consider the vulnerability information inside and outside the system, thereby improving the accuracy and reliability of the second model in predicting vulnerabilities.

[0085] The present disclosure does not specifically limit the form of storing vulnerability information in the vulnerability database. For example, it can be expressed as:

[0086] vul-database={(vulname,vul-OS,vul-port,vul-ser) i}

[0087] Among them, vul-database is the updated data, and vulname is the vulnerability name.

[0088] Optionally, in order to avoid the instability of the second model caused by the second model performing prediction and adjustment at the same time, the present disclosure is also provided with a third model. The third model is obtained based on the second model. Specifically, the parameters and structure of the third model can be the same as the second model, and the updated attack results and vulnerability prediction information can be used for training in the second time interval to obtain the adjusted third model, and the parameters of the adjusted third model can be copied to the second model in the third time interval. In this way, the separation of the learning process and the application process of the second model can be achieved, and the stability of the system can be improved. The present disclosure does not specifically limit the values ​​of the second time interval and the third time interval. The two can be the same or different, and can be determined according to actual conditions.

[0089] The present disclosure does not specifically limit the structure of the second model. Optionally, the second model may have an input layer, two hidden layers, and an output layer. In practical applications, network information may be input into the second model. For ease of understanding, the data of the input layer is presented in the following manner:

[0090]

[0091] Among them, X in That is, network information; OS is the operating system of the node; IP is the IP address; service / ver is the enabled service and version; port is the enabled port; action is the behavior of the host; vul is the vulnerability; patch is the patch, that is to say, the input layer is seven neurons.

[0092] The hidden layer can be eight neurons, and the output layer data can be expressed as:

[0093]

[0094] Among them, Q out is the output layer data, q1, q2…q i is the probability of existence of various vulnerabilities. I is an integer greater than 0.

[0095] Optionally, the data of the output layer can also be converted into the relative probability of each type of vulnerability. For example, a softmax function can be used, and the specific relationship can be satisfied as follows:

[0096]

[0097] Among them, P i is the relative probability of each type of vulnerability, and m is the total number of vulnerability types.

[0098] When the module deployed with the second model sends vulnerability prediction information to the module deployed with the first model, vulnerability prediction information of some vulnerabilities can be sent, and the vulnerabilities sent by the module deployed with the second model are the target vulnerabilities.

[0099] Exemplarily, when the module with the second model is deployed for screening, the vulnerabilities can be sorted from high to low based on their probability of existence, and added from high to low. When the sum of the probability of vulnerability existence is greater than a preset threshold, recording can be stopped. For example, P1+P2+...+P j ≥60%, and expressed as:

[0100] vul id ={(vulname1, P1), (vulname2, P2), ..., (vulname j , P j )}

[0101] Optionally, the present disclosure may also be provided with a network information acquisition module for maintaining network information in consideration of data security and timeliness. The present disclosure does not specifically limit the network information acquisition method of the network information acquisition module, including but not limited to: manual entry and automatic acquisition. In practical applications, the above two network information acquisition methods may also be used in combination, that is, the network information acquisition module may automatically acquire network information, and at the same time, some information may also be manually entered as a supplement.

[0102] In addition, the network information acquisition module can store network information in any manner. For example, the network information can be stored according to the network topology relationship.

[0103] For example, the relationship between nodes can be expressed as:

[0104] (G, I) = {(G1, I1), (G2, I2), ..., (G id , I id ),…,(G n , I n )};

[0105] Among them, (G, I) is the network topology information, G is the node connection information, and I is the network information.

[0106] Among them, I id =<OS,IP,service / ver,port,action,vul,patch> ;

[0107] Among them, G id = {g id-1 , g id-2 , …, g id-n}; If node id is reachable to node i, then g id-i is 1, otherwise it is 0.

[0108] The above is the vulnerability processing method provided by the present disclosure. The above embodiments can be combined according to actual conditions, and the present disclosure does not limit this.

[0109] In an illustrative embodiment, please refer to Figure 3 , Figure 3 FIG. 1 is a flowchart illustrating another vulnerability handling method according to an embodiment of the present disclosure. Figure 3 As shown, specifically including:

[0110] Step 1: Detect entity network information.

[0111] Step 2: perform vulnerability prediction and obtain vulnerability prediction information.

[0112] This step is implemented by the second model described above, using entity network information (ie, network information) to obtain vulnerability prediction information, and using the vulnerability prediction information to perform vulnerability prediction.

[0113] Step 3: Use the GetVul large model to obtain attack suggestions.

[0114] This step is equivalent to the aforementioned step S103, and the GetVul large model is the aforementioned first model. The GetVul large model obtains attack suggestions based on vulnerability prediction information.

[0115] Step 4: Use penetration tools to attack the target and obtain feedback information.

[0116] This step is equivalent to the penetration tool attacking the attack target (target vulnerability), and the feedback information obtained is the aforementioned attack result.

[0117] Step 5: GetVul large model determines the attack result.

[0118] In this step, if the attack fails and the feedback information does not meet the first condition, execute step 5', that is, generate a new attack suggestion. If the feedback information meets the first condition (that is, the vulnerability in step 6 does not exist / the attack is successful / the maximum number of times is reached), execute step 6.

[0119] Step 5', the GetVul large model gives new attack suggestions.

[0120] Step 6: When the attack result is that the vulnerability does not exist / the attack is successful / the maximum number of times is reached, the attack suggestion of this round is stored in the attack suggestion knowledge base. The attack method or failure information is given.

[0121] After obtaining the attack method or failure information, step 7 and / or step 8 may be executed.

[0122] Step 7: Use the fine-tuning supplementary database to perform RLHF fine-tuning on the GetVul large model.

[0123] The fine-tuning supplementary database is used to store vulnerability prediction information and attack information, and fine-tune the GetVul large model based on this. The data stored in the fine-tuning supplementary database and the subsequent vulnerability database can come from the attack suggestion knowledge base.

[0124] Step 8: Use vulnerability libraries, external databases, security bulletins, etc. to improve the predictive capabilities of vulnerability prediction.

[0125] From the above, it can be seen that in order to implement the method of the present disclosure, the present disclosure also provides a vulnerability processing method, which at least includes: a first model; a second model; a penetration tool; a vulnerability library; and an attack suggestion knowledge base.

[0126] The present disclosure also provides a vulnerability processing device. Figure 4 is a block diagram illustrating a vulnerability processing device according to an embodiment of the present disclosure. Figure 4 As shown, the vulnerability processing device 400 includes: an information acquisition module 401, which is used to acquire network information.

[0127] The vulnerability prediction module 402 is used to perform vulnerability prediction based on the network information to obtain vulnerability prediction information.

[0128] The information processing module 403 is used to process the vulnerability prediction information using the first model to obtain attack information; the first model is trained based on the reinforcement training principle.

[0129] The vulnerability attack module 404 is used to attack the target vulnerability based on the attack information.

[0130] In an exemplary embodiment, the vulnerability processing device 400 is further configured as follows: the reinforcement learning principle includes: artificial feedback reinforcement learning RLHF.

[0131] In an exemplary embodiment, the vulnerability processing device 400 is further configured as follows: the first model includes: a language model and a reward model; wherein the language model is used to learn the relationship between the vulnerability prediction information and the attack information; and the reward model is used to evaluate the reward value of the attack information.

[0132] In an exemplary embodiment, the vulnerability processing device 400 is further configured to: use the first model to process the vulnerability prediction information to obtain attack information, including: obtaining historical network information; using the language model to process the historical network information multiple times to obtain multiple output results; obtaining manual evaluation information of the multiple output results; using the historical network information, the multiple output results and the manual evaluation information as training samples to train the first model.

[0133] In an exemplary embodiment, the vulnerability processing device 400 is further configured to: use the first model to process the vulnerability prediction information to obtain attack information, including: obtaining first update data; the first update data at least includes vulnerability prediction information and attack information within a first time interval; obtaining a reward value corresponding to the attack information; and using the first update data and the reward value as training samples to perform supervised adjustment training on the first model.

[0134] In an exemplary embodiment, the vulnerability processing device 400 is further configured to: perform vulnerability prediction based on the network information to obtain vulnerability prediction information, including: use a second model to process the network information to obtain the vulnerability prediction information; wherein the second model is a neural network model.

[0135] In an exemplary embodiment, the vulnerability processing device 400 is further configured as follows: the vulnerability prediction information at least includes: the predicted vulnerability; the vulnerability prediction information also includes: the relative probability of the predicted vulnerability occurring.

[0136] In an exemplary embodiment, the vulnerability processing device 400 is further configured to: obtain second update data from a vulnerability library; wherein the vulnerability library maintains external vulnerability data; and perform secondary training on the second model using the second update data.

[0137] In an exemplary embodiment, the vulnerability processing device 400 is further configured to: obtain an attack result; and when the attack result indicates that the attack has failed, redetermine the attack information using the first model.

[0138] Figure 5 The electronic device 500 according to the embodiment of the present disclosure includes at least a memory, a processor and a computer program stored in the memory, and the processor executes the computer program to implement the signal transmission method described in any of the above embodiments.

[0139] Figure 5 The electronic device 500 shown specifically includes: a central processing unit (CPU) 501, a graphics processing unit (GPU) 502 and a memory 503. These units are interconnected via a bus 504. The central processing unit (CPU) 501 and / or the graphics processing unit (GPU) 502 can be used as the above-mentioned processor, and the memory 503 can be used as the above-mentioned memory for storing computer-readable instructions. In addition, the electronic device 500 may also include a communication unit 505, a storage unit 506, an output unit 507, an input unit 508 and an external device 509, which are also connected to the bus 504.

[0140] Figure 6A schematic diagram of a computer-readable storage medium provided in an embodiment of the present disclosure. Figure 6 As shown, a computer-readable storage medium 600 according to an embodiment of the present disclosure has computer-readable instructions 601 stored thereon. When the computer-readable instructions 601 are executed by a processor, the signal transmission method described in any of the foregoing embodiments of the present disclosure is implemented. The computer-readable storage medium includes, but is not limited to, for example, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, optical disk, magnetic disk, etc.

[0141] The present disclosure further provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the vulnerability processing method described in any of the foregoing embodiments of the present disclosure.

[0142] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, and are not limitations. The above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.

[0143] The block diagrams of the devices, apparatuses, equipment, and systems involved in this disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including," "comprising," "having," and the like are open words, referring to "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or," and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0144] Additionally, as used herein, "or" used in a list of items beginning with "at least one" indicates a separate list, so that, for example, a list of "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not mean that the example described is preferred or better than other examples.

[0145] It should also be noted that in the system and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0146] Various changes, substitutions, and modifications of the techniques described herein may be made without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of the present disclosure is not limited to the specific aspects of the processes, machines, manufactures, compositions of events, means, methods, and actions described above. Currently existing or later to be developed processes, machines, manufactures, compositions of events, means, methods, or actions that perform substantially the same functions or achieve substantially the same results as the corresponding aspects described herein may be utilized. Thus, the appended claims include such processes, machines, manufactures, compositions of events, means, methods, or actions within their scope.

[0147] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0148] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A vulnerability processing method, characterized in that: The method comprises: Get network information; Perform vulnerability prediction based on the network information to obtain vulnerability prediction information; The vulnerability prediction information is processed using a first model to obtain attack information; the first model is trained based on the reinforcement learning principle; An attack is performed on the target vulnerability based on the attack information.

2. The method according to claim 1, characterized in that: The reinforcement learning principle includes: artificial feedback reinforcement learning RLHF.

3. The method according to claim 1 or 2, characterized in that: The first model includes: a language model and a reward model; Wherein, the language model is used to learn the relationship between the vulnerability prediction information and the attack information; The reward model is used to evaluate the reward value of the attack information.

4. The method according to claim 3, characterized in that: The method further comprises: Get historical network information; Processing the historical network information multiple times using the language model to obtain multiple output results; Obtaining manual evaluation information of the plurality of output results; The historical network information, the plurality of output results and the manual evaluation information are used as training samples to train the first model.

5. The method according to claim 3, characterized in that: The method further comprises: Acquire first update data; the first update data at least includes vulnerability prediction information and attack information within a first time interval; Obtaining a reward value corresponding to the attack information; The first update data and the reward value are used as training samples to perform supervised adjustment training on the first model.

6. The method according to any one of claims 1 to 5, characterized in that: The performing vulnerability prediction based on the network information to obtain vulnerability prediction information includes: Processing the network information using the second model to obtain the vulnerability prediction information; Wherein, the second model is a neural network model.

7. The method according to claim 6, characterized in that The vulnerability prediction information includes at least: predicted vulnerabilities; The vulnerability prediction information also includes: a relative probability of the predicted vulnerability occurring.

8. The method according to claim 6, characterized in that The method further comprises: Acquire second update data from a vulnerability library; wherein the vulnerability library maintains external vulnerability data; The second model is trained again using the second updated data.

9. The method according to any one of claims 1 to 8, characterized in that: The method further comprises: Get the attack results; When the attack result indicates that the attack has failed, the attack information is re-determined using the first model.

10. A vulnerability processing device, characterized in that: The device comprises: An information acquisition module, used to acquire network information; A vulnerability prediction module, used to perform vulnerability prediction based on the network information to obtain vulnerability prediction information; An information processing module, used to process the vulnerability prediction information using a first model to obtain attack information; the first model is trained based on the reinforcement training principle; The vulnerability attack module is used to attack the target vulnerability based on the attack information.

11. A vulnerability processing system, characterized in that: The system comprises: First model; Second model; Penetration tools; Vulnerability library; Attack suggestion knowledge base.

12. An electronic device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the method according to any one of claims 1 to 9.

13. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the method according to any one of claims 1 to 9 is implemented.

14. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images