Network security evaluation system and method based on generative artificial intelligence

Abstract

The invention discloses a network security evaluation system and method based on generative artificial intelligence, and belongs to the technical field of network and information security, and the system comprises a dynamic attack scene generation module, an automatic evaluation module, a user feedback collection module, an adaptive strategy optimization module, a multi-modal data fusion evaluation module, and a visual report generation module. The method comprises the steps of collecting desensitization threat intelligence and constructing an attack template, generating simulation attack data in an isolation environment through a generative AI, scheduling a test task and calculating indexes such as a detection rate and a false alarm rate, combining a user feedback optimization strategy and fusing multi-source data to construct a risk map, and finally generating an interactive evaluation report. According to the network security evaluation system and method based on generative artificial intelligence, comprehensive, continuous and intelligent evaluation of the security protection capability of an information system is realized through dynamic attack scene generation, multi-modal data fusion and a self-evolution evaluation strategy.

Description

Technical Field

[0001] This invention relates to the field of network and information security technology, and in particular to a network security evaluation system and method based on generative artificial intelligence. Background Technology

[0002] As information system architectures become increasingly complex, cyberattack methods are becoming more covert and diversified. Traditional cybersecurity assessment schemes mainly rely on known vulnerability databases or fixed rule lists to run scans and tests. This approach is difficult to effectively assess a system's ability to cope with unknown threats, especially in scenarios involving zero-day vulnerabilities and advanced persistent threats (APTs).

[0003] In recent years, the development of generative artificial intelligence technology has provided new ideas for security technology. Using generative AI, different types of simulated threat samples can be constructed under safe and controllable conditions to verify the adaptability and resilience of defense systems. However, existing technical solutions still have the following shortcomings: most evaluation systems lack the ability to generate unknown attack patterns; they lack feedback-based adaptive testing strategy optimization mechanisms; and they have low integration of multi-source heterogeneous data, making it difficult to obtain a global security situation view.

[0004] Therefore, there is an urgent need for a method and system that can automatically generate simulated attack scenarios, perform automated security assessments, integrate multimodal security data, and continuously optimize strategies based on feedback, in order to solve the problems of existing assessment systems being static, single-dimensional, and poorly adaptable. Summary of the Invention

[0005] To address the shortcomings of the existing technologies, the present invention aims to provide a network security evaluation system and method based on generative artificial intelligence. Through dynamic attack scenario generation, multimodal data fusion, and self-evolving evaluation strategies, it achieves a comprehensive, continuous, and intelligent evaluation of the security protection capabilities of information systems.

[0006] The technical solution adopted by this invention to solve its technical problem is as follows:

[0007] A cybersecurity assessment system based on generative artificial intelligence is provided, including:

[0008] Dynamic attack scenario generation module: Based on de-identified threat intelligence and attack feature templates, it generates simulated attack vectors and interaction data through a generative artificial intelligence model, and executes it in an isolated test environment to generate simulated attack data;

[0009] Automated evaluation module: Based on the simulated attack data, it automatically executes test tasks on the system under test, collects detection results and performance indicators, and outputs evaluation results including detection rate, response time and false alarm rate according to preset evaluation standards;

[0010] User feedback collection module: Receives requests from security operations personnel for annotation, correction, and improvement of evaluation results;

[0011] Adaptive strategy optimization module: Combines historical evaluation data and user feedback data to dynamically adjust attack generation parameters and testing strategies;

[0012] Multimodal data fusion and assessment module: Integrates heterogeneous data from multiple sources such as network traffic, system logs, user behavior, and device status, and constructs a risk situation map through feature fusion;

[0013] Visualized report generation module: Generates and outputs an interactive assessment report based on the assessment results and risk situation map.

[0014] Furthermore, the dynamic attack scenario generation module includes:

[0015] Threat intelligence gathering unit: Acquires structured attack signature information from threat intelligence databases and attack sample databases;

[0016] Attack template building unit: generates attack behavior templates;

[0017] Generative Network Unit: Generative adversarial networks or variational autoencoders are used to generate simulated attack data under constraints;

[0018] Discrimination and Filtering Unit: Evaluates the degree of feature matching between the simulated data and real historical samples.

[0019] Furthermore, the automated evaluation module includes:

[0020] Task scheduling subunit: Generates a test task queue based on attack scenario metadata;

[0021] Baseline test subunit: Records the defense response and performance metrics of the system under test under default configuration;

[0022] The indicator calculation subunit calculates evaluation indicators such as detection rate, response time, and false alarm rate.

[0023] Furthermore, the task scheduling subunit employs an improved weighted priority queue scheduling algorithm, and the task priority score is calculated using the following formula:

[0024] P i =w1·Risk Level i +w2·Scenario Diversity i +w3·Historical False Positive Rate i

[0025] Among them, w1, w2, and w3 are dynamically adjustable weight coefficients, and the system scores P based on priority. i Allocate test resources from high to low.

[0026] Furthermore, the adaptive strategy optimization module uses a large language model combined with an enhanced retrieval mechanism to iteratively optimize the task scheduling strategy and simulation parameter set.

[0027] Furthermore, the multimodal data fusion evaluation module includes:

[0028] Feature extraction unit: Extracts temporal and semantic features from various data sources and stores them in vector form;

[0029] Feature fusion unit: Generates fused feature vectors using a multimodal attention mechanism;

[0030] Risk map construction unit: Generates a three-dimensional risk map based on fused features.

[0031] Furthermore, the feature fusion unit uses a one-dimensional convolutional neural network to extract network traffic features, a Transformer encoder to extract system log features, and a long short-term memory network to extract user behavior sequence features.

[0032] To enable features from different modalities to be computed in the same space, a fully connected layer is first used to process the feature vectors F of each modality. k Mapping to a unified dimension d, the mapping formula is as follows:

[0033]

[0034] Among them, W k Let F be the weight matrix of the k-th eigenvector. k b is the original feature vector. k For bias terms;

[0035] Next, an attention mechanism is used to assign importance weights to the features of each modality. First, the attention score of each modality is calculated, and then the Softmax function is used for normalization to obtain the weight α. k :

[0036]

[0037] Score(·) can be calculated using an attention mechanism;

[0038] Finally, using the calculated attention weights, the weighted sum of all mapped modal features is obtained to get the final multimodal fusion feature F. fusion :

[0039]

[0040] The fusion feature F fusionThe input will be fed into the risk mapping construction unit to generate a comprehensive risk assessment.

[0041] Specifically, Score(·) can be calculated using dot product or MLP.

[0042] A cybersecurity assessment method based on generative artificial intelligence includes the following steps:

[0043] S1: Obtain anonymized threat intelligence data and construct attack behavior templates;

[0044] S2: Generate simulated attack data under security constraints using generative AI models and execute it in an isolated environment;

[0045] S3: Schedule test tasks based on simulated attack data and perform automated evaluation;

[0046] S4: Indicator Collection and Calculation;

[0047] S5: Receive user annotations and feedback on the evaluation results;

[0048] S6: Dynamically optimize the evaluation strategy and attack generation parameters;

[0049] S7: Construct a risk situation map by integrating multi-source heterogeneous data;

[0050] S8: Generates an interactive evaluation report that includes defense capability analysis and improvement suggestions.

[0051] Furthermore, in S2, the process of generating simulated attack data includes:

[0052] The generator receives conditional labels and random noise, and outputs a feature sequence vector of the simulated attack.

[0053] The discriminator assesses the similarity of the generated data to the real historical samples in terms of statistical patterns, ensuring that its KL divergence is below a preset threshold;

[0054] Policy constraints ensure that all generated data is a non-executable, structured description.

[0055] Furthermore, the step of generating the evaluation report includes:

[0056] Transform assessment process data and risk maps into scalable topological views and time trend curves;

[0057] It supports users in filtering and backtracking analysis of single assets or attack paths.

[0058] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0059] 1. The network security evaluation system of this invention utilizes generative AI to dynamically generate simulated attack scenarios under security constraints, which can test the defense system's ability to respond to unknown threats and overcome the problem that traditional solutions can only detect known threats and cannot evaluate unknown ones.

[0060] 2. The network security evaluation system of this invention, through the adaptive policy optimization module of LLM+RAG, can generate policies and test task scheduling parameters based on real-time evaluation results, enabling the system to continuously optimize itself as the network threat situation evolves, breaking through the limitations of "static rules".

[0061] 3. The network security assessment system of this invention, through a multimodal data fusion assessment module and by using an attention mechanism for feature-level fusion, constructs a multi-dimensional risk map, thereby providing a more comprehensive and accurate global security situation view;

[0062] 4. The network security evaluation system exemplified in this invention integrates attack scenario generation, task scheduling, test execution, indicator calculation, and report generation into a fully automated closed-loop process. It can directly transform user feedback into the optimization of the next round of testing strategies, realize continuous and automated security monitoring and evaluation, and improve the efficiency of security operation and maintenance. Attached Figure Description

[0063] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0064] Figure 1 This is a schematic diagram of the system structure;

[0065] Figure 2 This is a flowchart illustrating the method. Detailed Implementation

[0066] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0067] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0068] Example 1: System Architecture and Module Collaboration

[0069] like Figure 1 As shown, this embodiment provides a cybersecurity evaluation system based on generative artificial intelligence, including:

[0070] Dynamic attack scenario generation module: Based on de-identified threat intelligence and attack feature templates, it generates simulated attack vectors and interaction data through a generative artificial intelligence model, and executes it in an isolated test environment to generate simulated attack data;

[0071] Automated evaluation module: Based on the simulated attack data, it automatically executes test tasks on the system under test, collects detection results and performance indicators, and outputs evaluation results including detection rate, response time and false alarm rate according to preset evaluation standards;

[0072] User feedback collection module: Receives requests from security operations personnel for annotation, correction, and improvement of evaluation results;

[0073] Adaptive strategy optimization module: Combines historical evaluation data and user feedback data to dynamically adjust attack generation parameters and testing strategies;

[0074] Multimodal data fusion and assessment module: Integrates heterogeneous data from multiple sources such as network traffic, system logs, user behavior, and device status, and constructs a risk situation map through feature fusion;

[0075] Visualized report generation module: Generates and outputs an interactive assessment report based on the assessment results and risk situation map.

[0076] Specifically, the implementation method of the dynamic attack scenario generation module is as follows:

[0077] Threat intelligence collection unit: Obtains structured attack feature information from threat intelligence databases and attack sample databases, and collects de-identified threat intelligence and feature data of historical attack events;

[0078] Attack template construction unit: Abstracts attack behavior into an information template containing protocol, data format, and timing characteristics to generate attack behavior template;

[0079] Generative Network Unit: Conditional Generative Adversarial Network (cGAN) or Variational Autoencoder (GAN) is used to generate simulated attack data under constraints. In this embodiment, a conditional generative adversarial network (cGAN) is used. The generator (G) receives input labels and random noise and outputs a new simulation feature sequence vector.

[0080] The discrimination and filtering unit evaluates the feature matching degree between the simulated data and the real historical samples. The discriminator (D) evaluates the similarity of the generated data and the historical samples in terms of statistical patterns and outputs a scalar that represents the probability that the input sequence is real data. In this embodiment, the threshold is set to 0.05. If the KL divergence is greater than this value, the generated data is considered unreasonable and is filtered out. Only the data that passes the discrimination will be sent to the next stage.

[0081] The specific implementation method of the automated evaluation module is as follows:

[0082] Task scheduling subunit: Generates a test task queue based on attack scenario metadata;

[0083] Baseline test subunit: Records the defense response and performance metrics of the system under test under default configuration;

[0084] The indicator calculation sub-unit calculates evaluation indicators such as detection rate, response time, and false alarm rate.

[0085] Specifically, the task scheduling subunit receives scenario metadata (such as scenario ID, risk level, diversity, etc.) output by the dynamic attack scenario generation module, and uses an improved weighted priority queue scheduling algorithm. The task priority score is calculated using the following formula:

[0086] P i =w1·Risk Level i +w2·Scenario Diversity i +w3·Historical False Positive Rate i

[0087] Among them, w1, w2, and w3 are dynamically adjustable weight coefficients, and the system scores P based on priority. i Allocate test resources from high to low.

[0088] The indicator calculation subunit counts the number of true positives (TP), false positives (FP), false negatives (FN), and true negatives (TN), and then calculates evaluation indicators such as detection rate, response time, and false alarm rate. Specifically:

[0089] Detection rate (DR):

[0090]

[0091] Where TP represents the number of times a real threat was correctly detected, and FN represents the number of times a real threat was missed.

[0092] Response time (RT):

[0093]

[0094] Among them, t alarm,i Let t be the trigger time of the i-th attack / event. attack,i The time when the system issues an alarm, where N is the total number of events;

[0095] False Alarm Rate (FPR):

[0096]

[0097] Wherein, FP represents the number of times a normal event was incorrectly marked as a threat, and TN represents the number of times a normal event was correctly identified as normal.

[0098] The user feedback collection module can automatically evaluate the results. If the system does not generate an alarm for a certain simulated attack or generates an erroneous alarm for normal traffic, the operators can manually annotate and correct it. This feedback will be stored in the feedback database.

[0099] The adaptive strategy optimization module uses a large language model (LLM) combined with an enhanced retrieval mechanism (RAG) to iteratively optimize the task scheduling strategy and simulation parameter set. The model periodically inputs new evaluation results and user feedback data into the LLM. The LLM, in conjunction with the RAG, retrieves relevant cases from the historical knowledge base, analyzes the shortcomings of the current strategy, and proposes adjustment suggestions.

[0100] The specific implementation method of the multimodal data fusion evaluation module is as follows:

[0101] Feature extraction unit: Extracts temporal and semantic features from various data sources and stores them in vector form;

[0102] Feature fusion unit: Generates fused feature vectors using a multimodal attention mechanism;

[0103] Risk map construction unit: Generates a three-dimensional risk map based on fused features.

[0104] Specifically, within a complete evaluation cycle, the feature extraction unit not only processes simulated attack data but also accesses real multi-source heterogeneous data. For network traffic, it uses a 1D CNN to extract pattern features such as packet size and time intervals. net For system logs, use the Transformer encoder to extract semantic features F. log For user behavior, LSTM is used to extract temporal features such as login location and operation sequence. user .

[0105] The feature fusion unit uses a one-dimensional convolutional neural network to extract network traffic features, a Transformer encoder to extract system log features, and a long short-term memory network to extract user behavior sequence features. net F log F user Features are mapped to the same dimension, and the weight α of each feature is calculated using an attention mechanism. k Finally, the fused feature vector F is generated. fusion .

[0106] To enable features from different modalities to be computed in the same space, a fully connected layer is first used to process the feature vectors F of each modality.k Mapping to a unified dimension d, the mapping formula is as follows:

[0107]

[0108] Among them, W k Let F be the weight matrix of the k-th eigenvector. k b is the original feature vector. k For bias terms;

[0109] Next, an attention mechanism is used to assign importance weights to the features of each modality. First, the attention score of each modality is calculated, and then the Softmax function is used for normalization to obtain the weight α. k :

[0110]

[0111] Score(·) can be calculated using either dot product or MLP. Dot product is the basic calculation method for attention mechanisms and is common knowledge in the field of natural language processing. It is widely used in the Transformer architecture. MLP stands for Multilayer Perceptron, which is a basic neural network structure and is widely used in computer vision (such as ResNet) and temporal models (such as LSTM). It will not be elaborated on here.

[0112] Finally, using the calculated attention weights, the weighted sum of all mapped modal features is obtained to get the final multimodal fusion feature F. fusion :

[0113]

[0114] The fusion feature F fusion The input will be fed into the risk map construction unit, which is based on the fusion feature F. fusion Construct a visualized 3D risk map that includes asset nodes, attack paths, and risk levels.

[0115] The visualization report generation module integrates automatically evaluated indicators and risk maps into an interactive report, which provides topology views, trend curves, improvement suggestions, and protection recommendations.

[0116] Example 2: Complete Workflow

[0117] like Figure 2 As shown, the method flow of this embodiment is described in detail below:

[0118] S1: Threat Intelligence Gathering and Template Construction: The system obtains information from de-identified threat intelligence sources and constructs attack templates such as "abnormal RDP login" and "suspicious PowerShell activity".

[0119] S2: Generative AI generates simulated attack data: Under security constraints, the generative network unit (such as GAN) generates a large number of simulated attack feature sequences based on the attack template, and after being filtered by the discrimination and filtering unit, it forms usable test cases.

[0120] S3: Automated scheduling and execution testing: The task scheduling subunit arranges the generated attack scenarios into a task queue according to the priority algorithm, and initiates tests on the system under test in an isolated environment in sequence.

[0121] S4: Indicator Collection and Calculation: The system automatically collects the response of the tested system and calculates key indicators such as detection rate, false alarm rate, and response time.

[0122] S5: User Feedback Collection: Security experts review the evaluation report, mark the system's missed and false alarms, and provide correction suggestions.

[0123] S6: Adaptive Strategy Optimization: The LLM optimization engine, combined with RAG, analyzes new evaluation results and user feedback, and automatically adjusts the weight parameters w1, w2, w3 of task scheduling and the parameters of the generated model to improve the effectiveness and relevance of the evaluation in the next round of testing.

[0124] S7: Multimodal Data Fusion and Risk Map Construction: The system integrates multi-source data such as network traffic, system logs, and user behavior to generate a global risk situation map, identifying the most vulnerable assets and the most likely attack paths.

[0125] S8: Generate interactive evaluation reports: The system generates interactive evaluation reports that include defense capability analysis and improvement suggestions to support security decisions.

[0126] Example 3: Key Algorithms and Data Examples

[0127] In this embodiment, w1 = 0.5, w2 = 0.3, and w3 = 0.2. As shown in Table 1, this is a statistical table of evaluation results for each scenario.

[0128] Table 1. Statistical Table of Evaluation Results for Each Scenario

[0129]

[0130] For scenario S001, with risk level = 0.9, scenario diversity = 0.7, and historical false negative rate = 0.3, its priority P = 0.5*0.9 + 0.3*0.7 + 0.2*0.3 = 0.72;

[0131] For scenario S002, P = 0.5 * 0.8 + 0.3 * 0.5 + 0.2 * 0.1 = 0.57;

[0132] For scenario S003, P = 0.5 * 0.7 + 0.3 * 0.8 + 0.2 * 0.4 = 0.67;

[0133] Scheduling order: S001(0.72)>S003(0.67)>S002(0.57).

[0134] Input scenario S001 data, system correct alarms (TP): 18, system missed alarms (FN): 2, system error alarms (FP): 2, system correctly ignored normal events (TN): 78.

[0135] Detection rate (DR) = 18 / (18+2)*100% = 90.0%;

[0136] False alarm rate (FPR) = 2 / (2+78)*100% = 2.5%.

[0137] Those skilled in the art should understand that the scope of the invention involved in this application is not limited to technical solutions formed by specific combinations of the above-mentioned technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-mentioned technical features or their equivalent features without departing from the inventive concept. For example, technical solutions formed by substituting the above-mentioned features with technical features disclosed in this application (but not limited to) that have similar functions.

Claims

1. A cybersecurity evaluation system based on generative artificial intelligence, characterized in that, include: Dynamic attack scenario generation module: Based on de-identified threat intelligence and attack feature templates, it generates simulated attack vectors and interaction data through a generative artificial intelligence model, and executes it in an isolated test environment to generate simulated attack data; Automated evaluation module: Based on the simulated attack data, it automatically executes test tasks on the system under test, collects detection results and performance indicators, and outputs evaluation results including detection rate, response time and false alarm rate according to preset evaluation standards; User feedback collection module: Receives requests from security operations personnel for annotation, correction, and improvement of evaluation results; Adaptive strategy optimization module: Combines historical evaluation data and user feedback data to dynamically adjust attack generation parameters and testing strategies; Multimodal data fusion and assessment module: Integrates heterogeneous data from multiple sources such as network traffic, system logs, user behavior, and device status, and constructs a risk situation map through feature fusion; Visualized report generation module: Generates and outputs an interactive assessment report based on the assessment results and risk situation map.

2. The network security evaluation system based on generative artificial intelligence according to claim 1, characterized in that, The dynamic attack scenario generation module includes: Threat intelligence gathering unit: Acquires structured attack signature information from threat intelligence databases and attack sample databases; Attack template building unit: generates attack behavior templates; Generative Network Unit: Generative adversarial networks or variational autoencoders are used to generate simulated attack data under constraints; Discrimination and Filtering Unit: Evaluates the degree of feature matching between the simulated data and real historical samples.

3. The network security evaluation system based on generative artificial intelligence according to claim 1, characterized in that, The automated evaluation module includes: Task scheduling subunit: Generates a test task queue based on attack scenario metadata; Baseline test subunit: Records the defense response and performance metrics of the system under test under default configuration; The indicator calculation sub-unit calculates evaluation indicators such as detection rate, response time, and false alarm rate.

4. The network security evaluation system based on generative artificial intelligence according to claim 3, characterized in that, The task scheduling subunit employs an improved weighted priority queue scheduling algorithm, and the task priority score is calculated using the following formula: P i =w1·Risk Level i +w2·Scenario Diversity i +w3·Historical False Positive Rate i Among them, w1, w2, and w3 are dynamically adjustable weight coefficients, and the system scores P based on priority. i Allocate test resources from high to low.

5. The network security evaluation system based on generative artificial intelligence according to claim 1, characterized in that, The adaptive strategy optimization module uses a large language model combined with an enhanced retrieval mechanism to iteratively optimize the task scheduling strategy and simulation parameter set.

6. The network security evaluation system based on generative artificial intelligence according to claim 1, characterized in that, The multimodal data fusion evaluation module includes: Feature extraction unit: Extracts temporal and semantic features from various data sources and stores them in vector form; Feature fusion unit: Generates fused feature vectors using a multimodal attention mechanism; Risk map construction unit: Generates a three-dimensional risk map based on fused features.

7. The network security evaluation system based on generative artificial intelligence according to claim 6, characterized in that, The feature fusion unit uses a one-dimensional convolutional neural network to extract network traffic features, a Transformer encoder to extract system log features, and a long short-term memory network to extract user behavior sequence features. To enable features from different modalities to be computed in the same space, a fully connected layer is first used to process the feature vectors F of each modality. k Mapping to a unified dimension d, the mapping formula is as follows: Among them, W k Let F be the weight matrix of the k-th eigenvector. k b is the original feature vector. k For bias terms; Next, an attention mechanism is used to assign importance weights to the features of each modality. First, the attention score of each modality is calculated, and then the Softmax function is used for normalization to obtain the weight α. k : Score(·) is calculated using an attention mechanism; Finally, using the calculated attention weights, the weighted sum of all mapped modal features is obtained to get the final multimodal fusion feature F. fusion : The fusion feature F fusion The input will be fed into the risk mapping construction unit to generate a comprehensive risk assessment.

8. A cybersecurity evaluation method based on generative artificial intelligence, characterized in that, Includes the following steps: S1: Obtain anonymized threat intelligence data and construct attack behavior templates; S2: Generate simulated attack data under security constraints using generative AI models and execute it in an isolated environment; S3: Schedule test tasks based on simulated attack data and perform automated evaluation; S4: Indicator Collection and Calculation; S5: Receive user annotations and feedback on the evaluation results; S6: Dynamically optimize the evaluation strategy and attack generation parameters; S7: Construct a risk situation map by integrating multi-source heterogeneous data; S8: Generates an interactive evaluation report that includes defense capability analysis and improvement suggestions.

9. The cybersecurity evaluation method based on generative artificial intelligence according to claim 8, characterized in that, In S2, the process of generating simulated attack data includes: The generator receives conditional labels and random noise, and outputs a feature sequence vector of the simulated attack. The discriminator assesses the similarity of the generated data to the real historical samples in terms of statistical patterns, ensuring that its KL divergence is below a preset threshold; Policy constraints ensure that all generated data is a non-executable, structured description.

10. The cybersecurity evaluation method based on generative artificial intelligence according to claim 8, characterized in that, The steps for generating the evaluation report include: Transform assessment process data and risk maps into scalable topological views and time trend curves; It supports users in filtering and backtracking analysis of single assets or attack paths.

Images