Network security situation awareness system capability evaluation method, system, device and medium

Abstract

The invention provides a network security situation awareness system capability evaluation method, system and device and a medium, and relates to the field of network security, and the method comprises the steps: determining an evaluation purpose based on a to-be-evaluated network security situation awareness system, and determining a test scene according to the evaluation purpose; based on the test scene, constructing a test network environment, selecting a simulation attack tool and an attack tactical, selecting network equipment to be attacked and a data type tool, generating a test flow, executing a test process, and collecting test data in the test process; and constructing a multi-dimensional network security situation awareness evaluation index based on the test scene, and based on the multi-dimensional network security situation awareness evaluation index, performing comparative analysis on the test data, the perception data obtained by the network security situation awareness system to be evaluated and the output perception result through the artificial intelligence large language model to obtain an evaluation result. According to the method and the system, the multi-aspect capability of the network security situation awareness system is evaluated, and the technical level of systematic network security evaluation is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of network security, and in particular to a network security situation awareness system capability evaluation method, system, device and medium. BACKGROUND

[0002] With the rapid development of new generation information technologies such as 5G, big data, artificial intelligence and the advent of the era of Internet of Everything, the threat sources and attack means of network security are constantly evolving, and the complexity and harm of network security are increasingly prominent, requiring a dynamic and comprehensive protection concept. Network security situation awareness is a technology that perceives, extracts, understands, evaluates and forecasts security elements that affect network situation in a large-scale network environment. Network security situation awareness system, as a real-time guardian of network security, is a main means to realize "all-weather and all-directional perception of network security situation". At present, there are a large number of network security situation awareness systems based on various technologies, and their network security situation awareness capabilities are uneven.

[0003] The current mainstream evaluation method generally uses virtualization technology to flexibly build attack and defense environments in the environment construction aspect, and focuses on respective dimensions in the evaluation aspect, and there is no automatic arrangement capability of automatically building evaluation environments and calling test tools according to test scenarios. SUMMARY

[0004] Therefore, the embodiments of the present application provide a network security situation awareness system capability evaluation method, system, device and medium, which realizes the evaluation of the capabilities of the network security situation awareness system in network security attack analysis, asset risk analysis, abnormal behavior analysis, situation display and the like, so as to improve the systematized network security evaluation technology level.

[0005] The embodiments of the present application provide the following technical solutions: a network security situation awareness system capability evaluation method, comprising: Based on the network security situation awareness system to be evaluated, determining an evaluation purpose, and determining a test scenario from a preset scenario library according to the evaluation purpose; Based on the test scenario, constructing a test network environment, selecting a simulated attack tool from a pre-constructed attack-type tool set and determining a corresponding attack tactic, selecting a network device to be attacked from a pre-constructed target machine-type tool set, and selecting a data-type tool from a pre-constructed data-type tool set; wherein the attack-type tool set, the target machine-type tool set and the data-type tool set are pre-constructed based on historical network security situation awareness and test data; According to the test network environment, the simulated attack tool and the corresponding attack tactic, the network device to be attacked and the data-type tool, generating a test flow and executing a test process, and collecting test data in the test process; Construct a multi-dimensional network security situation awareness evaluation index based on a test scene, and compare and analyze the test data with perception data and output perception results of a network security situation awareness system to be evaluated by using an artificial intelligence large language model based on the multi-dimensional network security situation awareness evaluation index, to obtain an evaluation result.

[0006] According to an embodiment of the present application, the containerized deployment operating system type image, the target machine type tool type image, the attack type tool type image, and the data type tool type image are managed in the form of container images.

[0007] According to an embodiment of the present application, the method further comprises: According to a comparison result of the test data and the perception data obtained by the network security situation awareness system to be evaluated, evaluation index scores of each network security situation awareness evaluation index are calculated, According to an influence degree of each network security situation awareness evaluation index on network security, weights of the network security situation awareness evaluation indexes are designed, and the evaluation result is obtained according to the evaluation index scores of the network security situation awareness evaluation indexes and the corresponding weights.

[0008] According to an embodiment of the present application, the multi-dimensional network security situation awareness evaluation index comprises: a visual global network topology generation capability, network traffic data collection integrity, log data collection integrity, open port and vulnerability information accuracy, threat intelligence accuracy, online asset identification capability, alarm pushing accuracy, and attack link restoration capability through multi-dimensional correlation events.

[0009] According to an embodiment of the present application, the attack type tools comprise vulnerability exploitation type attack tools, malicious code type attack tools, WEB application type attack tools, data stealing type attack tools, malicious email type attack tools, comprehensive orchestration attack type tools, malicious address access type tools, inappropriate content access type tools, and malicious behavior access type tools.

[0010] According to an embodiment of the present application, the method further comprises: after the construction of the test network environment and the determination of the network device to be attacked, simulating online network activity behaviors of a user to generate real network traffic through simulating operation behaviors of the user on network application programs.

[0011] According to an embodiment of the present application, the method further comprises: The real-time attack data stream and the offline attack data stream are started in parallel in the test network environment, traffic mirroring of the real-time attack data stream is performed on the network device of the attack path and is forwarded to the network security situation awareness system under test, and collection of real-time attack traffic data is completed; The offline attack data stream is labeled, denoised and normalized, a standardized test data set is generated, and then is sent to the network security situation awareness system under test.

[0012] The application also provides a network security situation awareness system capability evaluation system, comprising: A scene determination module is configured to determine an evaluation purpose based on the network security situation awareness system under test, and determine a test scene from a preset scene library according to the evaluation purpose; An environment construction and management module is configured to construct a test network environment based on the test scene, select a simulated attack tool and determine a corresponding attack tactic from a pre-constructed attack-type tool set, select a network device to be attacked from a pre-constructed target machine-type tool set, and select a data-type tool from a pre-constructed data-type tool set; wherein the attack-type tool set, the target machine-type tool set and the data-type tool set are pre-constructed based on historical network security situation awareness and test data; A test module is configured to generate a test process according to the test network environment, the simulated attack tool and the corresponding attack tactic, the network device to be attacked and the data-type tool, execute a test process, and collect test data in the test process; An evaluation module is configured to construct a multi-dimensional network security situation awareness evaluation index based on the test scene, compare and analyze the test data, awareness data obtained by the network security situation awareness system under test and output awareness results by an artificial intelligence large language model based on the multi-dimensional network security situation awareness evaluation index, and obtain an evaluation result.

[0013] The application also provides a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the network security situation awareness system capability evaluation method described above when executing the computer program.

[0014] The application also provides a computer readable storage medium storing a computer program for implementing the network security situation awareness system capability evaluation method described above.

[0015] Compared with the prior art, the at least one technical solution adopted by the embodiment of the present specification can achieve the beneficial effects at least including: the embodiment of the present application comprehensively evaluates the network security posture platform in terms of network security attack, asset risk analysis and abnormal behavior analysis, etc., the embodiment of the present application builds a network security posture perception product detection environment, uses virtualization technology and containerization technology to build various target machines, integrates various attack tools and data sets that can simulate mainstream attack scenes, creates a multi-dimensional network security posture perception capability evaluation index system, and uses an artificial intelligence large model to analyze the network security posture perception capability. The capability of the posture perception product in network attack, asset risk, abnormal behavior analysis, etc. can be comprehensively and effectively evaluated. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0017] Figure 1 is a first schematic diagram of the network security posture perception system capability evaluation method flow of the embodiment of the present application; Figure 2 is a second schematic diagram of the network security posture perception system capability evaluation method flow of the embodiment of the present application; Figure 3 is a first schematic diagram of the network security posture perception system capability evaluation system structure of the embodiment of the present application; Figure 4 is a second schematic diagram of the network security posture perception system capability evaluation system structure of the embodiment of the present application; Figure 5 is a structural schematic diagram of the computer device of the present application. DETAILED DESCRIPTION

[0018] The embodiments of the present application will be described in detail below with reference to the drawings.

[0019] Following make the application's implementation through specific concrete example, the person skilled in the art can easily understand the other advantages and efficacy of the application from the disclosure of this specification. Obviously, the described embodiments are only a part of the embodiments of the application, not all the embodiments. The application can also be implemented or applied by another different specific implementation, and the details in the specification can be based on different views and applications, various modifications or changes are made without departing from the spirit of the application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict. Based on the embodiments in the application, all other embodiments obtained by the person skilled in the art without creative labor belong to the scope of protection of the application.

[0020] The terms involved in the embodiments of the application are explained as follows: Threat: potential factor of undesirable event that may cause harm to system or organization.

[0021] Threat information: evidence-based knowledge, including context, attack mechanism, attack indicator, possible impact, etc. Used to describe existing or possible threats, so as to realize the response and prevention of threats.

[0022] Cybersecurity posture: through collecting network traffic, asset information, logs, vulnerability information, alarm information, threat information and other data, analyzing and processing network behavior and user behavior and other factors, mastering network security state, predicting network security trend, and carrying out display and early warning activities.

[0023] As shown in Figure 1 The embodiment of the application provides a network security posture system capability evaluation method, which comprises: S101. Based on the network security posture system to be evaluated, determine the evaluation purpose, and determine the test scene from the preset scene library according to the evaluation purpose; S102. Based on the test scene, construct a test network environment, select a simulated attack tool from a pre-constructed attack-type tool set and determine the corresponding attack tactics, select a network device to be attacked from a pre-constructed target machine-type tool set, and select a data-type tool from a pre-constructed data-type tool set; wherein the attack-type tool set, the target machine-type tool set and the data-type tool set are pre-constructed based on historical network security posture and test data; S103. According to the test network environment, the simulated attack tool and the corresponding attack tactics, the network device to be attacked and the data-type tool, generate a test process and execute a test process, and collect test data in the test process; S104. Construct a multi-dimensional network security situation awareness evaluation index based on the test scenario. Based on the multi-dimensional network security situation awareness evaluation index, compare and analyze the test data with the perception data and output perception results obtained by the network security situation awareness system to be evaluated through an artificial intelligence large language model to obtain the evaluation results.

[0024] This invention provides a method for evaluating the comprehensive capabilities of a network security situation awareness platform in network security attack analysis, asset risk analysis, abnormal behavior analysis, and situational awareness. This invention constructs a network security situation awareness product testing environment, establishes target machines simulating mainstream attack scenarios, and develops verification and analysis tools for the situation awareness system's ability to integrate network attacks, asset risks, and abnormal behaviors into security event analysis. This has resulted in a relatively systematic evaluation method, toolset, and system, thereby improving the level of systematic network security evaluation technology.

[0025] In one embodiment of the present invention, the method further includes: containerizing and deploying operating system images, target machine tool images, attack tool images, and data tool images, and managing the operating system images, target machine tool images, attack tool images, and data tool images in the form of container images.

[0026] In this embodiment, all network topology, operating system, target machine services, attack tools, and test data used in the evaluation network environment are made into Docker / OCI images (i.e., container images). These images are then placed into a private image repository according to their categories, with unified versions, unified pulls, and unified upgrades. In practice, each image can be tagged to ensure 100% reproducibility of the same evaluation scenario. The evaluation platform uses the `docker pull` command to pull images in seconds, enabling rapid distribution. Furthermore, a single command using `docker-compose` or Helm assembles the "OS + target machine + attack tools + data" into a complete topology, enabling rapid scenario assembly.

[0027] In one embodiment of the present invention, the method further includes: calculating the evaluation index scores of each network security situation awareness evaluation index based on the comparison results between the test data and the perception data obtained by the network security situation awareness system to be evaluated; designing weights for each network security situation awareness evaluation index based on the degree of influence of each network security situation awareness evaluation index on network security; and calculating the evaluation result based on the evaluation index scores and corresponding weights of each network security situation awareness evaluation index.

[0028] In specific implementation, such as Figure 2 As shown in this embodiment, a method for evaluating the capabilities of a network security situation awareness system includes the following steps: Based on the evaluation objectives, the required test scenarios are determined. Based on these scenarios, the algorithm automatically constructs the complex simulated network environment needed for the test, selects simulated attack tools and tactics, and builds the network devices to be attacked. It can quickly and automatically plan, deploy, expand, release, and schedule resources, allowing evaluation personnel to manage resources. Management of operating system images, target machine images, attack tool images, and data images is achieved using container images. The operating system images support mainstream operating system categories to meet the compatibility requirements of situational awareness probe deployment, target machine deployment, and attack tool deployment. Attack tools include: vulnerability exploitation tools, malware attack tools, web application attack tools, data theft attack tools, malicious email attack tools, comprehensive orchestration attack tools, malicious address access tools, inappropriate content access tools, and malicious behavior access tools. Attacked devices include application-based and host-based devices.

[0029] During the testing process, real data such as network traffic and host logs generated during the testing process are collected, and relevant data obtained from the product under test are collected and compared and analyzed.

[0030] A multi-dimensional network security situation awareness assessment index is constructed, and an artificial intelligence big language model is used to compare and analyze the various perception results of the network security situation awareness product to be evaluated with the actual situation.

[0031] The final evaluation result is calculated using S=u(x_1,x_2,...,x_n), where x represents the score of each individual evaluation indicator, and u is the overall evaluation function.

[0032] In this preferred embodiment, the multi-dimensional network security situation awareness assessment indicators include: Whether it can generate a visualized global network topology, whether network traffic data collection is complete, whether log data collection of hosts and security devices is complete, whether open port and vulnerability information is correct, whether threat intelligence is accurate, whether it can identify online assets, whether alarm push is accurate, and whether it can reconstruct a complete attack chain by correlating events through time, space, assets, and other dimensions.

[0033] In one embodiment of the present invention, the method further includes: after completing the construction of the test network environment and determining the network device to be attacked, simulating the user's online network activity behavior to generate real network traffic by simulating the user's operation behavior on the network application.

[0034] In this embodiment, user online network activity behavior is simulated to generate realistic network traffic by simulating user operations on network applications. Since there is definitely normal background traffic in a real network, if the target range only contains attack packets, the situational awareness platform can easily "identify the anomaly at a glance," leading to an inflated detection rate. However, by mixing in real network traffic generated by simulated user behavior, the test conditions become closer to reality, making the final measured detection rate, false alarm rate, and stress resistance more accurate.

[0035] In one embodiment of the present invention, the method further includes: starting real-time attack data streams and offline attack data streams in parallel in the test network environment; mirroring the real-time attack data streams on network devices along the attack path and forwarding them to the network security situation awareness system to be evaluated, thereby completing the collection of real-time attack traffic data; and labeling, denoising, and standardizing the offline attack data streams to generate a standardized test dataset, which is then sent to the network security situation awareness system to be evaluated.

[0036] In this embodiment of the invention, real-time attack data streams and offline attack data streams are launched in parallel in the test network environment. The evaluation system mirrors the currently occurring attack traffic (real-time attack data stream) and imports it into the network security situation awareness system under test in real time, enabling the system to perform real-time detection and test its real-time detection capabilities. On the other hand, this embodiment performs labeling, noise reduction, and normalization processing on the offline attack data stream to generate a standardized test dataset, which is then sent to the network security situation awareness system under test to measure its offline analysis capabilities.

[0037] like Figure 3 As shown, this application also provides a network security situation awareness system capability evaluation system 200, including: The scenario determination module 201 is used to determine the evaluation purpose based on the network security situation awareness system to be evaluated, and to determine the test scenario from the preset scenario library according to the evaluation purpose; The environment construction and management module 202 is used to construct a test network environment based on the test scenario, select simulated attack tools from a pre-built attack toolset and determine the corresponding attack tactics, select network devices to be attacked from a pre-built target toolset, and select data tools from a pre-built data toolset; wherein the attack toolset, the target toolset, and the data toolset are pre-built based on historical network security situation awareness and test data, respectively. The test module 203 is used to generate a test process based on the test network environment, the simulated attack tool and corresponding attack tactics, the network device to be attacked and the data tool, and then execute the test process and collect test data during the test process. The evaluation module 204 is used to construct a multi-dimensional network security situation awareness evaluation index based on the test scenario. Based on the multi-dimensional network security situation awareness evaluation index, the test data is compared and analyzed with the perception data and output perception results obtained by the network security situation awareness system to be evaluated through an artificial intelligence large language model to obtain the evaluation result.

[0038] This embodiment also includes a verification and analysis tool module, which provides a pre-built attack toolkit, the target machine toolkit, and the data toolkit.

[0039] In one embodiment, the network security situation awareness system capability evaluation system of this embodiment includes: The verification and analysis tools module provides attack tools, target machine tools, and data tools required for testing and evaluation. During the testing phase, the system can directly call the tools in this module.

[0040] The scenario determination module is used to determine the evaluation purpose based on the network security situation awareness system to be evaluated, and to determine the test scenario from a preset scenario library according to the evaluation purpose.

[0041] The environment building and management module is used to build and manage the network environment, attacking machines and attack tactics, and target machines to be attacked required for testing. It uses container images to manage operating system images, target machine images, attack tool images, and data images.

[0042] The testing module is used to select test scenarios, invoke verification and analysis tools, automate test process orchestration, and collect data such as network traffic and host logs generated during testing.

[0043] The evaluation module is used to construct a multi-dimensional network security situation awareness evaluation index system to evaluate the comprehensive capabilities of the network security situation awareness products under test in network security attack analysis, asset risk analysis, abnormal behavior analysis, and situation display.

[0044] like Figure 4 As shown, in practical implementation, this evaluation system mainly includes the following modules: The verification and analysis tools module provides attack tools, target machine tools, and data tools required for testing and evaluation. During the testing phase, the system can directly call these tools. This module offers three categories of verification and analysis tools: attack tools, target machine tools, and data tools. Attack tools simulate various attack scenarios to generate realistic attack characteristics. Target machine tools provide various types of target machines, including servers running various operating systems, personal computers, printers, etc. Data-based verification and analysis tools provide standardized and comprehensive data support for the testing environment through high-quality network security datasets. By deploying relevant offline attack traffic data on target machines and performing labeling, noise reduction, and normalization processing on the raw data, standardized test datasets are generated. These datasets simulate complex data scenarios in real network environments, allowing security products to asynchronously and offline analyze relevant attack data. This avoids the lack of security threat analysis capabilities in scenarios where traffic acquisition is impossible due to network environment or computing capacity limitations (such as large-scale denial-of-service attacks), and also verifies the applicability of security product detection algorithms in offline scenarios. The tool can directly transmit static data sets such as malicious traffic sets, malicious domains, inappropriate content, and sensitive data from data-driven tools to the situational awareness system to be evaluated, serving as data for its network security attack, asset risk analysis, and abnormal behavior analysis.

[0045] The environment building and management module can build and manage complex network environments, attack simulation tool environments, and target machine environments required for testing. It can quickly plan, deploy, scale, release, and schedule resources according to the requirements of attack tools, target machines, and testing scenarios. It manages operating system images, target machine images, attack tool images, and data images using container images. The operating system images support mainstream operating system categories to meet the compatibility requirements of situational awareness probe deployment, target machine deployment, and attack tool deployment.

[0046] The network environment construction and management unit primarily utilizes cloud virtualization, network topology simulation, network service simulation, and traffic simulation technologies to build the test range environment. Cloud virtualization enables efficient management and highly dynamic orchestration capabilities. Network topology simulation allows the system to create arbitrary network topologies, configure network services, and configure forwarding rules within the network range, achieving complex network topology simulation with connectivity, transparency, scalability, and practicality, supporting large-scale network experiments. Network service simulation allows the test range to simulate services such as social networks, routing protocols, service providers, domain name resolution, and public key infrastructure. The network simulator uses a discrete event model, allowing discrete steps and using events to communicate between simulated entities, thereby generating state transitions—that is, output results are generated based on event-triggered state changes. Network service simulation provides typical social media applications and application-level traffic simulation, increasing the realism of the test range scenario. In target machine environment construction, operating system images and target machine images are managed using container images. This allows for the rapid generation of various target machine types. The operating system images support mainstream operating system categories to meet the compatibility requirements of situational awareness probe deployment, target machine deployment, and attack tool deployment. Target machine images, attack tool images, and data images are configured with fixed image templates based on different testing scenarios, facilitating the rapid selection of relevant images for different scenarios and enabling fast and flexible deployment and environment construction.

[0047] The target machine environment construction unit is used to simulate actual target environments. By creating real or simulated target systems and data, it tests the effectiveness of attack tools and the capabilities of defense systems. It mainly includes three types: application-based target machines, host-based target machines, and malicious / sensitive data-based target machines. These target machines are primarily built using virtualization and containerization technologies for easy management and expansion. Among them: Application-based target machines are designed to simulate vulnerable systems by deploying specific web applications (such as Tomcat web containers) and databases (MySQL, Redis, etc.) as targets, thus mimicking potential victims of application service systems in real-world environments.

[0048] Host-based target machines are designed to simulate real-world end-user victims by deploying specific operating systems (Linux, Windows) as the targets.

[0049] Inappropriate and sensitive data target machines are used to simulate high-value data systems by placing sensitive data within the operating system or business system. This tests whether the security system can effectively detect unauthorized data acquisition attempts on such target machines during an attack.

[0050] After the network environment and target machine are built, user behavior simulation can be achieved. Real network traffic can be generated by simulating user operations on network applications, such as simulating online social networking activities such as web browsing and microblogging, as well as sending and receiving emails.

[0051] The attack tool environment construction and management unit includes tools for quickly generating various attack machines, setting attack targets, and providing a large number of known attack behaviors and effective attack tactics specific to certain organizations for simulation. These include: vulnerability exploitation tools, malware attack tools, web application attack tools, data theft attack tools, malicious email attack tools, comprehensive orchestration attack tools, malicious address access tools, inappropriate content access tools, and malicious behavior access tools. Among these, vulnerability exploitation tools can simulate attacks targeting system and application vulnerabilities (such as buffer overflows and SQL injection), ultimately testing the vulnerability detection capabilities of security products and the timeliness and effectiveness of patch updates; malware attack tools simulate the propagation and behavior of viruses, worms, Trojans, and other malicious code, ultimately verifying the effectiveness of antivirus software, sandbox analysis tools, and malware behavior monitoring systems; and web application attack tools can simulate common attacks on websites and web services, such as XSS and CSRF. The test includes several methods, including path traversal, to ultimately verify the security product's ability to detect web application attack traffic. Data theft attack tools simulate attacks that steal sensitive data (such as passwords and confidential files), ultimately verifying the security product's ability to detect attacks related to data encryption, access control, and data leakage. Malicious email attack tools simulate phishing and spam email attacks, ultimately verifying the security product's ability to detect potential malicious attack locations such as email addresses, email body content, related web hyperlinks, and email attachments. Malicious address access tools simulate active access to malicious IPs, domains, and websites, assessing whether the security product can utilize its own threat intelligence capabilities to detect corresponding malicious behavior, thus verifying the real-time and comprehensiveness of its threat intelligence capabilities. Inappropriate content access tools simulate active access to inappropriate or sensitive content, ultimately verifying whether the security product can detect user access to various inappropriate content. Malicious behavior access tools simulate abnormal malicious login operations such as mass brute-force SSH logins and MySQL logins, ultimately verifying whether the security product can effectively detect abnormal behavior of network devices.

[0052] The testing module can construct a complete test network environment by calling the attack tools, target machine tools, and data tools of the verification and analysis tool module according to the test scenario. After deploying the network security situation awareness platform, the system automatically orchestrates the test process based on the scenario and comprehensively collects, stores, and analyzes the network traffic, host logs, and other data generated during the test.

[0053] The evaluation module constructs a multi-dimensional network security situational awareness capability evaluation index system, mainly including dimensions such as whether it can generate a visualized global network topology, whether network traffic data collection is complete, whether log data collection from hosts and security devices is complete, whether open port and vulnerability information is correct, whether threat intelligence is accurate, whether it can identify online assets, whether alarm pushes are accurate, and whether it can reconstruct a complete attack chain by correlating events through time, space, assets, and other dimensions. An artificial intelligence large language model is used to compare the perception capabilities of each evaluation dimension with actual test results to obtain a score for each item. The final evaluation result is calculated using S=u(x_1,x_2,...,x_n), where x is the score of a single evaluation index, and u is the overall evaluation function.

[0054] In specific implementation, the evaluation process of this invention is as follows: The testers first select a test scenario based on the test objective. The system then automatically deploys the attack cluster and target machine cluster, selects data-driven tools, and automatically orchestrates the test process. It ensures proper isolation based on the test scenario. When an attack occurs, it guarantees that during the process of attack traffic traveling from the attack tool cluster to the target machine cluster, traffic mirroring can be easily performed on network devices along the attack path and forwarded to the situational awareness platform, thus completing the real-time collection of traffic data.

[0055] Testers can perform manual configuration. After automated deployment, testers can manually adjust settings, including network structure, attack cluster and target machine cluster size, and target machine type (host-type, server-type), providing more than six types of attack tools.

[0056] The system includes hundreds of known attack techniques and strategies from various organizations, which testers can configure accordingly.

[0057] Testers began implementing the tests. The system built attack tool services one by one. During this process, attack tool images needed to be selected from the image repository, and the node where the service container group resided needed to be chosen. Target machine services were then built one by one. This process also required selecting target machine images from the image repository and choosing the node where the service container group resided. Finally, data-driven tools were invoked to directly transmit static data sets, including malicious traffic sets, malicious domains, inappropriate content, and sensitive data, to the network security situation awareness system under test.

[0058] Recording and management of test process data. The system will record all attack behaviors and related data such as traffic, host logs, security devices, and ports generated during the test.

[0059] Monitoring and acquisition of network security situation awareness platform data. The system records all objective data collected and perceived by the tested platform, as well as all results derived by the platform based on this objective data, such as alarm information, asset risks, abnormal behavior analysis, and attack chain reconstruction.

[0060] Test Result Analysis. The system uses artificial intelligence to compare and analyze the actual test data and behaviors with the data and results obtained by the network security situation awareness platform. Based on the multi-dimensional network security situation awareness capability evaluation index system and overall evaluation function, the final evaluation result is obtained.

[0061] In one embodiment, a computer device is provided, such as Figure 5 As shown, it includes a memory 301, a processor 302, and a computer program stored on the memory 301 and executable on the processor 302. When the processor 302 executes the computer program, it implements the above-mentioned network security situation awareness system capability evaluation method.

[0062] Specifically, the computer device can be a computer terminal, a server, or a similar computing device.

[0063] In this embodiment, a computer-readable storage medium is provided, which stores a computer program that executes the above-described network security situation awareness system capability assessment method.

[0064] Specifically, computer-readable storage media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer-readable storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable storage media does not include transient media, such as modulated data signals and carrier waves.

[0065] Obviously, those skilled in the art should understand that the modules or steps of the above-described embodiments of the present invention can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the embodiments of the present invention are not limited to any particular hardware and software combination.

[0066] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for evaluating the capabilities of a network security situation awareness system, characterized in that, include: Based on the network security situation awareness system to be evaluated, the evaluation objective is determined, and test scenarios are selected from a preset scenario library according to the evaluation objective; Based on the test scenario, a test network environment is constructed, simulated attack tools are selected from a pre-constructed attack toolset and corresponding attack tactics are determined, network devices to be attacked are selected from a pre-constructed target toolset, and data tools are selected from a pre-constructed data toolset; wherein, the attack toolset, the target toolset, and the data toolset are pre-constructed based on historical network security situation awareness and test data, respectively. Based on the test network environment, the simulated attack tools and corresponding attack tactics, the network devices to be attacked and the data tools, a test process is generated and executed, and test data is collected during the test process. A multi-dimensional network security situation awareness assessment index based on test scenarios is constructed. Based on the multi-dimensional network security situation awareness assessment index, the test data is compared and analyzed with the perception data and output perception results obtained by the network security situation awareness system to be evaluated through an artificial intelligence large language model to obtain the evaluation results.

2. The network security situation awareness system capability evaluation method according to claim 1, characterized in that, Also includes: Containerize and deploy operating system images, target machine tool images, attack tool images, and data tool images, and manage these images using container images.

3. The method for evaluating the capabilities of a network security situation awareness system according to claim 1, characterized in that, Also includes: Based on the comparison results between the test data and the perception data obtained by the network security situation awareness system to be evaluated, the evaluation index scores of each network security situation awareness evaluation index are calculated. Based on the degree of impact of each network security situation awareness assessment indicator on network security, weights are designed for each network security situation awareness assessment indicator. The assessment results are calculated based on the assessment indicator scores and corresponding weights of each network security situation awareness assessment indicator.

4. The method for evaluating the capabilities of a network security situation awareness system according to claim 1, characterized in that, The multi-dimensional network security situation awareness assessment indicators include: the ability to generate a visualized global network topology, the completeness of network traffic data collection, the completeness of log data collection, the accuracy of open port and vulnerability information, the accuracy of threat intelligence, the ability to identify online assets, the accuracy of alarm push notifications, and the ability to reconstruct attack chains through multi-dimensional correlation events.

5. The method for evaluating the capabilities of a network security situation awareness system according to claim 1, characterized in that, The attack toolset includes vulnerability exploitation tools, malicious code attack tools, web application attack tools, data theft attack tools, malicious email attack tools, comprehensive orchestration attack tools, malicious address access tools, inappropriate content access tools, and malicious behavior access tools.

6. The method for evaluating the capabilities of a network security situation awareness system according to claim 1, characterized in that, Also includes: After completing the construction of the test network environment and identifying the network devices to be attacked, simulate the online network activity behavior of users to generate real network traffic by simulating the user's operation behavior on network applications.

7. The method for evaluating the capabilities of a network security situation awareness system according to claim 1, characterized in that, Also includes: In the test network environment, real-time attack data streams and offline attack data streams are started in parallel. The traffic of the real-time attack data stream is mirrored on the network devices along the attack path and forwarded to the network security situation awareness system to be evaluated, thus completing the collection of real-time attack traffic data. The offline attack data stream is labeled, denoised, and normalized to generate a standardized test dataset, which is then sent to the network security situation awareness system to be evaluated.

8. A network security situation awareness system capability evaluation system, characterized in that, include: The scenario determination module is used to determine the evaluation purpose based on the network security situation awareness system to be evaluated, and to determine the test scenario from the preset scenario library according to the evaluation purpose; The environment construction and management module is used to construct a test network environment based on the test scenario, select simulated attack tools from a pre-built attack toolset and determine the corresponding attack tactics, select network devices to be attacked from a pre-built target toolset, and select data tools from a pre-built data toolset; wherein the attack toolset, the target toolset, and the data toolset are pre-built based on historical network security situation awareness and test data, respectively. The testing module is used to generate a test process based on the test network environment, the simulated attack tool and corresponding attack tactics, the network device to be attacked and the data tool, and then execute the test process and collect test data during the test process. The evaluation module is used to construct a multi-dimensional network security situation awareness evaluation index based on the test scenario. Based on the multi-dimensional network security situation awareness evaluation index, the test data is compared and analyzed with the perception data and output perception results obtained by the network security situation awareness system to be evaluated through an artificial intelligence large language model to obtain the evaluation result.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the network security situation awareness system capability evaluation method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that executes the network security situation awareness system capability assessment method according to any one of claims 1 to 7.

Images