A sandbox monitoring data-based attack use case automatic scoring method and system

Abstract

The application discloses a kind of attack use case automatic scoring method and system based on sandbox monitoring data, belong to network security field.The method includes receiving the monitoring data package containing security scene label and behavior data;According to security scene label, the corresponding score configuration containing score dimension and original weight is matched;The original weight of each dimension is converted into dimension full score value;Get the check subitem of each dimension and the local weight of each subitem;Based on dimension full score value and the local weight sum of subitem, the subitem weight score value is calculated;For each check subitem, feature matching is carried out to behavior data, and binary determination result is output;The score of each dimension is calculated;The score of all dimensions is accumulated to obtain comprehensive score, according to comprehensive score, determine the availability level of attack use case, trigger corresponding downstream processing action.The application solves the problems of insufficient attack use case verification, weak security control, lack of auditability and scene adaptability in related technology.

Description

Technical Field

[0001] This invention belongs to the field of network security technology, specifically relating to an automated scoring method and system for attack cases based on sandbox monitoring data. Background Technology

[0002] In the field of cybersecurity, Business Intrusion and Attack Simulation (BAS) systems assess the security posture of enterprise networks by simulating real-world cyberattacks. Attack use cases, as a core element of the BAS system, directly determine the effectiveness of the security assessment based on their quality and availability. With the development of artificial intelligence technology, massive amounts of attack use cases can be automatically generated. How to systematically identify the availability of these use cases, ensure their security and controllability, and quantify their value has become a critical technical problem that urgently needs to be solved. Currently, existing technologies mainly suffer from the following shortcomings: The attack use case availability verification mechanism is insufficient: existing solutions mostly rely on manual review by experts or static code analysis, lacking verification of the actual execution effect of attack use cases in a real environment. For example, some studies only indirectly infer the validity of use cases through abstract syntax tree (AST) structure comparison and vector similarity matching, without designing actual vulnerability triggering and reproduction links, resulting in a high risk of distorted judgment results and inability to confirm whether the use cases are actually reachable.

[0003] Lack of security controls over attack cases: AI-generated attack cases may contain extremely high-risk or destructive payloads, and existing technologies lack effective isolation, verification, and risk filtering mechanisms. Deploying these cases without verification can easily disrupt real-world business networks.

[0004] The lack of an automated and auditable quantitative scoring mechanism: Existing scoring technologies rely on Transformer models for black-box reasoning, resulting in probabilistic and unreproducible outcomes. Furthermore, the source of scores is untraceable, failing to meet the stringent requirements of cybersecurity compliance reviews for a traceable and auditable judgment process. Simultaneously, existing scoring systems do not differentiate between security scenarios, failing to provide differentiated assessments based on the verification priorities of different scenarios such as perimeter, host, and endpoint.

[0005] The attack case library and vulnerability library are isolated: the attack cases and their corresponding vulnerability information lack effective correlation, resulting in data update delays, the inability to form an effective knowledge graph, and difficulty in supporting efficient threat assessment and response.

[0006] In summary, existing technologies have failed to address the technical challenges of systematically, automatically, and auditably performing usability verification, security control, and quality scoring of attack cases in the context of AI-generated attack cases in batches. Summary of the Invention

[0007] The purpose of this invention is to provide an automated attack case scoring method and system based on sandbox monitoring data, which can solve the problems of insufficient attack case verification, weak security control, and lack of auditability and scenario adaptability in related technologies.

[0008] In a first aspect, the present invention provides an automated attack case scoring method based on sandbox monitoring data, comprising the following steps: Receive monitoring data packets generated after attack test cases are executed in the sandbox environment. The monitoring data packets contain at least security scenario tags and behavioral data collected by monitoring probes. Based on the security scenario tags, match and load the scoring configuration corresponding to the current scenario from multiple pre-stored scenario scoring configurations. The scoring configuration contains multiple scoring dimensions and the original weights of each dimension under the scenario. Based on the sum of the original weights of all dimensions in the current scenario, the original weights of each dimension are converted into the full score of each dimension on a ten-point scale. For each scoring dimension, obtain at least one corresponding inspection sub-item and the local weight of each sub-item; based on the full score of the dimension and the sum of the local weights of all sub-items under that dimension, calculate the sub-item weight score of each sub-item; for each inspection sub-item, perform feature matching on the behavioral data in the monitoring data packet according to the preset automated judgment rules, and output a binary judgment result; calculate the score of each dimension, and the score of the dimension is equal to the sum of the sub-item weight scores of all the inspection sub-items that pass the inspection under that dimension; The scores from all dimensions are summed to obtain a comprehensive score. Based on the preset range in which the comprehensive score falls, the availability level of the attack case is determined, and corresponding downstream processing actions are triggered.

[0009] As an optional implementation, the scenario scoring configuration, inspection sub-items and their automated judgment rules are fixed in the configuration storage during system deployment, and are guaranteed not to be modified at runtime through a version hash locking mechanism; the monitoring probes include the eBPF kernel probe, the Zeek network resolver and the inotify file system monitor; the behavioral data includes system call sequences, network traffic logs and file system change logs.

[0010] As an alternative implementation method, security scenario tags are automatically generated based on the target environment characteristics of attack cases. The enumeration values ​​of security scenario tags include at least boundary security scenarios, host security scenarios, terminal security scenarios, traffic security scenarios, container security scenarios, and email security scenarios. The scoring dimension set and original weights in the scoring configuration corresponding to different security scenarios are different.

[0011] As an optional implementation, after calculating the sub-item weight score, a score calibration step is also included: The weight score of the last checked item in the same dimension is dynamically assigned to the dimension's full score minus the sum of the weight scores of all the checked items before it, to ensure that the sum of the weight scores of all items in the dimension equals the dimension's full score.

[0012] As an optional implementation, the automated judgment rules include regular expression matching, threshold comparison, or feature existence detection operations on specific structured fields in the behavioral data; the binary judgment results and the corresponding original monitoring data records are stored together to provide an auditable traceability path for the scoring results.

[0013] As an optional implementation, the preset interval includes at least: a first preset interval, a second preset interval, and a third preset interval; When the overall score is within the first preset range, the status of the attack test case will be marked as rejected; When the overall score is in the second preset range, the attack cases and the details of the sub-items that failed the inspection are manually reviewed. When the overall score is in the third preset range, the attack case status is marked as available, and the two-way synchronization update process between the attack case library and the vulnerability library is triggered.

[0014] As an optional implementation method, the bidirectional synchronous update process includes: Extract vulnerability association features from attack use cases that are marked as available. Vulnerability association features include at least CVE number, vulnerability type, affected components, and attack vector. Based on vulnerability association characteristics, establish or update a two-way mapping relationship between the attack case library and the vulnerability library; When a new valid attack case is added to the attack case database, the corresponding information is automatically synchronized back to the vulnerability database; and a full consistency check between the attack case database and the vulnerability database is triggered at a preset period.

[0015] As an alternative implementation, the method further includes a model iterative optimization step: Receive the results of manual review of the attack cases, and write the original data of the attack cases, the results of manual review, and the comprehensive score as labeled samples into the training data pool; When the number of labeled samples in the training data pool reaches a preset threshold, incremental training of the large language model used to generate attack cases or perform intelligent analysis is initiated to update the model parameters.

[0016] Secondly, this invention provides an automated attack case scoring system based on sandbox monitoring data, comprising: The scene recognition and configuration loading module is configured to: receive monitoring data packets generated after the attack test cases are executed in the sandbox environment. The monitoring data packets contain at least security scene tags and behavioral data collected by monitoring probes; and, based on the security scene tags, match and load the scoring configuration corresponding to the current scene from multiple pre-stored scene scoring configurations. The scoring configuration contains multiple scoring dimensions under the scene and the original weights of each dimension. The dimension weight normalization module is configured to: convert the original weights of each dimension into the full score of the dimension on a ten-point scale based on the sum of the original weights of all dimensions in the current scenario. The sub-item detection and scoring module is configured to: for each scoring dimension, obtain at least one corresponding inspection sub-item and the local weight of each sub-item; calculate the sub-item weight score of each sub-item based on the full score of the dimension and the sum of the local weights of all sub-items under that dimension; for each inspection sub-item, perform feature matching on the behavioral data in the monitoring data packet according to the preset automated judgment rules, and output a binary judgment result; calculate the score of each dimension, and the score of the dimension is equal to the sum of the sub-item weight scores of all the inspection sub-items that pass the inspection under that dimension; The comprehensive scoring and grading module is configured to: accumulate the scores of all dimensions to obtain a comprehensive score, and determine the availability level of the attack case based on the preset range of the comprehensive score, and trigger the corresponding downstream processing actions.

[0017] Thirdly, the present invention provides an electronic device including a memory and a processor, and computer instructions stored in the memory and running on the processor, wherein the computer instructions, when executed by the processor, perform the method described in the first aspect.

[0018] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention proposes an automated attack case scoring method and system based on sandbox monitoring data. By introducing multi-scenario isolated sandboxes to verify the actual execution of attack cases, and by automatically collecting heterogeneous big data from kernel-level monitoring components such as eBPF, Zeek, and inotify during execution, scoring is performed. This fundamentally solves the problem of distorted validity judgment caused by traditional solutions relying solely on static analysis or black-box reasoning. This invention achieves precise adaptation to different security scenarios such as boundaries, hosts, and terminals through automatic routing and loading of differentiated scoring configurations using scenario tags. Simultaneously, through a unique weight normalization and end-of-pipe correction mechanism, it eliminates precision drift caused by IEEE 754 floating-point operations at the algorithm level, ensuring that the results of the same attack case are completely consistent in any number of scoring attempts. Furthermore, each score can be fully audited and traced along the path of "total score → dimension → sub-item → original monitoring data record," meeting the compliance requirements of relevant regulations for auditable and traceable security testing processes. Furthermore, this invention constructs a closed-loop data governance mechanism that links the attack case library and the vulnerability library in two directions, and uses the expert review results to incrementally train the large model, realizing the system's self-evolution and significantly improving the efficiency and accuracy of automated quality inspection of massive attack cases. Attached Figure Description

[0019] Figure 1 This is a flowchart of an automated attack case scoring method based on sandbox monitoring data disclosed in an embodiment of the present invention; Figure 2 This is an architecture diagram of an automated attack case scoring system based on sandbox monitoring data disclosed in an embodiment of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0021] The technical solutions disclosed in the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0022] Example 1 like Figure 1 As shown, this embodiment provides an automated attack case scoring method based on sandbox monitoring data, including the following steps: Receive monitoring data packets generated after attack test cases are executed in the sandbox environment. The monitoring data packets contain at least security scenario tags and behavioral data collected by monitoring probes. Based on the security scenario tags, match and load the scoring configuration corresponding to the current scenario from multiple pre-stored scenario scoring configurations. The scoring configuration contains multiple scoring dimensions and the original weights of each dimension under the scenario. Based on the sum of the original weights of all dimensions in the current scenario, the original weights of each dimension are converted into the full score of each dimension on a ten-point scale. For each scoring dimension, obtain at least one corresponding inspection sub-item and the local weight of each sub-item; based on the full score of the dimension and the sum of the local weights of all sub-items under that dimension, calculate the sub-item weight score of each sub-item; for each inspection sub-item, perform feature matching on the behavioral data in the monitoring data packet according to the preset automated judgment rules, and output a binary judgment result; calculate the score of each dimension, and the score of the dimension is equal to the sum of the sub-item weight scores of all the inspection sub-items that pass the inspection under that dimension; The scores from all dimensions are summed to obtain a comprehensive score. Based on the preset range in which the comprehensive score falls, the availability level of the attack case is determined, and corresponding downstream processing actions are triggered.

[0023] The specific steps of this invention are as follows: Step S1: Scene recognition and configuration loading.

[0024] The scoring engine receives test case execution data packets pushed by the sandbox execution module via a message queue interface. This data packet is structured JSON or Protobuf formatted data and contains the following fields: 1. Security Scene Label (scene_label): Automatically generated by the sandbox execution module based on the objective characteristics of the target environment. Its enumerated values ​​include, but are not limited to, “boundary_security”, “host_security”, “endpoint_security”, “traffic_security”, “container_security”, and “email_security”.

[0025] 2. Behavioral data: eBPF system call sequence: [{timestamp, syscall_name, pid, uid, gid, args, ret}, ...]; Zeek traffic log: [{ts, uid, id.orig_h, id.orig_p, id.resp_h, id.resp_p,proto, service, payload}, ...]; inotify file change log: [{timestamp, path, mask, cookie, name}, ...]; 3. Execution status flags: "fully_succeeded", "partially_succeeded", or "failed". After the attack test case is executed, the sandbox execution module compares the actual collected sandbox behavior data (system call sequence, network traffic log, file change log) with the expected effect declared in the test case structured data, and outputs the verification results in three categories: "fully compliant / partially compliant / completely non-compliant".

[0026] The scoring engine parses the data packet, extracts the "safety scene label (scene_label)" field, and performs a string-by-string match against multiple scene scoring configurations pre-stored in the system configuration storage (such as ETCD or local YAML files). Each scene scoring configuration is a structured document that defines multiple dimensions used for scoring in that scene, the original weight of each dimension (reflecting the importance of that dimension), and the multiple inspection sub-items contained in each dimension and their judgment rules.

[0027] It is important to note that the security scenario label field (scene_label) in the data packet is not manually labeled, but is automatically generated by the sandbox execution module based on the objective characteristics of the target environment during the execution of attack test cases. Its automatic generation logic is as follows: During the environment preparation phase, the sandbox execution module reads the target environment fields (target_os, target_arch, target_services, etc.) from the structured data of the attack cases. Combined with the asset type identifier of the sandbox template, it automatically generates scenario tags and writes them into the monitoring data packet according to the following judgment rules: The target assets are network boundary devices such as firewalls, WAFs, and IDS, and the main monitoring data source is network traffic logs with the tag value of boundary_security.

[0028] The target asset is a Linux / Windows server operating system, and the main monitoring data sources include eBPF system call sequences and file system change logs → the tag value is host_security.

[0029] The target asset is a desktop terminal / mobile device, and EDR Agent telemetry data is available → the tag value is endpoint_security.

[0030] The target environment only provides network traffic PCAP data and has no executable sandbox environment → the label value is traffic_security.

[0031] The target asset is a Docker / K8s container instance, and the container runtime interface is reachable → with the tag value of container_security.

[0032] The target asset is a mail server / mail gateway, and the attack vector is based on the SMTP / IMAP protocol → the tag value is email_security.

[0033] The aforementioned judgment rules are pre-defined in the configuration storage of the sandbox execution module in the form of YAML / JSON configuration files, and use the same version hash locking mechanism as the scoring configuration object to ensure immutability. This automatic judgment mechanism ensures the objective consistency between scene labels and target environments, eliminating subjective biases that may be introduced by manual annotation.

[0034] When the same attack case is executed in different target environments, different scene_label values ​​may be generated due to different sandbox templates. This data-driven dynamic routing mechanism is a key technical prerequisite for the full automation of the scoring method, ensuring that the entire chain from sandbox execution to scoring and rating does not require manual selection of "which scoring table to use". After parsing the scene label, the scoring engine uses string matching to precisely match it with six sets of security scene scoring configuration objects preset in the system configuration storage to complete the scene configuration loading. The scoring configurations for each scene are shown in Tables 1 to 6.

[0035] Table 1. Scoring dimensions and weights for boundary security scenarios;

[0036] Table 2. Host security scenario scoring dimensions and weights;

[0037] Table 3. Scoring dimensions and weights for terminal security scenarios;

[0038] Table 4. Scoring dimensions and weights for traffic security scenarios;

[0039] Table 5. Scoring dimensions and weights for container security scenarios;

[0040] Table 6. Scoring dimensions and weights for email security scenarios;

[0041] Explanation of the technical sources for the scoring dimensions and weights of each scenario: The scoring dimensions for the six security scenarios in this solution, along with the original weights for each dimension, are not subjectively designed but derived from the following objective technical constraints: (1) Differences in monitoring data sources drive dimensional differentiation: There are fundamental differences in the sandbox monitoring data sources corresponding to different security scenarios. The core monitoring data for host security scenarios comes from eBPF system call sequences and inotify file change logs, while the core monitoring data for perimeter security scenarios comes from Zeek network traffic logs and Suricata / Snort rule matching results. Endpoint security scenarios rely on both EDR telemetry JSON data and eBPF process behavior events. Since the types of monitoring data that can be collected in each scenario are different, the types of sub-items that can be automatically detected by rules are necessarily different, which leads to the fact that the scoring dimension sets for each scenario are not interchangeable and must be designed differently. For example, the "privilege escalation and persistence" dimension in the host security scenario relies on the setuid / capset system call records collected by eBPF. These data are completely unavailable in the pure network traffic monitoring of the perimeter security scenario. Therefore, the perimeter security scenario cannot set this dimension. This defining limitation comes from the technical capability boundary of the monitoring data source, rather than subjective preference. Therefore, the dimensional configuration of the six scenarios is essentially an objective mapping of the hardware acquisition architecture capabilities of the sandbox cluster. The monitoring components (eBPF / Zeek / inotify / EDR Agent) deployed in the architecture determine which scoring dimensions can be used. The parameters can be pre-configured, but the constraints corresponding to the parameters are uniquely limited by the underlying hardware and data acquisition architecture and do not belong to the rules of intellectual activity.

[0042] (2) Weight allocation driven by the number of detectable sub-items: The original weight of each scoring dimension is positively correlated with the number of sub-items that can be automatically detected under that dimension and the discriminative power of the detection rules. Taking the host security scenario as an example: the "red team utilization value" dimension has the highest weight (28 / 100) because the system call sequence, file change log, and process behavior log that the sandbox execution module can collect are the most sufficient evidence collection for this dimension, the number of configurable sub-items is the largest (4), and the pass / fail judgment of each sub-item has a high degree of discrimination between high-quality use cases and low-quality use cases; the "command and script behavior" dimension has the lowest weight (12 / 100) because the detection of this dimension mainly relies on the statistics of system call types created by the process, and there are relatively few distinguishable sub-items. The weight ratio essentially reflects the availability of evidence and discriminative power of different dimensions in the automated detection environment, and is a resource allocation strategy objectively determined by the monitoring technology capabilities.

[0043] (3) Weight solidification mechanism throughout the system lifecycle: Six sets of scoring configuration objects are written to the configuration storage in YAML / JSON format during system deployment, and the version hash value of the configuration objects is calculated and stored at the same time. After the scoring engine loads the configuration objects at runtime, it first verifies whether the current version hash of the configuration object is consistent with the version hash recorded during deployment—if the hashes are inconsistent, loading is refused and an alarm is triggered. This mechanism technically eliminates the possibility of runtime weight tampering or drifting, ensuring that all attack cases are scored under a unified and immutable scoring configuration, and the weight parameters of any scoring for the same attack case are completely consistent.

[0044] Before performing multi-dimensional weighted scoring of the scene, a pre-state determination needs to be performed, specifically: The scoring engine parses the "execution status flag" field in the monitoring data packet and performs pre-processing routing: (1) If the execution status is marked as failed (sandbox execution failed, and the attack case did not generate any valid behavioral data), the scoring engine directly assigns a comprehensive score of 0, marks the status of the attack case as "unavailable", and terminates the subsequent sub-item rule matching and weight calculation process; at the same time, the inconsistency information between the previous judgment conclusion of the case and the sandbox test results is written into the training data pool for use in large model iterative optimization.

[0045] (2) If the execution status is marked as fully_succeeded or partially_succeeded, the scoring engine continues to execute steps S2 to S4 to calculate the comprehensive score based on the sub-item rule matching results.

[0046] Step S2: Dimension weight normalization.

[0047] After loading the scoring configuration for the current scene, the scoring engine performs dimension weight normalization. Let the current scene have n dimensions, and the original weight of the i-th dimension be w. i Let the total weight of all elements be Σw. Then, the full score M of the i-th dimension after conversion on a 10-point scale is... i Calculated by the following formula: M i =10× (w i / Σw); For example, in a host security scenario, with a total weight Σw=100, the full scores for each dimension are 1.2, 1.8, 2.0, 2.2, and 2.8, respectively, and the sum is strictly equal to 10.

[0048] Step S3: Sub-item detection and scoring.

[0049] For each scoring dimension, it is broken down into multiple inspection sub-items that can be automatically detected. The actual score of each sub-item is obtained through local weighting, while solving the problem of decimal precision error.

[0050] S31. Sub-item weight conversion and calibration: For the i-th dimension, let its full score be M. i There are m sub-items in this dimension, and the local weight of the j-th sub-item is v. ij The sum of the local weights of all sub-items under this dimension is Σv i Then the initial weight score p of the j-th sub-item. ij for: p ij =M i × (v ij / Σv i ); To prevent IEEE 754 floating-point operations from causing the accumulation of sub-item scores and the maximum score M of the dimension. i To address inconsistencies, this invention employs a tail-end compensation calibration mechanism. Specifically, for the first m-1 sub-items in this dimension, their scores are rounded to two decimal places; for the last sub-item (the m-th item), its score p im The value dynamically assigned is: p im =M i -Σ(p i1 +p i2 +...+p i(m-1) ); This forces the sum of the scores of all sub-items under this dimension to be always equal to M. i This completely eliminates calculation errors.

[0051] To address the negative score boundary issue in the end-of-line compensation calibration mechanism, this scheme employs a two-layer safeguard mechanism, as detailed below: Configuration constraint layer: The scoring configuration undergoes legality verification during system deployment. For each scoring dimension, the converted score of its last sub-item (the supplementary sub-item) is verified to be no less than the cumulative maximum rounding error of the preceding sub-items of that dimension (m-1)×0.005. Configuration objects that fail the verification are refused to be loaded and a configuration alarm is triggered. This constraint is very easy to satisfy in engineering (the weights are all positive integers and rounded down, such as the converted score of the last sub-item under the [50 / 30 / 20] configuration is about 0.4~0.6 points, which is much greater than the upper limit of the rounding error on the order of 0.01), thus eliminating the possibility of negative scores from a mathematical perspective.

[0052] Runtime bottom layer: Under extreme parameter configurations, the final sub-item score p calculated according to the formula im If p < 0, then p im Set it to 0, and then compare the actual total score of this dimension with the maximum score M of the dimension. i The deviation ε between the scores (with an upper bound not exceeding (m-1)×0.005, i.e., 0.025 points) is recorded in the scoring log; this deviation is then capped by the subsequent comprehensive scoring function S=min(10, ΣS i The uniform truncation and absorption has no substantial impact on the availability rating of attack use cases.

[0053] This solution differs from the simple weighted summation in existing technologies in the following three essential ways: (1) Automatic routing and loading of the scoring system: Existing general weighted algorithms require manual specification of scoring templates and weight parameters. This solution has created an original automatic routing and matching mechanism based on scene labels in the sandbox environment. The scoring engine parses the scene_label field in the monitoring data packet and automatically loads the exclusive scoring dimension set and weight configuration of the corresponding scene through string precise matching. No manual selection of scoring template is required throughout the process, realizing automatic mapping between heterogeneous monitoring data and differentiated scoring dimensions.

[0054] (2) Three-level scoring calibration mechanism: The existing general weighted algorithm only includes two steps: "weight assignment → weighted accumulation". This solution creates a unique three-level scoring calibration mechanism: "dimensional global normalization (converting the original weights of each dimension to a full score of 10) + sub-item local weight conversion (the full score of the dimension is distributed to each sub-item according to the sub-item weight ratio) + dynamic compensation for the last sub-item (the score of the last sub-item = the full score of the dimension - the sum of the scores of the first n-1 sub-items)". This mechanism is specifically designed to solve the precision drift problem of multi-level weighted accumulation under the IEEE 754 double-precision floating-point operation of computers. In binary representation, 0.1 is an infinitely repeating decimal 0.0001100110011.... After multiple multiplications, divisions and accumulations, the error gradually accumulates. The general weighted algorithm has no error elimination means. The end-compensation algorithm of this solution forcibly eliminates the floating-point accumulation error from the algorithm level, ensuring that the total score of a single-dimensional sub-item is strictly equal to the full score of the dimension and that the scoring results of any number of times for the same use case are completely consistent.

[0055] (3) Weighted input and output drive system linkage: The input of the existing general weighted algorithm is the score filled in manually or subjective evaluation, and the output is the conclusion for reference. The input of the weighted calculation in this scheme is limited to the automated binary judgment result (0 or 1) generated by kernel-level / network-level monitoring tools such as eBPF / Zeek / inotify after performing regular expression matching, threshold comparison and feature existence detection on massive sandbox execution data (tens of thousands of system call records, hundreds of file change events and dozens of network session logs). The weighted output directly triggers the status change of attack use cases (rejection / pending adjustment / available), work queue flow (pushing to expert review module), data synchronization (bidirectional update of use case library and vulnerability library) and writing of training data pool.

[0056] S32. Automated Sub-item Detection Based on Monitoring Data: For each inspection sub-item, the scoring engine calls its corresponding pre-compiled rule matcher to automatically determine the behavioral data in the monitoring data packet. The determination rules are deterministic operations on the structured fields of the monitoring data, including but not limited to regular expression matching, threshold comparison, and feature existence detection.

[0057] Taking the "privilege escalation and persistence" dimension in host security scenarios as an example, it includes three sub-items: Local privilege escalation exploit (local weight 50): The rule matcher traverses the eBPF system call sequence, checks for privilege escalation calls such as setuid(0) and capset(), and verifies whether the subsequent process UID becomes 0. If all conditions are met, output "Pass (1)", otherwise "Fail (0)".

[0058] Specific implementation logic: (1) Handling the problem of process identity association key - PID reuse.

[0059] Each system call record collected by eBPF is {timestamp, syscall_name, pid, uid, gid, args, ret}. After a process exits within the sandbox, its PID is reclaimed and reassigned; associating solely with the PID would lead to inconsistencies in process attribution. Therefore, a composite unique identifier of a process's lifecycle—"PID + process start timestamp"—is used; this is the basis for time-series association.

[0060] (2) Triggering conditions.

[0061] Any of the following privilege escalation evidence was detected and the call was successful (ret=0): Explicit privilege escalation calls: setuid(0), setresuid(0,0,0), capset(), etc., and return ret=0; Vulnerability-based privilege escalation: The eBPF kernel probe detects that kernel functions that modify credentials, such as commit_creds and prepare_kernel_cred, are being executed (overriding scenarios without explicit setuid calls).

[0062] (3) Verification time window.

[0063] The verification window T (configurable, aligned with the sandbox execution phase by default, recommended to be 2-10 seconds) is opened starting from the trigger time. The window must fall within the actual execution phase of the attack test case to avoid including process behaviors during the environment preparation phase and post-execution cleanup phase in the verification. If no evidence of credential change is found when the window times out, "Failed (0)" is output.

[0064] (4) Verification scope—process tree association.

[0065] The verification scope is limited to the triggering process itself plus its child process tree: The process tree is dynamically constructed using the PPID field in the fork / clone system call record; It covers the typical attack collaboration pattern of "parent process triggering vulnerability, child process inheriting root privileges"; Exclude other irrelevant processes globally (such as service processes that run as root within the sandbox itself).

[0066] (5) Judgment basis—document change event.

[0067] In the verification window, if the process credentials (uid / euid) of the triggering process or its child process are detected to change from non-zero to zero, or if the eBPF cred probe detects that the cred structure has been rewritten to root privileges, output "Pass (1)"; otherwise output "Fail (0)".

[0068] Supplementary explanation of the two types of privilege escalation paths: Explicit privilege escalation (setuid(0) returns success): Successful invocation itself proves that privilege escalation has been achieved. Subsequent credential verification serves as a double insurance to prevent the exception of "successful invocation but credential not effective".

[0069] Vulnerability-based privilege escalation (hijacking the cred structure, without explicit invocation): Determine whether the event depends on the eBPF credential probe and avoid this sub-item consistently scoring 0 points for this type of use case.

[0070] Persistent resident mechanism (local weight 30): The rule matcher traverses the inotify logs and checks for write operations to files in persistent paths such as / etc / crontab and ~ / .bashrc. If a write operation is detected, it outputs "Pass (1)"; otherwise, it outputs "Fail (0)".

[0071] Persistent Hiding of Permissions (Local Weight 20): The rule matcher checks for the existence of anti-forensic operations such as history -c. If detected, it outputs "Pass (1)", otherwise "Fail (0)".

[0072] The above sub-item judgment result x ij It is a binary value (1 or 0) and is associated with the corresponding monitoring data record row number and field value for storage, providing a basis for subsequent auditing.

[0073] Taking the three sub-items mentioned above as examples, all inspection sub-items in each scenario and dimension adopt the same approach—binding the sub-item detection conditions to specific, automatically detectable monitoring data feature fields. These detection rules cannot be directly executed by the human brain because the system call sequences generated during sandbox execution may reach tens of thousands and file change events may reach hundreds. The human brain cannot complete feature matching of system call sequences, regular expression comparison of path patterns, and event correlation analysis within a limited time.

[0074] S33. Dimensional Score Calculation: The final score S of the i-th dimension i It equals the sum of the weight scores of all passing sub-items under this dimension: S i =Σ(x ij ×p ij ); In other words, the score for a single dimension is the sum of the weighted scores of all passing sub-items under that dimension, while failing sub-items are not scored, thus accurately quantifying the quality of test cases in a single dimension.

[0075] Step S4: Comprehensive scoring and grading.

[0076] Overall rating: S total The sum of scores across all dimensions, capped at 10 points: S total =min(10,ΣS i ); After calculating the comprehensive score, the scoring engine triggers corresponding system actions based on the preset grading intervals: When 0≤S total If the value is ≤3, it is judged as "unavailable" and the attack use case status is set to "REJECTED".

[0077] When 4≤S total If the score is ≤7, it is judged as "to be adjusted", and the attack cases and details of failed sub-items are pushed to the work queue of the expert review module.

[0078] When 8≤S total When the value is ≤10, it is determined to be "AVAILABLE", the attack case status is updated to available, and the downstream data synchronization process is triggered.

[0079] Numerical Calculation Example: In order to more clearly illustrate the scoring process of the present invention, a numerical calculation is performed by taking the score of a certain attack use case in a host security scenario as an example.

[0080] 1. Weight normalization: As described in step S2, the full scores of each dimension in the host security scenario are determined as follows: command and script behavior (1.2), Agent event evidence (1.8), process / file impact (2.0), privilege escalation and persistence (2.2), red team utilization value (2.8).

[0081] 2. Sub-item detection and scoring (taking the "privilege escalation and persistence" dimension as an example): The full score of this dimension M=2.2, and the local weights of sub-item A (privilege escalation), sub-item B (persistence), and sub-item C (hiding) are 50, 30, and 20 respectively.

[0082] Score of sub-item A: 1.10; Score of sub-item B: 0.66; Score of sub-item C (using end compensation): 2.2-1.10-0.66=0.44.

[0083] Assuming that the attack use case successfully triggers privilege escalation (A passed) and persistence (B passed) during sandbox execution, but does not perform the hiding operation (C failed), then the score of this dimension S=1×1.10+1×0.66+0×0.44=1.76.

[0084] 3. Comprehensive scoring and grading: Assuming that the scores of the other four dimensions are: 1.2, 1.8, 2.0, 2.1 respectively.

[0085] Comprehensive score S total =1.76+1.2+1.8+2.0+2.1=8.86.

[0086] Since 8.86∈[8,10], the system determines that the attack use case is "AVAILABLE" and automatically triggers the two-way synchronous update of the attack use case library and the vulnerability library.

[0087] Dual-library linkage and model iteration: After the attack use case is marked as "available", the system triggers the two-way linkage update mechanism of the attack use case library and the vulnerability library. The data synchronization module extracts key features such as CVE number, vulnerability type, affected component, attack vector, and Payload from the structured data of the use case. Then, the multi-field binding relationship is established or updated between the attack use case library and the vulnerability library, forming a two-way mapping of "attack use case → vulnerability" and "vulnerability → attack use case". Meanwhile, the data and scoring details of the use case will be written into the training data pool.

[0088] Once the expert review module completes its review of "questionable" use cases, its judgment results are also fed back into the training data pool. When the accumulated labeled samples reach a threshold, the system automatically starts incremental training of the large model, solidifying expert experience into the model and achieving continuous iteration of judgment capabilities.

[0089] The method for bidirectional linkage and updating of the attack case library and vulnerability library is as follows: This method details the process of triggering a two-way synchronized update between the attack test case database and the vulnerability database after scoring is completed. This process is executed by the data synchronization module, ensuring the consistency of the data in the two databases and the real-time nature of their correlation.

[0090] This method includes two modes: periodic full verification and incremental real-time synchronization.

[0091] In the periodic full-data verification mode, the system's timed scheduler (e.g., configured to trigger at 00:00 daily) initiates a full-data traversal verification command. The data synchronization module simultaneously retrieves full data from both the attack test case library and the vulnerability library, traversing all established binding associations. During this traversal, the module cleans up erroneous associations across scenarios, expired invalid associations, and duplicate association records, regenerating accurate mapping relationships. Simultaneously, the module filters out data that has not been iterated for a long time, has mismatched features, or has failed sandbox testing, generating an abnormal data list and pushing alerts to administrators.

[0092] In incremental real-time synchronization mode, this process is triggered immediately when the scoring process in Example 1 marks an attack use case as "available". The specific steps are as follows: Step S201: Extract attack use case features.

[0093] The data synchronization module extracts vulnerability association features from the structured data of newly added "available" attack cases. These features include at least the CVE number, vulnerability type, affected components, and attack vector (payload). For example, for the Mirai use case, the module will extract its associated IoT vulnerability number, target component (such as Telnet service), and attack payload features.

[0094] Step S202: Establish a bidirectional mapping relationship.

[0095] The module uses these features as keys to retrieve corresponding vulnerability records in the vulnerability database. If a matching record is found, a bidirectional mapping relationship of "attack use case → vulnerability" and "vulnerability → attack use case" is established or updated between the corresponding attack use case record in the attack use case database and the corresponding vulnerability record in the vulnerability database. This association involves multiple fields, not just a simple binding of CVE numbers.

[0096] Step S203: Reverse synchronization of the vulnerability database.

[0097] The module will back-synchronize newly added valid use case information (including its attack vector, exploitation conditions, sandbox execution result summary, etc.) to the corresponding vulnerability record in the vulnerability database, enriching the practical information of the vulnerability.

[0098] Step S204: Trigger full consistency check.

[0099] As a periodic supplement, the system will also trigger a full consistency check between the attack case database and the vulnerability database at a preset period (e.g., weekly) to find and fix problems such as broken relationships or inconsistent data caused by various reasons (e.g., database anomalies), and ensure the long-term accuracy and integrity of the mapping relationship between the two databases.

[0100] The large-scale intelligent judgment and expert collaborative iterative method are as follows: This method describes a closed-loop process that utilizes a large model for initial screening of attack cases and combines expert review results for iterative model optimization. This method aims to address the problems of low efficiency in purely manual review and high false positive rates in purely model-based inference in the context of massive attack cases. The process mainly includes the following steps: Step S301: Structured analysis of attack test cases.

[0101] The NLP attack case parsing module receives unstructured attack case text (such as vulnerability descriptions or attack scripts uploaded by users). This module employs a supervised, fine-tuned DeepSeek-LLM, combined with few-shot prompts and JSON Schema structured output constraints, to transform the raw text into standardized JSON data containing key fields such as asset type, attack type, vulnerability number, target environment, and attack chain. During parsing, the model parameters are set to "temperature=0" and "top". k =1” to ensure the determinism of the output.

[0102] Step S302: Intelligent analysis of large models.

[0103] The large-scale intelligent analysis module receives the structured data output from step S301. This module is based on DeepSeek-LLM and uses RAG (Retrieval Enhanced Generation) technology to dynamically retrieve local vulnerability and attack case libraries, and performs usability assessment and applicable security scenario inference for attack cases.

[0104] Specifically, the module verifies the vulnerability version matching, whether the exploitation conditions are met, and the compatibility between the payload and the target environment. Simultaneously, it performs parallel inference through three independent prompts, taking the majority consensus. The inference process is also subject to hard constraints from the rule engine (e.g., the CVE number must exist in the vulnerability database). If the inference result is convincing, it directly outputs "available" or "unavailable" along with the basis for the judgment; if the model's confidence in a certain use case is low, it marks it as "questionable" and triggers step S303.

[0105] Step S303: Expert manual review.

[0106] "Questionable" attack cases and their model inference process information are pushed in real time (with a delay of ≤1 second) to the work queue of the expert review module. Attack and defense experts can view the original test cases, structured elements, inference logs, and sandbox replays (if any) through a visual front-end interface. Experts make a final judgment based on their experience and can revise the model's conclusions, supplement exploitation conditions, or adjust the risk level. The reviewed results overwrite the original model output and carry a "Manually Reviewed" label.

[0107] In the system flow of this invention, the large-scale model intelligent judgment module and the expert review module are pre-filtering layers. Their function is to perform preliminary usability screening on attack cases, eliminating obviously invalid or unusable cases to reduce the consumption of sandbox resources by invalid cases. However, the large-scale model judgment results rely on semantic reasoning and knowledge base matching, and their conclusions are probabilistic; only the actual execution results in the sandbox environment are the most objective criteria for the usability of attack cases. Therefore, when the usability conclusion output by the scenario multi-dimensional weighted scoring module based on sandbox monitoring data is inconsistent with the preliminary judgment conclusion of the large-scale model / expert, the conclusion of the scenario multi-dimensional weighted scoring module shall be the final decision.

[0108] Specifically: For attack cases judged as "usable" by the large model / experts but "unusable" by the scenario multidimensional weighted scoring module, the system will use the conclusion of the scenario multidimensional weighted scoring module as the standard, overwrite the status of the attack case as "rejected," and feed back the inconsistency information to the large model intelligent judgment module as a labeled sample for model optimization training; For attack cases judged as "usable" by the large model / experts but "needs adjustment" by the scoring module, the system will push the attack case and the details of the failed sub-items to the expert review module for secondary review.

[0109] Step S304: Iterative optimization of the model.

[0110] After expert review, the system writes the original attack cases, model output, expert correction records, and final scoring results into the training data pool as a labeled sample. When the number of labeled samples in the data pool reaches a preset threshold (e.g., 1000), an incremental training task is automatically triggered. The training unit merges the new samples with the historical training set and fine-tunes DeepSeek-LLM using supervised learning, guiding the model to learn the attack and defense logic behind the expert judgments. After training, the effect is evaluated using a validation set. If the judgment accuracy improvement meets expectations, the optimized model parameters are updated and deployed, completing the closed loop of "intelligent judgment - expert feedback - model evolution".

[0111] The scoring engine of this invention is a pure deterministic rule engine. During the scoring calculation process, no AI model is called for reasoning or scoring to ensure the reproducibility and auditability of the scoring results. The large language model belongs to the pre-scoring processing stage, and its function is limited to: (1) performing preliminary usability screening and applicable scenario identification for attack cases before they enter the sandbox for execution; (2) assisting security experts in mining candidate sub-items and judgment rules from historical attack cases and attack feature databases during the configuration generation stage of scoring dimensions, inspection sub-items and judgment rules (system development period). The generated rules are solidified in the configuration storage and version hash locking after being confirmed by security experts. During the scoring process, the scoring engine only executes the solidified deterministic judgment rules and weight calculation based on the monitoring data collected by the sandbox execution module, without involving any AI reasoning, thereby ensuring that the scoring results of any attack case are completely consistent and that each score can be audited and traced along the path of "total score → dimension → sub-item → original record of monitoring data".

[0112] The embodiments employ a combination of eBPF, Zeek, and inotify, but the invention is not limited thereto. In other embodiments, SystemTap can be used instead of eBPF, Snort or Suricata can be used instead of Zeek for network traffic analysis, or fanotify can be used instead of inotify for file system monitoring. All variations fall within the protection scope of this invention.

[0113] In this embodiment, the scoring configuration is fixed in the form of a YAML file. In other embodiments, the configuration can be stored in a relational database, a NoSQL database, or a distributed configuration center, and its integrity and immutability can be guaranteed by hash value verification or digital signature.

[0114] The embodiments illustrate six core security scenarios, but the present invention is not limited thereto. Those skilled in the art can add or adjust scenario categories and their corresponding scoring configurations according to actual business needs, which also constitutes a simple variation of the present invention.

[0115] Example 2 like Figure 2 As shown, this embodiment provides an automated attack case scoring system based on sandbox monitoring data, including: The scene recognition and configuration loading module is configured to: receive monitoring data packets generated after the attack test cases are executed in the sandbox environment. The monitoring data packets contain at least security scene tags and behavioral data collected by monitoring probes; and, based on the security scene tags, match and load the scoring configuration corresponding to the current scene from multiple pre-stored scene scoring configurations. The scoring configuration contains multiple scoring dimensions under the scene and the original weights of each dimension. The dimension weight normalization module is configured to: convert the original weights of each dimension into the full score of the dimension on a ten-point scale based on the sum of the original weights of all dimensions in the current scenario. The sub-item detection and scoring module is configured to: for each scoring dimension, obtain at least one corresponding inspection sub-item and the local weight of each sub-item; calculate the sub-item weight score of each sub-item based on the full score of the dimension and the sum of the local weights of all sub-items under that dimension; for each inspection sub-item, perform feature matching on the behavioral data in the monitoring data packet according to the preset automated judgment rules, and output a binary judgment result; calculate the score of each dimension, and the score of the dimension is equal to the sum of the sub-item weight scores of all the inspection sub-items that pass the inspection under that dimension; The comprehensive scoring and grading module is configured to: accumulate the scores of all dimensions to obtain a comprehensive score, and determine the availability level of the attack case based on the preset range of the comprehensive score, and trigger the corresponding downstream processing actions.

[0116] The above four modules all belong to the scenario-based multi-dimensional weighted scoring module and are part of the decision-making layer, making them core modules of the system. This module is configured to receive monitoring data packets pushed by the multi-scenario sandbox execution module through a message queue, and based on... Figure 1 The illustrated method flow executes scenario routing, configuration loading, normalization conversion, sub-item rule matching and binary judgment, score calculation, error correction, and comprehensive rating. This module is essentially a deterministic rule engine implemented in Go, where all scoring configurations are fixed in YAML structure during system deployment and locked via version hashing.

[0117] The automated scoring system of the present invention also includes: The NLP attack test case parsing module, belonging to the application layer, is configured to cleanse, semantically understand, and structurally map unstructured attack test case text uploaded by users or collected through interfaces, outputting standard structured data containing fields such as asset type, attack type, vulnerability number, risk level, and target scenario. Its technical implementation is based on a supervised, fine-tuned version of DeepSeek-LLM and Few-shot Prompt.

[0118] The large-scale intelligent analysis module, belonging to the decision-making layer, is configured to receive the structured data and, combined with knowledge from the local vulnerability database and attack case database dynamically retrieved by RAG, intelligently analyze the effectiveness, exploitation conditions, applicable scenarios, and severity level of attack cases, outputting a "usable / questionable / unusable" conclusion and its basis. Questionable attack cases are pushed to the expert review module.

[0119] The expert review module, belonging to the decision-making layer, is configured to provide a visual expert workbench to display the large model's assessment of questionable use cases and to receive the final rulings from attack and defense experts. These rulings are used to overwrite the model output and serve as labeled samples for subsequent iterations of training.

[0120] The multi-scenario sandbox execution module, belonging to the core execution layer, contains an environment management unit and an attack case execution unit. This module builds an isolated sandbox cluster based on Docker / gVisor / KVM, used to automatically execute attack cases deemed "usable" in the corresponding isolated environment. The module integrates eBPF, Zeek, and inotify as monitoring probes, collecting system calls, network traffic, and file behavior data throughout the execution process, and automatically destroying the sandbox environment upon completion.

[0121] The attack test case library and vulnerability library bidirectional linkage module belongs to the data layer. It is configured to execute the bidirectional linkage update method of the attack test case library and vulnerability library, maintain the multi-field bidirectional mapping relationship between attack test cases and vulnerabilities, and be responsible for the reverse synchronization of newly added test cases and periodic full consistency verification.

[0122] It should be noted that the above modules correspond to the steps in Embodiment 1, and the examples and application scenarios implemented by the above modules and their corresponding steps are the same, but are not limited to the content disclosed in Embodiment 1. It should also be noted that the above modules can be executed in a computer system as part of the system.

[0123] In further embodiments, the following is also provided: An electronic device includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, which, when executed by the processor, perform the method described in Embodiment 1. For brevity, further details are omitted here.

[0124] It should be understood that in this embodiment, the processor can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0125] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. An automated attack test case scoring method based on sandbox monitoring data, characterized in that, Includes the following steps: Receive monitoring data packets generated after attack test cases are executed in a sandbox environment, wherein the monitoring data packets contain at least security scenario labels and behavioral data collected by monitoring probes; Based on the security scenario label, the scoring configuration corresponding to the current scenario is matched and loaded from multiple pre-stored scenario scoring configurations. The scoring configuration includes multiple scoring dimensions and the original weights of each dimension under the scenario. Based on the sum of the original weights of all dimensions in the current scenario, the original weights of each dimension are converted into the full score of each dimension on a ten-point scale. For each scoring dimension, obtain at least one corresponding inspection sub-item and the local weight of each sub-item; based on the full score of the dimension and the sum of the local weights of all sub-items under that dimension, calculate the sub-item weight score of each sub-item; For each inspection sub-item, feature matching is performed on the behavioral data in the monitoring data packet according to the preset automated judgment rules, and a binary judgment result is output. Calculate the score for each dimension, where the score for a dimension is equal to the sum of the weight scores of all sub-items that passed the inspection under that dimension; The scores of all dimensions are summed to obtain a comprehensive score. Based on the preset range in which the comprehensive score falls, the availability level of the attack case is determined, and corresponding downstream processing actions are triggered.

2. The automated attack case scoring method based on sandbox monitoring data as described in claim 1, characterized in that, The scenario scoring configuration, the inspection sub-items, and their automated judgment rules are fixed in the configuration storage during system deployment and are guaranteed to be unmodifiable during runtime through a version hash locking mechanism; the monitoring probes include the eBPF kernel probe, the Zeek network resolver, and the inotify file system monitor; the behavioral data includes system call sequences, network traffic logs, and file system change logs.

3. The automated attack case scoring method based on sandbox monitoring data as described in claim 1, characterized in that, The security scenario labels are automatically generated based on the target environment characteristics of the attack use cases. The enumeration values ​​of the security scenario labels include at least boundary security scenarios, host security scenarios, terminal security scenarios, traffic security scenarios, container security scenarios, and email security scenarios. The scoring dimension set and original weights in the scoring configuration corresponding to different security scenarios are all different.

4. The automated attack case scoring method based on sandbox monitoring data as described in claim 1, characterized in that, After calculating the sub-item weight score, the process also includes a score calibration step: The weight score of the last checked item in the same dimension is dynamically assigned to the dimension's full score minus the sum of the weight scores of all the checked items before it, so as to ensure that the sum of the weight scores of all items in the dimension is equal to the dimension's full score.

5. The automated attack case scoring method based on sandbox monitoring data as described in claim 1, characterized in that, The automated judgment rules include regular expression matching, threshold comparison, or feature existence detection operations on specific structured fields in the behavioral data; the binary judgment results and the corresponding original monitoring data records are stored together to provide an auditable and traceable path for the scoring results.

6. The automated attack case scoring method based on sandbox monitoring data as described in claim 1, characterized in that, The preset interval includes at least: a first preset interval, a second preset interval, and a third preset interval; When the comprehensive score is within the first preset range, the status of the attack case is marked as rejected; When the overall score is within the second preset range, the attack cases and the details of the failed sub-items are manually reviewed. When the comprehensive score is within the third preset range, the status of the attack case is marked as available, and a two-way synchronous update process between the attack case library and the vulnerability library is triggered.

7. The automated attack case scoring method based on sandbox monitoring data as described in claim 6, characterized in that, The bidirectional synchronous update process includes: Extract vulnerability association features from attack use cases marked as available, wherein the vulnerability association features include at least CVE number, vulnerability type, affected components, and attack vector; Based on the vulnerability association characteristics, a bidirectional mapping relationship is established or updated between the attack use case library and the vulnerability library; When a new valid test case is added to the attack test case library, the corresponding information is automatically reverse-synchronized to the vulnerability library; and a full consistency check between the attack test case library and the vulnerability library is triggered at a preset period.

8. The automated attack case scoring method based on sandbox monitoring data as described in claim 1, characterized in that, The method also includes a model iterative optimization step: Receive the manual review results of the attack cases, and write the original data of the attack cases, the manual review results, and the comprehensive score as labeled samples into the training data pool; When the number of labeled samples in the training data pool reaches a preset threshold, incremental training of the large language model used to generate the attack cases or to perform intelligent analysis is initiated to update the model parameters.

9. An automated attack case scoring system based on sandbox monitoring data, characterized in that, include: The scene recognition and configuration loading module is configured to receive monitoring data packets generated after an attack case is executed in a sandbox environment. The monitoring data packets contain at least security scene tags and behavioral data collected by monitoring probes. Based on the security scenario label, the scoring configuration corresponding to the current scenario is matched and loaded from multiple pre-stored scenario scoring configurations. The scoring configuration includes multiple scoring dimensions and the original weights of each dimension under the scenario. The dimension weight normalization module is configured to: convert the original weights of each dimension into the full score of the dimension on a ten-point scale based on the sum of the original weights of all dimensions in the current scenario. The sub-item detection and scoring module is configured to: for each scoring dimension, obtain at least one corresponding sub-item to be checked and the local weight of each sub-item; and calculate the sub-item weight score of each sub-item based on the full score of the dimension and the sum of the local weights of all sub-items under that dimension. For each inspection sub-item, feature matching is performed on the behavioral data in the monitoring data packet according to the preset automated judgment rules, and a binary judgment result is output. Calculate the score for each dimension, where the score for a dimension is equal to the sum of the weight scores of all sub-items that passed the inspection under that dimension; The comprehensive scoring and grading module is configured to: accumulate the scores of all dimensions to obtain a comprehensive score, and determine the availability level of the attack use case based on the preset range in which the comprehensive score is located, and trigger the corresponding downstream processing actions.

10. An electronic device, characterized in that, It includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, which, when executed by the processor, perform the method according to any one of claims 1-8.

Images