Defense strategy blind spot detection method and system based on combined grouping test

Abstract

The invention discloses a defense strategy blind spot detection method and a defense strategy blind spot detection system based on a combination grouping test, which are characterized in that a combination test theory and a diagnosis model are deeply integrated into an attack simulation system (BAS), the number of test cases is compressed to a minimum and controllable scale, and the comprehensive combination test efficiency of the BAS is improved; secondly, in the aspect of vulnerability discovery capability, systematic coverage of complex blind spots caused by interaction is guaranteed, and combined security vulnerabilities which are hidden deeper are discovered more reliably and more efficiently; according to the method, the factor combination causing defense failure can be accurately positioned, the manual analysis and debugging time is greatly shortened, accurate repair guidance is provided for a security team, and the automation level and the operation efficiency of the whole security verification and response process are remarkably improved.

Description

Technical Field

[0001] This invention belongs to the field of network security, specifically relating to a method and system for detecting blind spots in defense strategies based on combined group testing. Background Technology

[0002] Breach and Attack Simulation (BAS) is a proactive and continuous security verification technology that has emerged in recent years. However, existing BAS systems face an inherent trade-off between testing efficiency and coverage depth when probing blind spots in complex defense strategies triggered by a combination of multiple factors.

[0003] Currently, when generating attack variants, BAS often uses brute force combinations, resulting in an explosion in the number of test cases, making comprehensive testing impractical, inefficient, and unable to achieve continuous verification. The use of random or simple mutations leads to incomplete test coverage, easily overlooking hidden vulnerabilities that can only be bypassed by specific combinations of multiple attack parameters. In addition, even if bypassable attacks are discovered, existing systems lack diagnostic capabilities and the remediation guidelines are unclear.

[0004] Therefore, there is an urgent need for an efficient diagnostic method that can be integrated into BAS to solve the problem of insufficient test case generation strategies. Summary of the Invention

[0005] Purpose of the invention: The purpose of this invention is to upgrade the existing BAS by introducing an optimization method based on combinatorial testing theory, so as to achieve rapid, comprehensive and accurate detection of defense blind spots.

[0006] Technical solution:

[0007] On the one hand, this invention proposes a method for detecting blind spots in defense strategies based on combined group testing, including:

[0008] Step 1: Based on the requirements of the target under test, select the set of scripts to be sent and the test combination coverage strength t of the attack simulation system; the test combination coverage strength indicates that the defense strategy of the target under test consists of t AND logics;

[0009] Step 2: Iterate through each script in the set, and for the current script, perform the following processing:

[0010] S1: Get all factors of the current script and the level of each factor;

[0011] S2: Generate a test set array that satisfies t-dimensional coverage;

[0012] S3: Traverse the test set array. For each element, the attack simulation system combines its factor levels to transform it into an attack instance, sends it through the attack simulation link, and obtains the feedback result vector.

[0013] S4: Calculate the suspicion level of each failure factor in the feedback result vector;

[0014] S5: Based on the preset suspicion threshold, filter out high suspicion factor levels and combine them according to t elements to form the candidate rule set for the current script;

[0015] S6: Traverse the candidate rule set, verify whether each candidate rule is consistent with the complete test result, and remove the rules that fail the verification from the candidate rule set;

[0016] S7: Each rule in the candidate rule set is considered a blind spot in the current script's defense strategy settings;

[0017] Step 3: Process each script in the script set to be sent using S1 to S7. The sum of the candidate rule sets for all scripts is the result of this blind spot detection.

[0018] Furthermore, the script to be sent includes a template and factors, generated from prior knowledge; the template defines the basic structure and fixed parts of the script, including the target IP, target port, and basic attack type; the factors are the variable content of the script, each factor corresponds to a variable of one dimension, and the value of each variable is the level of that factor.

[0019] Furthermore, generating the test set array that satisfies t-dimensional coverage includes:

[0020] S2.1: Initialize an empty array A set of tuples to be covered ,in Including all that must be tested Combinatorial factor interaction tuples;

[0021] S2.2: Randomly select an uncovered tuple. As a core objective that must be achieved in this iteration, create a partially defined test case. , so that it just contains The factors and levels, and the levels of other factors in the test cases are yet to be determined;

[0022] S2.3: Targeting For all the undetermined factors, select levels one by one until it becomes a complete test case; in each step of selection, choose the level that makes the test case cover more tuples.

[0023] S2.4: Add the test cases generated in step S2.3 to the coverage array. In the middle, at the same time from Remove the tuple corresponding to the test case from the list and update. ;when When the middle element is empty, If the covered array is completed, proceed to step S3; otherwise, proceed to step S2.2.

[0024] Furthermore, the suspicion level of each failure factor in the calculated feedback result vector includes:

[0025] For the feedback result vector Failure factor level Suspicion The calculation formula is:

[0026]

[0027] In the formula represent The number of all failed test cases in the test. represent The number of LF values ​​in all failed test cases. represent The number of LFs in all successful test cases.

[0028] Furthermore, verifying whether each candidate rule is consistent with the complete test result includes:

[0029] All test cases that satisfy any rule in the candidate rule set fail.

[0030] All failed test results correspond to test cases that satisfy any rule in the candidate rule set.

[0031] On the other hand, the present invention also proposes a defense strategy blind spot detection system based on combined group testing, comprising:

[0032] The requirement determination module is used to select the set of scripts to be sent and the test combination coverage strength t according to the requirements of the target under test; the test combination coverage strength indicates that the defense strategy of the target under test is composed of t AND logics.

[0033] The candidate rule set generation module is used to traverse each script in the set and perform the following processing for the current script:

[0034] S1: Get all factors of the current script and the level of each factor;

[0035] S2: Generate a test set array that satisfies t-dimensional coverage;

[0036] S3: Traverse the test set array. For each element, the attack simulation system combines its factor levels to transform it into an attack instance, sends it through the attack simulation link, and obtains the feedback result vector.

[0037] S4: Calculate the suspicion level of each failure factor in the feedback result vector;

[0038] S5: Based on the preset suspicion threshold, filter out high suspicion factor levels and combine them according to t elements to form the candidate rule set for the current script;

[0039] S6: Traverse the candidate rule set, verify whether each candidate rule is consistent with the complete test result, and remove the rules that fail the verification from the candidate rule set;

[0040] S7: Each rule in the candidate rule set is considered a blind spot in the current script's defense strategy settings;

[0041] The result summation module is used to input the candidate rule set generation module for each script in the set, and the sum of the candidate rule sets of all scripts is the result of this blind spot detection.

[0042] Furthermore, the script to be sent includes a template and factors, generated from prior knowledge; the template defines the basic structure and fixed parts of the script, including the target IP, target port, and basic attack type; the factors are the variable content of the script, each factor corresponds to a variable of one dimension, and the value of each variable is the level of that factor.

[0043] Furthermore, generating the test set array that satisfies t-dimensional coverage includes:

[0044] S2.1: Initialize an empty array A set of tuples to be covered ,in Including all that must be tested Combinatorial factor interaction tuples;

[0045] S2.2: Randomly select an uncovered tuple. As a core objective that must be achieved in this iteration, create a partially defined test case. , so that it just contains The factors and levels, and the levels of other factors in the test cases are yet to be determined;

[0046] S2.3: Targeting For all the undetermined factors, select levels one by one until it becomes a complete test case; in each step of selection, choose the level that makes the test case cover more tuples.

[0047] S2.4: Add the test cases generated in step S2.3 to the coverage array. In the middle, at the same time from Remove the tuple corresponding to the test case from the list and update. ;when When the middle element is empty, If the covered array is completed, proceed to step S3; otherwise, proceed to step S2.2.

[0048] Furthermore, the suspicion level of each failure factor in the calculated feedback result vector includes:

[0049] For the feedback result vector Failure factor level Suspicion The calculation formula is:

[0050]

[0051] In the formula represent The number of all failed test cases in the test. represent The number of LF values ​​in all failed test cases. represent The number of LFs in all successful test cases.

[0052] Furthermore, verifying whether each candidate rule is consistent with the complete test result includes:

[0053] All test cases that satisfy any rule in the candidate rule set fail.

[0054] All failed test results correspond to test cases that satisfy any rule in the candidate rule set.

[0055] Beneficial effects:

[0056] The test set generated by this invention can systematically cover all combinations consisting of t factors (t is a configurable parameter). By using strategies such as greedy algorithms, a test case set (coverage array) with a very small size but which can efficiently cover all t-dimensional factor combinations is dynamically constructed for the BAS script, which improves the efficiency of BAS testing, ensures the coverage depth, and thus efficiently discovers complex interactive vulnerabilities.

[0057] After the test is executed, this invention uses an innovative suspicion correlation analysis model. Based on the PASS / FAIL result vector after the BAS test, this method automatically calculates the correlation score between the level of each attack factor and the defense failure result, providing a quantitative basis for accurately locating the root cause of the vulnerability.

[0058] This invention automatically filters and combines high-suspicion factor levels based on calculated suspicion levels to form candidate defense blind spot rules. These rules precisely describe the conditions for bypassing defenses, and the accuracy of the diagnostic results is ensured through verification of the precision and completeness of full test results. Ultimately, this allows BAS to output remediation recommendations with clear guidance. Attached Figure Description

[0059] Figure 1 This is a flowchart of the method of the present invention;

[0060] Figure 2 This is a typical web attack example according to an embodiment of the present invention;

[0061] Figure 3 This is a structural diagram of the system of the present invention. Detailed Implementation

[0062] The present invention will be further explained below with reference to the accompanying drawings and specific embodiments.

[0063] The core process of message transmission in this invention is as follows: Figure 1 As shown, the specific steps include:

[0064] Step 1: Based on the needs of the target audience, select the set of scripts to be sent and the coverage strength of the test combination. .

[0065] The script to be sent is denoted as Based on the human analysis of the latest security vulnerabilities by cybersecurity experts, corresponding templates and factors are manually generated. The templates define the basic structure and fixed parts of the script, including the target IP, target port, and basic attack type; the factors are the variable content of the script, with each factor corresponding to a variable in one dimension. The content within each factor represents the level of that factor, indicating the range of values ​​for the corresponding variable.

[0066] The test combination coverage strength The complexity of the defense strategy to be detected in this test is defined, and is generally taken as... The goal is to discover all AND logic defense rules consisting of two conditions.

[0067] A typical example of a web attack in this embodiment is as follows: Figure 2 As shown. The core variable factors are represented by placeholders in the example. These placeholders will be replaced with different values ​​according to the test requirements during the actual operation of the system. Figure 2In this context, the three values ​​{{HTTP_METHOD}}, {{OGNL_PAYLOAD}}, and {{REQUEST_BODY}} are factors. Here, {{HTTP_METHOD}} is a placeholder for the HTTP method factor; {{OGNL_PAYLOAD}} is a placeholder for another factor, representing the OGNL attack payload; and {{REQUEST_BODY}} is the request body, the content of which must match the Content-Type header and HTTP_METHOD. Assuming the range of {{HTTP_METHOD}} is {GET, POST, PUT, DELETE}, then the level of this factor is {GET, POST, PUT, DELETE}.

[0068] Step 2: For each script in the set Iterate through them as follows:

[0069] S1: Obtain the script All factors of are denoted as ; Obtain each factor The corresponding level is denoted as , representing factor Total There are several optional values.

[0070] S2: Generate a combined test coverage array Specific generation methods include:

[0071] S2.1: For the current script, initialize an empty array. and a set of tuples to be covered. ,in Includes all that must be tested Combinatorial factor interaction tuples.

[0072] Hypothetical script There are three factors: proto, method, and payload. Their levels include: proto (protocol): {HTTP, HTTPS}, method: {GET, POST}, and payload: {none, SQLi, XSS}. This example sets... This means the goal is to discover all defense rules based on the AND logic consisting of two conditions. The system will generate pairwise combinations of each factor with other level factors, such as: {HTTP, GET}, {HTTP, POST}, {HTTP, SQLi}...{POST, XSS}, etc. These combinations constitute the set of tuples to be covered. In this embodiment There are 16 elements in total.

[0073] S2.2: Randomly select an uncovered tuple. As a core objective that must be achieved in this iteration, create a partially defined test case. , so that it just contains The factors and levels of [the factors] are to be determined, while the levels of other factors in the test cases are yet to be determined.

[0074] Assuming the selected For the current scenario, the test case is {POST, XSS}, but the level of one factor (proto) is still undetermined. Only after proto, method, and payload are all determined will the test case be complete. Represented as .

[0075] S2.3: Targeting For each of the undetermined factors, select a level one by one until a complete test case is formed. The selection criterion is: in each step, choose the level that maximizes the coverage of the final complete test case. The level of other uncovered tuples.

[0076] In this embodiment, the pending factor to be processed is proto, and its level can be either HTTP or HTTPS. When When, the tuples that can be covered are {HTTP,POST}, {HTTP,XSS}, and the target {POST,XSS}; when At that time, the tuples that can be covered are {HTTPS,POST}, {HTTPS,XSS}, and the target {POST,XSS}.

[0077] Assumption If the uncovered element in the data does not contain {HTTPS,POST}, then at this time... It can cover 3 tuples, and Overwrite 2 (because {HTTPS,POST} is not present) (in Chinese). Therefore, in this iteration, the final generated test cases are: .

[0078] S2.4: Add the test cases generated in step S2.3 to the coverage array. In the middle, at the same time from Remove the tuple corresponding to the test case from the list and update. ;when If the element in the middle is empty, proceed to step S3; otherwise, proceed to step S2.2.

[0079] The ultimate goal of S2 is to iteratively construct M until U is empty. Optionally, this step can be replaced by various other advanced combinatorial test case generation algorithms, such as heuristic search algorithms like simulated annealing, genetic algorithms, and tabu search, or deterministic algorithms based on mathematical constructs (such as orthogonal Latin squares). These algorithms have their own advantages and disadvantages in terms of generation speed and final test set size, but the goal is to generate a concise test set that satisfies t-dimensional coverage.

[0080] The final M obtained in this embodiment includes 7 elements. Using the method of this invention, only 7 test cases are needed to cover the specified intensity. It covers all the key testing points, instead of blindly combining the traditional 2*3*3=18 test cases.

[0081] S3: Traverse the final generated overlay array For each of its elements BAS transforms its factor level combination into a practical, executable attack instance, sends it through an attack simulation link, and retrieves feedback results, which are denoted as vectors. .

[0082] Specifically, the BAD system generates a test message to be sent using templates and specific variables of factors, and the BAS system senses the status of the sent test message and receives feedback results.

[0083] In this embodiment, a typical covering array and the corresponding test result vector As shown in Table 1, each row represents a specific The `result` field indicates the pass / fail status of the test case. A `FAIL` result means the test case represented by that row failed, posing a potential security risk. Since all test case messages in Table 1 originate from the same template, it can be inferred that the differences in the `result` field are due to a single factor level or a combination of factor levels.

[0084] Table 1. Coverage array elements (TC_1-TC_7) and corresponding feedback result vectors in this embodiment.

[0085] id proto method payload result TC_1 HTTP GET none PASS TC_2 HTTP POST SQLi FAIL TC_3 HTTP GET XSS PASS TC_4 HTTPS GET SQLi PASS TC_5 HTTPS POST none PASS TC_6 HTTPS POST XSS PASS TC_7 HTTPS GET none PASS

[0086] S4: Calculate the suspicion level for each failure factor level. For Level of each failure factor Based on test result vectors Calculate its correlation with test failure. .

[0087] The calculation method is as follows:

[0088]

[0089] This is the number of all failed test cases in the feedback result. In this embodiment, the result field of TC_2 is FAIL. The value is 1; LF represents the specific value of the factor; The number of LFs among all failed test cases. In this embodiment, the test cases that failed, TC_2, include three factor levels: HTTP, POST, and SQLi. This represents the number of LF values ​​in all successful test cases.

[0090] In this embodiment, the suspicion levels for TC_2 at both the POST and SQLi levels are calculated:

[0091] ,(TC_2); (TC_5, TC_6);

[0092]

[0093] ,(TC_2); ,(TC_4);

[0094]

[0095] Alternatively, the suspicion calculation formula in this step can be replaced with other well-established spectral diagnostic formulas in the field of fault localization, such as Ochiai, Jaccard, Tarantula, or DStar. These formulas each have their own emphasis when calculating correlation, but they are all used to effectively quantify the correlation between the level of a single factor and the test failure result.

[0096] S5: Based on the preset suspicion threshold High-suspicion factor levels were identified and categorized accordingly. Meta-combinations form a candidate rule set. .

[0097] The settings in this embodiment The levels of suspicion exceeding this value are {POST, SQLi}. Because... Therefore, the candidate rule set at this time .

[0098] Alternatively, alternatives here include: using dynamic thresholds or Top-K selection (i.e., selecting the K most suspicious factors) to increase flexibility; and when combining, it can also be extended from simple pairwise combinations to triadic combinations to diagnose more complex vulnerabilities, which corresponds to the value of the coverage strength t.

[0099] S6: Traverse the candidate rule set For each of the candidate rules To verify whether it is consistent with the complete test results, the verification must simultaneously satisfy accuracy (i.e., all tests that satisfy the rules fail) and completeness (i.e., all failed tests satisfy the rules). Remove rules that fail validation.

[0100] for Regarding the candidate rules:

[0101] (1) Accuracy check: The only test case that satisfies this rule is TC_2, and its result is FAIL, so the accuracy requirement is met; (2) Integrity check: In the vector In this case, all failed test cases meet the above rules, and the integrity requirement is met.

[0102] S7: For candidate rule sets Each rule in the script is considered a vulnerability in the current script's defense target rule settings.

[0103] The vulnerability of the defense rule of the defense device in this embodiment is described as: "When method=POST and payload=SQLi, the rule does not take effect."

[0104] Step 3: Process each scenario using steps S1 to S7 to obtain the candidate rule set for all scenarios. The sum of the candidate rule sets involved in all scenarios is the risk result of this detection.

[0105] On the other hand, the present invention also provides a blind spot detection system for a defense strategy based on combined group toilets, which executes all the steps of the aforementioned method. The system composition diagram is shown below. Figure 3 As shown, the Attack Simulation System (BAS) selects and sends corresponding scripts based on the target device's testing requirements. By comparing the scripts at both the sending and receiving ends, the number of test cases that penetrate the defense device can be determined. Penetration of the defense device indicates a deficiency in the device's ability to block specific types of attack packets, requiring targeted remediation.

Claims

1. A method for detecting blind spots in defense strategies based on combined group testing, characterized in that, include: Step 1: Based on the requirements of the target under test, select the set of scripts to be sent and the test combination coverage strength t of the attack simulation system; the test combination coverage strength indicates that the defense strategy of the target under test is composed of t AND logics; Step 2: Iterate through each script in the set, and for the current script, perform the following processing: S1: Get all factors of the current script and the level of each factor; S2: Generate a test set array that satisfies t-dimensional coverage; S3: Traverse the test set array. For each element, the attack simulation system combines its factor levels to transform it into an attack instance, sends it through the attack simulation link, and obtains the feedback result vector. S4: Calculate the suspicion level of each failure factor in the feedback result vector; S5: Based on the preset suspicion threshold, filter out high suspicion factor levels and combine them according to t elements to form the candidate rule set for the current script; S6: Traverse the candidate rule set, verify whether each candidate rule is consistent with the complete test result, and remove the rules that fail the verification from the candidate rule set; S7: Each rule in the candidate rule set is considered a blind spot in the current script's defense strategy settings; Step 3: Process each script in the script set to be sent using S1 to S7. The sum of the candidate rule sets for all scripts is the result of this blind spot detection.

2. The method for detecting blind spots in defense strategies according to claim 1, characterized in that, The script to be sent includes a template and factors, generated from prior knowledge; the template defines the basic structure and fixed parts of the script, including the target IP, target port, and basic attack type; The factors are the variable content of the script. Each factor corresponds to a variable in one dimension, and the value of each variable is the level of that factor.

3. The method for detecting blind spots in defense strategies according to claim 2, characterized in that, The generation of the test set array that satisfies t-dimensional coverage includes: S2.1: Initialize an empty array A set of tuples to be covered ,in Including all that must be tested Combinatorial factor interaction tuples; S2.2: Randomly select an uncovered tuple. As a core objective that must be achieved in this iteration, create a partially defined test case. , so that it just contains The factors and levels, and the levels of other factors in the test cases are yet to be determined; S2.3: Targeting For all the undetermined factors, select levels one by one until it becomes a complete test case; in each step of selection, choose the level that makes the test case cover more tuples. S2.4: Add the test cases generated in step S2.3 to the coverage array. In the middle, at the same time from Remove the tuple corresponding to the test case from the list and update. ;when When the middle element is empty, If the covered array is completed, proceed to step S3; otherwise, proceed to step S2.

2.

4. The method for detecting blind spots in defense strategies according to claim 3, characterized in that, The suspicion level of each failure factor level in the calculated feedback result vector includes: For the feedback result vector Failure factor level Suspicion The calculation formula is: In the formula represent The number of all failed test cases in the test. represent The number of LF values ​​in all failed test cases. represent The number of LFs in all successful test cases.

5. The method for detecting blind spots in defense strategies according to claim 4, characterized in that, The verification of whether each candidate rule is consistent with the complete test result includes: All test cases that satisfy any rule in the candidate rule set fail. All failed test results correspond to test cases that satisfy any rule in the candidate rule set.

6. A defense strategy blind spot detection system based on combined group testing, characterized in that, include: The requirements determination module is used to select the set of scripts to be sent and the test combination coverage intensity t based on the requirements of the target being tested; The test combination coverage strength indicates that the defense strategy of the target under test is composed of t AND logic elements; The candidate rule set generation module is used to traverse each script in the set and perform the following processing for the current script: S1: Get all factors of the current script and the level of each factor; S2: Generate a test set array that satisfies t-dimensional coverage; S3: Traverse the test set array. For each element, the attack simulation system combines its factor levels to transform it into an attack instance, sends it through the attack simulation link, and obtains the feedback result vector. S4: Calculate the suspicion level of each failure factor in the feedback result vector; S5: Based on the preset suspicion threshold, filter out high suspicion factor levels and combine them according to t elements to form the candidate rule set for the current script; S6: Traverse the candidate rule set, verify whether each candidate rule is consistent with the complete test result, and remove the rules that fail the verification from the candidate rule set; S7: Each rule in the candidate rule set is considered a blind spot in the current script's defense strategy settings; The result summation module is used to input the candidate rule set generation module for each script in the set, and the sum of the candidate rule sets of all scripts is the result of this blind spot detection.

7. The defense strategy blind spot detection system according to claim 6, characterized in that, The script to be sent includes a template and factors, generated from prior knowledge; the template defines the basic structure and fixed parts of the script, including the target IP, target port, and basic attack type; The factors are the variable content of the script. Each factor corresponds to a variable in one dimension, and the value of each variable is the level of that factor.

8. The defense strategy blind spot detection system according to claim 7, characterized in that, The generation of the test set array that satisfies t-dimensional coverage includes: S2.1: Initialize an empty array A set of tuples to be covered ,in Including all that must be tested Combinatorial factor interaction tuples; S2.2: Randomly select an uncovered tuple. As a core objective that must be achieved in this iteration, create a partially defined test case. , so that it just contains The factors and levels, and the levels of other factors in the test cases are yet to be determined; S2.3: Targeting For all the undetermined factors, select levels one by one until it becomes a complete test case; in each step of selection, choose the level that makes the test case cover more tuples. S2.4: Add the test cases generated in step S2.3 to the coverage array. In the middle, at the same time from Remove the tuple corresponding to the test case from the list and update. ;when When the middle element is empty, If the covered array is completed, proceed to step S3; otherwise, proceed to step S2.

2.

9. The defense strategy blind spot detection system according to claim 8, characterized in that, The suspicion level of each failure factor level in the calculated feedback result vector includes: For the feedback result vector Failure factor level Suspicion The calculation formula is: In the formula represent The number of all failed test cases in the test. represent The number of LF values ​​in all failed test cases. represent The number of LFs in all successful test cases.

10. The defense strategy blind spot detection system according to claim 9, characterized in that, The verification of whether each candidate rule is consistent with the complete test result includes: All test cases that satisfy any rule in the candidate rule set fail. All failed test results correspond to test cases that satisfy any rule in the candidate rule set.

Images