Test case sample cutting method, device and system and storage medium

Abstract

The embodiment of the invention provides a test case sample cutting method, device and system and a storage medium, and relates to the technical field of network security testing. The method comprises the following steps: constructing an industrial control system attack tactical library; wherein the tactical library comprises classified and arranged attack techniques and corresponding security attributes; performing information analysis and semantic analysis on the test case sample to generate structured data of the test case sample and / or function points in the test case sample; and cutting the test case sample according to the security attribute and / or the structured data and / or the function point to obtain a test fragment corresponding to the security attribute and / or the function point. According to the method, classification-based attack techniques and security attributes are cut by constructing a tactical library, key information can be accurately extracted and test intentions can be understood through information analysis and semantic analysis, redundant tests are avoided, cut test fragments can concentrate on specific security attributes or function points, the test coverage range is more accurate, and the test efficiency is improved. And the distributed security test effect is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of network security testing, in particular to a test case sample cutting method and device, system and storage medium. BACKGROUND

[0002] At present, with the development of information technology, the complexity of software and system increases, and security vulnerabilities occur frequently. Testing is a key link to ensure quality and safety. Traditional security testing relies on internal teams, which has limited coverage, high cost, and difficulty in discovering hidden vulnerabilities. Distributed security crowdsourcing platforms have emerged, which allow global security experts to participate in testing and can more extensively mine potential security problems. However, in distributed security crowdsourcing, test cases as core documents, due to the size and complexity of the test object and the diversity of the test environment, there are cases where a single test case is large and contains multiple scenarios and steps, which makes it difficult to write and maintain and reduces test execution efficiency.

[0003] To solve the above problems, related technologies cut complex test cases to improve test efficiency and effectiveness. For example, by manually analyzing the structure and function of the test case, it is divided into multiple smaller parts, or the test case is segmented based on simple rules.

[0004] In the process of implementing the embodiments of the present disclosure, it is found that at least the following problems exist in the related art:

[0005] The related technology improves the efficiency of distributed security testing to some extent. However, in actual application, when cutting test cases, the cutting logic often relies on fixed rules or manual experience, which may not accurately cover the scope of distributed security testing, and cannot cope with actual security risks in different test scenarios, resulting in poor effectiveness of distributed security testing. SUMMARY

[0006] The embodiments of the present application provide a test case sample cutting method and device, system and storage medium.

[0007] The first aspect of the embodiments of the present application provides a test case sample cutting method, comprising:

[0008] Constructing an industrial control system attack tactic library; wherein the tactic library includes classified attack methods and corresponding security attributes;

[0009] Performing information analysis and semantic analysis on the test case sample to generate structured data of the test case sample and / or function points in the test case sample;

[0010] According to the security attribute and / or the structured data and / or the function point cutting test case sample, a test fragment corresponding to the security attribute and / or the function point is obtained.

[0011] In an optional embodiment of the present application, the attack tactic library of the industrial control system is constructed, including:

[0012] According to the collected vulnerability information, a corresponding attack method is determined, and the attack method is classified according to the security attribute;

[0013] A tactic model corresponding to the attack method is constructed, and a simulation environment corresponding to the industrial control system is built, and an automatic script for enabling the tactic model to execute the corresponding attack method in the simulation environment is generated;

[0014] The vulnerability information, the security attribute, the attack method, the tactic model, the model environment and / or the automatic script are integrated into the tactic library.

[0015] In an optional embodiment of the present application, the attack tactic library of the industrial control system is constructed, including:

[0016] The attack method in the tactic library is functionally tested in the simulation environment; and / or, the performance index of the tactic library is tested; the performance index includes execution efficiency and / or resource occupation; and / or, the security of the tactic library is tested;

[0017] According to one or more test results, a test report of the tactic library is generated.

[0018] In an optional embodiment of the present application, the attack tactic library of the industrial control system is constructed, including:

[0019] The vulnerability information of the industrial control software system is obtained regularly, and the vulnerability information in the tactic library is updated; and / or,

[0020] According to the test result of the tactic library, the attack method in the tactic library is adjusted; and / or,

[0021] The content, time and / or reason of each log update of the tactic library are recorded.

[0022] In an optional embodiment of the present application, the test case sample is information parsed, and the structured data of the test case sample is generated, including:

[0023] The text of the test case sample is preprocessed, and the first key information is extracted;

[0024] According to the extracted first key information, the structured data of the test case sample is generated.

[0025] In an optional embodiment of the present application, the first key information includes input parameters, expected behavior and / or abnormal situation description.

[0026] In an optional embodiment of this application, semantic analysis is performed on the test case samples to generate functional points in the test case samples, including:

[0027] Semantic role labeling is performed on the text of the preprocessed test case samples;

[0028] Based on the semantic role annotation results, the logical intent of the test case samples is identified and classified, and secondary key information related to the logical intent is extracted from the text of the test case samples.

[0029] Generate contextual understanding results based on the contextual relationships of the test case samples;

[0030] Extract the functional points of the test case samples based on logical intent, secondary key information, and / or contextual understanding results.

[0031] In an optional embodiment of this application, the second key information includes the test target and / or the test scenario.

[0032] In an optional embodiment of this application, test case samples are segmented according to security attributes and / or structured data and / or function points to obtain test segments corresponding to security attributes and / or function points, including:

[0033] Ensure that the data format of test case samples, tactical libraries, and / or vulnerability triggering primitive models is consistent;

[0034] Use a preset method to identify redundant data in test case samples and mark the redundant data;

[0035] Map test case samples to security attributes in the tactics library to identify security attributes related to test case samples;

[0036] Based on the results of redundant data marking and / or security attributes and / or function points, analyze the structure of the test case samples to determine the cut-off points;

[0037] Determine the cutting strategy based on the location of the cutting point and / or testing requirements;

[0038] Based on the segmentation strategy, the test case samples are segmented into several test fragments.

[0039] In an optional embodiment of this application, after segmenting test case samples according to security attributes and / or structured data and / or function points to obtain test segments corresponding to security attributes and / or function points, the method further includes:

[0040] The industrial control system is tested based on the test segments.

[0041] In an optional embodiment of this application, the industrial control system is tested according to the cut test segments, including:

[0042] Classify and label the test segments;

[0043] Determine the priority of test segments based on their type and tags;

[0044] Test tasks are assigned based on the execution capabilities of each test node in the industrial control system and the priority of the corresponding test segments to test the industrial control system.

[0045] In an optional embodiment of this application, after segmenting test case samples according to security attributes and / or structured data and / or function points to obtain test segments corresponding to security attributes and / or function points, the method further includes:

[0046] Collect test data from industrial control systems using test segments;

[0047] Adjust the cutting strategy based on the test data.

[0048] In an optional embodiment of this application, adjusting the cutting strategy based on test data includes:

[0049] Perform statistical and / or correlation analysis on success and failure case data;

[0050] Based on the analysis results, identify and analyze the problems with the current segmentation strategy for test case samples;

[0051] Based on the problem identification and root cause analysis, adjust the test case sample segmentation strategy.

[0052] A second aspect of this application provides an apparatus for test case sample segmentation, including a processor and a memory storing program instructions. The processor is configured to execute the test case sample segmentation method as described in the first aspect of this application when running the program instructions.

[0053] A third aspect of this application provides a testing system, including:

[0054] Test the system itself; and,

[0055] The test case sample cutting device, as described in the second aspect of the embodiments of this application, is installed on the test system body.

[0056] A fourth aspect of the embodiments of this application provides a computer-readable storage medium storing program instructions, which, when executed, cause a computer to perform a method for segmenting test case samples as described in the first aspect of the embodiments of this application.

[0057] The method, apparatus, system, and storage medium for segmenting test case samples provided in the embodiments of this application have the following beneficial effects:

[0058] This application constructs an attack tactic library for industrial control systems, which includes categorized attack methods and corresponding security attributes. Information parsing and semantic analysis are performed on test case samples to generate structured data and / or functional points within the test case samples. Test case samples are then segmented based on security attributes and / or structured data and / or functional points to obtain test segments corresponding to those attributes and / or functional points. By constructing the tactic library, segmentation based on categorized attack methods and security attributes is ensured. Information parsing and semantic analysis accurately extract key information and understand test intent, avoiding redundant testing. This allows the segmented test segments to focus on specific security attributes or functional points, resulting in more comprehensive and accurate distributed security testing, thereby improving the effectiveness of distributed security testing. Attached Figure Description

[0059] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0060] Figure 1 This is a schematic diagram of a test case sample segmentation method provided in an embodiment of this application;

[0061] Figure 2 This is a schematic diagram of a test case sample segmentation device provided in an embodiment of this application.

[0062] Figure label:

[0063] 800: Test case sample segmentation device; 801: Processor; 802: Memory; 803: Communication interface; 804: Bus. Detailed Implementation

[0064] To make the technical solutions and advantages of the embodiments of this application clearer, the exemplary embodiments of this application will be described in further detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not an exhaustive list of all embodiments. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other.

[0065] Figure 1 This is a schematic diagram of the test case sample segmentation method provided in the embodiments of this application. Any of the following methods can be executed in the test system or in a server or terminal device that is connected to the test system.

[0066] likeFigure 1 As shown in the figure, this application provides a method for segmenting test case samples, including:

[0067] S01, Construct an attack tactic library for industrial control systems; the tactic library includes categorized attack methods and corresponding security attributes.

[0068] S02, perform information parsing and semantic analysis on the test case samples to generate structured data of the test case samples and / or functional points in the test case samples.

[0069] S03, cut the test case samples according to security attributes and / or structured data and / or function points to obtain test segments corresponding to security attributes and / or function points.

[0070] The test case sample segmentation method provided in this application constructs an industrial control system attack tactics library containing categorized attack methods and corresponding security attributes. This provides a systematic security attribute reference system for test case segmentation, enabling the segmentation process to revolve around core security attributes. Information parsing and semantic analysis are performed on the test case samples to generate structured data and functional points, achieving a deep understanding and logical decomposition of the test case content, and accurately extracting key information and functional units. Segmenting test case samples based on security attributes, structured data, and functional points can break down complex test cases into test fragments corresponding to security attributes and functional points. This avoids the blindness of traditional segmentation relying on fixed rules or manual experience, ensuring that the segmented test fragments accurately match the actual security risks of the industrial control system, effectively improving the accuracy of test case coverage of key security attributes, and thus enhancing the effectiveness of distributed security testing.

[0071] In an optional embodiment of this application, constructing an industrial control system attack tactic library includes: determining the corresponding attack methods based on collected vulnerability information and classifying the attack methods according to security attributes; constructing tactical models corresponding to the attack methods and building a simulation environment corresponding to the industrial control system, generating automated scripts that enable the tactical models to execute the corresponding attack methods in the simulation environment; and integrating vulnerability information, security attributes, attack methods, tactical models, model environments, and / or automated scripts into the tactic library.

[0072] In this way, when constructing the attack tactics library for industrial control systems, corresponding attack methods are determined based on collected vulnerability information, and these methods are categorized according to security attributes. This allows attack methods to form a systematic classification system based on security attributes, providing clear security attribute references for subsequent testing. Tactical models corresponding to the attack methods are constructed, and a simulation environment corresponding to the industrial control system is built. Automated scripts are generated to execute the corresponding attack methods in the simulation environment using the tactical models. The tactical models transform attack methods into executable operation sequences, while the simulation environment provides a scenario close to a real industrial control system for verifying attack methods. Automated scripts automate the attack process, improving the efficiency and repeatability of attack simulation. Integrating vulnerability information, security attributes, attack methods, tactical models, model environments, and / or automated scripts into the tactical library forms an attack tactics library containing multi-dimensional information, ensuring a close correlation between attack methods, security attributes, and actual attack scenarios. The tactical library provides comprehensive and structured security attribute and attack method references for segmenting test case samples, enabling the segmented test fragments to accurately correspond to the actual security risks of the industrial control system.

[0073] In an optional embodiment of this application, constructing an industrial control system attack tactic library further includes: performing functional tests on the attack methods in the tactic library in a simulated environment; and / or testing the performance indicators of the tactic library; the performance indicators include execution efficiency and / or resource consumption; and / or performing security tests on the tactic library; and generating a test report of the tactic library based on one or more of the above test results.

[0074] In this way, when constructing an attack tactic library for industrial control systems, functional testing of the attack methods within the library in a simulated environment can verify whether the attack methods can execute as expected and trigger potential security vulnerabilities, ensuring the effectiveness of the attack methods in the tactic library. Testing the execution efficiency, resource consumption, and other performance indicators of the tactic library can assess its resource consumption in practical applications, ensuring its applicability in real-world testing scenarios. Security testing of the tactic library can prevent malicious use or accidental damage, improving the library's own security. Generating a test report based on one or more of the above test results helps to comprehensively record the testing status of the tactic library, providing data support for subsequent optimization, improvement, and maintenance, enabling the tactic library to better serve the segmentation of test case samples and security testing.

[0075] In an optional embodiment of this application, constructing an industrial control system attack tactic library further includes: periodically acquiring vulnerability information of the industrial control system software and updating the vulnerability information in the tactic library; and / or adjusting the attack methods in the tactic library based on the test results of the tactic library; and / or recording the content, time and / or reason for each log update of the tactic library.

[0076] In this way, regularly acquiring vulnerability information and updating the attack tactics library for industrial control systems (ICS) during its construction ensures that the library promptly covers the latest security threats, guaranteeing the timeliness of attack methods and security attributes. This allows subsequent test case segmentation to be based on the latest security risks. Adjusting attack methods based on test results allows for optimization of the accuracy and effectiveness of attack methods within the library, enhancing its ability to simulate real-world attack scenarios. Recording the content, time, and reason for each update to the tactics library creates a complete update log, facilitating the tracking of the library's evolution. This provides clear historical data support for subsequent maintenance, optimization, and troubleshooting, ensuring the library remains adaptable to the security testing needs of ICS during continuous updates.

[0077] In an optional embodiment of this application, information parsing of test case samples to generate structured data of test case samples includes: preprocessing the text of test case samples and extracting first key information; and generating structured data representing test case samples based on the extracted first key information.

[0078] This text preprocessing, including removing irrelevant characters, standardizing formatting, word segmentation, part-of-speech tagging, and named entity recognition, provides standardized input for subsequent key information extraction, improving the accuracy and efficiency of information extraction. Next, it extracts the first key information, such as input parameters, expected behavior, and descriptions of abnormal situations, transforming the unstructured test case text into quantifiable and analyzable key elements. Finally, it generates structured data in JSON or XML formats based on this key information, systematically organizing the test case samples. This facilitates subsequent mapping with security attributes in the attack tactics library, laying the data foundation for accurate test case sample segmentation and effectively improving the standardization of test case processing and the efficiency of subsequent segmentation operations.

[0079] In an optional embodiment of this application, the first key information includes input parameters, expected behavior, and / or a description of abnormal situations.

[0080] Thus, the first key information includes input parameters, expected behavior, and / or descriptions of abnormal situations. Since input parameters are the fundamental conditions for test case execution, extracting their names, types, and value ranges clarifies the test variables and boundary conditions. Expected behavior is the concrete manifestation of the test objective; extracting the results the system should return or the operations it should perform provides a clear basis for test verification. Descriptions of abnormal situations cover boundary conditions such as input parameters exceeding range or insufficient system resources, along with their handling methods; extracting this information improves the test coverage. By extracting one or more of the above key information, unstructured test case text can be transformed into structured data containing core elements, laying the foundation for generating standardized structured representations of test cases. This makes the key information of test cases clearer and more standardized, facilitating mapping with security attributes in the attack tactics library, thereby providing effective data support for accurately segmenting test case samples.

[0081] In an optional embodiment of this application, semantic analysis is performed on the test case samples to generate functional points in the test case samples, including: semantic role labeling of the text of the preprocessed test case samples; identifying and classifying the logical intent of the test case samples based on the semantic role labeling results, and extracting second key information related to the logical intent from the text of the test case samples; generating contextual understanding results based on the contextual relationships of the test case samples; and extracting functional points of the test case samples based on the logical intent, the second key information, and / or the contextual understanding results.

[0082] This process involves semantic role labeling of the preprocessed text, identifying predicates, arguments, and their semantic roles to lay the foundation for understanding the semantic structure of test cases. Next, logical intent is identified and categorized based on the labeling results, while secondary key information such as test objectives and test scenarios is extracted to clarify the core purpose and application scenario of the test cases. Then, contextual understanding results are generated based on the contextual relationships of the test cases to comprehensively grasp the sequence and dependencies of test cases within the overall testing process. Finally, functional points are extracted by combining logical intent, secondary key information, and contextual understanding results, enabling the precise extraction of functional units such as specific function execution paths and exception handling logic from the test case text. By transforming unstructured test case text into structured information containing clear logical intent and functional boundaries, it facilitates subsequent mapping with security attributes in the attack tactics library, providing accurate functional dimension references for intelligent segmentation of test case samples, and effectively improving the matching degree between segmented test fragments and actual functional points.

[0083] In an optional embodiment of this application, the second key information includes the test target and / or the test scenario.

[0084] Thus, the second key information includes the test objective and / or test scenario. Since the test objective clearly defines the specific function or security attribute that the test case needs to verify, extracting it allows for precise identification of the core test intent. The test scenario describes the specific environment and conditions for test execution, and extracting it clarifies the applicable context of the test case. Therefore, by extracting this type of information, we can gain a deeper understanding of the logical purpose behind the test case from a semantic perspective, transforming unstructured text into structured information with clear objectives and scenario attributes. This not only provides semantic support for the subsequent extraction of functional points from test case samples, enabling functional points to more accurately reflect the actual functional units of the test case, but also facilitates precise mapping between the test objective and scenario and the security attributes in the attack tactics library during test case segmentation. This allows for the determination of reasonable segmentation points and strategies, making the segmented test fragments more focused on specific test objectives and scenarios, effectively improving the matching degree between test fragments and actual security testing requirements.

[0085] In an optional embodiment of this application, test case samples are segmented according to security attributes and / or structured data and / or functional points to obtain test segments corresponding to security attributes and / or functional points. This includes: ensuring that the data formats of the test case samples, the tactical library, and / or the vulnerability triggering primitive model are consistent; identifying redundant data in the test case samples using a preset method and marking the redundant data; mapping the test case samples to security attributes in the tactical library to identify security attributes related to the test case samples; analyzing the structure of the test case samples based on the redundant data marking results and / or security attributes and / or functional points to determine the segmentation points; determining the segmentation strategy based on the location of the segmentation points and / or test requirements; and segmenting the test case samples into several test segments according to the segmentation strategy.

[0086] This approach first ensures consistency in the data format of test case samples, tactical libraries, and / or vulnerability triggering primitive models, laying the foundation for unified data processing and analysis and avoiding processing obstacles caused by format differences. Next, redundant data in the test case samples is identified and marked using a pre-defined method, reducing repetitive content and improving testing efficiency. Then, the test case samples are mapped to security attributes in the tactical library to identify relevant security attributes, establishing a connection between test cases and actual security risks. Based on the redundant data marking results and / or security attributes and / or functional point analysis, the structure of the test case samples is determined, identifying cut points that are located between redundant parts or at the boundaries of security attributes / functional points for precise test case splitting. Subsequently, a cutting strategy is determined based on the location of the cut points and / or testing requirements, such as cutting by security attributes, functional points, or a combination thereof, making the cutting strategy more aligned with actual testing needs. Finally, the test case samples are divided into several test segments according to the cutting strategy, each segment focusing on verifying specific security attributes or functional points, thereby effectively improving the coverage of key security attributes by test cases and enhancing the execution efficiency and vulnerability discovery accuracy of distributed security testing.

[0087] In an optional embodiment of this application, after cutting test case samples according to security attributes and / or structured data and / or function points to obtain test segments corresponding to security attributes and / or function points, the method further includes: testing the industrial control system according to the test segments.

[0088] In this way, after segmenting test case samples based on security attributes and / or structured data and / or functional points to obtain test fragments, when testing the industrial control system based on these test fragments, the test fragments are first classified and tagged. Priority is determined according to type and tag. Simultaneously, hardware resources, software environment, and historical performance profiles of test nodes are constructed. Then, through dynamic task allocation strategies such as priority-based greedy allocation, load balancing allocation, and prediction-based intelligent allocation, the test fragments are assigned to suitable nodes for execution. During testing, resource usage is monitored in real time, task status is tracked, allocation strategies are dynamically adjusted, and fault tolerance and recovery mechanisms are employed to ensure test stability. By utilizing segmented test fragments focused on specific security attributes or functional points, combined with task scheduling optimization in a distributed testing environment, resource utilization can be maximized, the overall testing cycle shortened, and the accuracy and completeness of test results are ensured through node capability matching and priority management, thus improving the efficiency and reliability of industrial control system security testing.

[0089] In an optional embodiment of this application, testing the industrial control system according to test segments includes: classifying the test segments and establishing labels; determining the priority of the test segments according to their type and labels; and allocating test tasks according to the execution capabilities of each test node of the industrial control system and the priority of the corresponding test segments to test the industrial control system.

[0090] This method of categorizing and tagging test segments allows for a systematic organization of them based on functional classification (e.g., SQL injection, privilege escalation) and complexity assessment (simple, medium, complex), providing a clear basis for subsequent priority determination. Prioritization is determined based on the type and tags of the test segments, dynamically setting priorities based on business impact, vulnerability severity (referencing CVSS scores), and time sensitivity (time-limited vulnerability discovery). This ensures that tests with significant core business impact, high-risk vulnerability relevance, and time constraints are executed first. Test tasks are allocated based on the execution capabilities of each test node in the industrial control system (including hardware resource scores, software environment compatibility, and historical performance data) and the priority of the corresponding test segments. Through strategies such as priority-based greedy allocation, load balancing allocation, and prediction-based intelligent allocation, high-priority test segments are assigned to nodes with sufficient and compatible resources, avoiding overload of single nodes, maximizing resource utilization, and shortening the overall testing cycle.

[0091] In an optional embodiment of this application, after cutting test case samples according to security attributes and / or structured data and / or functional points to obtain test segments corresponding to security attributes and / or functional points, the method further includes: collecting test data of the industrial control system using the test segments for testing; and adjusting the cutting strategy according to the test data.

[0092] In this way, after obtaining test fragments by segmenting test case samples based on security attributes and / or structured data and / or functional points, test data of industrial control systems using test fragments for testing is collected. This includes the security attributes, functional points, execution paths, and input parameters of test fragments that trigger vulnerabilities in successful cases, as well as the error reasons and logs in failed cases. By cleaning, labeling, statistically analyzing, and performing correlation analysis on the above data, potential relationships between test fragments and security attributes and functional points can be uncovered. Problems such as unreasonable selection of segmentation points and inaccurate redundancy identification in the current segmentation strategy can be located. Then, the segmentation strategy can be adjusted based on the problem location and cause analysis, such as optimizing the segmentation point algorithm and improving the redundancy identification method, so that the segmentation algorithm is more in line with actual application scenarios in continuous iteration.

[0093] In an optional embodiment of this application, adjusting the segmentation strategy based on test data includes: performing statistical analysis and / or correlation analysis on successful case data and failed case data; identifying and analyzing problems with the current segmentation strategy for test case samples based on the analysis results; and adjusting the test case sample segmentation strategy based on the problem identification and cause analysis.

[0094] By performing statistical and / or correlation analysis on successful and failed case data, we can calculate the number and proportion of successful and failed cases, analyze the testing effects of different security attributes and functional points, and uncover the potential relationships between test segments and security attributes and functional points, identifying key factors affecting test performance. Based on the analysis results, we can pinpoint and analyze problems in the current test case sample segmentation strategy, clarifying issues such as unreasonable segmentation point selection, inaccurate identification of redundant parts, and incomplete extraction of security attributes and functional points, along with their causes, such as algorithmic logic defects, inaccurate data input, and insufficient model training. Based on the problem identification and root cause analysis, we can adjust the test case sample segmentation strategy by optimizing the segmentation point selection algorithm, improving the redundant part identification method, and refining the rules for extracting security attributes and functional points, making the segmentation strategy more aligned with actual application scenarios and thus improving the accuracy of test case segmentation.

[0095] In practical applications, when constructing an attack tactic library for industrial control systems (ICS), the first step is vulnerability collection and analysis. This involves gathering known ICS software vulnerability information from publicly available vulnerability databases such as CVE and NVD, vendor security bulletins, and security research reports. A detailed analysis is conducted on the type of each vulnerability (e.g., buffer overflow, SQL injection, privilege escalation), its impact scope (specific software version, operating system), and exploitation conditions (specific privileges, network environment), resulting in a vulnerability list containing a detailed description of the vulnerability, its impact scope, and exploitation conditions. Next, attack method research is conducted, investigating network scanning and vulnerability exploitation techniques targeting ICS software vulnerabilities. Common attack techniques such as malicious code injection and lateral movement are categorized by attack target, attack stage, and attack technology, generating a detailed attack technique classification table listing the name, description, and applicable scenarios of each technique. Then, tactical modeling is performed, designing an executable sequence of operations for each attack technique, including pre-attack preparation (information gathering, tool preparation), key steps in the attack process (vulnerability exploitation, privilege escalation), and post-attack processing (data theft, system damage). Flowcharts and state machines are used to construct the tactical model, clarifying the inputs, outputs, execution conditions, and subsequent steps for each step, forming a complete operational sequence. The process involves several steps: First, a tactical model document is created, detailing the model representation and specifications. Next, a simulation environment is built, similar to a real industrial control system (including control software, network devices, and operating systems). Known vulnerabilities are reproduced in this environment to ensure successful exploitation, generating a simulation environment configuration document describing the setup process, configuration parameters, and vulnerability reproduction. Then, a tactical library is implemented. Based on the tactical model, executable scripts or programs with error handling and logging functions are written and integrated into the tactical library to form a callable attack method module. This outputs a tactical library codebase containing all scripts or program code, along with a tactical library usage document describing usage methods, calling interfaces, and sample code. Following this, testing and verification are conducted. Each attack method is functionally tested in the simulation environment to ensure it executes as expected and triggers vulnerabilities. Performance metrics such as execution efficiency and resource consumption of the tactical library are tested. Security testing is performed on the tactical library itself to prevent malicious exploitation, generating a test report containing functional, performance, and security test results and analysis. Finally, maintenance and updates are performed. New vulnerability information is regularly monitored to update the vulnerability list and attack methods in the tactical library. Attack methods are optimized based on test results and actual feedback, and an update log is generated, recording the content, time, and reason for each update.

[0096] In a specific embodiment of this application, when parsing and semantically understanding test case samples, deep parsing of test cases is first performed. Irrelevant characters, spaces, line breaks, etc., are removed from the test case samples through text cleaning, and the text format is unified. NLP tools (such as NLTK and Jieba) are used for word segmentation and part-of-speech tagging. Named entities such as variable names, function names, and system component names are identified through named entity recognition (NER), resulting in preprocessed text data containing word segmentation results, part-of-speech tagging, and named entity recognition information. Then, input parameters (including parameter names, types, and value ranges), expected behaviors (results that the system should return, operations that should be performed, etc.), and abnormal situations (input parameters exceeding the range, insufficient system resources, etc., and processing methods or expected results) are extracted using regular expressions, keyword matching, and other methods, forming key information extraction results containing detailed descriptions of input parameters, expected behaviors, and abnormal situations. Finally, based on the extracted key information, data models in formats such as JSON and XML are designed and populated with information to generate structured test case data. Furthermore, during the semantic analysis of test cases, Stanford CoreNLP, LTP, and other semantic role annotation tools are used to annotate the preprocessed text, identify predicates, arguments, and their semantic roles in sentences, and extract semantic relationships between elements (such as the causal relationship between input parameters and expected behavior), resulting in semantic role annotation results containing detailed information on predicates, arguments, and their semantic roles. Based on the semantic role annotation results and domain knowledge, deep learning and other intent classification models are designed to classify the logical intent of test cases (such as verifying the correctness of functions, testing system performance, etc.) and extract key information such as test objectives and test scenarios, forming logical intent recognition results that include logical intent classification and key information extraction. Considering the contextual relationships such as the order and dependency between test cases, a context model is constructed, and the semantic information of individual test cases is fused with the context model to obtain contextual understanding results that include the semantic representation and relationships of test cases in the context. The consistency between the semantic analysis results and the test case text is checked, and errors or inconsistencies are corrected or annotated, generating semantic verification and correction results that include corrected semantic analysis results and error annotation information.

[0097] In a specific embodiment of this application, when implementing intelligent segmentation of test case samples, the following steps are performed: First, input preparation is conducted by receiving test case samples after deep parsing and semantic analysis, along with an attack tactic library and vulnerability triggering primitive models. Data formatting ensures consistency in the data formats of the three, resulting in formatted input data. Next, redundant parts are identified by using text similarity algorithms such as cosine similarity and Jaccard similarity, or machine learning models such as Siamese networks, to perform similarity analysis on the test case samples. This identifies duplicate or highly similar parts and marks their position and range within the test cases, forming a redundant part marking result containing position, range, and similarity scores. Then, security attributes and functional points are extracted by mapping the test case samples to security attributes in the attack tactic library, identifying security attributes such as authentication and authorization. Simultaneously, based on the semantic analysis results, specific function execution paths and exception handling logic are extracted, generating a list of security attributes and functional points containing descriptions and locations. Finally, segmentation points are determined, and the results are... The test case structure is analyzed by combining the results of marking redundant parts and the list of security attributes and function points. Cutting points are determined between redundant parts or at the boundaries of security attributes / function points. Based on the cutting point locations and test requirements, a cutting strategy is formulated based on security attributes, function points, or a combination thereof, resulting in a definite result including cutting point locations, cutting strategies, and the number of segments. Next, test case cutting is performed, dividing the test case sample into several independent small test segments focused on verifying specific security attributes or function points according to the cutting strategy. Each segment undergoes integrity and executability verification, forming a set of segments containing descriptions of the corresponding security attributes or function points. Subsequently, the results are optimized and adjusted, evaluating whether the cut test case segments contain missing security attributes or function points or unnecessary redundancy. Based on the evaluation results, the cutting strategy is adjusted, resulting in an optimized set of segments and a strategy adjustment record. Finally, output and feedback are provided: the cut test case segment set is output for subsequent testing, and feedback from testers on the cutting results is collected, generating the final set of segments and a feedback report.

[0098] In a specific embodiment of this application, when optimizing the testing process of the distributed security crowdsourcing platform, task preprocessing and priority allocation are performed first. Functional classification is based on the security attributes (such as SQL injection, privilege escalation) or functional modules (login module, payment module) of the test cases. Execution complexity (simple, medium, complex) is assessed based on historical data or expert experience, establishing a label system such as "high-risk" and "easy to execute." Priorities are dynamically set from three aspects: business impact (high priority is given to core business or high-value assets), vulnerability hazard (high-risk vulnerability test cases are prioritized based on CVSS scores), and time sensitivity (time-limited vulnerability discovery tasks have the highest priority). Next, a capability profile of the test nodes is constructed. Tools such as Zabbix and Nagios are used to monitor the usage of hardware resources such as CPU and memory in real time and calculate resource scores (e.g., CPU score = 100 - CPU utilization). The installed Burp Suite on the nodes is recorded. The system uses testing tools such as Suite and Nmap, matching their versions and specifications. Compatible nodes are selected based on Python version, database type, and other dependency requirements. Average execution time, success rate, and historical task interruption counts for various test cases are statistically analyzed to assess execution efficiency and stability. A dynamic task allocation strategy is then employed. Test cases are prioritized, and the highest-scoring node with sufficient resources and compatibility is selected for each case (priority-based greedy allocation). The number of tasks and resource utilization on each node are calculated and allocated to the node with the lowest load, with tasks periodically migrated (load balancing). Historical task data is collected to train regression models such as XGBoost and LSTM to predict execution time. The optimal node is selected by combining the predicted time with the node's resource score (prediction-based intelligent allocation). Real-time monitoring and dynamic adjustments are implemented, setting an 80% utilization threshold for CPU and memory resources. Exceeding this threshold triggers an alert and pauses new task allocation. The system tracks task status in real time, including running and completed tasks. Failed tasks are retried or migrated. When node resources are released, the system re-evaluates the task queue and prioritizes high-priority tasks. Long-running tasks are migrated based on node load. Fault tolerance and recovery mechanisms are also established. High-priority tasks are backed up and executed on multiple nodes, and the results are compared to verify consistency. Faulty nodes are flagged using heartbeat detection, and tasks are migrated to compatible nodes. Distributed file systems such as Ceph and GlusterFS are used to store test data and results. Transaction logs record the task allocation and execution process to ensure data consistency. Furthermore, collaboration and communication are optimized. Message queues such as RabbitMQ and Kafka are used to decouple task allocation and execution. Test nodes asynchronously transmit results. Distributed locks are used to avoid conflicts on shared resources such as database connections. The number of concurrent tasks on nodes is controlled to prevent resource contention. A workflow for the execution order and dependencies of test tasks is designed, and task status is tracked using a state machine.Finally, the effectiveness was evaluated using metrics such as resource utilization, task completion rate, and testing cycle. The performance of different allocation strategies was compared, parameters were continuously iterated, and feedback from testers was collected to automatically optimize the allocation strategy using reinforcement learning techniques.

[0099] In a specific embodiment of this application, the feedback mechanism and continuous improvement are achieved by introducing a closed-loop feedback mechanism. First, test result data is collected, recording the security attributes, functionalities, execution paths, input parameters, and triggering conditions of test case segments that successfully trigger vulnerabilities. Execution logs are collected, including execution time, CPU and memory usage, and network interaction information. Simultaneously, the reasons for failure of test case segments that fail to trigger vulnerabilities are recorded, such as invalid input parameters, execution path mismatch, or environment incompatibility. Error logs are collected, including error types, locations, and information. The collected data is then stored in distributed databases such as HBase and Cassandra, or in Hive and S... Data warehouses such as Nowflake are used; next, data cleaning and preprocessing are performed to remove invalid data such as duplicates, null values, and outliers, and to standardize data formats such as time and numerical units. Success and failure cases are labeled with information such as their security attributes, functionalities, and test environments. Then, data analysis and insights are conducted. Statistical analysis is used to calculate the number and proportion of success and failure cases, analyze the test effects of different security attributes and functionalities, and evaluate execution efficiency by statistically analyzing the execution time and resource consumption of test case fragments. Correlation analysis is used to uncover the correlation between success and failure cases and the key factors affecting test effects, such as input parameter combinations and execution path selection. The results are visualized using tools such as Tableau and Power BI. Finally, a feedback report containing success and failure case analysis, test effect evaluation, and optimization suggestions is generated and delivered to the development team, testing team, and relevant stakeholders via email, instant messaging tools, and project management platforms. Furthermore, during the test case segmentation and optimization, feedback reports are used to identify issues such as unreasonable segmentation point selection, inaccurate identification of redundant parts, and incomplete extraction of security attributes and functional points. The reasons for these issues, including algorithmic logic defects, inaccurate data input, and insufficient model training, are analyzed. Based on this, the segmentation algorithm logic is adjusted, such as optimizing segmentation point selection, improving redundancy identification, and refining the rules for extracting security attributes and functional points. If machine learning models are involved, parameters are adjusted, training data is increased, feature engineering is improved, and retraining is performed using new data to improve accuracy and generalization ability. Algorithm parameters such as similarity thresholds and resource requirement weights are adjusted, and the optimal combination is determined through experiments. After completing the algorithm code writing and debugging, new test case samples are used for testing and verification. The results before and after optimization are compared to evaluate the degree of improvement. The optimized algorithm is deployed to a distributed security crowdsourcing platform, and the running data and test results are monitored in real time. The actual performance of the algorithm is evaluated regularly, and running problems and new test results are fed back to the closed-loop feedback mechanism to form a continuous improvement cycle. This makes the segmentation algorithm more suitable for actual application scenarios and improves the effectiveness and relevance of test cases.

[0100] Combination Figure 2 As shown in the figure, this application provides a test case sample segmentation apparatus 800, including a processor 801 and a memory 802. Optionally, the apparatus may further include a communication interface 803 and a bus 804. The processor 801, communication interface 803, and memory 802 can communicate with each other via the bus 804. The communication interface 803 can be used for information transmission. The processor 801 can call logical instructions in the memory 802 to execute the test case sample segmentation method of the above embodiment.

[0101] Furthermore, the logic instructions in the aforementioned memory 802 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium.

[0102] The memory 802, as a computer-readable storage medium, can be used to store software programs and computer-executable programs, such as program instructions / modules corresponding to the methods in the embodiments of this application. The processor 801 executes functional applications and data processing by running the program instructions / modules stored in the memory 802, that is, it implements the test case sample segmentation method in the above embodiments.

[0103] The memory 802 may include a program storage area and a data storage area. The program storage area may store the operating system and application programs required for at least one function; the data storage area may store data created based on the use of the terminal device. Furthermore, the memory 802 may include high-speed random access memory and may also include non-volatile memory.

[0104] This application provides a testing system, including a testing system body and the aforementioned test case sample cutting device 800. The test case sample cutting device 800 is installed on the testing system body. The installation relationship described herein is not limited to placement within the testing system, but also includes installation connections with other components of the testing system, including but not limited to physical connections, electrical connections, or signal transmission connections. Those skilled in the art will understand that the test case sample cutting device 800 can be adapted to any feasible testing system body, thereby implementing other feasible embodiments.

[0105] This application provides a computer-readable storage medium storing computer-executable instructions configured to perform the above-described method for cutting test case samples.

[0106] The technical solutions of this application embodiment can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes one or more instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in this application embodiment. The aforementioned storage medium can be a non-transitory storage medium, including: USB flash drive, portable hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, and other media capable of storing program code.

[0107] The technical solutions of this application embodiment can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes one or more instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in this application embodiment. The aforementioned storage medium can be a non-transitory storage medium, including: USB flash drive, portable hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, and other media capable of storing program code.

[0108] The foregoing description and accompanying drawings fully illustrate embodiments of this application to enable those skilled in the art to practice them. Other embodiments may include structural, logical, electrical, procedural, and other changes. The embodiments represent only possible variations. Individual components and functions are optional unless explicitly required, and the order of operation may vary. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. Moreover, the terminology used in this application is for describing embodiments only and is not intended to limit the claims. As used in the description of embodiments and claims, the singular forms “a,” “an,” and “the” are intended to equally include the plural forms unless the context clearly indicates otherwise. Similarly, the term “and / or,” as used herein, means including one or more of the associated listed items and all possible combinations thereof. Additionally, when used in this application, the term "comprise" and its variations "comprises" and / or "comprising" refer to the presence of stated features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof. Without further limitations, an element defined by the phrase "comprises a..." does not exclude the presence of other identical elements in the process, method, or apparatus that includes said element. In this document, each embodiment may focus on the differences from other embodiments, and similar or identical parts between embodiments can be referred to mutually. For methods, products, etc., disclosed in the embodiments, if they correspond to the method section disclosed in the embodiments, the relevant parts can be referred to the description of the method section.

[0109] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0110] The methods and products (including but not limited to devices and equipment) disclosed in the embodiments herein can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units may be merely a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to implement this embodiment according to actual needs. In addition, the functional units in the embodiments of this application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0111] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than that shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. In the descriptions corresponding to the flowcharts and block diagrams in the accompanying drawings, the operations or steps corresponding to different blocks may also occur in a different order than disclosed in the description; sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. Each block in a block diagram and / or flowchart, and combinations of blocks in a block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

Claims

1. A method for segmenting test case samples, characterized in that, include: Construct an attack tactic library for industrial control systems; the tactic library includes categorized attack methods and corresponding security attributes; Perform information parsing and semantic analysis on the test case samples to generate structured data and / or functional points in the test case samples; Test case samples are segmented based on security attributes and / or structured data and / or function points to obtain test segments corresponding to security attributes and / or function points.

2. The method according to claim 1, characterized in that, Build a library of attack tactics for industrial control systems, including: Based on the collected vulnerability information, the corresponding attack methods are determined, and the attack methods are classified according to their security attributes. Construct tactical models corresponding to attack methods, build simulation environments corresponding to industrial control systems, and generate automated scripts that enable the tactical models to execute the corresponding attack methods in the simulation environment; Integrate vulnerability information, security attributes, attack methods, tactical models, model environments, and / or automated scripts into the tactical library.

3. The method according to claim 2, characterized in that, The construction of an industrial control system attack tactics library also includes: Perform functional testing on the attack methods in the tactical library in a simulated environment; and / or, test the performance metrics of the tactical library, including execution efficiency and / or resource consumption; and / or, perform security testing on the tactical library. Based on one or more of the above test results, generate a test report for the tactics library.

4. The method according to claim 2, characterized in that, The construction of an industrial control system attack tactics library also includes: Regularly acquire vulnerability information from industrial control system software and update the vulnerability information in the tactical database; and / or, Based on the test results of the tactics library, adjust the attack methods in the tactics library; and / or, Record the content, time, and / or reason for each log update to the tactics library.

5. The method according to claim 1, characterized in that, The test case samples are parsed to generate structured data for the test case samples, including: The text of the test case samples is preprocessed, and the first key information is extracted; Based on the extracted first key information, structured data representing test case samples is generated.

6. The method according to claim 5, characterized in that, The first key information includes input parameters, expected behavior, and / or a description of abnormal situations.

7. The method according to claim 1, characterized in that, Perform semantic analysis on the test case samples to generate functional points in the test case samples, including: Semantic role labeling is performed on the text of the preprocessed test case samples; Based on the semantic role annotation results, the logical intent of the test case samples is identified and classified, and second key information related to the logical intent is extracted from the text of the test case samples. Generate contextual understanding results based on the contextual relationships of the test case samples; Extract the functional points of the test case samples based on logical intent, secondary key information, and / or contextual understanding results.

8. The method according to claim 7, characterized in that, The second key piece of information includes the test objective and / or test scenario.

9. The method according to any one of claims 1 to 8, characterized in that, Based on security attributes and / or structured data and / or function points, test case samples are segmented to obtain test segments corresponding to the security attributes and / or function points, including: Ensure that the data format of test case samples, tactical libraries, and / or vulnerability triggering primitive models is consistent; Use a preset method to identify redundant data in test case samples and mark the redundant data; Map test case samples to security attributes in the tactics library to identify security attributes related to test case samples; Based on the results of redundant data marking and / or security attributes and / or function points, analyze the structure of the test case samples to determine the cut-off points; Determine the cutting strategy based on the location of the cutting point and / or testing requirements; Based on the segmentation strategy, the test case samples are segmented into several test fragments.

10. The method according to any one of claims 1 to 8, characterized in that, After segmenting test case samples based on security attributes and / or structured data and / or function points to obtain test segments corresponding to security attributes and / or function points, the test case samples also include: The industrial control system is tested based on the test segments.

11. The method according to claim 10, characterized in that, Based on the test segments, the industrial control system is tested, including: Classify and label the test segments; Determine the priority of test segments based on their type and tags; Test tasks are assigned based on the execution capabilities of each test node in the industrial control system and the priority of the corresponding test segments to test the industrial control system.

12. The method according to any one of claims 1 to 8, characterized in that, After segmenting test case samples based on security attributes and / or structured data and / or function points to obtain test segments corresponding to security attributes and / or function points, the test case samples also include: Collect test data from industrial control systems using test segments; Adjust the cutting strategy based on the test data.

13. The method according to claim 12, characterized in that, Adjust the cutting strategy based on test data, including: Perform statistical and / or correlation analysis on success and failure case data; Based on the analysis results, identify and analyze the problems with the current segmentation strategy for test case samples; Based on the problem identification and root cause analysis, adjust the test case sample segmentation strategy.

14. An apparatus for segmenting test case samples, comprising a processor and a memory storing program instructions, characterized in that, The processor is configured to perform the test case sample segmentation method as described in any one of claims 1 to 13 when executing the program instructions.

15. A testing system, characterized in that, include: Test the system itself; The test case sample segmentation device as described in claim 14 is installed on the test system body.

16. A computer-readable storage medium storing program instructions, characterized in that, When the program instructions are executed, they cause the computer to perform the method of test case sample segmentation as described in any one of claims 1 to 13.

Images