Method and device for generating test case based on natural language processing

Abstract

The invention provides a test case generation method and device based on natural language processing. The method comprises the following steps: preprocessing input data and generating a structured demand table; matching the rule base to determine compliance requirements; selecting the most similar test template through a semantic similarity algorithm; dynamically filling parameters to generate a test case and supplementing complete content; and outputting the structured test case and marking a risk level. Wherein the risk level is determined by comprehensively evaluating dimensions such as semantic similarity, risk matching degree, compliance matching degree and template use frequency through a weighted fusion algorithm. For high-risk test cases, test paths are expanded by creating and associating action test nodes. The test case is automatically generated through a natural language processing technology, a natural language based on efficiency and accuracy is remarkably improved, provided and generated, the test labor cost of processing is reduced, the test coverage range and the complex scene adaptability are enhanced through a multi-dimensional risk assessment and path extension mechanism, and the test efficiency is improved. Meanwhile, the test case is ensured to meet the requirements of regulations and standards.

Description

Technical Field

[0001] The present invention relates to the technical field of software testing, and in particular to a method, device, and storage medium for generating test cases based on natural language processing. Background Art

[0002] In the field of software testing, test case development is a crucial step in ensuring system quality and verifying functional implementation. Traditional test case generation relies primarily on testers manually writing test cases based on requirements documents. This is not only time-consuming and labor-intensive, but can also lead to incomplete test coverage and uneven test case quality due to misunderstandings and lack of experience. As software system complexity continues to increase, the demand for testing efficiency and accuracy is also increasing. Traditional manual methods are no longer able to meet the demands of rapid iteration and high-quality delivery.

[0003] In recent years, the industry has begun exploring methods for automatically generating test cases based on natural language processing technology. Some studies have attempted to improve the automation of test case writing by identifying keywords or sentence structures in requirement texts, extracting test scenarios, and matching them with predefined templates. However, these methods generally suffer from limited semantic understanding, low template matching accuracy, and a lack of risk assessment mechanisms. This results in poor applicability of the generated test cases, making them incapable of meeting the testing requirements of complex and changing business scenarios and high-risk functional modules.

[0004] Existing methods for prioritizing test cases often rely on manual judgment and lack quantitative evidence, hindering the rational allocation of test resources and the timely discovery of critical issues. Furthermore, when it comes to test path construction, most solutions only support static template filling and lack dynamic expansion capabilities, making it difficult to automatically expand test paths based on high-risk test cases.

[0005] Therefore, there is an urgent need for a test case generation method that can deeply integrate natural language processing, rule reasoning and intelligent evaluation mechanism to solve the problems existing in current technologies, such as low intelligence, unstable generation quality, weak risk identification ability, insufficient compliance assurance and limited test path construction ability, so as to comprehensively improve the efficiency, accuracy and practicality of test case generation. Summary of the Invention

[0006] The present invention protects a method for generating test cases based on natural language processing, comprising the following steps: preprocessing input data, extracting key entity information and generating a structured requirement table; matching a predefined rule library to determine compliance requirements based on the requirement information in the structured requirement table; matching the requirement information with test patterns in a test template library through a semantic similarity algorithm, and selecting the most similar test template; extracting parameters from the requirement information and dynamically filling them into the selected test template to generate a test case; supplementing the test case content with predefined test steps and expected results; outputting the generated test case into a structured format, and marking the risk level and compliance requirements of the test case.

[0007] Furthermore, the generated test cases are output into a structured format and the risk levels of the test cases are marked, including: comprehensively scoring multiple risk assessment dimensions through a weighted fusion algorithm, where the risk assessment dimensions include: a semantic similarity matching score, which is used to measure the degree of semantic matching between the requirement information and the test template; a risk matching degree, which is used to match the risk level of the requirement with the historical applicable risk level of the test template; a compliance matching degree, which is used to determine whether the test template covers the regulatory standards involved in the requirement; and a template usage frequency, which is used to calculate the template call frequency based on historical usage data to adjust the template priority. Based on the scores of the multiple risk assessment dimensions, weighted calculations are performed according to preset weight coefficients α, β, γ, and δ, and a comprehensive risk score is obtained using the following formula: Score = α×SemanticSimilarity+β×RiskMatch+γ×ComplianceMatch+δ×TemplateFrequency; where α is the semantic similarity weight, β is the risk matching weight, γ is the compliance matching weight, and δ is the template usage frequency weight. Based on the matching interval between the comprehensive contribution score and the threshold, the risk level of the test case is determined and marked.

[0008] Furthermore, when the risk level is high, the method also includes: dividing multiple test scenarios based on business needs, and creating an action test node for each test scenario; searching for corresponding action test nodes according to the requirement information corresponding to the test case with a high risk level; determining at least one action test node associated with it within a preset range based on the found action test node; creating and expanding the test path based on the found action test node, and creating and expanding the test path based on the associated action test node.

[0009] Furthermore, the test path is created and expanded based on the found action test node, and the test path is created and expanded based on the associated action test node, including: creating a test path centered on the found action test node based on the found action test node, and expanding the path based on the created test path, and creating a test path centered on the found action test node and / or the associated action test node based on the associated action test node, and expanding the test path based on the created test path.

[0010] Furthermore, a test case is generated based on the created test path; or a structured requirement table is generated based on the created test path.

[0011] Furthermore, the created test paths are mixed, and a test path that can cover multiple test scenarios is generated through a preset model, and the number of generated test paths is set as a model parameter condition, which satisfies the minimum number.

[0012] Furthermore, the input data is preprocessed to extract key entity information and generate a structured requirement table, including: using natural language processing tools to load a pre-trained language model; converting the input data into a document object, and identifying and extracting key entity information in the document through the language model; constructing a structured requirement table based on the extracted key entity information, and the requirement information in the structured requirement table includes requirement ID, action, condition, expected result and risk level.

[0013] Furthermore, the requirements are matched with the test patterns in the test template library through a semantic similarity algorithm, and the most similar test template is selected, including: converting the requirement text into a vector representation, using a pre-trained language model to extract the semantic features of the requirement text, and converting it into a vector representation of fixed dimension; vectorizing each test template in the test template library, and using a pre-trained language model to generate a vector representation for each test template in the test template library; calculating the semantic similarity between the requirement vector and each test template vector; and selecting the test template with the highest semantic similarity as the matching result based on the calculation result.

[0014] Furthermore, after calculating the semantic similarity between the requirement vector and each test template vector, it also includes: dynamically replacing the placeholders in the selected test template according to the values ​​and conditions extracted from the requirements, generating test cases based on the placeholders, and checking whether there are any unreplaced placeholders in the test cases; loading a predefined set of test steps, combining the test steps with the generated test cases, and supplementing the corresponding expected results.

[0015] Furthermore, it also includes: compliance verification of requirements through a rule engine; wherein, the rule engine checks whether the requirements contain specific conditions or actions according to predefined rules, and triggers exception processing if the rules are not met; the content of the requirements is analyzed through intelligent algorithms, and when it is detected that the requirements involve high-risk content or data integrity-related fields, risk tags are automatically added to the requirements.

[0016] The present invention protects a test case generation device based on natural language processing, including: a preprocessing module, used to preprocess input data, extract key entity information and generate a structured requirement table; a rule matching module, used to match a predefined rule library based on the requirement information in the structured requirement table to determine compliance requirements; a template matching module, used to match the requirement information with the test patterns in the test template library through a semantic similarity algorithm, and select the most similar test template; a use case generation module, used to extract parameters from the requirement information and dynamically fill them into the selected test template to generate preliminary test cases, and combine predefined test steps and expected results to complete the test case content; an output module, used to output the generated test cases into a structured format, marking the risk level and compliance requirements.

[0017] The present invention protects a test case generation method based on natural language processing, which performs structured processing on the input requirement text through natural language processing technology, and realizes the automatic generation of test cases in combination with the rule base and the test template base, thereby significantly improving the efficiency and accuracy of test case generation, reducing the workload and error rate of manually writing test cases, and at the same time ensuring that the generated test cases meet the requirements of relevant laws and regulations through the compliance matching mechanism, thereby improving the standardization and traceability of the testing process. Furthermore, the method introduces a multi-dimensional risk assessment model, comprehensively integrates indicators such as semantic similarity, risk matching, compliance matching and template usage frequency, and calculates the risk level of the test case through a weighted fusion algorithm, thereby improving the applicability and priority judgment ability of the test case in different application scenarios, helping testers to quickly identify high-risk test items and perform key verification. Furthermore, when the risk level is high, the method realizes the dynamic creation and expansion of the test path by constructing action test nodes and their associations, thereby enhancing the test coverage and test adaptability in complex scenarios, and effectively responding to the testing challenges brought by high-risk requirements. Furthermore, this method uses a pre-trained language model to vectorize the requirement text and test templates, and achieves precise matching through semantic similarity calculation. This improves the intelligence level of test template selection and matching accuracy, thereby ensuring the quality and applicability of generated test cases. Finally, after the test template is matched, the method also supports dynamic replacement of placeholders in the template based on requirement parameters, and completes the construction of the test case by combining predefined test steps with expected results, further improving the automation and practicality of test case generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 This is a flow chart of a method for generating test cases based on natural language processing according to an embodiment of the present application; Figure 2 A schematic diagram of a process for constructing a structured demand table according to an embodiment of the present application; Figure 3 A schematic diagram of a process for selecting the most similar test template according to an embodiment of the present application; Figure 4 This is a schematic diagram of the framework of a test case generation device based on natural language processing according to an embodiment of the present application; Figure 5 A schematic diagram of a test case generation device based on natural language processing according to an embodiment of the present application. DETAILED DESCRIPTION

[0020] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0021] Research has found that in the field of software testing, traditional test case generation methods mainly rely on manual parsing of requirement documents (URS / FS) and manual design of test scenarios and steps. However, this method is inefficient and prone to incomplete test coverage due to human negligence, especially when it comes to high-risk functions, which may miss key requirements. Even though some existing technologies attempt to generate test cases through automated means, they are usually based only on simple keyword matching or fixed template filling, and are unable to effectively parse the semantic information in complex requirement documents, nor can they conduct risk assessment and tagging in conjunction with compliance requirements. In addition, these methods lack systematic supplementation of test steps and expected results, and the generated test cases are not complete enough to meet the strict test case requirements of high-risk industries (such as pharmaceuticals and finance).

[0022] Based on this, see Figure 1 , an embodiment of the present application provides a method for generating test cases based on natural language processing, comprising the following steps: Preprocess the input data, extract key entity information and generate a structured requirement table; According to the requirement information in the structured requirement table, matching a predefined rule base to determine compliance requirements; Matching the requirement information with the test patterns in the test template library using a semantic similarity algorithm, and selecting the most similar test template; Extracting parameters from the requirement information and dynamically filling them into the selected test template to generate a test case; Complete the test case content by combining predefined test steps and expected results; The generated test cases are output in a structured format, and the risk levels of the test cases and the compliance requirements are marked.

[0023] The embodiment of the present application provides a test case generation method based on natural language processing, which can significantly improve the degree of automation of test case generation and reduce manual intervention. By preprocessing the input data and extracting key entity information, a structured requirement table is generated to ensure the comprehensiveness and accuracy of requirement analysis. The semantic similarity algorithm is used to match the requirement information with the test template library, and the most similar test template is selected, which avoids the limitations of the traditional keyword matching method and improves the intelligence level of matching. At the same time, combined with the predefined rule library and test steps, the test case content is supplemented and the risk level and compliance requirements are marked. The generated test cases not only have comprehensive coverage, but also meet the strict compliance requirements of high-risk industries, thereby reducing potential risks in the testing process and improving software quality assurance capabilities.

[0024] See also Figure 1 In step S101, the input data is preprocessed to extract key entity information and generate a structured requirements table. Specifically, this step aims to parse the input data using natural language processing technology, extract key information, and convert it into a structured form for subsequent processing, thereby converting unstructured requirements text into a structured requirements table.

[0025] To achieve the above functions, refer to Figure 2 As shown, in step S1011, a pre-trained language model is loaded using a natural language processing tool. For example, a mature NLP tool such as spaCy or NLTK can be used to load a suitable pre-trained language model (such as en_core_web_lg). Loading these models provides basic support for subsequent entity recognition and semantic analysis.

[0026] Next, in step S1012, the input data is converted into a document object, and the language model is used to identify and extract key entity information from the document. In the spaCy tool, this can be achieved by calling the nlp() function to convert input data (such as a requirement description) into a document object. The language model loaded in step S1011 is then used to identify and extract key entity information from the document. For example, for the sentence "URS-3.2: System shall lock account after 3 failed login attempts.", the input data is first converted into a document object. The loaded language model is then used to extract key entity information. For example, "3" is identified as a CARDINAL type, and "login attempts" is identified as a NOUN type. These key entity information constitute the core elements of the requirement.

[0027] Finally, in step S1013, a structured requirements table is constructed based on the extracted key entity information. This structured requirements table includes, but is not limited to, fields such as requirement ID, action, condition, expected result, and risk level, which constitute the requirement information. For example, for the above example, the following structured requirements table can be generated: requirement ID is URS-3.2, action is "lock account", condition is "3 failed logins", expected result is "account lock + audit log", and risk level is "high". In this way, unstructured input data can be converted into a clear, easy-to-process structured form, laying the foundation for the subsequent generation of test cases.

[0028] In summary, step S101 achieves the transformation from raw requirements to standardized data by preprocessing the input data, combining natural language processing technology with the construction of a structured requirements table. This process not only improves data processing efficiency but also provides a reliable input basis for subsequent steps.

[0029] In step S102, the predefined rule base is matched against the requirement information in the structured requirement table to determine compliance requirements. This process aims to ensure that the generated test cases comply with relevant regulations and standards through matching and verification of the rule base.

[0030] In this step, the rule engine performs compliance verification on the requirement information in the structured requirement table output in step S101. Specifically, the rule engine loads a predefined rule library, which is usually stored in JSON or other structured formats and contains various compliance requirements and corresponding check conditions. For example, for the requirement of "locking an account", the rule library may contain the following content: `{"rule_id":"SEC-001","action":"account_lock","requirements": ["audit_log","timestamp","user_identity"],"compliance":"21 CFR Part 11 §11.300"}`. In the rule base, "rule_id": "SEC-001" is the rule ID; "action": "account_lock" indicates the action "lock the account"; "requirements": ["audit_log", "timestamp", "user_identity"] are the required conditions, including audit logs, timestamps, and user identity information; and "compliance": "21 CFR Part 11 §11.300" indicates the regulatory standard to be met. The rule engine verifies the requirements based on these rules.

[0031] Furthermore, the rules engine checks whether requirements contain specific conditions or actions based on predefined rules. If the rules are not met, exception handling is triggered. For example, if requirement A does not include audit logging requirements, the rules engine will throw an exception such as `ValidationException("Missing audit log verification")`. This mechanism effectively avoids the risks caused by omitting key compliance requirements.

[0032] Furthermore, before the rule engine matches structured requirements with the rule base, an intelligent algorithm analyzes the requirements. If a requirement is detected as containing high-risk content or fields related to data integrity, it automatically adds a risk tag to the requirement. For example, if a requirement includes "Audit Trail" or is marked as "High Risk," the system automatically adds the "Data Integrity" tag to the requirement and sets its priority to P1. This intelligent processing approach not only improves the efficiency of compliance verification but also enhances the ability to identify potential risks.

[0033] In summary, step S102 achieves a comprehensive determination of compliance requirements by matching the requirements information in the structured requirements table with the predefined rule base, combined with validation by the rule engine and analysis by intelligent algorithms. This process not only ensures the accuracy of test case generation but also provides a solid foundation for subsequent risk management and compliance assurance.

[0034] In step S103, the requirement information is matched with the test patterns in the test template library through a semantic similarity algorithm, and the most similar test template is selected. This process aims to use natural language processing technology to compare the requirement information with the patterns in the test template library, thereby providing a suitable template basis for the generation of subsequent test cases. Specifically, in this step, it is first necessary to convert the requirement information and each test pattern in the test template library into a vector representation. For example, a pre-trained language model (such as BERT) can be used to encode the text to generate corresponding word vectors or sentence vectors. Then, the degree of matching is evaluated by calculating the semantic similarity between the requirement information and each test pattern. Commonly used similarity calculation methods include cosine similarity, Euclidean distance, etc.

[0035] The requirement information and test pattern are vectorized, and the test template most semantically similar to the requirement information is selected from the test template library. For example, suppose the requirement information is "lock account after 3 failed login attempts," and the test template library contains the following two patterns: "auth_failure: Verify system locks account after [n] failed attempts" and "password_policy: Verify password complexity enforcement." By calculating the semantic similarity between the requirement information and these two patterns, "auth_failure" is ultimately selected as the most similar test template.

[0036] In summary, step S103 uses a semantic similarity algorithm to match the requirement information with the test patterns in the test template library and select the most similar test template. This process not only improves the automation level of test case generation, but also ensures that the generated test cases accurately reflect the core content of the requirement.

[0037] In step S104, parameters are extracted from the requirement information and dynamically populated into the selected test template to generate a test case. This step combines the specific parameters in the requirement information with the selected test template to generate a preliminary test case that meets the requirements. Specifically, in this step, key parameters are first extracted from the requirement information. These parameters typically include numerical values, conditions, or other specific information. For example, in the requirement "lock account after 3 failed login attempts," the parameter "3" can be extracted as the number of failed login attempts. Using natural language processing techniques or regular expression matching, these parameters can be accurately identified and extracted from the requirement text. Next, the extracted parameters are dynamically populated into the selected test template. For example, assuming the selected test template is "Verify system locks account after [n] failed attempts," "[n]" is a placeholder for receiving the parameters extracted from the requirement information. By replacing the placeholder "[n]" with the parameter "3," the following preliminary test case can be generated: "Verify system locks account after 3 failed attempts." Furthermore, to ensure that the generated preliminary test cases fully reflect the core content of the requirements, we further verify that the templates after parameter filling conform to the expected logic. For example, we check for unreplaced placeholders or syntax errors, thereby ensuring that the generated test cases have basic usability.

[0038] In summary, step S104 generates test cases by extracting parameters from the requirement information and dynamically filling them into the selected test template. This process not only improves the flexibility and accuracy of test case generation, but also provides basic support for subsequent test case improvement.

[0039] In step S105, the test case content is supplemented by combining predefined test steps and expected results. This step combines the initially generated test case with predefined standard test steps and expected results to form a complete, executable test case. Specifically, in this step, a predefined set of test steps must first be loaded. These test steps are based on industry standards or internal enterprise specifications and provide detailed instructions from test environment preparation to specific operations. For example, for a login function test, predefined test steps might include: "1. Navigate to the login page," "2. Enter a valid username and an incorrect password," "3. Repeat the incorrect password twice (for a total of three failures)," and "4. Attempt to log in using the correct credentials." Next, these test steps are combined with the initially generated test case and supplemented with the corresponding expected results. Expected results specify the expected behavior of the system after each test step is completed. For example, for step "3. Repeat the incorrect password twice (for a total of three failures)," the expected result might be "The system displays the 'Account Locked' message and records the relevant audit log."

[0040] This ensures that test cases not only include operational instructions but also clearly define verification criteria. Furthermore, to enhance the completeness and readability of test cases, the system further optimizes the content format, ensuring that each section is logically clear and interconnected. For example, test steps are arranged in sequence, and detailed descriptions and corresponding expected results are provided for each step, ensuring that testers can accurately understand and execute test tasks.

[0041] In summary, step S105 supplements and improves the content of the preliminary test case by combining predefined test steps and expected results. This process not only improves the quality of the test case, but also provides a comprehensive and reliable basis for subsequent test execution.

[0042] In step S106, the generated test cases are output in a structured format, and the risk levels and compliance requirements are determined and marked. In this step, the structured output and marking mechanism ensures that the generated test cases are not only readable and executable, but also meet the requirements of risk management and regulatory compliance.

[0043] To achieve the above functionality, the generated structured test cases are exported to the manual review interface of the test management tool. Specifically, the test cases are exported in a standardized format (such as JSON, XML, or table) and integrated into common test management tools (such as JIRA and TestRail). For example, the following is an example of JSON format output: { "test_id":"TC-LOGIN-03", "title":"Verify account locks after 3 failed loginattempts", "steps": [ "1. Navigate to login page", "2. Enter valid username and wrong password", "3. Repeat step 2 two more times (total 3 failures)", "4. Attempt login with correct credentials"], "expected": [ "Account locked with message'Account locked for 30 minutes'", "Audit log contains timestamp, user ID, and event type'ACCOUNT_LOCK'"], "compliance": ["21 CFR Part 11 §11.300","EUAnnex 11 §9"], "risk":"high", "automation": true} During the manual review process, intelligent tools assist in the review of test cases. After completing the structured test case output in step S106, the assisted review system further determines whether the currently generated test case is marked as a "high" risk level. If a test case with a high risk level exists, the enhanced test path generation mechanism is triggered, providing risk and boundary testing recommendations and compliance warnings to improve the test coverage depth and logical integrity of high-risk function points.

[0044] After the test case is generated, the system collects evaluation data in four dimensions in parallel: Semantic Similarity: The BERT model is used to calculate the semantic embedding vectors between the requirement text and the test template. The cosine similarity algorithm is used to output a value on a scale of 0-1. For example, the similarity between the requirement "User login failure lockout" and the template "Account security exception handling" is calculated to be 0.87.

[0045] Risk Match: Matching is performed based on the template's historical risk database. A score of 0.9 is assigned when the required risk level is completely consistent with the template's historical level; a score of 0.5 is assigned when there is a one-level difference (e.g., high vs. medium); and a score of 0.1 is assigned when there is no match.

[0046] Compliance Match: Calculates the overlap between the regulatory provisions associated with the requirement and those covered by the template. For example, if a requirement involves five regulations and the template covers four of them, the score is 0.8.

[0047] Template Frequency: Based on historical call logs, the number of times a template is used is normalized to a value between 0 and 1 (current usage count / historical highest usage count).

[0048] After determining the data for each dimension, the weight coefficient of each dimension is determined. The weight coefficient can be configured in advance, determined based on a large model, or calculated based on expert scoring. This embodiment uses the configuration determination coefficient as an example to introduce the specific calculation process and method.

[0049] Before the system is running, the weight coefficients are determined through the front-end configuration interface. For example, the weight coefficients are configured as α=0.4, β=0.3, γ=0.2, and δ=0.1, where α is the semantic similarity weight, β is the risk matching weight, γ is the compliance matching weight, and δ is the template usage frequency weight. Weighted calculation is performed according to the formula: Score=α×SemanticSimilarity+β×RiskMatch+γ×ComplianceMatch+δ×TemplateFrequency; For example, the data determined by semantic similarity matching (SemanticSimilarity), risk matching (RiskMatch), compliance matching (ComplianceMatch), and template usage frequency (TemplateFrequency) are 0.87, 0.9, 0.8, and 0.6, respectively, and the final calculated result is 0.838.

[0050] The system has predefined threshold ranges for different risks. Here are some examples of relevant data ranges: Score ≥ 0.8: High risk (red label, triggering mandatory boundary testing extension); 0.8 > Score ≥ 0.4: Medium risk (yellow label, performing basic boundary checks); Score < 0.4: Low risk (green label, only validating normal flows). In the previous example, a score of 0.838 was marked as high risk. After determining the risk level, the structured data is output and the risk marker is written into the structured test case: json { "test_id":"TC-AUTH-07", "risk":"medium", "risk_detail": { / / Keep the original dimension score " SemanticSimilarity": 0.87, "risk_match": 0.9, "compliance_match": 0.8, "template_freq": 0.6 } } Automatically synchronized to tools such as TestRail / JIRA via the REST API, with risk levels mapped to priority fields (e.g., High → P0).

[0051] The system can analyze the steps in a test case, identify potential missed boundary conditions, and provide improvement suggestions. For example, if a login failure locks an account, the AI ​​might recommend adding a verification condition to ensure the account is unlocked after a second failure, thereby enhancing test coverage. Furthermore, the AI ​​checks whether the test case covers all relevant regulatory requirements. If any omissions are found (e.g., missing electronic signature verification), the system will generate a compliance warning.

[0052] Boundary testing recommendations include adding additional verification conditions for specific test steps, and compliance warnings indicate regulatory requirements not covered in the test case. For example, for the above example, AI may generate the following recommendations and warnings: Boundary test suggestion: [AI suggestion] Add boundary test: verify that the account is not locked after the second failure and requires subsequent verification; Compliance Warning: [Compliance Warning] Expected Results do not address electronic signature verification (reference 21 CFR Part 11 §11.200) These prompts are then integrated into the manual review interface for testers to refer to and decide whether to adopt. In this way, AI-assisted review not only improves the quality of test cases but also reduces the risks caused by missing key test conditions or regulatory requirements.

[0053] In summary, step S106 achieves comprehensive management and optimization of test cases by outputting the generated test cases into a structured format and incorporating risk level and compliance requirement tags. Furthermore, the assisted review function provided by AI technology further enhances the integrity and reliability of the test cases, providing a solid foundation for subsequent test execution.

[0054] In the aforementioned step S106 , how to output the generated test cases into a structured format and mark the risk levels and compliance requirements has been described in detail.

[0055] However, when the risk level is high, in order to further improve the test coverage and quality, the method further includes a series of enhanced processing steps for high-risk requirements. The implementation of these steps will be described in detail below.

[0056] First, based on business needs, divide the test scenarios into multiple categories and create action test nodes for each scenario. For example, you can divide the scenarios into login scenarios, query scenarios, payment scenarios, download scenarios, etc. Create corresponding test nodes for each scenario. For the login scenario, it can have a login node. This rough division can improve efficiency and reduce the amount of operation.

[0057] Action test nodes are used to describe specific test operations and their expected results. For example, if the requirement is to "lock an account," you can divide the test scenarios based on different conditions, such as the number of failed logins (such as 1, 2, and 3) and whether the lock mechanism is triggered. For each scenario, create a corresponding action test node. The following is a sample code snippet: test_scenarios = { "scenario_1":"Verify system behavior after 1 failed login attempt", "scenario_2":"Verify system behavior after 2 failed login attempts", "scenario_3":"Verify account lock after 3 failed loginattempts" } action_nodes = {} for scenario_id, scenario_description in test_scenarios.items(): action_nodes[scenario_id] = f"ActionNode for {scenario_description}" In the above code, the `test_scenarios` dictionary defines multiple test scenarios, and the `action_nodes` dictionary stores the action test nodes associated with each test scenario.

[0058] Next, the system searches for the corresponding action test nodes based on the requirements for the high-risk test cases. For example, if the generated test case involves a high-risk scenario called "lockout after multiple failures," the system automatically matches the action test nodes associated with that scenario, such as the "login" node.

[0059] Subsequently, based on the found action test node, at least one action test node associated with it within a preset range is determined. This process expands the coverage of the test path by analyzing the logical relationships between action test nodes. For example, if the found action test node is the "login" node, its associated nodes may be query, payment, download, and other nodes. The preset range content can be customized, such as within three action ranges.

[0060] Finally, the test path is created and expanded based on the action test nodes found, and the test path is created and expanded based on the associated action test nodes. Specifically, the system will dynamically generate or adjust the test path based on the action test nodes found and their associated nodes. For example, for the "login" node, the system may generate the following test path: enter the username -> enter the wrong password (three times) -> verify the account lock status -> record the audit log -> send a notification email. This dynamic path generation mechanism not only improves the flexibility of the test, but also ensures the integrity of the test in high-risk scenarios. There is no need to determine all the test paths and the test paths corresponding to all scenarios, which reduces the amount of calculation and improves efficiency. It is simpler and more efficient and avoids unnecessary complexity.

[0061] Furthermore, this path can be expanded and adjusted using a neural network model (which has been trained through expanded test paths). For the login node, this can be expanded to: Enter username -> Enter incorrect password (two or four times) -> Verify account lockout status. Alternatively, this can be expanded to: Enter username -> Enter incorrect password (two times) -> Enter correct password -> Verify account lockout status, and so on. This process aims to ensure comprehensive coverage of high-risk requirements by refining the test path. This also lays the foundation for subsequent testing.

[0062] For the "login" node, the system may further link to other related nodes, such as query, payment, download, etc., and construct corresponding test paths, such as entering username -> entering username -> entering incorrect password (2 or 4 times) -> verifying account lock status -> verifying payment status -> sending notification email, etc.

[0063] Furthermore, you can ensure comprehensive coverage of high-risk requirements by refining test scenarios. For example, for the requirement of "locking the account after failed login," you can divide it into the following test scenarios: normal login, locking after multiple failed logins, and unlocking the account. For each test scenario, create a corresponding action test node, such as "enter username," "enter incorrect password," and "verify account status." Alternatively, for login requirements, you can refine the scenarios to include login input, failed login, locked after multiple failed logins, and successful login.

[0064] For the "Enter an incorrect password three times" node in the scenario where the account is locked after multiple failures, the system may further link it to other related nodes, such as "Record an audit log" and "Send a notification email." The system can also generate the following test path: Enter username -> Enter incorrect password (three times) -> Verify account lockout status -> Record an audit log -> Send a notification email. This will not be detailed here.

[0065] In summary, when the risk level is high, a series of steps, including dividing test scenarios, creating action test nodes, searching for related nodes, and dynamically generating test paths, achieve deep coverage and refined testing of high-risk requirements. This enhanced processing complements the structured output and tagging mechanism in step S106, forming a complete natural language processing-based test case generation method, effectively improving test efficiency and quality.

[0066] Furthermore, an optimized implementation method can also mix the created test paths, and generate test paths that can cover multiple test scenarios through a preset model, and set the number of generated test paths as a model parameter condition, which satisfies the minimum number.

[0067] First, existing test paths are mixed. These test paths are generated by creating a test path centered on the found action test node based on the node, expanding the path based on the created test path, and creating a test path centered on the found action test node and / or the associated action test node based on the associated action test node, and expanding the test path based on the created test path. By combining and expanding these paths, new test paths can be generated that can cover more test scenarios. For example, the "user login" path is mixed with the "user upload file" path to generate a new path that includes "user upload file after login."

[0068] The new test paths are fed into a pre-trained neural network model. During training, the model is fed sample data consisting of a mixture of test paths. After generating multiple outputs, the n highest-scoring results are selected. When selecting paths, the goal is to minimize the number of generated paths while still covering multiple test scenarios. This can be achieved by conditioning the model parameters, for example, by using an optimization algorithm to select the optimal path combination.

[0069] like Figure 3 As shown, step S103 further includes multiple steps in matching the requirement information with the test patterns in the test template library through a semantic similarity algorithm and selecting the most similar test pattern.

[0070] In step S1031, the requirement text is converted into a vector representation. Specifically, in order to achieve semantic understanding and matching of the requirement text, the requirement text needs to be vectorized first. In this process, a pre-trained language model (such as spaCy, NLTK, or a Transformer-based model) can be used to extract the semantic features of the requirement text and convert it into a fixed-dimensional vector representation. For example, by loading the pre-trained model en_core_web_lg, its built-in word vector function can be used to process the requirement text. The following is an example code snippet: import spacy nlp = spacy.load("en_core_web_lg") doc = nlp("URS-3.2: System shall lock account after 3 failed login attempts.") req_vector = doc.vector # Get the vector representation of the requirement text In the above code, `doc.vector` returns the average vector representation of the entire document, which captures the key semantic information of the requirement text. This successfully converts the requirement text into a vector, providing a foundation for subsequent semantic similarity calculations. This step helps reduce noise interference, thereby improving the quality of the final matching results.

[0071] In step S1032, each test template in the test template library is vectorized. Based on the vectorization method of the requirement text in step S1031, a pre-trained language model can also be used to generate a vector representation for each test template in the test template library. Specifically, the test template library usually contains multiple predefined test templates, each template describes a specific test scenario or pattern. In order to achieve semantic matching with the requirement text, these test templates need to be converted into vector form one by one. The following is an example code snippet: templates = { "auth_failure":"Verify system locks account after [n]failed attempts", "password_policy":"Verify password complexityenforcement" }template_vectors = {} for template_id, template_text in templates.items(): template_doc = nlp(template_text) template_vectors[template_id] = template_doc.vector In the above code, the `templates` dictionary stores all templates in the test template library, with the key being the template ID and the value being the corresponding template text. By iterating over this dictionary, the same language model (such as `en_core_web_lg`) is invoked on each template text, converting it into a vector representation and ultimately storing it in the `template_vectors` dictionary.

[0072] It's important to note that the vectorization process for test templates should be consistent with the vectorization process for requirement documents to ensure comparability of semantic similarity calculations. Furthermore, if the test template library is large, consider batch processing or optimizing the vectorization method to improve efficiency. This step successfully converts each template in the test template library into a vector form, providing the necessary data support for subsequent semantic similarity calculations.

[0073] In step S1033, the semantic similarity between the requirement vector and each test template vector is calculated. Based on the requirement vector and test template vector generated in step S1031 and step S1032, respectively, a suitable semantic similarity algorithm can be used to quantify the degree of matching between them. Specifically, commonly used semantic similarity algorithms include cosine similarity, Euclidean distance, etc. Among them, cosine similarity is widely used in natural language processing tasks due to its good adaptability to text data. The following is an example code snippet: from sklearn.metrics.pairwise import cosine_similarity req_vector = req_vector.reshape(1, -1) # Adjust the shape to adapt to the cosine_similarity function similarity_scores = {} for template_id, template_vector in template_vectors.items(): template_vector = template_vector.reshape(1, -1) similarity = cosine_similarity(req_vector, template_vector)[0][0] similarity_scores[template_id] = similarity In the above code, the `cosine_similarity` function is used to calculate the cosine similarity between the requirement vector and each test template vector. By traversing all the test template vectors in the `template_vectors` dictionary, the similarity score between the requirement vector and each test template vector can be obtained and stored in the `similarity_scores` dictionary. It should be noted that in order to ensure the accuracy of the calculation results, the dimensions of the requirement vector and the test template vector must be consistent. In addition, if the test template library is large, you can consider optimizing the similarity calculation process, such as using matrix operations or a distributed computing framework to improve efficiency. Through this step, the semantic similarity between the requirement vector and each test template vector is successfully quantified, providing a basis for the subsequent selection of the most similar test template.

[0074] In step S1034, the test template with the highest semantic similarity is selected as the matching result based on the calculation results. Based on the semantic similarity scores between the requirement vector and each test template vector calculated in step S1033, the test template that best meets the requirements can be further screened. Specifically, by traversing the `similarity_scores` dictionary generated in step S1033, the test template ID with the highest similarity score is found, and the corresponding test template is used as the final matching result. The following is an example code snippet: best_match_id = max(similarity_scores, key=similarity_scores.get) best_match_template = templates[best_match_id] In the above code, the `max` function is combined with the `similarity_scores.get` method to find the key with the highest similarity score (i.e., the test template ID) from the dictionary, and the corresponding test template text is extracted from the original `templates` dictionary through the ID. It should be noted that if there are multiple test templates with the same similarity score and all are the highest values, you can select one of the templates according to actual needs, or further introduce other rules (such as template priority) for screening. In addition, in order to improve the reliability of the matching results, you can also set a similarity threshold. The match is considered successful only when the highest similarity score exceeds the threshold, otherwise no matching result is returned. Through this step, the test template with the highest semantic similarity is successfully selected from the test template library as the matching result of the requirement, thereby realizing the effective association between the requirement and the test template, and providing basic support for the subsequent generation of test cases.

[0075] See also Figure 4, Figure 4 This is a schematic diagram of the structure of a test case generation device based on natural language processing provided by an embodiment of the present application. Figure 4 As shown in , the generating device 300 includes: a preprocessing module 310, which is used to preprocess the input data, extract key entity information and generate a structured requirement table; a rule matching module 320, which is used to match the predefined rule library according to the requirement information in the structured requirement table to determine the compliance requirements; a template matching module 330, which is used to match the requirement information with the test pattern in the test template library through a semantic similarity algorithm, and select the most similar test template; a use case generating module 340, which is used to extract parameters from the requirement information and dynamically fill them into the selected test template to generate preliminary test cases, and combine the predefined test steps and expected results to complete the test case content; an output module 350, which is used to output the generated test cases into a structured format, marking the risk level and compliance requirements.

[0076] Furthermore, when the risk level is high, the generating device 300 also includes a scenario division module, which is used to: divide multiple test scenarios based on business needs, and create an action test node for each test scenario; search for corresponding action test nodes according to the requirement information corresponding to the test case with a high risk level; determine at least one action test node associated with it within a preset range based on the action test node found; create and expand the test path based on the action test node found, and create and expand the test path based on the associated action test node.

[0077] Furthermore, when the scenario division module is used to create and expand a test path based on the found action test node, and to create and expand a test path based on the associated action test node, the scenario division module is used to: create a test path centered on the found action test node based on the found action test node, and expand the path based on the created test path, and create a test path centered on the found action test node and / or the associated action test node based on the associated action test node, and expand the test path based on the created test path.

[0078] Furthermore, the generating device 300 further includes a path generating module, and the path generating module is used to generate a test case based on the created test path; or generate a structured requirement table based on the created test path.

[0079] Furthermore, the generating device 300 also includes a path mixing module, which is used to: mix the created test paths, and generate test paths that can cover multiple test scenarios through a preset model, and set the number of generated test paths as a model parameter condition, which satisfies the minimum number.

[0080] Furthermore, when the preprocessing module 310 is used to preprocess the input data, extract key entity information and generate a structured requirement table, the preprocessing module 310 is used to: use a natural language processing tool to load a pre-trained language model; convert the input data into a document object, and identify and extract key entity information in the document through the language model; construct a structured requirement table based on the extracted key entity information, and the requirement information in the structured requirement table includes requirement ID, action, condition, expected result and risk level.

[0081] Furthermore, when the template matching module 330 is used to match requirements with test patterns in the test template library through a semantic similarity algorithm and select the most similar test template, the template matching module 630 is used to: convert the requirement text into a vector representation; perform vectorization processing on each test template in the test template library; calculate the semantic similarity between the requirement vector and each test template vector; and select the test template with the highest semantic similarity as the matching result based on the calculation result.

[0082] Furthermore, when the template matching module 330 is used to calculate the semantic similarity between the demand vector and each test template vector, the template matching module 330 is used to: calculate the similarity value between the demand vector and each test template vector through the cosine similarity algorithm; and use the calculated similarity value as a quantitative indicator of semantic similarity.

[0083] Furthermore, when the template matching module 330 is used to calculate the similarity value between the demand vector and each test template vector through the cosine similarity algorithm, the template matching module 330 is used to: input the demand vector and each test template vector into the cosine similarity algorithm for calculation respectively; during the calculation process, normalize the dimensions of the demand vector and the test template vector.

[0084] Furthermore, when the use case generation module 340 is used to extract parameters from requirements and dynamically fill them into the selected test template to generate preliminary test cases, and to complete the test case content in combination with predefined test steps and expected results, the use case generation module 340 is also used to: dynamically replace placeholders in the selected test template according to the values ​​and conditions extracted from the requirements; and complete the test steps and expected results of the test case based on predefined standard operating steps.

[0085] Furthermore, the generating device 300 also includes a compliance verification module, which is used to: perform compliance verification on the requirements through a rule engine; wherein, the rule engine checks whether the requirements contain specific conditions or actions according to predefined rules, and triggers exception processing if the rules are not met; at the same time, the content of the requirements is analyzed through an intelligent algorithm, and when it is detected that the requirements involve high-risk content or data integrity-related fields, a risk mark is automatically added to the requirements.

[0086] Furthermore, the generating device 300 also includes an auxiliary audit module, which is used to: output the generated structured test cases to the manual audit interface in the test management tool; perform auxiliary audits on the test cases through intelligent tools, and provide boundary test suggestions and compliance warnings; wherein, the boundary test suggestions include operations to add additional verification conditions for specific test steps, and the compliance warnings prompt regulatory requirements not covered in the test cases.

[0087] The natural language processing-based requirements parsing and test case generation device provided in the embodiment of the present application preprocesses input data, extracts key entity information, and generates a structured requirements table. Based on the requirements information in the structured requirements table, it matches a predefined rule library to determine compliance requirements. It uses a semantic similarity algorithm to match the requirements information with test patterns in a test template library and select the most similar test template. It extracts parameters from the requirements information and dynamically fills them into the selected test template to generate preliminary test cases. It combines predefined test steps and expected results to complete the test case content. The generated test case is output in a structured format, marked with risk levels and compliance requirements. In this way, test cases can be automatically generated, testing efficiency can be improved, and labor costs can be reduced.

[0088] See also Figure 5 , Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 5 As shown in , the electronic device 400 includes a processor 410, a memory 420, and a bus 430. The memory 420 stores machine-readable instructions executable by the processor 410. When the electronic device 400 is running, the processor 410 and the memory 420 communicate with each other via the bus 430. When the machine-readable instructions are executed by the processor 410, the steps of the test case generation method based on natural language processing in the above-mentioned method embodiment can be performed. The specific implementation method can be found in the method embodiment and will not be repeated here.

[0089] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, it can execute the steps of the test case generation method based on natural language processing in the above method embodiment. The specific implementation method can be found in the method embodiment and will not be repeated here.

[0090] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0091] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of the present embodiment according to actual needs. Furthermore, the functional units in the various embodiments of the present application may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. If the functions are implemented in the form of software functional units and sold or used as independent products, they may be stored in a processor-executable non-volatile computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the portion that contributes to the prior art, or a portion of the technical solution, may be embodied in the form of a software product, stored in a storage medium and including instructions for enabling a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in the various embodiments of the present application. The aforementioned storage media include: USB flash drives, mobile hard drives, read-only memories (ROM), random access memories (RAM), magnetic disks or optical disks, and other media that can store program codes.

[0092] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The scope of protection of the present application is not limited thereto. Although the present application has been described in detail with reference to the above-mentioned embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-mentioned embodiments within the technical scope disclosed in the present application, or perform equivalent replacements for some of the technical features thereof. However, these modifications, changes, or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A method for generating test cases based on natural language processing, characterized in that: The following steps are involved: Preprocess the input data, extract key entity information and generate a structured requirement table; According to the requirement information in the structured requirement table, matching a predefined rule base to determine compliance requirements; Matching the requirement information with the test patterns in the test template library using a semantic similarity algorithm, and selecting the most similar test template; Extracting parameters from the requirement information and dynamically filling them into the selected test template to generate a test case; Complete the test case content by combining predefined test steps and expected results; The generated test cases are output in a structured format, and the risk levels of the test cases and the compliance requirements are marked.

2. The test case generation method based on natural language processing according to claim 1, characterized in that: The generated test cases are output in a structured format and the risk level of the test cases is marked. The risk level of the test cases is comprehensively scored by a weighted fusion algorithm based on multiple risk assessment dimensions, specifically including: Semantic Similarity: used to measure the degree of semantic matching between requirement information and test templates; RiskMatch, which is used to match the risk level of the requirement with the historical applicable risk level of the test template; Compliance Match, used to determine whether the test template covers the regulatory standards involved in the requirements; TemplateFrequency: This function calculates the frequency of template calls based on historical usage data. Based on the scores of the multiple risk assessment dimensions, weighted calculations are performed according to the preset weight coefficients α, β, γ, and δ, and a comprehensive score is obtained using the following formula: Score=α×SemanticSimilarity+β×RiskMatch+γ×ComplianceMatch+δ×TemplateFrequency; Among them, α is the semantic similarity weight, β is the risk matching weight, γ is the compliance matching weight, and δ is the template usage frequency weight; Based on a matching interval between the comprehensive score and a threshold, a risk level of the test case is determined and marked.

3. The test case generation method based on natural language processing according to claim 2, characterized in that: When the risk level is high, the method further includes: Based on business needs, multiple test scenarios are divided and action test nodes are created for each test scenario; Find the corresponding action test node based on the requirement information corresponding to the test case with a high risk level; Determine at least one action test node associated with the found action test node within a preset range; The test path is created and expanded according to the found action test node, and the test path is created and expanded according to the associated action test node.

4. The test case generation method based on natural language processing according to claim 3, characterized in that: Create and expand test paths based on the found action test nodes, and create and expand test paths based on the associated action test nodes, including: Create a test path centered on the found action test node based on the node, and expand the path based on the created test path; and create a test path centered on the found action test node and / or the associated action test node based on the associated action test node, and expand the test path based on the created test path.

5. The test case generation method based on natural language processing according to claim 3 or 4, characterized in that: Generate test cases based on the created test paths; or generate a structured requirement table based on the created test paths.

6. The test case generation method based on natural language processing according to claim 3 or 4, characterized in that: The created test paths are mixed, and a test path that can cover multiple test scenarios is generated through a preset model, and the number of generated test paths is set as a model parameter condition, and the condition is met with a minimum number of test paths.

7. The method for demand analysis and test case generation based on natural language processing according to claim 1, characterized in that: The preprocessing of input data, extraction of key entity information and generation of a structured requirements table includes: Use natural language processing tools to load pre-trained language models; Converting input data into document objects, and identifying and extracting key entity information in the document using the language model; A structured requirement table is constructed based on the extracted key entity information. The requirement information in the structured requirement table includes requirement ID, action, condition, expected result and risk level.

8. The test case generation method based on natural language processing according to claim 1, characterized in that: The method of matching the requirements with the test patterns in the test template library by using a semantic similarity algorithm and selecting the most similar test template includes: Convert the demand text into a vector representation, use a pre-trained language model to extract the semantic features of the demand text, and convert it into a vector representation of fixed dimension; Perform vectorization on each test template in the test template library, and use a pre-trained language model to generate a vector representation for each test template in the test template library; Calculate the semantic similarity between the requirement vector and each test template vector; According to the calculation results, the test template with the highest semantic similarity is selected as the matching result.

9. The test case generation method based on natural language processing according to claim 7, characterized in that: After calculating the semantic similarity between the demand vector and each test template vector, the method further includes: Dynamically replace placeholders in the selected test template based on the values ​​and conditions extracted from the requirements, generate test cases based on the placeholders, and check whether there are any unreplaced placeholders in the test cases; A predefined set of test steps is loaded, the test steps are combined with the generated test case, and the corresponding expected results are supplemented.

10. The test case generation method based on natural language processing according to claim 1, characterized in that: Also includes: Verify the compliance of requirements through the rule engine; The rule engine checks whether the requirement contains specific conditions or actions according to predefined rules, and triggers exception processing if the rules are not met; The requirement content is analyzed through intelligent algorithms, and when it is detected that the requirement involves high-risk content or data integrity-related fields, a risk tag is automatically added to the requirement.

11. A test case generation device based on natural language processing, characterized in that: include: The preprocessing module is used to preprocess the input data, extract key entity information and generate a structured requirement table; A rule matching module is used to match the predefined rule base to determine compliance requirements based on the requirement information in the structured requirement table; The template matching module is used to match the requirement information with the test patterns in the test template library through a semantic similarity algorithm and select the most similar test template; The use case generation module is used to extract parameters from the requirement information and dynamically fill them into the selected test template to generate preliminary test cases. It then combines predefined test steps and expected results to complete the test case content. The output module is used to output the generated test cases into a structured format, marking the risk level and compliance requirements.

Images