Method and system for automatically generating headless test case based on behavior description

Abstract

The invention discloses a method and system for automatically generating headless test cases based on behavior description. The method comprises the following steps: obtaining an original behavior description text, and carrying out standardization processing on the original behavior description text to generate a standardized behavior description text; performing semantic analysis on the standardized behavior description text to generate a structured intermediate representation; matching each IR step in the structured intermediate representation with a preset step library, and generating a corresponding target step for each IR step; according to all the target steps, corresponding code templates are selected from a preset template library and combined in sequence, and headless automatic test cases are generated; and for an interface on which the headless automatic test case depends, selecting a matched Mock data version from the Mock snapshot warehouse according to the interface contract hash, and performing contract consistency verification. According to the method, through semantic analysis, step library matching and template combination, BDD description is fully automatically converted into executable scripts, and testers are liberated from repeated coding work.

Description

Technical Field

[0001] This invention relates to the field of software test automation technology, and more specifically, to a method and system for automatically generating headless test cases based on behavioral descriptions. Background Technology

[0002] As software complexity and iteration speed continue to increase, software quality assurance faces severe challenges. Traditional testing methods mainly rely on manually writing automated test scripts or performing manual regression testing, which have the following inherent drawbacks: 1. Inefficient and costly: Manually writing and maintaining test scripts requires a large amount of professional test development resources, and the maintenance cost of scripts increases exponentially with business changes.

[0003] 2. Poor consistency and incomplete coverage: Test cases described in natural language are ambiguous, and the scripts written by different people have different styles and quality, making it difficult to systematically guarantee coverage of core business scenarios.

[0004] 3. Weak maintainability and reusability: The script logic is tightly coupled with the business logic. When the business logic changes, a large number of scattered scripts need to be manually located and modified, resulting in low reusability.

[0005] To partially address the aforementioned issues, existing technologies include recording and playback tools and BDD-based testing frameworks. While recording and playback tools can quickly generate scripts, the generated scripts are structurally fragile, have weak assertion capabilities, and are extremely difficult to maintain. BDD-based frameworks (such as Cucumber) separate use case descriptions from implementations by mapping natural language steps to code (Step Definitions), improving readability. However, their core mapping relationships still require manual writing and maintenance, failing to fundamentally solve the problems of automatic script generation and scalable reuse.

[0006] Therefore, there is an urgent need for an end-to-end solution that can directly and automatically generate high-quality, executable, and easily maintainable automated test cases from business behavior descriptions, and can intelligently manage test data and execution processes. Summary of the Invention

[0007] This invention provides a method and system for automatically generating headless test cases based on behavioral descriptions, in order to solve the problems of low efficiency, high cost, poor consistency, incomplete coverage, weak maintainability and reusability of manually written automated test scripts in the prior art, as well as the fragility, weak assertion capabilities, and extreme difficulty in maintenance of scripts quickly generated by recording and playback tools.

[0008] To achieve the above objectives, this invention provides a method for automatically generating headless test cases based on behavior descriptions. The method includes: S1, obtaining original behavior description text and standardizing it to generate standardized behavior description text; S2, performing semantic parsing on the standardized behavior description text to generate a structured intermediate representation; S3, matching each IR step in the structured intermediate representation with a pre-set step library to generate a corresponding target step for each IR step; S4, selecting corresponding code templates from a pre-set template library based on all target steps and combining them sequentially to generate executable headless automated test cases; S5, selecting matching Mock data versions from a Mock snapshot repository based on the interface contract hash for the interfaces on which the headless automated test cases depend, and performing contract consistency verification.

[0009] Optionally, S1 includes: obtaining the original behavior description text; performing keyword standardization and controlled word replacement on the original behavior description text to obtain the processed behavior description text; performing structural integrity verification on the processed behavior description text, and if the verification passes, obtaining the standardized behavior description text.

[0010] Optionally, S2 includes: decomposing the standardized behavior description text into multiple step units according to keyword boundaries; performing intent recognition on each step unit, and using an entity extractor to extract entity parameters and constraints from each step unit to generate the structured intermediate representation containing the intent, entity parameters, and constraints.

[0011] Optionally, S3 includes: sequentially performing the following processing on each IR step in the structured intermediate representation: based on the intent in the current IR step, filtering preliminary matching candidate steps in a preset step library; performing a weighted score on each candidate step based on entity matching degree and context matching degree; and automatically selecting the target step from the candidate steps or manually confirming the target step according to the score results.

[0012] Optionally, S4 includes: S41, selecting code templates matching the intent of the corresponding target steps from the preset template library according to the execution order of each target step; S42, replacing the placeholders in the code templates with entity parameters corresponding to the target steps to generate parameterized step code fragments; S43, concatenating the parameterized step code fragments in sequence and encapsulating them in a unified test scaffold structure to generate the executable headless automated test cases.

[0013] Optionally, after S5, the process includes: executing all headless automated test cases, collecting the execution results of each step during the execution of the test cases, and performing measurement and analysis on the execution results according to preset quality assessment rules to generate test quality assessment results.

[0014] On the other hand, the present invention provides a system for automatically generating headless test cases based on behavior descriptions. The system includes: a normalization unit for acquiring original behavior description text, normalizing the original behavior description text, and generating standardized behavior description text; a semantic parsing unit for performing semantic parsing on the standardized behavior description text and generating a structured intermediate representation; a step matching unit for matching each IR step in the structured intermediate representation with a pre-set step library to generate a corresponding target step for each IR step; a test case generation unit for selecting corresponding code templates from a pre-set template library and combining them sequentially according to all target steps to generate executable headless automated test cases; and a verification unit for selecting a matching Mock data version from a Mock snapshot repository based on the interface contract hash for the interfaces on which the headless automated test cases depend, and performing contract consistency verification.

[0015] Optionally, the normalization unit includes: an acquisition subunit for acquiring the original behavior description text; a replacement subunit for performing keyword normalization and controlled word replacement on the original behavior description text to obtain the processed behavior description text; and a verification subunit for performing structural integrity verification on the processed behavior description text. If the verification passes, the normalized behavior description text is obtained.

[0016] Optionally, the semantic parsing unit includes: a decomposition subunit, used to decompose the standardized behavior description text into multiple step units according to keyword boundaries; and a generation subunit, used to perform intent recognition on each step unit and use an entity extractor to extract entity parameters and constraints from each step unit to generate the structured intermediate representation containing the intent, entity parameters and constraints.

[0017] Optionally, the step matching unit includes: performing the following processing on each IR step in the structured intermediate representation in sequence: a filtering subunit, used to filter preliminary matching candidate steps in a preset step library based on the intent in the current IR step; a scoring subunit, used to perform a weighted score of entity matching degree and context matching degree on each candidate step; and a determination subunit, used to automatically select the target step or manually confirm the target step from the candidate steps according to the scoring results.

[0018] The beneficial effects of this invention are: This invention discloses a method and system for automatically generating headless test cases based on behavioral descriptions. The method automatically transforms BDD descriptions into executable scripts through semantic parsing, step library matching, and template combination, freeing testers from repetitive coding work and allowing them to focus on higher-value test case design and analysis. Practical results include: a reduction of over 70% in handwritten script workload, a decrease of approximately 50% in regression test case maintenance costs, and a several-fold reduction in the test design to script deployment cycle. Standardizing the original behavioral description text enforces a unified input language, eliminating ambiguity. Semantic parsing ensures accurate understanding of business intent, and combined with the reusability of the step library, it enables rapid, batch generation of test cases covering a wide range of scenarios (including positive, abnormal, and boundary scenarios). Practical results show that the automated test coverage of critical business paths systematically increases from approximately 30% in the manually-led stage to 70%-85%. The design of a "step library" and "template library" separates variable business logic (parameters) from invariant operational logic (code templates). When UI elements or interfaces change, typically only one step library definition or template library needs to be updated, and all test cases referencing that step will be automatically updated synchronously. This creates a maintainability advantage of "one change, everywhere takes effect," and the steps and templates themselves are highly reusable assets. This invention absorbs the excellent requirement expression form of BDD and breaks through its bottleneck of requiring manual coding in the last step. Through a fully automated link of "semantic parsing - step library matching - template combination," it truly achieves a seamless and automatic conversion from "requirement-side description" to "supply-side executable code." At the same time, the generated script has a clear structure and built-in robust logic, overcoming the inherent defects of recorded and replayed scripts. Attached Figure Description

[0019] Figure 1 This is a flowchart of a method for automatically generating headless test cases based on behavior description provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of a system for automatically generating headless test cases based on behavior description, provided in an embodiment of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0021] This invention provides a complete system for the automatic generation and execution of Behavioral Description (BDD) and Headless Test Cases (HTA). The following section uses a typical e-commerce application scenario—"user login and purchase of goods"—as an example to illustrate the implementation process of this invention in detail.

[0022] Figure 1 This invention provides a method for automatically generating headless test cases based on behavioral descriptions, such as... Figure 1 As shown, the method includes: S1. Obtain the original behavior description text, and perform normalization processing on the original behavior description text to generate standardized behavior description text; The goal of this step is to receive raw behavioral description text that may be written by product managers, testers, or developers and has inconsistent formatting and terminology, and process it into a standard, unambiguous format.

[0023] In an optional implementation, S1 includes: S11. Obtain the original behavior description text; The original behavior description text refers to the text content formed during the test design phase by testers, product managers, or business personnel using natural language to describe the expected behavior of the software system based on their business understanding.

[0024] Its text content typically has the following characteristics: (1) Written in natural language without being bound by a uniform format; (2) May mix Chinese, English or colloquial expressions from the field; (3) There may be multiple non-standard terms or synonyms for the same type of operation; (4) It usually follows the basic semantic structure of behavior-driven testing, but is not strictly standardized in terms of keywords, wording and format.

[0025] Suppose a tester or product manager wrote the following raw behavioral description text (BDD, a mixture of Chinese and non-standard terminology): If there is a registered user account When this user logs into the store When he found the "Smartphones" category on the homepage and clicked on it... He selected the top-ranked item and added it to his shopping cart. Then the number displayed on the shopping cart icon should change to 1. Then he should be able to see the information about the items he just added to his cart. S12. Perform keyword standardization and controlled word replacement on the original behavior description text to obtain the processed behavior description text; Keyword standardization was performed on the original behavioral description text described above; Specifically, the original behavior description text is read line by line, and each line of text is matched using pre-configured regular expression rules. When a keyword used to guide the semantics of the behavior is detected, it is replaced with the corresponding standard keyword according to the preset mapping rules.

[0026] For example, the mapping rules can be used to uniformly replace keywords such as "if / when / then" in the original behavior description text with the standard Given, When, Then.

[0027] The text after keyword standardization is: Given: There is a registered user account When: This user logged into the store. When: He found the "Smartphones" category on the homepage and clicked on it. When: He selected the top-ranked item and added it to his shopping cart. Then: The number displayed on the shopping cart icon should change to 1. Then: He should be able to see the information about the items he just added to his cart. The text after keyword standardization is subjected to controlled word replacement; After completing the keyword standardization process, in order to further eliminate the semantic uncertainty caused by the diversity of natural language in the business description, controlled word replacement processing is performed on the standardized behavioral description text.

[0028] The implementation logic of controlled word replacement is as follows: A controlled vocabulary corresponding to the target business domain is pre-constructed. This controlled vocabulary is used to maintain the mapping relationship between standard terms within the business domain and their non-standard expressions and synonyms. By uniformly replacing non-standard words appearing in the text with predefined standard terms, the semantic consistency of the behavioral description text is ensured.

[0029] Specifically, when processing text after keyword standardization, a controlled vocabulary list matching the current business scenario is loaded, and the text content is segmented and scanned. When a word matching a non-standard term in the controlled vocabulary list is detected in the text, the word is replaced with the corresponding standard term according to the mapping relationship. For words involving page elements or resource objects, they can be further specified as identifiable unique identifiers to facilitate subsequent automated processing.

[0030] For example, loading a "controlled thesaurus for e-commerce" contains standard terms within the business domain and their common synonym mappings. This thesaurus is stored as key-value pairs, for example: {"Login": "Login", "Click": "Click", "Drop": "Add to", "See": "Verification Display", "Shopping Cart Icon": "#shopping-cart-icon"}; The standardized text is segmented and scanned, and "login" is replaced with "login", "click in" is replaced with "click", "drop in" is replaced with "add to", "see" is replaced with "verify display", and "shopping cart icon" is specified as the page element identifier "#shopping-cart-icon".

[0031] The replaced text is: Given: There is a registered user account When: This user logged into the store When: He found the "Smartphones" category on the homepage and clicked on it. When: He selected the top-ranked item and added it to his shopping cart. Then: The number displayed on the #shopping-cart-icon should change to 1. Then: He should be able to verify and display the information of the items he just added to his cart. S13. Perform structural integrity verification on the processed behavior description text. If the verification passes, a standardized behavior description text is obtained.

[0032] The implementation logic for structural integrity verification includes at least two mutually cooperating verification dimensions: The first verification dimension is the existence verification of key structural statements or keywords: This is used to determine whether the processed behavior description text contains the necessary components to characterize the behavior scenario structure. Specifically, the system predefines a set of structural keywords to represent preconditions, behavior processes, and expected results, and verifies whether the text contains at least one structural keyword representing preconditions, at least one structural keyword representing the behavior process, and at least one structural keyword representing the expected result. If any type of structural keyword is missing, the behavior description text is deemed structurally incomplete.

[0033] The second verification dimension is the verification of the order of the structural keywords: This is used to determine whether the order of various structural keywords in the text conforms to preset structural logic rules. These preset structural logic rules are pre-configured by the system and describe the legal order of various structural statements in a behavior-driven scenario. These rules can be reused across multiple behavior description texts and do not depend on specific business code implementations. For example, the preset structural logic rules can stipulate that structural keywords representing preconditions should precede structural keywords representing behavioral processes, and structural keywords representing behavioral processes should precede structural keywords representing expected results, thus forming a structural order that conforms to the semantics of the behavior description.

[0034] Specifically, the structure of the processed behavior description text is verified (e.g., whether the Then assertion is missing). In this application, the processed behavior description text contains at least one Given, multiple When, and at least one Then, and the structure is complete.

[0035] The system verifies the logical order of Given / When / Then. If the verification fails, the system will capture the error location and throw an exception message containing the specific error reason (such as "missing precondition Given"), blocking subsequent processes until corrected. If the verification passes, a standardized behavior description text is obtained.

[0036] S2. Perform semantic parsing on the standardized behavior description text to generate a structured intermediate representation; The task of this step is to understand the business semantics of the normalized text and transform it into a structured intermediate representation (IR) to provide accurate input for subsequent code generation.

[0037] In an optional implementation, S2 includes: S21. The standardized behavior description text is broken down into multiple step units according to keyword boundaries; By utilizing the structural keywords in the original behavior description text that has been standardized, as natural semantic boundary markers, the continuous standardized behavior description text is divided into multiple independent and semantically complete step units.

[0038] The system predefines a set of structural keywords to represent steps of different semantic types and treats these structural keywords as step boundary markers. When processing standardized behavioral description text, the system scans sequentially. When a structural keyword is detected, it is used as the starting boundary of the current step unit, and subsequent continuous description content is grouped into the same step unit until the next structural keyword is detected or the text ends. In this way, a single standardized behavioral description text can be decomposed into multiple step units with clearly defined semantic types and boundaries.

[0039] For example, using “Given:”, “When:”, and “Then:” as delimiters, the standardized behavior description text is split into 6 independent step units, and their order in the original scene is recorded.

[0040] S22. Perform intent recognition for each step unit, and use an entity extractor to extract entity parameters and constraints from each step unit to generate the structured intermediate representation containing the intent, entity parameters and constraints.

[0041] First, intent recognition is performed for each step unit: The system predefines a set of standard intents to represent common behavior types and assertion types in the test scenario. When processing step units, the system analyzes the verbs, behavior objects, and semantic patterns within the step unit using an intent classifier to determine the standard intent identifier corresponding to that step unit. The intent classifier can perform matching based on keyword rules or classification based on a pre-trained machine learning model.

[0042] Secondly, after determining the intent identifier corresponding to the step unit, the entity extractor is started to extract the entity parameters and constraints of the step unit.

[0043] The entity extractor is used to identify key parameters related to the current intent from step units, including but not limited to the operation object, target element identifier, input parameter values, and expected results of assertions. Simultaneously, it parses implicit or explicit constraints in step units and converts them into structured constraint information. The entity extraction process can employ named entity recognition technology, location- or pattern-based rule matching, or a combination of both.

[0044] In some cases, if the necessary parameters required for a certain intention are not explicitly given in the current step unit, the system can mark the parameter as missing to indicate that the parameter needs to be completed by default configuration or external parameters in the subsequent library matching stage.

[0045] Finally, the system encapsulates the intent identifiers, extracted entity parameters, and constraints corresponding to the step units in a unified manner to generate a structured intermediate representation (IR).

[0046] This intermediate representation describes the semantic information of each step in the test scenario in the form of structured data, completely decoupling the natural language description from the specific code implementation, and serving as a general input for subsequent processes.

[0047] (1) Intent recognition: For the first "When" step unit: "This user logged into the online store."

[0048] The intent recognition process uses an intent classifier (which can be based on keyword rules or a trained machine learning model) to analyze the verb "login" and the object "shopping mall," classifying them as USER_LOGIN in the standard intent set, i.e., they are identified as the USER_LOGIN intent.

[0049] For the third "When" step unit: "He selected the top-ranked item and added it to his shopping cart."

[0050] The verb "add to" and the object "shopping cart" were identified as the ADD_PRODUCT_TO_CART intent.

[0051] For the first Then step unit: "The number displayed on the #shopping-cart-icon should become 1".

[0052] The keywords “display”, “become”, and the pattern “should” are identified as ASSERT_ELEMENT_TEXT or VERIFY_TEXT intents.

[0053] (2) Entity extraction Based on the identified intent, the entity extractor is initiated. Utilizing Named Entity Recognition (NER) technology or positional rules, the entity extractor extracts key parameters (such as the target element ID, input values, and expected assertion results) from each step unit. Simultaneously, it parses the constraints in each step unit (such as "must be greater than 0"), converting them into logical constraint objects.

[0054] For the USER_LOGIN intent, the entity extractor identifies the entity: role: "Registered User" (inherited from Given). Since login actions typically require credentials, but these are not provided in the text, a parameter gap `credentials: [REQUIRED]` is marked here, which can be filled by the default parameters of the steps library or external configuration.

[0055] For the ADD_PRODUCT_TO_CART intent, the extractor extracts product_selection_method: "by_rank", rank_index: 0 from "the first ranked product".

[0056] For the ASSERT_ELEMENT_TEXT intent, the extractor directly obtains the element selector: “#shopping-cart-icon” and the expected text: “1”.

[0057] Finally, the identified intent identifiers, extracted entity parameters, and constraints are assembled into a standardized JSON-formatted intermediate representation (IR). This IR is completely decoupled from the natural language and the specific code implementation, serving as a universal input for subsequent processes. The generated structured intermediate representation (IR) is as follows: { "scenario_id": "scenario_login_and_add_to_cart_001", "scenario_name": "User logs in and adds items to cart", "steps": [ { "step_id": "step_1", "type": "Given", "original_text": "There is a registered user account", "intent": "SETUP_USER_CONTEXT", "entities": {"user_type": "registered"} }, { "step_id": "step_2", "type": "When", "original_text": "This user has logged into the online store", "intent": "USER_LOGIN", "entities": {"role": "registered_user", "credentials": "[REQUIRED]"} }, { "step_id": "step_3", "type": "When", "original_text": "He found the 'Smartphones' category on the homepage and clicked on it," "intent": "NAVIGATE_AND_CLICK", "entities": {"page": "home", "target_category": "smartphones", "action": "click"}}, { "step_id": "step_4", "type": "When", "original_text": "He selected the top-ranked item and added it to his shopping cart", "intent": "ADD_PRODUCT_TO_CART", "entities": {"product_selection_method": "by_rank", "rank_index": 0} }, { "step_id": "step_5", "type": "Then", "original_text": "#The number displayed on the shopping-cart-icon should be 1", "intent": "ASSERT_ELEMENT_TEXT", "entities": {"element_selector": "#shopping-cart-icon", "expected_text": "1"} }, { "step_id": "step_6", "type": "Then", "original_text": "He should be able to verify and display the information of the items just added to the cart", "intent": "ASSERT_ELEMENT_VISIBLE", "entities": {"target": "last_added_cart_item"} / / Fuzzy entities, parsed using a step library } ] } This IR completely strips away natural language, using structured data to clearly express "who did what under what conditions, and what result was expected".

[0058] S3. Match each IR step in the structured intermediate representation with a preset step library to generate a corresponding target step for each IR step; This step serves as a bridge connecting "business intent" and "code implementation." It is responsible for finding the most suitable target step for each IR step from a pre-built step library.

[0059] In an optional implementation, S3 includes: Each IR step in the structured intermediate representation is processed sequentially as follows: In this context, the IR step is the smallest processing unit in the structured intermediate representation used to describe the semantics of a single behavior or assertion. Each IR step corresponds to a step unit in the standardized behavior description text, formed after semantic parsing.

[0060] S31. Based on the intent in the current IR step, filter preliminary matching candidate steps from the preset step library; Specifically, each pre-defined step in the step library is associated with at least one standard intent identifier, which declares the type of behavior that the step can support or implement. This association can be configured or labeled when the step is defined, so that the steps in the step library are organized in the form of a "step-intent" mapping relationship, thereby facilitating the retrieval and filtering of steps based on behavioral intent.

[0061] When processing an IR step, the system first reads the intent identifier carried in the IR step and uses the intent identifier as a filtering condition to search the step library for all step definitions that claim to support the intent identifier, thereby forming a set of candidate steps that match the current IR step at the behavioral semantic level.

[0062] First, read the intent in the current IR step. For example, the intent of IR step_4 is ADD_PRODUCT_TO_CART. Retrieve all candidate steps from the pre-defined step library that claim to support this intent. Assume two candidate steps are found: Candidate step A: add_product_to_cart_by_id (parameter: product_id) Candidate step B: add_product_to_cart_by_index (parameters: product_list_selector, product_index) S32. Calculate a weighted score for each candidate step based on entity matching degree and context matching degree; Each candidate step is scored in detail. The scoring formula uses a weighted algorithm: entity matching degree is given a high weight (e.g., 60%), which calculates the overlap rate between the entity parameters extracted in the current IR step and the parameters required by the candidate step; context matching degree is given a secondary weight (e.g., 40%), which evaluates whether the pre-dependent conditions of the candidate step are met; finally, a comprehensive matching score between 0 and 1 is obtained.

[0063] The entity matching degree is used to characterize the degree of matching between the entity parameters extracted in the current IR step and the parameters required in the candidate steps. Its calculation process includes the following steps: (1) Determine the parameter set Get the set of entity parameters extracted in the current IR step; get the set of parameter requirements declared in the step library for candidate steps.

[0064] (2) Establish parameter alignment relationship The system performs semantic alignment between entity parameters in the current IR step and parameters required in candidate steps based on parameter name, parameter semantic label, or parameter type, and determines whether there is a direct mapping or inferable correspondence.

[0065] (3) Calculate the parameter coverage ratio In the parameter set required by the candidate step, count the number of parameters that can be directly provided by the current IR step entity parameters or inferred from default values ​​and context, and calculate the ratio of this number to the total number of parameters required by the candidate step to obtain the parameter coverage ratio.

[0066] (4) Generate entity matching degree value The parameter coverage ratio is normalized to an entity matching degree value between 0 and 1, where a value of 0 is taken when no match is possible and a value of 1 is taken when all parameters are satisfied.

[0067] Context matching degree is used to characterize whether the preconditions for the execution of a candidate step can be satisfied in the execution context of the current step. Its calculation process includes the following steps: (1) Obtaining prerequisite dependencies Read the set of prerequisite dependencies declared by the candidate steps from the step library, which describes the state conditions that the step needs to meet before execution.

[0068] (2) Construct the current execution context state Based on the target step or the execution result of the IR step that was determined before the current IR step, construct the execution context state set of the current test scenario to represent the established state conditions.

[0069] (3) Dependency condition satisfaction judgment For each prerequisite dependency of a candidate step, determine whether it exists in the current execution context state set or can be logically derived from that state set.

[0070] (4) Generate context matching degree value Based on the ratio between the number of satisfied prerequisite dependencies and the total number of prerequisite dependencies, a context matching degree value between 0 and 1 is generated, where the value is 1 when all dependencies are satisfied and 0 when none are satisfied.

[0071] After obtaining the entity matching degree and the context matching degree, the system performs a weighted fusion of the two according to the preset weight coefficients to generate a comprehensive matching score for the candidate steps.

[0072] Specifically, the system assigns corresponding weight coefficients to entity matching degree and context matching degree respectively, and sums the product of entity matching degree and its weight coefficient with the product of context matching degree and its weight coefficient to obtain a comprehensive matching score between 0 and 1.

[0073] Entity matching degree calculation (weight 60%): The entity extracted in the current IR step is: {“product_selection_method”: “by_rank”, “rank_index”: 0}.

[0074] Candidate step A requires product_id, with a matching degree of 0.

[0075] Candidate step B requires `product_index`, which semantically overlaps with `rank_index`, resulting in a high match rate. Assuming `product_list_selector` has a default value or can be inferred from the context, this match rate is calculated to be 0.8.

[0076] Entity matching score: Candidate step A = 0, Candidate step B = 0.8 * 60% = 0.48.

[0077] Context matching degree calculation (weight 40%): Check dependencies: Candidate step B depends on ["user_logged_in", "on_product_list_page"].

[0078] From the current execution context (determined by the preceding steps step_2 and step_3), we know that the user is logged in (user_logged_in is true) and is located on the "Smartphones" category page (which can be considered as on_product_list_page being true).

[0079] The context is a perfect match, and the score is 1.0 * 40% = 0.4.

[0080] The dependencies of candidate step A may not be satisfied or may be unknown.

[0081] The overall score is calculated based on the entity matching score and the context matching score: Candidate step B: 0.48 + 0.4 = 0.88.

[0082] Candidate step A: The score is much lower than this.

[0083] S33. Based on the scoring results, automatically select the target step from the candidate steps or manually confirm the target step.

[0084] The system sorts the candidate steps based on the comprehensive matching scores of each candidate step obtained from S32, and determines whether the target step can be automatically determined or manual confirmation is required based on the pre-set automatic adoption threshold and the score differences between the candidate steps.

[0085] Specifically, the system executes the following judgment process: (1) Automatic adoption judgment The system pre-configures an automatic adoption threshold, which represents the minimum matching score required for a candidate step to be automatically selected as the target step. When the candidate step with the highest overall matching score has a score not lower than the automatic adoption threshold and has a significant difference in overall matching scores from other candidate steps, the system automatically identifies that candidate step as the target step.

[0086] (2) Manual confirmation trigger judgment When the candidate step with the highest overall matching score is not lower than the automatic adoption threshold, but the score difference between it and the candidate step with the second highest score is less than the preset difference threshold, the system determines that the matching degree between the current candidate steps is not obvious enough, automatically generates a list of candidates to be confirmed, and requests manual confirmation of the target step.

[0087] For example, the preset automatic adoption threshold is 0.85.

[0088] Candidate step B scored 0.88 > 0.85, and was significantly higher than other candidates.

[0089] Result: The system automatically binds the current IR step_4 to the add_product_to_cart_by_index step in the preset step library. The binding information includes specific parameter mappings: product_index = rank_index(0), and product_list_selector uses the inferred default value ".product-list-item".

[0090] If the two candidate scores are 0.86 and 0.84 respectively, with a very small difference, a "list to be confirmed" will be generated, requesting manual specification of which step definition to use.

[0091] S4. Based on all the target steps, select the corresponding code templates from the pre-set template library and combine them in sequence to generate executable headless automated test cases. This step generates the final executable headless automated test cases based on all the target steps and the code template corresponding to each target step.

[0092] In an optional implementation, S4 includes: S41. Based on the execution order of each target step in all target steps, select a code template from the preset template library that matches the intent of the corresponding target step. The execution order is derived from the sequence information of the IR steps in the structured intermediate representation. Specifically, in the aforementioned steps, the standardized behavior description text has been decomposed into multiple step units according to the structural keyword boundaries, and the order in which each step unit appears in the original behavior description text is recorded during the decomposition process. After semantic parsing, each step unit generates a corresponding IR step and inherits the sequence information. Subsequently, in the step matching stage, each IR step determines its corresponding target step. Therefore, each target step establishes an association with its corresponding IR step and inherits the sequence identifier of that IR step.

[0093] S42. Replace the placeholders in the code template with entity parameters corresponding to the target step to generate parameterized step code snippets; The method for determining the correspondence between placeholders and entity parameters is as follows: Each placeholder in the code template represents the position of a parameter to be filled, and is associated with a specific parameter semantics during the template definition phase. Specifically: In the template library, each code template declares its supported intent types and required parameter set in advance when it is defined; each parameter in the parameter set corresponds one-to-one with at least one placeholder in the code template, and the correspondence can be marked by parameter name, parameter identifier or parameter metadata; therefore, after the template is selected, the system can determine the parameter type and parameter name corresponding to each placeholder according to the parameter semantics declared in the template.

[0094] In this way, the placeholders in the code template are semantically mapped to the parameters required by the target step without relying on the specific program code implementation.

[0095] Sources and acquisition methods of entity parameters The entity parameters used to replace the placeholders originate from the aforementioned semantic parsing and step matching stage, specifically including one or a combination of the following sources: (1) Entity parameters extracted directly Entity parameters can be directly derived from the entity parameter information extracted and stored from each step unit by an entity extractor when performing entity extraction on each step unit. (2) Mapping parameters generated during the step matching stage During the determination of the target step, the system has established a mapping relationship between the entity parameters of the IR step and the parameters of the target step, and the mapping result serves as the direct source of the entity parameters; (3) Default parameters or context-inferred parameters For parameters required by the target step but not explicitly given in the IR step, the system can infer the corresponding parameter values ​​based on the default parameter values ​​defined in the template or based on the current execution context. (4) External configuration parameters In some cases, entity parameters can also be derived from external configuration files or test environment configurations to complete the parameter information required for execution.

[0096] After determining the correspondence between placeholders and entity parameters and obtaining the corresponding entity parameter values, the system scans the code template, replaces each placeholder in the template with the corresponding entity parameter value, and thus generates parameterized step code snippets.

[0097] S43. The parameterized step code fragments are concatenated in sequence and encapsulated in a unified test scaffold structure to generate the executable headless automated test cases.

[0098] The order of execution in this step is the same as that in S41.

[0099] Specifically, the scaffolding initialization test should be performed first: Create a new test file, for example, test_login_and_purchase.spec.js.

[0100] Write the header description block and preparatory code block required by the test framework.

[0101] The header description block is used to describe the test scenario, test scope, or test group of the current headless automated test case, and serves as the entry structure for the test framework to identify and organize test cases. The header description block typically contains the test case name identifier, test scenario description, and declaration information related to the test framework, used to clarify the test context to which the code snippets of subsequent steps belong.

[0102] Writing method: When generating headless automated test cases, after creating a new test file, the system first writes a header description block into the test file. The content of the header description block can be automatically generated based on the scene identifier or scene name recorded in the structured intermediate representation, and organized according to the syntax structure required by the target testing framework.

[0103] The pre-processing code block is used to initialize common resources required for test execution before executing specific step code snippets. These common resources include, but are not limited to, the creation of the test execution environment, the initialization of page or session objects, and the loading of global configurations. The pre-processing code block ensures that each step code snippet executes in a consistent and controllable initial environment.

[0104] Writing method: When generating headless automated test cases, the system writes a pre-preparation code block after the header description block. This pre-preparation code block declares and initializes the shared resources required by subsequent code snippets. The specific content of the pre-preparation code block can be generated according to the lifecycle management mechanism of the selected test framework. For example, it may create a browser instance or page object before the test begins and save the object as a shared variable that can be referenced by subsequent code snippets.

[0105] Specifically as follows: javascript const { test, expect} = require('@playwright / test'); / / Assuming the Playwright framework is used test.describe('User login and purchase process', () => { let page; test.beforeAll(async ({ browser}) => { page = await browser.newPage(); / / Global initialization code }); Next, the target steps bound to each IR step are traversed sequentially. The corresponding code template strings are extracted from the pre-built template library. A regular expression replacement algorithm is used to scan all placeholders (such as {{username}}, {{url}}) in the code template and replace them with the specific entity parameters parsed from the target step. If any unparsed parameters exist, default values ​​or undefined markers are automatically injected to avoid syntax errors.

[0106] Furthermore, to improve the stability of automated testing, a "retry and exception handling logic (Retry Wrapper)" is automatically wrapped when generating parameterized step code snippets for each target step. This logic sets the maximum number of retries and the exponential backoff time, automatically retrying when the operation fails, and automatically inserting "snapshot capture instructions" before and after the operation for context restoration and debugging in case of failure.

[0107] Furthermore, after all parameterized step code snippets have been generated, a post-cleanup code block is automatically appended to close the browser and disconnect the database. Finally, all code snippets are concatenated and formatted, outputting executable headless automated test cases.

[0108] S5. For the interfaces that the headless automated test cases depend on, select the matching Mock data version from the Mock snapshot repository according to the interface contract hash, and perform contract consistency verification.

[0109] This step ensures that the test is performed in a stable and controlled data environment.

[0110] A mock snapshot is a data object created by structurally storing the request patterns and response data of a specific interface under a particular contract version. It is used to simulate the return behavior of the interface during test execution. Each mock snapshot is associated with the contract structure of the corresponding interface, representing an example state of the interface's returned data under that contract structure.

[0111] The Mock snapshot repository is used to centrally store multiple Mock snapshot versions of multiple interfaces, and manages these Mock snapshots according to interface identifier, contract version, and time information. Through the Mock snapshot repository, the testing system can select the Mock data version that matches the current interface contract based on changes in the interface contract.

[0112] Example of a Mock snapshot repository: Json { "snapshot_id": "snap_cart_add_20231001_v1", "contract_hash": "a1b2c3d4e5...", "api_path": " / api / cart / add", "request_pattern": {"product_id": "number"}, “response_data”: { "success": true, “cartId”: “cart_123”, “itemCount”: 1, “totalPrice”: 2999.00 "serverTime": "2023-10-01T12:00:00Z" / / Dynamic field }, "tags": ["add_to_cart", "happy_path"] } The system establishes a relationship between target steps and interface calls at the step level. When it is determined that a target step will trigger a call to a specific interface during execution, it first obtains the currently effective interface contract definition for that interface and calculates the corresponding structure hash value based on the interface contract definition. Subsequently, the system uses the structure hash value as a search condition to find a Mock snapshot data version in the Mock snapshot repository that is consistent with the interface contract structure.

[0113] When multiple Mock snapshot data versions matching the structure hash value are retrieved, the system further filters the multiple Mock snapshots based on a preset selection strategy. The selection strategy includes at least time priority and scenario tag matching rules, which are used to determine the most suitable Mock data version for the current test scenario from multiple candidate Mock snapshots.

[0114] By using the above methods, it can be ensured that the version of the Mock data used by the test cases during execution is consistent with the current interface contract at the interface structure level, thereby avoiding test instability issues caused by interface evolution.

[0115] For example, in a specific implementation scenario, when generating executable headless automated test cases, it is identified that the step `addToCartByIndex` triggers a call to the ` / api / cart / add` interface. The current version of the OpenAPI definition fragment for this interface is obtained, and its structure hash value is calculated, assuming it is `HASH_CART_ADD_V2`. The snapshot (Mock data version) with a structure hash value equal to `HASH_CART_ADD_V2` is searched in the Mock snapshot repository. `snap_cart_add_20231001_v1` is found. If multiple Mock data versions exist, the version with the latest time and matching tags is selected first.

[0116] After selecting a snapshot, a verification algorithm is run, which compares the snapshot's response_data structure with the response structure declared in the current interface definition. If the verification passes (i.e., they are consistent), the snapshot is associated with a headless automated test case. If the verification fails (i.e., they are inconsistent), a degradation strategy (attempting to find a suboptimal version) is automatically triggered or a contract inconsistency warning is thrown.

[0117] Furthermore, when the headless automated test case captures the actual response data of the interface during execution, the system performs a structural-level comparative analysis of the actual response data with the response data in the currently associated Mock snapshot to identify whether the interface return behavior has undergone substantial changes. After identifying the type of change, the system only incrementally updates the Mock snapshot for the parts that are determined to be substantial business changes, thereby generating a new Mock snapshot version, while retaining the original snapshot to support historical backtracking.

[0118] For example: After an executable headless automated test case was released, the response of the ` / api / cart / add` interface added a new field "discountApplied": false, but this field was not present in the old snapshot. After the test was executed, the system captured the real response data `real_response`. The JSON Diff algorithm compared `real_response` with the old snapshot's `response_data` to generate a difference tree. It discovered the newly added field `discountApplied` and that the `serverTime` field value was different. `serverTime` was defined as a dynamic noise field and filtered out from the substantive changes, determining that `discountApplied` was a substantive business field change. A minimal difference patch was generated and applied to the old snapshot, generating a new snapshot `snap_cart_add_20231105_v2`, with the structure hash value still `HASH_CART_ADD_V2` (because the interface structure definition remained unchanged, only the return value had additional optional fields). The change log was logged: "Added discount application status field". The old snapshot was retained for retrospective purposes.

[0119] In an optional implementation, after S5, the following is included: Execute all headless automated test cases, collect the execution results of each step in the test case execution process, and perform measurement and analysis on the execution results according to the preset quality assessment rules to generate test quality assessment results.

[0120] Specifically, the implementation logic includes at least the following processing steps: (1) Test case dependency resolution and execution scheduling Before executing tests, the system first parses the dependencies between the headless automated test cases within the test suite and constructs a test case execution relationship graph based on these dependencies. This execution relationship graph represents the sequential dependencies and concurrency relationships between test cases. The system schedules test cases according to this execution relationship graph, ensuring that the corresponding test case is executed only after the dependency conditions are met, and concurrently executing test cases without dependency conflicts, provided the dependencies allow.

[0121] (2) Collection of execution results During the execution of test cases, the system collects the execution results of each headless automated test case and its internal steps. The execution results include at least whether the step was executed successfully, the reason for failure, and the execution status information, which are used for subsequent statistical analysis.

[0122] (3) Indicator Calculation and Measurement Analysis Based on the collected execution result data, the system calculates multiple test quality metrics to reflect the overall execution quality of the test set from different dimensions. These test quality metrics may include pass rate, functional coverage, and critical path health, among others.

[0123] (4) Quality rule determination and evaluation result generation The system compares the test quality indicators with the preset quality assessment rules, generates a test quality assessment conclusion based on the judgment result, and decides whether to allow the subsequent release process to continue or trigger the blocking and alarm mechanism.

[0124] Specifically, suppose there is a test suite containing four headless automated test cases: A. Login, B. Browse products (depending on A), C. Query orders (depending on A), and D. Payment (depending on B). Dependencies are resolved, and a Directed Acyclic Graph (DAG) is constructed: A -> B -> D; A -> C. Then, a scheduler allocates tasks: A is executed first. After A succeeds, B and C can be executed concurrently (e.g., assigned to two different executors). After B succeeds, D is executed.

[0125] After the four headless automated test cases were executed, the following data was collected: Total steps: 100 steps, 98 successful steps, 2 failed steps => 98% pass rate. The step library defines 50 step types, and this execution involved 45 of them => 90% functional coverage. All four preset core business paths (e.g., "login-add to cart-place order-pay") passed => 100% critical path health.

[0126] The preset quality assessment rules are: (1) Critical path health must be 100%. (2) Overall pass rate >= 95%. (3) Functional coverage >= 80%.

[0127] The execution results are measured and analyzed according to preset quality assessment rules. Critical path health 100% => Pass.

[0128] Overall pass rate 98% > 95% => Pass.

[0129] Functionality coverage 90% > 80% => Pass.

[0130] Test quality assessment result: All conditions are met, and the release process is allowed to continue.

[0131] If the critical path health is only 80% (meaning one path has failed), the access control will be triggered. Clearly identify which core path failed, block the release, and notify the relevant personnel.

[0132] This invention achieves a complete transformation and lifecycle management from vague natural language requirements to precise, stable, and measurable automated test assets, constructing an efficient, reliable, and governable modern software quality assurance system. By automating test generation to replace a large amount of manual script writing, testers can focus on higher-level behavioral design. In real-world projects, the automated test coverage of critical paths can be increased from approximately 30% in the manual stage to 70%–85%. When business logic changes, only the BDD description, step library, or template library needs to be updated to trigger automatic updates of relevant test cases, reducing maintenance costs by approximately 50%. The reuse mechanism of the step library and template library avoids redundant work.

[0133] Figure 2 This invention provides a system for automatically generating headless test cases based on behavior descriptions. The system includes: The normalization unit 201 is used to obtain the original behavior description text, perform normalization processing on the original behavior description text, and generate standardized behavior description text. Semantic parsing unit 202 is used to perform semantic parsing on the standardized behavior description text to generate a structured intermediate representation; Step matching unit 203 is used to match each IR step in the structured intermediate representation with a preset step library to generate a corresponding target step for each IR step; Test case generation unit 204 is used to select the corresponding code template from the pre-set template library and combine them in sequence according to all the target steps to generate executable headless automated test cases. The verification unit 205 is used to select a matching Mock data version from the Mock snapshot repository based on the interface contract hash for the interface on which the headless automated test case depends, and to perform contract consistency verification.

[0134] In an optional implementation, the normalization unit 201 includes: Get sub-unit 2011, used to retrieve the original behavior description text; Replacement subunit 2012 is used to perform keyword standardization and controlled word replacement on the original behavior description text to obtain the processed behavior description text; The verification subunit 2013 is used to perform structural integrity verification on the processed behavior description text. If the verification passes, a standardized behavior description text is obtained.

[0135] In an optional implementation, the semantic parsing unit 202 includes: The decomposition subunit 2021 is used to decompose the standardized behavior description text into multiple step units according to keyword boundaries; The generation subunit 2022 is used to perform intent recognition for each step unit and use an entity extractor to extract entity parameters and constraints from each step unit to generate the structured intermediate representation containing the intent, entity parameters and constraints.

[0136] In an optional implementation, the step matching unit 203 includes: Each IR step in the structured intermediate representation is processed sequentially as follows: The filtering subunit 2031 is used to filter preliminary matching candidate steps in a preset step library based on the intent in the current IR step. The scoring subunit 2032 is used to give a weighted score of entity matching degree and context matching degree for each candidate step; The determination subunit 2033 is used to automatically select the target step from the candidate steps or manually confirm the target step based on the scoring results.

[0137] In an optional implementation, the test case generation unit 204 includes: Select subunit 2041, which is used to select code templates that match the intent of the corresponding target step from the preset template library according to the execution order of each target step in all target steps; Processing subunit 2042 is used to replace the placeholders in the code template with entity parameters corresponding to the target step, and generate parameterized step code fragments; The splicing subunit 2043 is used to splice the parameterized step code fragments in sequence and encapsulate them in a unified test scaffold structure to generate the executable headless automated test cases.

[0138] In an optional implementation, the system further includes a quality assessment unit 206, which executes all headless automated test cases, collects the execution results of each step during the execution of the test cases, and performs measurement and analysis on the execution results according to preset quality assessment rules to generate test quality assessment results.

[0139] The system of the present invention corresponds to the method described above, and the specific implementation of the system will not be repeated here.

[0140] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for automatically generating headless test cases based on behavioral descriptions, characterized in that, include: S1. Obtain the original behavior description text, and perform normalization processing on the original behavior description text to generate standardized behavior description text; S2. Perform semantic parsing on the standardized behavior description text to generate a structured intermediate representation; S3. Match each IR step in the structured intermediate representation with a preset step library to generate a corresponding target step for each IR step; S4. Based on all the target steps, select the corresponding code templates from the pre-set template library and combine them in sequence to generate executable headless automated test cases. S5. For the interfaces that the headless automated test cases depend on, select the matching Mock data version from the Mock snapshot repository according to the interface contract hash, and perform contract consistency verification.

2. The method according to claim 1, characterized in that, S1 includes: Obtain the original behavior description text; The original behavior description text is subjected to keyword standardization and controlled word replacement to obtain the processed behavior description text. The processed behavior description text is subjected to structural integrity verification. If the verification passes, a standardized behavior description text is obtained.

3. The method according to claim 1, characterized in that, S2 includes: The standardized behavior description text is broken down into multiple step units according to keyword boundaries; Intent recognition is performed for each step unit, and entity parameters and constraints are extracted from each step unit using an entity extractor to generate the structured intermediate representation containing the intent, entity parameters, and constraints.

4. The method according to claim 1, characterized in that, S3 includes: Each IR step in the structured intermediate representation is processed sequentially as follows: Based on the intent in the current IR step, preliminary matching candidate steps are filtered from the pre-defined step library; For each candidate step, a weighted score is calculated based on entity matching degree and context matching degree; The target step is automatically selected from the candidate steps based on the scoring results, or the target step is manually confirmed.

5. The method according to claim 1, characterized in that, S4 includes: S41. Based on the execution order of each target step in all target steps, select a code template from the preset template library that matches the intent of the corresponding target step. S42. Replace the placeholders in the code template with entity parameters corresponding to the target step to generate parameterized step code snippets; S43. The parameterized step code fragments are concatenated in sequence and encapsulated in a unified test scaffold structure to generate the executable headless automated test cases.

6. The method according to claim 1, characterized in that, Following S5, it also includes: Execute all headless automated test cases, collect the execution results of each step in the test case execution process, and perform measurement and analysis on the execution results according to the preset quality assessment rules to generate test quality assessment results.

7. A system for automatically generating headless test cases based on behavioral descriptions, characterized in that, include: The normalization unit is used to obtain the original behavior description text, perform normalization processing on the original behavior description text, and generate standardized behavior description text. A semantic parsing unit is used to perform semantic parsing on the standardized behavior description text to generate a structured intermediate representation; The step matching unit is used to match each IR step in the structured intermediate representation with a preset step library to generate a corresponding target step for each IR step. The test case generation unit is used to select the corresponding code templates from the pre-set template library and combine them in sequence according to all the target steps to generate executable headless automated test cases. The verification unit is used to select a matching Mock data version from the Mock snapshot repository based on the interface contract hash for the interface on which the headless automated test case depends, and to perform contract consistency verification.

8. The system according to claim 7, characterized in that, The normalization unit includes: Get sub-units to retrieve the original behavior description text; The replacement subunit is used to perform keyword standardization and controlled word replacement on the original behavior description text to obtain the processed behavior description text. The verification subunit is used to perform structural integrity verification on the processed behavior description text. If the verification passes, a standardized behavior description text is obtained.

9. The system according to claim 7, characterized in that, The semantic parsing unit includes: The decomposition subunit is used to decompose the standardized behavior description text into multiple step units according to keyword boundaries; A sub-unit is generated to perform intent recognition for each step unit and to extract entity parameters and constraints from each step unit using an entity extractor, thereby generating the structured intermediate representation containing the intent, entity parameters, and constraints.

10. The system according to claim 7, characterized in that, The step matching unit includes: Each IR step in the structured intermediate representation is processed sequentially as follows: The filtering subunit is used to filter preliminary matching candidate steps from a pre-defined step library based on the intent in the current IR step. The scoring subunit is used to give a weighted score of entity matching degree and context matching degree for each candidate step; The sub-unit is determined to automatically select the target step from the candidate steps or manually confirm the target step based on the scoring results.

Images