Method, device and equipment for generating test case based on large language model

Abstract

The invention provides a method, device and equipment for generating a test case based on a large language model, and the method comprises the steps: obtaining a test demand of the test case, analyzing the test demand, and obtaining the type, the application field, the business process and the first key test information of the test case; on the basis of the type of the test case, the application field, the business process and the first key test information, constructing a cue word for generating the test case; and based on the cue word, adopting a pre-trained large language model to generate a test case. According to the method and the device, the cue word is constructed by considering the application field and the business process to which the test case belongs, so that the large language model can construct the test case in combination with knowledge information of the application field and context constraints of related business processes, and the generated test case naturally conforms to system business logic and a state machine contract; and seamless embedding with the existing system is realized.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of artificial intelligence, and in particular, to a method, device and equipment for generating test cases based on large language models. BACKGROUND

[0002] Under the tide of enterprise digital transformation, automated test case generation technology gradually replaces manual writing, significantly improving test efficiency and relieving the coverage pressure of complex scenarios. However, the test cases generated by existing tools often conflict with system workflows, resulting in test case execution failures or invalid results, and the test efficiency is not improved but decreased and the maintenance cost is increased. Therefore, how to seamlessly embed the generated test cases into the system workflow has become a technical problem to be solved by those skilled in the art. SUMMARY

[0003] Therefore, the present disclosure proposes a method, device, equipment and storage medium for generating test cases based on large language models, which can make the generated test cases naturally comply with the system business logic and state machine contract, and realize seamless embedding with existing systems.

[0004] According to a first aspect of the present disclosure, a method for generating test cases based on a large language model is provided, comprising:

[0005] Obtaining a test requirement of a test case, and parsing the test requirement to obtain a type, an application field, a business process and first key test information of the test case to which the test case belongs;

[0006] Based on the type, the application field, the business process and the first key test information of the test case to which the test case belongs, a prompt word for generating the test case is constructed;

[0007] Based on the prompt word, a pre-trained large language model is used to generate the test case.

[0008] In a possible implementation, when the test requirement is parsed to obtain the type, the application field, the business process and the first key test information of the test case to which the test case belongs, it includes:

[0009] The test requirement is parsed to obtain the type, the application field and the business process of the test case to which the test case belongs;

[0010] Based on the type and the application field, key test information in the test requirement is extracted to obtain the first key test information of the test case.

[0011] In a possible implementation, when the key test information in the test requirement is extracted based on the type and the application field to obtain the first key test information of the test case, it includes:

[0012] acquire knowledge information matching the application field;

[0013] based on the knowledge information matching the application field, extract key test information in the test requirement by using a key test information extraction algorithm matching the type, to obtain first key test information of the test case.

[0014] In a possible implementation, the type to which the test case belongs includes one of a functional test case and an interface test case.

[0015] In a possible implementation, when the prompt word for generating the test case is constructed based on the type to which the test case belongs, the application field, the business process, and the first key test information, the following operations are included:

[0016] filter a target business process associated with the test case from the business process to which the test case belongs, and acquire a test case of the target business process;

[0017] perform semantic fusion analysis on the first key test information and the test case of the target business process, to obtain second key test information;

[0018] generate the prompt word for the test case based on the type to which the test case belongs, the application field, and the second key test information.

[0019] In a possible implementation, when the test case is generated based on the prompt word by using a pre-trained large language model, the following operations are included:

[0020] generate an initial test case based on the prompt word by using a pre-trained large language model;

[0021] perform structural adjustment on the initial test case, to obtain the test case.

[0022] In a possible implementation, after the test case is generated, the following operation is further included: performing compliance auditing on the test case.

[0023] According to a second aspect of the present disclosure, a device for generating a test case based on a large language model is provided, including:

[0024] a data acquisition module, configured to acquire a test requirement of a test case, and parse the test requirement to obtain a type, an application field, a business process, and first key test information to which the test case belongs;

[0025] The prompt word generation module is configured to construct a prompt word for generating the test case based on the type, application field, business process, and first key test information to which the test case belongs.

[0026] The test case generation module is configured to generate the test case based on the prompt word by using a pre-trained large language model.

[0027] According to a third aspect of the present disclosure, a device for generating a test case based on a large language model is provided, comprising a processor, a memory for storing processor-executable instructions, wherein the processor is configured to execute the method according to the first aspect of the present disclosure.

[0028] According to a fourth aspect of the present disclosure, a non-volatile computer-readable storage medium is provided, which stores computer program instructions, wherein the computer program instructions are executed by a processor to implement the method according to the first aspect of the present disclosure.

[0029] The present disclosure provides a method, device, equipment and storage medium for generating a test case based on a large language model, which comprises: obtaining a test requirement of a test case, and analyzing the test requirement to obtain a type, an application field, a business process and first key test information to which the test case belongs; constructing a prompt word for generating the test case based on the type, the application field, the business process and the first key test information to which the test case belongs; and generating the test case based on the prompt word by using a pre-trained large language model. In the present disclosure, the prompt word is constructed based on the application field and the business process to which the test case belongs, so that the large language model can construct the test case in combination with the knowledge information of the application field and the context constraint of the related business process, and thus the generated test case naturally conforms to the system business logic and the state machine contract, and realizes seamless embedding with the existing system.

[0030] Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the drawings. BRIEF DESCRIPTION OF DRAWINGS

[0031] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the present disclosure and serve to explain the principles of the present disclosure.

[0032] Figure 1 A flowchart of a method for generating a test case based on a large language model according to an embodiment of the present disclosure is shown;

[0033] Figure 2 A schematic block diagram of a device for generating a test case based on a large language model according to an embodiment of the present disclosure is shown;

[0034] Figure 3A schematic block diagram of a device for generating a test case based on a large language model according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0035] Various exemplary embodiments, features, and aspects of the present disclosure will be explained in greater detail below with reference to the accompanying drawings. Like reference numerals may be used to refer to like elements throughout. While various aspects of embodiments are illustrated, the embodiments need not be used to scale unless specifically stated.

[0036] The term "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any implementation described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other implementations.

[0037] In addition, for the purpose of convenience and brevity, detailed descriptions of well-known functions and structures incorporated in the present disclosure can be omitted. It will be appreciated that those skilled in the art will be able to devise various modes of implementing the present disclosure without the benefits of these presently omitted details.

[0038] <Method Embodiment>

[0039] Figure 1 A flowchart of a method for generating a test case based on a large language model according to an embodiment of the present disclosure is shown. As shown in the flowchart, the method includes steps S1100-S1300. Figure 1

[0040] S1100, obtaining a test requirement of a test case, and parsing the test requirement to obtain a type, an application field, a business process, and first key test information to which the test case belongs.

[0041] The system for implementing the method of the present disclosure is provided with a test requirement uploading control, and a user can upload a test requirement of a test case through the test requirement uploading control. After the test requirement uploading is completed, the system can obtain the test requirement of the test case. When uploading the test requirement, pure text data can be directly uploaded, so that the system can directly obtain the test requirement in the form of text; documents in the formats of.docx,.pdf,.txt, etc. can be uploaded, so that the system can obtain the test requirement in the form of text through parsing of the document data; a web page URL including the test requirement can also be uploaded, so that the system can crawl the test requirement in the form of text in the web page through the URL. It should be noted that for a functional test case, the test requirement thereof needs to include a type, an application field, a business process, first key test information, and an expected result, etc. to which the test case belongs, and for an interface test case, the test requirement thereof needs to include an interface API document. ​

[0042] After obtaining the test requirement of the test case, the test requirement can be parsed to obtain the type, application field, business process and first key test information of the test case. The type of the test case includes one of a function test case and an interface test case. The application field of the test case is an industry or professional category served by the test case. The business process of the test case is an end-to-end business operation sequence served by the test case. The first key test information of the test case is a minimum necessary element set extracted from the test requirement and driving the generation of the test case.

[0043] It should be noted that the type of the test case is different, and the corresponding first key test information is also different. Specifically, when the type of the test case is a function test case, the first key test information includes a key entity, a key action and a constraint condition. The key entity is a core data object or a system component bearing a business function in the test requirement. The key action is a business operation or a state change instruction acting on the key entity in the test requirement. The constraint condition is a business logic and a technical specification limiting the execution boundary and verification rule of the key action in the test requirement. For example, the test requirement is to verify that after a user creates an order, the payment process is completed within 30 minutes, and the key entity extracted from the test requirement is an order, the key action is to create an order and pay for the order, and the constraint condition is that the payment time is less than or equal to 30 minutes. When the type of the test case is an interface test case, the first key test information includes an interface name, a parameter name, an operation instruction and a constraint condition.

[0044] In a possible implementation, when the test requirement is parsed to obtain the type, application field, business process and first key test information of the test case, the following steps can be included:

[0045] First, the test requirement is parsed to obtain the type, application field and business process of the test case. It should be noted that in order to ensure the accuracy of the test case generation, the type, application field and business process of the test case have been recorded in the test requirement. Therefore, when the test requirement is parsed, the type, application field and business process of the test case can be directly parsed.

[0046] Second, the key test information in the test requirement is extracted based on the parsed type and application field to obtain the first key test information of the test case. Specifically, the knowledge information matching the application field is obtained; then based on the knowledge information matching the application field, the key test information in the test requirement is extracted by using the key test information extraction algorithm matching the type to obtain the first key test information of the test case.

[0047] ​It should be noted that the system is provided with a domain knowledge base, which stores knowledge information corresponding to each application domain, so that the corresponding knowledge information can be extracted from the domain knowledge base after the application domain to which the test requirement belongs is parsed. The knowledge information of each domain can include the term table, data dictionary and domain rules of the domain. The term table is used to record the professional terms commonly used in the domain. The data dictionary is used to record the detailed information of each professional term commonly used in the domain, for example, the definition of the patent term, the data type, the data length, the constraint condition and the like. The domain rules are used to record the normative documents that need to be followed in the domain. For example, for the banking business domain, the domain rules can include: the localized Capital Adequacy Ratio Management Approach for ensuring capital adequacy, the Liquidity Risk Management Guidelines for controlling liquidity risk, the Commercial Bank Credit Asset Management Approach for post-loan management, the Deposit Insurance Regulations for ensuring deposit safety, the KYC / AML policy for anti-money laundering and customer identity recognition, and the Banking Industry Internal Control Guidelines for internal control and audit, etc.

[0048] It should be further noted that the system pre-configures different key test information extraction algorithms for each test case type, and after the type to which the test case belongs is parsed, the corresponding key test information extraction algorithm will be matched according to the type to which the test case belongs.

[0049] After obtaining the knowledge information of the application domain to which the test case belongs and matching the key test information extraction algorithm according to the type to which the test case belongs, the key test information in the test requirement can be extracted based on the knowledge information matched with the application domain to which the test case belongs, using the key test information extraction algorithm matched with the type to which the test case belongs, to obtain the first key test information of the test case.

[0050] In the case where the type to which the test case belongs is a functional test case, when the key test information in the test requirement is extracted based on the knowledge information matched with the application domain to which the test case belongs, using the key test information extraction algorithm matched with the type to which the test case belongs, to obtain the first key test information of the test case, the following steps can be included: first, based on the term table in the knowledge information matched with the application domain, the corresponding key entities, key actions and constraint conditions are extracted from the test requirement. Second, the corresponding key entities, key actions and constraint conditions are at least one of data conversion processing of data format conversion, coding conversion and data type conversion, so that the converted key entities, key actions and constraint conditions are matched with the requirements of the data dictionary in the knowledge information matched with the application domain. Finally, the converted key entities, key actions and constraint conditions are taken as the first key test information of the test case, and stored in the short-term memory module in the system to provide the first key test information of the planning plan for subsequent test case generation. The above data extraction (Extract), data conversion (Transform) and data storage (Load) processes are the first key test information extraction process based on the ETL pipeline.

[0051] In the case where the type to which the test case belongs is an interface test case, when the key test information (i.e. the first key test information of the test case) in the test requirement is extracted based on the knowledge information matched with the application domain to which the test case belongs, using the key test information extraction algorithm matched with the type to which the test case belongs, the following steps can be included:

[0052] First, analyze the test requirement to determine whether the associated interface (such as a pre-dependent interface or a downstream trigger interface) of the test case exists in the business process to which the test case belongs.

[0053] In the case where there is no associated interface, a lightweight ETL pipeline is used to extract the first key test information. The specific steps are as follows:

[0054] 1. Term table matching: based on the term table in the knowledge information matched with the application domain, the corresponding interface name, parameter name, operation instruction and constraint condition elements are extracted from the test requirement.

[0055] 2. Data conversion: at least one of data conversion processing of data format conversion, coding conversion and data type conversion is performed on the elements extracted in step 1, to ensure that the converted elements are consistent with the data dictionary format in the knowledge information matched with the application domain.

[0056] 3. Short-term memory storage: the converted elements are taken as the first key test information of the test case, and stored in the short-term memory module in the system to provide the standardized input for subsequent test case generation.

[0057] In the case of associated interfaces, the multi-source fusion processing method is used to extract the first key test information. The specific steps are as follows:

[0058] 1. Multi-source element extraction

[0059] 1) For the demand side: based on the term table in the knowledge information matched with the application field, extract the corresponding interface name, parameter name, operation instruction and constraint condition elements from the test demand.

[0060] 2) For the associated interface side: use the same method to extract the interface elements related to the current test demand from the history or external test cases of the associated interface:

[0061] Interface name and parameter: after extraction, standardization processing is required, such as uniform case, removing prefixes or suffixes, etc.

[0062] Operation instruction sequence: parse the multi-stage calling steps and generate the operation instruction sequence according to the multi-stage calling steps (such as calling interface A first, then calling interface B to confirm);

[0063] Constraint condition set: merge the verification or business constraints of the same field in different test cases to form a constraint condition set (such as inventory > 0, user status = active);

[0064] Context data: extract example values, default values and abnormal value ranges from multi-source data (including test requirements, historical or external test cases).

[0065] 2. Deep data processing

[0066] 1) Complex conversion: batch standardization processing is performed on all extracted elements (such as uniform date / time zone format, encoding method, numerical precision, etc.) to ensure consistency across sources;

[0067] 2) Multi-source fusion: merge the constraint conditions, example values and default values of the demand side and associated interface side test cases to build a unified business model;

[0068] 3) Field mapping: based on the synonym library and business semantics, automatically identify and map the names pointing to the same physical field in different sources.

[0069] 3. COT thought chain construction: using the thought chain (COT) technology, organize the deeply processed data in the following logical order:

[0070] 1) Preparation: define the input set (demand interface and its elements, associated interface and its elements);

[0071] 2) Execute the following reasoning steps in turn:

[0072] Inference 1, naming alignment, to unify naming and format, ensure comparability;

[0073] Inference 2, dependency recognition, to determine the dependency order between fields, combined with business rules;

[0074] Inference 3, constraint synthesis, to generate complete constraint checking procedures based on multi-source fusion results.

[0075] 3) Chain output: generate a clear parameter-constraint-call order chain as the final format of the first key test information.

[0076] The following is an example of the association processing process between the submit order interface (OMS.submitOrder) and the inventory check interface (INV.checkStock). The generation process of the first key test information in the case of associated interfaces is described in detail as follows:

[0077] 1. Multi-source element extraction

[0078] 1) Extract the following elements for the demand side (OMS.submitOrder):

[0079] Interface name: OMS.submitOrder

[0080] Parameter name: userId, productId, quantity

[0081] Constraint condition: quantity>0

[0082] Business rule: stock>=quantity (need to be implemented through associated interfaces)

[0083] 2) Extract the following elements for the associated interface side (INV.checkStock):

[0084] Interface name: INV.checkStock

[0085] Parameter name: productId

[0086] Return value: availableStock

[0087] Constraint condition: availableStock>=quantity (business rule in historical use cases).

[0088] 2. Deep data processing

[0089] 1) Complex conversion: unified naming specification (such as all lowercase, remove redundant prefixes):

[0090] quantity→orderQuantity

[0091] availableStock→stockLevel

[0092] 2) Field mapping: based on synonym library and business semantic association:

[0093] quantity (demand side) orderQuantity (unified field)

[0094] availableStock (association side) stockLevel (unified field)

[0095] 3) Multi-source fusion:

[0096] Merge constraints:

[0097] Demand side constraints: orderQuantity > 0

[0098] Association side constraints: stockLevel >= orderQuantity

[0099] Generate unified business model: orderQuantity > 0 AND stockLevel >= orderQuantity

[0100] 3. (CoT) Thought chain construction

[0101] 1) Premise preparation input set:

[0102] Demand interface: OMS.submitOrder (parameters userId, productId, orderQuantity)

[0103] Association interface: INV.checkStock (parameters productId, return value: stockLevel)

[0104] 2) Reasoning steps:

[0105] Reasoning 1, named alignment:

[0106] Unified field naming (quantity→orderQuantity, availableStock→stockLevel).

[0107] Reasoning 2, dependency relationship identification:

[0108] Business logic dependency: call INV.checkStock to verify stock first, then call OMS.submitOrder to submit order.

[0109] Inference 3: constraint composition:

[0110] Fusion of multi-source constraints to generate complete verification process:

[0111] text

[0112] Step 1: Verify that orderQuantity>0

[0113] Step 2: Call INV.checkStock to get stockLevel

[0114] Step 3: Verify that stockLevel>=orderQuantity.

[0115] Chain output (first key test information):

[0116] text

[0117] Call order sequence chain:

[0118] 1. INV.checkStock(productId)→returns stockLevel

[0119] 2. OMS.submitOrder(userId,productId,orderQuantity)

[0120] Parameter-constraint model:

[0121] -Input parameters:

[0122] userId: [example value]

[0123] productId: [example value]

[0124] orderQuantity: [example value] (constraint: >0)

[0125] -Cross-interface constraints:

[0126] stockLevel (from INV.checkStock)≥orderQuantity.

[0127] After generating the type, application field, business process and first key test information to which the test case belongs, the prompt word for generating the test case can be constructed based on the type, application field, business process and first key test information to which the test case belongs. Specifically, it can include the following steps:

[0128] First, the target business process associated with the test case is screened out from the business process to which the test case belongs, and the test case of the target business process is obtained.

[0129] First of all, it should be pointed out that a long-term memory module is provided in the system, which is used to record a plurality of business processes constructed in the system which the test case needs to be embedded and a plurality of test cases corresponding to the plurality of business processes. In this way, after obtaining the business process to which the test case belongs, the target business process associated with the test case can be screened out from the long-term memory module, and the test case of the target business process is extracted.

[0130] Second, the first key test information and the test case of the target business process are subjected to semantic fusion analysis to obtain the second key test information. Specifically, the following steps are included: 1. Multi-source fusion: merging the constraint conditions and context data of both (refer to the method of "deep data processing - multi-source fusion"); 2. Dependency expansion: based on the business process topology, adding cross-case timing dependency rules (such as interface A needs to be called before interface B); 3. Output the enhanced second key test information (including complete parameter-constraint chain across business processes).

[0131] Third, based on the type to which the test case belongs, the application field and the second key test information, the prompt word of the test case is generated. Specifically, the prompt word template matching the type is obtained; the application field and the second key test information are filled into the prompt word template, and the prompt word for generating the test case is obtained.

[0132] S1300, based on the prompt word, a pre-trained large language model is used to generate a test case. The specific steps are as follows:

[0133] First, based on the prompt word, a pre-trained large language model is used to generate an initial test case. Specifically, the model parameters of the large language model are first set, the model parameters including at least one of a temperature coefficient and a maximum generation degree. Then, the prompt word is input into the pre-trained large language model, and the large language model can automatically generate the corresponding initial test case.

[0134] In one possible implementation, for a functional test case, the large language model includes the following steps when generating an initial test case based on the prompt word:

[0135] 1. Requirement analysis: the model first performs semantic analysis on the input prompt word, extracts the function point name, involved entity, operation action and constraint condition by means of the built-in dictionary or learned domain knowledge; at the same time, the business process or user expectation implied in the prompt word is mapped into a test scene element.

[0136] 2. Scenario grooming: Based on the extracted functional elements, automatically divide typical positive scenarios (normal flow) and negative scenarios (input exceptions, boundary conditions, abnormal interruptions, etc.), and label the preconditions and environmental configurations involved (such as login status, data preparation, interface availability, etc.) for each scenario.

[0137] 3. Test step generation: Based on the "operation step" template, fill in the following in sequence: preconditions (Pre-conditions), operation actions (Step-by-step), input data or parameter values, and for each step of action, the model will complete the required clicks, calls, or input instructions based on the context prompts.

[0138] 4. Expected result writing: For each test step, generate specific expected output or system state description based on the constraints given in the requirement document or prompt words. If the scenario is an abnormal flow, describe error prompts, log records, or recovery mechanisms, etc.

[0139] 5. Boundary and exception expansion: Based on the built-in boundary value analysis strategy, automatically derive boundary value use cases for key input parameters (such as maximum value, minimum value, null value, super-long value, special character input); when the prompt words contain keywords such as "exception handling", "error code", etc., the model further expands to generate more negative test scenarios.

[0140] 6. Format verification and standardization: Format the generated initial test cases according to the pre-defined structured document template (including fields such as case number, case title, priority level, executor, version number, etc.) for output; and check the completeness and naming standardization of each field against the pre-set test specification (such as ISTQB standard, enterprise internal specification).

[0141] 7. Iterative optimization: After the initial test cases are generated, based on external feedback (such as manual review comments, test execution results), optimize and iterate the initial test cases, including: merging duplicate test cases, enriching operation step descriptions, and correcting or supplementing expected results.

[0142] 8. Final output: Output the optimized test cases in editable format (such as Markdown) for subsequent maintenance and execution.

[0143] In one possible implementation, for interface test cases, the large language model can include the following steps when generating initial test cases based on prompt words:

[0144] 1. Requirement analysis and structure mapping:

[0145] 1) Parse the second key test information in the prompt words (i.e. parameter-constraint-call sequence chain), extract the following structured elements:

[0146] Interface call sequence (e.g. INV.checkStock→OMS.submitOrder)

[0147] Uniform parameter set (e.g. userId, productId, orderQuantity, stockLevel)

[0148] Cross-interface constraint rules (e.g. orderQuantity>0 AND stockLevel≥orderQuantity)

[0149] 2) According to the application domain (e.g. e-commerce system) that the test case belongs to, load the domain-specific data specification (e.g. date format is YYYY-MM-DD, currency unit is RMB).

[0150] 2. Test scenario construction:

[0151] 1) According to the call sequence chain, generate positive scenarios (satisfy all constraints) and negative scenarios (violate single / multiple constraints):

[0152] Example positive scenario: submit order when inventory is sufficient

[0153] Example negative scenario: submit order when inventory is insufficient, order quantity is 0

[0154] 2) Label each scenario with preconditions (e.g. dependent interface availability, test environment identification);

[0155] 3) According to the domain rules matched by the application domain, derive compliance test scenarios (e.g. financial domain needs to add verification scenarios required by the Deposit Insurance Regulations).

[0156] 3. Test data generation:

[0157] 1) Based on parameter constraints and domain specifications, automatically generate compliance data:

[0158] Standard value (e.g. orderQuantity=5)

[0159] Boundary value (e.g. orderQuantity=1, orderQuantity=MAX_INT)

[0160] Abnormal value (e.g. orderQuantity=0, productId="")

[0161] 2) Extract context data (e.g. example value userId="U1001") from second key test information.

[0162] 4. Request & Expected Result Generation:

[0163] 1) Request Construction:

[0164] Generate request method (GET / POST), request headers, parameters and values for each interface in call order.

[0165] Example Request 1 (INV.checkStock):

[0166] http

[0167] GET / inventory / check?productId=P2001

[0168] Headers: {"Content-Type":"application / json"}

[0169] Example Request 2 (OMS.submitOrder):

[0170] http

[0171] POST / order / submit

[0172] Body: {"userId":"U1001","productId":"P2001","orderQuantity":5}

[0173] 2) Expected Result Generation:

[0174] Response Code (e.g. 200 OK or 400 Bad Request)

[0175] Response Body Checkpoints (e.g. check stockLevel >= orderQuantity)

[0176] Cross-Interface Business Rule Verification (e.g. check order status is "Created")

[0177] 5. Exception Handling Extension:

[0178] When the prompt word contains the "exception handling" keyword, automatically derive error scenarios based on constraint rules, example: generate stockLevel < orderQuantity when expected to return error code INSUFFICIENT_STOCK.

[0179] 6. Structured Output: Format output according to pre-defined interface test case template, fields include:

[0180]

[0181]

[0182] Secondly, the initial test case is structurally adjusted to obtain a test case. Specifically, a standard test case template of each type of test case is pre-set in the system, after the initial test case is generated, a standard test case template matching the type of the test case is obtained, and then the initial test case is split and integrated according to the form of the standard test case template, and finally a standardized test case is obtained.

[0183] In a possible implementation, the standard test case template matched with the function test case includes the following structured fields: case number, case name, precondition, interface request parameter, expected response effect, etc.

[0184] In a possible implementation, the standard test case template matched with the interface test case can include the following structured fields:

[0185] 1. Case number: a number or code uniquely identifying the test case;

[0186] 2. Case name: a brief description of the interface function or test purpose;

[0187] 3. Interface URL: the complete path of the interface to be tested (including host, port, resource path);

[0188] 4. Request method: GET / POST / PUT / DELETE / PATCH, etc.

[0189] 5. Request header: necessary Header information (such as Content-Type, Authorization, etc.);

[0190] 6. Request parameter: query parameter (Query Params), path parameter (Path Params), form parameter (FormData), JSON / XML, etc. Request Body;

[0191] 7. Precondition: environment configuration (such as test environment identification), dependent interface call sequence (such as calling interface A first and then calling interface B), or data preparation;

[0192] 8. Test data: standard data, boundary value, abnormal value;

[0193] 9. Expected response: HTTP status code, response header (Headers), response body structure and field check (field name, type, mandatory), business check point (such as return data correctness, business logic check);

[0194] 10. Postcondition: clean up data (such as delete test generated records), restore environment to initial state, priority / level, identify the importance or execution order of the use case;

[0195] 12. Performer: specify the person responsible for executing the test;

[0196] 13. Execution result: actual response status and content, result determination (pass / fail);

[0197] 14. Remark: other supplementary notes or special handling matters.

[0198] In one possible implementation, after the test case is generated, an operation of performing compliance auditing on the test case is further included. Specifically, a domain rule matched with an application domain (such as the Capital Adequacy Management Approach in the banking domain) is called, and it is detected whether the test case conforms to the business specification and the data security standard: if yes, the test case is directly output as a final test case; if not, the non-compliant content is output and prompted to facilitate the developer to make targeted adjustment, and the adjusted test case is output as a final test case. The preset auditing rule includes at least one of a domain rule (i.e., a business logic rule of the domain) to which the test case belongs, a data security specification, and an interface calling specification.

[0199] The present disclosure provides a method for generating a test case based on a large language model, comprising: obtaining a test requirement of the test case, and parsing the test requirement to obtain a type, an application domain, a business process, and first key test information of the test case; constructing a prompt word for generating the test case based on the type, the application domain, the business process, and the first key test information of the test case; and generating the test case based on the prompt word by using a pre-trained large language model. In the present disclosure, the prompt word is constructed based on the application domain and the business process to which the test case belongs, so that the large language model can construct the test case in combination with the knowledge information of the application domain and the context constraint of the related business process, and thus the generated test case naturally conforms to the system business logic and the state machine contract, and seamless embedding with the existing system is achieved.

[0200] <Device Embodiment>

[0201] Figure 2 A schematic block diagram of a device for generating a test case based on a large language model according to an embodiment of the present disclosure is shown. As shown in the figure, Figure 2 the device 100 comprises:

[0202] The data acquisition module 110 is configured to acquire test requirements of the test case, and parse the test requirements to obtain a type, an application field, a business process and first key test information to which the test case belongs.

[0203] The prompt word generation module 120 is configured to construct a prompt word for generating the test case based on the type, the application field, the business process and the first key test information to which the test case belongs.

[0204] The test case generation module 130 is configured to generate the test case based on the prompt word and by using a pre-trained large language model.

[0205] <Device Embodiment>

[0206] Figure 3 A schematic block diagram of a device for generating a test case based on a large language model according to an embodiment of the present disclosure is shown. As shown in the figure, the device 200 for generating a test case based on a large language model includes a processor 210 and a memory 220 for storing executable instructions of the processor 210. Wherein the processor 210 is configured to implement the method for generating a test case based on a large language model as described above when executing the executable instructions. Figure 3

[0207] Here, it should be pointed out that the number of processors 210 can be one or more. Meanwhile, the device 200 for generating a test case based on a large language model in the embodiment of the present disclosure can also include an input device 230 and an output device 240. Wherein the processor 210, the memory 220, the input device 230 and the output device 240 can be connected through a bus, or can be connected through other ways, which is not limited here.

[0208] The memory 220 as a kind of computer readable storage medium can be used to store software programs, computer executable programs and various modules, such as programs or modules corresponding to the method for generating a test case based on a large language model in the embodiment of the present disclosure. The processor 210 executes the software programs or modules stored in the memory 220, thereby performing various functional applications and data processing of the device 200 for generating a test case based on a large language model.

[0209] The input device 230 can be used to receive input numbers or signals. Wherein the signals can be key signals related to user settings and function control of the device / terminal / server. The output device 240 can include display devices such as display screens.

[0210] <Storage Medium Embodiment>

[0211] ​According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium is also provided, which has stored thereon computer program instructions, which when executed by a processor 210, implement any of the above-mentioned methods for generating test cases based on a large language model.

[0212] Embodiments of the present disclosure have been described above, the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes are obvious to those skilled in the art without departing from the scope and spirit of the described embodiments. The choice of terms used herein is intended to best explain the principles, practical applications, or technical improvements of the technology in the market, or to enable other ordinary skilled persons in the art to understand the embodiments disclosed herein.

Claims

1. A method for generating test cases based on a large language model, characterized in that: include: Obtaining test requirements of the test case, and parsing the test requirements to obtain the type, application field, business process and first key test information of the test case; Constructing prompt words for generating the test case based on the type, application field, business process and first key test information of the test case; Based on the prompt words, a pre-trained large language model is used to generate the test case.

2. The method according to claim 1, characterized in that When the test requirement is parsed to obtain the type, application field, business process and first key test information of the test case, it includes: Analyze the test requirements to obtain the type, application field, and business process of the test case; Key test information in the test requirement is extracted based on the type and the application field to obtain first key test information of the test case.

3. The method according to claim 2, characterized in that When extracting key test information from the test requirement based on the type and the application field to obtain first key test information of the test case, the method includes: Acquiring knowledge information matching the application field; Based on the knowledge information matching the application field, a key test information extraction algorithm matching the type is used to extract the key test information in the test requirement to obtain the first key test information of the test case.

4. The method according to claim 1, wherein The type of the test case includes one of a functional test case and an interface test case.

5. The method according to claim 1, wherein When constructing prompt words for generating the test case based on the type, application field, business process and first key test information of the test case, the following steps are included: Filtering a target business process associated with the test case from the business processes to which the test case belongs, and obtaining a test case for the target business process; Performing semantic fusion analysis on the first key test information and the test case of the target business process to obtain second key test information; A prompt word for the test case is generated based on the type and application field of the test case and the second key test information.

6. The method according to claim 1, characterized in that When generating the test case based on the prompt word and using a pre-trained large language model, the method includes: Based on the prompt words, using a pre-trained large language model to generate initial test cases; The initial test case is structurally adjusted to obtain the test case.

7. The method according to claim 1, characterized in that After the test case is generated, the process also includes performing a compliance review on the test case.

8. A device for generating test cases based on a large language model, characterized in that: include: A data acquisition module is used to obtain the test requirements of the test case and parse the test requirements to obtain the type, application field, business process and first key test information of the test case; A prompt word generation module, configured to construct prompt words for generating the test case based on the type, application field, business process, and first key test information of the test case; The test case generation module is used to generate the test case based on the prompt word using a pre-trained large language model.

9. A device for generating test cases based on a large language model, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to implement the method according to any one of claims 1 to 7 when executing the executable instructions.

10. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images