An automated testing method and system for non-embedded interfaces under test based on large models

Abstract

This invention discloses an automated testing method and system for non-embedded interfaces based on a large model. The method includes: first, semantic parsing the interface description document to extract interface metadata; then, inputting the metadata and prompt information into a first large language model to generate test cases covering normal and abnormal boundary scenarios; next, automatically converting the test cases into executable scripts using a configurable code template engine, and collecting actual response data after execution; finally, calling a second large language model to perform semantic-level difference analysis between the response data and expected assertions, and automatically generating a test report. This achieves automation and intelligence of the entire interface testing process, eliminating the need for manual writing of test cases and scripts, significantly reducing the testing threshold and maintenance costs. Through the semantic understanding capabilities of the large model, it can process complex natural language descriptions and generate comprehensive and diverse test cases, effectively improving test coverage and defect detection rate.

Description

Technical Field

[0001] This invention relates to the field of embedded interface testing technology, and in particular to an automated testing method and system for non-embedded interfaces under test based on a large model. Background Technology

[0002] Currently, in the software development lifecycle, interface testing is a crucial step in ensuring system quality and verifying the correctness of interactions between services. Non-embedded interfaces provide services externally through standard application layer network protocols, and their testing typically relies on testers manually writing test cases, maintaining test scripts, and analyzing test results. In the field of automated interface testing, traditional methods usually require testers to manually read the interface description document, extract metadata such as interface address, request method, parameter constraints, and response structure, then manually write test cases and test scripts based on this metadata, and finally execute the scripts and manually analyze the differences between the response results and expectations.

[0003] However, the above process has the following technical drawbacks:

[0004] Insufficient test case coverage: Manually written test cases are often limited to normal processes and are difficult to systematically cover scenarios such as abnormal inputs and boundary values. Moreover, when the interface version is updated, the workload of test case maintenance is large and it is easy to miss some.

[0005] Response assertions struggle to handle dynamic data and semantic differences: Real-world responses often contain dynamic values ​​such as timestamps and random IDs, which can easily lead to false alarms with traditional exact matching methods; at the same time, they cannot perform semantic-level verification automatically and still rely on manual judgment. Summary of the Invention

[0006] To address the problems mentioned above, this invention provides an automated testing method and system for non-embedded interfaces under test based on a large model, which solves the problems mentioned in the background art, such as insufficient test case coverage caused by manual testing and analysis of interface differences, and the difficulty of response assertions in handling dynamic data and semantic differences.

[0007] An automated testing method for non-embedded interfaces under test based on a large model includes the following steps:

[0008] Semantic parsing is performed on the description documents of the non-embedded interfaces to be tested to extract interface metadata, including interface network address, request method, request parameter constraints and response structure;

[0009] The interface metadata and the constructed prompt information are input into the first language model to generate interface test cases that cover normal business scenarios and abnormal boundary scenarios. Each interface test case includes the combination of request parameter values ​​and the expected response assertion conditions.

[0010] The configurable code template engine automatically converts interface test cases into executable test scripts that match the preset test framework. The executable test scripts are then run in the test environment to initiate network requests to the non-embedded interface under test and collect the corresponding actual response data.

[0011] The second largest language model is invoked to perform semantic-level difference analysis on the actual response data and the expected response assertion conditions, and a test report is automatically generated based on the analysis results.

[0012] Preferably, the semantic parsing of the acquired description document of the non-embedded interface to be tested, and the extraction of interface metadata including the interface network address, request method, request parameter constraints, and response structure, includes:

[0013] Obtain the description document of the non-embedded interface to be tested and identify the format type of the description document. Extract basic metadata from the description document according to the structured parsing rules corresponding to the format type.

[0014] The system locates the natural language description fields in the description document, combines the located natural language description fields with the pre-constructed parsing prompt template, and inputs them into the first language model. This drives the first language model to output a formatted expression of the request parameter constraints and a semantic definition of the response fields.

[0015] The basic metadata is integrated with the formatted expressions and semantic definitions output by the first major language model to construct standardized interface metadata.

[0016] Preferably, the interface metadata and the constructed prompt information are input together into the first large language model to generate interface test cases covering normal business scenarios and abnormal boundary scenarios. Each interface test case includes a combination of request parameter values ​​and expected response assertion conditions, including:

[0017] The prompt information for constructing and adapting the test case generation task is defined in the prompt information, which includes a scenario coverage instruction and an output format instruction. The scenario coverage instruction is used to indicate that the test cases to be generated must cover normal business scenarios, abnormal input scenarios, and boundary value scenarios. The output format instruction is used to specify that each test case must include a scenario category label, a combination of request parameter values, and an expected response assertion condition.

[0018] The interface metadata is converted into a structured representation and concatenated with the prompt information to form the model input sequence;

[0019] The model input sequence is fed into the first language model, which then generates a combination of request parameter values ​​that meet the constraints for different scenarios based on the request parameter constraint rules.

[0020] Based on the response structure definition and scenario category, derive the expected response assertion conditions corresponding to the combination of request parameter values, and output a set of structured interface test cases.

[0021] Preferably, a configurable code template engine automatically converts interface test cases into executable test scripts that match a preset test framework. These executable test scripts are then run in the test environment to initiate network requests to the non-embedded interface under test and collect corresponding actual response data, including:

[0022] Determine the frame type of the preset test framework, and select the corresponding request construction template and assertion template from the preset template library according to the frame type;

[0023] The system parses the combination of request parameter values, extracts the request method, interface network address, path parameters, query parameters, request header and request body parameters, and fills the extracted parameter values ​​into the placeholders of the request construction template through the parameter injection engine to generate the request construction code segment.

[0024] Parse the expected response assertion conditions, extract the status code assertion, response body field assertion, and response header assertion, and fill them into the assertion template to generate the assertion code segment;

[0025] Combine the request construction code segment and the assertion code segment to generate a complete executable test script that matches the syntax rules of the preset test framework;

[0026] In the test environment, launch an executor compatible with the preset test framework to run executable test scripts, so as to initiate network requests to the non-embedded interface under test through standard application layer network protocols and receive the corresponding actual response data.

[0027] The actual response data is associated with and stored with the corresponding interface test case identifier.

[0028] Preferably, the step of calling the second language model to perform semantic-level difference analysis on the actual response data and the expected response assertion conditions, and automatically generating a test report based on the analysis results, includes:

[0029] The actual response data is preprocessed with dynamic field normalization to identify dynamic values ​​in the actual response data, replace the dynamic values ​​with semantically equivalent normalized placeholders, and generate normalized response data.

[0030] The normalized response data, expected response assertion conditions, and the scene category labels are input into the second language model, which drives the second language model to compare the normalized response data with the expected response assertion conditions at the semantic level, and outputs structured difference analysis entries containing difference judgment results, difference location paths, and natural language defect cause inferences.

[0031] Collect all structured difference analysis entries corresponding to all interface test cases in the same test execution batch and aggregate them into a test report.

[0032] Preferably, the step of parsing the expected response assertion conditions, extracting the status code assertion, response body field assertion, and response header assertion, and filling them into the assertion template to generate an assertion code segment includes:

[0033] Extract status code assertions, response header assertions, and response body field assertions from the structured format description parameters of the expected response assertion conditions;

[0034] Determine the frame type of the preset test framework, and select the code template fragments corresponding to the status code assertion, response header assertion, and response body field assertion respectively from the preset assertion template library according to the frame type;

[0035] Parse the expected response assertion conditions, extract the expected status code value, the expected key-value pair set of the response header, and the response body field assertion entry list consisting of field location path, assertion operator, and expected value;

[0036] Fill the expected status code value into the first placeholder of the code template fragment corresponding to the status code assertion, and fill the expected response header key-value pair into the second placeholder of the code template fragment corresponding to the response header assertion;

[0037] Iterate through the list of field assertion entries in the response body, select the corresponding field assertion sub-template based on the assertion operator of each entry, inject the field location path and expected value into the field assertion sub-template, and generate response body field assertion statements one by one.

[0038] The filled status code assertion code snippet, response header assertion code snippet, and all response body field assertion statements are combined according to the assertion statement organization rules of the preset test framework to form an assertion code segment.

[0039] Preferably, the step of acquiring the description document of the non-embedded interface to be tested and identifying the format type of the description document, and extracting basic metadata from the description document through the structured parsing rules corresponding to the format type, includes:

[0040] Receive a document retrieval instruction pointing to the description document of the non-embedded interface under test, and retrieve the original content of the description document by reading it through the file system based on the document location information carried in the document retrieval instruction.

[0041] The document's format type is identified based on its file extension, structured identifier fields in the document content, and specification version declarations within the document content.

[0042] Based on the identified format type, the structured parsing rules corresponding to the format type are loaded from the preset parsing rule library. The structured parsing rules define the document node paths and field mapping relationships used to locate and extract basic metadata under the format type.

[0043] Based on the structured parsing rules, the original content of the description document is parsed, the document node path is traversed to locate the target node, and the field values ​​are extracted from the target node according to the field mapping relationship. Based on the field values, the basic metadata of the non-embedded interface to be tested is generated.

[0044] Preferred options also include:

[0045] The obtained interface metadata and the constructed prompt information are input into the first and third language models respectively to generate the first test case set and the second test case set. The first and third language models adopt different pre-training bases or different fine-tuning strategies. Each test case includes a scenario category label, a combination of request parameter values, and an expected response assertion condition.

[0046] Perform test case-level alignment on the test cases in the first test case set and the second test case set, calculate the difference degree of each aligned test case pair, and mark the test case pairs with a difference degree exceeding a preset threshold as test case pairs to be arbitrated.

[0047] The two test cases in the pair of test cases to be arbitrated, along with the interface metadata and the scenario category tag, are input into the arbitration discriminator to output a consistency score.

[0048] When the consistency score is lower than the judgment threshold, the first and third language models are driven to output the inference paths that generate the corresponding test cases.

[0049] The two inference paths are input into the arbitration discriminator again, which selects to adopt the test case corresponding to one of the inference paths or triggers the generation of fusion test cases, and outputs the finally adopted test case as the target test case.

[0050] The target test cases are converted into executable test scripts and executed, and the actual response data is collected and the second largest language model is called to perform semantic-level difference analysis.

[0051] If the difference analysis results indicate that the failure of the target test case is due to a model generation error rather than an interface defect, then construct an error pattern vector.

[0052] Error pattern vectors are converted into natural language correction instructions and dynamically inserted into the context windows of the first and third language models to adjust the generation behavior of subsequent test cases.

[0053] Regularly collect test cases, execution results, and correction records to form a training dataset. Perform low-rank adaptation fine-tuning on the first and third largest language models. The objective function of the fine-tuning includes the incremental term of the scenario coverage of the test cases.

[0054] Preferably, the process of inputting interface metadata and constructed prompt information into the first language model to generate interface test cases covering normal business scenarios and abnormal boundary scenarios also includes:

[0055] Obtain multiple interface test cases generated by the first major language model, covering normal business scenarios and abnormal boundary scenarios;

[0056] Obtain the main and secondary test function parameters for each interface test case, and determine the test priority index for each interface test case based on the main and secondary test function parameters;

[0057] Determine the business testing rules for each interface test case, and determine the multi-dimensional coverage factor for each interface test case based on the adaptation parameters of the business testing rules;

[0058] Retrieve historical test parameters for each interface test case, and determine the test time, test dependent resources, and test execution complexity for each interface test case based on the historical test parameters;

[0059] The test execution cost of each interface test case is determined based on test time, test dependent resources, and test execution complexity.

[0060] The test fit index of each interface test case is calculated based on its test priority index, multi-dimensional coverage factor, and test execution cost:

[0061]

[0062] in, This represents the test fit index of the i-th interface test case. This represents the business compatibility weight, with a value of 0.2. This represents the multi-dimensional coverage factor of the i-th interface test case. This represents the number of test scenarios covered by the i-th interface test case. This represents the number of assertion type coverages for the i-th interface test case. This represents the boundary coverage gain of the i-th interface test case between normal business scenarios and abnormal boundary scenarios. This represents the test priority weight, with a value of 0.4. Let represent the test priority index of the i-th interface test case. This represents the cost assessment weight, with a value of 0.4. Let $C$ represent the test execution cost of the $i$-th interface test case.

[0063] The first and second interface test cases with the highest test adaptability index were selected as the final interface test cases covering normal business scenarios and abnormal boundary scenarios.

[0064] An automated testing system for non-embedded interfaces under test based on a large model, the system comprising:

[0065] The extraction module is used to perform semantic parsing on the description document of the acquired non-embedded interface to be tested, and extract the interface metadata, including the interface network address, request method, request parameter constraints and response structure.

[0066] The first generation module is used to input the interface metadata and the constructed prompt information into the first language model to generate interface test cases that cover normal business scenarios and abnormal boundary scenarios. Each interface test case includes the combination of request parameter values ​​and the expected response assertion conditions.

[0067] The data collection module is used to automatically convert interface test cases into executable test scripts that match the preset test framework through a configurable code template engine. The executable test scripts are then run in the test environment to initiate network requests to the non-embedded interface under test and collect the corresponding actual response data.

[0068] The second generation module is used to call the second largest language model to perform semantic-level difference analysis on the actual response data and the expected response assertion conditions, and automatically generate a test report based on the analysis results.

[0069] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description and the accompanying drawings.

[0070] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0071] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0072] Figure 1 The flowchart illustrates the workflow of an automated testing method for non-embedded interfaces based on a large model, as provided in this invention.

[0073] Figure 2Another flowchart of the automated testing method for non-embedded interfaces under test based on a large model provided by the present invention;

[0074] Figure 3 This is another flowchart of an automated testing method for non-embedded interfaces under test based on a large model provided by the present invention.

[0075] Figure 4 This is a schematic diagram of the structure of an automated testing system for non-embedded interfaces under test based on a large model, provided by the present invention. Detailed Implementation

[0076] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0077] An automated testing method for non-embedded interfaces under test based on a large model, such as... Figure 1 As shown, it includes the following steps:

[0078] Step S101: Perform semantic parsing on the obtained description document of the non-embedded interface to be tested, and extract the interface metadata including the interface network address, request method, request parameter constraints and response structure.

[0079] Step S102: Input the interface metadata and the constructed prompt information into the first language model to generate interface test cases that cover normal business scenarios and abnormal boundary scenarios. Each interface test case includes the combination of request parameter values ​​and the expected response assertion conditions.

[0080] Step S103: Through the configurable code template engine, the interface test cases are automatically converted into executable test scripts that match the preset test framework. The executable test scripts are run in the test environment to initiate network requests to the non-embedded interface under test and collect the corresponding actual response data.

[0081] Step S104: Call the second language model to perform semantic-level difference analysis on the actual response data and the expected response assertion conditions, and automatically generate a test report based on the analysis results.

[0082] In this embodiment, the method is applied to a test environment independent of the system under test, and the test process does not embed test agents or stub code into the system under test.

[0083] In this embodiment, the non-embedded interface under test is a software interface that provides services to the outside world through standard application layer network protocols.

[0084] In this embodiment, semantic-level difference analysis refers to the large language model determining whether the actual response semantically satisfies the expected assertion conditions based on its understanding of the business meaning of the response content.

[0085] In this embodiment, a configurable code template engine refers to an engine that supports dynamic modification of code templates through external configuration files, databases, or user input, and can adapt to different testing frameworks or coding styles without modifying the engine source code.

[0086] In this embodiment, the first major language model refers to a deep learning model, which has the ability to understand and generate natural language, and is used to convert interface metadata and prompt information into structured test cases.

[0087] In this embodiment, the second language model refers to another deep learning model instance that is independent or shares physical resources. It is used to perform semantic-level difference analysis on the actual response data and the expected assertion conditions, and output the difference judgment results, location path and defect cause.

[0088] The beneficial effects of the above technical solution are as follows: By semantically parsing the interface description document to extract interface metadata, test cases covering normal business scenarios and abnormal boundary scenarios are automatically generated using the first major language model. These test cases are then converted into executable test scripts using a code template engine and run to collect actual responses. Finally, the second major language model is called to perform semantic-level difference analysis and generate a test report. This achieves full automation and intelligence of the interface testing process, eliminating the need for manual writing of test cases and scripts, significantly reducing the testing threshold and maintenance costs. Through the semantic understanding capabilities of the large model, it can handle complex natural language descriptions and generate comprehensive and diverse test cases, effectively improving test coverage and defect detection rate. This solves the problems mentioned in the background technology, such as insufficient test case coverage due to manual testing and analysis of interface differences, and the difficulty in handling dynamic data and semantic differences in response assertions.

[0089] In one embodiment, such as Figure 2 As shown, the semantic parsing of the obtained description document of the non-embedded interface to be tested extracts interface metadata, including the interface network address, request method, request parameter constraints, and response structure, including:

[0090] Step S201: Obtain the description document of the non-embedded interface to be tested and identify the format type of the description document. Extract basic metadata from the description document through the structured parsing rules corresponding to the format type.

[0091] Step S202: Locate the natural language description fields in the description document, combine the located natural language description fields with the pre-constructed parsing prompt template, and input them into the first language model to drive the first language model to output the formatted expression of the request parameter constraints and the semantic definition of the response fields.

[0092] Step S203: Integrate the basic metadata with the formatted expressions and semantic definitions output by the first major language model to construct standardized interface metadata.

[0093] In this embodiment, the basic metadata includes: interface path and HTTP method.

[0094] In this embodiment, the natural language description field contains explanatory text on the constraints of the request parameters and the semantics of the response fields.

[0095] In this embodiment, the interface metadata includes the interface network address formed by concatenating the host address, the base path, and the interface path, the request method, the request parameter list with constraint rules, and the response structure definition with field-level semantic annotations.

[0096] In this embodiment, formatted expression refers to the explicit expression of the constraint rules of request parameters using a structured data description language.

[0097] In this embodiment, semantic definition refers to describing the business meaning of response fields using natural language or structured tags.

[0098] In this embodiment, the parsing prompt template refers to a pre-constructed text template used to guide the large language model to output formatted information.

[0099] The beneficial effects of the above technical solution are as follows: By identifying the format type of the description document, basic metadata is extracted using structured parsing rules, and semantic parsing of the natural language description fields is performed using the first major language model. This outputs formatted parameter constraints and response field definitions, ultimately merging them into standardized interface metadata. It is compatible with various interface document formats and compensates for the omissions in natural language descriptions by pure structured parsing, improving the completeness and accuracy of metadata extraction and laying a reliable foundation for subsequent test case generation.

[0100] In one embodiment, the interface metadata and constructed prompt information are input into a first language model to generate interface test cases covering normal business scenarios and abnormal boundary scenarios. Each interface test case includes a combination of request parameter values ​​and expected response assertion conditions, including:

[0101] The prompt information for constructing and adapting the test case generation task is defined in the prompt information, which includes a scenario coverage instruction and an output format instruction. The scenario coverage instruction is used to indicate that the test cases to be generated must cover normal business scenarios, abnormal input scenarios, and boundary value scenarios. The output format instruction is used to specify that each test case must include a scenario category label, a combination of request parameter values, and an expected response assertion condition.

[0102] The interface metadata is converted into a structured representation and concatenated with the prompt information to form the model input sequence;

[0103] The model input sequence is fed into the first language model, which then generates a combination of request parameter values ​​that meet the constraints for different scenarios based on the request parameter constraint rules.

[0104] Based on the response structure definition and scenario category, derive the expected response assertion conditions corresponding to the combination of request parameter values, and output a set of structured interface test cases.

[0105] In this embodiment, the structured representation describes the request parameters and their constraint rules, response structure and its semantic annotations using key-value pairs or semi-structured language.

[0106] In this embodiment, the scenario category tag refers to the label used to identify the scenario to which the test case belongs.

[0107] In this embodiment, the derivation method is to automatically generate the corresponding assertion logic based on the scenario category and the semantic definition of the fields in the response structure.

[0108] The beneficial effects of the above technical solution are as follows: By constructing prompt information containing scenario coverage instructions and output format instructions, the interface metadata is converted into a structured representation and input into the first major language model. This drives the model to generate request parameter combinations under normal business, abnormal input, and boundary value scenarios based on parameter constraints, and derives the corresponding expected response assertion conditions. This ensures that the generated test cases automatically cover multiple test scenarios, outputting a structured and reusable set of test cases, reducing the subjectivity and omissions of manual test case design, and improving the systematicness and comprehensiveness of testing.

[0109] In one embodiment, a configurable code template engine automatically converts interface test cases into executable test scripts that match a preset test framework. These executable test scripts are then run in the test environment to initiate network requests to the non-embedded interface under test and collect corresponding actual response data, including:

[0110] Determine the frame type of the preset test framework, and select the corresponding request construction template and assertion template from the preset template library according to the frame type;

[0111] The system parses the combination of request parameter values, extracts the request method, interface network address, path parameters, query parameters, request header and request body parameters, and fills the extracted parameter values ​​into the placeholders of the request construction template through the parameter injection engine to generate the request construction code segment.

[0112] Parse the expected response assertion conditions, extract the status code assertion, response body field assertion, and response header assertion, and fill them into the assertion template to generate the assertion code segment;

[0113] Combine the request construction code segment and the assertion code segment to generate a complete executable test script that matches the syntax rules of the preset test framework;

[0114] In the test environment, launch an executor compatible with the preset test framework to run executable test scripts, so as to initiate network requests to the non-embedded interface under test through standard application layer network protocols and receive the corresponding actual response data.

[0115] The actual response data is associated with and stored with the corresponding interface test case identifier.

[0116] In this embodiment, the request construction template defines a request sending code skeleton that conforms to the test framework syntax, and the assertion template defines a response verification code skeleton that conforms to the test framework assertion library.

[0117] In this embodiment, the parameter injection engine refers to the automated component that fills the actual values ​​of the parsed request parameters into the placeholders of the request construction template.

[0118] In this embodiment, standard application layer network protocols refer to TCP / IP-based application layer protocols such as HTTP, HTTPS, HTTP / 2, gRPC, and WebSocket.

[0119] In this embodiment, an executable test script refers to a code file that conforms to the syntax rules of the selected test framework and can be directly run by the corresponding executor.

[0120] The beneficial effects of the above technical solution are as follows: By using a parameter injection engine to populate templates and generate request construction and assertion code segments, these segments are combined to form a complete executable script, which is then run in a test environment to collect actual responses. This solution supports multiple mainstream testing frameworks, achieves automatic conversion of test cases into executable scripts, eliminates the need for manual code writing, and significantly improves test execution efficiency and script maintainability.

[0121] In one embodiment, the step of invoking the second language model to perform semantic-level difference analysis on the actual response data and the expected response assertion conditions, and automatically generating a test report based on the analysis results, includes:

[0122] The actual response data is preprocessed with dynamic field normalization to identify dynamic values ​​in the actual response data, replace the dynamic values ​​with semantically equivalent normalized placeholders, and generate normalized response data.

[0123] The normalized response data, expected response assertion conditions, and the scene category labels are input into the second language model, which drives the second language model to compare the normalized response data with the expected response assertion conditions at the semantic level, and outputs structured difference analysis entries containing difference judgment results, difference location paths, and natural language defect cause inferences.

[0124] Collect all structured difference analysis entries corresponding to all interface test cases in the same test execution batch and aggregate them into a test report.

[0125] In this embodiment, dynamic fields refer to fields that generate unique or unpredictable values ​​with each API call, such as timestamps and random IDs; normalization preprocessing refers to replacing the actual values ​​of these fields with uniform semantic placeholders.

[0126] In this embodiment, semantically equivalent normalized placeholders refer to different dynamic values ​​being mapped to the same placeholder.

[0127] In this embodiment, the difference location path refers to a structured path used to indicate the location of the difference between the actual response and the expected assertion.

[0128] In this embodiment, natural language defect cause inference refers to the description of code or logic problems that may lead to test failure, generated by a large language model and described in natural language.

[0129] The beneficial effects of the above technical solution are as follows: It performs dynamic field normalization preprocessing on the actual response data to avoid misjudgments caused by dynamic values; then, the normalized response, expected assertions, and scene tags are input into the second large language model for semantic-level comparison, outputting structured analysis entries containing difference judgments, location paths, and natural language defect causes, ultimately aggregating to generate a test report. This solves the problem of traditional exact matching being sensitive to dynamic fields, improves judgment accuracy through semantic understanding, and can automatically generate highly readable defect analyses, facilitating testers to quickly locate problems.

[0130] In one embodiment, parsing the expected response assertion conditions, extracting the status code assertion, response body field assertion, and response header assertion and filling them into the assertion template to generate an assertion code segment includes:

[0131] Extract status code assertions, response header assertions, and response body field assertions from the structured format description parameters of the expected response assertion conditions;

[0132] Determine the frame type of the preset test framework, and select the code template fragments corresponding to the status code assertion, response header assertion, and response body field assertion respectively from the preset assertion template library according to the frame type;

[0133] Parse the expected response assertion conditions, extract the expected status code value, the expected key-value pair set of the response header, and the response body field assertion entry list consisting of field location path, assertion operator, and expected value;

[0134] Fill the expected status code value into the first placeholder of the code template fragment corresponding to the status code assertion, and fill the expected response header key-value pair into the second placeholder of the code template fragment corresponding to the response header assertion;

[0135] Iterate through the list of field assertion entries in the response body, select the corresponding field assertion sub-template based on the assertion operator of each entry, inject the field location path and expected value into the field assertion sub-template, and generate response body field assertion statements one by one.

[0136] The filled status code assertion code snippet, response header assertion code snippet, and all response body field assertion statements are combined according to the assertion statement organization rules of the preset test framework to form an assertion code segment.

[0137] In this embodiment, the assertion operator refers to the logical operator used to compare the actual value of the response field with the expected value.

[0138] In this embodiment, the field location path refers to the addressing expression used to uniquely identify a field in the response structure.

[0139] In this embodiment, the structured format description parameter refers to the structured data format that describes the expected response assertion conditions.

[0140] In this embodiment, the structured data format describing the expected response assertion conditions refers to the writing order, grouping method, and syntax structure of the assertion code specified by a specific testing framework.

[0141] The beneficial effects of the above technical solution are as follows: Expected values ​​are extracted according to the type of the expected assertion; corresponding code template fragments are selected based on the test framework type; the expected status code value, response header key-value pairs, field paths, and assertion operators are filled into the template; assertion statements are generated line by line and combined into a complete assertion code segment. This refines the processing of different assertion types, supports multiple assertion operators, and can adapt to the syntax of different test frameworks, improving the accuracy and flexibility of assertion code generation.

[0142] In one embodiment, such as Figure 3As shown, the process of obtaining the description document of the non-embedded interface under test and identifying the format type of the description document, and extracting basic metadata from the description document through the structured parsing rules corresponding to the format type, includes:

[0143] Step S301: Receive a document retrieval instruction pointing to the description document of the non-embedded interface under test, and read the original content of the description document through the file system according to the document location information carried in the document retrieval instruction.

[0144] Step S302: Identify and describe the document's format type based on the file extension, the structured identifier field of the document content, and the specification version declaration declared in the document content;

[0145] Step S303: Based on the identified format type, load the structured parsing rules corresponding to the format type from the preset parsing rule library. The structured parsing rules define the document node paths and field mapping relationships used to locate and extract basic metadata under the format type.

[0146] Step S304: According to the structured parsing rules, parse the original content of the description document, traverse the document node path to locate the target node, extract the field values ​​from the target node according to the field mapping relationship, and generate the basic metadata of the non-embedded interface to be tested based on the field values.

[0147] In this embodiment, the specification version declaration refers to the specification name and its version number that are explicitly declared in the interface description document.

[0148] In this embodiment, structured parsing rules refer to predefined configuration information used to locate and extract basic metadata from documents of a specific format, including document node paths and field mapping relationships.

[0149] In this embodiment, a document node path refers to an addressing expression for locating a specific node in the tree structure of a document.

[0150] In this embodiment, basic metadata refers to the core information extracted directly from the description document through structured rules, without relying on the semantic parsing of a large model.

[0151] The beneficial effects of the above technical solution are as follows: it automatically identifies the format type of the description document by using the file extension, structured identifier field, and specification version declaration; loads the corresponding structured parsing rules; and extracts field values ​​by traversing the document node path to generate basic metadata. This achieves automatic identification and parsing of various interface description document formats. The parsing rules are configurable, enhancing the system's scalability and compatibility, and reducing reliance on manually preset formats.

[0152] In one embodiment, it also includes:

[0153] The obtained interface metadata and the constructed prompt information are input into the first and third language models respectively to generate the first test case set and the second test case set. The first and third language models adopt different pre-training bases or different fine-tuning strategies. Each test case includes a scenario category label, a combination of request parameter values, and an expected response assertion condition.

[0154] Perform test case-level alignment on the test cases in the first test case set and the second test case set, calculate the difference degree of each aligned test case pair, and mark the test case pairs with a difference degree exceeding a preset threshold as test case pairs to be arbitrated.

[0155] The two test cases in the pair of test cases to be arbitrated, along with the interface metadata and the scenario category tag, are input into the arbitration discriminator to output a consistency score.

[0156] When the consistency score is lower than the judgment threshold, the first and third language models are driven to output the inference paths that generate the corresponding test cases.

[0157] The two inference paths are input into the arbitration discriminator again, which selects to adopt the test case corresponding to one of the inference paths or triggers the generation of fusion test cases, and outputs the finally adopted test case as the target test case.

[0158] The target test cases are converted into executable test scripts and executed, and the actual response data is collected and the second largest language model is called to perform semantic-level difference analysis.

[0159] If the difference analysis results indicate that the failure of the target test case is due to a model generation error rather than an interface defect, then construct an error pattern vector.

[0160] Error pattern vectors are converted into natural language correction instructions and dynamically inserted into the context windows of the first and third language models to adjust the generation behavior of subsequent test cases.

[0161] Regularly collect test cases, execution results, and correction records to form a training dataset. Perform low-rank adaptation fine-tuning on the first and third largest language models. The objective function of the fine-tuning includes the incremental term of the scenario coverage of the test cases.

[0162] In this embodiment, the third major language model refers to another major language model instance that uses a different pre-training base or a different fine-tuning strategy than the first major language model. It is used to generate more diverse test cases for comparison and arbitration.

[0163] In this embodiment, test case alignment refers to matching the test cases in the first test case set and the second test case set one-to-one according to the same scenario category label and request parameter constraint structure, so that each pair of test cases after alignment describes the same test intent.

[0164] In this embodiment, the difference degree refers to an indicator used to quantify the degree of difference between two test cases. It can be obtained by weighting the edit distance of the combination of request parameter values, the similarity and difference between field paths and operators in assertion conditions, and the difference in response structure, etc. The value range is [0,1], and the larger the value, the greater the difference.

[0165] In this embodiment, the arbitration discriminator refers to the decision module used to evaluate the consistency of test cases or select the optimal test cases. It can be a rule-based scoring function, a trained classifier, or the large language model itself, and outputs a consistency score.

[0166] In this embodiment, the reasoning path refers to the logical deduction process or intermediate generation steps on which the large language model generates test cases, and is recorded in the form of natural language sequences or structured logs.

[0167] In this embodiment, the fusion test case refers to a compromise test case regenerated by the arbitrator based on the common parts of the two test cases and the arbitration logic when neither of the two inference paths can be fully adopted.

[0168] In this embodiment, the error pattern vector refers to a numerical vector with a fixed dimension, which is used to encode the specific error type of the test failure. For example, the dimension may include: error type (parameter missing / type error / business rule violation), missing field path, semantic difference category between expected value and actual value, etc. Each dimension can take the value of 0 / 1 or continuous value.

[0169] In this embodiment, low-rank adaptation fine-tuning refers to an efficient parameter fine-tuning method that introduces a low-rank decomposition matrix for incremental training while keeping the original weights of the large language model unchanged.

[0170] In this embodiment, the context window refers to the maximum length range of the input token sequence that a large language model can accept in a single inference.

[0171] The beneficial effects of the above technical solution are as follows: By introducing a third language model to collaboratively generate test cases with the first language model, and using test case-level alignment, difference calculation, and arbitration discriminator to automatically adjudicate divergent test cases, combined with inference path analysis and a fusion test case generation mechanism, the accuracy and scenario coverage diversity of test cases are significantly improved. At the same time, when a model generation error rather than an interface defect is detected, an error pattern vector is constructed and converted into a natural language correction instruction, which is then dynamically inserted into the model context window, achieving adaptive correction of test case generation behavior. Furthermore, by combining low-rank adaptation fine-tuning with scenario coverage increment as the target, the model is continuously optimized, thereby forming a closed-loop self-evolution capability, effectively reducing the need for manual intervention, and improving the robustness, coverage, and long-term maintenance efficiency of test case generation.

[0172] In one embodiment, the process of inputting interface metadata and constructed prompt information into the first language model to generate interface test cases covering normal business scenarios and abnormal boundary scenarios also includes:

[0173] Obtain multiple interface test cases generated by the first major language model, covering normal business scenarios and abnormal boundary scenarios;

[0174] Obtain the main and secondary test function parameters for each interface test case, and determine the test priority index for each interface test case based on the main and secondary test function parameters;

[0175] Determine the business testing rules for each interface test case, and determine the multi-dimensional coverage factor for each interface test case based on the adaptation parameters of the business testing rules;

[0176] Retrieve historical test parameters for each interface test case, and determine the test time, test dependent resources, and test execution complexity for each interface test case based on the historical test parameters;

[0177] The test execution cost of each interface test case is determined based on test time, test dependent resources, and test execution complexity.

[0178] The test fit index of each interface test case is calculated based on its test priority index, multi-dimensional coverage factor, and test execution cost:

[0179]

[0180] in, This represents the test fit index of the i-th interface test case. This represents the business compatibility weight, with a value of 0.2. This represents the multi-dimensional coverage factor of the i-th interface test case. This represents the number of test scenarios covered by the i-th interface test case. This represents the number of assertion type coverages for the i-th interface test case. This represents the boundary coverage gain of the i-th interface test case between normal business scenarios and abnormal boundary scenarios. This represents the test priority weight, with a value of 0.4. Let represent the test priority index of the i-th interface test case. This represents the cost assessment weight, with a value of 0.4. Let $C$ represent the test execution cost of the $i$-th interface test case.

[0181] The first and second interface test cases with the highest test adaptability index were selected as the final interface test cases covering normal business scenarios and abnormal boundary scenarios.

[0182] In this embodiment, the main test function parameters refer to the input parameters that have a decisive impact on the correctness of the core business logic of the interface.

[0183] In this embodiment, the test priority index is a value used to quantify the importance of interface test cases. It can be calculated by weighting factors such as whether the parameter is a core required field, the sensitivity of the parameter value range boundary, and the historical defect detection rate. The higher the value, the higher the priority of execution.

[0184] In this embodiment, business testing rules refer to constraints or verification rules related to the business logic of the interface under test.

[0185] In this embodiment, the adaptation parameter refers to a set of parameters used to describe the scope of application and coverage requirements of the business test rules, and is used to quantify the degree of coverage of each test case to the business rules.

[0186] In this embodiment, the multi-dimensional coverage factor refers to the numerical value that comprehensively measures the performance of a test case across multiple coverage dimensions.

[0187] In this embodiment, historical test parameters refer to statistical data recorded when the same or similar interface tests were previously performed.

[0188] In this embodiment, test time refers to the predicted time required to execute the test case, which can be estimated based on the average execution time of similar historical test cases or based on request complexity and response data volume.

[0189] In this embodiment, test dependent resources refer to the external resources required to execute each interface test case.

[0190] In this embodiment, test execution complexity refers to a comprehensive indicator that measures the difficulty of executing test cases.

[0191] In this embodiment, test execution cost refers to the total resource overhead required to execute a single test case.

[0192] In this embodiment, the test fit index refers to a comprehensive indicator used to evaluate the business coverage capability, priority, and execution cost of test cases.

[0193] In this embodiment, the number of test scenarios covered refers to the number of different business scenario categories covered by each interface test case. The scenario categories include at least: normal business scenarios, abnormal input scenarios, and boundary value scenarios.

[0194] In this embodiment, the assertion type coverage refers to the number of different assertion categories used in each interface test case. Assertion types include: status code assertions, response header assertions, response body field value assertions, response body structure assertions, etc.

[0195] In this embodiment, boundary coverage gain refers to a coefficient used to quantify the additional coverage value added by test cases when switching between normal scenarios and abnormal boundaries. The value range is usually (0, 2]. When a test case covers both normal scenarios and relevant boundary scenarios, the gain is greater than 1; otherwise, it is 1 or less than 1.

[0196] In this embodiment, the first two refer to the two test cases ranked first and second after all the calculated test fit indices are sorted from high to low.

[0197] The beneficial effects of the above technical solution are as follows: By quantitatively evaluating multiple interface test cases generated from a large language model, and comprehensively considering the test priority index, multi-dimensional coverage factors, and test execution costs of each test case, a test fit index is calculated, and the two test cases with the highest fit indices are selected as the final test cases. This technical solution achieves automated test case selection, effectively controlling test execution costs while ensuring coverage of high-priority business scenarios and boundary scenarios. It avoids resource waste caused by blindly executing all generated test cases, thereby improving the overall efficiency, economy, and scenario representativeness of the test suite.

[0198] In one embodiment, this embodiment also discloses an automated testing system for non-embedded interfaces under test based on a large model, such as... Figure 4 As shown, the system includes:

[0199] The extraction module 401 is used to perform semantic parsing on the obtained description document of the non-embedded interface to be tested, and extract the interface metadata including the interface network address, request method, request parameter constraints and response structure.

[0200] The first generation module 402 is used to input the interface metadata and the constructed prompt information into the first large language model to generate interface test cases covering normal business scenarios and abnormal boundary scenarios. Each interface test case includes the combination of request parameter values ​​and the expected response assertion conditions.

[0201] The data collection module 403 is used to automatically convert interface test cases into executable test scripts that match the preset test framework through a configurable code template engine. The executable test scripts are then run in the test environment to initiate network requests to the non-embedded interface under test and collect the corresponding actual response data.

[0202] The second generation module 404 is used to call the second largest language model to perform semantic-level difference analysis on the actual response data and the expected response assertion conditions, and automatically generate a test report based on the analysis results.

[0203] The working principle and beneficial effects of the above technical solution have been explained in the method embodiments, and will not be repeated here.

[0204] Those skilled in the art should understand that the terms "first" and "second" in this invention simply refer to different application stages.

[0205] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0206] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A large model-based automatic test method for a non-embedded interface under test, characterized in that, Includes the following steps: Semantic parsing is performed on the description documents of the non-embedded interfaces to be tested to extract interface metadata, including interface network address, request method, request parameter constraints and response structure; The interface metadata and the constructed prompt information are input into the first language model to generate interface test cases that cover normal business scenarios and abnormal boundary scenarios. Each interface test case includes the combination of request parameter values ​​and the expected response assertion conditions. The configurable code template engine automatically converts interface test cases into executable test scripts that match the preset test framework. The executable test scripts are then run in the test environment to initiate network requests to the non-embedded interface under test and collect the corresponding actual response data. The second largest language model is invoked to perform semantic-level difference analysis on the actual response data and the expected response assertion conditions, and a test report is automatically generated based on the analysis results.

2. The automated testing method for non-embedded interfaces under test based on a large model according to claim 1, characterized in that, The semantic parsing of the obtained description document of the non-embedded interface to be tested is performed to extract interface metadata, including interface network address, request method, request parameter constraints, and response structure, including: Obtain the description document of the non-embedded interface to be tested and identify the format type of the description document. Extract basic metadata from the description document according to the structured parsing rules corresponding to the format type. The system locates the natural language description fields in the description document, combines the located natural language description fields with the pre-constructed parsing prompt template, and inputs them into the first language model. This drives the first language model to output a formatted expression of the request parameter constraints and a semantic definition of the response fields. The basic metadata is integrated with the formatted expressions and semantic definitions output by the first major language model to construct standardized interface metadata.

3. The automated testing method for non-embedded interfaces under test based on a large model according to claim 1, characterized in that, The interface metadata and constructed prompt information are input into the first large language model to generate interface test cases covering normal business scenarios and abnormal boundary scenarios. Each interface test case includes a combination of request parameter values ​​and expected response assertion conditions, including: The prompt information for constructing and adapting the test case generation task is defined in the prompt information, which includes a scenario coverage instruction and an output format instruction. The scenario coverage instruction is used to indicate that the test cases to be generated must cover normal business scenarios, abnormal input scenarios, and boundary value scenarios. The output format instruction is used to specify that each test case must include a scenario category label, a combination of request parameter values, and an expected response assertion condition. The interface metadata is converted into a structured representation and concatenated with the prompt information to form the model input sequence; The model input sequence is fed into the first language model, which then generates a combination of request parameter values ​​that meet the constraints for different scenarios based on the request parameter constraint rules. Based on the response structure definition and scenario category, derive the expected response assertion conditions corresponding to the combination of request parameter values, and output a set of structured interface test cases.

4. The automated testing method for non-embedded interfaces under test based on a large model according to claim 3, characterized in that, A configurable code template engine automatically converts interface test cases into executable test scripts that match a preset test framework. These executable test scripts are then run in the test environment to initiate network requests to the non-embedded interface under test and collect corresponding actual response data, including: Determine the frame type of the preset test framework, and select the corresponding request construction template and assertion template from the preset template library according to the frame type; The system parses the combination of request parameter values, extracts the request method, interface network address, path parameters, query parameters, request header and request body parameters, and fills the extracted parameter values ​​into the placeholders of the request construction template through the parameter injection engine to generate the request construction code segment. Parse the expected response assertion conditions, extract the status code assertion, response body field assertion, and response header assertion, and fill them into the assertion template to generate the assertion code segment; Combine the request construction code segment and the assertion code segment to generate a complete executable test script that matches the syntax rules of the preset test framework; In the test environment, launch an executor compatible with the preset test framework to run executable test scripts, so as to initiate network requests to the non-embedded interface under test through standard application layer network protocols and receive the corresponding actual response data. The actual response data is associated with and stored with the corresponding interface test case identifier.

5. The automated testing method for non-embedded interfaces under test based on a large model according to claim 1, characterized in that, The process involves calling the second largest language model to perform semantic-level difference analysis on the actual response data and the expected response assertion conditions, and automatically generating a test report based on the analysis results, including: The actual response data is preprocessed with dynamic field normalization to identify dynamic values ​​in the actual response data, replace the dynamic values ​​with semantically equivalent normalized placeholders, and generate normalized response data. The normalized response data, expected response assertion conditions, and the scene category labels are input into the second language model, which drives the second language model to compare the normalized response data with the expected response assertion conditions at the semantic level, and outputs structured difference analysis entries containing difference judgment results, difference location paths, and natural language defect cause inferences. Collect all structured difference analysis entries corresponding to all interface test cases in the same test execution batch and aggregate them into a test report.

6. The automated testing method for non-embedded interfaces under test based on a large model according to claim 4, characterized in that, The process involves parsing the expected response assertion conditions, extracting status code assertions, response body field assertions, and response header assertions, and filling them into the assertion template to generate an assertion code segment, including: Extract status code assertions, response header assertions, and response body field assertions from the structured format description parameters of the expected response assertion conditions; Determine the frame type of the preset test framework, and select the code template fragments corresponding to the status code assertion, response header assertion, and response body field assertion respectively from the preset assertion template library according to the frame type; Parse the expected response assertion conditions, extract the expected status code value, the expected key-value pair set of the response header, and the response body field assertion entry list consisting of field location path, assertion operator, and expected value; Fill the expected status code value into the first placeholder of the code template fragment corresponding to the status code assertion, and fill the expected response header key-value pair into the second placeholder of the code template fragment corresponding to the response header assertion; Iterate through the list of field assertion entries in the response body, select the corresponding field assertion sub-template based on the assertion operator of each entry, inject the field location path and expected value into the field assertion sub-template, and generate response body field assertion statements one by one. The filled status code assertion code snippet, response header assertion code snippet, and all response body field assertion statements are combined according to the assertion statement organization rules of the preset test framework to form an assertion code segment.

7. The automated testing method for non-embedded interfaces under test based on a large model according to claim 2, characterized in that, The process involves acquiring the description document of the non-embedded interface under test, identifying the format type of the description document, and extracting basic metadata from the description document using the structured parsing rules corresponding to the format type. This includes: Receive a document retrieval instruction pointing to the description document of the non-embedded interface under test, and retrieve the original content of the description document by reading it through the file system based on the document location information carried in the document retrieval instruction. The document's format type is identified based on its file extension, structured identifier fields in the document content, and specification version declarations within the document content. Based on the identified format type, the structured parsing rules corresponding to the format type are loaded from the preset parsing rule library. The structured parsing rules define the document node paths and field mapping relationships used to locate and extract basic metadata under the format type. Based on the structured parsing rules, the original content of the description document is parsed, the document node path is traversed to locate the target node, and the field values ​​are extracted from the target node according to the field mapping relationship. Based on the field values, the basic metadata of the non-embedded interface to be tested is generated.

8. The automated testing method for non-embedded interfaces under test based on a large model according to claim 1, characterized in that, Also includes: The obtained interface metadata and the constructed prompt information are input into the first and third language models respectively to generate the first test case set and the second test case set. The first and third language models adopt different pre-training bases or different fine-tuning strategies. Each test case includes a scenario category label, a combination of request parameter values, and an expected response assertion condition. Perform test case-level alignment on the test cases in the first test case set and the second test case set, calculate the difference degree of each aligned test case pair, and mark the test case pairs with a difference degree exceeding a preset threshold as test case pairs to be arbitrated. The two test cases in the pair of test cases to be arbitrated, along with the interface metadata and the scenario category tag, are input into the arbitration discriminator to output a consistency score. When the consistency score is lower than the judgment threshold, the first and third language models are driven to output the inference path that generates the corresponding test cases. The two inference paths are input into the arbitration discriminator again, which selects to adopt the test case corresponding to one of the inference paths or triggers the generation of fusion test cases, and outputs the finally adopted test case as the target test case. The target test cases are converted into executable test scripts and executed, and the actual response data is collected and the second largest language model is called to perform semantic-level difference analysis. If the difference analysis results indicate that the failure of the target test case is due to a model generation error rather than an interface defect, then construct an error pattern vector. Error pattern vectors are converted into natural language correction instructions and dynamically inserted into the context windows of the first and third language models to adjust the generation behavior of subsequent test cases. Regularly collect test cases, execution results, and correction records to form a training dataset. Perform low-rank adaptation fine-tuning on the first and third largest language models. The objective function of the fine-tuning includes the incremental term of the scenario coverage of the test cases.

9. The automated testing method for non-embedded interfaces under test based on a large model according to claim 1, characterized in that, The process of inputting interface metadata and constructed prompts into the first language model to generate interface test cases covering normal business scenarios and abnormal boundary scenarios also includes: Obtain multiple interface test cases generated by the first major language model, covering normal business scenarios and abnormal boundary scenarios; Obtain the main and secondary test function parameters for each interface test case, and determine the test priority index for each interface test case based on the main and secondary test function parameters; Determine the business testing rules for each interface test case, and determine the multi-dimensional coverage factor for each interface test case based on the adaptation parameters of the business testing rules; Retrieve historical test parameters for each interface test case, and determine the test time, test dependent resources, and test execution complexity for each interface test case based on the historical test parameters; The test execution cost of each interface test case is determined based on test time, test dependent resources, and test execution complexity. The test fit index of each interface test case is calculated based on its test priority index, multi-dimensional coverage factor, and test execution cost: in, This represents the test fit index of the i-th interface test case. This represents the business compatibility weight, with a value of 0.

2. This represents the multi-dimensional coverage factor of the i-th interface test case. This represents the number of test scenarios covered by the i-th interface test case. This represents the number of assertion type coverages for the i-th interface test case. This represents the boundary coverage gain of the i-th interface test case between normal business scenarios and abnormal boundary scenarios. This represents the test priority weight, with a value of 0.

4. Let represent the test priority index of the i-th interface test case. This represents the cost assessment weight, with a value of 0.

4. Let $C$ represent the test execution cost of the $i$-th interface test case. The first and second interface test cases with the highest test adaptability index were selected as the final interface test cases covering normal business scenarios and abnormal boundary scenarios.

10. An automated testing system for non-embedded interfaces under test based on a large model, characterized in that, The system includes: The extraction module is used to perform semantic parsing on the description document of the acquired non-embedded interface to be tested, and extract the interface metadata, including the interface network address, request method, request parameter constraints and response structure. The first generation module is used to input the interface metadata and the constructed prompt information into the first language model to generate interface test cases that cover normal business scenarios and abnormal boundary scenarios. Each interface test case includes the combination of request parameter values ​​and the expected response assertion conditions. The data collection module is used to automatically convert interface test cases into executable test scripts that match the preset test framework through a configurable code template engine. The executable test scripts are then run in the test environment to initiate network requests to the non-embedded interface under test and collect the corresponding actual response data. The second generation module is used to call the second largest language model to perform semantic-level difference analysis on the actual response data and the expected response assertion conditions, and automatically generate a test report based on the analysis results.

Images