Test case arrangement method and electronic equipment
By introducing a large language model and an automated interface testing framework, interface test cases are automatically orchestrated, solving the problems of cumbersome and time-consuming interface testing processes and the inability to combine test cases into complex business processes, thus achieving efficient and accurate interface testing.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHINA MOBILE (SUZHOU) SOFTWARE TECH CO LTD
- Filing Date
- 2026-02-04
- Publication Date
- 2026-05-01
AI Technical Summary
Currently, the interface testing process is cumbersome and time-consuming. Manual drag-and-drop operations are inefficient, and the limitations of tag-based classification make it impossible to divide test cases in detail. Independent test cases cannot be combined into complex business processes, and automated interface testing cannot be executed automatically.
By introducing a large language model, user business requirements information is obtained, interface description text information is generated using the large language model, the interface business process is determined, and automatic orchestration and execution are achieved in conjunction with an interface automation testing framework.
It improves the efficiency and orchestration accuracy of interface testing, reduces manual intervention, lowers labor costs, reduces human error, and achieves a closed loop of automated testing.
Smart Images
Figure CN121958129A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of interface automation testing, and more specifically, to a test case orchestration method and an electronic device. Background Technology
[0002] Interface testing is a type of software testing that focuses on examining the interactions between system components. Specifically, interface testing tests application programming interfaces (APIs) to verify the correctness, functionality, performance, security, and stability of data exchange between different software modules or services. In daily interface testing, testers organize and plan interface test cases to efficiently and systematically verify interface functionality.
[0003] However, at present, the interface testing process requires manual operation by testers or the tagging of test cases. The interface testing process is cumbersome, time-consuming, and inefficient. Summary of the Invention
[0004] This application addresses some of the shortcomings mentioned in the background technology by providing a test case orchestration method and an electronic device. This solution introduces a large language model, enabling automatic orchestration of interface test cases. It effectively solves the problems of tedious and time-consuming manual drag-and-drop operations, limitations of tag-based classification leading to incomplete test case segmentation, and the inability to combine independent test cases into complex business processes, significantly improving testing efficiency and orchestration accuracy.
[0005] Firstly, a test case orchestration method is provided. This method includes: acquiring user business requirement information; obtaining interface description text information output by a first language model based on the user business requirement information, the interface description text information describing multiple interfaces arranged in a first order; acquiring the correspondence between multiple interface test cases and the multiple interfaces, wherein each interface test case in the correspondence includes interface request information indicating the requested interface; and determining interface business process information based on the correspondence and the multiple interfaces arranged in the first order, the interface business process information indicating the multiple interface test cases arranged in the first order.
[0006] This solution allows users to automatically generate correctly ordered interface business process information simply by inputting their business requirements, replacing tedious manual drag-and-drop operations. Furthermore, by reducing the workload of manually orchestrating test cases, it effectively lowers labor costs and minimizes additional expenses caused by human error.
[0007] In conjunction with the first aspect, in a possible implementation of the first aspect, obtaining user business requirement information includes: obtaining natural language text input by the user, the natural language text being used to describe multiple requirements of the user; determining the user business requirement information based on the natural language text, wherein the user business requirement information includes multiple execution process descriptions corresponding to the multiple requirements and a first interval identifier, the first interval identifier being used to separate every two adjacent execution process descriptions.
[0008] This solution can transform user-input natural language requirements into structured business requirements, facilitating subsequent model processing.
[0009] In conjunction with the first aspect, in a possible implementation of the first aspect, obtaining the interface description text information output by the first language model based on the user business requirement information includes: segmenting the user business requirement information into words based on the first interval identifier in the user business requirement information to obtain the multiple execution process description information; generating multiple vector feature representations corresponding one-to-one with the multiple execution process description information; and determining the interface description text information based on the multiple vector feature representations, wherein the interface description text information includes the multiple interfaces and a second interval identifier, the second interval identifier being used to separate every two adjacent interfaces.
[0010] This solution leverages the powerful semantic understanding and text generation capabilities of large language models to accurately generate interface description text containing the required interfaces and their order based on structured business requirements.
[0011] In conjunction with the first aspect, in a possible implementation of the first aspect, determining the interface business process information based on the correspondence and the multiple interfaces arranged in the first order includes: performing word segmentation and vectorization processing on the interface description text information including the multiple interfaces to obtain the first vectorized representation information corresponding to the interface description text information; performing word segmentation and vectorization processing on the interface request information of the multiple interface test cases to obtain the second vectorized representation information corresponding to the interface request information of the multiple interface test cases; and determining the interface business process information based on the Euclidean distance between the first vectorized representation information and the second vectorized representation information.
[0012] This scheme, through vectorization and distance calculation, can accurately match interface descriptions with existing interface test cases, find the test cases that best match the description, and determine their order.
[0013] In conjunction with the first aspect, in a possible implementation of the first aspect, the method further includes: determining the interface business process information based on the Jaccard similarity between the first vectorized representation information and the second vectorized representation information.
[0014] This scheme provides a method for calculating Jaccard similarity, which further enhances the flexibility and accuracy of matching.
[0015] In conjunction with the first aspect, in a possible implementation of the first aspect, before inputting the user's business requirement information into the first large language model, the method further includes: obtaining a second large language model; and fine-tuning the second large language model based on a large model fine-tuning algorithm to obtain the first large language model.
[0016] This approach fine-tunes the basic large language model to better suit the specific task of generating interface description text, thereby improving the accuracy and reliability of the model's output.
[0017] In conjunction with the first aspect, among the possible implementations of the first aspect, the method further includes: calling the interface automation testing framework to present the business process corresponding to the interface business process information.
[0018] This solution not only enables automatic orchestration of test cases, but also allows the generated business processes to be presented or executed by calling the automated testing framework, forming a complete automated testing closed loop.
[0019] In a second aspect, an electronic device is provided. The electronic device includes one or more processors; one or more memories; the one or more memories storing one or more computer programs, the one or more computer programs including instructions that, when executed by the one or more processors, cause a method as described in the first aspect or any possible implementation thereof to be performed.
[0020] Thirdly, a computer-readable storage medium is provided. This computer-readable storage medium stores computer instructions that, when executed on a computer, cause a method as described in the first aspect or any possible implementation thereof to be performed.
[0021] Fourthly, a computer program product is provided. When the computer program product is run on a computer, it causes the computer to perform the methods as described in the first aspect or any possible implementation thereof. Attached Figure Description
[0022] Figure 1 A schematic flowchart of the test case orchestration method provided in an embodiment of this application is shown; Figure 2 A flowchart of the interface testing process provided in an embodiment of this application is shown; Figure 3 A schematic diagram of a test case orchestration process provided in an embodiment of this application is shown; Figure 4This is a structural schematic diagram of a device provided in an embodiment of this application; Figure 5 This is a structural schematic diagram of a system on a chip (SoC) provided in an embodiment of this application. Detailed Implementation
[0023] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.
[0024] The terminology used in the following embodiments is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to also include expressions such as “one or more,” unless the context clearly indicates otherwise. It should also be understood that in the following embodiments of this application, “at least one” and “one or more” refer to one, two, or more than two. The term “and / or” is used to describe the relationship between related objects, indicating that three relationships may exist; for example, A and / or B can indicate: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character “ / ” generally indicates that the preceding and following related objects are in an “or” relationship.
[0025] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.
[0026] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.
[0027] First, the background technology involved in the embodiments of this application will be introduced.
[0028] As mentioned earlier, API testing verifies the correctness, functionality, performance, security, and stability of data exchange between different software modules or services. A well-organized API test case program ensures coverage of all critical scenarios while improving testing efficiency and accuracy. Structured test case programming facilitates test case maintenance and updates, especially when APIs change, making it easier to adjust testing strategies. Furthermore, similar or identical test logic can be reused across different projects or versions, saving time and resources.
[0029] However, the current interface testing process is cumbersome and time-consuming, resulting in low efficiency.
[0030] For example, testers need to manually drag and drop to arrange the data, which is cumbersome and time-consuming. This is especially problematic when there are a large number of test cases requiring manual drag-and-drop arrangement, particularly when the test flow needs repeated adjustments and optimizations, impacting testing efficiency and project schedule. Furthermore, as time goes on and the project evolves, interface changes or testing requirements increase, manually adjusting these graphical flowcharts can become chaotic and difficult to manage, resulting in high maintenance costs.
[0031] For another example, it is necessary to label the use cases, such as setting the use cases as performance use cases or security use cases. However, the label items can only roughly classify the use case types and have limitations. They do not contain detailed use case descriptions, such as what specific functions the use case can test. Therefore, it is not possible to further subdivide the interface use cases.
[0032] For another example, this solution can only generate independent interface test cases. However, daily testing tasks typically involve complex test logic, and this solution cannot further orchestrate multiple interface test cases to form business processes with complex business logic. Furthermore, this solution does not integrate with automated testing for execution and cannot automatically execute automated interface test cases and output test reports.
[0033] To address the aforementioned issues, this application proposes a test case orchestration method 100. Figure 1 A schematic flowchart of the test case orchestration method 100 provided in an embodiment of this application is shown. Figure 1 As shown, method 100 includes steps S110 to S140. In method 100, a large language model is introduced to realize the automatic orchestration of interface test cases, which effectively solves the problems of tedious and time-consuming manual drag-and-drop operations, the limitations of tag-based classification leading to the inability to finely divide test cases, and the inability of independent test cases to be combined into complex business processes, thus significantly improving testing efficiency and orchestration accuracy.
[0034] Step S110: Obtain user business requirement information.
[0035] The user business requirement information can be a description of the interface business process. The interface business process can be a complex business process that can be created by arranging interface test cases according to a business requirement description.
[0036] Optionally, in embodiments of this application, the user-input natural language can also be converted into structured business requirement information through natural language processing. Specifically, obtaining the user's business requirement information includes: obtaining the user-input natural language text, which describes multiple requirements of the user; and determining the user's business requirement information based on the natural language text.
[0037] In the embodiments of this application, user business requirement information may refer to requirement data that has undergone structured processing, such as including multiple execution process descriptions and a first separator identifier. For example, the first separator identifier includes special characters such as semicolons, commas, or custom word vectors (such as "<|div|>"), used to separate the description information.
[0038] Step S120: Using the first language model, based on the user's business requirements information, obtain the interface description text information output by the first language model.
[0039] The interface description text describes multiple interfaces arranged in first order. Large language models (LLMs) are deep learning-based natural language processing models characterized by training on very large datasets and containing a large number of parameters. These models, trained on massive amounts of text data, are capable of understanding and generating human language. These models can typically perform a variety of tasks, including but not limited to: language translation, text generation, text summarization, sentiment analysis, and dialogue systems.
[0040] Optionally, in embodiments of this application, the large language model can be fine-tuned before input. Specifically, before inputting the user's business requirement information into the first large language model, the method further includes: obtaining a second large language model; and fine-tuning the second large language model based on a large model fine-tuning algorithm to obtain the first large language model.
[0041] It should be understood that, in the embodiments of this application, the first large language model may refer to a fine-tuned language model, such as one fine-tuned by a large model fine-tuning algorithm like the P-Tuning V2 algorithm, used to generate interface description text.
[0042] For example, large language models require parameter fine-tuning training before deployment. The fine-tuning algorithm used is P-TuningV2, and the goal is to ensure that the model's predicted interface business processes meet requirements. P-Tuning v2 is a soft-prompt tuning technique that can adjust the model's behavior by learning a small number of trainable vectors without updating the model's main parameters. This helps to avoid overfitting during fine-tuning to some extent. Furthermore, to further prevent overfitting, L2 norm penalty can be introduced during fine-tuning.
[0043] Optionally, in embodiments of this application, vectorization processing can be used to improve generation accuracy. Specifically, obtaining the interface description text information output by the first language model based on the user business requirement information includes: segmenting the user business requirement information into words based on the first interval identifier in the user business requirement information to obtain the multiple execution process description information; generating multiple vector feature representations corresponding one-to-one with the multiple execution process description information; and determining the interface description text information based on the multiple vector feature representations. The interface description text information includes the multiple interfaces and a second interval identifier, which is used to separate every two adjacent interfaces.
[0044] For example, after a user inputs a specific business requirement description, the first language model performs word segmentation, dividing the input text into individual word units. Next, the segmented words are converted into vector representations, i.e., embedding, mapping the words from the original text space to a high-dimensional vector space, ensuring that semantically similar words are close in distance. The embedded words then enter the first language model for further processing, undergoing a series of transformations and processing to extract features from the input text. Finally, the model generates the next possible word (token), based on the features extracted in the previous steps and the language model learned during training and fine-tuning. Through a recursive process, the first language model generates the next most probable word based on the previously input text and the generated text. This results in the orchestration of interface use case text information, with the generated result conforming to multiple "interface name: description" text formats.
[0045] For example, in the embodiments of this application, the interface description text information may refer to the text describing the interface sequence, such as "interface name 1: description 1 <|div|>,..., <|div|> interface name n: description n", and the second interval identifier as described above "<|div|>".
[0046] Step S130: Obtain the correspondence between multiple interface test cases and the multiple interfaces.
[0047] In this correspondence, each test case for the interface includes interface request information, which indicates the interface to be requested.
[0048] It should be understood that in the embodiments of this application, the correspondence can refer to data association. For example, an interface test case includes interface request information such as API call parameters, which are used to indicate the requested interface such as the login API.
[0049] Step S140: Based on the correspondence and the multiple interfaces arranged in the first order, determine the interface business process information, which is used to indicate the test cases of the multiple interfaces arranged in the first order.
[0050] Optionally, in embodiments of this application, various similarity calculation methods may also be employed. Specifically, determining the interface business process information based on the correspondence and the plurality of interfaces arranged in the first order includes: performing word segmentation and vectorization processing on the interface description text information including the plurality of interfaces to obtain a first vectorized representation information corresponding to the interface description text information; performing word segmentation and vectorization processing on the interface request information of the plurality of interface test cases to obtain a second vectorized representation information corresponding to the interface request information of the plurality of interface test cases; and determining the interface business process information based on the Euclidean distance between the first vectorized representation information and the second vectorized representation information.
[0051] Optionally, the method further includes: determining the interface business process information based on the Jaccard similarity between the first vectorized representation information and the second vectorized representation information.
[0052] It should be understood that, in the embodiments of this application, the interface business process information may refer to a sequence of test cases, such as test cases arranged in a first order, like login test cases and query test cases. The following embodiments will describe this process in detail, and will not be repeated here.
[0053] Optionally, in embodiments of this application, an automated testing framework may also be integrated. Specifically, the method further includes: invoking the interface automated testing framework to present the business process corresponding to the interface business process information. It should be understood that in embodiments of this application, the interface automated testing framework may refer to tools such as Postman or Selenium, used to execute or display the business process.
[0054] The API automation testing framework is a set of tools and specifications designed to simplify and standardize the API testing process. It helps teams efficiently write, manage, and run automated API test cases while ensuring the accuracy of test results.
[0055] In summary, the method 100 of this application improves the efficiency of the interface testing process by acquiring user business requirement information, generating interface descriptions using a large language model, obtaining the corresponding relationships of test cases, and ultimately determining business process information, thereby achieving automatic orchestration of interface test cases and replacing tedious manual operations. Furthermore, this application also improves the model's adaptability and matching accuracy by converting natural language into structured requirements, fine-tuning the large language model, vectorizing the data, and invoking an automated testing framework.
[0056] The following describes an embodiment of method 100 with reference to the accompanying drawings.
[0057] This application embodiment combines a large language model with an interface automation testing framework to support the automatic orchestration of user requirements into business processes with complex business logic, consisting of multiple interface automation test cases. The generated business processes can be automatically executed through the interface automation testing framework. Figure 2 A flowchart illustrating the interface testing process provided in an embodiment of this application is shown. Figure 2 As shown, the main implementation steps are as follows: test data preparation, model fine-tuning, interface test case orchestration, automated test framework integration, and test execution.
[0058] First, during the test data preparation process, it is necessary to train the large language model (i.e., the aforementioned second large language model) for subsequent test case orchestration.
[0059] Specifically, the training data required includes the following three items: (1) Interface test cases. Also known as API test cases, these are the basic units organized in this invention. The data includes the test case name, test case description, and request information (which may include the request body, request method, request parameters, URL, etc.). In the embodiments of this application, the large language model mainly uses the test case name and test case description, while the automated testing framework mainly uses the interface request information.
[0060] (2) Interface business process. This data is a collection of interface test cases, which consists of a business process composed of several ordered interface test cases.
[0061] (3) Business requirements. This data is a description of the requirements for the interface business process. That is, based on a business requirement description, the interface test cases can be sorted into a business process with complex business logic.
[0062] In this embodiment of the application, when collecting interface test cases, the name and detailed description of the test cases can be extracted. The test case name can be obtained directly. The description of the test case includes extracting the title and describe fields from the JSON data of the test case, cleaning up irregular symbols and semantically meaningless garbled characters, and using this part of the data as the test case description information.
[0063] Based on the understanding of the requirements, the interface test cases are orchestrated into a business process that conforms to those requirements, ensuring that the requirements are associated with the business process. The requirements can be actual content or a description of the business process to be orchestrated.
[0064] Optionally, to help the model better understand the user's requirements, the user's requirements are rewritten and reconstructed to create interface test cases and requirement data that conform to a key:value JSON file. For example, the text in "demanda" can be rewritten as the structured text "apiflow" as follows: { "demanda": "Manages a folder, including folder creation, modification, dragging and dropping operations"; "apiflow": "Add folder: Add a folder <|div|>Drag folder: Drag a folder from one directory to another <|div|>Edit folder: Change the name of a folder <|div|>.... <|div|>Delete folder: Delete a folder"; } Here, "<|div|>" is a custom word vector token (i.e., the aforementioned first or second separator). Before subsequent model fine-tuning, the tokenizer is loaded from the pre-trained model, and then this custom token is added to the tokenizer. Finally, the updated tokenizer is saved. During subsequent model fine-tuning, the new tokenizer is used to segment and vectorize the text. By using the above text to fine-tune the model, the model can learn to add the "<|div|>" separator between each "key:value" unit.
[0065] Finally, all datasets are split into training, validation, and test sets in a 6:2:2 ratio. The training set is used to train the model to fit the specific task. The validation set is used to tune hyperparameters and prevent overfitting during training. The test set is used to finally evaluate the model's performance. Ultimately, a large language model (i.e., the second largest language model mentioned above) can be trained for test case orchestration.
[0066] Next, a model fine-tuning process can be performed, that is, fine-tuning the aforementioned second-largest language model to obtain the aforementioned first-largest language model. For example, before the second-largest language model is put into use, parameter fine-tuning training is required. The fine-tuning algorithm uses the P-Tuning V2 algorithm, and the goal of fine-tuning is to ensure that the interface business process predicted by the model meets the requirements. P-Tuning v2 is a fine-tuning technique based on soft prompting, which can adjust the model's behavior by learning a small number of trainable vectors without updating the main parameters of the model. In this way, overfitting can be avoided to a certain extent during the fine-tuning process. At the same time, to further avoid overfitting, L2 norm penalty can be introduced during the fine-tuning process. Finally, after fine-tuning, the aforementioned first-largest language model is obtained.
[0067] Then, the interface test case orchestration process is carried out, which is to generate interface business process scripts that meet the requirements through the first major language model.
[0068] Specifically, corresponding to step S110, after receiving the specific business requirement description input by the user (i.e., the natural language text input by the user), the structured text of the business requirement description is obtained based on the aforementioned text rewriting process (i.e., the user business requirement information in step S110 obtained after word segmentation). For example, the text in "demanda" is rewritten as the structured text "apiflow" as follows: { "demanda": "Manages a folder, including folder creation, modification, dragging and dropping operations"; "apiflow": "Add folder: Add a folder <|div|>Drag folder: Drag a folder from one directory to another <|div|>Edit folder: Change the name of a folder <|div|>.... <|div|>Delete folder: Delete a folder"; } Next, corresponding to step S120, the first large language model first performs word segmentation on the text, dividing the input text into individual word units. The segmented words need to be converted into vector representations, i.e., embeddings. Then, the words are mapped from the original text space to a high-dimensional vector space, so that semantically similar words are close in distance in the vector space. The embedded words are then processed within the large model, undergoing a series of transformations and processing to extract the features of the input text. Finally, the model generates the next possible word (token), based on the features extracted in the previous steps and the language model learned during fine-tuning. This process is recursive; the first large language model generates the next most probable word based on the previously input text and the generated text. This yields the orchestration interface use case text information in the fine-tuning task, with the generated result conforming to multiple "interface name: description" text formats (i.e., the interface description text information in step S110).
[0069] Next, corresponding to steps S130 and S140, after obtaining the generated text "Interface Name 1: Description 1 <|div|>,..., <|div|> Interface Name n: Description n" (i.e., interface description text information), it is necessary to match the generated text with the interface test cases (i.e., match the interface) to finally obtain an executable business process composed of several interface test cases (i.e., the interface business process information in step S140).
[0070] First, the generated text needs to be segmented into n units, each containing an interface name and description, based on the custom delimiter "<|div|>" (i.e., the aforementioned second separator). These units can be called interface text units. Corresponding to the mapping in step S130, the interface text units are matched against the interface test cases. Based on the data preparation method described earlier, the request interface name and description text information of the interface test cases are obtained. Then, the interface text units and the text extracted from the interface test cases are matched using a vector matching and Jaccard similarity fusion scheme. For ease of description, the interface text unit will be denoted as "query," and the text extracted from the test cases will be denoted as "doc."
[0071] For example, in the vector matching process, the query is first segmented into words, then the word vector model is used to vectorize the segmentation results, and finally the mean of all word vector results is obtained as the vectorized representation of the query. The same steps are used to obtain the vectorized representation of each document, and then the Euclidean distance between the query and the vectorized result of each document is calculated. The smaller the distance, the higher the similarity. The Euclidean distance between the query and the nth document is recorded as follows: .
[0072] Exemplarily, the present application can also perform matching based on the Jaccard similarity. Specifically, in the first step, word segmentation is performed to remove meaningless stop words, such as "de", "ye", etc. In this way, a set of word segmentation results can be obtained. The query word segmentation set is denoted as , and the word segmentation set of the doc is denoted as . The Jaccard similarity calculation formula is as follows:
[0073] Record the Jaccard similarity between the query and the nth doc as . Finally, use a weight coefficient to fuse and , as the final similarity result, as shown in the following formula:
[0074] In the embodiment of the present application, the test case with the smallest value is used as the most similar matching result. At the same time, a threshold t can be set. If is greater than t, it is considered that no relevant test case is matched.
[0075] Finally, after matching to the specific interface cases, based on the front-end technology, the orchestrated multiple interfaces are presented to the user in a visual manner, and the user is supported to further modify and adjust the order of the interface cases to generate an executable business process.
[0076] Exemplarily, Figure 3 shows a schematic diagram of a test case orchestration process provided by an embodiment of the present application. As Figure 3 shown, corresponding to step S110, the user requirement obtained is described as "manage a folder, including operations such as folder creation, modification, and dragging". Corresponding to step S120, the foregoing user requirement description is input into the large language model. After parsing the requirement by the large language model, multiple corresponding interface cases are generated, including adding a folder, dragging a folder, editing a folder, deleting a folder, etc. Finally, corresponding to steps S130 and S140, based on these interface cases, a complete operation link of "add folder → drag folder → edit folder → delete folder" is constructed to achieve the full-process management of the folder.
[0077] Finally, this embodiment of the application can execute the generated business process by calling the interface automation testing framework. The orchestrated interface business process script includes multiple request URLs for request interfaces, and request parameters include headers, params, body, etc. Simultaneously, this embodiment of the application can also include expected result verification, forming these parts into a standardized JSON data structure and inputting it into the interface automation testing framework. The interface automation testing framework automatically parses the script, executes interface tests, and compares the interface execution results with expectations to determine whether the interface execution result is successful or unsuccessful. After all test cases have been executed, execution information is automatically collected and test results are summarized, and a test report is finally generated.
[0078] In summary, this application proposes an automatic test case orchestration method based on a large language model. This method can automatically orchestrate existing automated test cases for interfaces according to requirements, generate business processes with complex business logic, and present them to users in a visual manner, simplifying the process of testers writing these processes. Furthermore, through integration with an automated interface testing framework, it automatically runs interface test cases and returns test results.
[0079] Compared to the background technologies, the large language generation model used in this application has stronger generalization ability, stronger contextual understanding ability, and can automatically learn features from data, reducing reliance on manual feature engineering, making it more powerful, flexible, and efficient. Furthermore, this proposal can automatically invoke an automated testing framework to execute tests and output test reports.
[0080] This application provides a computer program product that, when run on an electronic device, causes the electronic device to execute the technical solutions described in the above embodiments. Its implementation principle and technical effects are similar to those of the related embodiments described above, and will not be repeated here.
[0081] This application provides a readable storage medium containing instructions that, when executed by an electronic device, cause the electronic device to perform the technical solution described in the above embodiments. The implementation principle and technical effects are similar and will not be repeated here.
[0082] This application provides a chip for executing instructions. When the chip is running, it executes the technical solutions described in the above embodiments. Its implementation principle and technical effects are similar and will not be repeated here.
[0083] The hardware module of this application is described below, which can be used to implement the aforementioned method 100, and can also implement... Figure 2 and Figure 3 The process is shown below.
[0084] Now for reference Figure 3The diagram shows a block diagram of a device 400 according to one embodiment of this application. Device 400 may include one or more processors 401 coupled to a controller hub 403. In at least one embodiment, the controller hub 403 communicates with the processor 401 via a multi-branch bus such as a front side bus (FSB), a point-to-point interface such as a quickpath interconnect (QPI), or a similar connection 410. The processor 401 executes instructions controlling general types of data processing operations. In one embodiment, the controller hub 403 includes, but is not limited to, a graphics memory controller hub (GMCH) (not shown) and an input / output hub (IOH) (which may be on a separate chip) (not shown), wherein the GMCH includes memory and a graphics controller and is coupled to the IOH.
[0085] Device 400 may also include a coprocessor 402 and a memory 404 coupled to a controller hub 403. Alternatively, one or both of the memory and the GMCH may be integrated within the processor, with memory 404 and coprocessor 402 directly coupled to processor 401 and controller hub 403, which resides on a single chip with the IOH. Memory 404 may be, for example, dynamic random access memory (DRAM), phase change memory (PCM), or a combination of both. In one embodiment, coprocessor 402 is a dedicated processor, such as, for example, a high-throughput many integrated core (MIC) processor, a network or communication processor, a compression engine, a graphics processor, a general-purpose computing on GPU (GPGPU), or an embedded processor, etc. Optional properties of coprocessor 402 are indicated by dashed lines. Figure 4 middle.
[0086] Memory 404, as a computer-readable storage medium, may include one or more tangible, non-transitory computer-readable media for storing data and / or instructions. For example, memory 404 may include any suitable non-volatile memory such as flash memory and / or any suitable non-volatile storage device such as one or more hard-disk drives (HDD(s)), one or more compact disc (CD) drives, and / or one or more digital versatile disc (DVD) drives.
[0087] In one embodiment, device 400 may further include a network interface controller (NIC) 406. NIC 406 may include a transceiver for providing a radio interface to device 400, thereby enabling communication with any other suitable device, such as a front-end module, antenna, etc. In various embodiments, NIC 406 may be integrated with other components of device 400. NIC 406 can implement the functionality of the communication unit in the above embodiments.
[0088] Device 400 may further include input / output (I / O) device 405. I / O 405 may include: a user interface designed to enable a user to interact with device 400; a peripheral component interface designed to enable peripheral components to also interact with device 400; and / or sensors designed to determine environmental conditions and / or location information related to device 400.
[0089] It is worth noting that, Figure 4 This is merely an example. That is, although... Figure 4 The diagram shows that device 400 includes multiple devices such as processor 401, controller hub 403, and memory 404. However, in actual applications, devices using the methods of this application may include only a portion of the devices in device 400. For example, it may include only processor 401 and NIC 406. Figure 4 The properties of the optional devices are shown in dashed lines. According to some embodiments of this application, the memory 404, which is a computer-readable storage medium, stores instructions that, when executed on a computer, cause the device 400 to perform the methods according to the above embodiments. Specific details can be found in the methods of the above embodiments, and will not be repeated here.
[0090] Now for reference Figure 5 The diagram shown is a block diagram of a system-on-chip (SoC) 500 according to an embodiment of this application. Figure 5In the diagram, similar components share the same reference numerals. Additionally, dashed boxes are an optional feature for more advanced SoCs. Figure 5 In this SoC 500, the following are included: an interconnect unit 550 coupled to an application processor 510; a system proxy unit 580; a bus controller unit 590; an integrated memory controller unit 540; a group or one or more coprocessors 520, which may include integrated graphics logic, an image processor, an audio processor, and a video processor; a static random access memory (SRAM) unit 530; and a direct memory access (DMA) unit 560. In one embodiment, the coprocessor 520 includes a dedicated processor, such as, for example, a network or communication processor, a compression engine, a GPGPU, a high-throughput MIC processor, or an embedded processor.
[0091] The static random-access memory (SRAM) unit 530 may include one or more computer-readable media for storing data and / or instructions. The computer-readable storage medium may store instructions, specifically, temporary and permanent copies of those instructions. These instructions may include, when executed by at least one unit in the processor, causing the SoC 500 to perform the attention training method according to the above embodiments, as detailed in the methods described above, which will not be repeated here.
[0092] Various embodiments of the mechanisms disclosed in this application can be implemented in hardware, software, firmware, or combinations of these implementation methods. Embodiments of this application can be implemented as computer programs or program code executable on a programmable system, the programmable system including at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.
[0093] Program code can be applied to input instructions to execute the functions described in this application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, the processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application-specific integrated circuit (ASIC), or a microprocessor.
[0094] The program code can be implemented using a high-level procedural language or an object-oriented programming language to communicate with the processing system. Assembly language or machine language can also be used when needed. In fact, the mechanisms described in this application are not limited to any particular programming language. In either case, the language can be a compiled language or an interpreted language.
[0095] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried or stored thereon on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, the instructions may be distributed via a network or through other computer-readable media. Therefore, machine-readable media may include any mechanism for storing or transmitting information in a machine-readable (e.g., computer-readable) form, including but not limited to floppy disks, optical disks, CD-ROMs, compact disc read-only memory (CD-ROMs), magneto-optical disks, read-only memory (ROM), random-access memory (RAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic cards or optical cards, flash memory, or tangible machine-readable storage for transmitting information (e.g., carrier waves, infrared signals, digital signals, etc.) using the Internet in the form of electrical, optical, acoustic, or other forms of propagated signals. Therefore, machine-readable media includes any type of machine-readable medium suitable for storing or transmitting electronic instructions or information in a machine-readable (e.g., computer-readable) form.
[0096] In the accompanying drawings, some structural or methodological features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order may not be necessary. Rather, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the accompanying drawings. Furthermore, including structural or methodological features in a particular figure does not imply that such features are required in all embodiments, and in some embodiments, these features may be omitted or may be combined with other features.
[0097] It should be noted that all units / modules mentioned in the device embodiments of this application are logical units / modules. Physically, a logical unit / module can be a physical unit / module, a part of a physical unit / module, or a combination of multiple physical units / modules. The physical implementation of these logical units / modules themselves is not the most important factor; the combination of functions implemented by these logical units / modules is the key to solving the technical problems proposed in this application. Furthermore, to highlight the innovative aspects of this application, the above-described device embodiments of this application have not introduced units / modules that are not closely related to solving the technical problems proposed in this application. This does not mean that the above-described device embodiments do not contain other units / modules.
[0098] It should be noted that in the examples and description of this patent, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0099] Although this application has been illustrated and described with reference to certain preferred embodiments thereof, those skilled in the art should understand that various changes in form and detail may be made thereto without departing from the spirit and scope of this application.
Claims
1. A test case orchestration method, characterized in that, include: Obtain user business requirements information; Using the first language model, based on the user business requirement information, the interface description text information output by the first language model is obtained. The interface description text information is used to describe multiple interfaces arranged in a first order. Obtain the correspondence between multiple interface test cases and the multiple interfaces, wherein each interface test case in the correspondence includes interface request information, and the interface request information is used to indicate the requested interface; Based on the correspondence and the plurality of interfaces arranged in the first order, interface business process information is determined, and the interface business process information is used to indicate the test cases of the plurality of interfaces arranged in the first order.
2. The method according to claim 1, characterized in that, The acquisition of user business requirement information includes: Obtain natural language text input by the user, the natural language text being used to describe multiple needs of the user; Based on the natural language text, the user business requirement information is determined, wherein the user business requirement information includes multiple execution process descriptions corresponding to the multiple requirements and a first interval identifier, the first interval identifier being used to separate every two adjacent execution process descriptions.
3. The method according to claim 2, characterized in that, The step of obtaining the interface description text information output by the first large language model based on the user business requirement information includes: Based on the first interval identifier in the user business requirement information, the user business requirement information is segmented into words to obtain the multiple execution process description information; Generate multiple vector feature representations that correspond one-to-one with the multiple execution process description information; Based on the plurality of vector feature representations, the interface description text information is determined, wherein the interface description text information includes the plurality of interfaces and a second interval identifier, the second interval identifier being used to separate every two adjacent interfaces.
4. The method according to claim 3, characterized in that, The step of determining the interface business process information based on the correspondence and the plurality of interfaces arranged in the first order includes: The interface description text information, which includes the multiple interfaces, is segmented and vectorized to obtain the first vectorized representation information corresponding to the interface description text information; The interface request information of the multiple interface test cases is segmented and vectorized to obtain the second vectorized representation information corresponding to the interface request information of the multiple interface test cases; The interface business process information is determined based on the Euclidean distance between the first vectorized representation information and the second vectorized representation information.
5. The method according to claim 4, characterized in that, The method further includes: The interface business process information is determined based on the Jaccard similarity between the first vectorized representation information and the second vectorized representation information.
6. The method according to claim 1 or 2, characterized in that, Before obtaining the interface description text information output by the first language model based on the user business requirement information, the method further includes: Obtain the second largest language model; The second largest language model is fine-tuned based on the large model fine-tuning algorithm to obtain the first largest language model.
7. The method according to claim 1 or 2, characterized in that, The method further includes: The API automated testing framework is invoked to present the business process corresponding to the API business process information.
8. An electronic device, characterized in that, It includes one or more processors; one or more memories; said one or more memories storing one or more computer programs, said one or more computer programs including instructions that, when executed by said one or more processors, cause the method of any one of claims 1 to 7 to be performed.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed on a computer, cause the method as described in any one of claims 1 to 7 to be performed.
10. A computer program product, characterized in that, When the computer program product is run on a computer, it causes the computer to perform the method as described in any one of claims 1 to 7.