Interface testing method and device, electronic equipment and storage medium

Abstract

The invention relates to the technical field of computers, in particular to an interface test method and device, electronic equipment and a storage medium, which are used for meeting increasing test requirements, improving the interface test coverage and improving the accuracy of test results. The method comprises the steps of generating a test case based on a test requirement and an interface test sample, and executing the test case to obtain a current execution result of the current time; based on the current execution result and an expected result, a current execution matching score is determined, the current execution matching score and the expected matching score are compared, an assertion result of the current execution result is determined, and the assertion result represents whether execution of the test case is passed or not when the test case is executed at the current time; and updating the expected matching score or updating the test case based on the assertion result of the current execution result and the assertion results of the plurality of historical execution results of the test case.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular, to an interface testing method, apparatus, electronic device, and storage medium. Background Art

[0002] In the process of software development, interface testing is an important part of ensuring software quality. It can help the development team discover and solve potential integration problems at an early stage, and ensure the normal interaction between different modules. Interface testing is a type of software testing, and its purpose is to replace the front-end page or third-party caller in the case where the front-end components are not integrated, so as to verify whether the interface parameter passing, function implementation, output result, and fault tolerance processing conform to the interface specification. This helps to improve the quality, stability, and reliability of the software, and reduce the later repair cost.

[0003] Currently, interface testing often relies on preset parameters and scenarios for interface testing, with insufficient testing depth, unable to meet the growing testing requirements, and the assertion ability is relatively single, resulting in incorrect results. Summary of the Invention

[0004] Embodiments of this application provide an interface testing method, apparatus, electronic device, and storage medium to meet the growing testing requirements and improve the accuracy of test results.

[0005] The specific technical solutions provided by the embodiments of this application are as follows:

[0006] In a first aspect, an interface testing method is provided. The method includes:

[0007] Generate test cases based on test requirements and interface test samples, and execute the test cases to obtain the current execution result of the current time;

[0008] Determine the current execution matching score based on the current execution result and the expected result, and compare the current execution matching score with the expected matching score to determine the assertion result of the current execution result. The assertion result represents whether the test case passes when executed at the current time;

[0009] Update the expected matching score or update the test case based on the assertion result of the current execution result and the assertion results of multiple historical execution results of the test case.

[0010] In a possible embodiment, generating test cases based on test requirements and interface test samples includes:

[0011] Generate target variable information based on the test requirements and the sample variable information of the interface test samples;

[0012] Assemble the target variable information and the sample quantitative information of the interface test samples to generate test cases.

[0013] By the above method, the test cases are split into quantitative and variable parts, and by changing the variable information, a large number of test cases that meet the requirements can be quickly generated, achieving the effect of fully covering the interface to be tested and also lowering the entry threshold for interface testing.

[0014] In a possible embodiment, the sample variable information includes: sample request parameters. Then, based on the test requirements and the sample variable information of the interface test cases, target variable information is generated, including:

[0015] Extract the parameter names and parameter types of the sample request parameters for semantic analysis to obtain the semantic analysis result of the sample request parameters;

[0016] According to the test requirements and the semantic analysis result of the sample request parameters, generate target request parameters to obtain target variable information.

[0017] By the above method, the target request parameters can be obtained quickly and accurately, providing accurate request parameters for continuous interface automation testing.

[0018] In a possible embodiment, before executing the test case to obtain the current execution result, it further includes:

[0019] Execute the test case for the first time to obtain the first execution result;

[0020] Perform a refinement process on the first execution result to obtain the expected result.

[0021] By the above method, by refining the first execution result, an accurate expected result can be obtained, avoiding misjudgment caused by text inconsistency but semantic similarity between the current execution result and the first execution result, so as to accurately determine the assertion result.

[0022] In a possible embodiment, comparing the current execution matching score and the expected matching score to determine the assertion result of the current execution result includes:

[0023] If the current execution matching score is greater than the expected matching score, determine that the assertion result of the current execution result is passed;

[0024] If the current execution matching score is less than or equal to the expected matching score, determine that the assertion result of the current execution result is not passed.

[0025] By the above method, based on the matching score, the assertion result can be determined quickly and accurately.

[0026] In a possible embodiment, based on the assertion result of the current execution result and the assertion results of multiple historical execution results of the test case, update the expected matching score or update the test case, including:

[0027] When the assertion result of the current execution result passes, calculate the first expected value of the execution matching scores corresponding to each passed execution result, and use the first expected value as the expected matching score, where each passed execution result is each execution result among the current execution result and multiple historical execution results whose assertion result passes.

[0028] When the assertion result of the current execution result fails, based on the first expected value and the current execution matching score, combine the gradient descent algorithm to determine the anomaly detection result of the test case, and when the anomaly detection result is that the test case is abnormal, update the test case.

[0029] Through the above method, during the process of repeatedly executing the test case, due to the evolution and iteration of the system, unknown errors may occur. Therefore, by automatically correcting the assertion content by the machine, the goal of self-optimizing the accuracy of the assertion result is achieved, the accuracy of the expected matching score is improved, thereby improving the accuracy of the assertion result, making up for the defect that the interface test result is inaccurate due to the single means of interface test assertion for the preset scenario being a fixed result, and making the interface test have higher accuracy and execution reliability. In addition, based on the execution matching scores of each failed execution result, combined with the gradient descent algorithm, judge the true applicability of the variables of the test case, and can accurately determine the abnormal test case.

[0030] In a possible embodiment, based on the first expected value and the current execution matching score, combine the gradient descent algorithm to determine the anomaly detection result of the test case, including:

[0031] Based on the first expected value and the current execution matching score, determine the absolute value of the gradient for the current time, and calculate the second expected value of the matching score loss value corresponding to each failed execution result as the target gradient value, where each failed execution result is each execution result among the current execution result and multiple historical execution results whose assertion result fails;

[0032] When the absolute value of the gradient is less than the target gradient value, determine that the anomaly detection result is that the test case is abnormal.

[0033] Through the above method, the abnormal test case can be accurately determined.

[0034] In a possible embodiment, determining to update the test case includes:

[0035] Discard the test case, and generate a new test case based on the test requirements and the interface test sample.

[0036] By the above method, the abnormal test cases that fail to pass are discarded, which reduces the waste of resources during the test run, increases the data depth of the interface test, automatically optimizes the abnormal test cases, fine-tunes the request parameters in the variables of the abnormal test cases, optimizes the test cases, and provides more accurate variable generation of test cases for continuous interface automation testing.

[0037] In a second aspect, the present application provides an interface test device, and the device includes:

[0038] An execution module, configured to generate test cases based on test requirements and interface test samples, execute the test cases, and obtain the current execution result of the current time;

[0039] An assertion module, configured to determine the current execution matching score based on the current execution result and the expected result, compare the current execution matching score with the expected matching score, and determine the assertion result of the current execution result, where the assertion result represents whether the test case passes when executed at the current time;

[0040] A feedback module, configured to update the expected matching score or update the test case based on the assertion result of the current execution result and the assertion results of multiple historical execution results of the test case.

[0041] In a possible embodiment, when generating test cases based on test requirements and interface test samples, the execution module is further configured to:

[0042] Generate target variable information based on the sample variable information of the test requirements and the interface test samples;

[0043] Assemble the target variable information and the sample quantitative information of the interface test samples to generate test cases.

[0044] In a possible embodiment, the sample variable information includes sample request parameters. When generating target variable information based on the sample variable information of the test requirements and the interface test samples, the execution module is further configured to:

[0045] Extract the parameter names and parameter types of the sample request parameters for semantic analysis to obtain the semantic analysis result of the sample request parameters;

[0046] Generate target request parameters according to the test requirements and the semantic analysis result of the sample request parameters to obtain target variable information.

[0047] In a possible embodiment, before executing the test case to obtain the current execution result, the execution module is further configured to:

[0048] Execute the test case for the first time to obtain the first execution result;

[0049] Refine the first execution result to obtain the expected result.

[0050] In a possible embodiment, when comparing the current execution matching score and the expected matching score to determine the assertion result of the current execution result, the assertion module is further configured to:

[0051] If the current execution matching score is greater than the expected matching score, determine that the assertion result of the current execution result is passed.

[0052] If the current execution matching score is less than or equal to the expected matching score, determine that the assertion result of the current execution result is not passed.

[0053] In a possible embodiment, when updating the expected matching score or determining to update the test case based on the assertion result of the current execution result and the assertion results of multiple historical execution results of the test case, the feedback module is further configured to:

[0054] When the assertion result of the current execution result is passed, calculate the first expected value of the execution matching scores corresponding to each passed execution result, and use the first expected value as the expected matching score, where each passed execution result is each execution result among the current execution result and multiple historical execution results whose assertion result is passed.

[0055] When the assertion result of the current execution result is not passed, based on the first expected value and the current execution matching score, combine the gradient descent algorithm to determine the anomaly detection result of the test case, and when the anomaly detection result is that the test case is abnormal, update the test case.

[0056] In a possible embodiment, when combining the gradient descent algorithm to determine the anomaly detection result of the test case based on the first expected value and the current execution matching score, the feedback module is further configured to:

[0057] Based on the first expected value and the current execution matching score, determine the absolute value of the gradient for the current time, and calculate the second expected value of the matching score loss values corresponding to each non-passed execution result as the target gradient value, where each non-passed execution result is each execution result among the current execution result and multiple historical execution results whose assertion result is not passed;

[0058] When the absolute value of the gradient is less than the target gradient value, determine that the anomaly detection result is that the test case is abnormal.

[0059] In a possible embodiment, when updating the test case, the feedback module is further configured to:

[0060] Discard the test case, and generate a new test case based on the test requirements and the interface test samples.

[0061] In a third aspect, the present application provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the steps of the method described in any one of the first aspects above are implemented.

[0062] In a fourth aspect, the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described in any one of the first aspects above are implemented.

[0063] In a fifth aspect, the present application provides a computer program product, which includes: computer program code. When the computer program code runs on a computer, the computer is caused to execute the method described in any one of the first aspects.

[0064] In the embodiments of the present application, first, based on test requirements and interface test samples, test cases are generated and executed, and the current execution result of the current time is obtained. Then, based on the current execution result and the expected result, the current execution matching score is determined, and the current execution matching score and the expected matching score are compared to determine the assertion result of the current execution result. The assertion result indicates whether the execution of the test case passes during the current execution. Finally, based on the assertion result of the current execution result and the assertion results of multiple historical execution results of the test case, the expected matching score is updated or the test case is updated. In this way, only by setting test requirements and interface test samples, a large number of test cases can be generated, increasing the depth of interface test data, improving the interface test coverage, meeting the growing test requirements, and providing a feedback mechanism. Based on the assertion result of the current execution result and the assertion results of multiple historical execution results of the test case, the expected matching score is updated or the abnormal detection result of the test case is determined. Without manual supervision, abnormal test cases can be detected in a timely manner, the expected matching score can be updated in a timely manner, the maintenance workload of traditional interface testing tools can be reduced, and the wrong results caused by the inflexibility of preset assertions in interface testing can be avoided, improving the accuracy of test results. Description of the Drawings

[0065] Figure 1 is the system architecture diagram in the embodiments of the present application;

[0066] Figure 2 is the implementation flowchart of an interface testing method provided in the embodiments of the present application;

[0067] Figure 3 is the schematic diagram of the test data matrix in the embodiments of the present application;

[0068] Figure 4 is the schematic diagram of generating target request parameters in the embodiments of the present application;

[0069] Figure 5 Schematic diagram for determining assertion results in an embodiment of this application;

[0070] Figure 6 Schematic diagram for determining semantic matching scores in an embodiment of this application;

[0071] Figure 7 Schematic diagram of correction feedback in an embodiment of this application;

[0072] Figure 8 Schematic diagram of the workflow of an interface testing method in an embodiment of this application;

[0073] Figure 9 Schematic diagram of the working logic of an interface testing method in an embodiment of this application;

[0074] Figure 10 Schematic diagram of the structure of an interface testing device provided in an embodiment of this application;

[0075] Figure 11 Schematic diagram of the structure of an electronic device in an embodiment of this application. Detailed implementation manners

[0076] To make the objectives, technical solutions and advantages of this application clearer and more understandable, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of this application. Apparently, the described embodiments are only some of the embodiments of this application, rather than all of them. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of this application without creative efforts shall fall within the protection scope of this application. Without conflict, the embodiments in this application and the features in the embodiments may be combined with each other arbitrarily. And although the logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than here.

[0077] The terms "first" and "second" in the specification, claims and above-mentioned drawings of this application are used to distinguish different objects, rather than to describe a specific order. In addition, the term "comprising" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally further include steps or units not listed, or may optionally further include other steps or units inherent to these processes, methods, products or devices. "Multiple" in this application may mean at least two, for example, it may be two, three or more, and the embodiments of this application do not make limitations.

[0078] The following describes exemplary embodiments of the present application with reference to the accompanying drawings. Various details of the embodiments of the present application are included to facilitate understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of the disclosure of the present application. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following. It should be noted that in the embodiments of the present application, certain existing industry solutions such as software, components, models, etc. may be mentioned, and they should be considered exemplary. Their purpose is only to illustrate the feasibility in the implementation of the technical solution of the present application, but it does not mean that the applicant has already or necessarily used this solution.

[0079] In the technical solution of the present application, the acquisition, transmission, storage, use, etc. of data all comply with the requirements of relevant national laws and regulations.

[0080] The following briefly introduces the design concept of the embodiments of the present application:

[0081] In the process of software development, interface testing is an important part of ensuring software quality. It can help the development team discover and solve potential integration problems early, and ensure the normal interaction between different modules. Interface testing is a type of software testing. Its purpose is to replace the front-end page or third-party caller to verify whether the interface parameter passing, function implementation, output result, and fault tolerance processing conform to the interface specification when the front-end components are not integrated. This helps to improve the quality, stability, and reliability of the software, and reduce the later repair cost.

[0082] Regarding the current emphasis on interface testing in the testing industry, most current interface tests rely on preset parameters and scenarios, with insufficient testing depth, unable to meet the growing testing needs. Moreover, the assertions are not flexible enough, and false positive results are likely to occur during the process, leading to incorrect results. At the same time, manual assertion setting is required, resulting in a large amount of work for interface testing and tool maintenance. Coupled with the rise of exploratory testing and special security interface testing, the development and improvement of software quality are more inclined to diversification and the goal of rich testing means. The traditional interface testing solution is no longer applicable to the rapidly developing software volume and the increasingly large testing needs, and the existing testing capabilities will also not meet the future sustainable testing concept.

[0083] In view of this, in the embodiments of the present application, an interface testing method, device, electronic device, and storage medium are provided. Based on test requirements and interface test samples, test cases are automatically generated, the test cases are executed, and the current execution result of the current execution is obtained. Then, based on the current execution result and the expected result, the current execution matching score is determined, and the current execution matching score and the expected matching score are compared to determine the assertion result of the current execution result. The assertion result indicates whether the test case passes when executed in the current execution. Finally, based on the assertion result of the current execution result and the assertion results of multiple historical execution results of the test case, the expected matching score is updated, or the test case is updated based on the assertion result of the current execution result and the assertion results of multiple historical execution results of the test case. In this way, only by setting test requirements and interface test samples, a large number of test cases can be generated, the depth of interface test data can be increased, the interface test coverage can be improved, the growing test requirements can be met, and a feedback mechanism is provided. Based on the assertion result of the current execution result and the assertion results of multiple historical execution results of the test case, the expected matching score is updated or the abnormal detection result of the test case is determined. No manual supervision is required, abnormal test cases can be detected in a timely manner, the maintenance workload of traditional interface testing tools can be reduced, and the error results caused by the inflexibility of preset assertions in interface testing can be avoided, improving the accuracy of test results.

[0084] The preferred embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0085] Refer to Figure 1 As shown, it is the system architecture diagram in the embodiments of the present application. It includes: an interface execution engine 110, a language model analysis engine 120, and a result feedback module 130.

[0086] The interface execution engine 110 is used to execute test cases and return the execution results of the test cases to the language model analysis engine 120. It mainly includes: data normalization, execution unit, result normalization, and use case set screening.

[0087] The language model analysis engine 120 is used to generate test cases and transmit them to the interface execution engine 110, receive the execution results of the test cases returned by the interface execution engine 110, and determine the assertion result of the execution result and transmit it to the result feedback module 130 according to the expected result and the expected matching score. It mainly includes: semantic analysis, comparison and matching, matching accuracy analysis, and accuracy improvement.

[0088] The result feedback module 130 is used to receive the assertion results transmitted by the language model analysis engine 120, determine a new expected matching score or identify abnormal test cases based on the assertion results, and transmit the new expected matching score to the language model analysis engine 120 so that the language model analysis engine 120 can update the expected matching score, and transmit the abnormal test cases to the interface execution engine 110 so that the interface execution engine 110 discards the abnormal test cases. It mainly includes: result display and data analysis.

[0089] Refer to Figure 2 As shown, it is a flowchart of the implementation of an interface testing method provided by an embodiment of the present application. The specific implementation process of this method is as follows:

[0090] Step 20: Generate test cases based on test requirements and interface test samples, execute the test cases, and obtain the current execution result of the current time.

[0091] In the embodiment of the present application, a large number of test cases that meet the test requirements are generated based on the test requirements and interface test samples, and the test cases are executed in a real test environment. After the execution is completed, the current execution result obtained is incorporated into the test data matrix. The test data matrix containing the execution results will be assembled into data groups in units of test cases for transmission.

[0092] Among them, the test requirements include boundary testing, equivalence testing, security testing, etc.; the interface test samples are pre-entered. The interface test samples include: sample quantitative information and sample variable information. The sample quantitative information is the path information and purpose of the interface, and the sample variable information is the request parameters of the interface, the request method, the parameter form, and whether to use certificate encryption for the request.

[0093] In addition, it is worth noting that the test data matrix is a virtual concept, and its goal is to uniformly store various types of data.

[0094] Exemplarily, refer to Figure 3 As shown, it is a schematic diagram of the test data matrix in the embodiment of the present application. One data group in the test data matrix corresponds to one test case. Before the test case is executed, the data group includes the attribute information of the test case (such as: interface request method, interface request path, and interface request parameters). After the test case is executed, the interface test execution environment and execution result will be incorporated into the data group. After the assertion result of the execution result is determined, the assertion result will be incorporated into the data group.

[0095] Optionally, in the embodiment of the present application, a possible embodiment is provided for generating test cases based on test requirements and interface test samples, and the following specific operations are performed:

[0096] Step 200: Generate target variable information based on the test requirements and the sample variable information of the interface test cases.

[0097] In the embodiments of the present application, the sample variable information of the interface test cases is parsed, and the sample variable information is normalized and processed as a data group and stored in the test data matrix. The test data matrix is updated with data to generate new variable information, that is, target variable information. If the test data matrix is updated with data multiple times, multiple new variable information will be generated to obtain multiple target variable information. Specifically, the sample request parameters in the sample variable information are iterated with data based on the data iterator of the large model to generate target request parameters and store them in the test data matrix, and other sub-information in the sample variable information is inferred to obtain target other sub-information and store it in the test data matrix.

[0098] Optionally, in the embodiments of the present application, a possible embodiment is provided for generating target variable information based on the test requirements and the sample variable information of the interface test cases, and the following operations are specifically performed:

[0099] Step 2000: Extract the parameter names and parameter types of the sample request parameters for semantic analysis to obtain the semantic analysis results of the sample request parameters.

[0100] Exemplarily, the sample request parameter is {"name": "zhangsan"}. The data iterator of the large model extracts the parameter name of this sample request parameter as "name" and the parameter type as text, and then performs semantic analysis to obtain the semantic analysis result of this sample request parameter: the value of the parameter name "name" in this sample request parameter is of text type, and the example length is 8.

[0101] In addition, it is worth noting that the data iterator in the embodiments of the present application can also be a data iterator based on a model other than the large model, and the embodiments of the present application do not limit this.

[0102] Step 2001: Generate target request parameters according to the test requirements and the semantic analysis results of the sample request parameters to obtain target variable information.

[0103] In the embodiments of the present application, data iteration generation is performed according to the test requirements and the semantic analysis results of the sample request parameters to generate target request parameters, and target variable information is obtained based on the target request parameters and target other sub-information.

[0104] In addition, it is worth noting that the data iterator in the embodiments of the present application will generate a large number of target request parameters according to the sample request parameters, thereby generating a large number of test cases to achieve the effect of covering all interfaces to be tested.

[0105] Exemplarily, if the test requirement is boundary testing, boundary variables are generated; if the test requirement is security testing, variables related to database injection or other security injections are generated.

[0106] Exemplarily, referring to Figure 4 As shown, it is a schematic diagram of generating target request parameters in an embodiment of the present application. The data iterator of the large model performs semantic parsing on the parameter names and parameter types of the sample request parameters, and parses the test requirements to obtain the target request parameters corresponding to the test requirements.

[0107] In this way, by using the semantic analysis function of the large model, the target request parameters can be obtained quickly and accurately, providing accurate request parameters for continuous interface automation testing.

[0108] Step 201: Assemble the target variable information and the sample quantitative information of the interface test case to generate a test case.

[0109] In an embodiment of the present application, the data assembly module reorganizes the data groups in the test data matrix, parses the content of the test data matrix, and assembles the target variable information and the sample quantitative information of the interface test case in the test data matrix to obtain an executable test case.

[0110] Briefly speaking, the test case is obtained from the interface test case. Since the interface test case contains sample variable information, when the sample variable information is updated, a new interface test case is obtained, and thus a new test case is obtained. Therefore, by changing the sample variable information in the interface test case, a large number of test cases that meet the requirements can be generated quickly, achieving the effect of covering all interfaces to be tested, reducing the operation difficulty of interface testing, improving the efficiency of interface testing, and also lowering the entry threshold of interface testing.

[0111] Step 21: Based on the current execution result and the expected result, determine the current execution matching score, and compare the current execution matching score with the expected matching score to determine the assertion result of the current execution result.

[0112] Among them, the assertion result represents whether the execution passes when the test case is executed this time.

[0113] In an embodiment of the present application, the assertion tool of the large model extracts the current execution result from the test data matrix, calculates the matching degree between the current execution result and the expected result to obtain the current execution matching score, then compares the current execution matching score with the expected matching score to determine the assertion result of the current execution result, and stores the assertion result and the current execution matching score in the test data matrix. Finally, a test report is generated through statistics and the test report is reported.

[0114] In addition, it is worth noting that the assertion tool in the embodiments of the present application may also be an assertion tool based on a model other than the large model, and the embodiments of the present application do not limit this.

[0115] Exemplarily, participate Figure 5 As shown, it is a schematic diagram of determining the assertion result in the embodiments of the present application. The assertion tool based on the large model calculates the semantic matching score, type matching score, numerical matching score, and inclusion relationship matching score between the current execution result and the expected result, so as to obtain the current execution matching score, and then compares the numerical size of the current execution matching score with the expected matching score to determine the assertion result of the current execution result.

[0116] Exemplarily, participate Figure 6 As shown, it is a schematic diagram of determining the semantic matching score in the embodiments of the present application. The response data keys and values of the current execution result content are semantically parsed, the content of the expected result is semantically parsed, and then the semantic matching degree between the current execution result and the expected result is calculated to obtain the semantic matching score.

[0117] In this way, the assertion tool based on the model calculates the matching degree, and can accurately determine the assertion result.

[0118] Optionally, in the embodiments of the present application, when determining the assertion result of the current execution result, the following operations are specifically performed:

[0119] Step 210: Determine whether the current execution matching score is greater than the expected matching score. If so, execute step 211; otherwise, execute step 212.

[0120] Step 211: Determine that the assertion result of the current execution result is passed, that is, the current execution of the test case passes.

[0121] For example, assume that the current execution matching score is 0.7 and the expected matching score is 0.6. Then the current execution matching score 0.7 is greater than the expected matching score 0.6, and the current execution of the test case passes.

[0122] Step 212: Determine that the assertion result of the current execution result is not passed, that is, the current execution of the test case does not pass.

[0123] For example, assume that the current execution matching score is 0.3 and the expected matching score is 0.6. Then the current execution matching score 0.3 is less than the expected matching score 0.6, and the current execution of the test case does not pass.

[0124] In this way, the assertion result can be determined quickly and accurately.

[0125] In addition, it is worth noting that in the embodiments of the present application, the expected result and the expected matching score can be obtained based on the first execution result of the test case, or can be manually written, where the expected matching score can also be optimized based on the historical execution results.

[0126] Specifically, in the embodiments of the present application, a possible embodiment is provided to obtain the expected result and the expected matching score, and the following operations are specifically performed:

[0127] Step A1: First execute the test case to obtain the first execution result.

[0128] In the embodiments of the present application, after the test case is first generated, the test case will be automatically executed in the real test environment to obtain the first execution result.

[0129] Step A2: Refine the first execution result to obtain the expected result.

[0130] In the embodiments of the present application, the first execution result is formatted, and semantic analysis is performed on each item in the execution result. The type, structure, and text information semantics of the result item are refined, and unnecessary modal particles and useless information are removed to obtain the expected result.

[0131] Step A3: Obtain the expected matching score based on the similarity of the feature vectors of the expected result and the first execution result.

[0132] In the embodiments of the present application, the similarity between the feature vector of the expected result and the feature vector of the first execution result is calculated to obtain the expected matching score.

[0133] In this way, by refining the first execution result, an accurate expected result can be obtained, avoiding misjudgment caused by text inconsistency but semantic similarity between the current execution result and the first execution result, so that the assertion result can be accurately determined.

[0134] Step 22: Update the expected matching score or update the test case based on the assertion result of the current execution result and the assertion results of multiple historical execution results of the test case.

[0135] In the embodiments of the present application, based on the assertion result and the current execution matching score of the current execution result, and the assertion results and historical execution matching scores of multiple historical execution results of the test case, feedback correction is performed according to the assertion result of the current execution result. The feedback correction includes two correction situations: updating the expected matching score and updating the test case.

[0136] In addition, it is worth noting that in the embodiments of the present application, the multiple historical execution results may be all the historical execution results obtained by executing the test case in the real test environment, or the previous consecutive m historical execution results of the current execution, where m is an integer greater than 1.

[0137] Correction case 1: When the assertion result of the current execution result is passed, update the expected matching score.

[0138] In the embodiments of the present application, when the assertion result of the current execution result is passed, calculate the first expected value of the execution matching scores corresponding to each passed execution result, and use the first expected value as the expected matching score.

[0139] Among them, each passed execution result is each execution result with the assertion result of passed among the current execution result and the multiple historical execution results.

[0140] In the embodiments of the present application, the first expected value can be expressed as: n is the total number of all passed execution results, x n is the execution matching score of the nth passed execution result, p n is the probability of the occurrence of x n this value.

[0141] In this way, during the process of repeatedly executing the test case, due to the evolution and iteration of the system, unknown errors may occur. Therefore, by automatically correcting the assertion content by the machine, the goal of self-optimizing the accuracy of the assertion result is achieved, the accuracy of the expected matching score is improved, thereby improving the accuracy of the assertion result, and making up for the defect that the interface test result is inaccurate due to the single fixed result of the interface test assertion method in the preset scenario, making the interface test have higher accuracy and execution reliability.

[0142] In addition, it is worth noting that the execution result and the execution matching score are usually stored in the form of key-value pairs for easy data analysis.

[0143] Correction case 2: When the assertion result of the current execution result is not passed, update the test case.

[0144] In the embodiments of the present application, when the assertion result of the current execution result is not passed, based on the first expected value and the current execution matching score, combined with the gradient descent algorithm, determine the anomaly detection result of the test case, and when the anomaly detection result is that the test case is abnormal, update the test case.

[0145] Optionally, in the embodiments of the present application, a possible embodiment is provided for determining the anomaly detection result of the test case based on the first expected value and the current execution matching score, and specifically perform the following operations:

[0146] Step B1: Based on the first expected value and the current execution matching score, determine the absolute value of the gradient for the current time, and calculate the second expected value of the matching score loss value corresponding to each non-passing execution result as the target gradient value.

[0147] Among them, the non-passing execution results are: among the current execution result and multiple historical execution results, the execution results with the assertion result of non-passing.

[0148] In the embodiment of the present application, based on the first expected value, the current execution matching score, and the gradient of the loss function, determine the absolute value of the gradient for the current time.

[0149] In the embodiment of the present application, the loss function can be expressed as: L(θ) = (P(θ) - E(x)) 2 .

[0150] Among them, P(θ) is the execution matching score of the non-passing execution result, and E(x) is the expected value of the execution matching score corresponding to each passing execution result. Based on this, the matching score loss value for the current time = (P(θ k ) - E1(x)) 2 , P(θ k ) is the current execution matching score for the current time, and E1(x) is the first expected value.

[0151] In the embodiment of the present application, the gradient of the loss function can be expressed as:

[0152]

[0153] Among them, P(θ) is the execution matching score of the non-passing execution result, and E(x) is the expected value of the execution matching score corresponding to each passing execution result. The gradient of the loss function can obtain the change rate of the error between P(θ) and E(x). Based on this, P(θ k ) is the current execution matching score for the current time, and E1(x) is the first expected value.

[0154] In the embodiment of the present application, by calculating the gradient of the parameter θ after the k-th iteration calculation to determine the trend of the execution result and the execution matching score, the rule of gradient descent is: Among them, θ k is the parameter value, and α is the learning rate.

[0155] In the embodiment of the present application, the second expected value can be expressed as:

[0156]

[0157] Among them, k is the total number of non-passing execution results, L(θ k) is the matching score loss value of the k-th non-passing execution result, p k is the probability of the occurrence of L(θ k ) this value.

[0158] Step B2: When the absolute value of the gradient is less than the target gradient value, determine that the anomaly detection result is a test case anomaly.

[0159] In the embodiment of the present application, the absolute value of the gradient is compared with the target gradient value. If the absolute value of the gradient is less than the target gradient value, it is determined that the anomaly detection result is a test case anomaly, that is, the test case is an abnormal test case.

[0160] In this way, based on the execution matching scores of each non-passing execution result and combined with the gradient descent algorithm, the true applicability of the variables of the test case can be judged, and the abnormal test case can be accurately determined.

[0161] Further, in the embodiment of the present application, after obtaining the anomaly detection result of the test case, when the anomaly detection result is a test case anomaly, the test case is discarded, and based on the test requirements and interface test samples, a new test case is generated, that is, the request parameters in the abnormal test case with execution failure are discarded and re-assigned to obtain a new test case.

[0162] In this way, discarding the abnormal test cases with non-passing execution reduces the waste of resources during the test run, increases the data depth of the interface test, automatically optimizes the abnormal test cases, fine-tunes the request parameters in the variables of the abnormal test cases, optimizes the test cases, and provides more accurate variable generation of test cases for continuous interface automation testing.

[0163] Exemplarily, referring to Figure 7 As shown, it is a schematic diagram of correction feedback in the embodiment of the present application. First, after the test task is executed, automatic assertion is performed, then the assertion result is analyzed, and then correction feedback is performed, feeding back the assertion information (i.e., the new expected matching score) and the use case data (i.e., the abnormal test case), and combining the research and judgment result (i.e., manual research and judgment) to update the test task, and finally execute the updated test task.

[0164] Based on the above embodiment, referring to Figure 8 As shown, it is a schematic diagram of the working process of an interface test method in the embodiment of the present application, including a script information module, an execution module, and a large model module, for testing interface A.

[0165] Based on the above embodiment, referring to Figure 9 As shown, it is a schematic diagram of the working logic of an interface test method in the embodiment of the present application. The specific implementation process of this method is as follows:

[0166] Step 90: Enter interface information to form an interface sample.

[0167] Step 91: Parse the sample quantitative information and sample variable information of the interface sample, and normalize the foregoing information.

[0168] Step 92: The large model data iterator generates data iteration for the normalized result of the request parameters, and generates target variable information according to the current test requirements.

[0169] Data assembly module, which is used to organize the data groups generated by the data iterator, and assemble the target variable information and sample quantitative information into test cases.

[0170] Step 94: The executor applies the test cases assembled by the data assembly module in a real test environment for execution and obtains the execution result.

[0171] Step 95: The large model matching assertion is used to compare the execution result obtained by the executor with the expected result, extract the key assertion items in the interface test result, and calculate the current execution matching score.

[0172] Use case analysis module, which is used to record and analyze the assertion result of the large model matching assertion, and feedback it to the data iterator and the large model matching assertion.

[0173] Based on the same inventive concept, an embodiment of the present application provides an interface test device. Please refer to Figure 10 , and the device includes:

[0174] Execution module 1001, which is used to generate test cases based on test requirements and interface test samples, execute the test cases, and obtain the current execution result of the current time;

[0175] Assertion module 1002, which is used to determine the current execution matching score based on the current execution result and the expected result, compare the current execution matching score with the expected matching score, and determine the assertion result of the current execution result. The assertion result represents whether the test case passes when executed at the current time;

[0176] Feedback module 1003, which is used to update the expected matching score or update the test cases based on the assertion result of the current execution result and the assertion results of multiple historical execution results of the test cases.

[0177] In a possible embodiment, when generating test cases based on test requirements and interface test samples, the execution module 1001 is further used to:

[0178] Generate target variable information based on the test requirements and the sample variable information of the interface test sample;

[0179] Assemble the target variable information and the sample quantitative information of the interface test case to generate a test case.

[0180] In a possible embodiment, when the sample variable information includes: sample request parameters, and the execution module 1001 is further configured to generate target variable information based on the test requirements and the sample variable information of the interface test case:

[0181] Extract the parameter names and parameter types of the sample request parameters for semantic analysis to obtain the semantic analysis result of the sample request parameters;

[0182] Generate target request parameters according to the test requirements and the semantic analysis result of the sample request parameters, and obtain the target variable information.

[0183] In a possible embodiment, before executing the test case to obtain the current execution result, the execution module 1001 is further configured to:

[0184] Execute the test case for the first time to obtain the first execution result;

[0185] Perform a refinement process on the first execution result to obtain the expected result.

[0186] In a possible embodiment, when comparing the current execution matching score and the expected matching score to determine the assertion result of the current execution result, the assertion module 1002 is further configured to:

[0187] If the current execution matching score is greater than the expected matching score, determine that the assertion result of the current execution result is passed;

[0188] If the current execution matching score is less than or equal to the expected matching score, determine that the assertion result of the current execution result is not passed.

[0189] In a possible embodiment, when updating the expected matching score or determining to update the test case based on the assertion result of the current execution result and the assertion results of multiple historical execution results of the test case, the feedback module 1003 is further configured to:

[0190] When the assertion result of the current execution result is passed, calculate the first expected value of the execution matching scores corresponding to each passed execution result, and use the first expected value as the expected matching score, where each passed execution result is each execution result among the current execution result and multiple historical execution results whose assertion result is passed.

[0191] When the assertion result of the current execution result is not passed, based on the first expected value and the current execution matching score, combine the gradient descent algorithm to determine the anomaly detection result of the test case, and update the test case when the anomaly detection result is that the test case is abnormal.

[0192] In a possible embodiment, when determining the anomaly detection result of a test case based on a first expected value and a current execution matching score and in combination with the gradient descent algorithm, the feedback module 1003 is further configured to:

[0193] Based on the first expected value and the current execution matching score, determine the absolute value of the gradient for the current time, and calculate a second expected value of the matching score loss value corresponding to each non-pass execution result as the target gradient value, where each non-pass execution result is: among the current execution result and multiple historical execution results, each execution result with an assertion result of non-pass.

[0194] When the absolute value of the gradient is less than the target gradient value, determine that the anomaly detection result is that the test case is abnormal.

[0195] In a possible embodiment, when updating a test case, the feedback module 1003 is further configured to:

[0196] When the anomaly detection result is that the test case is abnormal, discard the test case, and generate a new test case based on the test requirements and interface test samples.

[0197] Based on the above embodiments, refer to Figure 11 The following is a schematic structural diagram of an electronic device in an embodiment of the present application.

[0198] An embodiment of the present application provides an electronic device, which may include a processor 1110 (Center Processing Unit, CPU), a memory 1120, an input device 1130, an output device 1140, etc. The input device 1130 may include a keyboard, a mouse, a touch screen, etc., and the output device 1140 may include a display device, such as a liquid crystal display (Liquid Crystal Display, LCD), a cathode ray tube (Cathode Ray Tube, CRT), etc.

[0199] The memory 1120 may include a read-only memory (ROM) and a random access memory (RAM), and provide program instructions and data stored in the memory 1120 to the processor 1110. In an embodiment of the present application, the memory 1120 may be used to store a program of any interface test method in an embodiment of the present application.

[0200] By invoking the program instructions stored in the memory 1120, the processor 1110 is configured to execute any interface test method in an embodiment of the present application according to the obtained program instructions.

[0201] Based on the above embodiments, in an embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the interface test method in any method embodiment above is implemented.

[0202] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0203] The present application is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0204] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including instruction means that implement the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0205] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0206] Obviously, those skilled in the art can make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application is also intended to include these modifications and variations.

Claims

1. An interface testing method, characterized in that, Including: Generating test cases based on test requirements and interface test cases, and executing the test cases to obtain the current execution result of the current time; Determining the current execution matching score based on the current execution result and the expected result, comparing the current execution matching score with the expected matching score, and determining the assertion result of the current execution result, where the assertion result represents whether the test case passes when executed at the current time; Updating the expected matching score or updating the test case based on the assertion result of the current execution result and the assertion results of multiple historical execution results of the test case.

2. The method according to claim 1, wherein, The generating test cases based on test requirements and interface test cases includes: Generating target variable information based on the test requirements and the sample variable information of the interface test cases; Assembling the target variable information and the sample quantitative information of the interface test cases to generate the test cases.

3. The method according to claim 2, characterized in that, If the sample variable information includes sample request parameters, then the generating target variable information based on the test requirements and the sample variable information of the interface test cases includes: Extracting the parameter names and parameter types of the sample request parameters for semantic analysis to obtain the semantic analysis result of the sample request parameters; Generating target request parameters according to the test requirements and the semantic analysis result of the sample request parameters to obtain the target variable information.

4. The method according to claim 1, characterized in that, Before the executing the test cases to obtain the current execution result, it further includes: Executing the test cases for the first time to obtain the first execution result; Performing a reduction process on the first execution result to obtain the expected result.

5. The method according to claim 1, wherein The comparing the current execution matching score with the expected matching score and determining the assertion result of the current execution result includes: If the current execution matching score is greater than the expected matching score, determining that the assertion result of the current execution result is passing; If the current execution matching score is less than or equal to the expected matching score, determining that the assertion result of the current execution result is not passing.

6. The method according to claim 1, wherein The updating the expected matching score or updating the test case based on the assertion result of the current execution result and the assertion results of multiple historical execution results of the test case includes: When the assertion result of the current execution result is passing, calculating the first expected value of the execution matching scores corresponding to the passing execution results, and using the first expected value as the expected matching score, where the passing execution results are the execution results with the assertion result of passing among the current execution result and the multiple historical execution results; When the assertion result of the current execution result is not passing, determining the anomaly detection result of the test case based on the first expected value and the current execution matching score in combination with the gradient descent algorithm, and updating the test case when the anomaly detection result is that the test case is abnormal.

7. The method according to claim 6, characterized in that, The determining the anomaly detection result of the test case based on the first expected value and the current execution matching score in combination with the gradient descent algorithm includes: Based on the first expected value and the current execution matching score, determine the absolute value of the gradient for the current time, and calculate the second expected value of the matching score loss value corresponding to each non-passing execution result as the target gradient value, where the non-passing execution results are: among the current execution result and the multiple historical execution results, the execution results with an assertion result of non-passing; When the absolute value of the gradient is less than the target gradient value, determine that the anomaly detection result is that the test case is anomalous.

8. The method according to claim 6, wherein The updating of the test case includes: Discard the test case, and generate a new test case based on the test requirement and the interface test sample.

9. An interface testing device, characterized in that, including: An execution module, configured to generate a test case based on a test requirement and an interface test sample, and execute the test case to obtain the current execution result of the current time; An assertion module, configured to determine the current execution matching score based on the current execution result and the expected result, compare the current execution matching score with the expected matching score, and determine the assertion result of the current execution result, where the assertion result represents whether the test case passes when executed at the current time; A feedback module, configured to update the expected matching score or update the test case based on the assertion result of the current execution result and the assertion results of the multiple historical execution results of the test case.

10. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

Images