Automatic testing method for RPC API of large industrial system

Abstract

The invention discloses an automatic testing method oriented to a large industrial system RPC API (Remote Procedure Call Application Program Interface). According to the method, API documents and historical calling data of tenants are used as input, and automatic API testing is achieved. The method mainly comprises three stages of API dependency model construction, API test sequence generation and API sequence parameter rendering. The construction of the API dependency model is realized based on an API document and historical calling data of tenants, and comprises three steps: static analysis of the API document, parameter clustering based on a large model, and dependency relationship filtering based on the historical calling data of the tenants. The generation of the API test sequence is based on an API dependency graph generated in the previous stage, a to-be-tested API is taken as a starting point, a front dependency API is searched for continuously in an iteration mode to generate the sequence, and finally filtering is carried out through CRUD semantics. The rendering of the API sequence parameters is realized by constructing a dynamic dictionary and recommending large model parameters aiming at the API sequence generated in the previous stage.

Description

Technical Field

[0001] The present invention relates to the field of software testing, and in particular to an automated testing method for RPC API of a large industrial system. Background Art

[0002] The two mainstream Open API design styles on the market are REST and RPC. There have been related studies on REST API testing since it was proposed in 2000, but the number has always been small and the results are average. However, since the launch of EvoMaster and RESTler, the research on REST API has increased. However, there is still very little research on RPC API. The following will introduce the corresponding tools related to the present invention.

[0003] As a white-box testing tool, EvoMaster accesses the source code of the API and uses a genetic algorithm to dynamically analyze the Java bytecode to generate test cases. Under the action of the heuristic function, it continuously guides the generation tendency of test cases to improve code coverage. It was evaluated as the best tool in the review.

[0004] RESTler is the first stateful black-box fuzz testing tool for REST APIs. Stateful means that it can test certain states at a deeper level of the tool. RESTler analyzes API documentation, infers the producer-consumer dependency between APIs, and combines real-time dynamic feedback during test execution to update the request sequence and record abnormal situations to complete the test. RESTler and EvoMaster introduced testing of REST APIs in the fields of black-box and white-box testing, respectively, and more research work has emerged since then.

[0005] Different from RESTler's model-free random fuzz testing, RestTestGen attempts to solve the problem of insufficient modeling of API dependencies by establishing ODG (Operation Dependency Graph) to model API dependencies. During testing, the leaf nodes in ODG are called one by one and filtered according to semantics. For data variable matching and numerical rendering, the designed data name matching fault tolerance mechanism is also used to achieve this, in an attempt to solve the problem of uneven quality of API documentation.

[0006] Compared with RESTler and RestTestGen, which focus on the dependency issues between multiple APIs, RESTest focuses on the dependency issues between API parameters and supports automatic analysis of the dependency relationship between parameters. The dependency between parameters requires the use of the inter-parameter dependency language IDL (Inter-parameter Dependency Language). IDL needs to be written by the developer in advance, and then a constraint solver is used to generate effective test cases.

[0007] Morest combines the advantages of previous work RESTler and RestTestGen, and builds RPG (RESTful-service Property Graph) with richer information than ODG to model the dependencies between operations and parameters, and updates the connection of RPG during execution to achieve feedback updates. The testing process is divided into two steps. First, RPG is built with API documentation, and then RPG is dynamically updated during the test. During the test, all parameter nodes are traversed and extended at both ends according to the dependent connection, thus forming multiple test sequences.

[0008] RestCT uses the t-way technology of combinatorial testing to solve the problem of not being able to exhaustively explore the input space, while also considering the dependencies or constraints between operations and parameters. The generation process is divided into two stages: first, a constraint sequence coverage array is generated to determine the order of operation execution; then, an adaptive strategy is applied to concretize several of the constraint coverage arrays. In terms of constraint modeling, RestCT's method is more complete than the previous methods: for API dependency modeling, the resource hierarchy and CRUD semantics are considered; for parameter constraint modeling, the natural language tool spaCy is applied to design 23 parameter constraint patterns, which are extracted using pattern matching.

[0009] RESTInfer makes full use of the potential of natural language processing to analyze the description field in the API documentation to infer possible parameter constraints. Compared with RESTest, which requires manual writing of IDL parameter dependencies, RESTInfer is more convenient. Its main process is: first extract constraint information through the entity recognition tool NERSE and the relationship extraction tool Stanford parser, then formalize the parameter constraints into propositional logic, and finally connect to Morest to continue the experiment.

[0010] In 2022, Arcuri, the developer of EvoMaster, and the Meituan team applied EvoMaster to the testing of modern RPC APIs. A REST layer was added to the original EvoMaster to adapt to the RPC-style API, and many other parts of EvoMaster were also modified to be RPC-compatible, including interfaces, search spaces, and adaptation functions.

[0011] However, there are still many problems with the existing technology. First, the black-box testing tool is based on the API documentation, which is available for any API that is open to the public. There is no access permission requirement for the source code of the object being tested, and there are no access requirements or restrictions on the logic and language behind it. White-box testing tools represented by EvoMaster require access to the source code and the running process, which involves the information security of the enterprise. There are many hidden dangers in the application and use of the testing tools, and effective testing is impossible for many academic researchers and even researchers who are not in the specific business team of the enterprise.

[0012] Secondly, RPC API lacks information compared to REST API. Directly adapting existing REST API-related testing tools to RPC API will greatly reduce the testing effect.

[0013] Tools such as RestTestGen and Morest use dependency graphs to model API dependencies and generate API sequences by searching based on the constructed graphs. Tools such as RestTest rely on manually written IDL to model parameter-restricted dependencies. Tools such as RestCT and RestInfer use template matching, natural language processing and other tools to analyze text descriptions of API documents, which is not accurate or convenient enough. Summary of the invention

[0014] The present invention aims to address the deficiencies of the prior art and to propose an automated testing method for RPC API of a large industrial system.

[0015] The objective of the present invention is to achieve the following technical solution: an automated testing method for RPC API of a large industrial system, the method comprising:

[0016] S1. Perform static analysis on the RPC style API document to obtain API parameter information and API dependency graph;

[0017] S2, re-evaluate the API parameter similarity based on the large model and add new dependencies to the API dependency graph;

[0018] S3. Filter the dependency relationships of the API dependency graph based on the tenant's historical call data to obtain a more accurate and usable API dependency graph;

[0019] S4. Automatically generate API call sequences: First, weight the edges of the API dependency graph according to the tenant's historical call data, then generate API sequences based on the weighted API dependency graph, and finally filter the API sequences based on the API's add, delete, modify, and query attributes extracted by the big model to obtain API sequences that can be used for testing;

[0020] S5. Automatically fill in parameters for the API sequence to form a complete API test case: Use the dynamic parameters generated by the API sequence to fill in some parameters of the subsequent API, and automatically fill in the remaining parameters based on the large language model. The large model input includes parameter-related information extracted from the API document, and the large model output is a recommended combination of API parameters;

[0021] S6. Encode and encrypt the complete test case according to the request specification of the test server, and perform the test in the test server to obtain the final test result.

[0022] Furthermore, the static analysis specifically includes: parsing the API document in JSON format, extracting the API name, API meaning, API parameters and their descriptions, API return information, error code and error information, and storing them in the API parameter information for subsequent analysis.

[0023] Furthermore, the S2 specifically includes:

[0024] Add API dependencies based on parameter similarity; extract the input and output parameters of the API, first determine the correlation of the parameters based on the strong correlation between the parameters, cluster and connect the related output and input parameters, and add new dependencies from the API corresponding to the output parameter to the API corresponding to the input parameter;

[0025] Then design prompts, use the big model to determine whether different parameters are correlated, measure the weak correlation between parameters, and add new dependencies in the same way;

[0026] In the judgment of the related items, if the number of parameters is small, the judgment is made by comparing the parameters pairwise; if the number of parameters is large, a two-step method of coarse screening + fine screening is adopted; first, all parameters and their brief descriptions are combined into prompt words, and preliminary clustering is performed through a large model; then, in each coarse screening group, more structural information is provided for the parameters, and clustering is performed again to obtain the final classification result, and the parameters in the same category are regarded as related parameters.

[0027] Furthermore, the filtering of the dependency relationship of the API dependency graph based on the tenant's historical call data specifically includes: collecting historical call data of different users within a certain period of time, inferring whether there may be a dependency relationship between the APIs by limiting the API call time interval and whether the APIs have the same parameter value input and output parameters; verifying the obtained API dependency relationship, if there is actual call data to prove it, the API dependency relationship is considered to exist, otherwise it is deleted.

[0028] Furthermore, the weighting of the edges of the API dependency graph according to the tenant's historical call data is specifically: the weight of each dependency is the frequency of occurrence of the API dependency in the historical data. The formula is as follows:

[0029]

[0030] Furthermore, the generation of the API sequence based on the weighted API dependency graph includes: taking the specified API to be tested as the starting point, iteratively searching its dependencies to obtain the API request sequence of each API to be tested; if some input parameters cannot find the corresponding predecessor API to generate, they are regarded as parameters that need to be specified separately, which will be subsequently specified by the user or recommended by the large model. During the search process, the access order will be recorded, and loops will be ignored to prevent entering an infinite loop.

[0031] Furthermore, the formula used in S4 to filter the API sequence according to the addition, deletion, modification and query attributes of the API extracted by the large model is as follows:

[0032]

[0033] Furthermore, in the large model recommendation parameters, the prompt words for the large model include: role description, API documentation, output format requirements and special output restrictions;

[0034] The role description tells the big model that it will play the role of a test development engineer, giving the task background; the API document provides relevant information about the API; the output format requires that a JSON file that can be directly used by the tool be returned; the special output restrictions are used to prevent unexpected behavior; finally, the data returned by the big model is passed through a JSON format analyzer, and answers that do not meet the requirements are marked and re-requested until the format meets the requirements; the filtered information will be used as a parameter recommendation combination to automatically fill in the API parameters.

[0035] On the other hand, the specification of the present invention also provides an automated testing device for RPC API of large industrial systems, including a memory and one or more processors, wherein the memory stores executable code, and when the processor executes the executable code, it implements the automated testing method for RPC API of large industrial systems.

[0036] On the other hand, the specification of the present invention also provides a computer-readable storage medium on which a program is stored. When the program is executed by a processor, the automated testing method for the RPC API of a large industrial system is implemented.

[0037] Beneficial effects of the present invention:

[0038] The present invention belongs to a black box testing tool, which realizes the automated testing of API and reduces the manual dependence by automatically analyzing API documents and generating parameter and sequence automatic requests for testing.

[0039] The present invention constructs an API dependency graph based on parameter similarity and corrects the API dependency graph through tenant call records. In the context of the lack of rich information such as hierarchical structure of RPC API, the understanding of API parameters is enhanced through a large model, and the accuracy of the automatically generated API dependency graph is improved, thereby improving the coverage and efficiency of the test sequence. The present invention also utilizes actual production data and introduces the dependencies it contains, effectively reducing the impact of errors caused by API document errors, improving the error detection rate, and improving the reliability of test cases.

[0040] The present invention fills parameter values ​​based on dynamic parameters and large model recommended parameters. Dynamic parameters improve the accuracy of parameter information compared to random parameters or example parameters; compared to tools using other natural language processing technologies, the present invention uses large models to improve the ability to understand API parameter information; at the same time, it more directly recommends parameter combinations, reduces process complexity, and improves test efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A system architecture diagram provided for an embodiment of the present invention;

[0042] Figure 2 An example of API documentation provided for an embodiment of the present invention;

[0043] Figure 3 An example of a prompt word for two-step clustering provided in an embodiment of the present invention;

[0044] Figure 4 An example of an API dependency graph provided for an embodiment of the present invention;

[0045] Figure 5A schematic diagram of an automated black box testing device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0046] The specific implementation modes of the present invention are further described in detail below with reference to the accompanying drawings.

[0047] like Figure 1 As shown, the present invention provides an automated testing method for RPC API of large industrial systems. The design of the present invention includes the generation of API test sequences and the rendering of API sequence parameters, wherein the API sequence generation includes the construction of an API dependency model and the generation of an API sequence based on the dependency model, and the API parameter rendering is to concretize the various parameters of the API sequence generated in the previous order.

[0048] Specifically, the present invention first constructs an API dependency model. In the first step, the present invention performs static analysis on the RPC-style API document to obtain API parameter information and an API dependency graph. In the second step, the present invention re-evaluates parameter similarity based on the large model and adds new dependencies to the API dependency graph. In the third step, the present invention filters the dependencies of the API dependency graph based on the tenant's historical call data. Finally, a more accurate and usable API dependency graph is obtained.

[0049] Then, the present invention automatically generates an API call sequence. First, the edges of the API dependency graph are weighted according to the tenant's historical call data, and then the API sequence is generated according to the weighted API dependency graph. Finally, the API sequence is filtered according to the API's CRUD attributes extracted by the big model to obtain an API sequence that can be used for testing.

[0050] Finally, the present invention automatically fills in parameters for the API sequence to form a complete API test case. On the one hand, the present invention uses the dynamic parameters generated by the API sequence to fill in some parameters of the subsequent API, and on the other hand, it automatically fills in the remaining parameters based on a large language model. The input of the large model includes parameter-related information extracted from the API document, and the output of the large model is the recommended combination of API parameters.

[0051] Therefore, the present invention realizes the automated black box testing of the RPC API of a large industrial system.

[0052] The application scenarios of this method include but are not limited to the testing of various RPC-style APIs: API testing of Alibaba Cloud's computing, storage, network, and CDN services; API testing of Tencent Cloud's cloud servers, cloud disks, cloud functions, and other services; API testing of AWS's EC2, VPC, RDS, and other services. First, the tool will read the Open API document. An example of the API document is as follows: Figure 2As shown, it generally includes information such as API name, API description, API type, API input parameters and specific specifications, API output parameters and specific structure, possible error codes and corresponding meanings.

[0053] Specifically, the first part of the present invention is to build an API dependency model based on parameter similarity and historical call data.

[0054] The first step is to perform static analysis. Parse the API document in JSON format to extract information such as API name, API meaning, API parameters and their descriptions (including but not limited to name, type, default value, sample value, meaning, etc.), API return information (i.e. return parameters and their descriptions), error codes and error messages. This information is stored in the API parameter information for subsequent analysis.

[0055] The second step is to add API dependencies based on parameter similarity. Extract the input and output parameters of the API, first determine the correlation of the parameters based on the strong correlation between the parameters (whether the parameter names are the same or not, whether the parameter specifications are consistent), cluster the related output and input parameters, and add new dependencies from the API corresponding to the output parameter to the API corresponding to the input parameter. Then design a prompt, use the large model to determine whether there is a correlation between different parameters, measure the weak correlation between parameters, and add new dependencies in the same way. Figure 3 The following is an example of a prompt for two-step clustering provided by an embodiment of the present invention, which includes three main parts: task description, parameter information, and output format. The first part describes the role and corresponding capabilities that the large model should play, and describes the tasks that need to be completed. Figure 3 The two templates set the role of the big model as a cloud service expert, whose ability is to be good at solving problems related to cloud infrastructure. The second part describes the required parameter information. In the coarse screening stage, the parameter information is a list that stores the names of all parameters; in the fine screening stage, it contains the names of the parameters in the group and the description information after the parameters are summarized, provided in the form of key-value pairs. The third part explains the expected output content and the corresponding format. Both stages are grouping tasks, so it is expected that the big model can group the parameters and the parameters in the group have similar meanings. For the format, the output is required to return multiple parameter groups, including the group number and the corresponding parameters in the group, and an example in json format is provided to illustrate.

[0056] For the correlation judgment between parameters, the present invention designs two different strategies according to the number of parameters. For the case of a small number, the method of pairwise comparison between parameters is adopted to obtain better matching effect through richer information. For the case of a large number, a two-step method of coarse screening + fine screening is adopted; first, all parameters and their brief descriptions are composed into prompt words, and preliminary clustering is performed through a large model; then, in each coarse screening group, more structural information is provided for the parameters, and clustering is performed again to obtain the final classification result. Parameters in the same category are regarded as related parameters. The number mainly depends on the context window size of the large model used and its processing ability for long texts. If the number of parameters is large, the prompt word length of the pairwise comparison scheme is greater than the context window length of the large model, or the length is too long and the model processing effect is poor, it belongs to the second case, and a two-step solution should be used.

[0057] The third step is to filter API dependencies based on tenant historical call data. Collect historical call data from different users within the selected time, and infer whether there may be dependencies between APIs by limiting the API call time interval and whether the APIs have the same parameter values ​​for input and output parameters. Verify the API dependencies obtained in the first two steps. If there is actual call data to prove it, the API dependency is considered to exist, otherwise it is deleted. The selected time is determined according to the time range obtained by the tool user.

[0058] Finally, an API dependency graph is formed. An example of an API dependency graph is as follows: Figure 4 As shown in the figure, the blue node represents an API, the red node represents a parameter, and the directed edge between the nodes represents the relationship between the API and the parameter. The directed edge from the blue node to the red node means that the parameter is one of the output parameters of the API, and the directed edge from the red node to the blue node means that the parameter is one of the input parameters of the API.

[0059] The second part is API sequence generation based on weighted dependency graph.

[0060] First, the API dependency graph is weighted according to the tenant's historical call data. The weight of each dependency is the frequency of occurrence of the API dependency in the historical data. The formula is as follows:

[0061]

[0062] Where weight refers to the weight of the dependency relationship in the directed graph; API begin Refers to the starting API of the dependency relationship in the directed graph, that is, the dependent API; APIend refers to the ending API of the dependency relationship in the directed graph, that is, the API that depends on the previous item; PARAM iRefers to the dependency relationship formed by the i-th parameter in the directed graph, that is, the dependency relationship between the two specified APIs implemented by this parameter; N i It refers to the number of occurrences of the dependency realized by the i-th parameter after specifying the start and end APIs by analyzing the tenant's historical call data; condition refers to the tenant's historical call data judgment condition designed by us. Specifically: if within a certain time interval (such as 5 minutes), the same user visits two APIs successively, and the output parameter value of the first API is equal to the input parameter value of the second API, then the two APIs are considered to have realized a dependency relationship through this parameter value. If the parameter names are equal, the parameter name can be directly used to refer to PARAM i , otherwise use parameter pairs (an output parameter of the previous API and an input parameter of the next API).

[0063] Then, the sequence is generated, starting from the specified API to be tested, and its dependencies are iteratively searched, that is, the search is continuously forward until all the preceding APIs are covered, so as to obtain the API request sequence of each API to be tested. If some input parameters cannot find the corresponding preceding API to generate, they are considered to be parameters that need to be specified separately, and will be specified by the user or recommended by the large model later. Since loops may appear in the graph, the access order will be recorded during the search process, and the loop will be ignored to prevent entering an infinite loop.

[0064] Finally, these API dependency sequences are ranked to determine their test order. The score of the API dependency sequence is calculated by summing the weights of each dependency in the sequence, which corresponds to the first term (α term) of the following calculation formula. Many other indicators are also included. (a) The length of the API sequence: Shorter sequences are more likely to be used in actual scenarios because developers aim to simplify the use of the API. On the other hand, longer sequences may introduce redundant APIs. Corresponding to the second term (β term) of the following calculation formula. (b) The number of manual inputs: The fewer the number of manual inputs, the less human intervention and the stronger the automation capability. Corresponding to the third term (γ term) of the following calculation formula. (c) CRUD semantics: This will be discussed in the next paragraph. By leveraging these criteria, we can effectively rank the API request sequences of each API under test using the following equation. Corresponding to the δ term of the following calculation formula.

[0065] The calculation formula used is as follows:

[0066]

[0067] Where score refers to the score of the API sequence; API_Seq{a1,a2,…,a n} refers to a irefers to the API sequence composed of the order of APIs; α is the coefficient of the sum of the weights of the first API sequence. The sum of the weights of the API sequence measures the importance of the API sequence, which is achieved by summing up the weights of the dependencies of the API sequence; β is the coefficient of the number of APIs in the second API sequence, n is the number of APIs in the API sequence; γ is the coefficient of the number of parameters that need to be manually entered in the third API sequence, N Manual_Input That is, the sum of the number of parameters that need to be manually input in the API sequence; δ is the coefficient of the API sequence score, which is measured by whether the API sequence conforms to the CRUD semantics, that is, examining any two APIs, if they operate on the same resource, whether they conform to the CRUD semantic order of first create-then read and update-finally delete.

[0068] The third part is the API parameter generation based on the large model.

[0069] API parameter generation is achieved by combining API output dynamic parameters and large model recommended parameters.

[0070] On the one hand, according to the API sequence generated by the API dependency graph, some parameters can be filled from the output of the previous API to the input of the next API. This parameter is called a dynamic parameter because it is dynamically generated by the previous API during the test for use by the subsequent API. It will be stored in the dynamic parameter dictionary for automatic parameter filling.

[0071] On the other hand, the API description document will be provided to the big model to recommend parameter combinations as a supplement to the dynamic parameter dictionary, so that the remaining parameters can be automatically filled in. The prompt words for the big model include role description, API documentation, output format requirements, and special output restrictions. The role description tells the big model that it will play the role of a test development engineer to give the task background; the API documentation provides relevant information about the API so that the big model has an information source; the output format requires that the returned JSON file can be directly used by the tool; the special output restrictions are due to the fact that the big model's grasp of the task is easy to deviate, so special restrictions are set to prevent unexpected behaviors such as excessive output or output of illegal characters. Finally, the data returned by the big model is verified by the JSON format analyzer of the present invention to verify whether the string returned by the big model can be successfully serialized into a Python dictionary, and to verify whether it contains the name of the parameter combination, the key-value pair of the parameter name and the parameter value. If the verification is successful, it can be saved for use by the test tool. If an answer that does not meet the requirements is identified, an error mark is made and a request is made again until the format meets the requirements. The filtered information will be used as a parameter recommendation combination to automatically fill in the API parameters.

[0072] Finally, test case generation and execution are performed. For the API sequence generated based on the API weighted dependency graph, the API parameters are filled using the dynamic parameter dictionary and the parameters recommended by the large model to form the original test case. These test cases will be encoded according to the request specification of the Alibaba Cloud API, encrypted using the account AK / SK, and sent to the Alibaba Cloud server for test requests.

[0073] After the test is completed, the tool will give a test report in the form of a log, which contains rich information such as the parameters and responses of each request, and finally gives the final API coverage and use case efficiency. The empirical data generated during the test will also be deposited in JSON format for subsequent testing, including but not limited to the actual request parameter combination and the corresponding response information, API dependency graph data, API parameter information, etc.

[0074] The RPC style API in the present invention is only used to distinguish it from the REST style API. For API designs that do not comply with these two API design styles, if there is no clear hierarchical relationship between the APIs, they can be decoupled and independent, and relevant information about input and output parameters can be provided, and the tool of the present invention can also be used for testing.

[0075] Although the present invention is aimed at API testing, for forms such as API, that is, interfaces or similar services that do not expose internal implementation logic to the outside and only provide input and output information, this tool can also be used for testing through simple adaptation (such as writing equivalent API documents).

[0076] Corresponding to the aforementioned embodiment of an automated testing method for a large-scale industrial system RPC API, the present invention further provides an embodiment of an automated testing device for a large-scale industrial system RPC API.

[0077] See also Figure 5 An embodiment of the present invention provides an automated testing device for RPC API of a large industrial system, including a memory and one or more processors. The memory stores executable code. When the processor executes the executable code, it is used to implement an automated testing method for RPC API of a large industrial system in the above embodiment.

[0078] The embodiment of the automated testing device for the RPC API of a large industrial system provided by the present invention can be applied to any device with data processing capabilities, and the device with data processing capabilities can be a device or apparatus such as a computer. The device embodiment can be implemented through software, or through hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor of any device with data processing capabilities in which it is located reading the corresponding computer program instructions in the non-volatile memory into the internal memory for execution. From the hardware level, if Figure 5 As shown in FIG. 1 , a hardware structure diagram of an automated testing device for RPC API of a large industrial system provided by the present invention is provided for any device with data processing capability. Figure 5 In addition to the processor, memory, network interface, and non-volatile memory shown, any device with data processing capabilities in which the apparatus in the embodiments is located may also include other hardware, generally based on the actual functions of the device with data processing capabilities, which will not be described in detail.

[0079] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0080] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of the present invention. Ordinary technicians in this field can understand and implement it without paying creative work.

[0081] An embodiment of the present invention further provides a computer-readable storage medium on which a program is stored. When the program is executed by a processor, an automated testing method for RPC API of a large industrial system in the above embodiment is implemented.

[0082] The computer-readable storage medium may be an internal storage unit of any device with data processing capability described in any of the aforementioned embodiments, such as a hard disk or a memory. The computer-readable storage medium may also be an external storage device of any device with data processing capability, such as a plug-in hard disk, a smart media card (SMC), an SD card, a flash card, etc. equipped on the device. Furthermore, the computer-readable storage medium may also include both an internal storage unit and an external storage device of any device with data processing capability. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capability, and may also be used to temporarily store data that has been output or is to be output.

[0083] The present invention also provides a computer program product, including a computer program, and when the computer program is executed by a processor, the automated testing method for the RPC API of a large industrial system is implemented.

[0084] Those skilled in the art will readily appreciate other embodiments of the present application after considering the description and practicing the contents disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The description and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the claims.

[0085] It should be understood that the above general description and the detailed description below are only exemplary and explanatory and cannot limit the present application. The present application is not limited to the precise structure described above and shown in the drawings, and various modifications and changes can be made without departing from the scope thereof. The scope of the present application is limited only by the attached claims.

Claims

1. An automated testing method for RPC API of large industrial systems, characterized in that: The method includes: S1. Perform static analysis on the RPC style API document to obtain API parameter information and API dependency graph; S2, re-evaluate the API parameter similarity based on the large model and add new dependencies to the API dependency graph; S3. Filter the dependency relationships of the API dependency graph based on the tenant's historical call data to obtain a more accurate and usable API dependency graph; S4. Automatically generate API call sequences: First, weight the edges of the API dependency graph according to the tenant's historical call data, then generate API sequences based on the weighted API dependency graph, and finally filter the API sequences based on the API's add, delete, modify, and query attributes extracted by the big model to obtain API sequences that can be used for testing; S5. Automatically fill in parameters for the API sequence to form a complete API test case: Use the dynamic parameters generated by the API sequence to fill in some parameters of the subsequent API, and automatically fill in the remaining parameters based on the large language model. The large model input includes parameter-related information extracted from the API document, and the large model output is a recommended combination of API parameters; S6. Encode and encrypt the complete test case according to the request specification of the test server, and perform the test in the test server to obtain the final test result.

2. According to claim 1, an automated testing method for RPC API of a large industrial system is characterized in that: The static analysis specifically includes: parsing the API document in JSON format, extracting the API name, API meaning, API parameters and their descriptions, API return information, error code and error information, and storing them in the API parameter information for subsequent analysis.

3. The automated testing method for RPC API of a large industrial system according to claim 1, characterized in that: The S2 specifically includes: Add API dependencies based on parameter similarity; extract the input and output parameters of the API, first determine the correlation of the parameters based on the strong correlation between the parameters, cluster and connect the related output and input parameters, and add new dependencies from the API corresponding to the output parameter to the API corresponding to the input parameter; Then, the prompt is designed to determine whether different parameters are correlated through the large model, measure the weak correlation between parameters, cluster and connect the related output and input parameters, and add new dependencies; In the judgment of the related items, different strategies are used according to the context window size of the large model and its processing ability for long texts. If the number of parameters does not affect the effect of the pairwise comparison scheme, the judgment is made by pairwise comparison of the parameters. If the number of parameters is large, the prompt word length of the pairwise comparison scheme is greater than the context window length of the large model, or the length is too long and the model processing effect is not good, then a two-step method of coarse screening + fine screening is adopted; first, all parameters and their brief descriptions are combined into prompt words, and preliminary clustering is performed through the large model; then, in each coarse screening group, more structural information is provided for the parameters, and clustering is performed again to obtain the final classification result, and parameters in the same category are regarded as related parameters.

4. The automated testing method for RPC API of a large industrial system according to claim 1, characterized in that: The filtering of the dependency relationship of the API dependency graph based on the tenant's historical call data specifically includes: collecting historical call data of different users in a selected time period, and inferring whether there is a dependency relationship between APIs by limiting the API call time interval and whether the APIs have input and output parameters with the same parameter values; verifying the obtained API dependency relationship, and if there is actual call data to prove it, the API dependency relationship is considered to exist, otherwise it is deleted.

5. The automated testing method for RPC API of a large industrial system according to claim 1, characterized in that: The weighting of the edges of the API dependency graph according to the tenant's historical call data is specifically: the weight of each dependency is the frequency of occurrence of the API dependency in the historical data, and the formula is as follows: Where weight refers to the weight of the dependency relationship in the directed graph; API begin Refers to the starting API of the dependency relationship in the directed graph, that is, the dependent API; APIend refers to the ending API of the dependency relationship in the directed graph, that is, the API that depends on the previous item; PARAM i Refers to the dependency relationship formed by the i-th parameter in the directed graph, that is, the dependency relationship between the two specified APIs implemented by this parameter; N i It refers to the number of occurrences of the dependency realized by the ith parameter after analyzing the tenant's historical call data and specifying the start and end APIs; condition refers to the tenant's historical call data judgment condition designed by us. If the parameter names are equal, the parameter name is directly used to refer to PARAM i , otherwise use the parameter pair.

6. The automated testing method for RPC API of a large industrial system according to claim 1, characterized in that: The generation of the API sequence based on the weighted API dependency graph includes: starting from the specified API to be tested, iteratively searching its dependencies, and obtaining the API request sequence of each API to be tested; if some input parameters cannot find the corresponding predecessor API to generate, they are regarded as parameters that need to be specified separately, which will be specified by the user or recommended by the large model later. The access order will be recorded during the search process, and loops will be ignored to prevent entering an infinite loop.

7. The automated testing method for RPC API of a large industrial system according to claim 1, characterized in that: The specific formula used in S4 to filter the API sequence according to the addition, deletion, modification and query attributes of the API extracted by the large model is as follows: Where score refers to the score of the API sequence; API_Seq{a1,a2,…,a n } refers to a i The API sequence is composed of the order of the APIs referred to; α is the coefficient of the weight sum of the first API sequence. The API sequence weight sum measures the importance of the API sequence, which is achieved by summing up the weights of the dependencies of the API sequence; β is the coefficient of the number of APIs in the second API sequence, and n is the number of APIs in the API sequence; γ is the coefficient of the number of parameters that need to be manually entered in the third API sequence, N Manual_Input That is, the sum of the number of parameters that need to be manually input in the API sequence; δ is the coefficient of the API sequence score, which is measured by whether the API sequence conforms to the CRUD semantics, that is, examining any two APIs, if they operate on the same resource, whether they conform to the CRUD semantic order of first create, then read and update, and finally delete.

8. The automated testing method for RPC API of a large industrial system according to claim 1, characterized in that: In the large model recommended parameters, the prompt words for the large model include: role description, API documentation, output format requirements and special output restrictions; The role description informs the big model that it will play the role of a test development engineer given the task background; the API document provides relevant information about the API; the output format requires that a JSON file that can be directly used by the tool is returned; the special output restrictions are used to prevent unexpected behavior; finally, the data returned by the big model is passed through a JSON format analyzer, and answers that do not meet the requirements are marked and re-requested until the format meets the requirements; the filtered information will be used as a recommended parameter combination to automatically fill in API parameters; the JSON format analyzer verifies whether the string returned by the big model can be successfully serialized into a Python dictionary, and verifies whether it contains the name of the parameter combination, and the key-value pairs of the parameter name and parameter value. If the verification is successful, it will be saved; if an answer that does not meet the requirements is identified, it will be marked as an error.

9. An automated testing device for RPC API of a large industrial system, comprising a memory and one or more processors, wherein the memory stores executable code, characterized in that: When the processor executes the executable code, an automated testing method for a large-scale industrial system RPC API is implemented as described in any one of claims 1-8.

10. A computer-readable storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, an automated testing method for RPC API of a large industrial system according to any one of claims 1 to 8 is implemented.

Images