Software system testing method and device

Abstract

The embodiment of the present application provides a software system testing method and device, which can be used in the field of big data technology. The method includes: searching the target test object corresponding to the target API that has been changed in the software system in the API dependency graph corresponding to the software system, and the target test object includes: the target API and all dependent APIs corresponding to the target API, and the application or subsystem to which the target API and the dependent API respectively belong; obtaining the test case set corresponding to the target test object and generating a target test plan to test the target test object in the software system. The present application can effectively improve the efficiency and automation of searching for all test objects affected by the target API that has been changed, thereby effectively improving the efficiency and automation of the software system testing process, and can improve the test reliability and comprehensiveness of the software system testing process on the basis of improving test efficiency and reducing redundancy.

Description

Technical Field

[0001] The present application relates to the field of software testing technology, in particular to the field of big data technology, and specifically to a software system testing method and device. Background Art

[0002] With the rapid development of technologies such as cloud computing and cloud native, users' demands for software systems have become more diversified, the complexity of software systems has increased, and they are developing in the direction of distributed, highly available, scalable, and intelligent operation and maintenance. At present, software systems are gradually evolving towards a microservice technology architecture. Under the microservice technology framework, complex software systems are built by a large number of microservices with independent processes and independent development, testing, deployment, and operation and maintenance capabilities. Microservices are connected to each other by application programming interfaces (APIs). Changes to the API have a significant impact on the stability of the entire software system. Therefore, it is necessary to quickly develop unit testing and system integration testing plans for API changes and evaluate the test results.

[0003] In the existing software system testing process, when some APIs in the software system change, it is necessary to rely on manual or pre-defined mapping tables to determine the affected subsystems for integration testing. When the API changes, it is necessary to rely on manual or pre-defined mapping tables to determine the affected APIs, applications, and subsystems to be tested. Therefore, the existing software system testing methods have problems such as low test efficiency, high redundancy, and poor test reliability and comprehensiveness due to the omission of test objects. Summary of the invention

[0004] In response to the problems in the prior art, the present application provides a software system testing method and device, which can effectively improve the efficiency and automation of finding all test objects affected by the current target API change, thereby effectively improving the efficiency and automation of the software system testing process, and can improve the test reliability and comprehensiveness of the software system testing process on the basis of improving test efficiency and reducing redundancy.

[0005] In order to solve the above technical problems, this application provides the following technical solutions:

[0006] In a first aspect, the present application provides a software system testing method, comprising:

[0007] Searching the target test object corresponding to the target API currently changed in the software system in the API dependency graph corresponding to the software system, wherein the target test object includes: the target API and all dependent APIs corresponding to the target API, and the applications or subsystems to which the target API and the dependent APIs respectively belong;

[0008] A test case set corresponding to the target test object is obtained and a target test plan is generated, so as to test the target test object in the software system based on the target test plan.

[0009] Furthermore, before searching the API dependency graph corresponding to the software system for the target test object corresponding to the target API currently changed in the software system, the method further includes:

[0010] Obtaining metadata of each API corresponding to the software system, wherein the API metadata is used to store a unique identifier of a uniquely corresponding API and a dependency relationship between the API and other APIs, and the API metadata is also used for an application or subsystem to which the uniquely corresponding API belongs;

[0011] Each of the API metadata is taken as a node, and directed edges between each of the nodes are determined according to the dependency relationship between each of the APIs, so as to generate an API dependency graph consisting of each of the nodes and directed edges between each of the nodes.

[0012] Furthermore, searching the API dependency graph corresponding to the software system for a target test object corresponding to a target API currently changed in the software system includes:

[0013] Receiving an API change message for the software system, wherein the API change message includes a unique identifier of a target API to which the change is sent and a change type, wherein the change type includes: adding, deleting, or modifying;

[0014] Based on the unique identifier of the target API, an API dependency subgraph corresponding to the target API is searched in the API dependency graph corresponding to the software system, wherein the API dependency subgraph contains a target test object corresponding to the target API.

[0015] Further, the obtaining of the test case set corresponding to the target test object and generating a target test plan includes:

[0016] Obtaining an API coverage map corresponding to each API in the target test object, wherein each API coverage map is in a one-to-one correspondence with each test case;

[0017] According to the API dependency subgraph and the API coverage graph, select a test case set corresponding to the target test object;

[0018] A target test plan for the target test object is generated based on the test case set.

[0019] Furthermore, before obtaining the test case set corresponding to the target test object and generating the target test plan, it also includes:

[0020] Obtain test cases corresponding to each subsystem or application in the software system, wherein the test cases include: unit test cases for covering APIs in a single subsystem or application, and integration test cases for covering APIs between multiple subsystems or applications;

[0021] Generate an API coverage graph corresponding to each of the test cases according to the API dependency graph currently corresponding to the software system;

[0022] The API coverage map corresponding to the unit test case is used to store the APIs in a single subsystem or application covered by the unit test case. If there are multiple APIs in a single subsystem or application, the API coverage map corresponding to the unit test case is also used to store the dependency relationships between the APIs in the single subsystem or application covered by the unit test case.

[0023] The API coverage graph corresponding to the integration test case is used to store the APIs between multiple subsystems or applications covered by the integration test case and the dependencies between the APIs.

[0024] Furthermore, selecting a test case set corresponding to the target test object according to the API dependency subgraph and the API coverage graph includes:

[0025] Determine, according to the API dependency subgraph and the API coverage graph, the identifiers and total number of test cases corresponding to each of the APIs in the API dependency subgraph;

[0026] If there is an API with a total number of test cases of 1 in the API dependency subgraph, the identifier of the test case corresponding to the API with a total number of test cases of 1 is added to the test case set;

[0027] If there are APIs with a total test case count greater than 1 in the API dependency subgraph, and the test cases corresponding to the APIs cover the preset hotspot subsystems and / or hotspot APIs, the identifiers of the test cases covering the preset hotspot subsystems and / or hotspot APIs will be added to the test case set.

[0028] Furthermore, the API coverage graph is also used to store the test cost of the corresponding test case;

[0029] Correspondingly, the selecting a test case set corresponding to the target test object according to the API dependency subgraph and the API coverage graph further includes:

[0030] If there are APIs with a total number of test cases greater than 1 in the API dependency subgraph, and the test cases corresponding to the APIs do not cover the hot subsystems and / or hot APIs, then the corresponding test case identifiers are selected and added to the test case set based on the test costs and API coverage corresponding to each of the test cases corresponding to the API.

[0031] Furthermore, it also includes:

[0032] Receive an API addition message for the software system, wherein the API addition message includes API metadata corresponding to the API to be added;

[0033] According to the API metadata corresponding to the API to be added, directed edges between the node corresponding to the API to be added and other nodes having dependency relationships with the API to be added are added to the API dependency graph.

[0034] Furthermore, it also includes:

[0035] Receiving an API deletion message for the software system, wherein the API deletion message includes API metadata corresponding to the API to be deleted;

[0036] According to the API metadata corresponding to the API to be deleted, the node corresponding to the API to be deleted and the directed edges between other nodes having dependency relationships with the API to be deleted are deleted in the API dependency graph.

[0037] Furthermore, it also includes:

[0038] Receiving an API modification message for the software system, wherein the API modification message includes API modification metadata of the API to be modified;

[0039] Determine the content to be modified that is affected by the modification of the API to be modified according to the original API metadata corresponding to the API to be modified in the API dependency graph and the API modification metadata, wherein the content to be modified includes at least one of the dependent APIs of the API to be modified, the dependency relationship between the target API to be deleted and other APIs, and the dependency relationship between the target API to be added and other APIs;

[0040] Based on the content to be modified, the directed edges between other nodes in the API dependency graph that have dependency relationships with the API to be modified are modified.

[0041] Furthermore, it also includes:

[0042] Receiving an API implicit call relationship record, wherein the API implicit call relationship record is obtained in advance from the backtracking accident cause data corresponding to the production accident log of the software system;

[0043] According to the API implicit calling relationship record, a corresponding API implicit calling relationship is added to the API dependency graph.

[0044] Furthermore, it also includes:

[0045] Extracting hotspot objects from logs generated by the production environment of the software system, wherein the hotspot objects include hotspot subsystems and / or hotspot APIs;

[0046] The API dependency subgraph is updated according to the hotspot object.

[0047] In a second aspect, the present application provides a software system testing device, comprising:

[0048] A graph search module is used to search the API dependency graph corresponding to the software system for a target test object corresponding to the target API currently changed in the software system, wherein the target test object includes: the target API and all dependent APIs corresponding to the target API, and the applications or subsystems to which the target API and the dependent APIs respectively belong;

[0049] The testing module is used to obtain a test case set corresponding to the target test object and generate a target test plan, so as to test the target test object in the software system based on the target test plan.

[0050] In a third aspect, the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the software system testing method when executing the program.

[0051] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the software system testing method when executed by a processor.

[0052] It can be seen from the above technical scheme that the present application provides a software system testing method and device, the method comprising: searching for a target test object corresponding to a target API that has been changed in the software system in an API dependency graph corresponding to the software system, wherein the target test object comprises: the target API and all dependent APIs corresponding to the target API, and the application or subsystem to which the target API and dependent API respectively belong; obtaining a test case set corresponding to the target test object and generating a target test plan to test the target test object in the software system based on the target test plan; by presetting the API dependency graph corresponding to the target software system, and when there is a changed target API in the target software system, searching in the API dependency graph Finding all dependent APIs that have dependencies with the target API, as well as the applications or subsystems to which the target API and the associated APIs belong, can effectively improve the efficiency and automation of finding all test objects affected by the currently changed target API, thereby effectively improving the efficiency and automation of the software system testing process; at the same time, since there is no need to conduct a comprehensive test of the target software system, the redundancy of the software system testing can be further reduced; on this basis, it can also avoid the omission of the objects to be tested, and effectively improve the comprehensiveness and reliability of the acquisition of the objects to be tested. Therefore, on the basis of improving the test efficiency and reducing redundancy, the test reliability and comprehensiveness of the software system testing process can be improved, thereby improving the reliability and effectiveness of the software system testing results. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0054] Figure 1 It is a schematic diagram of the relationship between the software system testing device and the client device in the embodiment of the present application.

[0055] Figure 2 It is a first flow chart of the software system testing method in the embodiment of the present application.

[0056] Figure 3 It is a second flow chart of the software system testing method in the embodiment of the present application.

[0057] Figure 4 It is a third flow chart of the software system testing method in the embodiment of the present application.

[0058] Figure 5 It is a fourth flow chart of the software system testing method in the embodiment of the present application.

[0059] Figure 6 It is a fifth flow chart of the software system testing method in the embodiment of the present application.

[0060] Figure 7 It is a flowchart of step 220 in the software system testing method in the embodiment of the present application.

[0061] Figure 8 This is a sixth flow chart of the software system testing method in the embodiment of the present application.

[0062] Fig. 9 It is a flowchart of step 310 and step 320 in the software system testing method in the embodiment of the present application.

[0063] Fig.10 It is a flowchart of step 410 and step 420 in the software system testing method in the embodiment of the present application.

[0064] Fig.11 It is a flowchart of steps 510 to 520 in the software system testing method in the embodiment of the present application.

[0065] Fig.12 It is a flowchart of step 610 and step 620 in the software system testing method in the embodiment of the present application.

[0066] Fig.13 It is the seventh flow chart of the software system testing method in the embodiment of the present application.

[0067] Fig.14 It is a structural diagram of a software system testing device in an embodiment of the present application.

[0068] Fig.15 It is a logical schematic diagram of the application software system testing system provided by the application example of this application to implement the software system testing method.

[0069] Fig.16 This is an example diagram of the core content of the API metadata provided by the application example of this application.

[0070] Fig.17 This is an example diagram of the dependency relationship between the calling API node and the called API node provided by the application example of this application.

[0071] Fig.18 This is an example diagram of the transitivity of the API dependency relationship provided by the application example of this application.

[0072] Fig.19This is an example diagram of an API dependency diagram of all subsystems, APIs, and applications provided by the application example of this application.

[0073] Fig. 20 This is an example diagram of the affected APIs and subsystems provided by the application example of this application.

[0074] Fig.21 This is an example diagram of the API coverage diagram of the test case provided by the application example of this application.

[0075] Fig. 22 It is a schematic diagram of the structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0076] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0077] It should be noted that the software system testing method and device disclosed in the present application can be used in the field of big data technology, and can also be used in any field other than the field of big data technology. The application field of the software system testing method and device disclosed in the present application is not limited.

[0078] In traditional API unit testing and system integration testing methods, when some APIs in a software system change, it is necessary to rely on manual or pre-defined mapping tables to determine the affected subsystems for integration testing. This traditional API unit testing and system integration testing method is inefficient and the generated test solutions have high redundancy. When the subsystems and APIs in a complex software system change frequently, it is impossible to quickly generate targeted test solutions, and it is impossible to meet the user's needs for automated unit testing and system integration testing of API changes within the software system.

[0079] Based on this, for the existing software system testing methods, when the API changes, it is necessary to rely on manual or pre-defined mapping tables to determine the affected APIs, applications, and subsystems waiting for test objects. Therefore, the existing software system testing methods have problems such as low testing efficiency, high redundancy, and poor testing reliability and comprehensiveness caused by the omission of objects to be tested. The embodiment of the present application provides a software system testing method, which presets the API dependency graph corresponding to the target software system, and when there is a changed target API in the target software system, searches the API dependency graph for all dependent APIs that have a dependency relationship with the target API, as well as the target The application or subsystem to which the target API and the associated API belong can effectively improve the efficiency and automation of finding all test objects affected by the currently changed target API, thereby effectively improving the efficiency and automation of the software system testing process; at the same time, since there is no need to conduct a comprehensive test of the target software system, the redundancy of the software system testing can be further reduced; on this basis, it can also avoid the omission of the objects to be tested, and effectively improve the comprehensiveness and reliability of the acquisition of the objects to be tested. Therefore, on the basis of improving test efficiency and reducing redundancy, the test reliability and comprehensiveness of the software system testing process can be improved, thereby improving the reliability and effectiveness of the software system testing results.

[0080] Based on the above content, the present application also provides a software system testing device for implementing the software system testing method provided in one or more embodiments of the present application. The software system testing device can be a server, see Figure 1 The software system testing device can communicate and connect with each client device in sequence by itself or through a third-party server, etc. The software system testing device can receive the software system testing instructions sent by the client device, and search for the target test object corresponding to the target API that has been changed in the software system in the API dependency graph corresponding to the software system according to the software system testing instructions. The software system testing device can also send the test results of the target test object to the user's client device, etc.

[0081] In another practical application scenario, the part of the software system testing device that performs the software system testing can be executed in the server as described above, or all operations can be completed in the user terminal device. The specific selection can be made based on the processing capability of the user terminal device and the limitations of the user's usage scenario. This application does not limit this. If all operations are completed in the user terminal device, the user terminal device can also include a processor for specific processing of the software system testing.

[0082] It is understandable that the mobile terminal may include any mobile device capable of loading applications, such as a smart phone, a tablet electronic device, a network set-top box, a portable computer, a personal digital assistant (PDA), a vehicle-mounted device, a smart wearable device, etc. Among them, the smart wearable device may include smart glasses, smart watches, smart bracelets, etc.

[0083] The above-mentioned mobile terminal may have a communication module (i.e., a communication unit) that can communicate with a remote server to achieve data transmission with the server. The server may include a server on the task scheduling center side, and other implementation scenarios may also include a server on an intermediate platform, such as a server on a third-party server platform that has a communication link with the task scheduling center server. The server may include a single computer device, or a server cluster consisting of multiple servers, or a server structure of a distributed device.

[0084] The server and the mobile terminal may communicate with each other using any suitable network protocol, including network protocols that have not yet been developed on the date of filing this application. The network protocols may include, for example, TCP / IP, UDP / IP, HTTP, HTTPS, etc. Of course, the network protocols may also include, for example, RPC (Remote Procedure Call Protocol) and REST (Representational State Transfer) protocols used on top of the above protocols.

[0085] In one or more embodiments of the present application, the microservice technology framework refers to providing one or more APIs for each microservice.

[0086] In one or more embodiments of the present application, an application programming interface (API) is a set of predefined interfaces (such as functions, HTTP interfaces), or a convention for connecting different components of a software system. It is used to provide a set of routines that application developers can access based on certain software or hardware without having to access the source code or understand the details of the internal working mechanism.

[0087] The details are described in detail through the following embodiments and application examples.

[0088] In order to solve the existing software system testing method, when the API changes, it is necessary to rely on manual or pre-defined mapping tables to determine the affected APIs, applications and subsystems to be tested. Therefore, the existing software system testing method has the problems of low test efficiency, high redundancy, and poor test reliability and comprehensiveness caused by the omission of test objects. The present application provides an embodiment of a software system testing method, see Figure 2 The software system testing method performed by the software system testing device specifically includes the following contents:

[0089] Step 100: Search the API dependency graph corresponding to the software system for the target test object corresponding to the target API that is currently changed in the software system, wherein the target test object includes: the target API and all dependent APIs corresponding to the target API, and the applications or subsystems to which the target API and dependent APIs respectively belong.

[0090] It is understandable that the software system testing device can automatically discover the calling relationship between APIs based on API implementation and API metadata in advance, and establish a directed cyclic graph of API dependencies. The API asset management library is updated as the API changes. When a new API is introduced or the calling relationship between APIs changes due to API changes, the API dependency graph is rebuilt.

[0091] Step 200: Acquire a test case set corresponding to the target test object and generate a target test plan, so as to test the target test object in the software system based on the target test plan.

[0092] It is understandable that when a part of the API in the software system changes, it is necessary to select test cases for the change and generate a test plan. The test plan needs to meet the following conditions: 1. The test plan needs to cover the changed API, as well as all subsystems, APIs, and applications that depend on the API; 2. The test plan needs to cover hot subsystems, APIs, and applications as much as possible; 3. The test plan needs to reduce the testing cost while ensuring 1 and 2. The test plan takes the list of changed APIs, the API dependency diagram, the list of hot subsystems and APIs, the API coverage diagram of the test cases, and the comprehensive cost of the test cases as input, and outputs a list of a set of test cases.

[0093] From the above description, it can be seen that the software system testing method provided in the embodiment of the present application, by presetting the API dependency graph corresponding to the target software system, and when there is a changed target API in the target software system, searching in the API dependency graph for all dependent APIs that have a dependency relationship with the target API, as well as the application or subsystem to which the target API and the associated API respectively belong, can effectively improve the efficiency and automation of finding all test objects affected by the currently changed target API, thereby effectively improving the efficiency and automation of the software system testing process; at the same time, since there is no need to conduct a comprehensive test of the target software system, the redundancy of the software system testing can be further reduced; on this basis, it can also avoid the omission of the objects to be tested, effectively improving the comprehensiveness and reliability of the acquisition of the objects to be tested, so that on the basis of improving the test efficiency and reducing redundancy, the test reliability and comprehensiveness of the software system testing process can be improved, thereby improving the reliability and effectiveness of the software system testing results.

[0094] In order to further improve the efficiency and automation of searching for target test objects corresponding to target APIs that have been changed in the software system in the API dependency graph corresponding to the software system, an embodiment of the software system testing method provided in the present application is provided. Figure 3 , the software system testing method further specifically includes the following contents before step 100:

[0095] Step 010: Obtain the API metadata corresponding to the software system, wherein the API metadata is used to store the unique identifier of a uniquely corresponding API and the dependency relationship between the API and other APIs. The API metadata is also used for the application or subsystem to which the uniquely corresponding API belongs.

[0096] Step 020: Take each of the API metadata as a node, and determine the directed edges between each of the nodes according to the dependency relationship between each of the APIs, so as to generate an API dependency graph consisting of each of the nodes and the directed edges between each of the nodes.

[0097] It is understandable that the metadata of the API is provided by the software system developer. Each API contains a unique identifier and one or more links to other APIs, which are used to identify the list of APIs called (dependent) by the API. Traverse all APIs and metadata and establish a node for each API. If the API contains calls to other APIs, traverse the call links contained in the API, find the called API node, and establish a directed edge from the called node to the calling node, indicating the dependency of the calling API node on the called API node. The transitivity of the API dependency is automatically established in the process of traversing all APIs. When all APIs registered and recorded in the API asset management module are traversed, an API dependency graph containing all subsystems, APIs, and applications can be established.

[0098] From the above description, it can be seen that the software system testing method provided in the embodiment of the present application, by generating an API dependency graph composed of each of the nodes and the directed edges between each of the nodes, and the API dependency graph covers the unique identifier of each API, the dependency relationship between each API and other APIs, and the application or subsystem to which each API belongs, can further improve the efficiency and automation level of searching for the target test object corresponding to the target API that has been changed in the software system in the API dependency graph corresponding to the software system, and further avoid the omission of the object to be tested, effectively improving the comprehensiveness and reliability of the acquisition of the object to be tested.

[0099] In order to more intuitively determine the target test object corresponding to the changed target API, in an embodiment of the software system testing method provided in this application, see Figure 4 , step 100 of the software system testing method specifically includes the following contents:

[0100] Step 110: Receive an API change message for the software system, wherein the API change message includes a unique identifier of a target API to which the change is sent and a change type, wherein the change type includes: addition, deletion or modification.

[0101] Step 120: Based on the unique identifier of the target API, search for an API dependency subgraph corresponding to the target API in the API dependency graph corresponding to the software system, wherein the API dependency subgraph contains a target test object corresponding to the target API.

[0102] Specifically, the software system testing device can query the APIs, subsystems and applications affected by the API change based on the API dependency graph, and calculate the dependency subgraph of the affected APIs, subsystems and applications.

[0103] From the above description, it can be seen that the software system testing method provided in the embodiment of the present application can more intuitively determine the target test object corresponding to the changed target API by directly searching the API dependency subgraph corresponding to the target API in the API dependency graph, thereby further improving the efficiency and automation of finding all test objects affected by the currently changed target API, thereby further improving the efficiency and automation of the software system testing process.

[0104] In order to improve the efficiency and automation of determining the test case set corresponding to the target test object, an embodiment of the software system testing method provided in this application is as follows: Figure 5 , step 200 of the software system testing method specifically includes the following contents:

[0105] Step 210: Obtain an API coverage map corresponding to each API in the target test object, wherein there is a one-to-one correspondence between each API coverage map and each test case.

[0106] Step 220: According to the API dependency subgraph and the API coverage graph, select a test case set corresponding to the target test object.

[0107] Step 230: Generate a target test plan for the target test object based on the test case set.

[0108] From the above description, it can be seen that the software system testing method provided in the embodiment of the present application, based on the API dependency graph, can effectively improve the efficiency and automation of determining the test case set corresponding to the target test object by adopting the API coverage graph of the test case, and can further improve the comprehensiveness of determining the test case set corresponding to the target test object, thereby further improving the efficiency and automation of the software system testing process, and can further improve the comprehensiveness and reliability of the software system testing process on the basis of reducing software system testing redundancy.

[0109] In order to further improve the efficiency and automation of determining the test case set corresponding to the target test object, an embodiment of the software system testing method provided in this application is shown in FIG. Figure 6 The software system testing method further specifically includes the following contents before step 200:

[0110] Step 021: Obtain test cases corresponding to each subsystem or application in the software system, wherein the test cases include: unit test cases for covering APIs in a single subsystem or application, and integration test cases for covering APIs between multiple subsystems or applications;

[0111] Step 022: Generate an API coverage map corresponding to each of the test cases based on the API dependency graph currently corresponding to the software system; wherein, the API coverage map corresponding to the unit test case is used to store the API in a single subsystem or application covered by the unit test case. If there are multiple APIs covering a single subsystem or application, the API coverage map corresponding to the unit test case is also used to store the dependency relationships between the various APIs in the single subsystem or application covered by the unit test case; the API coverage map corresponding to the integration test case is used to store the various APIs between multiple subsystems or applications covered by the integration test case and the dependency relationships between the various APIs.

[0112] Specifically, the software system testing device establishes an API coverage map for the test case and records the comprehensive cost of the test case. Unit test cases cover one or more APIs within a single subsystem; integration test cases cover multiple APIs between multiple subsystems. The API coverage map of the test case is a directed cyclic graph, which is a subgraph of the API dependency graph in the API asset management library. The comprehensive cost value of the test case is comprehensively assessed by the tester, test equipment, test running time and other expenses. The test case management library is updated with the changes of the test case and the API asset management library. When the test case changes, the API coverage map of the test case is updated according to the design and implementation of the test case.

[0113] From the above description, it can be seen that the software system testing method provided in the embodiment of the present application can further improve the efficiency and automation of determining the test case set corresponding to the target test object by pre-constructing API coverage maps corresponding to unit test cases and integration test cases, and can further improve the comprehensiveness of determining the test case set corresponding to the target test object, thereby further improving the efficiency and automation of the software system testing process, and can further improve the comprehensiveness and reliability of the software system testing process on the basis of reducing software system testing redundancy.

[0114] In order to cover all APIs and calling relationships between APIs affected by this change, an embodiment of the software system testing method provided in this application is provided, see Figure 7 , step 220 in the software system testing method specifically includes the following contents:

[0115] Step 221: According to the API dependency subgraph and the API coverage graph, determine the identifiers and total number of test cases corresponding to each of the APIs in the API dependency subgraph.

[0116] Step 222: If there is an API with a total number of test cases of 1 in the API dependency subgraph, the identifiers of the test cases corresponding to the API with a total number of test cases of 1 are added to the test case set.

[0117] Step 223: If there are APIs with a total number of test cases greater than 1 in the API dependency subgraph, and the test cases corresponding to the APIs cover the preset hotspot subsystems and / or hotspot APIs, the identifiers of the test cases covering the preset hotspot subsystems and / or hotspot APIs are added to the test case set.

[0118] It can be understood that all API nodes in the dependency subgraph of the changed API can be traversed, and for each API node, the number of test cases and the list of test cases that can cover the API in the candidate test schemes are calculated; if the number of test case coverage of an API node is 1, then the test case is selected as the only coverage test case and added to the final scheme; traverse the remaining test cases, and if the test case covers hot subsystems and APIs, then the test case is added to the final test scheme.

[0119] From the above description, it can be seen that the software system testing method provided by the embodiment of the present application can select a set of unit test cases and integration test cases based on the API dependency subgraph affected by the API change and the test case coverage graph to generate a test plan for the API change. The test cases in the test plan cover all APIs affected by the change and the calling relationships between APIs.

[0120] In order to further remove test cases with high repetition rate and high test cost and reduce the test cost, in one embodiment of the software system testing method provided in the present application, the API coverage map is also used to store the test cost of the corresponding test case, see Figure 8 , the software system testing method further specifically includes the following contents after step 210:

[0121] Step 224: If there are APIs with a total number of test cases greater than 1 in the API dependency subgraph, and the test cases corresponding to the APIs do not cover the hot subsystems and / or hot APIs, then the corresponding test case identifiers are selected and added to the test case set based on the test costs and API coverage corresponding to each of the test cases corresponding to the APIs.

[0122] Specifically, for the final test plan that has been selected, the subsystem and API coverage are calculated and compared with the API node list in the dependency subgraph of the changed API. The API list that has not been covered by the final test plan can be covered by multiple remaining test cases in the candidate test plan. At this time, test cases with low comprehensive test cost values ​​and high coverage are gradually selected to supplement the coverage of the remaining APIs until the remaining APIs are fully covered.

[0123] From the above description, it can be seen that the software system testing method provided in the embodiment of the present application can generate a test plan that can completely cover the changed API and the subsystems, APIs, and applications affected by the change, ensuring that the hot subsystems, APIs, and applications affected by the change can be covered by the test, removing test cases with high repetition rates and high testing costs, and reducing the testing costs.

[0124] In order to further improve the application reliability and effectiveness of the API dependency graph, an embodiment of the software system testing method provided in this application is provided. Fig. 9 , the software system testing method further specifically includes the following contents after step 012:

[0125] Step 310: receiving an API addition message for the software system, wherein the API addition message includes API metadata corresponding to the API to be added;

[0126] Step 320: According to the API metadata corresponding to the API to be added, directed edges between the node corresponding to the API to be added and other nodes having dependency relationships with the API to be added are added to the API dependency graph.

[0127] Specifically, when software developers add a new API to the software system, they register the new API with the API asset management module and provide API metadata. The API asset management module registers a new node in the API dependency graph for the new API, then traverses the API's call relationship list, finds the API's dependent API node list in the API dependency graph, establishes a directed edge pointing to the API for the API's dependent node, completes the registration of the new API, and completes the update and maintenance of the API dependency graph.

[0128] From the above description, it can be seen that the software system testing method provided in the embodiment of the present application can effectively improve the application reliability and effectiveness of the API dependency graph by adding nodes corresponding to the API to be added and directed edges between other nodes that have dependency relationships with the API to be added in the API dependency graph, and timely update and maintenance of the API dependency graph can be realized, thereby effectively improving the accuracy, comprehensiveness and reliability of finding the target test objects corresponding to the target API that has been changed in the software system, thereby further improving the test accuracy, comprehensiveness and reliability of the software system testing process.

[0129] In order to further improve the application reliability and effectiveness of the API dependency graph, an embodiment of the software system testing method provided in this application is provided. Fig.10 , the software system testing method further specifically includes the following contents after step 012:

[0130] Step 410: receiving an API deletion message for the software system, wherein the API deletion message includes API metadata corresponding to the API to be deleted;

[0131] Step 420: According to the API metadata corresponding to the API to be deleted, delete the node corresponding to the API to be deleted and the directed edges between other nodes that have dependency relationships with the API to be deleted in the API dependency graph.

[0132] Specifically, when a software developer deletes an API in a software system, the API asset management module traverses the API's dependent API nodes based on the API's metadata and deletes the edges of the dependent API nodes pointing to the changed API. After all dependent API nodes have been traversed, the API node is deleted to complete the update and maintenance of the API dependency graph.

[0133] From the above description, it can be seen that the software system testing method provided in the embodiment of the present application can effectively improve the application reliability and effectiveness of the API dependency graph by deleting the nodes corresponding to the API to be deleted and the directed edges between other nodes that have dependency relationships with the API to be deleted in the API dependency graph, and timely realize the update and maintenance of the API dependency graph, thereby effectively improving the accuracy, comprehensiveness and reliability of finding the target test objects corresponding to the target API that has been changed in the software system, thereby further improving the test accuracy, comprehensiveness and reliability of the software system testing process.

[0134] In order to further improve the application reliability and effectiveness of the API dependency graph, an embodiment of the software system testing method provided in this application is provided. Fig.11, the software system testing method further specifically includes the following contents after step 012:

[0135] Step 510: Receive an API modification message for the software system, wherein the API modification message includes API modification metadata of the API to be modified.

[0136] Step 520: Determine the content to be modified that is affected by the modification of the API to be modified based on the original API metadata corresponding to the API to be modified in the API dependency graph and the API modification metadata, wherein the content to be modified includes at least one of the dependent APIs of the API to be modified, the dependency relationship between the target API to be deleted and other APIs, and the dependency relationship between the target API to be added and other APIs.

[0137] Step 530: Based on the content to be modified, in the API dependency graph, modify the directed edges between other nodes that have dependency relationships with the API to be modified.

[0138] Specifically, when software developers submit API changes to the software system, they submit updated API metadata to the API asset management module. The API asset management module compares the API call relationship lists in the new and old API metadata, and finds the affected dependent API nodes. For API call relationships deleted in the change, the API asset management module deletes the edges in the original dependent API nodes pointing to the changed API nodes; for API call relationships newly added in the change, the API asset management module adds edges from the dependent API nodes pointing to the changed API nodes. After completing the reconstruction of the API dependency graph, the API asset management module overwrites the old API metadata with the new API metadata to complete the update and maintenance of the API dependency graph.

[0139] From the above description, it can be seen that the software system testing method provided in the embodiment of the present application can effectively improve the application reliability and effectiveness of the API dependency graph by modifying the directed edges between other nodes that have dependency relationships with the API to be modified in the API dependency graph, and timely update and maintenance of the API dependency graph, thereby effectively improving the accuracy, comprehensiveness and reliability of finding the target test objects corresponding to the target API that has been changed in the software system, thereby further improving the test accuracy, comprehensiveness and reliability of the software system testing process.

[0140] In order to further improve the API dependency graph, an embodiment of the software system testing method provided in this application is provided. Fig.12 , the software system testing method further specifically includes the following contents after step 012:

[0141] Step 610: Receive an API implicit call relationship record, wherein the API implicit call relationship record is pre-acquired from the backtracking accident cause data corresponding to the production accident log of the software system.

[0142] Step 620: Add a corresponding API implicit call relationship in the API dependency graph according to the API implicit call relationship record.

[0143] Specifically, in the production accident analysis, if it is found that there is no explicit calling relationship between APIs, but there is an implicit dependency, the software system developer or tester can use manual analysis methods to add implicit calling relationship records to the API metadata, and then update the API dependency graph through the method described in step 3.2 above. Implicit dependencies will also be involved in the generation of subsequent test plans to ensure that when some APIs are changed, the subsystems, APIs, and applications affected by the implicit dependencies of the API can also be covered by the test plan.

[0144] From the above description, it can be seen that the software system testing method provided in the embodiment of the present application can further improve the API dependency diagram by adding corresponding API implicit call relationships in the API dependency diagram. The implicit dependency relationship will also participate in the generation of subsequent test plans to ensure that when some APIs are changed, the subsystems, APIs, and applications affected by the API implicit dependency can also be covered by the test plan.

[0145] In order to further improve the API dependency subgraph, an embodiment of the software system testing method provided in this application is provided, see Fig.13 The software system testing method further specifically includes the following contents between step 120 and step 200:

[0146] Step 130: Extract hotspot objects from the logs generated by the production environment of the software system, wherein the hotspot objects include hotspot subsystems and / or hotspot APIs.

[0147] Step 140: Update the API dependency subgraph according to the hotspot object.

[0148] Specifically, hot subsystems and APIs are defined as subsystems and APIs with large user access volume, wide fault impact range, and high test priority. When the test coverage of multiple subsystems and APIs is the same, hot subsystems and APIs are given priority for testing, which can ensure that while covering key applications, the test time and test costs are solved. Hot subsystems and APIs can be automatically discovered from the log traffic information in the production data, or they can be manually marked. The automatic discovery of hot subsystems and APIs is not within the scope of discussion of the application examples of this application. The production data monitoring module is used to automatically or manually extract hot subsystems and APIs as a supplementary input to the test plan generation module.

[0149] From the above description, it can be seen that in the software system testing method provided by the embodiment of the present application, hotspot APIs and subsystems are core components or modules in the software system, which have a significant impact on the operation of the entire software system. Depending on the test phase and test requirements, hotspot APIs and subsystems can be added to the above API dependency subgraph so that the calculated test solution can cover the hotspot APIs and subsystems in the production environment.

[0150] From the software level, in order to solve the existing software system testing method, when the API changes, it is necessary to rely on manual or pre-defined mapping tables to determine the affected APIs, applications and subsystems waiting for test objects, which leads to the existing software system testing method having problems such as low test efficiency, high redundancy and poor test reliability and comprehensiveness caused by the omission of test objects. The present application provides an embodiment of a software system testing device for executing all or part of the contents of the software system testing method, see Fig.14 , the software system testing device specifically includes the following contents:

[0151] The graph query module 10 is used to search the target test object corresponding to the target API that is currently changed in the software system in the API dependency graph corresponding to the software system, wherein the target test object includes: the target API and all dependent APIs corresponding to the target API, and the applications or subsystems to which the target API and the dependent APIs respectively belong.

[0152] The testing module 20 is used to obtain a test case set corresponding to the target test object and generate a target test plan, so as to test the target test object in the software system based on the target test plan.

[0153] The embodiment of the software system testing device provided in the present application can be specifically used to execute the processing flow of the embodiment of the software system testing method in the above-mentioned embodiment. Its functions will not be repeated here, and reference can be made to the detailed description of the above-mentioned method embodiment.

[0154] From the above description, it can be seen that the software system testing device provided in the embodiment of the present application, by presetting the API dependency graph corresponding to the target software system, and when there is a changed target API in the target software system, searching in the API dependency graph for all dependent APIs that have a dependency relationship with the target API, as well as the application or subsystem to which the target API and the associated API respectively belong, can effectively improve the efficiency and automation of finding all test objects affected by the currently changed target API, thereby effectively improving the efficiency and automation of the software system testing process; at the same time, since there is no need to conduct a comprehensive test of the target software system, the redundancy of the software system testing can be further reduced; on this basis, it can also avoid the omission of the objects to be tested, effectively improving the comprehensiveness and reliability of the acquisition of the objects to be tested, so that on the basis of improving the test efficiency and reducing redundancy, the test reliability and comprehensiveness of the software system testing process can be improved, thereby improving the reliability and effectiveness of the software system testing results.

[0155] In order to further illustrate this solution and solve the current API interface testing problem, this application example provides a software system testing method implemented by an application software system testing system, which relates to the field of software testing technology and is an automated and semi-automated software system testing method for application programming interface API unit testing and system integration testing. The specific description is as follows:

[0156] (1) The application example of this application provides a method and apparatus for API asset management, which is used to establish and maintain the API list of the software system, each subsystem and each application, automatically discover the calling relationship between APIs based on API implementation and API metadata, and establish a directed cyclic graph of API dependencies. The API asset management library is updated as the API changes. When a new API is introduced or the calling relationship between APIs changes due to API changes, the API dependency graph is rebuilt.

[0157] (2) The application example of the present application provides a method and apparatus for test case management, which is used to maintain unit test cases and integration test cases within a software system, establish an API coverage map for test cases, and record the comprehensive cost of test cases. Unit test cases cover one or more APIs within a single subsystem; integration test cases cover multiple APIs between multiple subsystems. The API coverage map of the test case is a directed cyclic graph, which is a subgraph of the API dependency graph in the API asset management library. The comprehensive cost value of the test case is comprehensively assessed by the overhead of testers, test equipment, test running time, etc. The test case management library is updated with changes in the test cases and the API asset management library. When a test case changes, the API coverage map of the test case is updated according to the design and implementation of the test case.

[0158] (3) The application example of this application provides an automated generation method for an API test plan for an API change, including: querying the APIs, subsystems, and applications affected by the API change based on the API dependency graph, and calculating the dependency subgraph of the affected APIs, subsystems, and applications; selecting a set of unit test cases and integration test cases based on the API dependency subgraph affected by the API change and the test case coverage graph to generate a test plan for the API change. The test cases in the test plan cover all APIs affected by the change and the calling relationships between APIs, and the combined comprehensive cost of all test cases is the lowest.

[0159] (4) The application example of this application provides a test scheme verification and optimization method and device based on production data. The device extracts hot APIs and subsystems from the logs generated by the production environment. Hot APIs and subsystems are core components or modules in the software system, which have a significant impact on the operation of the entire software system. Depending on the test phase and test requirements, hot APIs and subsystems can be added to the above-mentioned API dependency subgraph, so that the calculated test scheme can cover the hot APIs and subsystems in the production environment.

[0160] (5) The application example of this application provides a production accident backtracking method, which is used to trace the cause of the accident from the production accident log, find missing API test points and API dependencies, and improve the test plan. There may be implicit dependencies between different APIs. Ignoring the implicit dependencies between APIs may result in the generated test plan being insufficient to cover all business processes. Production accident backtracking can discover the implicit calling relationship between APIs through manual or machine learning-based methods, add the implicit dependencies between APIs to the API asset management library and participate in the test plan generation calculation, thereby increasing the coverage of the test plan.

[0161] Based on the above content, the application example of this application is mainly composed of an API asset management module, a test case management module, a production accident backtracking module, a production data monitoring module, and a test plan generation module. The API asset management module records the call relationship information of all subsystems, APIs, and applications in the software system, and is used to generate an API dependency graph. The production accident backtracking module manually reviews the log information in the production accident, adds API hermit dependencies, and supplements and improves the API dependency graph. The test case management module records the test cases of unit testing and system integration testing in software testing, as well as the subsystems, APIs, and application information covered by each test case, and provides the comprehensive cost of the test case. The production data monitoring module collects information such as logs and access traffic of the production environment, and provides a list of hot subsystems and APIs. The test plan generation module takes the API dependency graph, the API coverage graph of the test case, the comprehensive cost of the test case, the hot API, and the subsystem list as input, automatically selects test cases according to the API that has changed, and generates a test plan.

[0162] See also Fig.15 The software system testing method implemented by the application software system testing system provided by the application example of this application specifically includes the following contents:

[0163] Step 1: Build API metadata. API metadata is provided by software system developers. The core content of API metadata is as follows: Fig.16 As shown, it specifically includes the definition of the calling relationship between APIs in the API metadata. Each API includes a unique identifier and one or more links to other APIs, which are used to identify the API list called (dependent) by the API.

[0164] Step 2: Build an API dependency graph. Traverse all APIs and metadata in the API asset management module and create a node for each API. If an API contains calls to other APIs, traverse the call links contained in the API, find the called API node, and create a directed edge from the called node to the calling node, indicating the dependency of the calling API node on the called API node, such as Fig.17 As shown in the figure, if API_1 calls API_2, changes on API_2 will affect the operation of API_1. Define API_1 and API_2 as two nodes in the directed cyclic graph of API dependencies, and there is a directed edge from API_2 to API_1.

[0165] Transitive nature of API dependencies ( Fig.18) is automatically created during the process of traversing all APIs. If the application APP calls API_1, and API_1 calls API_2, the changes on API_2 will be passed to API_1 and APP.

[0166] When all APIs registered in the API asset management module are traversed, an API dependency graph including all subsystems, APIs, and applications can be established, such as Fig.19 As shown in the figure. When any API changes, all APIs, subsystems, and applications affected by the API node change can be calculated through the directed edges between API nodes. This allows the API dependency subgraph to be generated based on the API dependency graph. If API_1 in subsystem 1 changes, the following can be generated: Fig. 20 The affected APIs and subsystems shown can be used as inputs to the test plan generation module to select test cases to cover nodes and edges.

[0167] Step 3: Update and maintain the API dependency graph. The update of the API dependency graph is usually triggered by API changes and completed by the API asset management module. API changes can include the addition of new APIs, changes to existing API dependencies, and deletion of APIs. Specifically, it includes the following steps 3.1 to 3.3.

[0168] Step 3.1: Add a new API. When a software developer adds a new API to the software system, he registers the new API with the API asset management module and provides API metadata. The API asset management module registers a new node in the API dependency graph for the new API, then traverses the API's call relationship list, finds the API's dependent API node list in the API dependency graph, establishes a directed edge pointing to the API for the API's dependent node, completes the registration of the new API, and completes the update and maintenance of the API dependency graph.

[0169] Step 3.2: Change the API. When software developers submit API changes to the software system, they submit updated API metadata to the API asset management module. The API asset management module compares the API call relationship lists in the new and old API metadata, and finds the affected dependent API nodes. For API call relationships deleted in the change, the API asset management module deletes the edges in the original dependent API nodes pointing to the changed API nodes; for API call relationships newly added in the change, the API asset management module adds edges from the dependent API nodes pointing to the changed API nodes. After completing the reconstruction of the API dependency graph, the API asset management module overwrites the old API metadata with the new API metadata to complete the update and maintenance of the API dependency graph.

[0170] Step 3.3: Delete API. When a software developer deletes an API in the software system, the API asset management module traverses the API's dependent API nodes based on the API's metadata and deletes the edges of the dependent API nodes pointing to the changed API. After all dependent API nodes have been traversed, the API node is deleted to complete the update and maintenance of the API dependency graph.

[0171] Step 4: Based on the production accident backtest defects, improve the API dependency graph. In the production accident analysis, if it is found that there is no explicit call relationship between APIs, but there is an implicit dependency, the software system developer or tester can use manual analysis methods to add implicit call relationship records to the API metadata, and then update the API dependency graph through the method described in step 3.2 above. Implicit dependencies will also be involved in the generation of subsequent test plans to ensure that when some APIs are changed, the subsystems, APIs, and applications affected by the implicit dependencies of the API can also be covered by the test plan.

[0172] Step 5: Discovery and annotation of hot subsystems and APIs. Hot subsystems and APIs are defined as subsystems and APIs with large user access volume, wide fault impact range, and high test priority. When the test coverage of multiple subsystems and APIs is the same, hot subsystems and APIs are given priority for testing, which can ensure that while covering key applications, the test time and test costs are solved. Hot subsystems and APIs can be automatically discovered from the log traffic information in the production data, or they can be manually labeled. The automatic discovery of hot subsystems and APIs is beyond the scope of discussion of the application examples of this application. The production data monitoring module is used to automatically or manually extract hot subsystems and APIs as a supplementary input to the test plan generation module.

[0173] Step 6: Build the API coverage map of the test case. The API coverage map of the test case is provided by the software tester, see Fig.21 The coverage graph of the test case is a subgraph of the API dependency graph, which indicates the API nodes that the test case can cover and the calls / dependencies between APIs. Usually, a unit test case can cover one or more APIs in a single subsystem, and a system integration test case can cover multiple APIs in multiple subsystems.

[0174] Step 7: Build a comprehensive cost list of test cases. The comprehensive cost of a test case is defined as the comprehensive cost of running the test case, which may include the running time of the test case, the system resources consumed to run the test case, the testers required to run the test case, etc. The comprehensive cost of the test case can be used to calculate the consumption of the test plan during the test plan generation process. When multiple test plans meet the test coverage requirements, the test plan with the lower comprehensive cost can be selected.

[0175] Step 8: Build a test plan. When some of the APIs in the software system change, you need to select test cases for the change and generate a test plan. The test plan needs to meet the following conditions: 1. The test plan needs to cover the changed API, as well as all subsystems, APIs, and applications that depend on the API; 2. The test plan needs to cover hot subsystems, APIs, and applications as much as possible; 3. The test plan needs to reduce testing costs while ensuring 1 and 2. The test plan takes the list of changed APIs, the API dependency diagram, the list of hot subsystems and APIs, the API coverage diagram of the test cases, and the comprehensive cost of the test cases as input, and outputs a list of test cases. The following steps can be used to build a test plan:

[0176] Step 8.1: Annotate the changed API. Fig.21 The list of changed APIs is provided by the developers of the software system, and the test plan generation module marks the changed APIs in the API dependency graph.

[0177] Step 8.2: Change the dependency subgraph of the API. In the API dependency graph, starting from the changed API node, traverse all dependent API nodes and application nodes, and mark the dependent API and application nodes. When the traversal is completed, all marked nodes constitute the dependency subgraph of the changed API.

[0178] Step 8.3: Construct candidate test plans. Candidate test plans can be defined as a collection of test cases that cover all API nodes in the dependency subgraph of the changed API. The combination of test cases in the candidate test plan can completely cover the changed API and the subsystems, APIs, and applications affected by the change, and there may be duplicate coverage for some APIs. Without considering the test cost, the test plan can be submitted to the test system for testing.

[0179] Step 8.4: Optimize the test plan. Considering the test cost and the need to select and remove duplicate test plans, optimize the test plan as follows:

[0180] 1. Traverse all API nodes in the dependency subgraph of the changed API, and for each API node, calculate the number of test cases and the list of test cases that can cover the API in the candidate test schemes;

[0181] 2. If the number of test cases covering a certain API node is 1, then this test case is selected as the only covered test case and added to the final solution;

[0182] 3. Traverse the remaining test cases. If the test case covers the hot subsystem and API, add the test case to the final test plan;

[0183] 4. For the final test plan that has been selected, calculate the subsystem and API coverage, and compare it with the API node list in the dependency subgraph of the changed API. The API list that has not been covered by the final test plan can be covered by multiple remaining test cases in the candidate test plan. At this time, gradually select test cases with low comprehensive test cost values ​​and high coverage to supplement the coverage of the remaining APIs until the remaining APIs are fully covered.

[0184] The test plan generated in this step can completely cover the changed API and the subsystems, APIs, and applications affected by the change, ensuring that the hot subsystems, APIs, and applications affected by the change can be covered by the test, removing test cases with high repetition rates and high testing costs, and reducing the overall cost of testing.

[0185] The software system testing method provided by the application example of this application has the following beneficial effects:

[0186] 1. Automatically generate API asset library and API dependency diagram based on API implementation and original data, avoiding the tedious manual configuration and maintenance of API, while improving accuracy.

[0187] 2. It can establish a complete API coverage map for test cases, improve the management and performance evaluation of test cases; and automatically generate the most optimized test plan through API dependency diagram, test case API coverage map, and test case comprehensive cost value. While ensuring full coverage of API and API call path, it selects the test case combination with the lowest comprehensive cost value to reduce test redundancy.

[0188] 3. A production data verification mechanism and a production accident backtracking mechanism can be established to introduce production operation data and production accident data into the test plan generation process to achieve sustainable growth in test efficiency.

[0189] From the hardware level, in order to solve the existing software system testing method, when the API changes, it is necessary to rely on manual or pre-defined mapping tables to determine the affected APIs, applications and subsystems to be tested, which leads to the existing software system testing method having problems such as low test efficiency, high redundancy, and poor test reliability and comprehensiveness due to omission of objects to be tested. The present application provides an embodiment of an electronic device for implementing all or part of the contents of the software system testing method, and the electronic device specifically includes the following contents:

[0190] Fig. 22FIG. 9 is a schematic block diagram of the system structure of the electronic device 9600 according to an embodiment of the present application. Fig. 22 As shown, the electronic device 9600 may include a central processor 9100 and a memory 9140; the memory 9140 is coupled to the central processor 9100. It is worth noting that Fig. 22 is exemplary; other types of structures may also be used to supplement or replace this structure to implement telecommunication functions or other functions.

[0191] In one embodiment, the software system test function may be integrated into the central processing unit. The central processing unit may be configured to perform the following control:

[0192] Step 100: Search the API dependency graph corresponding to the software system for the target test object corresponding to the target API that is currently changed in the software system, wherein the target test object includes: the target API and all dependent APIs corresponding to the target API, and the applications or subsystems to which the target API and dependent APIs respectively belong.

[0193] Step 200: Acquire a test case set corresponding to the target test object and generate a target test plan, so as to test the target test object in the software system based on the target test plan.

[0194] From the above description, it can be seen that the electronic device provided in the embodiment of the present application, by presetting the API dependency graph corresponding to the target software system, and when there is a changed target API in the target software system, searching in the API dependency graph for all dependent APIs that have a dependency relationship with the target API, as well as the application or subsystem to which the target API and the associated API respectively belong, can effectively improve the efficiency and automation of finding all test objects affected by the currently changed target API, thereby effectively improving the efficiency and automation of the software system testing process; at the same time, since there is no need to conduct a comprehensive test of the target software system, the redundancy of the software system testing can be further reduced; on this basis, it can also avoid the omission of the objects to be tested, effectively improving the comprehensiveness and reliability of the acquisition of the objects to be tested, so that on the basis of improving the test efficiency and reducing redundancy, the test reliability and comprehensiveness of the software system testing process can be improved, thereby improving the reliability and effectiveness of the software system testing results.

[0195] In another embodiment, the software system testing device may be configured separately from the central processing unit 9100. For example, the software system testing device may be configured as a chip connected to the central processing unit 9100, and the software system testing function may be implemented under the control of the central processing unit.

[0196] like Fig. 22As shown, the electronic device 9600 may also include: a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It is worth noting that the electronic device 9600 does not necessarily have to include Fig. 22 In addition, the electronic device 9600 may also include Fig. 22 For components not shown, reference may be made to the prior art.

[0197] like Fig. 22 As shown, the central processing unit 9100 is sometimes also referred to as a controller or an operation control, and may include a microprocessor or other processor device and / or logic device, which receives input and controls the operation of various components of the electronic device 9600.

[0198] The memory 9140 may be, for example, one or more of a cache, a flash memory, a hard drive, a removable medium, a volatile memory, a non-volatile memory or other suitable devices. The above-mentioned information related to the failure may be stored, and a program for executing the relevant information may also be stored. The CPU 9100 may execute the program stored in the memory 9140 to implement information storage or processing, etc.

[0199] The input unit 9120 provides input to the central processing unit 9100. The input unit 9120 is, for example, a key or a touch input device. The power supply 9170 is used to provide power to the electronic device 9600. The display 9160 is used to display display objects such as images and texts. The display may be, for example, an LCD display, but is not limited thereto.

[0200] The memory 9140 may be a solid-state memory, such as a read-only memory (ROM), a random access memory (RAM), a SIM card, etc. It may also be a memory that saves information even when the power is off, can be selectively erased, and is provided with more data, examples of which are sometimes referred to as EPROMs, etc. The memory 9140 may also be some other type of device. The memory 9140 includes a buffer memory 9141 (sometimes referred to as a buffer). The memory 9140 may include an application / function storage unit 9142, which is used to store application programs and function programs or processes for executing the operation of the electronic device 9600 through the central processor 9100.

[0201] The memory 9140 may also include a data storage unit 9143 for storing data, such as contacts, digital data, pictures, sounds, and / or any other data used by the electronic device. The driver storage unit 9144 of the memory 9140 may include various drivers for communication functions of the electronic device and / or for executing other functions of the electronic device (such as messaging applications, address book applications, etc.).

[0202] The communication module 9110 is a transmitter / receiver 9110 that sends and receives signals via an antenna 9111. The communication module (transmitter / receiver) 9110 is coupled to the central processor 9100 to provide input signals and receive output signals, which may be the same as the case of a conventional mobile communication terminal.

[0203] Based on different communication technologies, multiple communication modules 9110 may be provided in the same electronic device, such as a cellular network module, a Bluetooth module and / or a wireless LAN module, etc. The communication module (transmitter / receiver) 9110 is also coupled to a speaker 9131 and a microphone 9132 via an audio processor 9130 to provide an audio output via the speaker 9131 and receive an audio input from the microphone 9132, thereby realizing a common telecommunication function. The audio processor 9130 may include any suitable buffer, decoder, amplifier, etc. In addition, the audio processor 9130 is also coupled to the central processor 9100, so that recording can be performed on the local machine through the microphone 9132, and the sound stored on the local machine can be played through the speaker 9131.

[0204] The embodiments of the present application also provide a computer-readable storage medium capable of implementing all the steps in the software system testing method in the above embodiments. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, all the steps in the software system testing method in the above embodiments are implemented by the execution subject being a server or a client. For example, when the processor executes the computer program, the following steps are implemented:

[0205] Step 100: Search the API dependency graph corresponding to the software system for the target test object corresponding to the target API that is currently changed in the software system, wherein the target test object includes: the target API and all dependent APIs corresponding to the target API, and the applications or subsystems to which the target API and dependent APIs respectively belong.

[0206] Step 200: Acquire a test case set corresponding to the target test object and generate a target test plan, so as to test the target test object in the software system based on the target test plan.

[0207] From the above description, it can be seen that the computer-readable storage medium provided in the embodiment of the present application can effectively improve the efficiency and automation of finding all test objects affected by the currently changed target API by presetting the API dependency graph corresponding to the target software system, and when there is a changed target API in the target software system, searching in the API dependency graph for all dependent APIs that have a dependency relationship with the target API, as well as the application or subsystem to which the target API and the associated API respectively belong, thereby effectively improving the efficiency and automation of the software system testing process; at the same time, since there is no need to conduct a comprehensive test of the target software system, the redundancy of the software system testing can be further reduced; on this basis, it can also avoid the omission of the objects to be tested, effectively improving the comprehensiveness and reliability of the acquisition of the objects to be tested, so that on the basis of improving the test efficiency and reducing the redundancy, the test reliability and comprehensiveness of the software system testing process can be improved, thereby improving the reliability and effectiveness of the software system testing results.

[0208] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0209] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (apparatus), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0210] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0211] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0212] The present invention uses specific embodiments to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A software system testing method, characterized in that: include: Searching the target test object corresponding to the target API currently changed in the software system in the API dependency graph corresponding to the software system, wherein the target test object includes: the target API and all dependent APIs corresponding to the target API, and the applications or subsystems to which the target API and the dependent APIs respectively belong; Acquire a test case set corresponding to the target test object and generate a target test plan, so as to test the target test object in the software system based on the target test plan; The step of searching, in the API dependency graph corresponding to the software system, for a target test object corresponding to a target API currently changed in the software system includes: Receiving an API change message for the software system, wherein the API change message includes a unique identifier of a target API to which the change is sent and a change type, wherein the change type includes: adding, deleting, or modifying; Based on the unique identifier of the target API, searching for an API dependency subgraph corresponding to the target API in the API dependency graph corresponding to the software system, wherein the API dependency subgraph contains a target test object corresponding to the target API; The obtaining of the test case set corresponding to the target test object and generating a target test plan includes: Obtaining an API coverage map corresponding to each API in the target test object, wherein each API coverage map is in a one-to-one correspondence with each test case; According to the API dependency subgraph and the API coverage graph, select a test case set corresponding to the target test object; Generate a target test plan for the target test object based on the test case set; The selecting, according to the API dependency subgraph and the API coverage graph, a test case set corresponding to the target test object comprises: Determine, according to the API dependency subgraph and the API coverage graph, the identifiers and total number of test cases corresponding to each of the APIs in the API dependency subgraph; If there is an API with a total number of test cases of 1 in the API dependency subgraph, the identifier of the test case corresponding to the API with a total number of test cases of 1 is added to the test case set; If there are APIs with a total number of test cases greater than 1 in the API dependency subgraph, and the test cases corresponding to the APIs cover the preset hotspot subsystems and / or hotspot APIs, the identifiers of the test cases covering the preset hotspot subsystems and / or hotspot APIs will be added to the test case set, wherein the hotspot subsystem represents a subsystem with a large user access volume, a wide range of fault impact, and a high test priority, and the hotspot API represents an API with a large user access volume, a wide range of fault impact, and a high test priority.

2. The software system testing method according to claim 1, characterized in that: Before searching the API dependency graph corresponding to the software system for the target test object corresponding to the target API currently changed in the software system, the method further includes: Obtaining metadata of each API corresponding to the software system, wherein the API metadata is used to store a unique identifier of a uniquely corresponding API and a dependency relationship between the API and other APIs, and the API metadata is also used for an application or subsystem to which the uniquely corresponding API belongs; Each of the API metadata is taken as a node, and directed edges between each of the nodes are determined according to the dependency relationship between each of the APIs, so as to generate an API dependency graph consisting of each of the nodes and directed edges between each of the nodes.

3. The software system testing method according to claim 1, characterized in that: Before obtaining the test case set corresponding to the target test object and generating the target test plan, the method further includes: Obtain test cases corresponding to each subsystem or application in the software system, wherein the test cases include: unit test cases for covering APIs in a single subsystem or application, and integration test cases for covering APIs between multiple subsystems or applications; Generate an API coverage graph corresponding to each of the test cases according to the API dependency graph currently corresponding to the software system; The API coverage map corresponding to the unit test case is used to store the APIs in a single subsystem or application covered by the unit test case. If there are multiple APIs in a single subsystem or application, the API coverage map corresponding to the unit test case is also used to store the dependency relationships between the APIs in the single subsystem or application covered by the unit test case. The API coverage graph corresponding to the integration test case is used to store the APIs between multiple subsystems or applications covered by the integration test case and the dependencies between the APIs.

4. The software system testing method according to claim 1, characterized in that: The API coverage graph is also used to store the test cost of the corresponding test case; Correspondingly, the selecting a test case set corresponding to the target test object according to the API dependency subgraph and the API coverage graph further includes: If there are APIs with a total number of test cases greater than 1 in the API dependency subgraph, and the test cases corresponding to the APIs do not cover the hot subsystems and / or hot APIs, then the corresponding test case identifiers are selected and added to the test case set based on the test costs and API coverage corresponding to each of the test cases corresponding to the API.

5. The software system testing method according to claim 2, characterized in that: Also includes: Receive an API addition message for the software system, wherein the API addition message includes API metadata corresponding to the API to be added; According to the API metadata corresponding to the API to be added, directed edges between the node corresponding to the API to be added and other nodes having dependency relationships with the API to be added are added to the API dependency graph.

6. The software system testing method according to claim 2, characterized in that: Also includes: Receiving an API deletion message for the software system, wherein the API deletion message includes API metadata corresponding to the API to be deleted; According to the API metadata corresponding to the API to be deleted, the node corresponding to the API to be deleted and the directed edges between other nodes having dependency relationships with the API to be deleted are deleted in the API dependency graph.

7. The software system testing method according to claim 2, characterized in that: Also includes: Receiving an API modification message for the software system, wherein the API modification message includes API modification metadata of the API to be modified; Determine the content to be modified that is affected by the modification of the API to be modified according to the original API metadata corresponding to the API to be modified in the API dependency graph and the API modification metadata, wherein the content to be modified includes at least one of the dependent APIs of the API to be modified, the dependency relationship between the target API to be deleted and other APIs, and the dependency relationship between the target API to be added and other APIs; Based on the content to be modified, the directed edges between other nodes in the API dependency graph that have dependency relationships with the API to be modified are modified.

8. The software system testing method according to claim 2, characterized in that: Also includes: Receiving an API implicit call relationship record, wherein the API implicit call relationship record is obtained in advance from the backtracking accident cause data corresponding to the production accident log of the software system; According to the API implicit calling relationship record, a corresponding API implicit calling relationship is added to the API dependency graph.

9. The software system testing method according to claim 1, characterized in that: Also includes: Extracting hotspot objects from logs generated by the production environment of the software system, wherein the hotspot objects include hotspot subsystems and / or hotspot APIs; The API dependency subgraph is updated according to the hotspot object.

10. A software system testing device, characterized in that: include: A graph search module is used to search the API dependency graph corresponding to the software system for a target test object corresponding to the target API currently changed in the software system, wherein the target test object includes: the target API and all dependent APIs corresponding to the target API, and the applications or subsystems to which the target API and the dependent APIs respectively belong; A test module, used to obtain a test case set corresponding to the target test object and generate a target test plan, so as to test the target test object in the software system based on the target test plan; The graph search module searches the API dependency graph corresponding to the software system for a target test object corresponding to a target API currently changed in the software system, specifically for: Receiving an API change message for the software system, wherein the API change message includes a unique identifier of a target API to which the change is sent and a change type, wherein the change type includes: adding, deleting, or modifying; Based on the unique identifier of the target API, searching for an API dependency subgraph corresponding to the target API in the API dependency graph corresponding to the software system, wherein the API dependency subgraph contains a target test object corresponding to the target API; The test module obtains the test case set corresponding to the target test object and generates a target test plan, which is specifically used to: Obtaining an API coverage map corresponding to each API in the target test object, wherein each API coverage map is in a one-to-one correspondence with each test case; According to the API dependency subgraph and the API coverage graph, select a test case set corresponding to the target test object; Generate a target test plan for the target test object based on the test case set; The selecting, according to the API dependency subgraph and the API coverage graph, a test case set corresponding to the target test object comprises: Determine, according to the API dependency subgraph and the API coverage graph, the identifiers and total number of test cases corresponding to each of the APIs in the API dependency subgraph; If there is an API with a total number of test cases of 1 in the API dependency subgraph, the identifier of the test case corresponding to the API with a total number of test cases of 1 is added to the test case set; If there are APIs with a total number of test cases greater than 1 in the API dependency subgraph, and the test cases corresponding to the APIs cover the preset hotspot subsystems and / or hotspot APIs, the identifiers of the test cases covering the preset hotspot subsystems and / or hotspot APIs will be added to the test case set, wherein the hotspot subsystem represents a subsystem with a large user access volume, a wide range of fault impact, and a high test priority, and the hotspot API represents an API with a large user access volume, a wide range of fault impact, and a high test priority.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the software system testing method according to any one of claims 1 to 9 is implemented.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the software system testing method according to any one of claims 1 to 9 is implemented.

13. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the steps of the software system testing method according to any one of claims 1 to 9 are implemented.

Images