System testing method and device, computer equipment, computer readable storage medium and computer program product

Abstract

The invention relates to a system testing method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: in the process of executing each test case of the distributed system, obtaining a call tree and a coverage rate of the test case, obtaining a case weight corresponding to the test case according to the coverage rate, clustering all the test cases according to the call tree to obtain multiple groups of clusters, and sending the multiple groups of clusters to the distributed system; and determining a test priority corresponding to each test case under each group of clusters according to the case weight, and carrying out regression testing on the distributed system based on the test priorities. By adopting the method, the distributed system can be tested more comprehensively and specifically, and the test time and resources are saved, so that the stability and reliability of the system are ensured.

Description

Technical Field

[0001] The present application relates to the field of security testing technology, and in particular to a system testing method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] Distributed systems achieve optimal resource allocation and efficient utilization by distributing storage and computing tasks across multiple physical or virtual nodes. This system architecture not only significantly improves system scalability, availability, and fault tolerance, but also fully utilizes heterogeneous resources and promotes resource sharing, thereby supporting more flexible and diverse service requirements.

[0003] Currently, testing methods for distributed systems mainly include black-box testing under business load. This testing method tests distributed systems by simulating user behavior and load in actual business scenarios. Testers do not focus on the specific implementation within the system, but instead verify the system's functions and performance from the user's perspective.

[0004] However, due to the complexity and uncertainty of distributed systems, simple black-box testing often fails to cover all possible paths and branches, resulting in a low hit rate of test cases. Summary of the Invention

[0005] Based on this, it is necessary to provide a system testing method, device, computer equipment, computer-readable storage medium and computer program product that can comprehensively and effectively implement distributed system testing in response to the above technical problems.

[0006] In a first aspect, the present application provides a system testing method, comprising:

[0007] During the execution of each test case in the distributed system, the call tree and coverage of the test case are obtained; the call tree is used to represent the hierarchical relationship of the test case, and the coverage is used to represent the degree of code coverage of the test case;

[0008] Get the test case weight corresponding to the test case based on coverage;

[0009] Cluster all test cases according to the call tree to obtain multiple clusters, and determine the test priority of each test case in each cluster according to the test case weight;

[0010] Regression testing of distributed systems based on test priorities.

[0011] In one embodiment, the step of obtaining a call tree of a test case includes:

[0012] Get the functions triggered during the execution of the test case;

[0013] The call tree of the test case is obtained according to the calling order of the functions; the root node of the call tree represents the initial function of the test case, and the child nodes represent the remaining functions of the test case; the node order of the call tree is consistent with the calling order.

[0014] In one embodiment, the coverage includes statistical values ​​of multiple dimensions; and the step of obtaining the use case weight corresponding to the test case according to the coverage includes:

[0015] Assign corresponding weights to the statistical values ​​of each dimension based on the operating environment of the distributed system;

[0016] Perform weighted summation on the statistical values ​​of all dimensions and the corresponding weights to obtain the use case weight corresponding to the test case.

[0017] In one embodiment, the step of clustering all test cases according to the call tree to obtain multiple groups of clusters includes:

[0018] Calculate the similarity of all test cases based on the call tree, and construct a similarity matrix based on all the calculated similarities; each element of the similarity matrix represents the similarity between two call trees;

[0019] The similarity matrix is ​​clustered to obtain multiple groups of clusters.

[0020] In one embodiment, the step of determining the test priority corresponding to each test case in each cluster according to the test case weight includes:

[0021] For each cluster, all test cases in the cluster are sorted in descending order according to the test case weight to obtain the test priority ranking.

[0022] In one embodiment, the step of performing regression testing on a distributed system based on test priorities includes:

[0023] Determine the cluster to be tested based on the test requirements, and use the test case with the highest test priority in the cluster to be tested as the target test case;

[0024] Execute target use cases to regression test distributed systems.

[0025] In a second aspect, the present application further provides a system testing device, comprising:

[0026] The execution module is used to obtain the call tree and coverage of each test case during the execution of each test case in the distributed system; the call tree is used to represent the hierarchical relationship of the test case, and the coverage is used to represent the degree of code coverage of the test case;

[0027] The acquisition module is used to obtain the test case weight corresponding to the test case based on the coverage;

[0028] The clustering module is used to cluster all test cases according to the call tree to obtain multiple groups of clusters, and determine the test priority corresponding to each test case in each cluster according to the case weight;

[0029] A testing module for regression testing distributed systems based on test priorities.

[0030] In a third aspect, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements any one of the method steps in the first aspect when executing the computer program.

[0031] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which implements any one of the method steps in the first aspect when the computer program is executed by a processor.

[0032] In a fifth aspect, the present application further provides a computer program product, comprising a computer program, which implements any one of the method steps in the first aspect when executed by a processor.

[0033] The above-mentioned system testing method, apparatus, computer equipment, computer-readable storage medium and computer program product obtain the call tree and coverage of each test case, cluster the test cases according to the call tree, and determine the case weight and priority ranking of each cluster according to the coverage, so that testing resources can be more accurately invested in test cases with high system code coverage and involving key business logic, avoiding indiscriminate repeated testing of all test cases, saving testing time and resources, and thus testing the distributed system more comprehensively and specifically, timely discovering and solving problems that arise in the system during development and maintenance, and ensuring the stability and reliability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments of the present application or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying any creative work.

[0035] Figure 1 An application environment diagram of a system testing method in one embodiment;

[0036] Figure 2 1 is a flow chart of a system testing method according to an embodiment;

[0037] Figure 3 is a flow chart of a system testing method according to another embodiment;

[0038] Figure 4 is a structural block diagram of a system testing device in one embodiment;

[0039] Figure 5 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0041] The system testing method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store data that server 104 needs to process. The data storage system can be integrated with server 104, or placed on a cloud or other network server. Terminal 102 is used to obtain the call tree and coverage of each test case in the distributed system during execution, obtain the test case weight corresponding to the test case based on the coverage, cluster all test cases based on the call tree to obtain multiple clusters, and determine the test priority corresponding to each test case in each cluster based on the test case weight. Regression testing of the distributed system is performed based on the test priority. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart car devices, projectors, etc. Portable wearable devices can include smart watches, smart bracelets, head-mounted devices, etc. Head-mounted devices can include virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc. The server 104 may be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services.

[0042] In an exemplary embodiment, Figure 2 As shown, a system testing method is provided, which is applied to Figure 1 Taking the terminal 102 in the example as an example, the following steps 202 to 208 are included. Among them:

[0043] S202: During the execution of each test case of the distributed system, a call tree and coverage rate of the test case are obtained; the call tree is used to represent the hierarchical relationship of the test case, and the coverage rate is used to represent the degree of coverage of the code by the test case.

[0044] Optionally, a distributed system test case refers to a set of inputs, execution conditions, and expected outputs used to verify various characteristics of the distributed system, such as functionality, performance, and reliability. It serves as a specific example of testing whether the distributed system meets the requirements of the requirements specification. Executing a test case is the process of actually performing operations on the distributed system (or other software system) under test according to the steps, input data, and expected results described in the pre-designed test case documentation. During execution, a call tree and coverage are obtained for each test case. A call tree is a data structure that displays the hierarchical relationship and call sequence between test cases. In a distributed system, the implementation of a function often involves multiple sub-functions, which may be verified by different test cases. The purpose of the call tree is to clearly demonstrate how these test cases are interconnected and called. Coverage is a metric that measures the extent to which a test case exercises the distributed system code. It is primarily used to assess test adequacy, namely, whether the test case covers key parts of the system code, including statements, branches, functions, etc. By obtaining coverage, weak links in the test can be identified and any untested code can be determined.

[0045] S204: Obtain the test case weight corresponding to the test case according to the coverage.

[0046] Alternatively, in distributed systems, where functions are numerous and complex, and different functions have varying degrees of importance to the overall operation of the system, by introducing weights for test cases, testing resources can be allocated to high-weighted test cases corresponding to key functions. Furthermore, after introducing weights, the order of testing can be arranged based on the weights of the test cases. When resources are limited, test cases with higher weights can be prioritized, improving the efficiency of problem discovery.

[0047] S206: Clustering all test cases according to the call tree to obtain multiple clusters, and determining the test priority corresponding to each test case in each cluster according to the test case weight.

[0048] Optionally, all test cases can be clustered based on the hierarchical relationships of the call tree. This allows grouping test cases with similar hierarchical relationships or closely related business functions. Once multiple clusters are generated, the previously determined test case weights are combined to prioritize the individual test cases within each cluster. Test cases with higher weights are typically given higher priority within their clusters, meaning they are prioritized for subsequent regression testing and other tasks.

[0049] S208: Perform regression testing on the distributed system based on the test priority.

[0050] Optionally, regression testing involves re-executing previous test cases after system modifications (such as fixing a vulnerability or adding a new feature) to ensure that the modification has not introduced new issues. Regression testing should be performed on high-priority test cases first, following the previously determined test priorities. This allows for rapid verification of key system components and important functions potentially affected by the modification, allowing for the timely identification and resolution of potential issues.

[0051] In the above system testing method, by obtaining the call tree and coverage of each test case, clustering the test cases according to the call tree, and determining the case weight and priority ranking of each cluster according to the coverage, test resources can be more accurately invested in test cases with high coverage of system code and involving key business logic, avoiding indiscriminate repeated testing of all test cases, saving testing time and resources, and thus testing the distributed system more comprehensively and specifically, timely discovering and solving problems that arise in the system during development and maintenance, and ensuring the stability and reliability of the system.

[0052] In an exemplary embodiment, the step of obtaining a call tree of a test case includes: obtaining functions triggered during the execution of the test case; obtaining a call tree of the test case according to the calling order of the functions; the root node of the call tree represents the initial function of the test case, and the child nodes represent the remaining functions of the test case; the node order of the call tree is consistent with the calling order.

[0053] Optionally, when executing a test case, all functions called during the execution of the test case are recorded. This can be achieved by inserting a specific tracking tool into the code or using the debugging function provided by the programming language itself. When the test case is running, each time a function is called, it will be recorded. For all recorded functions, a call tree is constructed according to their calling order. The call tree is a tree-shaped data structure used to intuitively display the calling relationship between functions. The root node of the call tree represents the initial function of the test case, which is the starting point of the entire calling process. It is directly triggered by the test case, and the child nodes represent the remaining functions of the test case. These functions are called during the execution of the initial function or other parent node functions. In this way, it can be clearly seen how the system calls each function step by step to complete the entire function after the test case starts executing.

[0054] In this embodiment, by obtaining the call tree of the test case, the calling sequence and hierarchical relationship of the functions during the test case execution process can be clearly obtained, thereby providing accurate data support for subsequent clustering processing and further improving the accuracy of clustering.

[0055] In an exemplary embodiment, the coverage includes statistical values ​​of multiple dimensions; the step of obtaining the use case weight corresponding to the test case based on the coverage includes: assigning corresponding weights to the statistical values ​​of each dimension according to the operating environment of the distributed system; and performing weighted summation on the statistical values ​​and corresponding weights of all dimensions to obtain the use case weight corresponding to the test case.

[0056] Alternatively, in software testing, coverage is a comprehensive concept that can measure the degree of system code coverage by test cases from multiple perspectives. Common dimensions include line coverage, branch coverage, and instruction coverage. Line coverage, also known as statement coverage, refers to the percentage of executed code statements compared to the total number of code statements during test case execution. Branch coverage refers to the percentage of executed branches compared to the total number of branches during test case execution. Instruction coverage refers to the machine instructions in the program executed by the test case.

[0057] Furthermore, the operating environments of distributed systems are complex and diverse, and different environmental factors can influence the importance of various coverage dimensions. For example, in a distributed trading system with extremely strict real-time requirements, path coverage may be relatively more important, as different paths in the transaction processing process may involve different risk controls and transaction processes. In a distributed data storage system, statement coverage corresponding to data read and write operations may be given a higher weight. Based on the specific characteristics and operational requirements of the system, each coverage dimension is assigned a weight, and the overall weight of the test case is calculated through a weighted summation.

[0058] For example, assuming that the weights of the three dimensions of line coverage, branch coverage and instruction coverage are w1, w2, and w3 respectively, and the coverage rates obtained during the execution of a certain test case are c1, c2, and c3 respectively, then the weight T of the test case can be expressed as: T=w1×c1+w2×c2+w3×c3, where w1+w2+w3=w, and w is a constant.

[0059] In this embodiment, by considering multiple coverage dimensions and performing weighted summation, the coverage of the test cases on the distributed system can be measured more comprehensively and accurately, thereby accurately obtaining the weight of each test case and implementing targeted testing of the distributed system.

[0060] In an exemplary embodiment, clustering all test cases according to the call tree to obtain multiple groups of clusters includes: calculating similarities for all test cases according to the call tree, and constructing a similarity matrix based on all the calculated similarities; each element of the similarity matrix represents the similarity between two call trees; and clustering the similarity matrix to obtain multiple groups of clusters.

[0061] Optionally, a call tree represents the order and hierarchical relationships of function calls during the execution of a test case. When calculating the similarity between test cases, the similarity can be obtained based on the structural similarity and edit distance of the call trees. Structural similarity is a metric that measures the structural similarity between different call trees, focusing on structural characteristics such as the shape, hierarchical relationships, and the connection between nodes. Edit distance is a metric that quantifies the degree of difference between two call trees. It refers to the minimum number of operations required to transform one call tree into another through a series of insertions, deletions, and substitutions. The similarity between each test case and all other test cases is calculated. Assuming there are n test cases, the constructed similarity matrix is ​​an n×n matrix, where each element (i, j) in the matrix represents the similarity between the i-th test case and the j-th test case.

[0062] Specifically, various clustering algorithms can be used during clustering, such as hierarchical clustering and K-Means clustering. Taking hierarchical clustering as an example, it starts with each test case as a separate class and then gradually merges similar classes based on a similarity matrix. At each step, it finds the two most similar classes and merges them into a new class. This process continues until a stopping condition is met, such as reaching a predetermined number of clusters or the similarity between all classes falls below a certain threshold. After clustering, the test cases are divided into multiple groups, and the test cases within each group have a high degree of similarity in the structure of the call tree and the function call relationships.

[0063] Optionally, a specific label is assigned to each cluster. This label is used to identify a certain feature or attribute of the cluster. The cluster label can be set according to the common characteristics of the test cases contained in the cluster. Through the cluster label, the type of each cluster can be more intuitively understood, so that the test case group of the specified type can be quickly located in subsequent tests.

[0064] In this embodiment, similar test cases are grouped together through clustering, which can more accurately find test case groups related to specific functional modules or business processes. At the same time, different test resources can be allocated to different clusters to improve the effectiveness of the test.

[0065] In an exemplary embodiment, the step of determining the test priority corresponding to each test case in each group of clusters according to the use case weight includes: for each group of clusters, sorting all the test cases in the cluster in descending order of the use case weight to obtain a test priority ranking.

[0066] Optionally, the use case weight reflects the importance of the test case. The test cases in the cluster are arranged from large to small according to the weight, with the test cases with high weights placed in front to obtain the test priority ranking. In regression testing or testing after system updates, the use cases with high weights are tested first in order of priority, so that the core parts of the system can be quickly focused.

[0067] In this embodiment, by sorting according to the use case weights, the test priority ranking is obtained, which can more specifically discover potential problems of the system during the test process, give priority to the execution of high-weight test cases, and reduce the consumption of test resources.

[0068] In an exemplary embodiment, the steps of performing regression testing on a distributed system based on test priority include: determining a cluster to be tested according to test requirements, and taking the test case ranked highest in the test priority ranking of the cluster to be tested as a target test case; and executing the target test case to perform regression testing on the distributed system.

[0069] Optionally, during distributed system testing, testing requirements may arise from a variety of situations. For example, after a system undergoes a feature update, performance optimization, or security vulnerability fix, regression testing may be required to ensure that these changes have not introduced new issues. Based on the functional scope of the change or the modules that may be affected, the clusters that require testing are identified. The test cases ranked highest in the cluster are selected as target cases, which are considered the most important and most likely to uncover issues. Actual operations are performed on the distributed system, and during execution, the actual behavior of the test system is tested to ensure that it meets expectations.

[0070] For example, in distributed systems, data read and write operations are common requirements. These operations include write, read, modify, and delete. Due to the division of labor among team members and the different stages of system development, test cases may include Case 1: Write + Read, Case 2: Write + Read + Delete, Case 3: Write + Read + Modify + Delete, and Case 4: Write + Read + Modify + Read + Delete. Each of these four cases is meaningful. In actual testing, the test case count is large and the testing cycle is long. Running all four cases at once is inefficient. Furthermore, relying on manual identification and selection of test cases is very costly. By running all test cases in full iterations, these cases are automatically clustered, and Case 4 has the highest weight, followed by Case 3, Case 2, and Case 1. When performing regression testing or automated testing, a weight identifier can be specified to run only Case 4, ensuring coverage and reducing testing costs.

[0071] For example, taking the chaos test of a distributed block storage system as an example, scenario one is a large-scale multi-hard disk read and write test, and scenario two is a small-scale multiple execution of hard disk and server add and delete operations. After executing for a period of time, both test scenarios have the problem of process exit. By deeply comparing the execution paths of the two scenarios during the test process and using the similarity measurement method, it is found that the trajectories of the two different scenarios are highly similar, which can quickly assist in locating the trigger point and root cause of the problem. In addition, through multiple rounds of chaos testing, a reliable similarity model is gradually built. This model can effectively cluster chaos test scenarios, thereby avoiding the blind random injection of anomalies and improving the targetedness and efficiency of the test.

[0072] In this embodiment, when regression testing resources are limited, the testing resources can be quickly concentrated on the key functions and important scenarios that are most likely to be affected by the system changes. This allows regression testing to more specifically verify those functions that are likely to be affected by the changes, further improving testing efficiency and reducing repeated testing.

[0073] In an exemplary embodiment, Figure 3 As shown, a system testing method is provided, which includes the following steps:

[0074] In the process of executing each test case of the distributed system, the functions triggered during the execution of the test case are obtained; the call tree of the test case is obtained according to the calling order of the functions; the root node of the call tree represents the initial function of the test case, and the child nodes represent the remaining functions of the test case; the node order of the call tree is consistent with the calling order.

[0075] Get the coverage of the test cases. The call tree is used to represent the hierarchical relationship of the test cases, and the coverage is used to represent the degree of code coverage by the test cases. The coverage includes statistical values ​​of multiple dimensions.

[0076] According to the operating environment of the distributed system, a corresponding weight is assigned to the statistical value of each dimension; the statistical values ​​of all dimensions and the corresponding weights are weighted summed to obtain the use case weight corresponding to the test case.

[0077] The similarity of all test cases is calculated according to the call tree, and a similarity matrix is ​​constructed based on all the calculated similarities; each element of the similarity matrix represents the similarity between two call trees; the similarity matrix is ​​clustered to obtain multiple groups of clusters.

[0078] For each cluster, all test cases in the cluster are sorted in descending order according to the test case weight to obtain the test priority ranking.

[0079] According to the test requirements, the cluster to be tested is determined, and the test case ranked first in the test priority ranking of the cluster to be tested is used as the target test case; the target test case is executed to perform regression testing on the distributed system.

[0080] In this embodiment, by obtaining the call tree and coverage of each test case, clustering the test cases according to the call tree, and determining the case weight and priority ranking of each cluster according to the coverage, test resources can be more accurately allocated to test cases with high coverage of system code and involving key business logic, avoiding indiscriminate repeated testing of all test cases, saving testing time and resources, and thus testing the distributed system more comprehensively and specifically, promptly discovering and resolving problems that arise in the system during development and maintenance, and ensuring the stability and reliability of the system.

[0081] It should be understood that, although the steps in the flowcharts of the above embodiments are shown in sequence as indicated by the arrows, these steps are not necessarily performed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be performed in other orders. Moreover, at least a portion of the steps in the flowcharts of the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily to be performed in sequence, but can be performed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0082] Based on the same inventive concept, embodiments of the present application also provide a system testing device for implementing the aforementioned system testing method. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations in one or more system testing device embodiments provided below can be found in the above-described limitations on the system testing method and will not be further elaborated here.

[0083] In an exemplary embodiment, Figure 4 As shown, a system testing device is provided, comprising: an execution module 10, an acquisition module 20, a clustering module 30 and a testing module 40, wherein:

[0084] The execution module 10 is used to obtain the call tree and coverage of the test case during the execution of each test case of the distributed system; the call tree is used to represent the hierarchical relationship of the test case, and the coverage is used to represent the degree of coverage of the code by the test case.

[0085] The acquisition module 20 is used to obtain the test case weight corresponding to the test case according to the coverage.

[0086] The clustering module 30 is used to cluster all test cases according to the call tree to obtain multiple groups of clusters, and determine the test priority corresponding to each test case in each cluster according to the case weight.

[0087] The testing module 40 is used to perform regression testing on the distributed system based on the test priority.

[0088] In an exemplary embodiment, the execution module 10 is also used to obtain the functions triggered during the execution of the test case; obtain the call tree of the test case according to the calling order of the functions; the root node of the call tree represents the initial function of the test case, and the child nodes represent the remaining functions of the test case; the node order of the call tree is consistent with the calling order.

[0089] In an exemplary embodiment, the coverage includes statistical values ​​of multiple dimensions; the acquisition module 20 is also used to assign corresponding weights to the statistical values ​​of each dimension according to the operating environment of the distributed system; and the statistical values ​​of all dimensions and the corresponding weights are weighted summed to obtain the use case weight corresponding to the test case.

[0090] In an exemplary embodiment, the clustering module 30 is also used to calculate the similarity of all test cases based on the call tree, and construct a similarity matrix based on all the calculated similarities; each element of the similarity matrix represents the similarity between two call trees; the similarity matrix is ​​clustered to obtain multiple groups of clusters.

[0091] In an exemplary embodiment, the clustering module 30 is further configured to sort all test cases in each cluster in descending order of case weights to obtain a test priority ranking.

[0092] In an exemplary embodiment, the test module 40 is also used to determine the cluster to be tested according to the test requirements, and use the test case ranked first in the test priority ranking of the cluster to be tested as the target test case; execute the target test case to perform regression testing on the distributed system.

[0093] Each module in the system testing apparatus described above may be implemented in whole or in part through software, hardware, or a combination thereof. Each module may be embedded in or independent of a processor in a computer device in the form of hardware, or may be stored in a memory in the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0094] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 5 As shown. The computer device includes a processor, memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals via wired or wireless means, and the wireless means can be implemented via Wi-Fi, a mobile cellular network, near-field communication (NFC), or other technologies. When executed by the processor, the computer program implements a system testing method. The display unit of the computer device is used to form a visually visible image, and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device casing, or an external keyboard, touchpad or mouse.

[0095] Those skilled in the art will understand that Figure 5The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0096] In an exemplary embodiment, a computer device is provided, comprising a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the following steps when executing the computer program: obtaining a call tree and coverage of the test case during execution of each test case of a distributed system; the call tree is used to characterize the hierarchical relationship of the test case, and the coverage is used to characterize the degree of coverage of the code by the test case; obtaining a use case weight corresponding to the test case according to the coverage; clustering all test cases according to the call tree to obtain multiple groups of clusters, and determining the test priority corresponding to each test case under each group of clusters according to the use case weight; and performing regression testing on the distributed system based on the test priority.

[0097] In one embodiment, obtaining a call tree of a test case involved in executing a computer program by a processor includes: obtaining functions triggered during the execution of the test case; obtaining a call tree of the test case according to the calling order of the functions; the root node of the call tree represents the initial function of the test case, and the child nodes represent the remaining functions of the test case; the node order of the call tree is consistent with the calling order.

[0098] In one embodiment, the coverage includes statistical values ​​of multiple dimensions; obtaining the use case weight corresponding to the test case based on the coverage when the processor executes the computer program includes: assigning a corresponding weight to the statistical value of each dimension according to the operating environment of the distributed system; and performing weighted summation on the statistical values ​​and corresponding weights of all dimensions to obtain the use case weight corresponding to the test case.

[0099] In one embodiment, when a processor executes a computer program, clustering all test cases according to a call tree to obtain multiple groups of clusters includes: calculating similarities for all test cases according to the call tree, and constructing a similarity matrix based on all calculated similarities; each element of the similarity matrix represents the similarity between two call trees; and clustering the similarity matrix to obtain multiple groups of clusters.

[0100] In one embodiment, when a processor executes a computer program, the process involves determining the test priority corresponding to each test case in each group of clusters based on the use case weight, including: for each group of clusters, sorting all the test cases in the cluster in descending order of the use case weight to obtain a test priority sorting.

[0101] In one embodiment, a processor executing a computer program involves performing regression testing on a distributed system based on test priority, including: determining a cluster to be tested based on test requirements, and using the test case that is ranked highest in the test priority ranking of the cluster to be tested as a target test case; and executing the target test case to perform regression testing on the distributed system.

[0102] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented: in the process of executing each test case of a distributed system, a call tree and coverage rate of the test case are obtained; the call tree is used to characterize the hierarchical relationship of the test case, and the coverage rate is used to characterize the degree of coverage of the code by the test case; the use case weight corresponding to the test case is obtained according to the coverage rate; all test cases are clustered according to the call tree to obtain multiple groups of clusters, and the test priority corresponding to each test case under each group of clusters is determined according to the use case weight; and the distributed system is regression tested based on the test priority.

[0103] In one embodiment, obtaining a call tree of a test case involved in execution of a computer program by a processor includes: obtaining functions triggered during the execution of the test case; obtaining a call tree of the test case according to the calling order of the functions; the root node of the call tree represents the initial function of the test case, and the child nodes represent the remaining functions of the test case; the node order of the call tree is consistent with the calling order.

[0104] In one embodiment, the coverage includes statistical values ​​of multiple dimensions; when the computer program is executed by the processor, obtaining the use case weight corresponding to the test case based on the coverage includes: assigning a corresponding weight to the statistical value of each dimension according to the operating environment of the distributed system; and performing weighted summation on the statistical values ​​and corresponding weights of all dimensions to obtain the use case weight corresponding to the test case.

[0105] In one embodiment, when a computer program is executed by a processor, clustering processing is performed on all test cases according to a call tree to obtain multiple groups of clusters, including: calculating similarities for all test cases according to the call tree, and constructing a similarity matrix based on all calculated similarities; each element of the similarity matrix represents the similarity between two call trees; and clustering processing is performed on the similarity matrix to obtain multiple groups of clusters.

[0106] In one embodiment, when a computer program is executed by a processor, the method involves determining the test priority corresponding to each test case in each group of clusters based on the use case weight, including: for each group of clusters, sorting all test cases in the cluster in order from large to small according to the use case weight to obtain a test priority ranking.

[0107] In one embodiment, when a computer program is executed by a processor, regression testing of a distributed system based on test priority is performed, including: determining a cluster to be tested according to test requirements, and taking the test case ranked first in the test priority ranking of the cluster to be tested as a target test case; and executing the target test case to perform regression testing on the distributed system.

[0108] In one embodiment, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the following steps: obtaining a call tree and coverage of each test case in a distributed system; the call tree is used to characterize the hierarchical relationship of the test case, and the coverage is used to characterize the degree of code coverage by the test case; obtaining a use case weight corresponding to the test case based on the coverage; clustering all test cases based on the call tree to obtain multiple groups of clusters, and determining the test priority corresponding to each test case in each group of clusters based on the use case weight; and performing regression testing on the distributed system based on the test priority.

[0109] In one embodiment, obtaining a call tree of a test case involved in execution of a computer program by a processor includes: obtaining functions triggered during the execution of the test case; obtaining a call tree of the test case according to the calling order of the functions; the root node of the call tree represents the initial function of the test case, and the child nodes represent the remaining functions of the test case; the node order of the call tree is consistent with the calling order.

[0110] In one embodiment, the coverage includes statistical values ​​of multiple dimensions; when the computer program is executed by the processor, obtaining the use case weight corresponding to the test case based on the coverage includes: assigning a corresponding weight to the statistical value of each dimension according to the operating environment of the distributed system; and performing weighted summation on the statistical values ​​and corresponding weights of all dimensions to obtain the use case weight corresponding to the test case.

[0111] In one embodiment, when a computer program is executed by a processor, clustering processing is performed on all test cases according to a call tree to obtain multiple groups of clusters, including: calculating similarities for all test cases according to the call tree, and constructing a similarity matrix based on all calculated similarities; each element of the similarity matrix represents the similarity between two call trees; and clustering processing is performed on the similarity matrix to obtain multiple groups of clusters.

[0112] In one embodiment, when a computer program is executed by a processor, the method involves determining the test priority corresponding to each test case in each group of clusters based on the use case weight, including: for each group of clusters, sorting all test cases in the cluster in order from large to small according to the use case weight to obtain a test priority ranking.

[0113] In one embodiment, when a computer program is executed by a processor, regression testing of a distributed system based on test priority is performed, including: determining a cluster to be tested according to test requirements, and taking the test case ranked first in the test priority ranking of the cluster to be tested as a target test case; and executing the target test case to perform regression testing on the distributed system.

[0114] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), quantum computing-based data processing logic devices, artificial intelligence (AI) processors, and the like.

[0115] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0116] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A system testing method, characterized in that: The method comprises: In the process of executing each test case of the distributed system, a call tree and coverage rate of the test case are obtained; the call tree is used to characterize the hierarchical relationship of the test case, and the coverage rate is used to characterize the coverage degree of the test case on the code; Obtaining a test case weight corresponding to the test case according to the coverage; Clustering all test cases according to the call tree to obtain multiple groups of clusters, and determining the test priority corresponding to each test case in each group of clusters according to the test case weight; Regression testing is performed on the distributed system based on the test priority.

2. The method according to claim 1, characterized in that The obtaining of the call tree of the test case comprises: Obtaining the functions triggered during the execution of the test case; A call tree of the test case is obtained according to the calling order of the functions; the root node of the call tree represents the initial function of the test case, and the child nodes represent the remaining functions of the test case; the node order of the call tree is consistent with the calling order.

3. The method according to claim 1, characterized in that The coverage rate includes statistical values ​​of multiple dimensions; and obtaining the use case weight corresponding to the test case according to the coverage rate includes: According to the operating environment of the distributed system, a corresponding weight is assigned to the statistical value of each dimension; The statistical values ​​of all dimensions and the corresponding weights are weighted summed to obtain the test case weight corresponding to the test case.

4. The method according to claim 1, characterized in that: The clustering process is performed on all test cases according to the call tree to obtain multiple groups of clusters, including: Calculate the similarity of all test cases according to the call tree, and construct a similarity matrix according to all the calculated similarities; each element of the similarity matrix represents the similarity between two call trees; The similarity matrix is ​​clustered to obtain multiple groups of clusters.

5. The method according to claim 4, characterized in that Determining the test priority corresponding to each test case in each cluster according to the test case weight includes: For each group of clusters, all test cases in the cluster are sorted in descending order of the test case weights to obtain a test priority ranking.

6. The method according to claim 5, characterized in that The performing regression testing on the distributed system based on the test priority includes: According to the test requirements, determine the cluster to be tested, and use the test case ranked first in the test priority ranking of the cluster to be tested as the target test case; The target use case is executed to perform regression testing on the distributed system.

7. A system testing device, characterized in that: The device comprises: An execution module, used to obtain a call tree and coverage rate of each test case in the process of executing the test case of the distributed system; the call tree is used to characterize the hierarchical relationship of the test case, and the coverage rate is used to characterize the coverage degree of the test case to the code; An acquisition module, used for acquiring a test case weight corresponding to the test case according to the coverage; A clustering module, used to cluster all test cases according to the call tree to obtain multiple groups of clusters, and determine the test priority corresponding to each test case under each group of clusters according to the test case weight; A testing module is used to perform regression testing on the distributed system based on the test priority.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images