A test case screening method, device, equipment and storage medium
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINA MOBILE (SUZHOU) SOFTWARE TECH CO LTD
- Filing Date
- 2022-06-10
- Publication Date
- 2026-06-02
Smart Images
Figure CN116804967B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of software testing, and includes, but is not limited to, a test case screening method, apparatus, device, and storage medium. Background Technology
[0002] Software testing has become a crucial part of the software lifecycle and a vital means of ensuring software quality. In current technology, testers write test cases based on design and requirements documents to uncover defects in the software. During the test case design process, to maximize defect detection, testing techniques are employed to cover as many aspects of the delivered software's functionality as possible, thereby ensuring product quality.
[0003] However, in reality, after a product is launched, there are numerous iterations, both large and small. How to select the most effective test cases for each iteration has always been a difficult problem for testers, especially functional testers. If every iteration is tested with all test cases, although this would reduce missed functionalities, it's impractical due to the very short iteration cycles and limited manpower. Furthermore, selecting all test cases in each iteration inevitably leads to redundancy and significant waste. If test cases are selected for each iteration, since most functional testers are unaware of the code's content, their selection is primarily based on their personal understanding of the business logic. This is heavily influenced by personal experience, making accuracy difficult to guarantee, resulting in blind testing, high risk, and a high rate of missed tests. Summary of the Invention
[0004] In view of this, embodiments of this application provide a test case screening method, apparatus, device, and storage medium.
[0005] The technical solution of this application embodiment is implemented as follows:
[0006] In a first aspect, embodiments of this application provide a test case filtering method, the method comprising:
[0007] Obtain a full set of test cases, wherein the test cases in the full set of test cases are used to test N services of the cloud platform, where N is an integer greater than or equal to 1; determine the target service from the M services involved in the upgrade of the cloud platform, where M is an integer greater than or equal to 1 and less than or equal to N; use the Apriori algorithm to filter out target test cases that support the target service from the full set of test cases; add the target test cases to the test case set of the target service.
[0008] Secondly, embodiments of this application provide a test case screening device, the device comprising:
[0009] The module includes an acquisition module for acquiring a full set of test cases, wherein the test cases in the full set are used to test N services of the cloud platform, where N is an integer greater than or equal to 1; a determination module for determining a target service from M services involved in the upgrade of the cloud platform, where M is an integer greater than or equal to 1 and less than or equal to N; a filtering module for using the Apriori algorithm to filter out target test cases supporting the target service from the full set of test cases; and an addition module for adding the target test cases to the test case set of the target service.
[0010] Thirdly, embodiments of this application provide an electronic device, including a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor executes the program to implement the above-described method.
[0011] Fourthly, embodiments of this application provide a storage medium storing executable instructions for inducing a processor to execute the above-described method.
[0012] In this embodiment, the entire set of test cases is first obtained. Then, the target service is determined from the M services involved in the cloud platform upgrade. The Apriori algorithm is used to filter out target test cases supporting the target service from the entire set of test cases. Finally, the target test cases are added to the test case set of the target service. In this way, based on the characteristics of the cloud platform, the Apriori algorithm can be used to filter a set of test cases that can support the upgrade service from the entire set of test cases, and this set is used as the test case set for testing the upgraded cloud platform. This effectively avoids errors caused by experience in manual test case selection, improves testing efficiency, and reduces the risk of missed tests. Attached Figure Description
[0013] Figure 1 A schematic diagram of a cloud platform architecture provided in an embodiment of this application;
[0014] Figure 2 A schematic diagram illustrating the implementation process of a test case filtering method provided in an embodiment of this application;
[0015] Figure 3 A schematic diagram illustrating the implementation process of a test case filtering method provided in an embodiment of this application;
[0016] Figure 4 A schematic diagram illustrating the implementation process of a test case filtering method provided in an embodiment of this application;
[0017] Figure 5 A schematic diagram of the composition structure of a test case screening device provided in an embodiment of this application;
[0018] Figure 6 This is a schematic diagram of a hardware entity of an electronic device provided in an embodiment of this application. Detailed Implementation
[0019] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the specific technical solutions of the embodiments will be further described in detail below with reference to the accompanying drawings. The following embodiments are used to illustrate this application, but are not intended to limit the scope of this application.
[0020] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.
[0021] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.
[0022] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.
[0023] Figure 1 This is a schematic diagram of a cloud platform architecture provided in an embodiment of this application, such as... Figure 1 The diagram includes: an access presentation layer 11, a cloud management service layer 12, an OpenStack service management layer 13, and a basic equipment layer 14, wherein...
[0024] Access the presentation layer 11: Implement the front-end page of the portal website, namely the operation and management platform portal, which can be identified by Cfro.
[0025] Cloud Management Service Layer 12: Provides efficient operation and high availability services for common business operations, including at least one of the following services: Computation Management Service (Cbac-cal), Storage Management Service (Cbac-cin), Network Management Service (Cbac-net), Order Management Service (Cbac-ord), User Management Service (Cbac-usr), and Log Management Service (Cbac-oper);
[0026] OpenStack Service Management Layer 13: Provides services such as compute virtualization, software-defined storage, and software-defined networking, including at least one of the following services: OpenStack compute service (nova), OpenStack network service (neutron), and OpenStack storage service (cinder); OpenStack compute service can be identified by Cnov, OpenStack network service by Cneu, and OpenStack storage service by Ccin.
[0027] Infrastructure Layer 14: This mainly includes physical machines, network devices, and storage devices in the computing, storage, and network resource pools.
[0028] The cloud platform is built on OpenStack technology, which simplifies the process of enabling resources such as servers, virtual machines, and storage, and provides complete resource service capabilities and unified operation and maintenance management capabilities for computing, storage, network, and security products.
[0029] From the cloud platform architecture diagram Figure 1 It can be seen that the upgrade of the cloud platform mainly involves the upgrade of the access presentation layer, i.e., the front end; the upgrade of the cloud management service layer, i.e., the upgrade of the various modules of the data provided to the presentation layer; and the upgrade of the various modules of the OpenStack service management layer.
[0030] There is no specific method for selecting test cases for cloud platforms in the existing technology. Before each iteration test, the corresponding test cases need to be manually selected according to the different upgrade modules. In addition to the extra labor cost, there are also errors in the selection of test cases due to the different experience levels of individuals.
[0031] This application addresses the characteristics of the aforementioned cloud platforms by allowing each test case to be annotated in multiple dimensions based on whether it involves the front-end, various modules of the OpenStack service management layer, and various modules of the cloud management service layer. Therefore, the vector for each test case can be represented as TC. i= {Cnov, Cneu, Ccin, Cfro, Cbac-cal, Cbac-net, Cbac-cin, Cbac-usr, Cbac-ord, Cbac-oper}. Where Cnov indicates whether it involves OpenStack compute services, Cneu indicates whether it involves OpenStack network services, Ccin indicates whether it involves OpenStack storage services, Cfro indicates whether it involves platform front-end services, Cbac-cal indicates whether it involves cloud-managed compute services (compute management services), Cbac-net indicates whether it involves cloud-managed network services (network management services), Cbac-cin indicates whether it involves cloud-managed storage services (storage management services), Cbac-usr indicates whether it involves cloud-managed user services (user management services), Cbac-ord indicates whether it involves cloud-managed order services (order management services), and Cbac-oper indicates whether it involves cloud-managed log services (log management services). During implementation, all values are represented by 0 and 1. For example, if it is determined that the use case involves OpenStack computing services, the “Cnov” in the vector is set to 1; if it is determined that the use case does not involve OpenStack network services, the “Cneu” in the vector can be set to 0.
[0032] This application provides a test case filtering method applied to cloud platform testing, such as... Figure 2 As shown, the method includes:
[0033] Step S210: Obtain a full set of test cases, wherein the test cases in the full set of test cases are used to test N services of the cloud platform, and N is an integer greater than or equal to 1;
[0034] Testing cloud platforms, whether in unit testing or system testing, requires designing a certain number of test cases to verify the correctness of the system or program's operation. The basis for designing test cases is the testing requirements. First, using existing testing techniques, a complete set of test cases can be designed based on the requirements document. This set can be expanded or reduced according to actual needs, ultimately forming a complete test case set: TC = {TC1, TC2, ... TC}. q Here, the full test case set is the complete set of unfiltered test cases used to test the cloud platform software, specifically the N services of the cloud platform.
[0035] During implementation, the above N services include: OpenStack computing services, OpenStack network services, OpenStack storage services, platform front-end services, cloud-managed computing services, cloud-managed network services, cloud-managed storage services, cloud-managed user services, cloud-managed order services, and cloud-managed log services.
[0036] Step S220: Determine the target service from the M services involved in the cloud platform upgrade, where M is an integer greater than or equal to 1 and less than or equal to N;
[0037] During implementation, each cloud platform upgrade may involve M services out of N services. These M services are identified as target services. For each of these M target services, a set of test cases corresponding to each target service can be selected from the full set of test cases.
[0038] During implementation, the upgrade itemset P can be constructed as follows, based on the requirements of each cloud platform iteration upgrade:
[0039] P = {{nova}(OpenStack compute service upgrade), {cinder}(OpenStack storage service upgrade), {neutron}(OpenStack network service upgrade), {fornt}(platform front-end service upgrade), {bac-cal}(cloud management compute service upgrade), {bac-cin}(cloud management storage service upgrade), {bac-net}(cloud management network service upgrade), {bac-usr}(cloud management user service upgrade), {bac-ord}(cloud management order service upgrade), {bac-oper}(cloud management log service upgrade)}.
[0040] During implementation, the upgrade items in the upgrade item set can be determined first, that is, the target services identified from the M services involved in the cloud platform upgrade. Then, the target services in the upgrade item set are set to 1, and the remaining services are set to 0. For example, in the case of OpenStack compute service and OpenStack storage service among the M services involved in the cloud platform upgrade, the upgrade item set P can be set to {1, 1, 0, 0, 0, 0, 0, 0, 0, 0}.
[0041] Step S230: Use the Apriori algorithm to filter out target test cases that support the target service from the full set of test cases;
[0042] Here, the Apriori algorithm is the first and most classic association rule mining algorithm. It uses an iterative, layer-by-layer search method to find relationships between itemsets in a database to form rules. This process consists of joins (matrix-like operations) and pruning (removing unnecessary intermediate results). In this algorithm, an itemset is a set of items. The frequency of an itemset is the number of transactions containing that itemset. If an itemset satisfies the minimum support, it is called a frequent itemset. The basic idea of the Apriori algorithm is: first, find all frequent itemsets whose frequency is at least equal to the predefined minimum support. Then, generate strong association rules from these frequent itemsets; these rules must satisfy both minimum support and minimum confidence. Next, use the found frequent itemsets to generate the desired rules, producing all rules containing only items from the set, where each rule has only one item on the right-hand side (using the definition of a middle rule). Once these rules are generated, only those rules with a confidence greater than the user-given minimum are retained.
[0043] During implementation, the Apriori algorithm can be used to sequentially filter out target test cases that support each target service from the full set of test cases.
[0044] Step S240: Add the target test case to the test case set of the target service.
[0045] In this embodiment, the entire set of test cases is first obtained. Then, the target service is determined from the M services involved in the cloud platform upgrade. The Apriori algorithm is used to filter out target test cases supporting the target service from the entire set of test cases. Finally, the target test cases are added to the test case set of the target service. In this way, based on the characteristics of the cloud platform, the Apriori algorithm can be used to filter a set of test cases that can support the upgrade service from the entire set of test cases, and this set is used as the test case set for testing the upgraded cloud platform. This effectively avoids errors caused by experience in manual test case selection, improves testing efficiency, and reduces the risk of missed tests.
[0046] In some embodiments, step S230 above, "using the Apriori algorithm to filter out target test cases supporting the target service from the full set of test cases," is as follows: Figure 3 As shown, this can be achieved through the following steps:
[0047] Step S310: Determine the total number of test cases in the full set of test cases;
[0048] Step S320: Obtain the service support degree of the target service, wherein the service support degree is used to represent the percentage of the number of first test cases associated with the target service in the total number of test cases;
[0049] Here, we can first determine the service associated with each test case in the full test set, then determine the number of first test cases associated with the target service, and finally determine the percentage of the number of first test cases associated with the target service in the total number of test cases, so as to obtain the service support level of the target service.
[0050] During implementation, before determining the target service, the service support level for each of the N services can be determined first. For example, if the N services include OpenStack compute service, OpenStack network service, OpenStack storage service, platform front-end service, cloud-managed compute service, cloud-managed network service, cloud-managed storage service, cloud-managed user service, cloud-managed order service, and cloud-managed log service, the service support levels for the following services can be determined sequentially: OpenStack compute service, OpenStack network service, OpenStack storage service, platform front-end service, cloud-managed compute service, cloud-managed network service, cloud-managed storage service, cloud-managed user service, cloud-managed order service, and cloud-managed log service. Then, the M service support levels corresponding to the M services associated with the cloud platform upgrade service can be obtained from these N service support levels.
[0051] Step S330: If it is determined that the test cases to be judged in the full set of test cases involve the target service, obtain the test case support of the test cases to be judged, wherein the test case support is used to represent the proportion of the number of second test cases that are associated with the same service as the test cases to be judged in the total number of test cases;
[0052] During implementation, each test case in the full set of test cases can be identified and sorted. Then, based on the identification of the test cases, it can be determined whether the test cases to be judged involve the target service from the full set of test cases in turn.
[0053] If it is determined that the test case to be judged in the full set of test cases involves the target service, first determine the number of second test cases that are associated with the same service as the test case to be judged, and then determine the percentage of the number of second test cases that are associated with the same service as the test case to be judged in the total number of test cases, so as to obtain the test case support of the test case to be judged.
[0054] Step S340: Based on the service support and the use case support, determine the test case to be judged as the target test case.
[0055] During implementation, the confidence level of the test case to be judged can be determined using the confidence algorithm formula in the Apriori algorithm based on service support and test case support. Then, based on the confidence level, it can be determined whether the test case to be judged is the target test case.
[0056] In this embodiment, the total number of test cases in the full test case set is first determined; then, the service support level of the target service is obtained; if it is determined that the test case to be judged in the full test case set involves the target service, the test case support level of the test case to be judged is obtained; finally, based on the service support level and the test case support level, the test case to be judged is determined to be the target test case. In this way, based on the service support level and the test case support level, it is possible to effectively determine whether the test case to be judged is a target test case, that is, to effectively determine whether the test case to be judged can be added to the test case set of the target service.
[0057] In some embodiments, step S230 above, "using the Apriori algorithm to filter out target test cases that support the target service from the full set of test cases," further includes the following step:
[0058] Step 350: If it is determined that the test case to be judged does not involve the target service, then the test case to be judged is not the target test case.
[0059] During implementation, if it is determined that the test case to be judged does not involve the target service, that is, when the test case to be judged TC is determined... i With upgrade item P j In the case of no association, i.e., TC i With upgrade item P j Mutual exclusion ensures that the test case to be judged is not the target test case.
[0060] In this embodiment of the application, if it is determined that the test case to be judged does not involve the target service, the test case to be judged can be discarded.
[0061] In some embodiments, the above step S320, "obtaining the service support level of the target service," can be achieved through the following steps:
[0062] Step 321: Based on the K services of the cloud platform involved in each test case, label each test case with an N-dimensional vector. Each dimension of the N-dimensional vector corresponds to determining whether the test case is associated with a service. K is an integer greater than or equal to 1 and less than or equal to N.
[0063] Here, we can see from the Apriori algorithm that determining associations requires two steps: first, finding all frequency sets, and then generating strong association rules from these frequency sets. The algorithm's complexity is O(n^2). 4 To maximize the efficiency of the Apriori algorithm, this application treats each service in the upgraded services as a frequency set and sets its support to 1, meaning that the vector dimension corresponding to each test case is the same. The algorithm then identifies the test case itemsets that are strongly correlated with the comparison itemsets, resulting in a time complexity of only O(n^2). 2 This reduces the complexity of the traditional Apriori algorithm.
[0064] During implementation, the vector representation of the full test case set can be initialized through the log processing program. First, you can check the API call history of each test case in the weighted test case set. For example, if a test case has no API calls, it means that the test case is simply a front-end test case, involving one service of the cloud platform, with K set to 1. In this case, only Cfro needs to be set to 1 in the vector, meaning the vector representation of the test case is: TC i ={0,0,0,1,0,0,0,0,0,0}.
[0065] If a test case involves API calls, the corresponding backend log information can be obtained based on the called API. The service called is then labeled as 1. Taking the test case of creating a cloud host as an example, this test case involves the cloud management service's computing service, network service, order service, user service, and log service, as well as OpenStack's computing and network services, totaling seven services of the cloud platform. With K set to 7, the vector for this test case can be represented as: TC i ={1, 1, 0, 0, 1, 1, 0, 1, 1, 1}.
[0066] Initializing each test case in the full test case set yields a set of all test cases TC = (TC1, TC2, ..., TC...). n ), of which TC i ={Cnov, Cneu, Ccin, Cfro, Cbac-cal, Cbac-net, Cbac-cin, Cbac-usr, Cbac-ord, Cbac-oper}.
[0067] Step 322: Based on the vector of each test case, determine the number of first test cases associated with the target service in the full set of test cases;
[0068] During implementation, the number of first test cases associated with the target service in the full test case set can be determined based on the elements in the test case vector that describe the target service. For example, the number of test cases with a Cnov value of 1 in the test case vector can be determined. Since Cnov indicates whether OpenStack compute services are involved, this determines the number of test cases managed by OpenStack compute services.
[0069] Step 323: Divide the number of the first test cases by the total number of test cases to obtain the service support level of the target service.
[0070] For example, if the target service is an OpenStack computing service, the service support level of the nova service can be obtained using the following formula (1):
[0071]
[0072] In this embodiment, each test case is first labeled with an N-dimensional vector based on the K services of the cloud platform involved in each test case. Each dimension of the N-dimensional vector determines whether the test case is associated with a service. This allows the Apriori algorithm to achieve maximum efficiency while reducing the complexity of the traditional Apriori algorithm.
[0073] In this embodiment, based on the vector of each test case, the number of first test cases associated with the target service is determined in the full test case set; the service support level of the target service is obtained by dividing the number of first test cases by the total number of test cases. In this way, the service support level of the target service can be determined based on the vector of the test cases.
[0074] In some embodiments, the step S330 above, "obtaining the test case support degree of the test case to be judged", can be achieved through the following steps:
[0075] Step 331: Based on the vector of each test case, determine the number of second test cases in the full set of test cases that support the same test service as the test case to be judged;
[0076] During implementation, the number of second test cases in the full test case set that support the same test service as the test case to be judged can be determined based on the elements in the test case vector that describe the target service. For example, the number of test cases in the test case vector where only Cnov is 1 can be determined, which means the number of test cases in the full test case set that are only associated with OpenStack compute services.
[0077] Step 332: Divide the number of the second test cases by the total number of test cases to obtain the test case support degree of the test cases to be judged.
[0078] For example, the test case support of the i-th test case can be obtained using the following formula (2):
[0079]
[0080] In this embodiment of the application, based on the vector of each test case, the number of second test cases in the full set of test cases that support the same test service as the test case to be judged is determined; the number of second test cases is divided by the total number of test cases to obtain the test case support degree of the test case to be judged. In this way, the test case support degree of the test case to be judged can be determined based on the vector of the test cases.
[0081] In some embodiments, step S340 above, "determining the test case to be judged as the target test case based on the service support and the use case support," can be implemented through the following steps:
[0082] Step 341: Determine the target confidence level of the test case to be judged in relation to the target service based on the service support level and the test case support level;
[0083] Here, TC can be used. i The element corresponding to the upgrade term in the vector determines TC. i With upgrade item P j Whether it is related, i.e., TC i If the element of the upgrade term in the vector is 1, then the upgrade term is associated; TC i If the element of the upgrade term in the vector is 0, then the upgrade term is not associated.
[0084] During implementation, when determining the test cases (TCs) to be evaluated... i With upgrade item P j In the case of no correlation, i.e., TC i With upgrade item P j Mutually exclusive. If TC i With upgrade item P j If they are mutually exclusive, then the confidence level ρ equals 0, meaning it can be determined that the test to be judged is used in relation to the upgrade item P. j The lack of correlation.
[0085] After determining the test cases to be judged (TC) i With upgrade item P j In the case of association, the target confidence of the associated target service of the test case to be judged can be determined based on service support and test case support.
[0086] Step 342: If the target confidence level is determined to be greater than the preset confidence threshold, the test case to be judged is determined to be the target test case;
[0087] During implementation, confidence thresholds can be set according to actual needs. The higher the target confidence, the stronger the relevance of the use case to the service; the lower the target confidence, the weaker the relevance of the use case to the service.
[0088] If the target confidence level is greater than or equal to the confidence threshold, that is, if the test case to be judged satisfies the strength of the association relationship with the upgrade item, the test case to be judged can be designated as the target test case.
[0089] Step 343: If the target confidence level is less than the preset confidence threshold, determine that the test case to be judged is not the target test case.
[0090] During implementation, if the target confidence level is determined to be less than the confidence level threshold, that is, if the test case to be judged does not meet the correlation strength of the upgrade item, the test case to be judged can be discarded.
[0091] Similarly, since step S360 is in the process of determining the test case TC to be judged... i With upgrade item P j In the case of no association, i.e., TC i With upgrade item P j Mutually exclusive. That is, in this case, the confidence level ρ equals 0. The test case to be judged can be discarded.
[0092] In this embodiment, the target confidence level of the associated target service of the test case to be judged is first determined based on service support and test case support. If the target confidence level is greater than a preset confidence threshold, the test case to be judged is determined to be a target test case. If the target confidence level is less than the preset confidence threshold, the test case to be judged is determined not to be a target test case. In this way, it is possible to effectively determine whether the test case to be judged is a target test case based on the confidence level.
[0093] In some embodiments, step 341 above, "determining the target confidence level of the test case to be judged associated with the target service based on the service support and the use case support," can be implemented through a process:
[0094] The target confidence level is obtained by dividing the service support level by the use case support level.
[0095] During implementation, when determining the test cases (TCs) to be evaluated... i With upgrade item P j In the case of association, i.e., determining TC i With upgrade item Pj Related. If TC i With upgrade item P j If there is a correlation, the confidence level ρ is calculated using the following formula (3):
[0096]
[0097] Among them, T j For upgrade item P j Service support, PC i Test Case TC i Use case support. Due to TC i ∈P j Then P(TC) i P j )=P(TC i ) = P ci .
[0098] In this embodiment of the application, dividing the service support score by the use case support score can effectively obtain the target confidence score.
[0099] In some embodiments, the cloud platform services include at least one of the following: OpenStack computing services, OpenStack network services, OpenStack storage services, platform front-end services, cloud-managed computing services, cloud-managed network services, cloud-managed storage services, cloud-managed user services, cloud-managed order services, and cloud-managed log services. Thus, the above test case filtering method can be used to filter test cases corresponding to the above services.
[0100] Figure 4 This is a flowchart illustrating a test case selection method provided in an embodiment of this application, as shown below. Figure 4 As shown, it includes the following steps:
[0101] Step S401: Execute the log processing program;
[0102] Step S402: Initialize the test case set and obtain the full test case set TC, which includes i test cases;
[0103] Here, initializing the test case set can be achieved by labeling each test case in the full test case set with an N-dimensional vector, where each dimension of the N-dimensional vector corresponds to determining whether the test case is associated with a service. For example, each test case can be labeled with multiple dimensions based on whether it involves the front-end, various modules of the OpenStack service management layer, and various modules of the cloud management service layer. Then, the vector of each test case can be represented as TCi = {Cnov, Cneu, Ccin, Cfro, Cbac-cal, Cbac-net, Cbac-cin, Cbac-usr, Cbac-ord, Cbac-oper}. In this vector, Cnov indicates whether OpenStack compute services are involved, Cneu indicates whether OpenStack network services are involved, Ccin indicates whether OpenStack storage services are involved, Cfro indicates whether the platform front-end is involved, Cbac-cal indicates whether cloud-managed compute services are involved, Cbac-net indicates whether cloud-managed network services are involved, Cbac-cin indicates whether cloud-managed storage services are involved, Cbac-usr indicates whether cloud-managed user services are involved, Cbac-ord indicates whether cloud-managed order services are involved, and Cbac-oper indicates whether cloud-managed log services are involved. During implementation, values are represented using 0 and 1. For example, if it is determined that the use case involves OpenStack compute services, "Cnov" in the vector is set to 1; if it is determined that the use case does not involve OpenStack network services, "Cneu" in the vector can be set to 0.
[0104] During implementation, automated test case tools such as Robot Framework can be used. When executing each test case in the full test case set, first check the API calls for each test case. For example, if a test case has no API calls, it means that the test case is simply performing a front-end test. In this case, only Cfro in the test case vector needs to be set to 1. That is, the test case vector can be represented as: TC i ={0,0,0,1,0,0,0,0,0,0}.
[0105] If a test case involves API calls, the corresponding backend log information can be obtained based on the called API. The service being called is labeled as 1. Taking the test case of creating a cloud host as an example, this test case involves the cloud management service's computing service, network service, order service, user service, log service, as well as OpenStack's computing and network services. Therefore, the vector for this test case can be represented as: TC i ={1, 1, 0, 0, 1, 1, 0, 1, 1, 1}.
[0106] Initializing each test case in the full test case set yields a set of all test cases TC = (TC1, TC2, ..., TC...). n ), of which TC i ={Cnov, Cneu, Ccin, Cfro, Cbac-cal, Cbac-net, Cbac-cin, Cbac-usr, Cbac-ord, Cbac-oper}.
[0107] Here, we can see from the Apriori algorithm that determining associations requires two steps: first, finding all frequency sets, and then generating strong association rules from these frequency sets. The algorithm's complexity is O(n^2). 4 To maximize the efficiency of the Apriori algorithm, this application treats each service in the upgraded service as a frequency set and sets its support to 1. It then identifies the use case itemsets that are strongly correlated with the comparison itemsets, resulting in an algorithm complexity of only O(n^2). 2 This reduces the complexity of the traditional Apriori algorithm.
[0108] Step S403: Obtain the upgrade item set P, which includes j upgrade items;
[0109] During implementation, the upgrade itemset P can be constructed as follows, based on the upgrade requirements of each iteration:
[0110] P = {{nova}(OpenStack compute service upgrade), {cinder}(OpenStack storage service upgrade), {neutron}(OpenStack network service upgrade), {fornt}(platform front-end service upgrade), {bac-cal}(cloud management compute service upgrade), {bac-cin}(cloud management storage service upgrade), {bac-net}(cloud management network service upgrade), {bac-usr}(cloud management user service upgrade), {bac-ord}(cloud management order service upgrade), {bac-oper}(cloud management log service upgrade)}.
[0111] Each time, an upgrade item is determined in the upgrade item set. Then, the entire set of test cases is traversed, and test cases that are strongly associated with each upgrade item are determined from the combination of all test cases. These test cases are then used to form the test cases for each upgrade service.
[0112] Step S404: Calculate the service support level T for each upgrade item based on the full set of test cases and the upgrade item set. j and the test case support score (PC) for each test case i ;
[0113] Here, service support level T jThat is, calculate the percentage of the number of the first test cases associated with the service in the total number of test cases. For example, the service support level of the Nova service can be obtained using the following formula (1):
[0114]
[0115] Use Case Support PC i That is, calculate the percentage of the number of second test cases that are associated with the same service as the test case in the total number of test cases. For example, the test case support of the i-th test case can be obtained using the following formula (2):
[0116]
[0117] Step S405: Obtain upgrade item P from the upgrade item set. j ;
[0118] During implementation, an upgrade item P with a flag of 1 can be obtained from the upgrade item set P. j This means identifying a service associated with this iteration and upgrade.
[0119] Step S406, based on service support level T j and use case support PC i Determine the test cases to be judged and the upgrade item set P. j The target confidence level;
[0120] During implementation, in the test cases to be judged (TC) i With upgrade item P j In the case of no correlation, i.e., TC i Involving TC i With upgrade item P j Mutually exclusive. If TC i With upgrade item P j If they are mutually exclusive, then the confidence level ρ equals 0, meaning it can be determined that the test to be judged is used in relation to the upgrade item P. j If the test case is not related, proceed to step S409, which means discarding the test case to be judged.
[0121] After determining the test cases to be judged (TC) i With upgrade item P j In the case of association, i.e., determining TC i With upgrade item P j Related. If TC i With upgrade item P j If there is a correlation, the confidence level ρ is calculated using the following formula (3):
[0122]
[0123] Among them, Tj For upgrade item P j Service support, PC i Test Case TC i Use case support. Due to TC i ∈P j Then P(TC) i P j )=P(TC i ) = P ci .
[0124] Step S407: Determine the relationship between the target confidence level and the confidence threshold;
[0125] During implementation, confidence thresholds can be set according to actual needs. The higher the target confidence, the stronger the relevance of the use case to the service; the lower the target confidence, the weaker the relevance of the use case to the service.
[0126] Step S408: If the target confidence level is determined to be greater than or equal to the confidence threshold, add the test case to be judged to the upgrade item P. j Test Case Set TC upgrade ;
[0127] During implementation, if the target confidence level is determined to be greater than or equal to the confidence threshold, i.e., if the strength of the association between the test case to be judged and the upgrade item is determined, the test case to be judged can be added to the upgrade item P. j Test Case Set TC upgrade This is used to test the upgrade.
[0128] Step S409: If the target confidence level is determined to be less than the confidence level threshold, discard the test case to be judged.
[0129] During implementation, if the target confidence level is determined to be less than the confidence level threshold, that is, if the test case to be judged does not meet the correlation strength of the upgrade item, the test case to be judged can be discarded.
[0130] Step S410: Determine whether there are any test cases to be judged in the full set of test cases;
[0131] If it is determined that there is a test case to be judged in the full set of test cases, proceed to step S406 to select a new test case to be judged in order to determine whether the test case to be judged can be added to the test case set of the upgrade item; until it is determined that there is no test case to be judged in the test case set, proceed to step S411.
[0132] Step S411: If it is determined that there is no test case to be judged in the test case set, the upgrade item P is obtained. j Test Case Set TCupgrade ;
[0133] Step S412: Determine if there are any more upgrade items in the upgrade item set;
[0134] If an upgrade item is found to exist in the upgrade item set, proceed to step S405 to re-determine the test case set for an upgrade item; if no upgrade item is found to exist in the upgrade item set, end the test case screening process to complete the screening of test cases for all upgrade items involved in all version upgrades.
[0135] In this embodiment, based on the characteristics of the cloud platform, a vector representation of the test case set is constructed according to the upgrade modules of the platform modules. An improved Apriori algorithm is used to construct upgrade module itemsets. Iterative test cases for the corresponding upgrade modules are selected by finding test case sets strongly correlated with the upgrade itemsets. This proposes a method for selecting iterative test case sets based on a cloud platform. The test case sets are annotated in multiple dimensions according to the specific characteristics of the cloud platform. An improved Apriori algorithm is used to construct upgrade module itemsets. Iterative test cases for the corresponding upgrade modules are selected by finding test case sets strongly correlated with the upgrade itemsets. Using a quantitative method to select test cases effectively avoids errors caused by experience in manual test case selection, improves testing efficiency, and reduces the risk of missed tests.
[0136] Based on the foregoing embodiments, this application provides a test case screening device. The device includes various modules, each module includes sub-modules, and each sub-module includes units. It can be implemented by a processor in an electronic device; of course, it can also be implemented by specific logic circuits. In the implementation process, the processor can be a central processing unit (CPU), microprocessor (MPU), digital signal processor (DSP), or field programmable gate array (FPGA), etc.
[0137] Figure 5 This is a schematic diagram of the composition structure of the test case screening device provided in the embodiments of this application, as shown below. Figure 5 As shown, the device 500 includes:
[0138] The acquisition module 510 is used to acquire a full set of test cases, wherein the test cases in the full set of test cases are used to test N services of the cloud platform, and N is an integer greater than or equal to 1;
[0139] The determination module 520 is used to determine the target service from the M services involved in the upgrade of the cloud platform, where M is an integer greater than or equal to 1 and less than or equal to N;
[0140] The filtering module 530 is used to filter out target test cases that support the target service from the full set of test cases using the Apriori algorithm;
[0141] Add module 540, which is used to add the target test case to the test case set of the target service.
[0142] In some embodiments, the filtering module 530 includes a first determining submodule, a first obtaining submodule, a second obtaining submodule, and a second determining submodule. The first determining submodule is used to determine the total number of test cases in the full set of test cases. The first obtaining submodule is used to obtain the service support degree of the target service, wherein the service support degree represents the percentage of first test cases associated with the target service in the total number of test cases. The second obtaining submodule is used to obtain the test case support degree of the test case to be judged when it is determined that the test case to be judged in the full set of test cases involves the target service, wherein the test case support degree represents the percentage of second test cases associated with the test case to be judged in the total number of test cases. The second determining submodule is used to determine the test case to be judged as the target test case based on the service support degree and the test case support degree.
[0143] In some embodiments, the filtering module 530 further includes a third determining submodule, configured to determine that the test case to be judged is not the target test case if it is determined that the test case to be judged does not involve the target service.
[0144] In some embodiments, the second acquisition submodule includes an annotation unit, a first determination unit, and a first division operation unit, wherein the annotation unit is used to annotate each test case using an N-dimensional vector based on the cloud platform services involved in each test case, and each dimension of the N-dimensional vector corresponds to determining whether the test case is associated with a service; the first determination unit is used to determine the number of first test cases associated with the target service in the full set of test cases based on the vector of each test case; the first division operation unit is used to divide the number of first test cases by the total number of test cases to obtain the service support degree of the target service.
[0145] In some embodiments, the second acquisition submodule includes a second determining unit and a second division operation unit, wherein the second determining unit is used to determine, based on the vector of each test case, the number of second test cases in the full set of test cases that support the same test service as the test case to be judged; the second division operation unit is used to divide the number of second test cases by the total number of test cases to obtain the test case support degree of the test case to be judged.
[0146] In some embodiments, the second determining submodule includes a third determining unit, a fourth determining unit, and a fifth determining unit, wherein the third determining unit is configured to determine the target confidence level of the test case to be judged associated with the target service based on the service support level and the test case support level; the fourth determining unit is configured to determine the test case to be judged as the target test case if the target confidence level is greater than or equal to a preset confidence threshold; and the fifth determining unit is configured to determine that the test case to be judged is not the target test case if the target confidence level is less than the preset confidence threshold.
[0147] In some embodiments, the third determining unit is further configured to divide the service support score by the use case support score to obtain the target confidence score.
[0148] In some embodiments, the cloud platform services include at least one of the following: OpenStack computing services, OpenStack network services, OpenStack storage services, platform front-end services, cloud-managed computing services, cloud-managed network services, cloud-managed storage services, cloud-managed user services, cloud-managed order services, and cloud-managed log services.
[0149] The descriptions of the above device embodiments are similar to those of the above method embodiments, and have similar beneficial effects. For technical details not disclosed in the device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.
[0150] It should be noted that, in the embodiments of this application, if the above methods are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to related technologies, can be embodied in the form of software products. These computer software products are stored in a storage medium and include several instructions to cause electronic devices (such as mobile phones, tablets, laptops, desktop computers, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware and software combination.
[0151] Correspondingly, embodiments of this application provide a storage medium on which a computer program is stored. When the computer program is executed by a processor, it implements the steps in the test case screening method provided in the above embodiments.
[0152] Correspondingly, embodiments of this application provide an electronic device, Figure 6 A schematic diagram of a hardware entity of an electronic device provided in an embodiment of this application, such as... Figure 6 As shown, the hardware entity of the device 600 includes a memory 601 and a processor 602. The memory 601 stores a computer program that can run on the processor 602. When the processor 602 executes the program, it implements the steps in the test case screening method provided in the above embodiments.
[0153] The memory 601 is configured to store instructions and applications executable by the processor 602, and can also cache data to be processed or already processed (e.g., image data, audio data, voice communication data and video communication data) of the processor 602 and various modules in the electronic device 600, and can be implemented by flash memory or random access memory (RAM).
[0154] It should be noted that the descriptions of the storage medium and device embodiments above are similar to the descriptions of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.
[0155] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above-described embodiments are merely descriptive and do not represent the superiority or inferiority of the embodiments.
[0156] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.
[0157] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.
[0158] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.
[0159] In addition, each functional unit in the various embodiments of this application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.
[0160] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.
[0161] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a mobile phone, tablet computer, laptop computer, desktop computer, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, magnetic disks, or optical disks.
[0162] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.
[0163] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.
[0164] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method or device embodiments.
[0165] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A test case selection method, characterized in that, The method, applied to cloud platform testing, includes: Obtain a full set of test cases, wherein the test cases in the full set of test cases are used to test N services of the cloud platform, where N is an integer greater than or equal to 1; The target service is determined from the M services involved in the cloud platform upgrade, where M is an integer greater than or equal to 1 and less than or equal to N; Determine the total number of test cases in the full test case set; Obtain the service support score of the target service, wherein the service support score is used to represent the percentage of the number of first test cases associated with the target service in the total number of test cases; If it is determined that the test cases to be judged in the full set of test cases involve the target service, the test case support of the test cases to be judged is obtained, wherein the test case support is used to represent the proportion of the number of second test cases that are associated with the same service as the test cases to be judged in the total number of test cases; Based on the service support and the use case support, the test case to be judged is determined as the target test case; Add the target test case to the test case set of the target service; The step of determining the test case to be judged as the target test case based on the service support and the test case support includes: If the target confidence level is greater than or equal to a preset confidence threshold, the test case to be judged is determined as the target test case; the target confidence level is obtained by dividing the service support level by the test case support level.
2. The method as described in claim 1, characterized in that, The method further includes: If it is determined that the test case to be judged does not involve the target service, then the test case to be judged is determined not to be the target test case.
3. The method as described in claim 1, characterized in that, The process of obtaining the service support level of the target service includes: Based on the cloud platform services involved in each of the test cases, each test case is labeled with an N-dimensional vector, and each dimension of the N-dimensional vector corresponds to determining whether the test case is associated with a service. Based on the vector of each test case, determine the number of first test cases associated with the target service in the full set of test cases; The service support level of the target service is obtained by dividing the number of the first test cases by the total number of test cases.
4. The method as described in claim 3, characterized in that, The process of obtaining the test case support degree of the test case to be judged includes: Based on the vector of each test case, determine the number of second test cases in the full set of test cases that support the same test service as the test case to be judged; Divide the number of the second test cases by the total number of test cases to obtain the test case support degree of the test case to be judged.
5. The method as described in claim 1, characterized in that, The step of determining the test case to be judged as the target test case based on the service support and the test case support further includes: The target confidence level of the test case to be judged is determined based on the service support and the test case support, and associated with the target service. If the target confidence level is determined to be less than the preset confidence threshold, then the test case to be judged is determined not to be the target test case.
6. The method according to any one of claims 1 to 5, characterized in that, The cloud platform services include at least one of the following: OpenStack computing services, OpenStack network services, OpenStack storage services, platform front-end services, cloud-managed computing services, cloud-managed network services, cloud-managed storage services, cloud-managed user services, cloud-managed order services, and cloud-managed log services.
7. A test case screening device, the device comprising: The acquisition module is used to acquire a full set of test cases, wherein the test cases in the full set of test cases are used to test N services of the cloud platform, and N is an integer greater than or equal to 1; The determination module is used to determine the target service from M services involved in the upgrade of the cloud platform, where M is an integer greater than or equal to 1 and less than or equal to N; A filtering module is used to determine the total number of test cases in the full set of test cases; obtain the service support score of the target service, wherein the service support score represents the percentage of first test cases associated with the target service in the total number of test cases; if it is determined that the test case to be judged in the full set of test cases involves the target service, obtain the test case support score of the test case to be judged, wherein the test case support score represents the percentage of second test cases associated with the same service as the test case to be judged in the total number of test cases; and determine the test case to be judged as the target test case based on the service support score and the test case support score. The add module is used to add the target test case to the test case set of the target service; The filtering module is further configured to determine the test case to be judged as the target test case when the target confidence level is greater than or equal to a preset confidence threshold; the target confidence level is obtained by dividing the service support level by the test case support level.
8. An electronic device comprising a memory and a processor, the memory storing a computer program executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method according to any one of claims 1 to 6.
9. A storage medium, characterized in that, It stores executable instructions for causing a processor to execute, thereby implementing the steps of the method according to any one of claims 1 to 6.