Fair method for selecting regression cases

Abstract

Methods, systems, and non-transitory processor-readable storage media for a test selection system are provided herein. An example method includes selecting a regression test case from a plurality of regression test cases in a software test lifecycle system. A test selection system calculates scores for regression test cases using weights assigned by an Analytical Hierarchical Process (AHP). Using the score, the test selection system visualizes a relationship between the regression test case and a target associated with the regression test job. The test selection system selects regression test cases for use in the regression test work based on the visualized relationship, and executes the regression test cases on the system.

Description

Technical Field

[0001] The field relates generally to optimizing selection of regression test cases, and more particularly to quantifying and optimizing selection of regression test cases in information processing systems. Background Art

[0002] System testing is an important part of the quality engineering activities before the release of information processing systems. There are two important parts in the system testing phase: new feature testing and regression test cases. For complex information processing systems, such as storage products, there may be thousands of system regression test cases. Summary of the invention

[0003] The illustrative embodiments provide techniques for implementing a test selection system in a storage system. For example, the illustrative embodiments select regression test cases from a plurality of regression test cases in a software testing lifecycle system. The test selection system calculates scores for regression test cases using weights assigned by an analysis hierarchy process (AHP). Using the scores, the test selection system visualizes the relationship between regression test cases and goals associated with regression testing work. The test selection system selects regression test cases for use in regression testing work based on the visualized relationship, and executes the regression test cases on the system. Other types of processing devices may be used in other embodiments. These and other illustrative embodiments include, but are not limited to, apparatus, systems, methods, and processor-readable storage media. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] Figure 1 An information handling system including a test selection system in an illustrative embodiment is shown.

[0005] Figure 2 A flow chart of a process for testing a selection system in an illustrative embodiment is shown.

[0006] Figure 3 A hierarchy diagram in an illustrative embodiment is shown.

[0007] Figure 4 A comparison table based on standards in an illustrative embodiment is shown.

[0008] Figure 5 A comparison table with normalized data in an illustrative embodiment is shown.

[0009] Fig. 6A and Figure 6B Feature coverage and test point coverage visualized in an elbow plot in an illustrative embodiment are shown.

[0010] Figure 7 and Figure 8An example of a processing platform that may be used to implement at least a portion of an embodiment of a test selection system is shown. DETAILED DESCRIPTION

[0011] Illustrative embodiments will be described herein with reference to an exemplary computer network and associated computers, servers, network devices, or other types of processing devices. However, it should be understood that these and other embodiments are not limited to use with the specific illustrative network and device configurations shown. Therefore, the term "computer network" as used herein is intended to be interpreted broadly to include, for example, any system including multiple networked processing devices.

[0012] A technique for implementing a test selection system is described below, which can be used to provide regression test case selection quantification and optimization, etc. by selecting regression test cases from multiple regression test cases in a software testing lifecycle system. The test selection system calculates a score for the regression test case using weights assigned by an analysis hierarchy process (AHP). Using the score, the test selection system visualizes the relationship between the regression test case and a goal associated with the regression test effort. The test selection system selects a regression test case for use in the regression test effort based on the visualized relationship, and executes the regression test case on the system.

[0013] The system testing phase of pre-release quality engineering activities for information processing systems may include thousands of regression test cases. Regression test cases must be executed to ensure that newly implemented features do not affect existing functionality. As mentioned above, there may be thousands of regression test cases. However, the resources to execute these regression test cases may be limited. Therefore, it is critical to select the most valuable set of regression test examples to execute from the set of regression test cases.

[0014] Traditional techniques do not provide a method to evaluate the value and weight of regression test cases. It may be difficult for engineers to select appropriate regression test cases, especially when they are not the owners of the regression test cases. Hardware resources and human resources may change and may be reduced due to product releases. The team may not have the bandwidth to rerun the entire set of regression test cases for a new product release. Traditional techniques do not provide a method to select the best regression test cases to adequately provide regression test coverage for new features in a new product release, especially when resources are limited. Traditional techniques do not quantify the number of best regression test cases and then select those best regression test cases. Traditional techniques measure the value of regression test cases based on a single factor, such as the number of defects detected by the regression test case or the amount of test code covered, and tend to ignore the value of regression test cases in other aspects. For example, a regression test case developed to test a new feature may be executed less often and therefore may detect fewer defects. In this example scenario, the value of the newly developed regression test case is measured based on the number of defects detected incorrectly, and the newly developed regression test case is evaluated as lower than its actual value. Traditional techniques do not consider multiple factors. Traditional techniques do not assign fair weights to each factor of the regression test case. Traditional techniques do not consider feature overlap, test point overlap, execution time, and regression test case creation time when evaluating regression test cases. Traditional techniques do not maximize some factors and minimize others when evaluating the overall value of regression test cases. Traditional techniques do not use the Analytical Hierarchy Process (AHP) to assign weights to the factors used to evaluate regression test cases.

[0015] In contrast, in at least some implementations of the current technology described herein, regression test case selection is optimized by selecting a regression test case from a plurality of regression test cases in a software testing lifecycle system. The test selection system calculates a score for the regression test case using weights assigned by an analysis hierarchy process (AHP). Using the score, the test selection system visualizes the relationship between the regression test case and a goal associated with the regression testing effort. The test selection system selects the regression test case for use in the regression testing effort based on the visualized relationship and executes the regression test case on the system.

[0016] Therefore, an object of the present technology is to provide a method and system for providing a test selection system that can quantify the optimal number of regression test cases and select those optimal regression test cases. Another object is to measure the value of regression test cases based on multiple weighted factors. Another object is to assign fair weights to each factor of regression test cases. Another object is to consider feature overlap, test point overlap, execution time, and regression test case creation time when evaluating regression test cases. Another object is to maximize some factors while minimizing other factors when evaluating the overall value of regression test cases. Yet another object is to use the Analysis Hierarchy Process (AHP) to assign weights to the factors used to evaluate regression test cases.

[0017] In at least some implementations according to the present technology described herein, use of a test selection system may provide one or more of the following advantages: quantifying an optimal number of regression test cases and selecting those optimal regression test cases; measuring the value of regression test cases based on multiple factors; assigning a fair weight to each factor of the regression test cases; considering feature overlap, test point overlap, execution time, and regression test case creation time when evaluating regression test cases; maximizing certain factors and minimizing other factors when evaluating the overall value of regression test cases; and using an analysis hierarchy process (AHP) to assign weights to factors used to evaluate regression test cases.

[0018] Compared to conventional techniques, in at least some implementations of the current techniques described herein, regression test cases are quantified and optimized by selecting regression test cases from a plurality of regression test cases in a software testing lifecycle system. The test selection system calculates scores for the regression test cases using weights assigned by an analysis hierarchy process (AHP). Using the scores, the test selection system visualizes the relationships between the regression test cases and the goals associated with the regression testing effort. The test selection system selects the regression test cases for use in the regression testing effort based on the visualized relationships, and executes the regression test cases on the system.

[0019] In an example embodiment of the present technology, a test selection system identifies a plurality of valuable factors associated with a regression test case.

[0020] In an example embodiment of the present technology, a test selection system normalizes multiple valuable factors associated with regression test cases.

[0021] In an example embodiment of the present technology, a test selection system normalizes multiple valuable factors using min-max normalization.

[0022] In an example embodiment of the present technology, a test selection system uses AHP to determine a respective weight associated with each of a plurality of valuable factors associated with a regression test case.

[0023] In an example embodiment of the present technology, a test selection system maximizes a subset of valuable factors associated with regression test cases.

[0024] In an example embodiment of the present technology, a subset of valuable factors includes feature coverage, code coverage, test point coverage, and defect list.

[0025] In an example embodiment of the present technology, a test selection system minimizes a subset of valuable factors associated with regression test cases.

[0026] In an example embodiment of the present technology, a subset of valuable factors includes feature overlap, test point overlap, execution time, and use case creation time.

[0027] In an example embodiment of the present technology, a test selection system calculates a corresponding score for each regression test case in a plurality of regression test cases.

[0028] In an example embodiment of the present technology, the test selection system uses the elbow rule to visualize this relationship.

[0029] In an example embodiment of the present technology, a test selection system ranks a plurality of scores associated with a plurality of regression test cases.

[0030] In an example embodiment of the present technology, a test selection system visualizes a subset of a plurality of regression test cases associated with a highest score among a plurality of ranked scores.

[0031] In an example embodiment of the present technology, a test selection system visualizes this relationship to determine an optimal number of regression test cases to execute on the system.

[0032] In an example embodiment of the present technology, a test selection system selects an optimal number of regression test cases to execute on a system.

[0033] In an example embodiment of the present technology, a test selection system selects an optimal number of regression test cases from a subset of a plurality of regression test cases, wherein the subset of regression test cases has a highest score among the plurality of regression test cases.

[0034] In an example embodiment of the present technology, a test selection system executes a subset of the plurality of regression test cases, wherein the subset of regression test cases has a highest score among the plurality of regression test cases.

[0035] In an example embodiment of the present technology, a test selection system validates selection of regression test cases for use in the regression test cases based on changes to a defect manifest.

[0036] Figure 1A computer network (also referred to herein as an information processing system) 100 configured in accordance with an illustrative embodiment is shown. The computer network 100 includes a software testing lifecycle system 101, a test selection system 105, and test systems 102-N. The software testing lifecycle system 101, the test selection system 105, and the test systems 102-N are coupled to a network 104, wherein the network 104 in this embodiment is assumed to represent a sub-network or other related portion of a larger computer network 100. Therefore, both elements 100 and 104 are referred to herein as examples of a "network," but the latter is not necessarily the same in this embodiment. Figure 1 Components of the former are assumed in the context of the embodiments. Also coupled to the network 104 is a test selection system 105 which may reside on a storage system. Such a storage system may include any of a variety of different types of storage, including network attached storage (NAS), storage area network (SAN), direct attached storage (DAS), and distributed DAS, as well as combinations of these and other storage types, including software defined storage.

[0037] Each of the test systems 102-N may include, for example, a server and / or a portion of one or more server systems, as well as devices such as mobile phones, laptop computers, tablet computers, desktop computers, or other types of computing devices. Such devices are examples of what are more generally referred to herein as "processing devices." Some of these processing devices are also generally referred to herein as "computers."

[0038] In some embodiments, test system 102-N includes respective computers associated with a particular company, organization, or other enterprise. In addition, at least portions of computer network 100 may also be referred to herein as collectively forming an "enterprise network." As will be appreciated by those skilled in the art, many other operational scenarios involving various types and arrangements of processing devices and networks are possible.

[0039] Furthermore, it should be understood that in this context and elsewhere herein, the term "user" is intended to be interpreted broadly to include, for example, a person, a hardware, software, or firmware entity, and various combinations of these entities.

[0040] The network 104 is assumed to comprise a portion of a global computer network such as the Internet, although other types of networks may be part of the computer network 100, including a wide area network (WAN), a local area network (LAN), a satellite network, a telephone or wired network, a cellular network, a wireless network such as a Wi-Fi or WiMAX network, or various portions or combinations of these and other types of networks. Thus, in some embodiments, the computer network 100 comprises a combination of multiple different types of networks, each of which includes processing devices configured to communicate using the Internet Protocol (IP) or other related communication protocols.

[0041] Also associated with the test selection system 105 are one or more input-output devices, which illustratively include a keyboard, a display, or any combination of other types of input-output devices. Such input-output devices may be used, for example, to support one or more user interfaces to the test selection system 105, as well as to support communication between the test selection system 105 and other related systems and devices not explicitly shown. For example, a dashboard may be provided for a user to view the execution progress of the test selection system 105. One or more input-output devices may also be associated with any of the test systems 102-N.

[0042] In addition, assuming Figure 1 The test selection system 105 in the embodiment is implemented using at least one processing device. Each such processing device generally includes at least one processor and associated memory, and implements one or more functional modules for controlling certain features of the test selection system 105.

[0043] More specifically, the test selection system 105 in this embodiment may include a processor coupled to a memory and a network interface.

[0044] The processor illustratively includes a microprocessor, a microcontroller, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other type of processing circuit, as well as portions or combinations of these circuit elements.

[0045] The memory illustratively includes any combination of random access memory (RAM), read only memory (ROM), or other types of memory. The memory and other memories disclosed herein may be viewed as examples of what is more generally referred to as a "processor-readable storage medium" that stores executable computer program code or other types of software programs.

[0046] One or more embodiments include articles of manufacture such as computer-readable storage media. Examples of articles of manufacture include, but are not limited to, storage devices such as storage disks, storage arrays, or integrated circuits containing memory, as well as various other types of computer program products. The term "article of manufacture" as used herein should be understood to exclude transient propagating signals. These and other references to "discs" mentioned herein are intended to refer generally to storage devices including solid-state drives (SSDs), and therefore should not be considered to be limited in any way to spinning magnetic media.

[0047] The network interface allows the test selection system 105 to communicate with the software testing lifecycle system 101 and the test systems 102 -N via the network 104 and includes, for example, one or more conventional transceivers.

[0048] The test selection system 105 may be implemented at least in part in the form of software stored in a memory and executed by a processor, and may reside in any processing device. The test selection system 105 may be a stand-alone plug-in that may be included within a processing device.

[0049] It should be understood that Figure 1 The specific set of elements of the test selection system 105 shown in the figure relating to the software testing lifecycle system 101 and the test systems 102-N of the computer network 100 are presented only as illustrative examples, and additional or alternative elements may be used in other embodiments. Therefore, another embodiment includes additional or alternative systems, devices, and other network entities, as well as different arrangements of modules and other components. For example, in at least one embodiment, one or more of the test selection systems 105 may be on the same processing platform and / or part of the same processing platform.

[0050] References such as Figure 2 The flowchart of FIG. 1 describes an exemplary process of the test selection system 105 in the computer network 100 in more detail.

[0051] Figure 2 is a flow chart of a process for executing the test selection system 105 in an illustrative embodiment. It should be appreciated that this particular process is merely an example and that additional or alternative processes may be performed in other embodiments.

[0052] At 200, the test selection system 105 selects regression test cases from a plurality of regression test cases in the software testing lifecycle system. In an exemplary embodiment, the test selection system 105 selects each regression test case from the plurality of regression test cases individually to evaluate each regression test case to identify an optimal subset of regression test cases for each product release.

[0053] At 202, the test selection system 105 calculates the score of the regression test case using the weights assigned by the analysis hierarchy process (AHP). In an example embodiment, the test selection system 105 considers multiple factors simultaneously, including test point overlap and feature overlap, as well as other related factors, and quantifies these factors. In an example embodiment, the test selection system 105 identifies multiple valuable factors associated with the regression test case. For example, the test selection system 105 quantifies eight valuable factors. In an example embodiment, the valuable factors include runtime, creation time (i.e., when the regression test case was developed), feature coverage (i.e., the number of covered features), feature overlap (i.e., the number of overlapping features), test point coverage (i.e., the number of covered test points), test point overlap (i.e., the number of overlapping test points), defect list, and covered test code (i.e., the lines of test code covered by the regression test case). In an example embodiment, when evaluating the creation time value factor, the test selection system 105 may value the more recently developed regression test case higher than the earlier developed regression test case.

[0054] In an example embodiment, the test selection system 105 constructs a hierarchical graph to obtain the weights of valuable factors. Figure 3 An example hierarchy diagram is shown. In an example embodiment, the test selection system 105 performs a consistency check before normalizing the data.

[0055] Figure 4 A comparison table based on valuable factors to ensure consistency is shown. Listed below Figure 4 The values ​​and standards used in , where "I" is the horizontal axis and "J" is the vertical axis.

[0056] value standard 1 I is as important as J 3 I is slightly more important than J 5 I is obviously more important than J 7 I is much more important than J 9 I is much more important than J 2-8 Median of adjacent comparisons 1 / n Comparison between J and I

[0057] In an example embodiment, the test selection system 105 normalizes the multiple valuable factors associated with the regression test case. In an example embodiment, after passing the consistency check, the test selection system 105 normalizes the data. Normalization can be performed in a variety of ways. In an example embodiment, the test selection system 105 normalizes the multiple valuable factors using min-max normalization. Figure 5 An example of normalized data is shown.

[0058] In an example embodiment, the test selection system 105 uses AHP to determine a corresponding weight associated with each valuable factor in a plurality of valuable factors associated with a regression test case. The test selection system 105 uses AHP to ensure fair weighting of the valuable factors and to assign an appropriate weight to each valuable factor. In an example embodiment, the test selection system 105 obtains the corresponding weight of each valuable factor after normalizing the data.

[0059]

[0060] In an example embodiment, the test selection system 105 maximizes a subset of valuable factors associated with regression test cases, such as feature coverage, code coverage, test point coverage, and defect list. In an example embodiment, the test selection system 105 minimizes a subset of valuable factors associated with regression test cases, such as feature overlap, test point overlap, execution time, and case creation time. In an example embodiment, the test selection system 105 uses the weights and the valuable factors to calculate a corresponding score for each of the multiple regression test cases. In an example embodiment, after obtaining the weight of each valuable factor, the test selection system 105 uses the weights and the valuable factors to calculate a corresponding score for each of the multiple regression test cases.

[0061] At 204, using the score, the test selection system 105 visualizes the relationship between the regression test case and the goal associated with the regression test work. In an example embodiment, the test selection system 105 uses the elbow method to visualize the relationship. For example, the test selection system 105 draws an elbow diagram to visualize the relationship between the number of regression test cases in the multiple regression test cases and the goals of the regression test work, such as test feature coverage and test code coverage. In an example embodiment, the test selection system 105 sorts the multiple scores associated with the multiple regression test cases. In other words, the test selection system 105 sorts the multiple regression test cases based on their corresponding scores. In an example embodiment, the test selection system 105 visualizes a subset of the multiple regression test cases associated with the highest score in the multiple sorted scores. The test selection system 105 selects the regression test case with the highest score to be visualized using the elbow method.

[0062] In an example embodiment, the test selection system 105 visualizes the relationship to determine the optimal number of regression test cases to execute on the system. In an example embodiment, the test selection system 105 performs the selection and visualization as an iterative process. In an example embodiment, the test selection system 105 draws the elbow graph by gradually increasing the number of regression test cases and measuring the different coverage achieved by selecting those regression test cases. Each iteration selects the regression test case with the highest score, so the test coverage gradually increases until it reaches 100%. Because the number of regression test cases is much higher than the number of features, the graph is drawn similar to an elbow, as shown in FIG. Fig. 6A and Figure 6BAs shown. In an example embodiment, the inflection point is the recommended number of regression test cases. The inflection point in the elbow graph represents the point where the marginal benefit of adding more regression test cases begins to significantly decrease. In an example embodiment, the recommended number of regression test cases is set to the number of regression test cases at the inflection point.

[0063] At 206, the test selection system 105 selects regression test cases for the regression testing effort based on the visualized relationship. In an example embodiment, the test selection system 105 determines an optimal number of regression test cases to be executed on the system. As described above, the test selection system 105 visualizes the relationship to determine the optimal number of regression test cases to be executed on the system. The test selection system 105 selects the optimal number of regression test cases from a subset of the plurality of regression test cases, wherein the subset of regression test cases has the highest score among the plurality of regression test cases. In other words, the test selection system 105 obtains the optimal number of regression test cases to be executed from the visualization, and then selects the number of regression test cases from the ranked list of regression test cases (i.e., the regression test cases with the highest scores) to be executed during the regression testing effort.

[0064] At 208, the software testing lifecycle system 101 executes regression test cases on the system. In an example embodiment, the software testing lifecycle system 101 executes a subset of multiple regression test cases, wherein the subset of regression test cases has the highest score among the multiple regression test cases. In an example embodiment, the test selection system 105 verifies the selection of regression test cases used in the regression testing work based on changes in the defect list. For example, when new features are incorporated into a product, the number of defect lists may show an increasing trend because the new features may introduce additional defects in the product. One goal of regression testing is to detect these newly introduced defects as soon as possible, such as before production, rather than after the product is released. Therefore, the number of defect lists becomes a convergent trend, wherein the number of defect lists first increases and then decreases over time. In this example scenario, selecting the best regression test cases can detect more newly introduced defects earlier.

[0065] In another example embodiment, the newly added feature may cause an additional 20 defects identified during regression testing. In this example scenario, the test selection system 105 verifies the selection of the regression test case based on the number of defects detected by the selected regression test case. For example, if the selected regression test case detects 17 of the 20 additional defects caused by the newly added feature, then the selected regression test case is at least one of the best regression test cases for testing the newly added feature.

[0066] Therefore, combined with Figure 2The specific processing operations and other functions described in the flowcharts are presented only as illustrative examples and should not be interpreted as limiting the scope of the present disclosure in any way. For example, in other embodiments, the order of the process steps can be changed, or some steps can be performed simultaneously with each other rather than sequentially.

[0067] The above illustrative embodiments provide significant advantages over traditional methods. For example, some embodiments are configured to significantly improve the quantification and identification of the best subset of regression test cases for each product release. Compared with traditional methods, these and other embodiments can effectively improve the regression testing for each product release. For example, the embodiments disclosed herein measure the value of regression test cases based on multiple factors. The embodiments disclosed herein assign fair weights to each factor of regression test cases. The embodiments disclosed herein consider feature overlap, test point overlap, execution time, and regression test case creation time when evaluating regression test cases. The embodiments disclosed herein maximize some factors and minimize other factors when evaluating the overall value of regression test cases. The embodiments disclosed herein use the analysis hierarchy process (AHP) to assign weights to factors used to evaluate regression test cases.

[0068] It should be understood that the specific advantages described above and elsewhere herein are associated with specific illustrative embodiments and need not exist in other embodiments. In addition, the specific types of information processing system features and functions shown in the drawings and described above are merely exemplary, and many other arrangements may be used in other embodiments.

[0069] As previously mentioned, at least a portion of the information handling system 100 can be implemented using one or more processing platforms. A given such processing platform includes at least one processing device, which includes a processor coupled to a memory. In some embodiments, the processor and memory include corresponding processors and memory elements of a virtual machine or container provided using one or more underlying physical machines. The term "processing device" used herein is intended to be broadly interpreted as including a variety of different physical processors, memory and other device components and virtual instances of these components. For example, in some embodiments, a "processing device" may include one or more virtual processors or execute across one or more virtual processors. Therefore, a processing device may be physical or virtual and may execute across one or more physical or virtual processors. It should also be noted that a given virtual device may be mapped to a portion of a physical device.

[0070] Some illustrative embodiments of a processing platform for implementing at least a portion of an information processing system include a cloud infrastructure that includes virtual machines implemented using a hypervisor running on a physical infrastructure. The cloud infrastructure also includes a collection of applications running on each virtual machine under the control of the hypervisor. Multiple hypervisors may also be used, each hypervisor providing a collection of virtual machines using at least one underlying physical machine. Different sets of virtual machines provided by one or more hypervisors may be used to configure multiple instances of various components of the system.

[0071] These and other types of cloud infrastructures can be used to provide what is also referred to herein as a multi-tenant environment. One or more system components, or portions thereof, are illustratively implemented for use by tenants of such a multi-tenant environment.

[0072] As previously mentioned, the cloud infrastructure disclosed herein may include a cloud-based system. In an illustrative embodiment, a virtual machine provided in such a system may be used to implement at least a portion of a computer system.

[0073] In some embodiments, the cloud infrastructure additionally or alternatively includes multiple containers implemented using a container host device. For example, as described in detail herein, a given container of the cloud infrastructure illustratively includes a Docker container or other type of Linux container (LXC). The container runs on a virtual machine in a multi-tenant environment, although other arrangements are possible. Containers are used to implement various different types of functions within the information processing system 100. For example, containers can be used to implement corresponding processing devices that provide computing and / or storage services for a cloud-based system. Similarly, containers can be used in combination with other virtualization infrastructures, such as virtual machines implemented using a hypervisor.

[0074] Now refer to Figure 7 and Figure 8 Illustrative embodiments of the processing platforms are described in further detail. Although described in the context of information handling system 100, in other embodiments, these platforms may also be used to implement at least a portion of other information handling systems.

[0075] Figure 7 An example processing platform is shown that includes a cloud infrastructure 700. The cloud infrastructure 700 includes a combination of physical and virtual processing resources for implementing at least a portion of the information processing system 100. The cloud infrastructure 700 includes a plurality of virtual machines (VMs) and / or container sets 702-1, 702-2, ... 702-L implemented using a virtualization infrastructure 704. The virtualization infrastructure 704 runs on a physical infrastructure 705 and illustratively includes one or more hypervisors and / or operating system-level virtualization infrastructures. The operating system-level virtualization infrastructure illustratively includes a kernel control group of a Linux operating system or other type of operating system.

[0076] The cloud infrastructure 700 also includes a set of applications 710-1, 710-2, ... 710-L running on a corresponding set of VM / container sets 702-1, 702-2, ... 702-L under the control of the virtualization infrastructure 704. The VM / container set 702 includes a corresponding VM, a corresponding set of one or more containers, or a corresponding set of one or more containers running in a VM. Figure 7 In some implementations of the embodiments, the VM / container set 702 includes respective VMs implemented using a virtualization infrastructure 704 including at least one hypervisor.

[0077] A hypervisor platform may be used to implement a hypervisor within the virtualization infrastructure 704, wherein the hypervisor platform has an associated virtual infrastructure management system.The underlying physical machines include one or more distributed processing platforms including one or more storage systems.

[0078] exist Figure 7 In other implementations of the embodiment, the VM / container set 702 includes corresponding containers implemented using a virtualization infrastructure 704, which provides operating system-level virtualization functions, such as support for Docker containers running on a bare metal host or Docker containers running on a VM. The container is illustratively implemented using a corresponding kernel control group of the operating system.

[0079] As can be seen from the above, one or more processing modules or other components of information processing system 100 can be executed on a computer, server, storage device or other processing platform element, respectively. Such elements are considered as examples of what is more generally referred to as a "processing device" in this article. Figure 7 The cloud infrastructure 700 shown may represent at least a portion of a processing platform. Another example of such a processing platform is Figure 8 Processing platform 800 is shown.

[0080] The processing platform 800 in this embodiment comprises a portion of the information processing system 100 and includes a plurality of processing devices, denoted as 802 - 1 , 802 - 2 , 802 - 3 , . . . 802 -K, which communicate with each other via a network 804 .

[0081] Network 804 includes any type of network including, for example, a global computer network such as the Internet, a wide area network, a local area network, a satellite network, a telephone or wired network, a cellular network, a wireless network such as a Wi-Fi or WiMAX network, or various portions or combinations of these and other types of networks.

[0082] The processing device 802 - 1 in the processing platform 800 includes a processor 810 coupled to a memory 812 .

[0083] Processor 810 includes a microprocessor, a microcontroller, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other types of processing circuits, as well as portions or combinations of these circuit elements.

[0084] The memory 812 includes any combination of random access memory (RAM), read-only memory (ROM), or other types of memory. The memory 812 and other memories disclosed herein should be viewed as illustrative examples of what is more generally referred to as a "processor-readable storage medium" that stores executable program code for one or more software programs.

[0085] Articles of manufacture including such processor-readable storage media are considered to be illustrative embodiments. A given such article of manufacture includes, for example, a storage array, a storage disk, or an integrated circuit containing RAM, ROM or other electronic memory, or any of various other types of computer program products. The term "article of manufacture" as used herein should be understood to exclude transient propagation signals. Many other types of computer program products including processor-readable storage media may be used.

[0086] Also included in the processing device 802-1 is a network interface circuit 814 for interfacing the processing device with the network 804 and other system components, and may include a conventional transceiver.

[0087] It is assumed that the other processing devices 802 of the processing platform 800 are configured in a manner similar to the processing device 802 - 1 shown in the figure.

[0088] Likewise, the specific processing platform 800 shown in the figure is presented only as an example, and information processing system 100 may include additional or alternative processing platforms, as well as any combination of many different processing platforms, each such platform including one or more computers, servers, storage devices or other processing devices.

[0089] For example, other processing platforms for implementing the illustrative embodiments may include different types of virtualization infrastructures, instead of or in addition to virtualization infrastructures including virtual machines. Such virtualization infrastructures illustratively include container-based virtualization infrastructures configured to provide Docker containers or other types of LXC.

[0090] As another example, in some embodiments, portions of a given processing platform may include a fused infrastructure.

[0091] Therefore, it should be understood that in other embodiments, different arrangements of additional or alternative elements may be used.At least a subset of these elements may be implemented collectively on a common processing platform, or each such element may be implemented on a separate processing platform.

[0092] Additionally, many other arrangements of computers, servers, storage products or devices, or other components are possible in information handling system 100. These components may communicate with other elements of information handling system 100 via any type of network or other communication medium.

[0093] For example, in the illustrative embodiments, specific types of storage products that may be used to implement a given storage system of a distributed processing system include all-flash and hybrid flash storage arrays, scale-out all-flash storage arrays, scale-out NAS clusters, or other types of storage arrays. In the illustrative embodiments, a combination of multiple of these and other storage products may also be used to implement a given storage system.

[0094] It should be emphasized again that the above-described embodiments are presented for illustrative purposes only. Many variations and other alternative embodiments may be used. In addition, the specific configurations of the system and device elements illustratively shown in the accompanying drawings and the associated processing operations may be changed in other embodiments. Thus, for example, the specific types of processing devices, modules, systems, and resources deployed in a given embodiment and their corresponding configurations may be different. In addition, the various assumptions made in the process of describing the illustrative embodiments above should also be regarded as exemplary, rather than requirements or limitations of the present disclosure. Many other alternative embodiments within the scope of the appended claims will be apparent to those skilled in the art.

Claims

1. A method comprising: A regression test case is selected by a test selection system from a plurality of regression test cases in a software testing lifecycle system; Calculating a score for the regression test case using the weights assigned by the Analysis Hierarchy Process (AHP); visualizing, by the test selection system, relationships between the regression test cases and goals associated with a regression testing effort using the scores; selecting, by the test selection system, the regression test case based on the visualized relationship for use in the regression testing work; as well as The regression test case is executed on a system, wherein the method is implemented by at least one processing device including a processor coupled to a memory.

2. The method of claim 1 , wherein calculating the score for the regression test case comprises: A plurality of valuable factors associated with the regression test case are identified.

3. The method of claim 1 , wherein calculating the score for the regression test case comprises: A plurality of valuable factors associated with the regression test case are normalized.

4. The method according to claim 3, wherein normalizing the plurality of valuable factors comprises: The plurality of valuable factors are normalized using min-max normalization.

5. The method of claim 1 , wherein calculating the score for the regression test case comprises: AHP is used to determine a respective weight associated with each of a plurality of valuable factors associated with the regression test case.

6. The method of claim 1 , wherein calculating the score for the regression test case comprises: A subset of valuable factors associated with the regression test case is maximized.

7. The method of claim 6, wherein the subset of valuable factors includes feature coverage, code coverage, test point coverage, and defect list.

8. The method of claim 1, wherein calculating the score for the regression test case comprises: Minimize a subset of valuable factors associated with the regression test case.

9. The method of claim 8, wherein the subset of valuable factors includes feature overlap, test point overlap, execution time, and use case creation time.

10. The method according to claim 1, wherein calculating the score of the regression test case comprises: A corresponding score is calculated for each regression test case in the plurality of regression test cases.

11. The method of claim 1 , wherein visualizing, by the test selection system, the relationship between the regression test cases and goals associated with a regression testing effort using the scores comprises: Use the elbow rule to visualize the relationship.

12. The method of claim 1 , wherein visualizing, by the test selection system, the relationship between the regression test cases and goals associated with a regression testing effort using the scores comprises: Sort multiple scores associated with multiple regression test cases.

13. The method according to claim 12, further comprising: A subset of the plurality of regression test cases associated with a highest score among a plurality of the ranked scores is visualized.

14. The method of claim 1 , wherein visualizing, by the test selection system, the relationship between the regression test cases and goals associated with a regression testing effort using the scores comprises: The relationship is visualized to determine an optimal number of regression test cases to execute on the system.

15. The method according to claim 1, wherein selecting, by the test selection system, the regression test case for use in the regression testing effort based on the visualized relationship comprises: Select the optimal number of regression test cases to execute on the system.

16. The method according to claim 15, further comprising: The optimal number of regression test cases is selected from a subset of a plurality of regression test cases, wherein the subset of regression test cases has a highest score among the plurality of regression test cases.

17. The method according to claim 1, executing the regression test case on the system comprises: A subset of a plurality of regression test cases is executed, wherein the subset of regression test cases has a highest score among the plurality of regression test cases.

18. The method according to claim 1, executing the regression test case on the system comprises: The selection of the regression test cases for use in the regression testing effort is validated based on changes in the defect inventory.

19. A system comprising: at least one processing device including a processor coupled to a memory; The at least one processing device is configured to: A regression test case is selected by a test selection system from a plurality of regression test cases in a software testing lifecycle system; Calculating a score for the regression test case using the weights assigned by the Analysis Hierarchy Process (AHP); visualizing, by the test selection system, relationships between the regression test cases and goals associated with a regression testing effort using the scores; The test selection system selects the regression test case based on the visualized relationship for use in the regression test work; as well as The regression test cases are executed on the system.

20. A computer program product comprising a non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code, when executed by at least one processing device, causes the at least one processing device to: A regression test case is selected by a test selection system from a plurality of regression test cases in a software testing lifecycle system; Calculating a score for the regression test case using the weights assigned by the Analysis Hierarchy Process (AHP); visualizing, by the test selection system, relationships between the regression test cases and goals associated with a regression testing effort using the scores; selecting, by the test selection system, the regression test case based on the visualized relationship for use in the regression testing work; as well as The regression test cases are executed on the system.

Images