Test case scoring model training method and test case selection method

Abstract

The present invention discloses a test case scoring model training method and a test case selection method. The test case scoring model training method includes: determining multiple historical test cases for software testing; generating training samples based on the historical test cases, wherein the training samples include feature information of the historical test cases and the defect discovery rate of the historical test cases; and performing model training based on horizontal federated learning in conjunction with other participants based on the training samples. The test case scoring model finally obtained is used to predict the corresponding defect discovery rate based on the feature information of multiple alternative test cases, so as to select actual test cases from the multiple alternative test cases based on the predicted defect discovery rate. By combining multiple participants to realize training of the model based on a large amount of sample data, it can be used to predict the defect discovery rate of the test case, provide an effective reference for the selection of test cases, improve the efficiency of test case selection, and improve the efficiency of software testing.

Description

Technical Field

[0001] The present invention relates to the field of computers, and in particular to a test case scoring model training method and a test case selection method. Background Art

[0002] With the prevalence of smart devices, the types and amount of software installed on these devices are also increasing. Before software is released, it's necessary to test all its functions, which requires the use of test cases. A test case is a description of the testing tasks for a specific software product, embodying the test plan, methods, techniques, and strategies. It includes test objectives, test environment, input data, test steps, expected results, test scripts, and other content, ultimately resulting in documentation.

[0003] As software functionality increases, so do the number of test cases required. However, due to release time constraints, there may not be enough time to execute all test cases. This necessitates selecting a subset of test cases from the vast pool to conduct software testing as efficiently as possible.

[0004] For a project without sufficient testing experience, the test cases can only be selected based on the experience of the testers. The test cases selected in this way may not be applicable to the current project, resulting in low test case selection efficiency and low software testing efficiency. Summary of the Invention

[0005] The main purpose of the present invention is to provide a test case scoring model training method and a test case selection method, so as to train a test case scoring model based on a large amount of test data, realize automatic selection of test cases, and improve the test case selection efficiency and software testing efficiency.

[0006] To achieve the above objectives, in a first aspect, the present invention provides a test case scoring model training method, comprising:

[0007] Identify multiple historical test cases for software testing;

[0008] Generating a training sample based on the historical test case, wherein the training sample includes feature information of the historical test case and a defect discovery rate of the historical test case;

[0009] Based on the training samples, in collaboration with other participants, model training is performed based on horizontal federated learning. The final test case scoring model is used to predict the corresponding defect detection rate based on the feature information of multiple alternative test cases, so as to select actual test cases from multiple alternative test cases based on the predicted defect detection rate.

[0010] Optionally, the model training is performed based on horizontal federated learning in collaboration with other participants according to the training samples, including:

[0011] Get the initial test case scoring model parameters from the server;

[0012] Performing model training using the training samples to obtain a loss function gradient;

[0013] Sending the loss function gradient to the server so that the server aggregates and updates the loss function gradients sent by each participant;

[0014] Receive the updated loss function gradient sent by the server;

[0015] The model training is continued based on the updated loss function gradient, and the iteration is performed until the loss function converges to obtain a trained test case scoring model.

[0016] Optionally, the characteristics of the historical test cases include: software form, software business, use case type, module to which it belongs, use case status, whether the operation steps involve input boxes, whether the operation steps involve buttons, whether the operation steps involve zooming, whether the verification point includes page display, whether the verification point includes database verification, and whether the verification point includes log verification.

[0017] In a second aspect, the present invention provides a test case selection method, comprising:

[0018] Get multiple alternative test cases;

[0019] Predicting the defect discovery rate of the candidate test cases based on a trained test case scoring model; the test case scoring model is trained using the test case scoring model training method as described in the first aspect;

[0020] According to the predicted defect discovery rate, actual test cases are selected from multiple alternative test cases.

[0021] Optionally, selecting an actual test case from a plurality of candidate test cases based on the predicted defect discovery rate includes:

[0022] Sort multiple alternative test cases from high to low according to the defect discovery rate;

[0023] Select N alternative test cases with the highest defect discovery rate as actual test cases.

[0024] Optionally, selecting an actual test case from a plurality of candidate test cases based on the predicted defect discovery rate includes:

[0025] For each test case, set weights for defect discovery rate, importance, and execution time;

[0026] Determine the weighted sum of defect discovery rate, importance, and execution time;

[0027] Sort the plurality of candidate test cases from largest to smallest according to the weighted sum value;

[0028] Select the N alternative test cases with the largest weighted sum value as the actual test cases.

[0029] In a third aspect, the present invention provides a test case scoring model training device, comprising:

[0030] A historical test case determination module, used to determine multiple historical test cases for software testing;

[0031] A training sample generation module is used to generate training samples based on the historical test cases, wherein the training samples include feature information of the historical test cases and defect discovery rates of the historical test cases;

[0032] The training module is used to perform model training based on horizontal federated learning in conjunction with other participants based on the training samples. The test case scoring model finally obtained is used to predict the corresponding defect detection rate based on the feature information of multiple alternative test cases, so as to select actual test cases from multiple alternative test cases based on the predicted defect detection rate.

[0033] Optionally, the training module is specifically used to:

[0034] Get the initial test case scoring model parameters from the server;

[0035] Performing model training using the training samples to obtain a loss function gradient;

[0036] Sending the loss function gradient to the server so that the server aggregates and updates the loss function gradients sent by each participant;

[0037] Receive the updated loss function gradient sent by the server;

[0038] The model training is continued based on the updated loss function gradient, and the iteration is performed until the loss function converges to obtain a trained test case scoring model.

[0039] Optionally, the characteristics of the historical test cases include: software form, software business, use case type, module to which it belongs, use case status, whether the operation steps involve input boxes, whether the operation steps involve buttons, whether the operation steps involve zooming, whether the verification point includes page display, whether the verification point includes database verification, and whether the verification point includes log verification.

[0040] In a fourth aspect, the present invention provides a test case selection device, comprising:

[0041] An acquisition module is used to obtain multiple alternative test cases;

[0042] A prediction module, configured to predict the defect discovery rate of the candidate test case based on a trained test case scoring model; the test case scoring model is trained using the test case scoring model training method described in the first aspect;

[0043] The selection module is used to select actual test cases from multiple alternative test cases based on the predicted defect discovery rate.

[0044] Optionally, the selection module is specifically configured to:

[0045] Sort multiple alternative test cases from high to low according to the defect discovery rate;

[0046] Select N alternative test cases with the highest defect discovery rate as actual test cases.

[0047] Optionally, the selection module is specifically configured to:

[0048] For each test case, set weights for defect discovery rate, importance, and execution time;

[0049] Determine the weighted sum of defect discovery rate, importance, and execution time;

[0050] Sort the plurality of candidate test cases from largest to smallest according to the weighted sum value;

[0051] Select the N alternative test cases with the largest weighted sum value as the actual test cases.

[0052] In a fifth aspect, the present invention provides an electronic device comprising: a memory, a processor, and a program stored in the memory and executable on the processor, wherein the program, when executed by the processor, implements the steps of the method described in the first aspect and / or the second aspect.

[0053] In a sixth aspect, the present invention provides a computer-readable storage medium having a program stored thereon, and when the program is executed by a processor, the steps of the method described in the first aspect and / or the second aspect are implemented.

[0054] In a seventh aspect, the present invention provides a computer program product, comprising a computer program, which implements the method described in the first aspect and / or the second aspect when executed by a processor.

[0055] The present invention provides a test case scoring model training method and a test case selection method. The test case scoring model training method includes: determining multiple historical test cases for software testing; generating training samples based on the historical test cases, wherein the training samples include feature information of the historical test cases and the defect discovery rate of the historical test cases; and performing model training based on horizontal federated learning in conjunction with other participants based on the training samples. The test case scoring model finally obtained is used to predict the corresponding defect discovery rate based on the feature information of multiple alternative test cases, so as to select actual test cases from the multiple alternative test cases based on the predicted defect discovery rate. The solution of the present invention realizes training of the model based on a large amount of sample data by uniting multiple participants, and can be used to predict the defect discovery rate of the test case. The predicted defect discovery rate can reflect the value of the test case, provide an effective reference for the selection of the test case, improve the efficiency of test case selection, and further improve the efficiency of software testing. At the same time, model training based on horizontal federated learning can effectively guarantee the data privacy of each participant. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 A schematic diagram of an application scenario provided by an embodiment of the present invention;

[0057] Figure 2 A flowchart of a test case scoring model training method provided by an embodiment of the present invention;

[0058] Figure 3 A flowchart of a test case selection method provided by an embodiment of the present invention;

[0059] Figure 4 A schematic diagram of the structure of a test case scoring model training device provided by an embodiment of the present invention;

[0060] Figure 5 A schematic diagram of the structure of a test case selection device provided by an embodiment of the present invention;

[0061] Figure 6 A schematic structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0062] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0063] During the software testing process, it is generally necessary to use many test cases to test the completeness of the software functions as much as possible and to find problems in the software functions or logic.

[0064] As software features grow, more test cases are required. However, in some cases, release time constraints may only allow for a certain number of test cases to be executed. Therefore, it is important to select test cases that can effectively test software performance.

[0065] Among the test case performance indicators, the defect discovery rate is a relatively important one. The probability of discovering a software defect by executing a test case often indicates the value of the test case to a certain extent.

[0066] To use this metric for test case selection, a quick and accurate prediction of the defect discovery rate for each test case is necessary. However, current defect discovery rate predictions are based on empirical judgment by staff, resulting in low accuracy and a poorly preserved and inherited data.

[0067] Based on this, the present invention proposes a test case scoring model training method to build and train a model to achieve accurate and efficient prediction of defect discovery rates. This allows test cases to be selected based on the predicted defect discovery rates, further enabling efficient software testing.

[0068] At the same time, some new projects may not have sufficient historical data to serve as a reference. To enhance the applicability of the model, a federated learning training model is introduced. During the model training process, multiple parties are jointly trained. Each party provides a certain amount of historical data, which can expand the training sample and improve the model's prediction accuracy.

[0069] In many fields, technologies such as machine learning and model training based on big data are constantly developing. By mining big data, a wealth of valuable information can be obtained. With the development of technology, the sources of raw data are becoming increasingly diverse, even leading to cross-disciplinary collaboration.

[0070] In some business scenarios, multiple business platforms collect their own business data. This data is valuable and is often kept private by these platforms. Each platform does not want to share its raw data with others. However, in some scenarios, multiple platforms desire to collaborate on computing without disclosing their data to improve processing capabilities. This is why approaches like federated machine learning have emerged.

[0071] Federated Machine Learning, also known as Federated Learning, can unite all parties to use data and conduct collaborative modeling without leaving the local data, and is gradually becoming a common method in privacy-preserving computing.

[0072] For different data sets, federated learning can be divided into horizontal federated learning (Horizontal Federated Learning), vertical federated learning (Vertical Federated Learning) and federated transfer learning (Federated Transfer Learning).

[0073] Horizontal federated learning can be applied to scenarios where different datasets have significant overlap in user features but minimal user overlap. For example, consider a bank in location A and another in location B. While the two banks are located in different locations and have minimal user overlap, their data on users' income and expenditure behaviors, asset data, and other data points can be used to enable collaborative computing between the two banks.

[0074] Vertical federated learning can be applied to scenarios where different datasets have significant user overlap but less overlap in user characteristics. For example, consider two different participating institutions: a local bank and an e-commerce company. Their user bases likely encompass the majority of the local population, resulting in a significant overlap in user characteristics. However, because the bank's data involves user income and expenditure behavior and asset data, while the e-commerce company's data focuses on user browsing and purchase history, their user characteristics overlap less. In this scenario, vertical federated learning can be used to enable data collaborative computing between the bank and the e-commerce company.

[0075] Federated transfer learning can be applied to scenarios where there is little overlap between users and user characteristics across different datasets. For example, consider two different participating institutions: a bank in location A and an e-commerce company in location B. The user groups of these two participants have little overlap, and their user characteristics also have minimal overlap. In this case, transfer learning is necessary to improve the effectiveness of collaborative computing.

[0076] There are many machine learning algorithms that can be used in federated learning, including but not limited to neural networks, random forests, and other algorithms.

[0077] In the present invention, what is involved is the union of historical data from multiple parties, all of which involve data such as the characteristics of test cases and defect discovery rates. Therefore, it conforms to the characteristics of horizontal federated learning and belongs to the category of horizontal federated learning.

[0078] Figure 1A schematic diagram of an application scenario provided by an embodiment of the present invention. Figure 1 As shown in the figure, a project (corresponding to client terminal 1) needs to test newly developed software and collaborates with other projects (corresponding to client terminals 2, ..., k) to jointly train a model. Each client terminal trains the model based on local training samples and exchanges model parameters with the server. Ultimately, a test case scoring model is obtained.

[0079] For specific implementation methods, please refer to the following embodiments.

[0080] Figure 2 The flowchart of a test case scoring model training method provided by an embodiment of the present invention. The method of this embodiment can be applied to various participants in model training, such as Figure 2 As shown, the method of this embodiment may specifically include:

[0081] S201. Determine multiple historical test cases for software testing.

[0082] Software testing typically involves many test cases and associated test data. Test cases used in past software testing processes are called historical test cases. You can select some or all of these test cases.

[0083] S202: Generate training samples based on historical test cases, where the training samples include feature information of the historical test cases and defect discovery rates of the historical test cases.

[0084] Feature annotation is performed on each test case, and the corresponding defect discovery rate is determined to form a data set of training samples.

[0085] The defect discovery rate of a test case refers to the percentage of software problems discovered using this test case among all problems discovered during the software testing process.

[0086] S203. Based on the training samples, in collaboration with other participants, model training is performed based on horizontal federated learning. The resulting test case scoring model is used to predict the corresponding defect detection rate based on the feature information of multiple alternative test cases, so as to select actual test cases from the multiple alternative test cases based on the predicted defect detection rate.

[0087] Other participants also use steps S201 and S202 to generate training samples, and all parties jointly conduct horizontal federated learning to complete model training. Each participant can save the model parameters of the trained test case scoring model.

[0088] The test case scoring model can be used to select test cases. Specifically, the test case scoring model can be used to predict the defect detection rate of alternative test cases based on the characteristics of the alternative test cases, and then some alternative test cases can be selected as actual test cases based on the predicted defect detection rate for subsequent software testing.

[0089] The test case scoring model training method provided in this embodiment includes: determining multiple historical test cases for software testing; generating training samples based on the historical test cases, the training samples including the feature information of the historical test cases and the defect discovery rate of the historical test cases; based on the training samples, jointly with other participants, performing model training based on horizontal federated learning, and the test case scoring model finally obtained is used to predict the corresponding defect discovery rate based on the feature information of multiple alternative test cases, so as to select the actual test case from the multiple alternative test cases based on the predicted defect discovery rate. By jointly training the model based on a large amount of sample data with multiple participants, it can be used to predict the defect discovery rate of the test case. The predicted defect discovery rate can reflect the value of the test case, provide an effective reference for the selection of the test case, improve the efficiency of the test case selection and further improve the efficiency of software testing. At the same time, model training based on horizontal federated learning can effectively guarantee the data privacy of each participant.

[0090] In some embodiments, the above-mentioned model training based on horizontal federated learning in conjunction with other participants based on training samples may specifically include: obtaining initial test case scoring model parameters from the server; using training samples to train the model to obtain loss function gradients; sending the loss function gradients to the server so that the server aggregates and updates the loss function gradients sent by each participant; receiving the updated loss function gradients sent by the server; continuing model training based on the updated loss function gradients, iterating until the loss function converges, and obtaining a trained test case scoring model.

[0091] Still refer to Figure 1 The server and k client terminals jointly perform the model training process of horizontal federated learning. The server sends the global model to each client terminal. Each client terminal uses local data to train the global model sent by the server, and the trained model parameters are uploaded to the server. The server aggregates the model parameters uploaded by each client terminal to obtain the updated global model parameters. The above process is repeated in sequence until the aggregated global model converges.

[0092] Specifically, model parameters may include loss function, loss function gradient, etc.

[0093] Assuming that the tth round of iteration is currently in progress, the specific training process is as follows:

[0094] Step 1: The server sends the current model weights, loss function, loss function gradient and other model parameters to each client terminal C i .

[0095] Step 2: Client Terminal C i Using algorithms such as logistic regression and linear regression, combined with model parameters such as model weights, local training is performed based on its own local training samples, and parameters such as the loss function and loss function gradient are recalculated and uploaded to the server.

[0096] Step 3: The server receives the loss function, loss function gradient, and other parameters uploaded by all client terminals, aggregates them, and updates them. At the same time, the server can determine whether the loss function has converged based on the loss function gradient state. If not, the server repeats the above steps.

[0097] In addition, you can preset an iteration number before model training begins. When the iteration number reaches the preset value, you can determine that the model is in a convergence state.

[0098] To ensure data security, the client terminal and server can encrypt the data they interact with during the interaction process. This encryption mechanism protects the privacy of the participants through parameter exchange. The data itself is not transmitted, and participants do not need to expose their own data to other participants, nor can they infer the data of other participants. This effectively protects user privacy and ensures data security.

[0099] In some embodiments, the characteristics of the above-mentioned historical test cases may include: software form, software business, use case type, module to which it belongs, use case status, whether the operation steps involve input boxes, whether the operation steps involve buttons, whether the operation steps involve zooming, whether the verification point includes page display, whether the verification point includes database verification, whether the verification point includes log verification, etc.

[0100] Among them, the parameters corresponding to the software form characteristics can be web, app, api, etc.; the parameters corresponding to the software business characteristics can be mall, entertainment, tools, etc.; the parameters corresponding to the use case type characteristics can be function, performance, UI, etc.; the parameters corresponding to the module characteristics can be login, homepage, personal center, etc.; the parameters corresponding to the use case status characteristics can be normal or abnormal; the parameters corresponding to whether the operation steps involve the input box characteristics can be yes or no; the parameters corresponding to whether the operation steps involve the button characteristics can be yes or no; the parameters corresponding to whether the operation steps involve the zoom characteristics can be yes or no; the parameters corresponding to whether the verification point includes the page display characteristics can be yes or no; the parameters corresponding to whether the verification point includes the database verification characteristics can be yes or no; the parameters corresponding to whether the verification point includes the log verification characteristics can be yes or no.

[0101] It should be noted that the above features are only examples, and the features that can be marked in actual application may include one or more of the above features, or other features not mentioned.

[0102] By annotating the features of test cases in detail, the results of model training can be closer to the actual correspondence between features and defect discovery rate, making the prediction of defect discovery rate based on features more accurate.

[0103] Figure 3 The flowchart of a test case selection method provided by an embodiment of the present invention. The method of this embodiment can be applied to various participants in model training, such as Figure 3 The method of this embodiment may specifically include:

[0104] S301. Obtain multiple candidate test cases.

[0105] Alternative test cases refer to test cases that can be used for software testing, from which test cases actually used for software testing can be selected.

[0106] The relevant data of the alternative test cases can be obtained from the specified storage location.

[0107] S302: Based on the trained test case scoring model, predict the defect discovery rate of the alternative test cases.

[0108] The test case scoring model is trained using the test case scoring model training method of the above embodiment.

[0109] For alternative test cases, mark the corresponding features, input the test effort scoring model, and then output the predicted results of the defect discovery rate of the alternative test cases.

[0110] The marked test case characteristics can be found in the following table:

[0111] Use Case Number 11001 11002 Software form classification Web Web Software Business Classification App App Use case types Function UI …… …… ……

[0112] S303: Select an actual test case from multiple candidate test cases based on the predicted defect discovery rate.

[0113] The predicted defect discovery rate can, to a certain extent, reflect the value of the corresponding test case to software testing. Based on the predicted defect discovery rate, the actual test case can be selected from multiple candidate test cases.

[0114] For example, test cases with a defect discovery rate higher than a preset value can be selected as actual test cases to perform subsequent software testing.

[0115] This embodiment uses a horizontal federated learning model to predict defect discovery rates. This model is more reliable than manual judgment and uses a wider range of sample data than models trained using other methods, resulting in higher accuracy. The test cases selected using this method are more valuable, further improving software testing efficiency.

[0116] In some embodiments, the above-mentioned method of selecting actual test cases from multiple alternative test cases based on the predicted defect discovery rate can specifically include: sorting multiple alternative test cases from high to low according to the defect discovery rate; and selecting N alternative test cases with the highest defect discovery rates as actual test cases.

[0117] Using the defect discovery rate as a test case selection criterion, select the candidate test cases with the highest defect discovery rate as the actual test cases. This allows you to test as many possible software issues as possible within a shorter testing time, achieving higher testing efficiency.

[0118] In other embodiments, the above-mentioned method of selecting actual test cases from multiple alternative test cases based on the predicted defect detection rate can also be: for each test case, setting weights for the defect detection rate, importance, and execution time; determining the weighted sum of the defect detection rate, importance, and execution time; sorting multiple alternative test cases from large to small according to the weighted sum value; and selecting N alternative test cases with the largest weighted sum value as actual test cases.

[0119] Importance refers to the importance of a test case's performance within the software's functionality, while execution duration refers to the length of time it takes to execute a test case. In addition to these two metrics, other performance indicators can also be added.

[0120] We comprehensively consider multiple metrics, including defect discovery rate, importance, and execution time, and assign weights to each metric. The weighted sum is then used as the selection criterion. The candidate test cases with the highest weighted sums are selected as the actual test cases. This allows for a more comprehensive assessment of the value of each test case, allowing for the most comprehensive testing of potential software issues, resulting in higher testing efficiency.

[0121] In addition, after executing actual test cases to complete software testing, the use cases and actual defect discovery rates can be added to the sample data set to facilitate updating the model.

[0122] The device that performs model training and the device that performs test case selection can be the same device or different devices.

[0123] Figure 4 A structural diagram of a test case scoring model training device provided by an embodiment of the present invention is shown as follows: Figure 4 As shown, the test case scoring model training device 400 of this embodiment includes: a historical test case determination module 401, a training sample generation module 402, and a training module 403.

[0124] A historical test case determination module 401 is used to determine a plurality of historical test cases for software testing;

[0125] A training sample generating module 402 is configured to generate training samples based on historical test cases, wherein the training samples include feature information of the historical test cases and defect discovery rates of the historical test cases;

[0126] Training module 403 is used to perform model training based on horizontal federated learning in conjunction with other participants based on training samples. The final test case scoring model is used to predict the corresponding defect detection rate based on the feature information of multiple alternative test cases, so as to select actual test cases from multiple alternative test cases based on the predicted defect detection rate.

[0127] Optionally, the training module 403 is specifically configured to:

[0128] Get the initial test case scoring model parameters from the server;

[0129] Use training samples to train the model and obtain new model parameters;

[0130] Sending new model parameters to the server so that the server aggregates and updates the new model parameters sent by each participant;

[0131] Receive updated model parameters sent by the server;

[0132] Continue model training based on the updated model parameters, iterate until the new model converges, and obtain a trained test case scoring model.

[0133] Optionally, the characteristics of historical test cases include: software form, software business, use case type, module to which it belongs, use case status, whether the operation steps involve input boxes, whether the operation steps involve buttons, whether the operation steps involve zooming, whether the verification point includes page display, whether the verification point includes database verification, and whether the verification point includes log verification.

[0134] The device provided in this embodiment can execute the test case scoring model training method in the above embodiment to achieve the same technical effect, which will not be repeated here.

[0135] Figure 5 A schematic diagram of the structure of a test case selection device provided by an embodiment of the present invention is shown as follows: Figure 5As shown, the test case selection device 500 of this embodiment includes: an acquisition module 501 , a prediction module 502 , and a selection module 503 .

[0136] An acquisition module 501 is used to acquire multiple candidate test cases;

[0137] Prediction module 502, for predicting the defect discovery rate of candidate test cases based on the trained test case scoring model; the test case scoring model is trained using the test case scoring model training method in the above embodiment;

[0138] The selection module 503 is configured to select an actual test case from a plurality of candidate test cases according to the predicted defect discovery rate.

[0139] Optionally, the selection module 503 is specifically configured to:

[0140] Sort multiple alternative test cases from high to low according to the defect discovery rate;

[0141] Select N alternative test cases with the highest defect discovery rate as actual test cases.

[0142] Optionally, the selection module 503 is specifically configured to:

[0143] For each test case, set weights for defect discovery rate, importance, and execution time;

[0144] Determine the weighted sum of defect discovery rate, importance, and execution time;

[0145] Sort multiple alternative test cases from large to small according to their weighted sum values;

[0146] Select the N alternative test cases with the largest weighted sum value as the actual test cases.

[0147] The device provided in this embodiment can execute the test case selection method in the above embodiment to achieve the same technical effect, which will not be repeated here.

[0148] Figure 6 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention is shown in FIG. Figure 6 As shown, the electronic device 600 of this embodiment includes: a memory 601, a processor 602, and a program stored in the memory 601 and executable on the processor 602. When the program is executed by the processor 602, the test case scoring model training method in the above-mentioned embodiment and / or the steps of the test case selection method in the above-mentioned embodiment can be implemented to achieve the same technical effect as the above-mentioned embodiment, which will not be repeated here.

[0149] The present invention also provides a computer-readable storage medium having a program stored thereon. When the program is executed by a processor, the program implements the test case scoring model training method in the above embodiment and / or the steps of the test case selection method in the above embodiment.

[0150] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements the test case scoring model training method in the above embodiment and / or the test case selection method in the above embodiment.

[0151] In the several embodiments provided herein, it should be understood that the disclosed devices and methods may be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the module division is merely a logical functional division. In actual implementation, other division methods may be used. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not implemented.

[0152] The integrated modules implemented in the form of software function modules can be stored in a computer-readable storage medium. The software function modules stored in a storage medium include a number of instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute some of the steps of the methods of various embodiments of the present invention.

[0153] It should be understood that the processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), or application-specific integrated circuits (ASICs). A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the present invention may be directly implemented by a hardware processor or implemented by a combination of hardware and software modules in the processor.

[0154] The memory may include a high-speed RAM memory, and may also include non-volatile storage NVM, such as at least one disk memory, and may also be a USB flash drive, a mobile hard disk, a read-only memory, a magnetic disk or an optical disk.

[0155] The storage medium may be implemented by any type of volatile or non-volatile memory device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage medium may be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0156] An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be an integral part of the processor. The processor and the storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the storage medium can also exist as discrete components in an electronic device or a main control device.

[0157] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0158] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0159] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods of various embodiments of the present invention.

[0160] The above are only preferred embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A test case scoring model training method, characterized in that: include: Identify multiple historical test cases for software testing; Generating a training sample based on the historical test case, wherein the training sample includes feature information of the historical test case and a defect discovery rate of the historical test case; Based on the training samples, the model is trained in conjunction with other participants based on horizontal federated learning. Each participant uses its own local training samples to train the global model issued by the server in combination with model parameters, and interacts with the server through encrypted loss function gradients. The model parameters are issued by the server and include the current model weights, loss function, and loss function gradients. The server aggregates the loss function gradients of each participant and updates the global model parameters, iterating until the model converges. The resulting test case scoring model is used to predict corresponding defect discovery rates based on feature information of multiple candidate test cases, so as to select actual test cases from the multiple candidate test cases based on the predicted defect discovery rates.

2. The method according to claim 1, characterized in that The model training is performed based on horizontal federated learning in conjunction with other participants based on the training samples, including: Get the initial test case scoring model parameters from the server; Performing model training using the training samples to obtain a loss function gradient; Sending the loss function gradient to the server so that the server aggregates and updates the loss function gradients sent by each participant; Receive the updated loss function gradient sent by the server; The model training is continued based on the updated loss function gradient, and the iteration is performed until the loss function converges to obtain a trained test case scoring model.

3. The method according to claim 1 or 2, characterized in that The characteristics of the historical test cases include: software form, software business, use case type, module to which it belongs, use case status, whether the operation steps involve input boxes, whether the operation steps involve buttons, whether the operation steps involve zooming, whether the verification points include page display, whether the verification points include database verification, and whether the verification points include log verification.

4. A test case selection method, characterized in that: include: Get multiple alternative test cases; Predicting the defect discovery rate of the candidate test case based on a trained test case scoring model; the test case scoring model is trained using the test case scoring model training method according to any one of claims 1 to 3; According to the predicted defect discovery rate, actual test cases are selected from multiple alternative test cases.

5. The method according to claim 4, characterized in that The actual test case is selected from a plurality of alternative test cases based on the predicted defect discovery rate, including: Sort multiple alternative test cases from high to low according to the defect discovery rate; Select N alternative test cases with the highest defect discovery rate as actual test cases.

6. The method according to claim 4, characterized in that The actual test case is selected from a plurality of alternative test cases based on the predicted defect discovery rate, including: For each test case, set weights for defect discovery rate, importance, and execution time; Determine the weighted sum of defect discovery rate, importance, and execution time; Sort the plurality of candidate test cases from largest to smallest according to the weighted sum value; Select the N alternative test cases with the largest weighted sum value as the actual test cases.

7. A test case scoring model training device, characterized in that: include: A historical test case determination module, used to determine multiple historical test cases for software testing; A training sample generation module is used to generate training samples based on the historical test cases, wherein the training samples include feature information of the historical test cases and defect discovery rates of the historical test cases; A training module is configured to perform model training based on the training samples and in conjunction with other participants based on horizontal federated learning. Each participant uses its own local training samples in combination with model parameters to train the global model issued by the server, interacting with the server via encrypted loss function gradients. The model parameters are issued by the server and include the current model weights, loss function, and loss function gradients. The server aggregates the loss function gradients of each participant and updates the global model parameters, iterating until the model converges. The resulting test case scoring model is used to predict corresponding defect discovery rates based on feature information of multiple candidate test cases, so as to select actual test cases from the multiple candidate test cases based on the predicted defect discovery rates.

8. A test case selection device, characterized in that: include: An acquisition module is used to obtain multiple alternative test cases; A prediction module, configured to predict the defect discovery rate of the candidate test case based on a trained test case scoring model; the test case scoring model is trained using the test case scoring model training method according to any one of claims 1 to 3; The selection module is used to select actual test cases from multiple alternative test cases based on the predicted defect discovery rate.

9. An electronic device, characterized in that: The electronic device comprises: a memory, a processor, and a program stored in the memory and executable on the processor, wherein the program implements the steps of the method according to any one of claims 1 to 6 when executed by the processor.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a program, and when the program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

11. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images