An evaluation method and device for software testing, a computing device and a storage medium

Abstract

Embodiments of the present application provide a software testing evaluation method and device, a computing device and a storage medium. The method comprises: determining a target proportion of a test software according to historical proportions of a plurality of historical version softwares corresponding to the test software; determining a target defect quantity of the test software in a target sub-period of testing according to the target proportion, a cumulative number of test defects of the test software in a turning point sub-period, and an order of the turning point sub-period of the test software in testing; comparing a test defect quantity of the test software in the target sub-period with the corresponding target defect quantity, and determining whether the test software is sufficiently tested according to a comparison result. Since the test software has a greater correlation with its historical version software and is less affected by factors such as a testing environment and a tester, the target proportion of the test software is more accurately determined based on the historical proportions of the historical version softwares, so that whether the test software is sufficiently tested is accurately evaluated.

Description

Technical Field

[0001] This application relates to the field of financial technology (Fintech), and more particularly to an evaluation method, apparatus, computing device, and storage medium for software testing. Background Technology

[0002] With the development of computer technology, more and more technologies are being applied in the financial sector, and the traditional financial industry is gradually transforming into fintech. However, due to the security and real-time requirements of the financial industry, higher demands are being placed on technology. In the financial sector, to ensure the normal and efficient operation of software, it is necessary to test the software to fix defects. If the software is not adequately tested, it is impossible to effectively fix software defects.

[0003] In related technologies, code coverage is used to assess whether software testing is sufficient. However, the code written by developers is affected by their technical skills and experience, and the code they write may not fully cover the requirements. Based on the code that already has omissions, testers cannot accurately assess whether the software has been tested sufficiently.

[0004] In summary, there is an urgent need for a software testing evaluation method to accurately assess whether software has been adequately tested. Summary of the Invention

[0005] This application provides a software testing evaluation method, apparatus, computing device, and storage medium to accurately evaluate whether software has been adequately tested.

[0006] In a first aspect, embodiments of this application provide an evaluation method for software testing, the method comprising:

[0007] The target ratio of the test software is determined based on the historical ratios of multiple historical versions of the test software. The historical ratio of any historical version of the software is the ratio of the cumulative number of test defects in the corresponding historical inflection point sub-period to the total number of test defects in the corresponding historical test. The historical inflection point sub-period corresponding to any historical version of the software is the sub-period with the highest number of test defects among the multiple sub-periods of the corresponding historical test.

[0008] Based on the target ratio, the cumulative number of test defects of the test software in the inflection point sub-cycle, and the order of the inflection point sub-cycles of the test software in the test, the target number of defects of the test software in the target sub-cycle during the test is determined.

[0009] The number of test defects in the target sub-cycle is compared with the corresponding number of target defects, and the test software is used to determine whether the test is sufficient based on the comparison results.

[0010] In the above technical solution, since each software is closely related to its historical versions, meaning that the defect distribution of software in the same project is similar during testing, while the defect distribution of software in different projects varies greatly due to factors such as testing environment and testers, the target ratio of the cumulative number of defects in the inflection point sub-cycle relative to the total number of defects can be determined based on the historical proportions of multiple historical versions of the test software. Then, based on this target ratio, the target number of defects in the target sub-cycle when the test software is fully tested can be accurately determined. By comparing the number of test defects in the actual test in the target sub-cycle with the corresponding target number of defects when the test is fully tested, it can be accurately determined whether the test software has been fully tested.

[0011] Optionally, the target proportion of the test software is determined based on the historical proportions of multiple historical versions of the test software, including:

[0012] Filter out abnormal historical proportions from multiple historical proportions;

[0013] The average value of the filtered historical proportions is determined as the target proportion.

[0014] In the above technical solution, since some historical versions of software may be abnormal during testing, the defect distribution of these historical versions differs significantly from the defect distribution of other historical versions. These historical versions correspond to the abnormal historical proportion. Therefore, by filtering out the abnormal historical proportion from the historical proportions of multiple historical versions, and then determining the average of the filtered historical proportions as the target proportion, the impact of abnormal testing is reduced.

[0015] Optionally, the historical proportion of anomalies can be determined by the following methods:

[0016] Determine the standard deviation corresponding to the plurality of historical proportions;

[0017] For any historical proportion, if the deviation of the historical proportion from the average of the plurality of historical proportions is greater than the standard deviation, then the historical proportion is determined to be an abnormal historical proportion.

[0018] In the above technical solution, since the standard deviation of multiple historical proportions reflects the dispersion of multiple historical proportions, if the deviation of a certain historical proportion relative to the average of multiple historical proportions is greater than the above standard deviation, it indicates that the deviation of the historical proportion relative to the average is large, and it is an outlier among multiple historical proportions. Therefore, the historical proportion is determined to be an abnormal historical proportion.

[0019] Optionally, the target number of defects in the target sub-cycle of the testing software is determined based on the target ratio, the cumulative number of test defects in the inflection point sub-cycle of the testing software, and the order of the inflection point sub-cycles of the testing software during testing, including:

[0020] The ratio of the cumulative number of test defects in the inflection point sub-cycle of the test software to the target ratio is determined as the total number of target defects.

[0021] The number of target defects in an inflection point sub-cycle is determined based on the total number of target defects and the order of the inflection point sub-cycles of the testing software during testing; and / or the number of target defects in other target sub-cycles is determined based on the total number of target defects, the order of the inflection point sub-cycles of the testing software during testing, and the order of other target sub-cycles of the testing software besides the inflection point sub-cycles during testing.

[0022] In the above technical solution, since the target ratio is the ratio of the cumulative number of defects in the inflection point sub-cycle to the total number of defects when the test software is fully tested, determining the ratio of the cumulative number of test defects in the inflection point sub-cycle to this target ratio yields the target total number of defects when the test software is fully tested. Furthermore, based on this target total number of defects and the order of the inflection point sub-cycles in the test, the target number of defects in each target sub-cycle when the test software is fully tested can be accurately determined. These two methods improve the flexibility of determining the target number of defects. Moreover, compared to the first method which determines the target number of defects in the inflection point sub-cycle based on the order of the inflection point sub-cycles in the test, the second method, based on the first method and combining the order of the inflection point sub-cycles in the test with the order of other target sub-cycles of the test software (excluding the inflection point sub-cycle) in the test, determines the target number of defects. This improves the comprehensiveness of data calculation, reduces errors, and further improves the accuracy of determining the target number of defects.

[0023] Optionally, if the order of the inflection point sub-cycles of the test software is not the target order, then the target sub-cycle includes the inflection point sub-cycles of the test software and the sub-cycles of the target order; if the order of the inflection point sub-cycles is the target order, then the target sub-cycle includes the inflection point sub-cycles of the test software.

[0024] The target order is determined based on the total number of sub-cycles in the test and a first preset ratio.

[0025] In the above technical solution, since the sub-cycles in the target order are relatively stable when the test software is fully tested, and the inflection point sub-cycle is also an important period in the test, when the order of the inflection point sub-cycles of the test software is not the target order, the target sub-cycle includes the sub-cycles in the target order and the inflection point sub-cycle; when the order of the inflection point sub-cycles of the test software is the target order, the inflection point sub-cycle is the sub-cycle in the target order, and the target sub-cycle only needs to include the inflection point sub-cycle.

[0026] Optionally, determining whether the testing software has been sufficiently tested based on the comparison results includes:

[0027] If the change in the number of test defects in the target sub-cycle relative to the corresponding target defect number does not exceed a preset range, then the test software is deemed to have performed sufficient testing.

[0028] In the above technical solution, since the test software is relatively stable in the target sub-cycle when the test is fully tested, the change in the number of test defects relative to the corresponding target defect number will not be too large. Therefore, the change in the number of test defects in the target sub-cycle relative to the corresponding target defect number does not exceed the preset range, indicating that the test software is relatively stable in the target sub-cycle and the test software can be determined to be fully tested relatively accurately.

[0029] Optionally, before determining that the test software has been sufficiently tested, the method further includes:

[0030] The number of test defects in the last sub-cycle of the test software is determined to be no more than a second preset ratio relative to the total number of test defects in the test.

[0031] In the above technical solution, since the test software will not add many defects at the end of the test when the test is fully tested, the test software can be more accurately determined to be fully tested by first determining the number of test defects in the last sub-cycle of the test software in the test, and ensuring that the ratio of the number of test defects in the test software to the total number of test defects in the test does not exceed the second preset ratio.

[0032] Secondly, embodiments of this application also provide an evaluation apparatus for software testing, comprising:

[0033] The ratio determination module is used to determine the target ratio of the test software based on the historical ratios of multiple historical versions of the test software; wherein, the historical ratio of any historical version of the software is the ratio of the cumulative number of test defects in the corresponding historical inflection point sub-period to the total number of test defects in the corresponding historical test; the historical inflection point sub-period corresponding to any historical version of the software is the sub-period with the largest number of test defects among the multiple sub-periods of the corresponding historical test.

[0034] The quantity determination module is used to determine the target number of defects of the test software in the target sub-cycle during the test based on the target ratio, the cumulative number of test defects of the test software in the inflection point sub-cycle, and the order of the inflection point sub-cycles of the test software in the test.

[0035] The evaluation module is used to compare the number of test defects of the test software in the target sub-cycle with the corresponding number of target defects, and determine whether the test software has been tested sufficiently based on the comparison results.

[0036] Thirdly, embodiments of this application provide a computing device, including at least one processor and at least one memory, wherein the memory stores a computer program, and when the program is executed by the processor, the processor performs the evaluation method for software testing as described in any of the first aspects above.

[0037] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program executable by a computing device, which, when run on the computing device, causes the computing device to perform the evaluation method for software testing as described in any of the first aspects above.

[0038] Furthermore, the technical effects of any of the implementation methods in aspects two to four can be found in the technical effects of different implementation methods in aspect one, and will not be repeated here. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 A flowchart illustrating the first software testing evaluation method provided in this application embodiment;

[0041] Figure 2 A flowchart illustrating the second software testing evaluation method provided in this application embodiment;

[0042] Figure 3 A schematic diagram showing the defect distribution of the first historical version of the software in corresponding historical tests, provided for an embodiment of this application.

[0043] Figure 4 A schematic diagram illustrating the defect distribution of the second historical version software provided in the embodiments of this application in the corresponding historical tests;

[0044] Figure 5A schematic diagram showing the defect distribution of the third historical version software provided in the embodiments of this application in the corresponding historical tests;

[0045] Figure 6 A schematic diagram showing the defect distribution of the fourth historical version software provided in the embodiments of this application in the corresponding historical tests;

[0046] Figure 7 A schematic diagram showing the defect distribution of the fifth historical version software in corresponding historical tests, provided for embodiments of this application.

[0047] Figure 8 A flowchart illustrating the third software testing evaluation method provided in this application embodiment;

[0048] Figure 9 A flowchart illustrating the fourth software testing evaluation method provided in this application embodiment;

[0049] Figure 10 A schematic diagram of the structure of the evaluation apparatus for software testing provided in the embodiments of this application;

[0050] Figure 11 A schematic diagram of the structure of a computing device provided in an embodiment of this application. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0052] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0053] The term "test defect count" refers to the number of defects generated within the corresponding sub-cycle.

[0054] The term "cumulative number of test defects" refers to the sum of the number of defects generated within the corresponding sub-cycle and the number of defects generated before the corresponding sub-cycle.

[0055] The term "total number of test defects" refers to the total number of defects generated in the corresponding test.

[0056] To ensure the normal and efficient operation of software, testing is necessary to fix software defects. If the software is not adequately tested, defects cannot be effectively fixed. Code coverage can be used to assess whether software has been adequately tested; however, the code written by developers is influenced by their technical skills and experience, and may not fully cover all requirements. Testers, based on this existing code oversight, cannot accurately assess whether the software has been adequately tested.

[0057] In some embodiments, a uniform ratio is set for all software, which is the ratio of the cumulative number of defects in the inflection point sub-cycle to the total number of defects when the software is fully tested. Based on this ratio, the defect distribution when the software is fully tested is determined. Based on the defect distribution when the software is fully tested and the defect distribution during actual testing, it is determined whether the software has been fully tested.

[0058] However, different projects have different software testing environments. Some projects have relatively stable testing environments, while others have unstable ones. Furthermore, different projects have different software testers; some have experienced and highly skilled testers, while others have inexperienced and less capable testers. Therefore, the defect distribution varies significantly across projects due to factors such as testing environment and testers. It is difficult to accurately set a uniform ratio suitable for all projects. Consequently, based on a set uniform ratio, the defect distribution when software testing is sufficiently thorough cannot be accurately determined.

[0059] In view of this, embodiments of this application propose an evaluation method, apparatus, computing device, and storage medium for software testing. Since each software is highly correlated with its historical versions, meaning that software from the same project exhibits similar defect distributions during testing, while software from different projects shows significant differences in defect distributions due to factors such as testing environment and testers, this application proposes a method. Based on the historical proportions of multiple historical versions of the software being tested, a target proportion can be determined relative to the total number of defects at the inflection point sub-cycle when the software is sufficiently tested. This target proportion allows for the accurate determination of the target number of defects in the target sub-cycle when the software is sufficiently tested. By comparing the actual number of defects tested in the target sub-cycle with the corresponding target number of defects when testing is sufficient, it can be precisely determined whether the software has been sufficiently tested.

[0060] The technical solution of this application and how it solves the above-mentioned technical problems will be described in detail below with reference to the accompanying drawings and specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0061] This application provides a first method for evaluating software testing, such as... Figure 1 As shown, it includes the following steps:

[0062] Step S101: Determine the target ratio of the test software based on the historical ratios of multiple historical versions of the test software.

[0063] Wherein, the historical proportion of any historical version of software is the ratio of the cumulative number of test defects in the corresponding historical inflection point sub-cycle to the total number of test defects in the corresponding historical tests; the historical inflection point sub-cycle corresponding to any historical version of software is the sub-cycle with the highest number of test defects among the multiple sub-cycles of the corresponding historical tests.

[0064] In this embodiment, due to the influence of factors such as testing environment and testers on software from different projects, the defect distribution varies greatly during testing, making it difficult to accurately set a uniform ratio suitable for all projects. However, for software from the same project (for a given software, its corresponding historical versions are considered software from the same project), the testers are the same, and the testing environment is relatively similar. Therefore, based on the historical ratios of multiple historical versions of the tested software, the target ratio of the cumulative number of defects to the total number of defects at the inflection point sub-cycle can be determined when the tested software has been sufficiently tested.

[0065] Step S102: Determine the target number of defects in the target sub-cycle of the test software during the test based on the target ratio, the cumulative number of test defects in the test software at the inflection point sub-cycle, and the order of the inflection point sub-cycles of the test software during the test.

[0066] The inflection point sub-cycle corresponding to the above testing software is the sub-cycle with the highest number of defects detected among the multiple sub-cycles of testing.

[0067] In this embodiment, after determining the target ratio of the cumulative number of defects in the inflection point sub-cycle to the total number of defects when the testing software is fully tested, it is also necessary to determine the target number of defects in the target sub-cycle when the testing software is fully tested based on the target ratio.

[0068] Step S103: Compare the number of test defects in the target sub-cycle with the corresponding number of target defects, and determine whether the test software has performed sufficient testing based on the comparison result.

[0069] In implementation, the number of test defects in the target sub-cycle is the actual number of defects generated during testing within that sub-cycle; the target number of defects in the target sub-cycle is the number of defects that should be generated within the target sub-cycle when testing is sufficient. Therefore, these two numbers need to be compared. In some specific embodiments, the smaller the change in the number of test defects relative to the corresponding target number of defects, the closer the actual number of defects generated in the target sub-cycle is to the number of defects that should be generated when testing is sufficient, indicating that the testing software has performed more thorough testing.

[0070] In the above technical solution, since each software is closely related to its historical versions, meaning that the defect distribution of software in the same project is similar during testing, while the defect distribution of software in different projects varies greatly due to factors such as testing environment and testers, the target ratio of the cumulative number of defects in the inflection point sub-cycle relative to the total number of defects can be determined based on the historical proportions of multiple historical versions of the test software. Then, based on this target ratio, the target number of defects in the target sub-cycle when the test software is fully tested can be accurately determined. By comparing the number of test defects in the actual test in the target sub-cycle with the corresponding target number of defects when the test is fully tested, it can be accurately determined whether the test software has been fully tested.

[0071] This application provides a second method for evaluating software testing, such as... Figure 2 As shown, it includes the following steps:

[0072] Step S201: Filter out the abnormal historical proportions from the historical proportions of multiple historical versions of the test software.

[0073] As mentioned above, for software in the same project, the testers are the same and the test environment is also similar. Based on the historical proportions of multiple historical versions of the software corresponding to the test software, the target proportion of the cumulative number of defects relative to the total number of defects at the inflection point sub-cycle when the test software is fully tested can be determined more accurately.

[0074] However, among the multiple historical versions of the software being tested, some historical versions may have tested abnormally. The defect distribution of these historical versions during testing differs significantly from the defect distribution of other historical versions. These historical versions correspond to the historical proportion of abnormal versions.

[0075] See Figure 3-7 The image shows the defect distribution in corresponding historical tests for five historical versions of software belonging to the same project as the tested software:

[0076] See Figure 3As shown, in the corresponding historical tests, the historical inflection point sub-cycle of the first historical version of the software is the 9th cycle, the cumulative number of test defects in the historical inflection point sub-cycle is 74, the total number of test defects is 100, and the historical proportion of the first historical version of the software is A1 = 74 / 100 = 0.74 (74%).

[0077] See Figure 4 As shown, in the corresponding historical tests, the historical inflection point sub-cycle of the second historical version software is the 4th cycle, the cumulative number of test defects in the historical inflection point sub-cycle is 21, the total number of test defects is 29, and the historical proportion of the second historical version software A2 = 21 / 29 = 0.724 (72.4%).

[0078] See Figure 5 As shown, in the corresponding historical tests, the third historical version of the software has a historical inflection point sub-cycle of the 3rd cycle, a cumulative number of test defects of 23 in the historical inflection point sub-cycle, and a total number of test defects of 32. The historical proportion of the third historical version of the software is A3 = 23 / 34 = 0.676 (67.6%).

[0079] See Figure 6 As shown, in the corresponding historical tests, the historical inflection point sub-cycle of the fourth historical version software is the 3rd cycle, the cumulative number of test defects in the historical inflection point sub-cycle is 25, the total number of test defects is 32, and the historical proportion of the fourth historical version software A4 = 25 / 32 = 0.781 (78.1%).

[0080] See Figure 7 As shown, in the corresponding historical tests, the historical inflection point sub-cycle of the fifth historical version software is the second cycle. The cumulative number of test defects in the historical inflection point sub-cycle is 14, and the total number of test defects is 50. The historical proportion of the fifth historical version software is A5 = 14 / 50 = 0.28 (28%).

[0081] Clearly, the historical proportions of the fifth historical version of the software differ significantly from those of the other four versions. If the historical proportions of the fifth historical version are taken into account when determining the target proportion, it will affect the accuracy of the target proportion. Therefore, it is necessary to filter out the abnormal historical proportions from the historical proportions of the multiple historical versions of the test software (hereinafter referred to as multiple historical proportions). That is, filter out the historical proportion of the fifth historical version and only consider the historical proportions of the other four historical versions.

[0082] In some alternative implementations, the historical proportion of anomalies can be determined in the following ways:

[0083] Determine the standard deviation corresponding to the plurality of historical proportions;

[0084] For any historical proportion, if the deviation of the historical proportion from the average of the plurality of historical proportions is greater than the standard deviation, then the historical proportion is determined to be an abnormal historical proportion.

[0085] Taking the five historical software versions mentioned above as examples, the average value of the five historical ratios is B1 = (0.74 + 0.724 + 0.676 + 0.781 + 0.28) / 5 = 0.6402;

[0086] Standard deviation

[0087] Determine the deviation C1 of A1 relative to B1, C1 = |A1 - B1| = |0.74 - 0.6402| < 0.18;

[0088] Determine the deviation C2 of A2 relative to B1, C2 = |A2 - B1| = |0.724 - 0.6402| < 0.18;

[0089] Determine the deviation C3 of A3 relative to B1, C3 = |A3 - B1| = |0.676 - 0.6402| < 0.18;

[0090] Determine the deviation C4 of A4 relative to B1, C4 = |A4 - B1| = |0.781 - 0.6402| < 0.18;

[0091] Determine the deviation C5 of A5 relative to B1, C5 = |A5 - B1| = |0.28 - 0.6402| > 0.18.

[0092] The above method was used to determine that A5 (the historical proportion of the fifth historical version of the software) was an abnormal historical proportion.

[0093] Since the standard deviation of multiple historical proportions reflects the dispersion of multiple historical proportions, if the deviation of a certain historical proportion relative to the average of multiple historical proportions is greater than the above standard deviation, it means that the deviation of the historical proportion relative to the average is large, and it is an outlier among multiple historical proportions. Therefore, the historical proportion is identified as an abnormal historical proportion.

[0094] Step S202: Determine the average value of the filtered historical proportions as the target proportion.

[0095] In this embodiment, after filtering out the abnormal historical proportions from the above multiple historical proportions, the target proportion that reduces the impact of abnormal tests can be determined based on the filtered historical proportions.

[0096] Taking the five historical software versions mentioned above as examples, filter out A5 from A1, A2, A3, A4 and A5, and use the average value B2 of A1, A2, A3 and A4 as the target ratio. B2 = (A1 + A2 + A3 + A4) / 4 = (0.74 + 0.724 + 0.676 + 0.781) / 4 = 0.73.

[0097] The number of historical software versions and the historical proportion of each historical software version are merely illustrative examples, and this application does not impose any specific limitations on these parameters.

[0098] Step S203: Determine the target number of defects in the target sub-cycle of the test software during the test based on the target ratio, the cumulative number of test defects in the test software during the inflection point sub-cycle, and the order of the inflection point sub-cycles of the test software during the test.

[0099] Step S204: Compare the number of test defects in the target sub-cycle with the corresponding number of target defects, and determine whether the test software has performed sufficient testing based on the comparison result.

[0100] The specific implementation of steps S203-S204 can be found in the following embodiments, and will not be repeated here.

[0101] In the above technical solution, since some historical versions of software may be abnormal during testing, the defect distribution of these historical versions differs significantly from the defect distribution of other historical versions. These historical versions correspond to the abnormal historical proportion. Therefore, by filtering out the abnormal historical proportion from the historical proportions of multiple historical versions, and then determining the average of the filtered historical proportions as the target proportion, the impact of abnormal testing is reduced.

[0102] This application provides a third method for evaluating software testing, such as... Figure 8 As shown, it includes the following steps:

[0103] Step S801: Determine the target ratio of the test software based on the historical ratios of multiple historical versions of the test software.

[0104] Wherein, the historical proportion of any historical version of software is the ratio of the cumulative number of test defects in the corresponding historical inflection point sub-cycle to the total number of test defects in the corresponding historical tests; the historical inflection point sub-cycle corresponding to any historical version of software is the sub-cycle with the highest number of test defects among the multiple sub-cycles of the corresponding historical tests.

[0105] The specific implementation of step S801 can be found in the above embodiments, and will not be repeated here.

[0106] Step S802: The ratio of the cumulative number of test defects in the test software at the inflection point sub-cycle to the target ratio is determined as the target total number of defects.

[0107] In practice, since the target ratio is the ratio of the cumulative number of defects in the inflection point sub-cycle to the total number of defects when the test software is fully tested, the ratio of the cumulative number of test defects in the inflection point sub-cycle to this target ratio is the target total number of defects that should be generated when the test software is fully tested.

[0108] Step S803: Determine the number of target defects in the inflection point sub-cycle based on the total number of target defects and the order of the inflection point sub-cycles of the testing software during testing; and / or determine the number of target defects in other target sub-cycles based on the total number of target defects, the order of the inflection point sub-cycles of the testing software during testing, and the order of other target sub-cycles of the testing software besides the inflection point sub-cycles during testing.

[0109] In this embodiment, after determining the total number of target defects that should be generated when the test software is fully tested, the number of defects that should be generated in each sub-cycle when the test software is fully tested can be accurately determined based on the total number of target defects and the order of the inflection point sub-cycles in the test.

[0110] When software is sufficiently tested, the number of defects typically increases sequentially in the sub-cycles before the inflection point; the number of defects peaks in the inflection point sub-cycle; and decreases sequentially in the sub-cycles after the inflection point sub-cycle; with the number of defects approaching zero in the last sub-cycle. Therefore, when software is sufficiently tested, the distribution of the number of defects follows a Rayleigh distribution.

[0111] Based on this, in some specific embodiments, the number of target defects in each sub-cycle when the testing software is fully tested can be calculated using the Rayleigh distribution formula; or the number of target defects in the target sub-cycle when the testing software is fully tested can be calculated directly using the Rayleigh distribution formula. The Rayleigh distribution formula includes: the cumulative distribution function and the probability density function.

[0112] The cumulative distribution function is: F(ti)=K*(1-e -(ti / c)*(ti / c) ), t m Let ti be the order of the inflection point sub-cycle in the test, ti be the order of sub-cycle i in the test, K be the total number of target defects, and F(ti) be the cumulative number of target defects in sub-cycle i when the test software is fully tested.

[0113] The probability density function is: f(ti)=2*K*ti*(1 / c) 2 *e -(ti / c)*(ti / c) , t m Let ti be the order of the inflection point sub-cycle in the test, ti be the order of sub-cycle i in the test, K be the total number of target defects, and f(ti) be the number of target defects in sub-cycle i when the test software is fully tested.

[0114] Taking a test software with 10 sub-cycles and a target ratio of 0.73 (73%) as an example, the number of test defects in each sub-cycle is shown in Table 1:

[0115] Table 1

[0116] order 1 2 3 4 5 6 7 8 9 10 Number of defects tested 1 8 7 8 2 14 3 8 11 1

[0117] The inflection point sub-cycle is the 6th sub-cycle in the test (sequence number 6). The cumulative number of defects in the test software during the inflection point sub-cycle is 1+8+7+8+2+14=40. When the test software is fully tested, the total number of target defects K=40 / 73%=55.

[0118] When the testing software has performed sufficient testing, the target number of defects in the i-th sub-cycle is achieved.

[0119] Substituting the order of each sub-cycle into the above formula, we can obtain the target defect number for each sub-cycle as shown in Table 2:

[0120] Table 2

[0121] order 1 2 3 4 5 6 7 8 9 10 Target number of defects 1.5 2.9 4 4.9 5.4 5.6 5.4 5 4.5 3.8

[0122] Of course, in some embodiments, only the number of target defects in the target sub-cycles can be calculated. For example, if the 6th, 7th, and 10th sub-cycles are the target sub-cycles, only f(t6), f(t7), and f(t10) need to be calculated.

[0123] Step S804: Compare the number of test defects in the target sub-cycle with the corresponding number of target defects, and determine whether the test software has performed sufficient testing based on the comparison result.

[0124] The specific implementation of step S804 can be found in the following embodiments, which will not be repeated here.

[0125] In the above technical solution, since the target ratio is the ratio of the cumulative number of defects in the inflection point sub-cycle to the total number of defects when the test software is fully tested, determining the ratio of the cumulative number of test defects in the inflection point sub-cycle to the target ratio yields the target total number of defects when the test software is fully tested. Furthermore, based on the target total number of defects and the order of the inflection point sub-cycles in the test, the target number of defects in each target sub-cycle when the test software is fully tested can be accurately determined.

[0126] This application provides a fourth method for evaluating software testing, such as... Figure 9 As shown, it includes the following steps:

[0127] Step S901: Determine the target ratio of the test software based on the historical ratios of multiple historical versions of the test software.

[0128] Wherein, the historical proportion of any historical version of software is the ratio of the cumulative number of test defects in the corresponding historical inflection point sub-cycle to the total number of test defects in the corresponding historical tests; the historical inflection point sub-cycle corresponding to any historical version of software is the sub-cycle with the highest number of test defects among the multiple sub-cycles of the corresponding historical tests.

[0129] Step S902: Determine the target number of defects in the target sub-cycle of the test software during the test based on the target ratio, the cumulative number of test defects in the test software during the inflection point sub-cycle, and the order of the inflection point sub-cycles of the test software during the test.

[0130] In implementation, the aforementioned target sub-cycles can be determined based on the actual application scenario. Since the sub-cycles in the target testing sequence (e.g., mid-to-late stages) are relatively stable when the software testing is sufficient, the change in the number of test defects relative to the corresponding target defect number will not be too large; furthermore, the inflection point sub-cycle is also a crucial period in testing. Therefore, the aforementioned target sub-cycles need to cover the sub-cycles in the target testing sequence (e.g., mid-to-late stages) and also include the inflection point sub-cycle.

[0131] In some optional implementations, the target order is determined based on the total number of sub-cycles in the test and a first preset ratio. For example, the product of the total number of sub-cycles and the first preset ratio can be used as the target order, which represents the middle and later stages of the test. Taking a test software with 10 sub-cycles in the test as an example, the first preset ratio is 2 / 3, 10*(2 / 3)=20 / 3≈6.67. Since the target order is a positive integer, 7 is used as the target order.

[0132] The order of the inflection point sub-cycles in the above test software may not be the target order (the order of the inflection point sub-cycles is 6). In this case, the target sub-cycles include the inflection point sub-cycles (the 6th sub-cycle) and the sub-cycles of the target order (the 7th sub-cycle).

[0133] The order of the inflection point sub-cycles in the above-mentioned testing software may be the target order, in which case the target sub-cycles include the inflection point sub-cycles (the 7th sub-cycle).

[0134] Step S903: If the change in the number of test defects in the target sub-cycle relative to the corresponding target defect number does not exceed a preset range, then the test software is determined to have performed sufficient testing.

[0135] Since a sufficient number of defects will not be generated at the end of the testing process when the testing software is fully tested, in some alternative implementations, it is also necessary to determine, before determining whether the number of test defects in the last sub-cycle of the testing software does not exceed a second preset ratio relative to the total number of test defects during the testing process, before confirming that the testing software is fully tested.

[0136] The specific values ​​of the aforementioned preset amplitude and the second preset ratio can be set according to the actual application scenario, such as a preset amplitude of 30% (0.3) and a second preset ratio of 10% (0.1). The following are two specific embodiments for illustration:

[0137] 1) Referring to Table 3, the number of test defects and the number of target defects for a testing software in each sub-cycle are shown. In the inflection point sub-cycle (the 6th sub-cycle), the number of test defects is 14, and the number of target defects is 5.6. The corresponding change range D0 = |14-5.6| / 5.6 = 1.5 > 0.3, exceeding the preset range, which directly indicates that the testing software has not performed sufficiently.

[0138] Table 3

[0139] order 1 2 3 4 5 6 7 8 9 10 Number of defects tested 1 8 7 8 2 14 3 8 11 1 Target number of defects 1.5 2.9 4 4.9 5.4 5.6 5.4 5 4.5 3.8

[0140] 2) Referring to Table 4, the number of test defects and target defects of another testing software in each sub-cycle are shown. The number of test defects of the testing software in the inflection point sub-cycle (the second sub-cycle) is 13, and the number of target defects is 10.61. The corresponding change range D1 = |13-10.61| / 10.61 = 0.225 < 0.3, which does not exceed the preset range;

[0141] The number of defects tested by the testing software in the target sequence sub-cycle (the 4th sub-cycle) was 6, and the target number of defects was 4.74. The corresponding change range D2 = |6-4.74| / 4.74 = 0.269 < 0.3, which did not exceed the preset range.

[0142] The number of test defects in the last sub-cycle (the 6th sub-cycle) of the test software was 1, and the total number of test defects in the test was 37. The ratio of the number of test defects in the last sub-cycle to the total number of test defects was E = 1 / 37 = 0.027 < 0.1, which did not exceed the second preset ratio.

[0143] Ensure the testing software has been thoroughly tested.

[0144] Table 4

[0145] order 1 2 3 4 5 6 Number of defects tested 1 13 11 6 5 1 Target number of defects 7.72 10.61 8.52 4.74 1.92 0.58

[0146] The two specific embodiments described above are merely illustrative examples and are not intended to limit the scope of this application.

[0147] During implementation, if it is determined that the test software has not been tested sufficiently, test resources need to be adjusted to ensure the quality of the software.

[0148] Based on the same inventive concept, embodiments of this application provide an evaluation apparatus for software testing, see below. Figure 10 As shown, the software testing evaluation device 1000 includes:

[0149] The ratio determination module 1001 is used to determine the target ratio of the test software based on the historical ratios of multiple historical versions of the test software; wherein, the historical ratio of any historical version of the software is the ratio of the cumulative number of test defects in the corresponding historical inflection point sub-cycle to the total number of test defects in the corresponding historical test; the historical inflection point sub-cycle corresponding to any historical version of the software is the sub-cycle with the most test defects among the multiple sub-cycles of the corresponding historical test.

[0150] The quantity determination module 1002 is used to determine the target number of defects of the test software in the target sub-cycle during the test based on the target ratio, the cumulative number of test defects of the test software in the inflection point sub-cycle, and the order of the inflection point sub-cycles of the test software in the test.

[0151] The evaluation module 1003 is used to compare the number of test defects of the test software in the target sub-cycle with the corresponding number of target defects, and determine whether the test software has been tested sufficiently based on the comparison result.

[0152] Optionally, the ratio determination module 1001 is specifically used for:

[0153] Filter out abnormal historical proportions from multiple historical proportions;

[0154] The average value of the filtered historical proportions is determined as the target proportion.

[0155] Optionally, the ratio determination module 1001 is also used for:

[0156] Determine the standard deviation corresponding to the plurality of historical proportions;

[0157] For any historical proportion, if the deviation of the historical proportion from the average of the plurality of historical proportions is greater than the standard deviation, then the historical proportion is determined to be an abnormal historical proportion.

[0158] Optionally, the quantity determination module 1002 is specifically used for:

[0159] The ratio of the cumulative number of test defects in the inflection point sub-cycle of the test software to the target ratio is determined as the total number of target defects.

[0160] The number of target defects in an inflection point sub-cycle is determined based on the total number of target defects and the order of the inflection point sub-cycles of the testing software during testing; and / or the number of target defects in other target sub-cycles is determined based on the total number of target defects, the order of the inflection point sub-cycles of the testing software during testing, and the order of other target sub-cycles of the testing software besides the inflection point sub-cycles during testing.

[0161] Optionally, if the order of the inflection point sub-cycles of the test software is not the target order, then the target sub-cycle includes the inflection point sub-cycles of the test software and the sub-cycles of the target order; if the order of the inflection point sub-cycles is the target order, then the target sub-cycle includes the inflection point sub-cycles of the test software.

[0162] The target order is determined based on the total number of sub-cycles in the test and a first preset ratio.

[0163] Optionally, the evaluation module 1003 is specifically used for:

[0164] If the change in the number of test defects in the target sub-cycle relative to the corresponding target defect number does not exceed a preset range, then the test software is deemed to have performed sufficient testing.

[0165] Optionally, before determining that the test software has been adequately tested, the evaluation module 1003 is further configured to:

[0166] The number of test defects in the last sub-cycle of the test software is determined to be no more than a second preset ratio relative to the total number of test defects in the test.

[0167] Since this device is the same as the device in the method of this application embodiment, and the principle of the device in solving the problem is similar to that of the method, the implementation of the device can be referred to the implementation of the method, and the repeated parts will not be described again.

[0168] Based on the same technical concept, this application also provides a computing device 1100, such as... Figure 11 As shown, it includes at least one processor 1101 and a memory 1102 connected to at least one processor. In this embodiment, the specific connection medium between the processor 1101 and the memory 1102 is not limited. Figure 11 Taking the connection between processor 1101 and memory 1102 via bus 1103 as an example. The bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, Figure 11 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0169] The processor 1101 is the control center of the computing device. It can connect to various parts of the computing device using various interfaces and lines, and performs data processing by running or executing instructions stored in the memory 1102 and calling data stored in the memory 1102. Optionally, the processor 1101 may include one or more processing units. The processor 1101 may integrate an application processor and a modem processor. The application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles issuing instructions. It is understood that the modem processor may not be integrated into the processor 1101. In some embodiments, the processor 1101 and the memory 1102 may be implemented on the same chip; in some embodiments, they may also be implemented on separate chips.

[0170] Processor 1101 can be a general-purpose processor, such as a central processing unit (CPU), digital signal processor, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of the software testing evaluation method can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0171] Memory 1102, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory 1102 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic storage, magnetic disk, optical disk, etc. Memory 1102 can be any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but is not limited thereto. In the embodiments of this application, memory 1102 can also be a circuit or any other device capable of implementing storage functions for storing program instructions and / or data.

[0172] In this embodiment, the memory 1102 stores a computer program, which, when executed by the processor 1101, causes the processor 1101 to perform the following:

[0173] The target ratio of the test software is determined based on the historical ratios of multiple historical versions of the test software. The historical ratio of any historical version of the software is the ratio of the cumulative number of test defects in the corresponding historical inflection point sub-period to the total number of test defects in the corresponding historical test. The historical inflection point sub-period corresponding to any historical version of the software is the sub-period with the highest number of test defects among the multiple sub-periods of the corresponding historical test.

[0174] Based on the target ratio, the cumulative number of test defects of the test software in the inflection point sub-cycle, and the order of the inflection point sub-cycles of the test software in the test, the target number of defects of the test software in the target sub-cycle during the test is determined.

[0175] The number of test defects in the target sub-cycle is compared with the corresponding number of target defects, and the test software is used to determine whether the test is sufficient based on the comparison results.

[0176] Optionally, the processor 1101 is specifically used for:

[0177] Filter out abnormal historical proportions from multiple historical proportions;

[0178] The average value of the filtered historical proportions is determined as the target proportion.

[0179] Optionally, the processor 1101 is further configured to:

[0180] Determine the standard deviation corresponding to the plurality of historical proportions;

[0181] For any historical proportion, if the deviation of the historical proportion from the average of the plurality of historical proportions is greater than the standard deviation, then the historical proportion is determined to be an abnormal historical proportion.

[0182] Optionally, the processor 1101 is specifically used for:

[0183] The ratio of the cumulative number of test defects in the inflection point sub-cycle of the test software to the target ratio is determined as the total number of target defects.

[0184] The number of target defects in an inflection point sub-cycle is determined based on the total number of target defects and the order of the inflection point sub-cycles of the testing software during testing; and / or the number of target defects in other target sub-cycles is determined based on the total number of target defects, the order of the inflection point sub-cycles of the testing software during testing, and the order of other target sub-cycles of the testing software besides the inflection point sub-cycles during testing.

[0185] Optionally, if the order of the inflection point sub-cycles of the test software is not the target order, then the target sub-cycle includes the inflection point sub-cycles of the test software and the sub-cycles of the target order; if the order of the inflection point sub-cycles is the target order, then the target sub-cycle includes the inflection point sub-cycles of the test software.

[0186] The target order is determined based on the total number of sub-cycles in the test and a first preset ratio.

[0187] Optionally, the processor 1101 is specifically used for:

[0188] If the change in the number of test defects in the target sub-cycle relative to the corresponding target defect number does not exceed a preset range, then the test software is deemed to have performed sufficient testing.

[0189] Optionally, before determining that the test software test is sufficient, the processor 1101 is further configured to:

[0190] The number of test defects in the last sub-cycle of the test software is determined to be no more than a second preset ratio relative to the total number of test defects in the test.

[0191] Since the computing device is the same computing device in the method of this application embodiment, and the principle of the computing device in solving the problem is similar to that of the method, the implementation of the computing device can refer to the implementation of the method, and the repeated parts will not be described again.

[0192] Based on the same technical concept, embodiments of this application also provide a computer-readable storage medium storing a computer program executable by a computing device, which, when run on the computing device, causes the computing device to perform the steps of the evaluation method for the software test described above.

[0193] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0194] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0195] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0196] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1The steps of the function specified in one or more boxes.

[0197] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0198] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A software testing evaluation method, characterized in that, The method includes: The target ratio of the test software is determined based on the historical ratios of multiple historical versions of the test software. The historical ratio of any historical version of the software is the ratio of the cumulative number of test defects in the corresponding historical inflection point sub-period to the total number of test defects in the corresponding historical test. The historical inflection point sub-period corresponding to any historical version of the software is the sub-period with the highest number of test defects among the multiple sub-periods of the corresponding historical test. Based on the target ratio, the cumulative number of test defects of the test software in the inflection point sub-cycle, and the order of the inflection point sub-cycles of the test software in the test, the target number of defects of the test software in the target sub-cycle during the test is determined. The number of test defects in the target sub-cycle of the test software is compared with the corresponding number of target defects, and the test software is used to determine whether the test is sufficient based on the comparison results. Based on the target ratio, the cumulative number of test defects in the test software during the inflection point sub-cycle, and the order of the inflection point sub-cycles in the test, the target number of defects in the target sub-cycle of the test software during the test is determined, including: The ratio of the cumulative number of test defects in the inflection point sub-cycle of the test software to the target ratio is determined as the total number of target defects. The number of target defects in an inflection point sub-cycle is determined based on the total number of target defects and the order of the inflection point sub-cycles of the testing software during testing; and / or the number of target defects in other target sub-cycles is determined based on the total number of target defects, the order of the inflection point sub-cycles of the testing software during testing, and the order of other target sub-cycles of the testing software besides the inflection point sub-cycles during testing. If the order of the inflection point sub-cycles of the test software is not the target order, then the target sub-cycle includes the inflection point sub-cycles of the test software and the sub-cycles of the target order; if the order of the inflection point sub-cycles is the target order, then the target sub-cycle includes the inflection point sub-cycles of the test software. The target order is determined based on the total number of sub-cycles in the test software and a first preset ratio.

2. The method as described in claim 1, characterized in that, The target proportion of the test software is determined based on the historical proportions of multiple historical versions of the test software, including: Filter out abnormal historical proportions from multiple historical proportions; The average value of the filtered historical proportions is determined as the target proportion.

3. The method as described in claim 2, characterized in that, The historical proportion of anomalies is determined using the following methods: Determine the standard deviation corresponding to the plurality of historical proportions; For any historical proportion, if the deviation of the historical proportion from the average of the plurality of historical proportions is greater than the standard deviation, then the historical proportion is determined to be an abnormal historical proportion.

4. The method according to any one of claims 1-3, characterized in that, Determining whether the testing software has been adequately tested based on the comparison results includes: If the change in the number of test defects in the target sub-cycle relative to the corresponding target defect number does not exceed a preset range, then the test software is deemed to have performed sufficient testing.

5. The method as described in claim 4, characterized in that, Before determining that the test software has been adequately tested, the following steps are also included: The number of test defects in the last sub-cycle of the test software is determined to be no more than a second preset ratio relative to the total number of test defects in the test.

6. An evaluation device for software testing, characterized in that, include: The ratio determination module is used to determine the target ratio of the test software based on the historical ratios of multiple historical versions of the test software; wherein, the historical ratio of any historical version of the software is the ratio of the cumulative number of test defects in the corresponding historical inflection point sub-period to the total number of test defects in the corresponding historical test; the historical inflection point sub-period corresponding to any historical version of the software is the sub-period with the largest number of test defects among the multiple sub-periods of the corresponding historical test. The quantity determination module is used to determine the target number of defects of the test software in the target sub-cycle during the test based on the target ratio, the cumulative number of test defects of the test software in the inflection point sub-cycle, and the order of the inflection point sub-cycles of the test software in the test. The evaluation module is used to compare the number of test defects of the test software in the target sub-cycle with the corresponding target number of defects, and determine whether the test software has performed sufficient testing based on the comparison results. The quantity determination module is specifically used for: The ratio of the cumulative number of test defects in the inflection point sub-cycle of the test software to the target ratio is determined as the total number of target defects. The number of target defects in an inflection point sub-cycle is determined based on the total number of target defects and the order of the inflection point sub-cycles of the testing software during testing; and / or the number of target defects in other target sub-cycles is determined based on the total number of target defects, the order of the inflection point sub-cycles of the testing software during testing, and the order of other target sub-cycles of the testing software besides the inflection point sub-cycles during testing. If the order of the inflection point sub-cycles of the test software is not the target order, then the target sub-cycle includes the inflection point sub-cycles of the test software and the sub-cycles of the target order; if the order of the inflection point sub-cycles is the target order, then the target sub-cycle includes the inflection point sub-cycles of the test software. The target order is determined based on the total number of sub-cycles in the test software and a first preset ratio.

7. A computing device, characterized in that, It includes at least one processor and at least one memory, wherein the memory stores a computer program that, when executed by the processor, causes the processor to perform the method of any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, It stores a computer program executable by a computing device, which, when run on the computing device, causes the computing device to perform the method of any one of claims 1 to 5.

Images