Software testing system based on scene construction

Abstract

The invention relates to the technical field of software testing, in particular to a software testing system based on scene construction. Comprising a test group construction module used for constructing a to-be-tested group; the software test module is used for testing the software to obtain a performance efficiency test value, a reliability test value, a compatibility test value, a functional test value and an information security test value, determining an actual test value of the software, simulating a risk intrusion event to the software in test time if a security risk easily exists in a scene of the software, and determining the security risk of the software; and determining intrusion time, detection time, disposal time, an in-detection difference coefficient and an out-of-detection difference coefficient of the risk intrusion event, and determining an actual test value of the software so as to obtain an actual state of the software. By combining the risk intrusion event, the actual state data of the software for processing the risk intrusion time in the test process is determined, the accurate information security test value of the software under the condition is defined, and the actual test value is checked synchronously, so that the adaptation test of the software application scene is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of software testing, and in particular to a software testing system built based on scenarios. Background Art

[0002] After the software is built through code, it needs to pass a number of tests to determine whether it can function properly and meet the user's time requirements.

[0003] In the process of testing software, it is usually necessary to consider multiple test indicators to evaluate the actual status of the software. However, in the current testing process, the functionality, compatibility, and stability of the software are tested to determine the test data of the software. If the direction in which the software is applied is prone to risk invasion, such as Trojans, then how to provide a test method adapted to the application environment to deepen the test suitability of the software for this situation and facilitate the provision of more accurate test data is a problem that needs to be solved at present. Summary of the invention

[0004] In view of the above-mentioned shortcomings of the prior art, the present invention provides a software testing system built based on scenarios, which can effectively solve the problem in the prior art that when the software is tested in an environment where the direction in which the software is applied is susceptible to risk invasion, it is currently impossible to provide an adapted application scenario for testing, thereby affecting the actual data of the detection software.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0006] The present invention provides a software testing system based on scenario construction, which at least includes:

[0007] A test group building module, which is used to obtain the software to be tested and build a test group;

[0008] The software testing module is used to test the software in the test group according to the usage scenarios to obtain performance efficiency test values, reliability test values, compatibility test values, functional test values ​​and information security test values, thereby clarifying the current actual test values ​​of the software, and is characterized by:

[0009] If the scenario in which the software is used is prone to security risks, the method for determining the information security test value is as follows:

[0010] During the test time, simulate risk intrusion events to the software, and clarify the intrusion time RT, detection time JT and disposal time CT of the risk intrusion events, where:

[0011] In the tested software, determine whether there is an intrusion event with the same risk. If so, determine the entry difference coefficient TJT based on the detection time JT and the treatment time CT.s TCT c , among which,

[0012] If there is a risk intrusion event without detection time JT and handling time CT, the abnormal intrusion time YRT of the risk intrusion event is calculated, and the actual test value of the software is determined thereby to obtain the actual status of the software.

[0013] Furthermore, the intrusion time RT, the detection time JT and the treatment time CT are determined as follows:

[0014] When at least two risk intrusion events are formed, determine the corresponding intrusion time RT;

[0015] Clarify whether the software has detected a risk intrusion event and how to deal with the risk intrusion event:

[0016] If detected and handled, the detection time JT and handling time CT of the corresponding risk intrusion event are obtained respectively. The inspection time RJT is obtained according to the difference between the intrusion time RT and the detection time JT, and the inspection and handling time RCT is obtained according to the difference between the detection time JT and the handling time CT.

[0017] Furthermore, the method for determining the same risk intrusion events is:

[0018] When multiple risk intrusion events are detected by the software, the detection time JT and the handling time CT of each risk intrusion event are obtained;

[0019] Sequence the risk intrusion events according to their intrusion time RT from early to late;

[0020] The risk intrusion events with the same intrusion time RT are obtained, and the risk intrusion events are recorded as the same risk intrusion events.

[0021] Furthermore, the inspection difference coefficient TJT s TCT c The construction method is:

[0022] The difference between the corresponding detection times JT of the same risk intrusion events is obtained to obtain the same entry detection difference time TJT, thereby obtaining the product of multiple same entry detection difference times TJT to obtain the entry detection difference coefficient TJT s ;

[0023] If the same risk intrusion events are all handled by the software, the difference between the corresponding handling times CT of the same risk intrusion events is obtained to obtain the same handling difference time TCT. Thus, the product of multiple same handling difference times TCT is obtained to obtain the handling difference coefficient TCT c .

[0024] Furthermore, if there is a risk intrusion event after the intrusion time RT, and the software forms a detection time JT, when the risk intrusion event is handled abnormally:

[0025] The number of risk intrusion events that failed to be processed is obtained, recorded as N, and the time between the time when each risk intrusion event failed to be processed and its corresponding intrusion time RT is recorded as the alien time. The product of multiple alien times is calculated to obtain the total alien time YT. Based on the number N and the total alien time, the alien coefficient XYT is obtained.

[0026] Furthermore, the calculation formula of the information security test value is:

[0027]

[0028] Where: RJT is the inspection time, RCT is the inspection time, JQT is the risk control time, TJT s is the inspection error coefficient, TCT c is the same-location difference time, PN is the cracking coefficient, YRT is the abnormal intrusion time, XYT is the different-location coefficient, β1, β2, β3, β4, β5, β6, β7 and β8 are weight coefficients respectively, Atx is the number of evaluation items, and Rβ is the constant correction coefficient.

[0029] Furthermore, the actual test value is determined according to the following relationship:

[0030]

[0031] Where: PJA is the actual test value, Pre is the performance efficiency test value, Sre is the reliability test value, Jre is the compatibility test value, Rre is the functional test value, Af is the information security test value, τf1, τf2, τf3, τf4 and τf5 are the corresponding weight coefficients, and Gxp is the number of calculation items.

[0032] Furthermore, the calculation formula of the reliability test value is:

[0033]

[0034] Where: Sre is the reliability test value, up is the stable operation value of the software, um is the failure time value of the software, ug is the failure handling time, ut is the number of failures of the software during the test, ub is the number of crashes of the software during the test, ω1, ω2, ω3, ω4 and ω5 are the corresponding weight coefficients respectively, Sω is the constant correction coefficient, and Stx is the number of evaluation items.

[0035] The calculation formula for the functional test value is:

[0036]

[0037] Where: Rre is the functional test value, cr is the requirement coverage, cf is the functional correctness, ck is the number of defects in the software, φ1, φ2, φ3 and φ4 are weight coefficients, Rφ is the constant correction coefficient, and Rtx is the number of evaluation items.

[0038] When the software is tested for performance efficiency, reliability, compatibility, functionality and information security, the software is placed within the test time for testing.

[0039] Compared with the known prior art, the technical solution provided by the present invention has the following beneficial effects:

[0040] By dividing the software into performance testing, stability testing, compatibility testing, functional testing and security testing, we can implement comprehensive testing of the software during testing. By combining the actual application scenarios of the software, when there are greater security risks in the software application scenarios, we can combine risk intrusion events to clarify the actual status data of the software's processing of risk intrusion time during the test process. In this way, we can define the precise information security test value of the software in this case, and simultaneously verify the actual test value to implement adaptation testing for the software application scenario, which is conducive to accurately judging the software that can be applied to the scenario and facilitating the precise definition of the software's qualification level. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0042] Figure 1 It is the overall module block diagram of the present invention. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] In the process of testing software, it is usually necessary to consider multiple test indicators to evaluate the actual status of the software. However, in the current testing process, the functionality, compatibility, and stability of the software are tested to determine the test data of the software. If the direction in which the software is applied is susceptible to risk invasion, such as Trojans, etc., then how to provide a test method adapted to the application environment to deepen the test suitability of the software for this situation? Therefore, the present invention provides a software testing system built based on scenarios.

[0045] The present invention will be further described below in conjunction with the embodiments.

[0046] Example 1 (see Figure 1 ): A software testing system built based on scenarios, including at least:

[0047] The test group building module is used to obtain the software to be tested and build a test group, wherein the software in the test group is sorted according to the actual functions required by the user, specifically:

[0048] Get the actual functions that users need;

[0049] Obtaining software corresponding to actual functions in the tested software;

[0050] The software that corresponds to the actual functions one by one is classified as Class A in the test group. If there is no one-to-one correspondence with the actual functions, that is, there is a lack of corresponding actual functions, the number of corresponding software and actual functions is clarified, and the software is classified into Class B and Class C in descending order according to the number (the same number of actual functions is classified into one category), and so on;

[0051] In the above, the test order in the group to be tested is ABC, etc., so that the group to be tested is constructed according to the actual needs of the user, so that the software that meets the actual needs of the user can be pre-tested, thereby forming a targeted test, and when the user has corresponding needs in the future, more suitable software can be quickly provided to the user.

[0052] The software testing module is used to test the software in the test group according to the usage scenarios to obtain performance efficiency test values, reliability test values, compatibility test values, functional test values ​​and information security test values. Then, the actual test values ​​of the software can be obtained by synthesizing the above multiple test values ​​to output the best software to the user, wherein:

[0053] The calculation formula for the performance efficiency test value is:

[0054]

[0055] Where: Pre is the performance efficiency test value, tl is the number of requests processed by the software per unit time, th is the maximum number of users that the software can process simultaneously, tp is the CPU data of the software under load, tc is the memory data of the software under load, te is the time it takes the software to process the request, ε1, ε2, ε3, ε4 and ε5 are the corresponding weight coefficients respectively, Pε is the constant correction coefficient, Ptx is the number of evaluation items, which is preset to 5.

[0056] It should be noted that in this scheme, when calculating the performance efficiency test value, reliability test value, compatibility test value, functional test value and information security test value, a corresponding test time is set for the software (corresponding test time is formed according to different test items to collect indicator items, which is the current conventional preset scheme and will not be elaborated here) to obtain multiple indicator parameters for executing performance test, stability test, compatibility test, functional test and security test under the test time, thereby evaluating the actual corresponding performance efficiency test value, reliability test value, compatibility test value, functional test value and information security test value of the software.

[0057] The calculation formula of the reliability test value is:

[0058]

[0059] In the formula: Sre is the reliability test value, up is the stable operation value of the software, which is determined according to the ratio of the normal operation time of the software to the test time, um is the failure time value of the software, which is determined according to the ratio of the test time to the number of failures (failure to respond, mismatch between input instructions and output, etc.), ug is the fault handling time, which is determined according to the ratio of the repair time of the software during the test process to the number of failures, and the repair time is obtained based on the repair time statistics after each failure during the test time, ut is the number of failures of the software during the test process, ub is the number of crashes of the software during the test process (such as flash back, etc.), ω1, ω2, ω3, ω4 and ω5 are the corresponding weight coefficients respectively, Sω is the constant correction coefficient, and Stx is the number of evaluation items, which is preset to 5.

[0060] In this solution, it should be noted that before testing the software, a test time (at least 2 hours) needs to be preset to ensure that the software can be fully tested within the test time and that the data obtained is more accurate.

[0061] In the above, the compatibility test value is determined based on whether the software can run on different devices, browsers and operating systems. Each time it can run on a device (Android, Ios, etc.), browser (such as Chrome, Firefox, Safari, Edge, etc.) or operating system (such as Windows, macOS, Linux, etc.) is assigned a value of 1, and the values ​​are superimposed to obtain the compatibility test value Jre.

[0062] The calculation formula for the functional test value is:

[0063]

[0064] Where: Rre is the functional test value, cr is the requirement coverage, which is obtained by the ratio of the total number of tested functions to the actual total number of functions required by users, cf is the functional correctness, which is obtained by the ratio of the number of successful function executions within the test time to the total number of function tests, ck is the number of software defects, which is obtained by the sum of the number of defective codes, functional defects (the number of defects that are not displayed or executed according to the preset functions), and user interface defects (the sum of the number of incorrect link locations and the number of confusing interface layout locations) in the software, φ1, φ2, φ3 and φ4 are weight coefficients, Rφ is a constant correction coefficient, and Rtx is the number of evaluation items, which is preset to 4.

[0065] The method for determining the information security test value is as follows (usually it is necessary to obtain in advance whether the scenario used by the user has security situations such as attacks and intrusions that are susceptible to risk events, that is, whether there are security risks. If so, the information security test value is calculated according to the following method):

[0066] During the test, multiple risk intrusion events are simulated in the software, including but not limited to intrusion Trojans, viruses, etc.

[0067] When multiple (at least two) risk intrusion events are formed, the corresponding intrusion time RT is determined, and it is clear whether the software detects the risk intrusion event and handles the risk intrusion event. If the risk intrusion event is detected and handled (such as isolation or removal), the detection time JT and the handling time CT of the corresponding risk intrusion event are obtained respectively. The inspection time RJT is obtained according to the difference between the intrusion time RT and the detection time JT, and the inspection time RCT is obtained according to the difference between the detection time JT and the handling time CT.

[0068] Among them, when multiple risk intrusion events are detected by the software, the detection time JT and the disposal time CT of each risk intrusion event are obtained, and the risk intrusion events are sequenced (sorted) from early to late according to the intrusion time RT of the risk intrusion event, thereby obtaining the risk intrusion events with the same intrusion time RT, and the risk intrusion events are recorded as the same risk intrusion events, and the difference between the corresponding detection times JT of the same risk intrusion events is obtained to obtain the same inspection difference time TJT, thereby obtaining the product between multiple same inspection difference times TJT to obtain the inspection difference coefficient TJT s ;

[0069] and:

[0070] If the same risk intrusion events are all handled by the software, the difference between the corresponding handling times CT of the same risk intrusion events is obtained to obtain the same handling difference time TCT. Thus, the product of multiple same handling difference times TCT is obtained to obtain the handling difference coefficient TCT c ;

[0071] In the above, if there is at least one risk intrusion event that has not been detected and handled after the intrusion time RT is determined and until the end of the test time (i.e., the time for performing the security test), then the time between the intrusion time RT of the risk intrusion event and the end of the test time is obtained and recorded as the intrusion time QRT, and the sum of the intrusion times QRT of multiple risk intrusion events is counted and recorded as the abnormal intrusion time YRT;

[0072] When the detection time JT is formed, it is clear whether the database or folder in the software performs a security operation. The security operation refers to whether there is a key switch for the initially encrypted database or folder. If so, the time for executing the key switch is obtained and recorded as the switching time QT. The risk control time JQT is obtained based on the difference between the detection time JT and the switching time QT. Then, the cracking coefficient PN of the key after the key switch is obtained, where the cracking coefficient PN = key length × key bit number. The information security test value Af is calculated according to the following relationship:

[0073]

[0074] Wherein: β1, β2, β3, β4, β5, β6 and β7 are weight coefficients respectively, Atx is the number of evaluation items, which is preset to 7, and Rβ is the constant correction coefficient.

[0075] Furthermore, in the above scheme, if there is a risk intrusion event after the intrusion time RT, and the software forms a detection time JT, but the risk intrusion event handling fails, it means that the software cannot handle the risk intrusion event or fails to handle it successfully, and the risk intrusion event handling exception is obtained, such as the number of failed risk intrusion events is recorded as N, and the time between the time when each risk intrusion event fails to be handled and its corresponding intrusion time RT is recorded as the difference time, the product of multiple difference times is calculated and the total difference time YT is obtained, according to In the formula, XYT is the difference coefficient, TI and T2 are weight coefficients, so the difference coefficient is substituted into the above and the information security test value Af is recalculated:

[0076]

[0077] Where: β8 is the weight coefficient, based on which the actual test value PJA in the following text can be accurately calculated to clarify the actual status data of the current software and achieve accurate judgment of the software status.

[0078] Therefore, the actual test value of the software is obtained through the above-mentioned performance efficiency test value Pre, reliability test value Sre, compatibility test value Jre, functional test value Rre and information security test value Af, wherein the actual test value is determined according to the following relationship:

[0079]

[0080] Where: PJA is the actual test value, τf1, τf2, τf3, τf4 and τf5 are the corresponding weight coefficients, Gxp is the number of calculation items, which is preset to 5;

[0081] Thus, the actual test value PJA of all the software in the test group after testing is obtained, and the preset standard test value PBJA is obtained, so that the actual test value PJA is compared with the standard test value PBJA, so that the actual test value PJA lower than the standard test value PBJA is judged as unqualified, that is, the software corresponding to the actual test value PJA is unqualified, and thus, more accurate testing of the software can be achieved through the above scheme.

[0082] Finally, it is worth mentioning that the software in the test group can be sequenced from high to low according to the qualified actual test value PJA, so that the software suitable for the actual needs of the user can be recommended to the user.

[0083] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements will not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.

Claims

1. Software testing system based on scenario construction, including: A test group building module, which is used to obtain the software to be tested and build a test group; The software testing module is used to test the software in the test group according to the usage scenarios to obtain performance efficiency test values, reliability test values, compatibility test values, functional test values ​​and information security test values, thereby clarifying the current actual test values ​​of the software, and is characterized by: If the scenario in which the software is used is prone to security risks, the method for determining the information security test value is as follows: During the test time, simulate risk intrusion events to the software, and clarify the intrusion time RT, detection time JT and disposal time CT of the risk intrusion events, where: In the tested software, determine whether there is an intrusion event with the same risk. If so, determine the entry difference coefficient TJT based on the detection time JT and the treatment time CT. s TCT c , among which, If there is a risk intrusion event without detection time JT and handling time CT, the abnormal intrusion time YRT of the risk intrusion event is calculated, and the actual test value of the software is determined thereby to obtain the actual status of the software.

2. The software testing system based on scenario construction according to claim 1, characterized in that: The method for determining the intrusion time RT, the detection time JT and the treatment time CT is: When at least two risk intrusion events are formed, determine the corresponding intrusion time RT; Clarify whether the software has detected a risk intrusion event and how to deal with the risk intrusion event: If detected and handled, the detection time JT and handling time CT of the corresponding risk intrusion event are obtained respectively. The inspection time RJT is obtained according to the difference between the intrusion time RT and the detection time JT, and the inspection and handling time RCT is obtained according to the difference between the detection time JT and the handling time CT.

3. The software testing system based on scenario construction according to claim 1, characterized in that: The method for determining the same risk intrusion event is: When multiple risk intrusion events are detected by the software, the detection time JT and the handling time CT of each risk intrusion event are obtained; Sequence the risk intrusion events according to their intrusion time RT from early to late; The risk intrusion events with the same intrusion time RT are obtained, and the risk intrusion events are recorded as the same risk intrusion events.

4. The scenario-based software testing system according to claim 1, characterized in that: The inspection difference coefficient TJT s TCT c The construction method is: The difference between the corresponding detection times JT of the same risk intrusion events is obtained to obtain the same entry detection difference time TJT, thereby obtaining the product of multiple same entry detection difference times TJT to obtain the entry detection difference coefficient TJT s ; If the same risk intrusion events are all handled by the software, the difference between the corresponding handling times CT of the same risk intrusion events is obtained to obtain the same handling difference time TCT. Thus, the product of multiple same handling difference times TCT is obtained to obtain the handling difference coefficient TCT c .

5. The scenario-based software testing system according to claim 1, characterized in that: If there is a risk intrusion event after the intrusion time RT, and the software forms a detection time JT, and the risk intrusion event is handled abnormally: The number of risk intrusion events that failed to be processed is obtained, recorded as N, and the time between the time when each risk intrusion event failed to be processed and its corresponding intrusion time RT is recorded as the alien time. The product of multiple alien times is calculated to obtain the total alien time YT. Based on the number N and the total alien time, the alien coefficient XYT is obtained.

6. The scenario-based software testing system according to claim 1, characterized in that: The calculation formula of the information security test value is: Where: RJT is the inspection time, RCT is the inspection time, JQT is the risk control time, TJT s is the inspection error coefficient, TCT c is the same-location difference time, PN is the cracking coefficient, YRT is the abnormal intrusion time, XYT is the different-location coefficient, β1, β2, β3, β4, β5, β6, β7 and β8 are weight coefficients respectively, Atx is the number of evaluation items, and Rβ is the constant correction coefficient.

7. The scenario-based software testing system according to claim 1, characterized in that: The actual test value is determined according to the following relationship: Where: PJA is the actual test value, Pre is the performance efficiency test value, Sre is the reliability test value, Jre is the compatibility test value, Rre is the functional test value, Af is the information security test value, τf1, τf2, τf3, τf4 and τf5 are the corresponding weight coefficients, and Gxp is the number of calculation items.

8. The scenario-based software testing system according to claim 1, characterized in that: The calculation formula of the reliability test value is: Where: Sre is the reliability test value, up is the stable operation value of the software, um is the failure time value of the software, ug is the failure handling time, ut is the number of failures of the software during the test, ub is the number of crashes of the software during the test, ω1, ω2, ω3, ω4 and ω5 are the corresponding weight coefficients respectively, Sω is the constant correction coefficient, and Stx is the number of evaluation items.

9. The scenario-based software testing system according to claim 1, characterized in that: The calculation formula of the functional test value is: Where: Rre is the functional test value, cr is the requirement coverage, cf is the functional correctness, ck is the number of defects in the software, φ1, φ2, φ3 and φ4 are weight coefficients, Rφ is the constant correction coefficient, and Rtx is the number of evaluation items.

10. The scenario-based software testing system according to claim 1, characterized in that: When the software is tested for performance efficiency, reliability, compatibility, functionality and information security, the software is placed within the test time to perform the test.

Images