Software testing system and testing method
By building a test environment in the software testing system, automatically executing test cases, and dynamically adjusting test data based on the test results, the problem of poor test accuracy in the existing technology is solved, and more efficient and accurate software testing is achieved.
Patent Information
- Application Number
- CN202411383484.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-30
- Publication Date
- 2025-02-28
AI Technical Summary
When detecting software performance, existing software testing methods lack the ability to adjust test data in combination with feedback from test results, resulting in poor test accuracy.
Provide a software testing system, including data module, configuration module, automatic testing module and optimization module. By obtaining test documents, building a test environment, automatically executing test cases, and dynamically adjusting test data based on feedback of test results using preset generation algorithms such as simulated annealing algorithm.
By combining the feedback of test results, the test data is adjusted significantly improves the accuracy of the test and ensures the effectiveness and reliability of software testing.
Smart Images
Figure 00000000_0000_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of software testing, and in particular to a software testing system and a testing method. Background Art
[0002] Software testing is the process of running or measuring a software system using manual or automatic means, with the aim of verifying whether it meets the specified requirements or clarifying the difference between expected results and actual results.
[0003] Currently, before a software system is released, it is necessary to conduct system testing to test the performance of the system. Different business scenarios of the system require the configuration of corresponding test cases and test data; among which, test data is the data used to execute the test cases.
[0004] With the continuous development of computer technology, more and more software is being put into production and life. After the software is produced, it needs to be tested. Therefore, the efficiency of using only manual testing is very low. Although there are methods of using automated systems to test software, the testing methods are relatively simple and do not combine the feedback of test results to adjust the associated test data, resulting in the defect of poor test accuracy. Summary of the invention
[0005] In view of the technical defects mentioned in the background technology, an object of the embodiments of the present invention is to provide a software testing system and a testing method.
[0006] To achieve the above objectives, in a first aspect, an embodiment of the present invention provides a software testing system, the system comprising:
[0007] A data module, used to obtain a test document; wherein the test document includes a test purpose and a test case;
[0008] Configuration module, used to create a new automatic test project and build a test environment based on the test purpose;
[0009] The automatic testing module is used to automatically execute the corresponding test cases in the constructed test environment to obtain the test results;
[0010] The optimization module is used to dynamically adjust the associated test data based on the feedback of the test results and in combination with a preset generation algorithm.
[0011] As a specific implementation of the present application, the preset generation algorithm includes a simulated annealing algorithm, and the dynamic adjustment of the associated test data specifically includes:
[0012] If the test result indicates that a defect exists, the annealing rate is reduced to generate more adjacent data to find out the data boundary causing the defect.
[0013] As a specific implementation method of the present application, when more adjacent data are generated, the next set of test data is obtained according to the current annealing rate and the current data, and then it is determined whether to accept the new data according to the associated Metropolis criterion. If accepted, it will be included in the test data set.
[0014] As a specific implementation method of the present application, when setting up the test environment, multiple test environments are generated, and each test environment has at least one configuration difference, supporting customized test environment domain names.
[0015] As a specific implementation method of the present application, the test case is also generated based on fuzzy data; wherein the fuzzy data includes a newly generated seed file, and the newly generated seed file is generated based on an improved LeakGAN; the improved LeakGAN includes generator improvements and discriminator improvements.
[0016] In a second aspect, an embodiment of the present invention further provides a software testing method, which is applied to a software testing system according to the first aspect, and the method comprises the following steps:
[0017] Obtaining a test document; wherein the test document includes a test purpose and a test case;
[0018] Create a new automatic test project and build a test environment based on the test purpose;
[0019] Automatically execute the corresponding test cases in the built test environment to obtain the test results;
[0020] Based on the feedback of the test results, the associated test data is dynamically adjusted in combination with a preset generation algorithm.
[0021] As a specific implementation of the present application, the preset generation algorithm includes a simulated annealing algorithm, and the dynamic adjustment of the associated test data specifically includes:
[0022] If the test result indicates that a defect exists, the annealing rate is reduced to generate more adjacent data to find out the data boundary causing the defect.
[0023] As a specific implementation method of the present application, when more adjacent data are generated, the next set of test data is obtained according to the current annealing rate and the current data, and then it is determined whether to accept the new data according to the associated Metropolis criterion. If accepted, it will be included in the test data set.
[0024] As a specific implementation method of the present application, when setting up the test environment, multiple test environments are generated, and each test environment has at least one configuration difference, supporting customized test environment domain names.
[0025] As a specific implementation method of the present application, the test case is also generated based on fuzzy data; wherein the fuzzy data includes a newly generated seed file, and the newly generated seed file is generated based on an improved LeakGAN; the improved LeakGAN includes generator improvements and discriminator improvements.
[0026] The technical solution provided by the embodiment of the present invention first builds a test environment according to the test purpose; and automatically executes the corresponding test cases in the built test environment to obtain test results; then based on the feedback of the test results and in combination with a preset generation algorithm, dynamically adjusts the associated test data; and realizes the adjustment of the associated test data in combination with the feedback of the test results, thereby improving the corresponding test accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] In order to more clearly illustrate the specific implementation of the present invention or the technical solution in the prior art, the drawings required for use in the specific implementation or the description of the prior art are briefly introduced below.
[0028] Figure 1 It is a principle block diagram of a software testing system provided by an embodiment of the present invention;
[0029] Figure 2 The present invention provides a flowchart of a software testing method. DETAILED DESCRIPTION
[0030] The specific embodiments of the present invention will be described in detail below. It should be noted that the embodiments described herein are only for illustration and are not intended to limit the present invention. In the following description, a large number of specific details are set forth in order to provide a thorough understanding of the present invention. However, it is apparent to those of ordinary skill in the art that these specific details need not be adopted to implement the present invention.
[0031] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0032] In the present application, the terms "upper", "lower", "left", "right", "front", "back", "top", "bottom", "inner", "outer", "middle", "vertical", "horizontal", "lateral", "longitudinal" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the drawings. These terms are mainly used to better describe the present application and its embodiments, and are not used to limit the indicated devices, elements or components to have a specific orientation, or to be constructed and operated in a specific orientation.
[0033] In addition, some of the above terms may be used to express other meanings in addition to indicating orientation or positional relationship. For example, the term "on" may also be used to express a certain dependency or connection relationship in some cases. For those of ordinary skill in the art, the specific meanings of these terms in this application can be understood according to specific circumstances.
[0034] Throughout the specification, references to "one embodiment," "an embodiment," "an example," or "an example" mean that a particular feature, structure, or characteristic described in conjunction with the embodiment or example is included in at least one embodiment of the present invention. Therefore, the phrases "in one embodiment," "in an embodiment," "an example," or "an example" appearing in various places throughout the specification do not necessarily all refer to the same embodiment or example. Furthermore, particular features, structures, or characteristics may be combined in one or more embodiments or examples in any appropriate combinations and / or subcombinations.
[0035] It should be noted that, unless otherwise specified, the technical terms in this embodiment have the common meanings understood in the relevant technical field.
[0036] Please refer to Figure 1 , an embodiment of the present invention provides a software testing system, the system comprising:
[0037] A data module, used to obtain a test document; wherein the test document includes a test purpose and a test case;
[0038] Configuration module, used to create a new automatic test project and build a test environment based on the test purpose;
[0039] The automatic testing module is used to automatically execute the corresponding test cases in the constructed test environment to obtain the test results;
[0040] The optimization module is used to dynamically adjust the associated test data based on the feedback of the test results and in combination with a preset generation algorithm.
[0041] In this embodiment, the test document also includes test methods and progress requirements and other contents well known to those skilled in the art, which will not be described in detail here;
[0042] At the same time, analyze the automated test results to find defects as early as possible; if it is a system defect, it should be submitted to the developer for repair; if it is not a system defect, check the automated test script or test environment;
[0043] We also track test bugs: record the found bugs in the defect management tool for regular tracking and processing.
[0044] Furthermore, in order to improve the efficiency of the test, when setting up the test environment, multiple test environments are generated, and each test environment has at least one configuration difference, and supports custom test environment domain names.
[0045] Through the above solution, it is convenient to deploy it in a variety of different test environments, thereby improving the comprehensiveness and efficiency of the test.
[0046] In this embodiment, the preset generation algorithm includes a simulated annealing algorithm, and the dynamically adjusting associated test data specifically includes:
[0047] If the test result indicates that a defect exists, the annealing rate is reduced to generate more adjacent data to find out the data boundary causing the defect.
[0048] During implementation, when more adjacent data are generated, the next set of test data is obtained based on the current annealing rate and the current data, and then it is determined whether to accept the new data based on the associated Metropolis criterion. If accepted, it will be included in the test data set.
[0049] This is based on the fact that if a piece of test data finds a software defect, the adjacent test data is also likely to have software problems;
[0050] That is, during the test process, the annealing rate in the simulated annealing algorithm is dynamically adjusted according to whether software defects are found in the test result data, and then a decision is made whether to accept the new value, thereby forming the final test data set and improving the test coverage.
[0051] The above scheme first builds a test environment according to the test purpose; and automatically executes the corresponding test cases in the built test environment to obtain the test results; then based on the feedback of the test results and in combination with the preset generation algorithm, the associated test data is dynamically adjusted; it realizes the adjustment of the associated test data in combination with the feedback of the test results, thereby improving the corresponding test accuracy.
[0052] It should be noted that, in another embodiment, in order to implement security analysis so as to find security vulnerabilities;
[0053] The test case is also generated based on fuzzy data; wherein the fuzzy data includes a newly generated seed file, and the newly generated seed file is generated based on an improved LeakGAN; the improved LeakGAN includes generator improvements and discriminator improvements; the LeakGAN is a derivative model based on a generative adversarial network (GAN).
[0054] In this embodiment, the discriminator is improved as follows: RNN is selected as the core of the discriminator. On this basis, a self-attention layer is added after the input layer and the pooling connection layer, respectively, to help the discriminator focus on effective features and strengthen the grammatical structure information of the seed file in the feature vector, thereby improving the output quality of the feature vector and the model discrimination ability;
[0055] The generator is improved as follows: a batch normalization layer is added to the generator to make the input value distribution more stable, which is conducive to network training and accelerates the convergence of the model; at the same time, the Softmax sampling selected in LeakGAN can only produce deterministic samples, resulting in the inability to fully utilize the probability distribution information during sampling and poor exploratory performance. By introducing Gumbel-Max sampling to control the exploratory performance of generated samples, it is helpful to expand the word search space of the model and help the model better adapt to different data distributions during training and inference, thereby improving the generalization ability of the model and the diversity of generated seeds.
[0056] Reference Figure 2 Based on the same inventive concept, an embodiment of the present invention further provides a software testing method, which is applied to a software testing system described in the first aspect, and the method comprises the following steps:
[0057] S101, obtaining a test document; wherein the test document includes a test purpose and a test case;
[0058] S102, creating a new automatic test project and building a test environment based on the test purpose;
[0059] S103, automatically executing corresponding test cases in the constructed test environment to obtain test results;
[0060] S104: Based on the feedback of the test results and in combination with a preset generation algorithm, dynamically adjust the associated test data.
[0061] Furthermore, the preset generation algorithm includes a simulated annealing algorithm, and the dynamically adjusting associated test data specifically includes:
[0062] If the test result indicates that a defect exists, the annealing rate is reduced to generate more adjacent data to find out the data boundary causing the defect.
[0063] During implementation, when more adjacent data are generated, the next set of test data is obtained based on the current annealing rate and the current data, and then it is determined whether to accept the new data based on the associated Metropolis criterion. If accepted, it will be included in the test data set.
[0064] In this embodiment, when setting up the test environment, multiple test environments are generated, and each test environment has at least one configuration difference, and supports custom test environment domain names.
[0065] The test case is also generated based on fuzzy data; wherein the fuzzy data includes a newly generated seed file, and the newly generated seed file is generated based on an improved LeakGAN; the improved LeakGAN includes generator improvements and discriminator improvements.
[0066] It should be noted that for a more specific description of the workflow of the method embodiment, please refer to the aforementioned system embodiment part, which will not be repeated here.
[0067] The entire solution first builds a test environment according to the test purpose; then automatically executes the corresponding test cases in the built test environment to obtain the test results; then dynamically adjusts the associated test data based on the feedback of the test results and in combination with a preset generation algorithm; and realizes the adjustment of the associated test data in combination with the feedback of the test results, thereby improving the corresponding test accuracy.
[0068] In the several embodiments provided in the present application, it should be understood that the disclosed software testing system and testing method can be implemented in other ways. For example, the system embodiments described above are merely schematic, for example, the division of the modules can be divided in other ways in actual implementation, for example, multiple units or components can be combined or integrated into another system or device, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, or can be electrical, mechanical or other forms of connection.
[0069] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.
Claims
1. A software testing system, characterized in that: The system comprises: A data module, used to obtain a test document; wherein the test document includes a test purpose and a test case; Configuration module, used to create a new automatic test project and build a test environment based on the test purpose; The automatic testing module is used to automatically execute the corresponding test cases in the constructed test environment to obtain the test results; The optimization module is used to dynamically adjust the associated test data based on the feedback of the test results and in combination with a preset generation algorithm.
2. A software testing system as claimed in claim 1, characterized in that: The preset generation algorithm includes a simulated annealing algorithm, and the dynamically adjusting associated test data specifically includes: If the test result indicates that a defect exists, the annealing rate is reduced to generate more adjacent data to find out the data boundary causing the defect.
3. A software testing system as claimed in claim 2, characterized in that: When more adjacent data are generated, the next set of test data is obtained according to the current annealing rate and the current data, and then the associated Metropolis criterion is used to determine whether to accept the new data. If accepted, it will be included in the test data set.
4. A software testing system as claimed in claim 3, characterized in that: When setting up the test environment, multiple test environments are generated, and each test environment has at least one configuration difference, and supports custom test environment domain names.
5. A software testing system according to any one of claims 1 to 4, characterized in that: The test case is also generated based on fuzzy data; wherein the fuzzy data includes a newly generated seed file, and the newly generated seed file is generated based on an improved LeakGAN; the improved LeakGAN includes generator improvements and discriminator improvements.
6. A software testing method, characterized in that: Applied to a software testing system as claimed in claim 5, the method comprises the following steps: Obtaining a test document; wherein the test document includes a test purpose and a test case; Create a new automatic test project and build a test environment based on the test purpose; Automatically execute the corresponding test cases in the built test environment to obtain the test results; Based on the feedback of the test results, the associated test data is dynamically adjusted in combination with a preset generation algorithm.
7. A software testing method as claimed in claim 6, characterized in that: The preset generation algorithm includes a simulated annealing algorithm, and the dynamically adjusting associated test data specifically includes: If the test result indicates that a defect exists, the annealing rate is reduced to generate more adjacent data to find out the data boundary causing the defect.
8. A software testing method as claimed in claim 7, characterized in that: When more adjacent data are generated, the next set of test data is obtained according to the current annealing rate and the current data, and then the associated Metropolis criterion is used to determine whether to accept the new data. If accepted, it will be included in the test data set.
9. A software testing method as claimed in any one of claims 6 to 8, characterized in that: When setting up the test environment, multiple test environments are generated, and each test environment has at least one configuration difference, and supports custom test environment domain names.
10. A software testing method as claimed in claim 9, characterized in that: The test case is also generated based on fuzzy data; wherein the fuzzy data includes a newly generated seed file, and the newly generated seed file is generated based on an improved LeakGAN; the improved LeakGAN includes generator improvements and discriminator improvements.