Software testing method, computer equipment, readable storage medium and program product

Abstract

The invention relates to the technical field of electronic design, in particular to a software testing method, computer equipment, a readable storage medium and a program product. According to the software testing method provided by the invention, the testing scheme is generated according to the code of the to-be-detected software through the testing strategy module, so that the pertinence is improved, and the resource consumption of the server is reduced. According to the test scheme, the metadata module and the execution module are utilized to create and run the test case, flexible configuration and automation of the test process are achieved, manpower and time cost is reduced, and efficiency and accuracy are improved. Meanwhile, through multi-level test result judgment of the judgment module, public test judgment and a test case exclusive judgment mode are combined, it is ensured that problems are exposed at the fastest speed, and the test efficiency is improved. The modularized testing process enables the method to have high expandability, new requirements can be met, it is ensured that the software quality is continuously and effectively evaluated, and the testing failure risk is reduced.

Description

Technical Field

[0001] This invention relates to the field of electronic design technology, and in particular to a software testing method, computer equipment, readable storage medium, and program product. Background Technology

[0002] EDA (Electronic Design Automation) software is a set of tools used for designing, verifying, and manufacturing electronic systems (such as chips and circuit boards). It runs through the entire electronic design process, from concept to physical implementation, all relying on EDA tools. EDA software covers the three major stages of chip design: front-end design, back-end design, and manufacturing and packaging. Among them, back-end design software (PR tools) plays a crucial role in the chip design flow. PR tools transform the front-end design results into physical layout files (GDS) that can be used for production tape-out through operations such as placement and routing. In this process, it is necessary to optimize indicators such as power consumption, performance area, etc., as much as possible while meeting various constraints.

[0003] PR tools are specialized industrial software bridging design and manufacturing, with extremely high performance requirements, leading to stringent testing and acceptance standards during development. Traditional testing methods, to ensure software quality, typically perform full-process testing after each code modification and merging, determining the impact of code changes on output results under the same test inputs. However, this approach is difficult to implement effectively in practice, primarily due to the following significant drawbacks: Full-process testing with PR tools is costly, taking hours to days, increasing with the scale of the design. Memory requirements range from tens to hundreds of GB, placing high demands on server resources. Traditional methods address this challenge by adding more servers or sacrificing test cases. Furthermore, testing lacks specificity; most software quality defects appear early in development and are relatively easy to expose. Traditional full-process testing not only results in slow problem discovery but also wastes resources, leading to low automation or high automated maintenance costs. Additionally, key metrics such as PPA / QoR are sensitive to code modifications. Traditional testing workflows easily misinterpret reasonable metric changes or even increases as "execution failures," impacting development efficiency and hindering the effective implementation of the "shift-left" testing strategy. Summary of the Invention

[0004] To address the problems of low efficiency and lack of specificity in existing software testing methods, this invention provides a software testing method, which includes: It provides a test strategy module and the code of the software to be tested. The test strategy module generates a corresponding test plan based on the input code. The test plan contains test level information. A metadata module is provided, which includes a test case query interface and a test case library. The test case library contains test scripts and test scenarios. The generated test plan is input into the metadata module. The test case query interface queries the test case library for the corresponding test scripts and test scenarios based on the test plan. The metadata module then uses the queried test scripts and test scenarios to form a test case set. An execution module is provided, which sends the test case set to the server. The execution module calls the server to execute the test tasks related to the test case set. At the same time, the execution module monitors the server load during the test execution process and determines the test execution level of the test case set based on the test level information in the test plan. The execution module automatically generates a test task distribution strategy based on the server load and the test execution level of the test case set. Then, the execution module schedules the test tasks on the server according to the generated test task distribution strategy to complete the testing of the software under test and obtain the test execution results of the corresponding test case set.

[0005] Preferably, the test strategy module stores a test strategy template. The test strategy module generates a test plan based on the test strategy template. The test elements in the test strategy template include test level information, functional coverage requirements, test dimension coverage requirements, method coverage requirements, quality attribute coverage requirements, job priority, and execution strategy mode. When providing the code of the software to be tested, a test preference form is also provided. The test preference form contains modification information for the test elements in the test strategy template. The test strategy module adjusts the test plan generated from the test strategy template based on the test preference form to obtain a new test plan.

[0006] Preferably, when querying the corresponding test scenario based on the test script within the metadata module, if multiple test scenarios are found, the queried test scenarios are filtered according to the test requirements in the test plan. The test script and the filtered test scenarios are combined to obtain multiple test case sets. Different weights are defined for the queried test scripts and test scenarios. The Aging-DP algorithm is used to select the test case set with the highest weight among the multiple test case sets as the target test case set and output it to the execution module.

[0007] Preferably, the test task distribution strategy generated by the execution module includes monitoring the running load of each server node during the test, and actively terminating and marking when the test resource limit is exceeded; allocating each test task to different server nodes according to the resource requirements of each test task; test tasks at different test execution levels in the test case set have different execution priorities, and their priority gradually decreases as the test execution level increases; tasks at the same test execution level are executed simultaneously; when server resources are insufficient, low-priority test tasks are queued until available resources that meet the requirements are available for execution.

[0008] Preferably, a judgment module is provided, which sets up a test case execution database. The judgment module judges the test execution results obtained by the execution module. The judgment process includes basic program execution checks, function point checks, and performance checks in sequence. In the function point check, the expected functional output results and their judgment methods corresponding to the test cases are consulted in the test case execution database, and the test execution result of the test case is judged. In the performance check, the corresponding performance reference data and their judgment methods are consulted in the test case execution database, and the test execution result of the test case is judged.

[0009] Preferably, a storage module is provided to store the complete test execution results of the obtained test case set that failed, while storing only the graphic files or reports of the test execution results that passed.

[0010] Preferably, the generated test plan is a task form, which is represented by structured data and can be implemented in JSON, YAML, XML or DSL format.

[0011] To address the aforementioned technical problems, the present invention also provides a computer device, including a storage device, a processor, and a computer program stored on the storage device, wherein the processor executes the computer program to implement the steps of the software testing method described above.

[0012] To address the aforementioned technical problems, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a software testing method as described in any one of claims 1 to 7.

[0013] To address the aforementioned technical problems, the present invention also provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps of the software testing method described above.

[0014] Compared with the prior art, the software testing method, computer equipment, readable storage medium, and program product of the present invention have the following advantages: 1. The software testing method provided by this invention generates a test plan based on the code of the software to be tested through a test strategy module, inputs the test plan into a metadata module, queries the test case library for corresponding test scripts and test scenarios through a test case query interface, and forms a test case set; finally, the test case set is sent to the execution module, which schedules the server to execute the test task according to the hierarchical information in the test plan and the server load, completes the testing of the software, and obtains the test execution results of the test case set.

[0015] Once developers provide software code to trigger a test request, the test strategy module automatically generates a test plan based on the code of the software under test. This automated generation improves testing efficiency. Furthermore, the test strategy module allows for flexible adjustment of the test plan; as software versions are updated and new features are added, only the code of the software under test needs to be updated, and the test strategy module automatically adjusts the test plan to adapt to new testing requirements. This makes the testing method more targeted towards the software, avoiding the need for full-process testing as in traditional methods, reducing the server resources required to complete the tests, and enabling faster test results through targeted testing using the generated test plans. This testing method also uses a metadata module and an execution module to create and run corresponding test cases to obtain test results. The metadata module can query the corresponding test case set based on the input test plan, further improving the automation of the testing process, reducing manual intervention, and lowering the manpower and time costs in the testing process. The execution module, by combining server load and the test execution level of the test case set, rationally schedules server resources to complete different levels of test tasks, achieving flexible configuration and automated execution of the testing process, improving testing efficiency and accuracy.

[0016] 2. The test strategy module involved in this invention has built-in test strategy templates, which can be used to create test plans. These templates contain various test elements to ensure the flexibility of test case configuration. This testing method also allows for the provision of a test preference form, so that the test strategy module can make corresponding adjustments to the test plans automatically generated based on the test strategy templates according to the requirements in the test preference form.

[0017] The test preference form can include key testing areas, test coverage, and test execution priorities, making the test plan more closely aligned with actual testing needs. In this way, this testing method not only improves testing flexibility but also enhances its relevance and effectiveness. Furthermore, the built-in test strategy templates and configurable test preference form together constitute the core advantages of this testing method, making the testing process more intelligent and automated.

[0018] 3. In this invention, the metadata module can retrieve corresponding test scenarios based on the test script. If the search results contain multiple test scenarios, these data will be filtered and selected according to the requirements of the test plan. Subsequently, the test script is combined with the filtered scenario data to form multiple test case sets, enabling the rapid acquisition of test cases that meet the requirements of the test plan, thus improving the efficiency and accuracy of testing. Simultaneously, different weight values ​​are assigned to the retrieved test scripts and test scenarios. Finally, the Aging-DP algorithm is used to select the test case set with the highest weight and pass it as the target test case set to the execution module. In test scripts and test scenarios, critical or common functions and data often have higher weights. However, this can lead to some test cases being executed excessively frequently, while other test cases remain unexecuted for extended periods. The Aging-DP algorithm introduces a weight aging factor. Over time, the weights of test scripts and test scenarios are multiplied by a random factor greater than 1, and the weight decreases accordingly each time a test case is selected for execution. This prioritizes test cases that have not been executed for a long time, ensuring sufficient test coverage for various functions and scales over a longer period.

[0019] 4. The execution module of this invention can monitor the server load. Once the load exceeds a preset limit, it will stop the task and mark it. It will allocate tasks to different servers according to their resource requirements. In test tasks, tasks at different levels have different priorities, and the priority decreases as the level increases. Tasks at the same level can be executed in parallel. When resources are insufficient, low-priority tasks will be queued and wait for resources. The execution module divides test cases and test execution according to levels and, combined with server resource characteristics, completes the receiving, sending, and scheduling of test tasks on the server. Compared with traditional methods, this method significantly improves the relevance, coverage, and efficiency of testing.

[0020] 5. This invention includes a judgment module containing a test case execution database. The judgment module performs multi-level evaluation of test execution results, including program fundamentals, functional points, and performance checks. Functional point checks evaluate test results based on expected results and judgment methods in the test case database; performance checks are performed based on performance reference data and judgment methods. Traditional testing methods often use decision-makers that default to file string matching, which is unsuitable for complex comparison processes and checkpoint placement. In this testing method, the judgment module also calls the queried judgment methods, greatly increasing the flexibility of test case writing. Through multi-level test result judgment, the judgment module combines traditional common test judgment with test case-specific judgment methods, ensuring that problems are exposed as quickly as possible.

[0021] 6. This invention also provides a storage module that fully stores the execution results of failed test cases, while storing only graphical files or reports summarizing the execution results of successful test cases. Traditional testing methods often produce large outputs that consume significant disk space; uncontrolled storage leads to wasted disk resources. Conversely, complete non-storage makes it difficult to trace test problems and historical data. Therefore, the storage module in this testing method only fully stores failed test cases, while storing only small files of successful test cases. This saves disk space and facilitates quick location and analysis of failed test cases by testers. The storage module design fully considers the importance of test data and storage efficiency, making the testing process more efficient and controllable. Furthermore, the storage module supports the tracing of historical test data, providing testers with comprehensive test data support and contributing to improved test quality and efficiency.

[0022] 7. In this invention, the test plan generated by the test strategy module is presented as a work form. This form is displayed in a structured data format, supporting JSON, YAML, XML, or DSL formats to meet the diverse needs of testing teams. This design enables data exchange between modules in the testing method to use a structured language, facilitating maintenance and development.

[0023] 8. The present invention also provides a computer device that has the same beneficial effects as the above-described software testing method, which will not be described in detail here.

[0024] 9. The present invention also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed, it implements the above-described software testing method and has the same beneficial effects as the above-described software testing method, which will not be elaborated here.

[0025] 10. The present invention also provides a computer program product that has the same beneficial effects as the above-described software testing method, which will not be elaborated here. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is the step flow of a software testing method provided in the first embodiment of the present invention. Figure 1 .

[0028] Figure 2This is the step flow of a software testing method provided in the first embodiment of the present invention. Figure 2 .

[0029] Figure 3 This is a schematic diagram of the test implementation process of a software testing method provided in the first embodiment of the present invention.

[0030] Figure 4 This is a schematic diagram of a computer device structure provided in the second embodiment of the present invention.

[0031] Figure 5 This is a schematic diagram of a computer-readable storage medium structure provided in the third embodiment of the present invention. Figure 6 This is a schematic diagram of a computer program product structure provided in the fourth embodiment of the present invention.

[0032] Explanation of reference numerals in the attached diagram: 10. Computer equipment; 20. Computer-readable storage media; 30. Computer program products. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0034] As specialized software bridging design and manufacturing, PR tools have stringent performance requirements, leading to rigorous testing and acceptance standards. Traditional testing methods perform full-process testing after software code modifications to assess the impact of code changes on the output. However, this approach suffers from high testing costs, long processing times, high memory requirements, and demanding server resources. Problems are slowly exposed during testing, and the low level of automation results in high maintenance costs and wasted resources. Furthermore, traditional testing methods lack specificity; some key software metrics are sensitive to code modifications, and traditional testing may misjudge changes in these metrics, impacting development efficiency.

[0035] To address the issues of inefficiency and lack of specificity in existing software testing methods, please refer to [link / reference]. Figure 1 The first embodiment of the present invention provides a software testing method, which includes: S1: Provides the test strategy module and the code of the software to be tested. The test strategy module generates the corresponding test plan based on the input code. The test plan contains test level information. S2: Provides a metadata module, which includes a test case query interface and a test case library. The test case library contains test scripts and test scenarios. The generated test plan is input into the metadata module. The test case query interface queries the test case library for the corresponding test scripts and test scenarios based on the test plan. The metadata module then uses the queried test scripts and test scenarios to form a test case set. S3: Provides an execution module that sends the test case set to the execution module. The execution module calls the server to execute the test tasks related to the test case set. At the same time, the execution module monitors the server load during the test execution process and determines the test execution level of the test case set based on the test level information in the test plan. The execution module automatically generates a test task distribution strategy based on the server load and the test execution level of the test case set. Then, the execution module schedules the test tasks on the server according to the generated test task distribution strategy to complete the testing of the software under test and obtain the test execution results of the corresponding test case set.

[0036] The software testing method provided by this invention generates a test plan based on the code of the software to be tested through a test strategy module, inputs the test plan into a metadata module, queries the test case library for corresponding test scripts and test scenarios through a test case query interface, and forms a test case set; finally, the test case set is sent to the execution module, which schedules the server to execute the test task according to the hierarchical information in the test plan and the server load, completes the testing of the software, and obtains the test execution results of the test case set.

[0037] Once developers provide software code to trigger a test request, the test strategy module automatically generates a test plan based on the code of the software under test. This automated generation improves testing efficiency. Furthermore, the test strategy module allows for flexible adjustment of the test plan; as software versions are updated and new features are added, only the code of the software under test needs to be updated, and the test strategy module automatically adjusts the test plan to adapt to new testing requirements. This makes the testing method more targeted towards the software, avoiding the need for full-process testing as in traditional methods, reducing the server resources required to complete the tests, and enabling faster test results through targeted testing using the generated test plans. This testing method also uses a metadata module and an execution module to create and run corresponding test cases to obtain test results. The metadata module can query the corresponding test case set based on the input test plan, further improving the automation of the testing process, reducing manual intervention, and lowering the manpower and time costs in the testing process. The execution module, by combining server load and the test execution level of the test case set, rationally schedules server resources to complete different levels of test tasks, achieving flexible configuration and automated execution of the testing process, improving testing efficiency and accuracy.

[0038] It's important to note that in the chip design flow, PR (Print Rendering) tools are responsible for converting the results of front-end design into production-ready layout files (GDS) through steps such as placement and routing. During this process, PR tools must optimize key metrics such as power consumption, performance, and area as much as possible while meeting a series of constraints (such as timing closure and no design rule violations). When modifying the PR tool's code, these modifications may include bug fixes, algorithm optimizations, adding new features, or adjusting the implementation logic of existing features. To ensure the software quality of the modified PR tool, traditional testing methods perform full-process testing after each code merge. This involves comprehensive testing (such as placement, routing, optimization, and approval) to ensure no regression errors at any stage. Comparison of inputs and outputs (such as GDS files, timing reports, and power analysis) verifies that the code changes have no negative impact. Only after confirmation can the modified code be finally merged. However, full-process testing is time-consuming, slowing down development iteration. Furthermore, test cases need to cover a vast number of scenarios (such as different processes and design scales), making maintenance complex. Developers also need to wait for test results for extended periods, impacting debugging efficiency.

[0039] The software testing method provided by this invention allows developers to submit modified code to the PR tool, triggering a test request. The test strategy module generates a test plan based on the code, inputs the test plan into the metadata module to obtain a test case set, and sends the test case set to the execution module to complete the software testing and obtain the test execution results of the test case set, thereby confirming the impact of the code modification on the PR tool.

[0040] Understandably, in step S1, the test strategy module stores a test strategy template. The test strategy module generates a test plan based on the test strategy template. The test elements in the test strategy template include test level information, functional coverage requirements, test dimension coverage requirements, method coverage requirements, quality attribute coverage requirements, job priority, and execution strategy mode. When providing the code of the software to be tested, a test preference form is also provided. The test preference form contains modification information for the test elements in the test strategy template. The test strategy module adjusts the test plan generated from the test strategy template based on the test preference form to obtain a new test plan.

[0041] The Test Strategy module is a structured framework or toolset used to guide and manage testing activities. It ensures that tests cover all critical scenarios and efficiently verify the impact of code changes through clear rules, processes, and automation. The Test Strategy module uses built-in test strategy templates to create test plans, ensuring flexibility in test case configuration.

[0042] This testing method also allows for the provision of a test preference form, enabling the test strategy module to adjust the test plan automatically generated based on the test strategy template according to the requirements in the test preference form. The test preference form can include the key areas of testing, test coverage, and test execution priority. For example, it can increase the size of test cases when server resources are plentiful, or force the specification of test coverage for certain data scenarios to quickly verify bug fixes through regression testing, making the test plan more closely aligned with actual testing needs. In this way, this testing method not only improves testing flexibility but also enhances the relevance and effectiveness of testing. The built-in test strategy template and configurable test preference form make the testing process more intelligent and automated.

[0043] The testing strategies module utilizes methods such as regression testing, smoke testing, performance testing, compatibility testing, and fault tolerance testing. As one implementation method, smoke testing quickly verifies whether the critical path functions correctly. For smoke testing, the strategy module selects test cases around the modified functionality, and the test scenario data volume is relatively small to ensure that issues with basic or important functions / features are quickly exposed and reported to development. Additionally, resource requirements / constraints characterize the approximate time and server resource overhead required for the testing task. Selecting appropriate test cases based on this information provides prior information for test case selection. When submitted code contains infinite loops or memory leaks, it can lead to prolonged high server usage, impacting other tests. The testing strategy module ensures that bugs do not have a destructive impact on the server environment.

[0044] Understandably, in step S1, the generated test plan is a work form. This work form uses structured data representation and can be implemented in JSON, YAML, XML, or DSL format to suit the diverse needs of testing teams. This design allows for data exchange between modules within the testing method to utilize a structured language, facilitating maintenance and development.

[0045] Understandably, in step S2, when the metadata module queries the corresponding test scenario based on the test script, if multiple test scenarios are found, the queried test scenarios are filtered according to the test requirements in the test plan. The test script and the filtered test scenarios are combined to obtain multiple test case sets. Different weights are defined for the queried test script and test scenario. The Aging-DP algorithm is used to select the test case set with the highest weight among the multiple test case sets as the target test case set and output it to the execution module.

[0046] It should be noted that the Aging-DP algorithm is an optimization method that combines device aging analysis with dynamic programming (DP), primarily used for reliability optimization during the chip physical design phase (such as placement and routing). Its core objective is to predict the impact of transistor aging on circuit performance (such as timing degradation and increased power consumption) and proactively optimize the circuit structure during the design phase, thereby extending chip lifespan or ensuring performance stability throughout its lifespan.

[0047] The test case library includes TestScript-Metadata and Database-Metabase. The metadata module retrieves the corresponding test scripts from the test script data based on the functionalities / commands covered in the test plan. After selecting a test script, it retrieves the corresponding test scenarios from the test scenario library based on the test scenarios required by the script. The test script and test scenarios are then logically combined to form a complete test case. If the search results contain multiple test scenarios, these data are filtered according to the requirements of the test plan. Filtering can be performed using constraints such as instance size and execution time to obtain test scenarios that better meet the test requirements of the test plan, reducing the amount of test scenario data and improving testing efficiency. Subsequently, the test scripts are combined with the filtered test scenarios to form multiple test case sets. The metadata module enables the rapid retrieval of test cases that meet the requirements of the test plan, improving testing efficiency and accuracy.

[0048] Furthermore, by assigning different weight values ​​to the retrieved test scripts and test scenarios, the Aging-DP algorithm is used to select the test case set with the highest weight, which is then passed to the execution module as the target test case set. In test scripts and test scenarios, critical or common functions and data often have higher weights. However, this can lead to some test cases being executed excessively frequently, while others remain unexecuted for extended periods. The Aging-DP algorithm introduces a weight aging factor. Over time, the weights of test scripts and test scenarios are multiplied by a random factor greater than 1, and each time a test case is selected for execution, its weight decreases accordingly. This prioritizes test cases that have not been executed for a long time, ensuring sufficient test coverage for various functions and scales over a longer period, thus improving test coverage and accuracy.

[0049] Understandably, in step S3, the test task distribution strategy generated by the execution module includes monitoring the running load of each server node in the test, actively terminating and marking when the test resource limit is exceeded; allocating each test task to different server nodes according to the resource requirements of each test task; test tasks at different test execution levels in the test case set have different execution priorities, and their priority gradually decreases as the test execution level increases; tasks at the same test execution level are executed simultaneously; when server resources are insufficient, low-priority test tasks will be queued until available resources that meet the requirements are available for execution.

[0050] The execution of test tasks needs to consider load balancing to avoid wasting server resources or affecting test efficiency. The execution module is responsible for receiving, sending, and scheduling all test tasks to achieve load balancing. The execution module can monitor the server load; once it exceeds preset limits, it will stop the task and mark it. It will allocate tasks to different servers based on their resource requirements. Within the test tasks, tasks at different levels have different priorities, and the priority decreases as the level increases. Tasks at the same level can be executed in parallel. When resources are insufficient, low-priority tasks will be queued and wait for resources. The execution module divides test cases and test execution according to hierarchy and, combined with server resource characteristics, completes the receiving, sending, and scheduling of test tasks on the server. Compared with traditional methods, this approach significantly improves the relevance, coverage, and efficiency of testing.

[0051] As one implementation method, smoke test tasks can be set to have the highest priority. For smoke tests, if a test case fails, the current test round will be marked as terminated. For other levels of test tasks, if a test case fails, it will wait for the test cases of the same level to complete. This behavior can be modified through the test preference form.

[0052] Understandably, please refer to Figure 2 The software testing method also includes step S4: providing a judgment module, which sets up a test case execution database. The judgment module judges the test execution results obtained by the execution module. The judgment process includes basic program execution checks, function point checks, and performance checks in sequence. In the function point check, the expected functional output results and their judgment methods corresponding to the test case are consulted in the test case execution database, and the test execution result of the test case is judged. In the performance check, the corresponding performance reference data and their judgment methods are consulted in the test case execution database, and the test execution result of the test case is judged.

[0053] This invention employs a decision module to perform multi-level evaluation of test execution results, including program fundamentals, functionalities, and performance checks. Program fundamentals include checking exit codes, resource usage overflows, and quickly identifying refresh issues and memory leaks. Traditional testing methods rely on file string matching for checks, which is unsuitable for complex comparison processes and checkpoint placement. This invention, however, utilizes queried decision methods for functionalities and performance checks, significantly increasing the flexibility of test case writing. The decision module combines traditional common test decision methods with test case-specific decision methods through multi-level test result evaluation, ensuring issues are exposed as quickly as possible.

[0054] Understandably, please refer to Figure 2 This software testing method also includes step S5: providing a storage module to completely store the test execution results of the obtained test case set that failed into the storage module, and to store only the graphic files or reports of the test execution results that passed into the storage module.

[0055] Traditional testing methods often produce large outputs, consuming significant disk space, and uncontrolled storage leads to wasted disk resources; conversely, complete neglect of storage makes it difficult to trace test issues and historical data. Therefore, this testing method's storage module only stores failed test cases in their entirety, while storing successful test cases as small files. This saves disk space and facilitates quick location and analysis of failed test cases by testers. The storage module's design fully considers the importance of test data and storage efficiency, making the testing process more efficient and controllable. Furthermore, the storage module supports the tracing of historical test data, providing testers with comprehensive test data support and contributing to improved test quality and efficiency.

[0056] Please see Figure 3As one implementation method, the software testing method provided in the first embodiment of the present invention provides modified code using a PR tool. A test strategy module automatically generates test plans and summarizes them into a task form. The method enhances targeting based on test strategy templates and test preference forms, flexibly adjusts the test strategy module according to test requirements, reduces server resources, and quickly obtains test results. Simultaneously, the test case query interface in the metadata module retrieves a test case set. The execution module distributes test tasks evenly based on the test case set, rationally allocating server resources, reducing manpower and time costs, and improving testing efficiency and accuracy. A judgment module performs multi-level evaluation, including checks on program fundamentals, functionalities, and performance, increasing the flexibility of test case writing and ensuring rapid problem exposure. A storage module stores only failed test cases in their entirety, while successfully passed test cases are stored as small files, saving disk space and facilitating rapid location and analysis of failed test cases, improving the efficiency and controllability of the testing process. Finally, based on the test results reviewed in the storage module, the modified software code is submitted for saving or rejected for modification.

[0057] To resolve the above technical issues, please refer to [link / reference]. Figure 4 The second embodiment of the present invention also provides a computer device 10, including a storage device, a processor, and a computer program stored in the storage device. The processor executes the computer program to implement the steps of the above-described software testing method, and has the same beneficial effects as the above-described software testing method, which will not be described in detail here.

[0058] To resolve the above technical issues, please refer to [link / reference]. Figure 5 The third embodiment of the present invention also provides a computer-readable storage medium 20, on which a computer program is stored. When the computer program is executed by a processor, it implements the steps of the above-described software testing method and has the same beneficial effects as the above-described software testing method, which will not be described in detail here.

[0059] To resolve the above technical issues, please refer to [link / reference]. Figure 6 The fourth embodiment of the present invention also provides a computer program product 30, including a computer program / instruction. When the computer program / instruction is executed by a processor, it implements the steps of the above-described software testing method and has the same beneficial effects as the above-described software testing method, which will not be described in detail here.

[0060] It is understood that, according to the embodiments disclosed in this invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), it performs the functions defined in the methods of this application. It should be noted that the computer-readable medium described in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. Computer-readable storage media include, but are not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on a computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0061] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0062] In the embodiments provided by this invention, it should be understood that "B corresponding to A" means that B is associated with A, and B can be determined based on A. However, it should also be understood that determining B based on A does not mean determining B solely based on A; B can also be determined based on A and / or other information.

[0063] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of the invention. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Those skilled in the art should also recognize that the embodiments described in the specification are optional embodiments, and the actions and modules involved are not necessarily essential to the invention.

[0064] In various embodiments of the present invention, it should be understood that the sequence number of each process does not necessarily imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0065] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It is particularly important to note that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0066] The software testing method, computer equipment, readable storage medium, and program product disclosed in the embodiments of the present invention have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention. Any modifications, equivalent substitutions, and improvements made within the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A software testing method, characterized in that: It provides a test strategy module and the code of the software to be tested. The test strategy module generates a corresponding test plan based on the input code. The test plan contains test level information. A metadata module is provided, which includes a test case query interface and a test case library. The test case library contains test scripts and test scenarios. The generated test plan is input into the metadata module. The test case query interface queries the test case library for the corresponding test scripts and test scenarios based on the test plan. The metadata module then uses the queried test scripts and test scenarios to form a test case set. An execution module is provided, which sends the test case set to the server. The execution module calls the server to execute the test tasks related to the test case set. At the same time, the execution module monitors the server load during the test execution process and determines the test execution level of the test case set based on the test level information in the test plan. The execution module automatically generates a test task distribution strategy by combining the server load and the test execution level of the test case set. The execution module then schedules the test tasks on the server according to the generated test task distribution strategy to complete the testing of the software under test and obtain the test execution results of the corresponding test case set.

2. The software testing method according to claim 1, characterized in that: The test strategy module stores test strategy templates. Based on these templates, the module generates test plans. Test elements within the templates include test level information, functional coverage requirements, test dimension coverage requirements, method coverage requirements, quality attribute coverage requirements, task priority, and execution strategy mode. When providing the code of the software to be tested, a test preference form is also provided. This form contains modification information for the test elements in the test strategy templates. The module adjusts the generated test plans based on the preference form to obtain new test plans.

3. The software testing method according to claim 2, characterized in that: When querying the corresponding test scenario based on the test script within the metadata module, if multiple test scenarios are found, the queried test scenarios are filtered according to the test requirements in the test plan. The test script and the filtered test scenarios are combined to obtain multiple test case sets. Different weights are defined for the queried test scripts and test scenarios. The Aging-DP algorithm is used to select the test case set with the highest weight among the multiple test case sets as the target test case set and output it to the execution module.

4. The software testing method according to claim 1, characterized in that: The test task distribution strategy generated by the execution module includes monitoring the running load of each server node during testing, actively terminating and marking when the test resource limit is exceeded; allocating test tasks to different server nodes according to their resource requirements; test tasks at different execution levels in the test case set have different execution priorities, with the priority gradually decreasing as the test execution level increases; and tasks at the same test execution level are executed simultaneously. When server resources are insufficient, low-priority test tasks are queued until available resources that meet the requirements are available for execution.

5. A software testing method according to claim 4, characterized in that: A judgment module is provided, which sets up a test case execution database. The judgment module judges the test execution results obtained by the execution module. The judgment process includes basic program execution checks, functional point checks, and performance checks in sequence. In the functional point check, the expected functional output results and their judgment methods corresponding to the test cases are consulted in the test case execution database, and the test execution result of the test case is judged. In the performance check, the corresponding performance reference data and their judgment methods are consulted in the test case execution database, and the test execution result of the test case is judged.

6. The software testing method according to claim 1, characterized in that: A storage module is provided to store the complete test execution results of failed test cases, while only storing the successful test execution results as graphical files or reports.

7. A software testing method according to claim 2, characterized in that: The generated test plan is a task form, which is represented by structured data and can be implemented in JSON, YAML, XML or DSL format.

8. A computer device, comprising a storage unit, a processor, and a computer program stored in the storage unit, characterized in that: The processor executes the computer program to implement the steps of the software testing method as described in any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of a software testing method as described in any one of claims 1 to 7.

10. A computer program product comprising a computer program / instructions, characterized in that: When the computer program / instructions are executed by the processor, they implement the steps of a software testing method as described in any one of claims 1 to 7.

Images