Test case CBB design method, physical case generation method, and test evaluation method

Abstract

The application provides a test case CBB design method, comprising: designing a data dictionary according to product specification, adopting a black box test design method according to a CBB idea to design a logic test case, referencing the data dictionary to test data, and realizing separation of application data and logic; a physical case generation method is provided, logic test cases are selected according to scene requirements, and sub-cases are obtained by respectively processing network scene, device type and test data by using different test coverage technologies; different test coverage technologies are used on the sub-cases to obtain final physical cases; a test evaluation method is provided, and the association information in the design method and the generation method is used for test evaluation. The application solves the problem of case normalization, and solves the problems of completing risk-based test execution within a limited test period and evaluating test execution results.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to test case generation technology, and more particularly to a test case CBB design method, a physical test case generation method, and a test evaluation method. Background Technology

[0002] In contemporary technology products, especially domestically developed products, as the customer base gradually expands and customer needs diversify, product customization has become a trend and is becoming increasingly serious. This has led to an increasing number of product models and application scenarios within the same series, which has resulted in the following problems in the testing field: within a limited testing cycle, it is impossible for similar test cases to cover all device forms (i.e., test objects), all networking scenarios (i.e., test scenarios), and all user data (i.e., test data).

[0003] To address these issues, industry testing methodologies typically focus on coverage factors within a single dimension. This involves data analysis using equivalence partitioning and boundary value analysis, followed by combining data with different test coverage techniques to create test cases. However, conventional testing techniques suffer from two main drawbacks: firstly, they lack a comprehensive consideration of the test objects, test scenarios, and test data, making it difficult to accurately reproduce customer application scenarios and increasing the risk of defect leakage; secondly, without a well-developed, unified testing platform, managing the test execution process—including test design, test strategy formulation, test execution, and test evaluation—within such a multi-dimensional testing scope is challenging. If the testing process is uncontrollable, it may be difficult to accurately assess product quality before release. Summary of the Invention

[0004] This application provides a requirement-based test case design method (CBB), a risk-based physical test case generation method, and a requirement-based test evaluation method, which solve the problem of test case normalization, and address the problems of completing risk-based test execution within a limited test cycle and evaluating test execution results.

[0005] To achieve the above objectives, the present invention employs the following techniques:

[0006] A requirements-based test case design (CBB) method includes the following steps:

[0007] S101. Extract product testing requirements based on pre-defined product design requirements;

[0008] S102. Analyze the product testing requirements to obtain product specifications and test points. Design a data dictionary based on the product specifications and analyze the test points to form test case names.

[0009] S103. Design logical test cases using the black-box testing design method according to the CBB (Content-Based Test-Based) approach. Test data references a data dictionary and does not directly reflect instantiated rule data. Test logic is primarily based on requirement descriptions and does not reflect specific product implementation schemes. Test case attributes include test case name, number, level, preconditions, test steps, expected results, and associated requirements.

[0010] S104. Import the relationship between product design requirements and logical test cases into a relational database for maintenance.

[0011] The data dictionary is presented in a Key:Value format, where the Key includes one or more of the following: enumeration, minimum value, maximum value, random value, and typical value.

[0012] A risk-based physical test case generation method is used to generate physical test cases from logical test cases obtained through the test case CBB design method. The method includes generation steps for scenarios of new requirement delivery, applicable to the iterative development phase, and generation steps for scenarios of requirement acceptance, applicable to the regression testing phase.

[0013] Among them, the generation steps applicable to the iterative development phase for scenarios involving the delivery of new requirements include:

[0014] S201. Based on the product requirements list delivered during development, select the corresponding logical test cases;

[0015] S202, Full-dimensional analysis of the test execution coverage of logical test cases, including supported networking scenarios, device types, and test data;

[0016] S203. For each selected logical test case, based on the network scenario supported by the test risk analysis, different test coverage technologies are selected for processing to generate physical sub-test cases.

[0017] For each selected logical test case, based on the device type supported by the test risk analysis, different test coverage techniques are selected for processing to generate physical sub-test cases;

[0018] For each selected logical test case, the supported test data is processed using different test coverage techniques based on test risk analysis to generate physical sub-test cases.

[0019] S204. The physical sub-test cases generated in S203 according to each dimension are then processed using different test coverage techniques to generate the final executable physical test cases.

[0020] S205. After generating physical test cases in S204, the detailed information of the test coverage technology to be executed is appended to the physical test case name by means of a suffix. The association between logical test cases and physical test cases is backfilled into the relational database, and the executable physical test cases are imported into the third-party TestLink test case execution platform to carry out test case test execution.

[0021] S206. After the test is completed, import the test results and the relationship between the test results and the physical test cases into the relational database.

[0022] Specifically, for the requirement acceptance scenario, the generation steps applicable to the regression testing phase include:

[0023] S301. Based on the development bug fixes and version quality assessment, select the corresponding test logic cases;

[0024] S302. For each selected logical test case, based on the network scenarios supported by the selected test cases, and according to the test coverage and test result analysis performed in the previous iterative development phase, different test coverage techniques are selected for processing to generate physical sub-test cases.

[0025] For each selected logical test case, based on the device type supported by the previous iterative development phase, different test coverage techniques are selected for processing to generate physical sub-test cases, according to the test coverage and test result analysis performed in the previous iterative development phase.

[0026] For each selected logical test case, based on the test coverage and test result analysis performed in the previous iterative development phase, different test coverage techniques are selected for processing to generate physical sub-test cases.

[0027] S303. The physical sub-test cases generated in S302 according to each dimension are then processed using different test coverage techniques to generate the final executable physical test cases.

[0028] S304. After generating physical test cases in S303, the detailed information of the test coverage technology to be executed is appended to the physical test case name by means of a suffix. The association between logical test cases and physical test cases is backfilled into the relational database, and the executable physical test cases are imported into the third-party TestLink test case execution platform to carry out test case test execution.

[0029] S305. After the test is completed, import the test results and the relationship between the test results and the physical test cases into the relational database.

[0030] Furthermore, the test coverage techniques include single coverage, Parewise, and full orthogonal.

[0031] A requirement-based test evaluation method includes: after executing the physical test case generation method, using the association between physical test cases and test results, the association between physical test cases and logical test cases, and the association between logical test cases and product design requirements in the relational database, outputting the test execution status at the version level, including which iteration versions each logical test case has been executed in, and detailed execution records for each iteration: under what test scenario, what type of device was used, what test data, and what execution results were obtained.

[0032] The beneficial effects of this invention are as follows:

[0033] 1. It solves three problems in the existing R&D testing process: First, it solves the problem of test case normalization by designing test cases using the CBB (Contingent Protection, Comparison, and Optimization) concept; second, it solves the problem of completing risk-based test execution within a limited testing cycle by applying existing mature test coverage technologies; and third, it solves the problem of difficulty in evaluating test execution results.

[0034] 2. This paper provides a test case design method based on the CBB (Content-Based Block Design) concept, which separates data from logic. On the one hand, it decouples the product implementation scheme from the test steps of the test cases; on the other hand, it decouples the test steps of the test cases from the product specifications, thereby facilitating the maintenance of test cases in the later stage. For example, if the implementation scheme of a certain feature of the product changes, it will not lead to the redesign of the test cases; changes in the product specifications will not affect the updating of the test case steps, only the corresponding data dictionary needs to be updated.

[0035] 3. Provide a method or system architecture for generating physical test cases from logical test cases with multiple dimensions as input. Based on each CBB logical test case, combined with test scenarios, test data, and device form, multiple physical test cases are generated using different test coverage techniques. Physical test cases can be quickly generated based on logical test cases combined with test scenarios, test data, and device form. At the same time, when test case steps need to be refreshed, only the logical test cases need to be refreshed, which greatly improves the efficiency of test case maintenance.

[0036] 4. Provide a relational database to record the association between test cases and requirements, as well as the historical execution coverage of test cases in each iteration of the entire product testing cycle, which facilitates the formulation of test execution strategies for N+1 iterations and the evaluation of overall test coverage after the test is completed.

[0037] 5. This solution provides an end-to-end testing process and method that can be completed in one testing desktop, from test case design, test case strategy formulation, test case execution, and test case evaluation, which can also greatly improve work efficiency at the test management level. Attached Figure Description

[0038] Figure 1This is a diagram of the overall software system architecture involved in the design, generation, and evaluation methods of this application embodiment.

[0039] Figure 2 This is an example of a data dictionary in an embodiment of this application.

[0040] Figure 3 This is an example of a logic test case in an embodiment of this application.

[0041] Figure 4 This is an example of the generation rules in an embodiment of this application.

[0042] Figure 5 These are 15 physical test case examples generated from 3 logical test cases in the embodiments of this application.

[0043] Figure 6 This is an example of a test evaluation result record in an embodiment of this application.

[0044] Figure 7 This is a structural diagram of the test and evaluation system according to an embodiment of this application.

[0045] Figure 8 This is an example of designing a relational database using object-oriented principles, as described in this application. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the implementation methods of the present invention will be described in detail below with reference to the accompanying drawings. However, the embodiments described in this invention are only some embodiments of the present invention, and not all embodiments.

[0047] This application provides a requirement-based test case design (CBB) method, a risk-based method for generating logical test cases to physical test cases, and a requirement-based test evaluation method. The overall software system architecture involved is as follows: Figure 1 As shown.

[0048] Specifically, the design method, generation method and testing method of this embodiment will be demonstrated and explained using a sub-function of a satellite communication product as an example.

[0049] Test case CBB design phase:

[0050] S101. Extract product testing requirements based on pre-defined product design requirements.

[0051] S102. Analyze the product testing requirements to obtain product specifications and test points. Design a data dictionary based on the product specifications and analyze the test points to form test case names.

[0052] The data dictionary is presented in a Key:Value format, where the Key includes one or more of the following: enumeration, minimum value, maximum value, random value, and typical value.

[0053] like Figure 2 The data dictionary designed for this example is shown below: the dictionary names are IF_TDM_BW and IF_SpreedSpectrum, and the descriptions are TDM carrier bandwidth and spreading factor, respectively. IF_TDM_BW has valid and invalid values, and depending on the roll-off factor and device type, there are different maximum values, minimum values, typical value 1, typical value 2, ..., random values. IF_SpreedSpectrum has valid values, and depending on the carrier type, there are different default values, values ​​1, ..., values ​​6, etc.

[0054] S103. Following the CBB (Constant Best Practices) approach, black-box testing methods are used to design logical test cases. Test data references the data dictionary designed in S102 and does not reflect the specific product implementation, thus separating test case data from logic. Test logic is primarily based on requirement descriptions and does not reflect the specific product implementation. Test case attributes include test case name, number, level, preconditions, test steps, expected / predicted results, and associated requirement number, such as... Figure 3 As shown, the designed logical test cases are black-box test cases, applicable to any network topology and any device. The test case steps do not change with changes in the corresponding solution.

[0055] S104. Import the logical test cases into the relational database for maintenance, so as to realize the association between product design requirements and logical test cases backfilling into the relational database.

[0056] Then comes the generation phase from logical test cases to physical test cases. This phase includes two scenarios: one is for delivering new requirements, which is suitable for the iterative development phase; the other is for requirement acceptance, which is suitable for the regression testing phase.

[0057] For scenarios involving the delivery of new requirements, the generation steps applicable to the iterative development phase include:

[0058] S201. Based on the product requirements list delivered during development, select the corresponding logical test cases.

[0059] S202, a comprehensive analysis of the test execution coverage strategy for logical test cases, with dimensions including supported networking scenarios, device types, test data, test versions, etc.

[0060] S203. For each selected logical test case, based on the network scenario supported by the network, different test coverage techniques are selected and processed according to the test risk analysis to generate physical sub-test cases; for each selected logical test case, based on the device type supported by the device, different test coverage techniques are selected and processed according to the test risk analysis to generate physical sub-test cases; for each selected logical test case, based on the test data supported by the test, different test coverage techniques are selected and processed according to the test risk analysis to generate physical sub-test cases. The test coverage techniques include single coverage, Parewise, and full orthogonality.

[0061] S204. The physical sub-test cases generated in S203 according to each dimension are then processed using different test coverage techniques to generate the final executable physical test cases. Among them, the test coverage techniques include single coverage, Parewise, and full orthogonal.

[0062] In S203-S204, a coverage strategy is formed based on the selected test coverage technology. Specifically, a first-layer test combination A is generated for the test data according to the test coverage strategy; a first-layer test combination B is generated for the test network according to the coverage strategy; a first-layer test combination C is generated for the device type according to the coverage strategy; and a second-layer test combination D is generated for test combination A, test combination B, and test network C according to the coverage strategy. The background automatically generates physical test cases according to the physical test case generation rules, such as... Figure 4 The image shows an example of the generated physical use case.

[0063] S205. After generating physical test cases in S204, the detailed information of the test coverage strategy to be executed is appended to the test case name as a suffix to form the physical test cases that can be executed in the current version. Then, the data is backfilled into the relational database to realize the association between physical test cases and logical test cases and the backfilling of the relational database. The executable physical test cases are then imported into the third-party TestLink test case execution platform to carry out test case testing.

[0064] In this example, the three logical test cases ultimately generate 15 physical test cases according to the executed test coverage strategy, such as... Figure 5 As shown.

[0065] S206. After the test is completed, import the test results and the relationship between the test results and the physical test cases into the relational database.

[0066] Specifically, for the requirement acceptance scenario, the generation steps applicable to the regression testing phase include:

[0067] S301. This step differs from S201 in that it selects the corresponding test logic cases based on the development bug fixes and version quality assessment.

[0068] S302. For each selected logical test case, based on the network scenarios supported by the network scenarios, and according to the test coverage and test result analysis performed in the previous iterative development phase, different test coverage techniques are selected for processing to generate physical sub-test cases. For each selected logical test case, based on the device types supported by the device types, and according to the test coverage and test result analysis performed in the previous iterative development phase, different test coverage techniques are selected for processing to generate physical sub-test cases. The test coverage techniques include single coverage, Parewise, and fully orthogonal.

[0069] S303. The physical sub-test cases generated in S302 according to each dimension are then processed using different test coverage techniques to generate the final executable physical test cases. Among them, test coverage techniques include single coverage, Parewise, and full orthogonal.

[0070] S304. After generating physical test cases in S303, the detailed information of the test coverage strategy to be executed is appended to the test case name as a suffix to form the physical test cases that can be executed in the current version. Then, the association between logical test cases and physical test cases is populated back into the relational database, and the executable physical test cases are imported into the third-party TestLink test case execution platform to carry out test case test execution.

[0071] S305. After the test is completed, import the test results and the relationship between the test results and the physical test cases into the relational database.

[0072] After completing the above steps, proceed to the testing and evaluation method phase:

[0073] The backend retrieves the correlation between logical test cases and product design requirements from the relational database, the correlation between physical test cases and logical test cases from the test strategy formulation phase, and the correlation between physical test cases and test results from the test execution phase. It outputs the test execution status at the version level, including which iteration versions each logical test case has been executed in, and detailed execution records for each iteration: what device type was used in what test scenario, what test data, and what execution results were obtained.

[0074] Specifically, historical coverage data obtained through the front-end can be queried, including test scenarios, test data, covered devices, tested object versions, and execution results. This provides comprehensive and objective testing data for N+1 iteration execution strategies or test evaluation during requirement acceptance. Figure 6The table shows the test execution results after two rounds of iteration. It reveals that the test case "Spread Spectrum and Non-Spread Spectrum Carrier Remote Station Network Access Test" failed in version B020. Furthermore, the test data did not cover boundary values ​​and random values ​​for Hz bandwidth, nor did it cover boundary values ​​for adjustment modes; and it did not cover remote stations in terms of device type coverage. Therefore, these conclusions can be used as input to guide the development of the execution strategy for the third round of iteration.

[0075] In this example, the platform interface for implementing coverage strategy formulation and test execution evaluation can be a web page, an EXE file, or even an Excel file.

[0076] This example also provides a requirements-based test evaluation system for implementing the logical test case design, physical test case generation, and test evaluation processes described in the above example.

[0077] like Figure 7 The diagram shows the system framework. The system includes a CBB logical test case design module, a test case execution strategy module, a physical test case generation device, and a relational database. The CBB logical test case design module connects to the test case execution strategy module and the relational database, and also connects to the third-party TestLink test case execution platform. The test case execution strategy module connects to the relational database and the physical test case generation device. The physical test case generation device connects to the third-party TestLink test case execution platform, and the relational database connects to the third-party TestLink test case execution platform.

[0078] The CBB logic test case design module obtains pre-defined product design requirements from the data input end, extracts and analyzes product testing requirements based on these requirements, derives product specifications and test points, designs a data dictionary based on the product specifications, and analyzes the test points to form test case names. Following the CBB methodology, black-box testing is used to design logic test cases. Test data references the data dictionary and does not directly reflect instantiated rule data. Test logic is primarily based on requirement descriptions and does not reflect specific product implementation schemes. Test case attributes include test case name, number, level, preconditions, test steps, expected results, and associated requirements. The relationship between product design requirements and logic test cases is imported into a relational database for maintenance.

[0079] There are different physical sub-use case generation processes for the two different scenarios, as detailed below:

[0080] (1) In the scenario of delivering new requirements

[0081] The test case execution strategy module selects corresponding logical test cases based on the product requirement list delivered by development and analyzes test execution coverage from all dimensions, including supported network scenarios, device types, and test data. For each selected logical test case, based on the supported network scenarios and test risk analysis, different test coverage techniques are selected and executed through a physical test case generation device to generate physical sub-test cases. Similarly, for each selected logical test case, based on the supported device types and test risk analysis, different test coverage techniques are selected and executed through a physical test case generation device to generate physical sub-test cases. Finally, for each selected logical test case, based on the supported test data and test risk analysis, different test coverage techniques are selected and executed through a physical test case generation device to generate physical sub-test cases.

[0082] After generating physical sub-use cases for each dimension:

[0083] The test case execution strategy module will select different test coverage technologies based on the physical sub-test cases covered by the network scenario, physical sub-test cases covered by the device type, and physical sub-test cases covered by the test data, according to the test risk analysis. Then, through the physical test case generation device, it will generate executable physical test cases based on the network scenario, device type, and test data. The module will also append the detailed information of the executed test coverage technology to the physical test case name by renaming it, thus obtaining the association between the logical test cases and the test coverage technology strategy, that is, the association between the logical test cases and the physical test cases.

[0084] (2) In the scenario of requirements acceptance

[0085] The test case execution strategy module selects corresponding test logic test cases based on the development bug fix status and version quality assessment. For each selected logical test case, it selects different test coverage techniques based on the test coverage and test result analysis performed in the previous iteration development phase, based on the supported network scenarios. These techniques are then executed through a physical test case generation device to generate physical sub-test cases. Furthermore, for each selected logical test case, it selects different test coverage techniques based on the supported device types, based on the test coverage and test result analysis performed in the previous iteration development phase, and executes these techniques through a physical test case generation device to generate physical sub-test cases. Finally, for each selected logical test case, it selects different test coverage techniques based on the supported test data, based on the test coverage and test result analysis performed in the previous iteration development phase, and executes these techniques through a physical test case generation device to generate physical sub-test cases.

[0086] After generating physical sub-use cases for each dimension:

[0087] The test case execution strategy module will select different test coverage technologies based on the physical sub-test cases covered by the network scenario, the device type, and the test data, according to the test risk analysis. Through the physical test case generation device, it will generate executable physical test cases based on the network scenario, device type, and test data. The executed test coverage strategy information will be appended to the physical test case name by renaming, thus obtaining the association between the logical test cases and the test coverage technology strategy, that is, the association between the logical test cases and the physical test cases.

[0088] After the final executable physical use case is generated:

[0089] The test case execution strategy module backfills the association between logical test cases and physical test cases into the relational database. The physical test case generation device imports executable physical test cases into the third-party TestLink test case execution platform to carry out test case testing. The third-party TestLink test case execution platform is used to import the test results into the relational database after the test execution is completed.

[0090] In this example, to establish the association between physical use cases and execution results, physical use cases and logical use cases, and logical use cases and product requirements, a relational database is designed using object-oriented principles, such as... Figure 8 As shown. Specific relationships can be carried out through an external medium, which can be an SQL database, XML file, JSON file, Excel file, or Txt file.

[0091] The DRManage class is a product requirement set management class that provides the DRManage.MaintainDR() interface to add, delete, modify, and query product requirements.

[0092] DR class: Product requirement class, which carries the specific requirement description, and establishes a one-to-many relationship between itself and logical use cases through the DR.CaseNumIDList interface using LogicCase.CaseNum. It provides the DR.MaintainRelation interface to maintain this relationship and perform CRUD operations.

[0093] The LogicCaseManage class is a logic case set management class that provides the LogicCaseManage.MaintainCase interface to add, delete, modify, and query logic case sets.

[0094] The LogicCase class is a logical test case class that carries the test case attributes for each logical test case. It establishes a one-to-many relationship between itself and physical test cases using the PhyCase.CaseNum interface through the LogicCase.MaintainPhyCase interface. It also provides the LogicCase.CalcTestTResult interface to calculate the final test result of the logical test case based on the execution results of the corresponding multiple physical test cases.

[0095] The PhyCaseManage class is a physical test case set management class that provides the PhyCaseManage.MaintainCase() interface to add, delete, modify, and query physical test cases.

[0096] The PhyCase class is a physical test case class that contains detailed execution information for each physical test case. It manages the test data used during the execution of this physical test case via TestData.ID; the test environment used by this physical test case via TestEnv.ID; the test version of the object under test for this physical test case via Version.ID; and records the test execution result of this physical test case via Phycase.TestResult. It also provides three interfaces—Phycase.InitVersionRelation, Phycase.InitTestDataRelation, and Phycase.InitTestEnvRelation—to initialize the test version, test data, and test network associated with this physical test case.

[0097] The TestDataManage class is a test dataset management class that provides the TestDataManage.MaintainDataDict interface to perform CRUD operations on test data.

[0098] TestData class: Test data class, which contains specific test data.

[0099] The TestEnvManage class is a product requirement set management class that provides the TestEnvManage.MaintainEnvDict() interface to add, delete, modify, and query test groups.

[0100] TestEnv class: Test network class, which carries the specific test network.

[0101] The VersionManage class is a test version set management class that provides the VersionManage.MaintainVersion interface to add, delete, modify, and query test versions.

[0102] Version class: Test version class, which contains the specific test version.

[0103] Benefits of this application:

[0104] The solution presented in this example, when implemented in a satellite communication equipment development version, resulted in an end-to-end overall efficiency improvement of over 100%. The efficiency improvement can be broken down as follows:

[0105]

[0106] The above description is only a preferred embodiment of this application and is not intended to limit this application. Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application.

Claims

1. A risk-based physical use case generation method, characterized by, Used to generate physical test cases from logical test cases obtained through the CBB test case design method; The design method includes the following steps: S101. Extract product testing requirements based on pre-defined product design requirements; S102. Analyze the product testing requirements to obtain product specifications and test points. Design a data dictionary based on the product specifications and analyze the test points to form test case names. S103. Design logical test cases using the black-box testing design method according to the CBB (Content-Based Test-Based) approach. Test data references a data dictionary and does not directly reflect instantiated rule data. Test logic is primarily based on requirement descriptions and does not reflect specific product implementation schemes. Test case attributes include test case name, number, level, preconditions, test steps, expected results, and associated requirements. S104. Import the relationship between product design requirements and logical test cases into a relational database for maintenance; The generation method includes scenarios for delivering new requirements and is applicable to generation steps in the iterative development phase: S201. Based on the product requirements list delivered during development, select the corresponding logical test cases; S202, Full-dimensional analysis of the test execution coverage of logical test cases, including supported networking scenarios, device types, and test data; S203. For each selected logical test case, based on the network scenario supported by the test risk analysis, different test coverage technologies are selected for processing to generate physical sub-test cases. For each selected logical test case, based on the device type supported by the test risk analysis, different test coverage techniques are selected for processing to generate physical sub-test cases; For each selected logical test case, the supported test data is processed using different test coverage techniques based on test risk analysis to generate physical sub-test cases. S204. The physical sub-test cases generated in S203 according to each dimension are then processed using different test coverage techniques to generate the final executable physical test cases. S205. After generating physical test cases in S204, the detailed information of the test coverage technology to be executed is appended to the physical test case name by means of a suffix. The association between logical test cases and physical test cases is backfilled into the relational database, and the executable physical test cases are imported into the third-party TestLink test case execution platform to carry out test case test execution. S206. After the test is completed, import the test results and the relationship between the test results and the physical test cases into the relational database.

2. The physical use case generation method according to claim 1, characterized in that, The data dictionary is presented in a Key:Value format, where the Key includes one or more of the following: enumeration, minimum value, maximum value, random value, and typical value.

3. The physical use case generation method according to claim 1, characterized in that, The generation method also includes generation steps for the requirement acceptance scenario, applicable to the regression testing phase: S301. Based on the development bug fixes and version quality assessment, select the corresponding test logic cases; S302. For each selected logical test case, based on the network scenarios supported by the selected test cases, and according to the test coverage and test result analysis performed in the previous iterative development phase, different test coverage techniques are selected for processing to generate physical sub-test cases. For each selected logical test case, based on the device type supported by the previous iterative development phase, different test coverage techniques are selected for processing to generate physical sub-test cases, according to the test coverage and test result analysis performed in the previous iterative development phase. For each selected logical test case, based on the test coverage and test result analysis performed in the previous iterative development phase, different test coverage techniques are selected for processing to generate physical sub-test cases. S303. The physical sub-test cases generated in S302 according to each dimension are then processed using different test coverage techniques to generate the final executable physical test cases. S304. After generating physical test cases in S303, the detailed information of the test coverage technology to be executed is appended to the physical test case name by means of a suffix. The association between logical test cases and physical test cases is backfilled into the relational database, and the executable physical test cases are imported into the third-party TestLink test case execution platform to carry out test case test execution. S305. After the test is completed, import the test results and the relationship between the test results and the physical test cases into the relational database.

4. The physical use case generation method according to claim 3, characterized in that, Test coverage techniques include single coverage, Parewise, and full orthogonal.

5. A requirements-based testing and evaluation method, characterized in that, Including the following steps: Perform the physical use case generation method as described in any one of claims 1 to 4; Using the relationships between physical test cases and test results, physical test cases and logical test cases, and logical test cases and product design requirements in the relational database, the test execution status at the version level is output. This includes the cumulative number of iterations in which each logical test case has been executed, and detailed execution records for each iteration: what device type was used in what test scenario, what test data was used, and what execution results were obtained.

6. A requirements-based testing and evaluation system, characterized in that, It includes a CBB logical use case design module, a use case execution strategy module, a physical use case generation device, and a relational database; The CBB logical test case design module is used to extract and analyze product testing requirements based on predefined product design requirements, derive product specifications and test points, design a data dictionary based on product specifications, and analyze test points to generate test case names. Following the CBB methodology, black-box testing is used to design logical test cases. Test data references the data dictionary and does not directly reflect instantiated rule data. Test logic is primarily based on requirement descriptions and does not reflect specific product implementation schemes. Test case attributes include test case name, number, level, preconditions, test steps, expected results, and associated requirements. The relationship between product design requirements and logical test cases is imported into a relational database for maintenance. In the scenario of delivering new requirements: The test case execution strategy module is used to select corresponding logical test cases based on the product requirement list delivered by development and to analyze test execution coverage from all dimensions, including supported networking scenarios, device types, and test data. The test case execution strategy module is used to select different test coverage technologies for the network scenarios supported by each selected logical test case, based on test risk analysis, and execute the coverage through the physical test case generation device to generate physical sub-test cases; it is also used to select different test coverage technologies for the device types supported by each selected logical test case, based on test risk analysis, and execute the coverage through the physical test case generation device to generate physical sub-test cases; and the test case execution strategy module is used to select different test coverage technologies for the test data supported by each selected logical test case, based on test risk analysis, and execute the coverage through the physical test case generation device to generate physical sub-test cases. The test case execution strategy module will select different test coverage technologies based on the test risk analysis, and then generate executable physical test cases based on the network scenario, device type, and test data through the physical test case generation device. The module will also rename the physical test cases and append the detailed information of the executed test coverage technology to the physical test case name. In the scenario of requirements acceptance: The test case execution strategy module is used to select corresponding test logic test cases based on the development bug fix status and version quality assessment status. The test case execution strategy module is used to select different test coverage technologies for each logical test case, based on the test coverage and test result analysis performed in the previous iteration development phase, to generate physical sub-test cases by executing the coverage through the physical test case generation device, for each network scenario supported by the selected logical test case; it is also used to select different test coverage technologies for each device type supported by the selected logical test case, based on the test coverage and test result analysis performed in the previous iteration development phase, to execute the coverage through the physical test case generation device, to generate physical sub-test cases; and it is also used to select different test coverage technologies for each logical test case supported by the selected logical test case, based on the test coverage and test result analysis performed in the previous iteration development phase, to execute the coverage through the physical test case generation device, to generate physical sub-test cases. The test case execution strategy module is used to generate physical sub-test cases based on network scenario coverage, device type coverage, and test data coverage. Based on test risk analysis, it selects different test coverage technologies and generates executable physical test cases based on network scenario, device type, and test data through the physical test case generation device. The executed test coverage strategy information is then appended to the physical test case name by renaming. After the final executable physical use case is generated: The test case execution strategy module is used to backfill the association between logical test cases and physical test cases into the relational database. The physical test case generation device imports the executable physical test cases into the third-party TestLink test case execution platform to carry out test case testing. The third-party TestLink test case execution platform is used to import the test results into the relational database after the test execution is completed.

Images