Test method, electronic device, storage medium, and program product

Abstract

Embodiments of the present application provide a test method, an electronic device, a storage medium and a program product, relating to the technical field of artificial intelligence. The method determines a test benchmark by obtaining a to-be-tested module and a test requirement document, calls a test case recommendation model trained to convergence and a test case feature library collecting multiple historical case features, inputs the aforementioned three types of information into the model for operation to generate a recommended test case list adapted to the to-be-tested content, relies on model quantification matching to replace manual review of massive data and selection of test cases based on subjective experience, relies on standardized feature data and a converged model to ensure that test cases match the test requirements, saves manual labor in searching and comparing test cases one by one, alleviates the problem of a sharp increase in the workload of test case selection when batch modules are tested, relies on recommended test cases to carry out testing and output test results, effectively shortens the preparation time in the early stage, shortens the overall test cycle, and improves the test efficiency.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a testing method, electronic device, storage medium, and program product. Background Technology

[0002] With the rapid iteration of software products and the continuous release of various functional modules, module testing is a core aspect of controlling the quality of product launches, and the appropriate selection of test cases directly determines the test quality.

[0003] The current testing method generally adopts the implementation method of manually selecting test cases: after obtaining the module to be tested and the requirements document, the tester manually reviews the existing historical test case data, subjectively selects suitable test cases based on his own professional experience, and then executes the full test of the module to be tested one by one.

[0004] However, manual screening requires reviewing massive amounts of historical test case feature information one by one. Test case retrieval and screening consume a lot of manpower and time. When the modules to be tested are launched in batches, the workload of test case selection increases exponentially, lengthening the overall testing cycle and resulting in low efficiency of the entire module testing work. Summary of the Invention

[0005] This application provides a test method, electronic device, storage medium, and program product to improve test efficiency.

[0006] Firstly, this application provides a testing method, including:

[0007] Obtain the module to be tested and the test requirements document;

[0008] Obtain the test case recommendation model that has been trained to convergence;

[0009] Obtain the test case feature library, which includes feature data from multiple historical test cases;

[0010] The module to be tested, the test requirement document, and the test case feature library are input into the test case recommendation model that has been trained to convergence, and a list of recommended test cases is obtained. The list of recommended test cases includes multiple test cases that match the module to be tested and the test requirement document.

[0011] Based on the recommended test case list, the module to be tested is tested, and the test results are obtained.

[0012] Output the test results.

[0013] Secondly, embodiments of this application provide a testing apparatus, comprising:

[0014] The acquisition module is used to acquire the module to be tested and the test requirements document;

[0015] The acquisition module is also used to acquire test case recommendation models that have been trained to convergence;

[0016] The acquisition module is also used to acquire a test case feature library, which includes feature data of multiple historical test cases.

[0017] The testing module is used to input the module to be tested, the test requirements document, and the test case feature library into the test case recommendation model that has been trained to convergence, and obtain a list of recommended test cases. The list of recommended test cases includes multiple test cases that match the module to be tested and the test requirements document.

[0018] The testing module is also used to test the module to be tested based on the recommended test case list and obtain test results;

[0019] The output module is used to output the test results.

[0020] Thirdly, embodiments of this application provide an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0021] The memory stores instructions that the computer executes;

[0022] The processor executes computer execution instructions stored in memory to implement the methods provided above.

[0023] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the methods provided above.

[0024] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the method described above.

[0025] This application provides a testing method, electronic device, storage medium, and program product. It determines the testing benchmark by acquiring the module to be tested and the testing requirements document. It then retrieves a pre-trained, converged test case recommendation model and a test case feature library that compiles features from multiple historical test cases. These three types of information are input into the model to generate a list of recommended test cases suitable for the content to be tested. This model-based quantitative matching replaces the manual sifting of massive amounts of data and subjective experience in selecting test cases. It ensures that test cases match the testing requirements based on standardized feature data and a converged model, while eliminating the manpower cost of manually searching and comparing test cases one by one. This alleviates the problem of a surge in test case selection workload during batch module testing. Testing is conducted based on the recommended test cases, and test results are output, effectively reducing the preparation time, shortening the overall testing cycle, and improving testing efficiency. Attached Figure Description

[0026] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0027] Figure 1 A flowchart illustrating the testing method provided in the embodiments of this application;

[0028] Figure 2 This is a schematic diagram of the structure of the testing device provided in the embodiments of this application;

[0029] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0030] The accompanying drawings have illustrated specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to specific embodiments. Detailed Implementation

[0031] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0032] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of the relevant data all comply with the relevant laws, regulations, and standards of the relevant countries and regions, have taken necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation access points for users to choose to authorize or refuse.

[0033] Furthermore, the technical solution involved in this application, which involves big data analysis of user information (including but not limited to personal biometrics, identity data, consumption data, asset data, electronic terminal operation data, etc.) and the use of artificial intelligence technology for automated decision-making, and makes decisions that have a significant impact on personal rights based on the results of automated decision-making, provides users with corresponding operation entry points for users to choose to agree to or reject the results of automated decision-making; if the user chooses to reject, the process will proceed to the expert decision-making process.

[0034] It should be noted that the test methods, electronic devices, storage media, and program products provided in this application can be used in the field of artificial intelligence, or in any field other than artificial intelligence. The application fields of the test methods, electronic devices, storage media, and program products in this application are not limited.

[0035] In the context of deep digital transformation, the banking industry is comprehensively advancing the construction of financial technology. Various business modules, including mobile banking clients, personal loan approval systems, interbank fund clearing platforms, wealth management product trading modules, anti-fraud risk control engines, and corporate account management systems, are maintaining a high-frequency iteration pace. With the continuous expansion of online deposits and loans, investment and wealth management, quick payments, and insurance sales, banks launch new modules every month for functional optimization, rule upgrades, and compliance improvements. Since financial business is directly related to customer fund security and regulatory compliance requirements, compared to ordinary internet software, banking systems have unique characteristics such as extremely low fault tolerance, stringent regulatory constraints, and complex business logic chains. Any module vulnerability could potentially lead to significant risks such as accounting errors, fund theft, and violations of relevant regulations. Therefore, full-process testing before deployment has become an indispensable and crucial part of the bank's risk control system. In the testing phase, test cases serve as the basis for verifying module functions, compliance clauses, and execution of abnormal scenarios. The accuracy and rationality of the selected test cases directly determine the test coverage and the security level of the deployed system. Over the years of system development, banks have accumulated hundreds of thousands of historical test cases over decades of version iterations, covering all scenarios such as accounting, compliance verification, exception reporting, and boundary transactions. The management and reuse of this massive amount of existing test cases has gradually become a major challenge in the field of bank testing.

[0036] Currently, the testing of financial modules still largely relies on the traditional method of manually selecting test cases. After a bank project enters the testing phase, test engineers first receive a specification document for the module to be tested, along with two versions of test requirement documents containing business rules, regulatory details, and product requirements. In addition to standard product function descriptions, financial requirement documents also include additional constraints such as regulatory compliance clauses, accounting logic, and People's Bank of China clearing regulations. The document volume and information complexity far exceed those of general software requirements. After completing the requirement review, testers need to access historical test cases from different business lines across multiple internal document systems and test management platforms. These historical test cases belong to multiple independent databases, including core accounting, retail channels, and risk control / anti-fraud databases, lacking a unified structured index. Engineers can only rely on manually sifting through scattered archived test case materials. The screening process relies entirely on the testers' years of experience in the banking industry and their past experience in similar projects to make subjective judgments. They combine their understanding of regulatory policies and business logic to manually select test cases that are suitable for the current module under test from a massive number of existing test cases. After the test case list is finalized, the functional testing, compliance testing, and abnormal stress testing of the financial module under test are carried out step by step according to the list. There are no automated matching tools involved in the entire process from test case screening to test implementation. The entire process is completed by human subjective operation.

[0037] However, the manual test case selection model has revealed significant efficiency shortcomings in the context of large-scale version deployment in banks, becoming a key pain point restricting the on-time delivery of bank test projects. Banks typically conduct batch version iterations on a quarterly or monthly basis. A single iteration often simultaneously launches new modules for more than ten different businesses, such as wealth management, credit, and payment. At this time, the number of test cases that need to be matched increases exponentially. Banks have accumulated hundreds of thousands of historical test cases, covering thousands of transaction scenarios and hundreds of regulatory compliance verification points. Testers need to spend a huge amount of time searching, reviewing, and comparing the characteristics of each test case. The test case selection phase of a single medium-sized bank's quarterly version often occupies more than half of the test team's project cycle. When a large number of modules are simultaneously awaiting testing, the workload of test case selection increases exponentially. The originally reserved testing period is largely squeezed out by the preliminary screening work, and the overall testing period is forced to be extended. If the test case screening time is hastily shortened in order to catch up with the launch node, there is a high risk of missing key compliant test cases. In summary, the method of selecting test cases entirely manually ultimately results in low overall testing efficiency of banking and financial modules, which is not conducive to the efficient iteration and implementation of banking digital products.

[0038] To address the low testing efficiency caused by manual test case selection in existing banking and financial modules, this application proposes the following technical concept: First, comprehensively collect the data of the banking and financial module to be tested, along with supporting business and compliance test requirement documents. Simultaneously, prepare a test case recommendation model that has undergone convergence training in advance, and build a test case feature library that summarizes all historical financial test case feature data. Then, input the information of the module to be tested, the test requirement documents, and the full test case feature library data into the trained recommendation model. Relying on the model's algorithm, perform multi-dimensional data matching calculations and automatically output a list of recommended test cases that accurately matches the current financial module and compliance requirements. Finally, conduct item-by-item testing of the banking module directly based on the recommended test case list output by the model, collect and summarize the test results obtained from the module tests, and ultimately output a complete test report. This approach, relying on automatic matching by algorithms to replace the traditional mode of manually sifting through massive amounts of data and subjectively selecting test cases, solves the technical problems of time-consuming manual selection and low efficiency caused by the surge in batch testing workload from the source of test case selection.

[0039] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0040] Figure 1 A flowchart illustrating the testing method provided in the embodiments of this application is shown below. Figure 1 As shown, the method includes:

[0041] S101. Obtain the module to be tested and the test requirements document.

[0042] In this step, the executing entity can be a test scheduling server deployed in the test platform, an automated test management device, a task node in a continuous integration pipeline, or a terminal device with data processing capabilities. The module to be tested indicates the target module that needs to be tested, and the test requirement document characterizes the test objectives, test scope, or test constraints corresponding to the module to be tested. In practice, trigger information for the current test task can be received from the code repository, requirement management platform, or test management platform. Upon receiving a version submission test instruction, merge request trigger signal, pre-release verification task, or manually initiated test application, the task identifier, version number, branch information, service name, component name, or interface identifier is parsed to determine the module to be tested. The module to be tested can be a target module containing code changes, or any module whose functional correctness, stability, or business constraints need to be verified, such as an e-commerce settlement module, account login module, order processing module, payment interface module, access control module, or message distribution module. Test requirement documents can be text describing requirements in natural language or structured test requirement information, such as a combination of requirement title, requirement number, function point description, input constraints, expected output, acceptance conditions, boundary conditions, exception rules, and business priorities.

[0043] In practice, the code change records, commit notes, affected file list, interface definition changes, and module dependency information corresponding to the module under test can be obtained by calling the code repository's open interface. The requirement management platform can also be used to obtain the requirement items, iteration descriptions, business process change descriptions, and acceptance criteria associated with the module. Furthermore, the test management platform can be used to obtain the module identifiers and requirement attachments already entered in the test task form. For cases with multi-source data, association mapping can be performed according to version number, requirement number, module identifier, or service identifier, unifying data from multiple sources into the input object corresponding to the same test task. To facilitate subsequent model processing, this step can also preprocess the information of the module under test, such as extracting the module name, service, call chain position, code change file path, interface parameter changes, configuration item changes, and affected upstream and downstream relationships, and converting them into structured fields. Simultaneously, the test requirement document undergoes text cleaning, word segmentation, noise reduction, terminology normalization, and paragraph parsing to form a requirement information representation suitable for subsequent joint matching. Based on the above analysis, by simultaneously acquiring the module to be tested and the test requirements document at the beginning of the testing task, the actual attributes of the object under test and the constraints of the current testing objectives can be incorporated into the subsequent recommendation process. This avoids omissions caused by relying solely on a single code change or a single text similarity for test case selection, thus providing a complete input foundation for improving the coverage and relevance of test case recommendations. In one possible embodiment, the acquisition of the module to be tested can also be combined with static analysis results or service registration information to identify the actually affected sub-modules, interface sets, or business process nodes; the test requirements document can also include risk statements and key verification items supplemented by product, development, or testing personnel. It should be understood that the above examples are merely illustrative and not limiting.

[0044] S102. Obtain the test case recommendation model that has been trained to convergence.

[0045] In this step, the test case recommendation model receives the module to be tested, the test requirement document, and the test case feature library, and outputs a list of recommended test cases. Specifically, after training convergence, it serves as the core matching and ranking model. The module to be tested, the test requirement document, and the feature library containing feature data from multiple historical test cases are input into the model. Based on the model's calculation of the relationships between features, multiple test cases matching the module to be tested and the test requirement document are obtained. "Trained to convergence" indicates that the model has been trained using historical test data and has reached a state suitable for inference. Convergence conditions can be reflected in the following: the model parameter update magnitude is lower than a preset threshold, the validation set loss tends to stabilize over multiple consecutive training rounds, the ranking metric reaches a preset performance threshold, or the accuracy, recall, and cumulative gain of the recommendation results on the validation samples meet the deployment requirements.

[0046] In practice, the executing entity can load the corresponding version of the test case recommendation model from the model repository, model service platform, or local model directory, and read its model configuration, feature dictionary, input format constraints, vector encoding rules, and inference parameters. This model can be implemented using a deep learning ranking model, a dual-tower matching model, a cross-encoding semantic matching model, a graph association inference model, or a hybrid model that integrates rules and learning capabilities, as long as it can perform joint matching and ranking between the module to be tested, the test requirement document, and historical test case feature data. In an exemplary implementation, during the training phase, the model uses historical test tasks, requirement descriptions, executed test cases, and their execution effects as supervision data. The model parameters are continuously updated by minimizing the ranking loss function or classification loss function until the model's error on the validation set decreases to a preset range or the performance improvement plateaus. The trained model is then solidified as an inference model for use in this step. During deployment, multiple model versions can be maintained for different business lines, different microservice domains, or different testing phases. The executing entity selects the corresponding model version based on the business tag, system type, or environment type of the current test task.

[0047] To ensure inference stability, this step can also perform model validity verification, such as verifying the integrity of the model file, the consistency of parameter dimensions, whether the feature mapping table version matches, and whether the model service instance is available. After passing the verification, the model is loaded into the memory or container inference service, awaiting subsequent input. Based on the above analysis, obtaining a test case recommendation model that has been trained to convergence first allows the subsequent recommendation process to no longer rely on manually defined fixed selection rules. Instead, it can leverage the relationships accumulated in historical testing activities to jointly model the modules to be tested, test requirement documents, and historical test assets, thereby maintaining the stability and adaptability of the recommendation results as the software system continues to evolve, module responsibilities change, and historical test cases expand. In one possible embodiment, the model can also carry a business lexicon, a module embedding vector table, and a test case label encoder to improve the recognition ability of domain terms and cross-module relationships. It should be understood that the above examples are for demonstration purposes only and are not limiting.

[0048] S103. Obtain the test case feature library, which includes feature data of multiple historical test cases.

[0049] In this step, the test case feature library provides feature data of historical test cases as a feature set for model input. Specifically, it aggregates feature data from multiple historical test cases, serving as the foundational data source for subsequent recommendation model matching and ranking, thus supporting test case filtering and recommendation. The feature data of historical test cases may include at least one of the following: covered modules, associated requirements, step descriptions, input / output parameters, assertion conditions, execution duration, execution frequency, failure rate, defect association records, applicable environment, test case type, priority, update time, API call information, and historical coverage effects. To facilitate numerical calculations by the model, the test case feature library may also include feature vectors derived from the above features. For example, text descriptions may be converted into semantic vectors using an encoder; discrete features such as module identifiers, test case tags, and execution results may be converted into embedded representations; and statistical features such as execution duration, stability scores, and defect hit rates may be converted into normalized values.

[0050] In practice, historical test case data can be extracted from test case management platforms, historical execution log databases, defect management systems, continuous integration platforms, and quality analysis platforms. This data is then merged according to the unique identifier of each test case to form a unified test case record. For each historical test case record, information such as its title, preconditions, test steps, expected results, associated modules, associated interfaces, applicable version range, environmental requirements, dependent data, historical execution status, and associated defect number can be extracted. Data cleaning, deduplication, missing field completion, and format standardization are then performed. For textual features, terminology normalization, entity recognition, and semantic encoding can be performed. For structured features, field mapping, enumeration encoding, and statistical calculations can be performed. For defect association information, the defect level, recurrence frequency, and risk distribution of the corresponding module exposed by the test case can be further extracted. This ensures that historical test assets not only reflect the similarity of textual content but also reflect actual execution value and risk coverage capabilities.

[0051] In one possible embodiment, the test case feature library can be stored as a combination of a relational database, a vector retrieval library, and a document database. The relational database stores test case identifiers, tags, and statistical attributes; the document database stores long text steps and descriptions; and the vector retrieval library stores semantic vectors of test cases to support rapid matching and retrieval by the model in large-scale historical test cases. The test case feature library can also be indexed according to business domain, system module, environment type, and version range to reduce the interference of irrelevant historical test cases on the recommendation process. Based on the above analysis, by acquiring a test case feature library containing feature data from multiple historical test cases, long-term accumulated test assets can be transformed into computable, searchable, and sortable model input objects. This allows the model to comprehensively consider module relationships, requirement semantics, execution value, and risk information, achieving effective screening of massive historical test cases, thereby reducing the burden of manual retrieval and improving the completeness of the recommendation base data. In one possible embodiment, for historical test cases lacking some structured attributes, some feature fields can be automatically inferred based on their step text, execution logs, and associated modules to enhance the usability of the feature library. It should be understood that the above examples are merely illustrative and not limiting.

[0052] S104. Input the module to be tested, the test requirement document, and the test case feature library into the test case recommendation model that has been trained to convergence to obtain a recommended test case list. The recommended test case list includes multiple test cases that match the module to be tested and the test requirement document.

[0053] In this step, the recommended test case list serves as the basis for multiple matching test cases output by the model and is used as the target for subsequent test execution. Specifically, it is a list generated by the test case recommendation model based on the module to be tested and the test requirement document. The test cases in the list are matched with the module to be tested and the test requirement document for use in subsequent tests. During execution, firstly, the module information, dependencies, code change summaries, interface change information, and risk tags corresponding to the module to be tested are encoded into input representations recognizable by the model. Simultaneously, the requirement semantics, functional objectives, boundary conditions, acceptance criteria, and business rules corresponding to the test requirement document are encoded into requirement representations. Then, multiple historical test case feature data or historical test case feature vectors from the test case feature library are input into the model as a candidate set. During inference, the model calculates the association between the module to be tested and historical test cases, the matching relationship between the test requirement document and historical test cases, and the joint constraints between the module and the requirements. It outputs a matching score for each historical test case relative to the current test task and sorts them based on the matching scores to generate the recommended test case list.

[0054] In an exemplary implementation, candidate recall can be performed on the test case feature library using the information of the module to be tested, filtering out a set of historical test cases covering the same module, related interfaces, or adjacent dependency chains. This subset is then refined based on the test requirements document. The model can integrate module similarity scores, requirement semantic similarity scores, historical defect association weights, boundary scenario coverage weights, and execution stability weights to form a comprehensive ranking score. For example, the comprehensive scoring formula Score = α×M + β×R + γ×D + δ×S can be used, where M represents the module association score, R represents the requirement matching score, D represents the defect risk coverage score, S represents the historical execution stability score, and α, β, γ, and δ represent the corresponding weight parameters, with the sum of all weights set to 1. This formula maps the test value of different dimensions into a ranking score. The module association score ensures that the recommended results are relevant to the actual tested object; the requirement matching score ensures that the recommended results cover the current test objectives and business rules; the defect risk coverage score increases attention to high-risk historical problem scenarios; and the execution stability score reduces the interference of low-executability test cases on the results. The weight parameters can be determined based on historical validation results. For example, in scenarios where requirements change frequently, the weight of requirement matching score can be increased, and in regression testing scenarios, the weight of module association score and defect risk coverage score can be increased, so that the recommendation results are more in line with the characteristics of the current testing task.

[0055] After generating the recommended test case list, further processing can be performed, including quantity reduction, duplicate removal, type balancing, and dependency verification. For example, the top N test cases can be selected based on matching scores from highest to lowest, where N can be automatically determined based on test resources, execution window duration, or target coverage. For historical test cases with semantic repetition or highly overlapping steps, only those with higher ranking or execution value can be retained. For different types of test cases, such as functional, boundary, exception, security, and performance compatibility test cases, minimum quotas can be set to maintain coverage balance. For test cases with environmental, data, and sequence dependencies, execution prerequisites can be appended to the list. Based on the above analysis, this step is crucial for solving technical problems. By jointly inputting the module to be tested, test requirement documents, and test case feature library into a pre-trained and converged test case recommendation model, it is possible to overcome the limitations of single recommendation methods that rely solely on code change analysis or semantic matching of requirements. This allows the recommendation results to simultaneously reflect module adaptability and requirement relevance, and combines historical execution value to optimize ranking, thereby improving the accuracy, coverage, and continuous adaptability of test case recommendations in complex software evolution scenarios. In one possible embodiment, the recommended test case list may also include a reason for recommending each test case, such as the associated module, the hit requirement keywords, the covered boundary constraints, or the associated historical defects, to facilitate review by testers. It should be understood that the above example is merely illustrative and not limiting.

[0056] S105. Based on the recommended test case list, test the module to be tested and obtain the test results.

[0057] In this step, multiple test cases from the recommended test case list are used as the actual execution targets to verify the functional correctness, boundary conditions, exception handling capabilities, or business rule compliance of the module under test. In practice, the execution entity can distribute the recommended test case list to automated test execution environments, interface testing engines, front-end regression testing frameworks, service integration environments, or manually assisted execution terminals, and execute tests according to the preconditions, input parameters, step scripts, assertion conditions, and environment requirements defined in the test cases. For test cases that can be executed automatically, the scheduler can execute them serially or in parallel based on test case dependencies and environmental resource conditions. For test cases requiring manual intervention, the recommended results can be pushed to testers, who can then complete the manual operation guided by the list, while the system records the operation process and execution results. The module under test can be deployed in development environments, test environments, integration environments, pre-release environments, or isolated container instances. Before execution, the system can complete environment initialization, test data loading, account preparation, service activation, and configuration verification to ensure that the test cases are executable.

[0058] During execution, scripts can be triggered sequentially according to the priority of each test case in the recommended test case list. Corresponding request parameters, page interaction actions, or message queue events are input to the module under test. Return results, logs, status codes, page rendering status, database change records, and tracing information are collected, and assertions are made based on the expected results of the test cases. If the assertion is true, the test case is marked as passed; if the assertion is false, the test case is marked as failed, and the failure type, failure steps, difference between actual and expected output, exception stack trace, and related log fragments are recorded. If new problem clues are discovered during execution, defect candidate records can be generated and associated with specific test cases and the module under test. In one possible embodiment, the execution results of multiple test cases can be summarized and statistically analyzed to obtain the number of passes, failures, blocking, abnormal termination, number of covered functional points, and number of hit risk points, thereby forming the overall test results for the current module under test.

[0059] Based on the above analysis, testing the module under test based on the recommended test case list does not simply output recommended results. Instead, it establishes a closed loop between model recommendations and actual execution, ensuring that highly relevant test cases are directly implemented in testing activities. This verifies the effectiveness of the recommendations in a real-world environment and generates results data that can be used for quality assessment. This processing logic helps shorten the time from test task triggering to test conclusion output, improving response efficiency in regression testing, requirement change testing, and continuous integration testing scenarios. Furthermore, since the recommended list comprehensively considers module and requirement characteristics, the resulting test results typically better reflect the real risks brought about by current version changes. In one possible embodiment, test results can also be retained for subsequent model calibration and feature updates to enhance the model's adaptability to new versions and new business changes. It should be understood that the above examples are merely illustrative and not limiting.

[0060] S106. Output the test results.

[0061] In this step, outputting test results refers to providing the test execution conclusions obtained in the previous step in a displayable, storable, and traceable format. Test results can include test pass results, test failure results, defect discovery results, execution status of each test case, a summary of failure reasons, risk level assessment, coverage statistics, and other result information related to this testing task. In practice, test results can be output to a test management platform to update the test task status and archive execution records; they can be output to a test reporting system to generate structured or text-based reports; and they can also be output to a user interface, instant messaging channel, email, or continuous integration pipeline results page to display the current test status of the module under test to testers, developers, product personnel, or release managers.

[0062] In an exemplary implementation, the system can associate and store the recommended test case list with the actual execution results. For each test case, it records its recommended score, execution start time, execution end time, execution environment, assertion results, relevant log links, and defect ticket links, and outputs them in the form of tables, charts, or structured data objects. If there are failed test cases, the output can be further annotated with the functional points, boundary scenarios, or business rules where the failures occurred in a concentrated manner to help locate the problem. If all tests pass, a conclusion label applicable to online review or regression verification can be output. For automation platforms, the test results can also be encapsulated as standard interface responses or message events for continued use in subsequent quality access control, release approval, or risk assessment processes. To ensure the traceability of results, this step can also archive the test results by version, establishing a relationship between them and the version of the module under test, requirement number, test environment identifier, and execution batch.

[0063] Based on the above analysis, outputting test results formally implements the recommendations and execution conclusions formed by the testing method into the software development process. This ensures that testing activities not only complete recommendations and executions but also generate data products that can be recorded, analyzed, compared, and used for decision-making, thereby supporting version acceptance, quality review, and issue tracking. By structured outputting the results, it also provides foundational data for subsequent statistical analysis of recommendation accuracy, evaluation of test asset value, and continuous model optimization. This application's embodiment, by jointly inputting the module to be tested, test requirement documents, and test case feature library into a pre-trained and converged test case recommendation model, obtains a list of recommended test cases matching the current testing task. Based on this list, tests are executed and test results are output. This establishes a complete closed-loop processing mechanism from test task input, model recommendation, test execution, to result output. Compared to methods that solely rely on code change mapping or requirement text matching, this approach more effectively balances module applicability, requirement relevance, and historical test value, reduces manual screening costs, improves test coverage and recommendation accuracy, and enhances the method's adaptability in scenarios of continuous software evolution. Based on the above analysis, this application provides a testing method that includes acquiring the module to be tested and the test requirement document, acquiring a test case recommendation model that has been trained to convergence, acquiring a test case feature library, inputting the module to be tested, the test requirement document, and the test case feature library into the trained to convergence test case recommendation model to obtain a recommended test case list, testing the module to be tested based on the recommended test case list, obtaining test results, and outputting the test results. In this embodiment, by uniformly modeling and jointly reasoning the information of the object under test, the requirement constraint information, and the characteristics of historical test assets, the selection of test cases is transformed from manual retrieval to model-driven recommendation oriented to the current task. Combined with test execution and result output, a closed loop is formed, which can improve the efficiency of test resource utilization in complex software testing scenarios, reduce the risk of missing key test scenarios, and provide stable technical support for continuous integration, regression testing, and requirement change testing. It should be understood that the above examples are only illustrative and not limiting.

[0064] Based on the aforementioned embodiments, further, obtaining a test case recommendation model that has been trained to convergence includes: obtaining test data, which includes at least one of system source code, code structure information, functional requirement documents, historical test case sets, and defect reports; preprocessing the test data to obtain a test dataset; obtaining an initial test case recommendation model; and training the initial test case recommendation model to convergence based on the test dataset to obtain a test case recommendation model that has been trained to convergence.

[0065] The test data serves as the raw input data for training the converged test case recommendation model. It can originate from code repositories, requirement management platforms, test case management platforms, or defect tracking systems, and is aggregated uniformly before being used in model training. The test dataset is a collection of training data after the test data has been cleaned, denoised, formatted, and feature-enhanced, used to support subsequent model parameter updates. The initial test case recommendation model serves as the model base for training. It can typically be a neural network model with randomly initialized parameters, a pre-trained model transferred from another model, or an incremental model based on a historical recommendation model. In practical applications, other models can also be selected, and this embodiment does not limit the choice.

[0066] In practical implementation, after acquiring test data, the system source code and code structure information are first parsed to extract function, class, and interface call relationships, as well as module dependencies. The functional requirements document is then processed through word segmentation, stop word removal, and semantic vectorization. Historical test case sets are used to extract test case titles, execution steps, covered modules, and execution results. Defect reports are used to extract defect types, located modules, and related repair information. Subsequently, data from different sources are converted into a unified feature representation, and missing values, duplicate samples, and abnormal text are removed to form a test dataset for model learning. Based on this test dataset, the initial test case recommendation model undergoes supervised or joint training. By iteratively updating the model parameters, the loss function gradually converges to a preset threshold or the decrease in magnitude meets the convergence condition after several consecutive rounds, thus obtaining a converged test case recommendation model.

[0067] The model training process works by mapping code change information, requirement semantics, historical use case features, and defect exposure features to a unified representation space, enabling the model to learn the correlation patterns between test objects and historical test assets. Because the training data simultaneously covers multi-source information from code, requirements, use cases, and defects, the model can simultaneously capture functional similarity, structural dependencies, and risk correlations, thereby improving the recommendation model's adaptability to complex testing scenarios.

[0068] By adopting this implementation method, the test case recommendation model can form stable feature representations based on more complete data sources, reduce the bias caused by single text matching or single code association, and improve the accuracy and coverage of recommendations. At the same time, after convergent training, the model's parameter state is stable, which can provide a consistent reasoning basis for subsequent test case recommendations, reduce manual screening costs, and enhance adaptability to version evolution and demand change scenarios.

[0069] In one possible implementation, the test data is preprocessed, including at least one of the following: semantic vectorization of each requirement description text in the functional requirements document; classification and labeling of defect reports; standardization of code structure information; code feature extraction of system source code; and feature labeling of historical test case sets.

[0070] The functional requirements document (FQD) outlines the business objectives, constraints, and acceptance criteria for the modules under test. Each requirement description can be a natural language sentence, a list of requirements, or explanatory text containing business terminology. Semantic vectorization converts keywords, semantic relationships, and contextual representations into vector representations of predefined dimensions, enabling subsequent models to identify semantic similarities between requirements. Defect reports record problems, reproducibility conditions, and fixes encountered during historical testing or deployment. Classification and labeling are based on defect severity, module, defect type, or triggering scenario, allowing defect information to be correlated with test cases and code features. Code structure information represents module hierarchy, file organization, function call relationships, and dependencies. Standardization converts structural descriptions from different sources and formats into predefined fields and a unified encoding format to eliminate differences in expression between projects. System source code provides implementation-level test-related information. Code feature extraction extracts information such as interface calls, class hierarchy, branch complexity, parameter characteristics, and change fragments from the source code. Historical test case sets are used to provide past verification assets. Their feature annotation process involves labeling test names, preconditions, execution steps, covered modules, execution results, and applicable scenarios to form test case feature data that can be used for modeling.

[0071] In practical implementation, semantic vectorization can employ pre-trained language models, word vector models, or sentence vector models to map each requirement description text into a fixed-length floating-point vector. Multiple vectors are then stored as part of the test dataset in the order of requirement entries. Classification labels can be jointly generated by a rule engine, manual verification results, and historical defect mapping relationships. Labels can include functional defects, performance defects, interface anomalies, or boundary condition defects. Standardization can be achieved through unified file parsing, field alignment, naming conventions, and hierarchical encoding, ensuring that code structure information can be consistently read by different model modules. Code feature extraction can be performed using static analysis tools, and the extraction results can be written into a feature table or graph structure representation. Feature annotation of historical test case sets can be generated by combining test case titles, step text, associated requirements, execution records, and defect backtracking results to form label vectors, which are then used as training input along with other test data.

[0072] This preprocessing step transforms heterogeneous test data into a unified, computable, alignable, and trainable data representation, providing a consistent input foundation for subsequent test case recommendation models. By jointly modeling requirement semantics, defect information, code structure, source code features, and historical test case features, the model's comprehensive perception of business scenarios, code changes, and historical testing experience can be improved, thereby enhancing the accuracy and coverage of test case matching and its adaptability to version evolution.

[0073] Based on the aforementioned embodiments, further, a test case feature library is obtained, including: determining historical test case features, code association features, and test scenario features based on the test dataset. The historical test case features include at least one of the following: coverage module data, input parameters, output parameters, execution time, defect detection rate, and test case type. The code association features include at least one of the following: code complexity information, module coupling degree, and historical defect density. The test scenario features include at least one of the following: test environment information, test priority, and defect repair rate. Based on the historical test case features, code association features, and test scenario features, a test case feature library is constructed.

[0074] The test dataset can be preprocessed from at least one of the following: system source code, code structure information, functional requirements documents, historical test case sets, and defect reports. Historical test case features characterize the coverage, performance, and defect detection capabilities of existing test cases. Code association features reflect the degree of association between the test object and code risks. Test scenario features describe the impact of the execution environment and test task context on test case adaptability. Coverage module data records the business module identifiers actually covered by the test cases. Input and output parameters correspond to the request data and return results during test case execution, respectively. Execution duration characterizes the resource consumption of test case execution. Defect detection rate characterizes the test case's ability to detect defects in historical executions. Test case types can include regression test cases, smoke test cases, or interface test cases. Code complexity information can be characterized by cyclomatic complexity, code nesting level, or number of branches. Module coupling can be determined by the strength of dependencies between modules. Historical defect density can be calculated from the number of defects per unit of code. Test environment information can include operating system, database version, network conditions, or container orchestration environment. Test priority can be determined based on the scope of business impact and release urgency. Defect repair rate can be calculated based on the results of historical defect closed-loop processing.

[0075] When constructing the test case feature library, historical test case features, code-related features, and test scenario features are uniformly encoded and stored together to form a feature set for recommendation models. Each feature in the feature library can be saved using structured fields, vector representations, or key-value mappings, and associated with its corresponding historical test case identifier for direct retrieval during subsequent similarity calculations, weight matching, or model input. For data from different sources, missing value completion, dimension normalization, and field standardization are performed before writing the data into the feature library according to a preset data structure, ensuring consistency and comparability of feature representations. In practical applications, the feature library can also be deployed in relational databases, in-memory data structures, or distributed storage systems; this embodiment does not limit this.

[0076] Based on the above settings, the test case feature library can simultaneously reflect the attributes of historical test cases themselves, code risk attributes, and test scenario context attributes. This allows the recommendation model to quickly match suitable test cases from a multi-dimensional feature space after obtaining the module to be tested and the test requirement document. Because the feature library performs unified aggregation and standardized management of test data, it reduces the bias caused by relying solely on single code or text information, improves the coverage and relevance of test case recommendations, and enhances adaptability to version evolution and changes in the test environment.

[0077] In one possible implementation, a test case feature library is constructed based on historical test case features, code association features, and test scenario features. This includes: determining association rules based on historical test case features, code association features, and test scenario features; establishing semantic relationships between historical test case features, code association features, and test scenario features based on the association rules to obtain a feature knowledge graph; and integrating the feature knowledge graph, historical test case features, code association features, and test scenario features to obtain the test case feature library.

[0078] Among them, historical test case features are used to characterize the coverage, input / output attributes, execution performance, and defect detection capabilities of historical test cases; code association features are used to characterize the complexity, coupling relationship, and historical defect distribution of the code under test; and test scenario features are used to characterize the test environment, execution priority, and repair feedback status. Association rules are used to define the logical correspondence between the three types of features to support subsequent semantic association modeling; and feature knowledge graphs are used to represent the association network between the three types of features in a graph structure, thereby enhancing the contextual understanding of test case applicability and scenario relevance.

[0079] In practical implementation, association rules can be mined based on historical test records, defect repair records, and code change records. Support, confidence, and lift are used to filter the co-occurrence relationships between feature items, resulting in a set of rules that reflects the relationship between "module change—scenario constraint—test case hit." Subsequently, based on the rule set, test case identifiers, code module identifiers, environment identifiers, defect identifiers, and scenario tags are constructed into nodes, and the corresponding mapping relationships in the rules are constructed into edges, forming a feature knowledge graph. Nodes in the graph can further carry feature weights, timestamps, and version numbers to preserve the feature evolution trajectory at different iteration stages. The graph is then fused with the original historical test case features, code association features, and test scenario features to form a test case feature library containing structured features, relational features, and contextual features for subsequent use by the recommendation model.

[0080] This feature library transforms simple discrete features into computable relational representations during operation, enabling test cases to not only retain their own attributes but also carry semantic context corresponding to code changes and test scenarios, thereby improving the matching ability between the module under test and test requirements. By integrating the feature knowledge graph with the original features, the feature library's ability to reuse historical experience is enhanced, and more stable relational support is formed for new test cases, changed modules, and different test environments.

[0081] By adopting this implementation method, the test case feature library can more comprehensively describe the relationship between historical test cases, code changes and test scenarios, reduce the omission problems caused by relying solely on single text similarity or single code mapping, improve the coverage, accuracy and interpretability of recommended test cases, and enhance the adaptability of the feature library to be continuously updated as the version evolves.

[0082] Based on the aforementioned embodiments, the method further includes: obtaining test results, which are obtained after testing the module to be tested using a list of recommended test cases; determining a feedback dataset based on the test results; and incrementally training the test case recommendation model that has been trained to convergence based on the feedback dataset.

[0083] The test results characterize the execution output after the recommended test case list is applied to the module under test. The execution output may include test pass information, test failure information, exception logs, assertion results, interface return values, and coverage statistics. The test results can be uniformly summarized by the automated testing framework after executing the test cases. The feedback dataset aggregates the test results and serves as input for incremental model training. It organizes the test results into structured samples, which may contain test case identifiers, execution environments, test conclusions, failure reasons, defect association information, and coverage hit information for subsequent model parameter updates.

[0084] In its implementation, after the recommended test case list has completed testing of the modules to be tested, the test execution engine writes the runtime logs, output results, and judgment conclusions of each test case into the result storage unit, and associates multiple test records corresponding to the same module to be tested to form a complete set of test results. Subsequently, the data processing unit extracts effective feedback information based on the test results, such as merging the error stacks, uncovered branches, duplicate defect markers, and manual review conclusions corresponding to failed test cases, and filtering out noisy data irrelevant to the recommendations, thereby forming a feedback dataset that can be directly used for model learning. This feedback dataset can be stored as a structured table, vector samples, or graph association samples. In practical applications, other data organization forms can also be selected, and this embodiment does not limit this.

[0085] After acquiring the feedback dataset, the incremental training unit reads the already converged test case recommendation model and inputs the feedback dataset as new training samples into the model. It then locally updates the model's feature weights, similarity metrics, or classification boundaries, allowing the model to absorb the latest test feedback while maintaining its original convergence capability. During incremental training, a mini-batch update method can be used to iteratively calculate new samples, and historical parameter constraints can be set to prevent the model from losing existing knowledge due to excessive shifts in new data. In this way, the model can gradually adapt to the version evolution of the module under test, changes in business rules, and changes in defect exposure patterns.

[0086] The working principle of this scheme is to feed the results of actual test execution back into the recommendation model training phase. This allows the model to move beyond relying solely on historical static samples and continuously absorb real feedback information from the current test task, thus forming a closed-loop update mechanism of recommendation, testing, feedback, and retraining. Since the feedback dataset comes directly from the execution results of the recommended test case list, the model can adjust subsequent recommendation preferences based on real and effective test performance, thereby improving the accuracy of hitting the current module under test and its adaptability to different scenarios.

[0087] By adopting this implementation method, the test case recommendation model can be continuously updated with version iterations, reducing recommendation bias caused by outdated historical data, improving the matching quality between the recommended test case list and the modules to be tested, and enhancing the ability to identify failed test cases, boundary scenarios, and high-risk scenarios. This reduces the cost of manual review, improves the efficiency of test resource utilization, and ensures the stability and evolvability of the test recommendation capability in long-term operation.

[0088] Based on the aforementioned embodiments, the test case recommendation model further includes a first feature network, a first attention mechanism, and a reinforcement learning reward function. The first feature network is used to identify long-distance dependencies between historical test case features, code association features, and test scenario features. The first attention mechanism is used to assign corresponding weights based on the defect detection rate of each historical test case. The reinforcement learning reward function is used to optimize the sorting order of the recommended test case list.

[0089] The first feature network can be composed of multiple sequence modeling units, which can employ recurrent neural networks, gated recurrent units, long short-term memory networks, or self-attention-based encoding structures to jointly encode historical test case features, code association features, and test scenario features. Historical test case features may include covered modules, input parameters, output parameters, execution time, and defect detection rate; code association features may include code complexity, module coupling, and historical defect density; and test scenario features may include the test environment, test priority, and defect repair rate. By vectorizing the input of various features according to time order, module order, or association order, the first feature network can uncover long-distance dependencies across test cases, code modules, and test scenarios, thereby forming a unified feature representation for the current recommendation task. In practical applications, this network can also employ different hidden layer dimensions and the number of attention heads; this embodiment does not limit this.

[0090] The first attention mechanism is used to weight historical test cases based on feature representations. Historical test cases with higher defect detection rates and stronger correlation with the current module under test will receive higher weights. This can be implemented by calculating query vectors, key vectors, and value vectors on the vector sequence output by the first feature network, generating attention scores based on defect detection rates, and then normalizing these scores to obtain weight coefficients. These weighted coefficients are then used to aggregate the representations of each historical test case, highlighting key test cases that can reveal potential defects. The attention mechanism can be jointly trained end-to-end with the feature network to adaptively adjust weight allocation as the test dataset changes. In practical applications, this attention module can employ a multi-head attention or single-head attention structure; this embodiment does not limit the specific approach.

[0091] The reinforcement learning reward function is used to optimize the sorting order of the recommended test case list. It constructs reward values ​​based on test execution feedback, defect detection results, and sorting position. Higher rewards are given to test cases ranked higher that detect defects earlier, while lower rewards are given when low-relevance test cases occupy high positions in the recommended list. During model training, the current sorting strategy can be used as an action, and the improvement in test results and defect detection rate can be used as environmental feedback. The sorting parameters are updated by maximizing the cumulative reward, ensuring that the recommended test case list satisfies both relevance and the goal of prioritizing defect exposure. This reward function can also be used in conjunction with supervised loss to balance feature matching accuracy and sorting optimization effects. In practical applications, other reinforcement learning algorithms can also be used, and this embodiment does not limit this approach.

[0092] Based on the above structure, the first feature network first completes deep correlation modeling of multi-source features, the first attention mechanism then highlights high-value historical test cases, and the reinforcement learning reward function further guides the optimization of the recommendation order. This ensures that the output list of recommended test cases not only covers key risk points but also improves defect detection efficiency in terms of execution order. This approach enhances the model's comprehensive ability to identify complex code changes, requirement shifts, and historical defect patterns, improves the accuracy, stability, and adaptability of test case recommendations to version evolution, and reduces the cost of manual test case selection.

[0093] Based on the aforementioned embodiments, further, the module to be tested is tested based on the recommended test case list to obtain test results, including: generating test tasks based on the recommended test case list; inputting the test tasks into a preset test agent to test the module to be tested and obtaining intermediate test data, which includes the runtime data generated during the testing process of each test case in the recommended test case list; and generating test results based on the intermediate test data and the test requirement document.

[0094] In this implementation, the test task carries the execution content transformed from the recommended test case list. It may include test case identifiers, execution order, input parameters, environment configuration, and assertion conditions, enabling the test agent to call the corresponding automated scripts or interfaces to complete the test according to the unified task description. The test agent receives the test task and drives the module under test. It may consist of a test orchestration unit, an execution unit, and a result acquisition unit. Intermediate test data includes interface return values, log information, page status, exception stacks, resource usage, and execution time of each test case. The test requirements document provides verification objectives, constraints, and acceptance criteria, facilitating the analysis of intermediate test data against the requirements to generate test results.

[0095] In practical implementation, generating test tasks involves parsing each test case in the recommended test case list into an executable object and arranging them according to test dependencies, priorities, and preconditions to form a task package that can be recognized by the test agent. After receiving the task, the test agent starts the module to be tested according to the runtime environment defined in the task package, injects test data according to the test case script, and collects runtime information during execution. The obtained intermediate test data retains the behavioral trajectory and state changes of each test case during execution. Subsequently, the intermediate test data is compared with the functional points, constraints, and acceptance items in the test requirements document to identify pass items, fail items, and exception items, and based on this, test results containing test conclusions, defect prompts, and coverage information are generated. In practical applications, the test agent can also be selected from other models or implementation forms; this embodiment does not limit this.

[0096] This implementation transforms the recommended test case list into test tasks and assigns them to a test agent for execution, directly linking the test execution process with the test case recommendation results. This reduces the costs of manual configuration and regression testing. Because intermediate test data retains runtime information during test case execution and is combined with test requirement documents to summarize results, it improves the accuracy of the correspondence between test results and requirement objectives, enhances the ability to identify abnormal behavior, boundary failures, and requirement deviations, thereby increasing the degree of test automation and the reliability of results.

[0097] In this embodiment, test data for model training is first obtained. The test data includes at least one of the following: system source code, code structure information, functional requirements document, historical test case set, and defect reports. Semantic vectorization processing is performed on each requirement description text in the functional requirements document, classification and annotation are performed on the defect reports, standardization processing is performed on the code structure information, code feature extraction processing is performed on the system source code, and feature annotation processing is performed on the historical test case set, resulting in a test dataset. Based on this, an initial test case recommendation model is obtained, and the initial model is continuously trained using the test dataset until convergence, forming a converged test case recommendation model.

[0098] After forming the test dataset, historical test case features, code association features, and test scenario features are determined from the dataset. Historical test case features include at least one of the following: covered module data, input parameters, output parameters, execution time, defect detection rate, and test case type. Code association features include at least one of the following: code complexity information, module coupling, and historical defect density. Test scenario features include at least one of the following: test environment information, test priority, and defect repair rate. Subsequently, association rules are determined based on these features, and semantic relationships are established between historical test case features, code association features, and test scenario features to obtain a feature knowledge graph. This feature knowledge graph is then integrated with the historical test case features, code association features, and test scenario features to construct a test case feature library. The test case recommendation model includes a first feature network, a first attention mechanism, and a reinforcement learning reward function. The first feature network is used to identify long-distance dependencies between historical test case features, code association features, and test scenario features. The first attention mechanism assigns corresponding weights based on the defect detection rate of each historical test case, and the reinforcement learning reward function optimizes the sorting order of the recommended test case list.

[0099] During actual testing, the module to be tested and its test requirement document are obtained, and a test case recommendation model that has been trained to convergence is invoked. At the same time, the module to be tested, the test requirement document, and the test case feature library are input into the model. The model combines the module features of the module to be tested, the semantic information of the test requirement document, and the historical test case features, code association features, and test scenario features in the test case feature library to perform matching analysis, and outputs multiple test cases that match the module to be tested and the test requirement document to form a recommended test case list. The test cases in the list are arranged in an optimized sorting order.

[0100] Test tasks are generated based on a recommended test case list and input into a pre-defined test agent. The test agent then executes tests on the module under test according to each test case in the recommended test case list. During the test, runtime data generated by each test case is collected to obtain intermediate test data, which includes the runtime data generated by each test case in the recommended test case list during the test. Finally, the intermediate test data and the test requirements document are combined to generate test results, which are then output.

[0101] In one possible implementation, after obtaining the test results, the result information formed by testing the module to be tested through the recommended test case list is extracted, the feedback dataset is determined, and incremental training is performed on the test case recommendation model that has been trained to convergence based on the feedback dataset, so that the model parameters are continuously updated with new test results.

[0102] Figure 2This is a schematic diagram of the structure of the testing device provided in the embodiments of this application, as shown below. Figure 2 As shown, the testing device 20 provided in this embodiment includes an acquisition module 201, a testing module 202, and an output module 203.

[0103] Module 201 is used to obtain the module to be tested and the test requirements document;

[0104] The acquisition module 201 is also used to acquire the test case recommendation model that has been trained to convergence;

[0105] The acquisition module 201 is also used to acquire the test case feature library, which includes feature data of multiple historical test cases;

[0106] Test module 202 is used to input the module to be tested, the test requirement document, and the test case feature library into the test case recommendation model that has been trained to convergence, and obtain a recommended test case list. The recommended test case list includes multiple test cases that match the module to be tested and the test requirement document.

[0107] Test module 202 is also used to test the module to be tested based on the recommended test case list and obtain test results;

[0108] Output module 203 is used to output test results.

[0109] In one possible embodiment, the acquisition module 201 is specifically used for:

[0110] Acquire test data, which includes at least one of the following: system source code, code structure information, functional requirements document, historical test case set, and defect report;

[0111] The test data is preprocessed to obtain the test dataset;

[0112] Obtain the initial test case recommendation model;

[0113] Based on the test dataset, the initial test case recommendation model is trained until convergence, resulting in a converged test case recommendation model.

[0114] In one possible embodiment, the test module 202 is further configured to:

[0115] Perform semantic vectorization on the text describing each requirement in the functional requirements document;

[0116] Categorize and label defect reports;

[0117] Standardize the code structure information;

[0118] Perform code feature extraction processing on the system source code;

[0119] Feature annotation processing is performed on the historical test case set.

[0120] In one possible embodiment, the acquisition module 201 is specifically used for:

[0121] Based on the test dataset, determine the characteristics of historical test cases, code association characteristics, and test scenario characteristics. The characteristics of historical test cases include at least one of the following: coverage module data, input parameters, output parameters, execution time, defect detection rate, and test case type. The characteristics of code association include at least one of the following: code complexity information, module coupling degree, and historical defect density. The characteristics of test scenario include at least one of the following: test environment information, test priority, and defect repair rate.

[0122] A test case feature library is built based on historical test case features, code association features, and test scenario features.

[0123] In one possible embodiment, the acquisition module 201 is specifically used for:

[0124] Based on historical test case characteristics, code association characteristics, and test scenario characteristics, association rules are determined.

[0125] Based on association rules, semantic association relationships are established between historical test case features, code association features, and test scenario features to obtain a feature knowledge graph;

[0126] By integrating feature knowledge graphs, historical test case features, code association features, and test scenario features, a test case feature library is obtained.

[0127] In one possible embodiment, the test module 202 is further configured to:

[0128] Obtain the test results, which are obtained after testing the module to be tested using the recommended test case list.

[0129] Based on the test results, determine the feedback dataset;

[0130] Based on the feedback dataset, the test case recommendation model that has been trained to convergence is incrementally trained.

[0131] In one possible embodiment, the test case recommendation model includes a first feature network, a first attention mechanism, and a reinforcement learning reward function. The first feature network is used to identify long-distance dependencies between historical test case features, code association features, and test scenario features. The first attention mechanism is used to assign corresponding weights based on the defect detection rate of each historical test case. The reinforcement learning reward function is used to optimize the sorting order of the recommended test case list.

[0132] In one possible embodiment, the test module 202 is specifically used for:

[0133] Generate test tasks based on the recommended test case list;

[0134] The test task is input into the preset test agent to test the module to be tested and obtain intermediate test data. The intermediate test data includes the running data generated during the testing process of each test case in the recommended test case list.

[0135] Test results are generated based on intermediate test data and test requirements documents.

[0136] The testing device provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0137] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 3 As shown, the electronic device 30 provided in this embodiment includes at least one processor 301 and a memory 302. Optionally, the device 30 further includes a communication component 303. The processor 301, memory 302, and communication component 303 are connected via a bus.

[0138] In a specific implementation, at least one processor 301 executes computer execution instructions stored in memory 302, causing at least one processor 301 to perform the above-described method.

[0139] The specific implementation process of processor 301 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0140] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0141] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0142] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0143] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0144] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0145] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0146] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0147] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0148] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0149] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0150] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0151] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0152] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A testing method, characterized in that, include: Obtain the module to be tested and its test requirements document; Obtain the test case recommendation model that has been trained to convergence; Obtain a test case feature library, which includes feature data of multiple historical test cases; The module to be tested, the test requirement document, and the test case feature library are input into the test case recommendation model that has been trained to convergence to obtain a recommended test case list, which includes multiple test cases that match the module to be tested and the test requirement document. Based on the recommended test case list, the module to be tested is tested, and the test results are obtained. Output the test results.

2. The method according to claim 1, characterized in that, The process of obtaining the test case recommendation model that has been trained to convergence includes: Acquire test data, which includes at least one of the following: system source code, code structure information, functional requirements document, historical test case set, and defect report; The test data is preprocessed to obtain a test dataset; Obtain the initial test case recommendation model; Based on the test dataset, the initial test case recommendation model is trained until convergence, resulting in a converged test case recommendation model.

3. The method according to claim 2, characterized in that, The preprocessing of the test data includes at least one of the following: Semantic vectorization is performed on each requirement description text in the functional requirements document; The defect reports are categorized and labeled. The code structure information is standardized. The system source code is subjected to code feature extraction processing; The historical test case set is subjected to feature annotation processing.

4. The method according to claim 2, characterized in that, The acquisition of the test case feature library includes: Based on the test dataset, historical test case characteristics, code association characteristics, and test scenario characteristics are determined. The historical test case characteristics include at least one of the following: coverage module data, input parameters, output parameters, execution time, defect detection rate, and test case type. The code association characteristics include at least one of the following: code complexity information, module coupling degree, and historical defect density. The test scenario characteristics include at least one of the following: test environment information, test priority, and defect repair rate. A test case feature library is constructed based on the historical test case features, the code association features, and the test scenario features.

5. The method according to claim 4, characterized in that, The construction of a test case feature library based on the historical test case features, the code association features, and the test scenario features includes: Based on the characteristics of the historical test cases, the characteristics of the code association, and the characteristics of the test scenarios, the association rules are determined. Based on the association rules, semantic association relationships are established among the historical test case features, the code association features, and the test scenario features to obtain a feature knowledge graph. By integrating the feature knowledge graph, the historical test case features, the code association features, and the test scenario features, a test case feature library is obtained.

6. The method according to claim 1, characterized in that, Also includes: Obtain test results, which are obtained after testing the module to be tested using the recommended test case list; Based on the test results, determine the feedback dataset; Based on the feedback dataset, the test case recommendation model that has been trained to convergence is incrementally trained.

7. The method according to claim 4, characterized in that, The test case recommendation model includes a first feature network, a first attention mechanism, and a reinforcement learning reward function. The first feature network is used to identify long-distance dependencies between the features of the historical test cases, the code association features, and the test scenario features. The first attention mechanism is used to assign corresponding weights based on the defect detection rate of each historical test case. The reinforcement learning reward function is used to optimize the sorting order of the recommended test case list.

8. The method according to claim 1, characterized in that, The step of testing the module to be tested based on the recommended test case list and obtaining test results includes: Based on the recommended test case list, a test task is generated; The test task is input into a preset test agent to test the module to be tested and obtain intermediate test data. The intermediate test data includes the running data generated during the testing process of each test case in the recommended test case list. Test results are generated based on the intermediate test data and the test requirements document.

9. A testing device, characterized in that, include: The acquisition module is used to acquire the module to be tested and its test requirement documents. The acquisition module is also used to acquire a test case recommendation model that has been trained to convergence; The acquisition module is also used to acquire a test case feature library, which includes feature data of multiple historical test cases; The testing module is used to input the module to be tested, the test requirement document, and the test case feature library into the test case recommendation model that has been trained to convergence, and obtain a recommended test case list, which includes multiple test cases that match the module to be tested and the test requirement document. The testing module is also used to test the module to be tested based on the recommended test case list and obtain test results; The output module is used to output the test results.

10. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 8.

12. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 1 to 8.

Images