Self-adaptive pointer network test case priority ranking method based on meta reinforcement learning

Abstract

The invention relates to a test case priority ranking method based on meta reinforcement learning and an adaptive pointer network. The method comprises the following steps: constructing a self-adaptive pointer network fused with a meta-learning mechanism as a strategy network; the strategy network completes serialization processing of a test case set by means of a built-in encoder-decoder and a pointer attention mechanism, and generates a test case priority ranking sequence. In the process, hyper-parameters in the network are dynamically adjusted by a meta-learning mechanism according to performance feedback, and manual intervention is not needed. Meanwhile, according to the method, a multi-target reward function is constructed according to a test history execution result and execution time, and a dominant action evaluation algorithm is adopted to drive a pointer network to carry out strategy optimization. By performing meta-training on a large number of continuous integration tasks, the system obtains the ability to quickly adapt to new tasks. The method aims to solve the problems of poor dynamic adaptability, weak cross-project generalization ability and high hyper-parameter tuning cost of a sorting strategy, and finally achieves the purposes of quickly positioning defects, improving test efficiency and guaranteeing software quality.

Description

Technical Field

[0001] This invention relates to the field of software testing and program analysis technology, and is particularly applicable to test case optimization techniques for regression testing in continuous integration environments. Its purpose is to automatically and quickly generate an optimal test case execution sequence for the program under test, given a set of test cases, through an adaptive priority ranking method, thereby accelerating the software defect detection process and improving the efficiency of test resource utilization. This invention is an intelligent testing solution that can significantly reduce regression testing time and costs, and help ensure software quality. Background Technology

[0002] Software testing is a crucial step in ensuring software quality and reliability. Throughout the software development lifecycle, developers continuously modify code to adapt to changing requirements and fix defects. To ensure these modifications don't introduce new errors, testers need to retest the modified software—a process known as regression testing. However, as software iterates, its scale and complexity increase, leading to increasingly large test case sets. Executing all test cases within limited time and computing resources is often prohibitively expensive. Therefore, efficiently utilizing existing test case sets and prioritizing tests that quickly uncover errors is vital for reducing regression testing costs and improving testing efficiency.

[0003] In the field of regression test optimization, the main techniques include Test Suite Minisation (TSM), Test Case Selection (TCS), and Test Case Prioritization (TCP). TSM and TCS reduce the test suite size by eliminating redundant test cases, but both carry the risk of missing potential defects due to discarding certain test cases. TCP, on the other hand, without reducing the number of test cases, rearranges the execution order of test cases, prioritizing the execution of high-risk test cases, thereby accelerating the fault detection process and improving the efficiency of regression testing.

[0004] The core of TCP technology lies in its sorting criteria, which evaluate and rank test cases based on an objective function. Existing sorting criteria can be broadly categorized into single-objective and multi-objective approaches. Early research focused primarily on single-objective optimization. For example, Wang Ziyuan et al.'s invention patent, "A Test Case Priority Sorting Method Oriented to Parameter Value Switching," proposed a test case sorting method with minimizing test execution time cost as its single optimization objective. Its aim was to accurately quantify and model the parameter setting overhead, cancellation overhead, and execution cost during the testing process, and then use a greedy algorithm to reorder test cases, thereby effectively reducing the total time cost of regression testing. However, this single-objective optimization process cannot consider other dimensions of testing needs. The sorting result might prioritize saving a few seconds of switching time, placing high-risk test cases later, thus delaying the discovery of serious defects. Therefore, researchers have begun to shift towards multi-objective TCP technology, aiming to achieve a balance between multiple key indicators such as error detection rate and test execution time. For example, Xia Chunyan et al.'s invention patent, "A Test Case Priority Ranking Method Based on Multi-Objective Optimization," includes a Pareto-optimal non-dominated sorting genetic algorithm that simultaneously uses average error detection rate, average statement coverage, and effective execution time as optimization objectives, thereby obtaining test sequences that perform well across multiple objectives. However, its dynamic adaptability is poor, making it difficult to adapt to the dynamic changes in test requirements caused by frequent code changes in continuous integration environments. Secondly, it lacks cross-project generalization ability; a sorting model or strategy that performs well in one project often experiences a significant performance drop when directly migrated to another heterogeneous project, lacking universal applicability. This forces testers to redesign or adjust the sorting method for each new project, resulting in high costs.

[0005] To address this issue, this invention proposes an adaptive pointer network test case priority ranking method based on meta-reinforcement learning. The basic idea of ​​this invention is to construct an adaptive pointer network incorporating a meta-learning mechanism as the agent's policy network, construct a multi-objective reward function using historical test execution results and execution times, and drive the network to perform end-to-end policy optimization using a dominant action evaluation algorithm. Through meta-training on a large number of continuous integration tasks, the model acquires the inherent ability to automatically adjust its internal parameters to adapt to new tasks. When applied to new continuous integration projects, this method can directly sequence test cases into the optimal ranking due to its rapid adaptive capability. This method effectively solves the problems of poor dynamic adaptability, weak cross-project generalization ability, and high cost of manual hyperparameter tuning in existing test case ranking strategies. Compared with traditional methods, this invention is more conducive to quickly and adaptively locating software faults in complex continuous integration environments, significantly reducing the execution time and resource consumption of regression testing, and realizing the intelligence and automation of the testing process. Summary of the Invention

[0006] This invention provides a test case priority ranking method based on meta-reinforcement learning and adaptive pointer networks. This method effectively addresses the shortcomings of current continuous integration environments, such as poor dynamic adaptability of test case priority ranking, weak cross-project generalization ability, and high cost of hyperparameter tuning. As a result, it improves the defect detection efficiency of regression testing, reduces test resource consumption, and ultimately achieves adaptive optimization of the testing process and continuous improvement of testing efficiency.

[0007] To achieve the above objectives, this invention employs a meta-reinforcement learning framework. Through training on a large number of continuous integration tasks, the system acquires the adaptive capability to dynamically adjust hyperparameters for continuous integration scenarios. Subsequently, within each integration cycle, a priority ranking sequence of test cases is generated based on a pointer attention mechanism, and the ranking strategy is continuously optimized based on test execution feedback, constructing a complete adaptive optimization closed loop of "hyperparameter adjustment - test ranking generation - strategy optimization - hyperparameter update". Given a continuous integration environment and a test case set, the system first dynamically adjusts the training hyperparameters of the deep reinforcement learning agent through a meta-hyperparameter adjustment module. Then, the agent uses a policy network based on a pointer attention mechanism to serialize the test case set into a priority ranking. After the ranking result is tested, the feedback generated is used to update the ranking strategy through a dominant action evaluation algorithm, and also serves as the performance input for the meta-hyperparameter adjustment module for the next round of hyperparameter adjustment. After meta-training on a large number of tasks and acquiring rapid adaptive capabilities, the system can be directly applied to new continuous integration projects, efficiently generating optimal test sequences.

[0008] The implementation of the method includes the following steps.

[0009] Meta-hyperparameter tuning module. This module aims to dynamically adjust and optimize the training hyperparameters of a deep reinforcement learning agent through an outer loop meta-training process. First, it initiates meta-training by initializing a set of meta-hyperparameters, including the base learning rate. Basic discount factor Number of basic training cycles Adjustment factor .

[0010] Second, the inner loop deep reinforcement learning training uses the current hyperparameter configuration in an inner loop manner. Train the deep reinforcement learning agent.

[0011] Third, performance evaluation: After the inner loop training is completed, the agent's performance is evaluated on the validation set. Performance metrics The ranking is comprehensively evaluated using both the Average Percentage Defect Detection Rate (APFD) and the Normalized Rank Percentile Average (NRPA). The calculation methods for APFD and NRPA are as follows:

[0012] a. The APFD metric is calculated by plotting the sum of the execution positions of all failed test cases in a test case sequence, and relating it to the total length of the sequence and the total number of defects. This metric represents the defect detection rate of the test case sequence. The calculation formula is as follows:

[0013]

[0014] in, This represents the sorted sequence of test case executions. For a specific test case in the sequence. Test cases The execution position in the sequence, Test cases The execution decision (1 for failure, 0 for success). It is the total number of test cases in the sequence. It is the total number of software defects revealed by the test case set in this cycle.

[0015] b. The NRPA metric calculates the ranking difference of each test case in the generated sequence and the optimal reference sequence, and uses normalization to represent how close the ranking result is to the optimal ranking. The calculation formula is as follows:

[0016]

[0017] in, This represents the sorted sequence generated by the model. This represents the optimal reference sequence. This indicates the total number of test cases. Represents a sequence Each test case in the database, Represents test cases In sequence Index in Represents test cases In the optimal sequence The index in.

[0018] Fourth, meta-feedback and dynamic hyperparameter adjustment: the meta-controller compares current performance. Compared with historical performance Calculate the change in performance ,according to With preset threshold The relationship between the training epochs and the hyperparameters is automatically adjusted. Among the hyperparameters are the number of training epochs and the number of training cycles. The learning rate is used to control the number of iterations in the intelligent training loop, directly affecting the model's convergence and training cost. The step size used to control the update of policy network parameters is related to the convergence speed and stability of model training; discount factor. This is used to weigh the importance of current rewards against long-term future returns, influencing the agent's decision-making perspective. The calculation formula is as follows:

[0019] a. Number of training cycles The adjustment rules are given by the following formula:

[0020]

[0021] in, This is the adjusted number of training cycles. and These are the minimum and maximum allowed number of training cycles. It is a meta-adjustment factor. It is the threshold for performance change.

[0022] b. Learning rate The adjustment rules are given by the following formula:

[0023]

[0024] in, This is the adjusted learning rate. This is the current learning rate.

[0025] c. Discount factor The adjustment rules are given by the following formula:

[0026]

[0027] in, This is the adjusted discount factor. and These are the minimum and maximum allowed values ​​for the discount factor.

[0028] Fifth, meta-training convergence judgment: Repeat the inner loop training and hyperparameter tuning; if the convergence occurs continuously... Performance variation for each task All less than the threshold And hyperparameter configuration If no adjustments are made during this period, the meta-training is considered to have converged. At this point, the meta-training phase ends, and the system gains the ability to quickly adapt to new tasks.

[0029] 2) Pointer Attention Mechanism and Advantageous Action Evaluation Algorithm. This module constitutes the core of the agent that performs specific sorting operations and policy learning within each continuous integration cycle. It is responsible for mapping the test case feature sequence to the optimal execution order and using test feedback for self-optimization. First, the encoder generates a hidden state sequence, which maps the test case feature sequence to the optimal execution order. Input a bidirectional long short-term memory network encoder, which processes the sequence in both forward and backward directions, and outputs a hidden state that incorporates the global context for each test case. Finally, the hidden state sequence is obtained. .

[0030] Second, decoder initialization and pointer attention calculation: the decoder is a long short-term memory network, and its initial state... Set as the hidden state at the final moment of the encoder At each step of decoding The decoder uses a pointer attention mechanism to calculate the probability of selecting all unselected test cases. This includes the attention score and the test case... In the The probability of a step being selected is calculated as follows:

[0031] a. Attention score The calculation formula is as follows:

[0032]

[0033] in, It is a trainable parameter vector. and It is a trainable weight matrix. It is the first The encoder hidden state of each test case. Is the decoder in the 1st The hidden state of the step.

[0034] b. Test cases In the The probability of a step being selected is given by the following formula:

[0035]

[0036] in, It was before The generated partial sequence has been processed. This represents the overall state of the current test case set; the summation in the denominator applies to all test cases that have not yet been selected in the current step. .

[0037] Third, sequence generation: at each step Select the test case with the highest probability. As the next element in the sequence, the test case is removed from the candidate set, and its feature vector is input into the decoder to update its state. Repeat this process until all test cases are sorted, and output the complete sorted sequence. .

[0038] Fourth, multi-objective reward calculation and advantage function estimation: execution of ranking sequence Then, a reward signal is calculated based on the test results. The reward function combines the normalized average percentage defect detection rate (NAPFD) with efficiency considerations based on execution time; and then calculates the advantage function. Among them, NAPFD and the dominance function The calculation method is as follows:

[0039] a. The NAPFD metric uses the relationship between the execution position of failed test cases in a test sequence and the total length of the sequence to represent the defect detection efficiency of the test case sequence. The calculation formula is as follows:

[0040]

[0041] in, It is the percentage of defects detected within the testing budget. , It is a set of test examples that reveal defects. Test cases Position in the sequence, This is the original size of the test case set.

[0042] b. Dominant Function The method used is to calculate the actual reward received. Value predicted by commentator network The difference is used to represent the quality of the action performed under the current strategy relative to the average level. The calculation formula is as follows:

[0043]

[0044] in, It is the actual total reward obtained after executing the sorting sequence. It is a commentator network based on state The predicted expected cumulative reward.

[0045] Fifth, the strategy and value update of the dominant action evaluation algorithm: utilizing the dominant function. The parameters of the executor network (pointer network) are updated using the policy gradient ascent method. The parameters of the commenter network are updated by minimizing the mean squared error of the value function. The policy update is based on the formula for calculating the policy gradient, and the value update is based on the loss function of the commenter network. The specific methods are as follows:

[0046] a. The policy gradient is calculated by multiplying the probability gradient of the policy network's output action with the advantage function to represent the direction and magnitude of the parameter update. The calculation formula is as follows:

[0047]

[0048] in, It is the policy objective function with respect to the parameters gradient estimation, This indicates that for all in the sequence Gradient average of each decision step, It is the gradient of the log probability of the selected action.

[0049] b. The commenter network loss function uses the calculation of actual reward. With network predictive value The mean squared error is used to represent the accuracy of value prediction, and its calculation formula is as follows:

[0050]

[0051] in, These are coefficients used to facilitate gradient calculation. This represents the squared error.

[0052] Furthermore, the specific steps of step 1) above are as follows:

[0053] Step 1)-1: Initial state;

[0054] Steps 1)-2: Begin the meta-training process;

[0055] Steps 1)-3: Initialize meta-hyperparameters, including the base learning rate. Basic discount factor Number of basic training cycles Adjustment factor ;

[0056] Steps 1)-4: Enter the inner loop: Use the current hyperparameter configuration Training deep reinforcement learning intelligence

[0057] Steps 1)-5: After the inner loop training is completed, evaluate the agent's performance on the validation set, using performance metrics. It is composed of the average percentage defect detection rate (APFD) and the normalized rank percentile mean (NRPA);

[0058] Steps 1)-6: Determine if performance has improved: The meta controller compares the current performance. Compared with historical performance Calculate the change in performance ;

[0059] Steps 1)-7: If performance is improved ( If the hyperparameters are maintained or fine-tuned, then the hyperparameters are maintained or fine-tuned.

[0060] Steps 1)-8: If performance is not improved ( If so, the hyperparameters will be adjusted according to the preset rules;

[0061] Step 1)-9: Determine if the meta-training convergence condition has been met; if not, return to Step 1)-4 to continue the inner loop training and hyperparameter tuning.

[0062] Step 1)-10: If the convergence condition is met, output the optimized set of hyperparameters;

[0063] Step 1)-11: End state.

[0064] Furthermore, the specific steps of step 2) above are as follows:

[0065] Step 2)-1: Initial state;

[0066] Step 2)-2: Enter the current test case set and its characteristic sequences The features include historical execution results and historical execution time;

[0067] Steps 2)-3: Encoder Processing: Input the test case feature sequence into the bidirectional long short-term memory network encoder to generate the hidden state sequence. ;

[0068] Step 2)-4: Decoder Initialization: Set the decoder to its initial state. Set as the hidden state at the final moment of the encoder ;

[0069] Steps 2)-5: Pointer attention calculation: at each step of decoding For each test case that has not yet been selected Calculate their attention score and the probability of being selected ;

[0070] Steps 2)-6: Select the test case with the highest probability. As the next execution use case in the sequence;

[0071] Steps 2)-7: Select the test cases Remove it from the candidate set and input its feature vector into the decoder to update the decoder state. ;

[0072] Step 2)-8: Determine if all test cases have been sorted; if not, return to Step 2)-5 to continue generating the sequence;

[0073] Step 2)-9: If yes, output the complete test case sorting sequence. ;

[0074] Step 2)-10: Execute the sorted test case sequence and collect test feedback;

[0075] Step 2)-11: Multi-objective reward calculation: Calculate the reward signal based on the test results. The reward function combines the normalized average percentage defect detection rate (NAPFD) with considerations based on execution time.

[0076] Step 2)-12: Advantage function estimation: The commentator network estimates the advantage function based on the initial state. Predicted State Value Advantage function Calculated as ;

[0077] Steps 2)-13: Strategy and Value Update of the Dominant Action Evaluation Algorithm: Utilizing the Dominant Function The parameters of the executor network (pointer network) are updated through policy gradient. The parameters of the commentator network are updated by minimizing the mean squared error of the value function. ;

[0078] Step 2)-14: End this round of training. Attached Figure Description

[0079] Figure 1 This is an overall flowchart of the test case priority ranking method based on meta-reinforcement learning and adaptive pointer network implemented in this invention.

[0080] Figure 2 for Figure 1 Flowchart of the Zhongyuan hyperparameter adjustment module.

[0081] Figure 3 for Figure 1 Flowchart of the action evaluation algorithm that integrates the middle pointer attention mechanism. Detailed Implementation

[0082] To better understand the technical content of this invention, specific embodiments are described below in conjunction with the accompanying drawings.

[0083] Figure 1 This is an overall flowchart of the test case priority ranking method based on meta-reinforcement learning and adaptive pointer network implemented in this invention.

[0084] A test case priority ranking method based on meta-reinforcement learning and adaptive pointer networks is characterized by the following steps.

[0085] The S1 meta-hyperparameter tuning module, given a test case set TS, dynamically optimizes the training hyperparameters of the deep reinforcement learning agent through a meta-training process, including the automatic adjustment of the learning rate, discount factor, number of training epochs, and meta-tuning parameters, thereby generating an optimized hyperparameter configuration.

[0086] The S2 pointer attention mechanism integrates the dominant action evaluation algorithm. Given the hyperparameter configuration and test case feature sequence generated by S1, it guides optimization through a reward function that integrates factors such as NAPFD and test execution time. Based on the pointer attention mechanism, it generates a test case ranking sequence, updates the policy network parameters through the dominant action evaluation algorithm, and finally outputs the ranked test case sequence.

[0087] Figure 2 This is a flowchart of the meta-hyperparameter tuning module. After starting meta-training, the hyperparameters are first initialized, including the base learning rate. Basic discount factor Number of basic training cycles Adjustment factor Then, the inner loop of deep reinforcement learning training begins, using the current hyperparameter configuration to train the agent. After training, performance is evaluated, and the APFD / NRPA metric is used to determine if performance has improved, thus deciding whether to maintain or fine-tune the hyperparameters, or to adjust them further. This process is repeated until convergence is achieved, and the optimized hyperparameters are output. The specific steps are as follows:

[0088] Step 1: Initial state; Step 2: Start the meta-training process; Step 3: Initialize meta-hyperparameters, including the base learning rate. Basic discount factor Number of basic training cycles Adjustment factor Step 4: Enter the inner loop and use the current hyperparameter configuration. Train the deep reinforcement learning agent; Step 5: After the inner loop training is completed, evaluate the agent's performance on the validation set. The performance metric P is composed of the average percentage defect detection rate (APFD) and the normalized rank percentile mean (NRPA); Step 6: Determine whether the performance has improved. Compared with historical performance Calculation performance change Step 7: If performance improves ( If the performance does not improve ( ), then maintain or fine-tune the hyperparameters; Step 8: If the performance does not improve ( If the hyperparameters are adjusted according to the preset rules, then step 9: if in continuous Performance variation for each task All less than the threshold And hyperparameter configuration If no adjustment is made during this period, the meta-training is considered to have converged. If it has not converged, return to step 4 to continue the inner loop training and hyperparameter adjustment. Step 10: If the convergence condition is met, output the optimized hyperparameter set. Step 11: End state.

[0089] Figure 3 This is a flowchart illustrating the integration of the pointer attention mechanism with the dominant action evaluation algorithm. After inputting the test case feature sequence, the encoder first generates a hidden state sequence. Then, the decoder is initialized, and the pointer attention mechanism is used to calculate the selection probability of each test case. The test case with the highest probability is selected as the next execution test case, and this process is repeated until all test cases are sorted. After outputting the complete sorted sequence, the dominance function is calculated and updated using the dominant action evaluation algorithm. The specific steps are as follows:

[0090] Step 1: Initial state; Step 2: Input the current test case set TS and its feature sequence. The features include historical execution results and historical execution time; Step 3: Encoder processing, inputting the test case feature sequence into the bidirectional long short-term memory network encoder to generate the hidden state sequence. Step 4: Decoder initialization, setting the decoder to its initial state. Set as the hidden state at the final moment of the encoder Step 5: Pointer attention mechanism calculation, in each step of decoding. For each test case that has not yet been selected Calculate their attention score and the probability of being selected Step 6: Select the test case with the highest probability. As the next test case to be executed in the sequence; Step 7: Select the test case Remove it from the candidate set and input its feature vector into the decoder to update the decoder state. Step 8: Determine if all test cases have been sorted. If not, return to Step 5 to continue generating the sequence. Step 9: If yes, output the complete test case sorting sequence. Step 10: Execute the sorted test case sequence and collect test feedback; Step 11: Calculate multi-objective reward and reward signals based on test results. The reward function combines the normalized average percentage defect detection rate (NAPFD) with execution time considerations; Step 12: Advantage function estimation, the commentator network estimates the advantage function based on the initial state. Predicted State Value Advantage function Calculated as Step 13: Strategy and value update of the dominant action evaluation algorithm, utilizing the dominance function. The parameters of the executor network (pointer network) are updated through policy gradient. The parameters of the commentator network are updated by minimizing the mean squared error of the value function. Step 14: End this round of training.

[0091] In summary, the purpose of this invention is to solve the problems of poor dynamic adaptability, weak cross-project generalization ability, and high cost of hyperparameter tuning in test case priority ranking in continuous integration environments. By deeply integrating meta-reinforcement learning and adaptive pointer networks, adaptive optimization of the testing process and continuous improvement of testing efficiency can be achieved.

Claims

1. A test case priority ranking method based on meta-reinforcement learning and adaptive pointer networks, characterized in that, For test case sets in a continuous integration environment, a meta-reinforcement learning framework is used to train the system on a large number of continuous integration tasks, enabling the system to dynamically adjust hyperparameters. In each integration cycle, a pointer attention mechanism is used to generate a test case ranking, and the ranking strategy is continuously optimized based on test execution feedback, thus constructing a complete adaptive optimization closed loop of "hyperparameter tuning - test ranking generation - strategy optimization - hyperparameter update". Given a continuous integration environment and test case set, the system first dynamically adjusts the training hyperparameters of the deep reinforcement learning agent through a meta-hyperparameter tuning module. This design, by integrating a meta-learning mechanism and a pointer network to construct an adaptive policy network, overcomes the limitations of traditional test case ranking models with fixed structures and reliance on manual parameter tuning, achieving hyperparameter optimization. The system dynamically adapts and adjusts; subsequently, the agent uses a policy network based on pointer attention to serialize the test case set into a priority ranking; after the ranking result is tested, the feedback generated is used to update the ranking strategy through the dominant action evaluation algorithm, and also serves as the performance input of the meta-hyperparameter adjustment module to drive the next round of hyperparameter adjustment; this method precisely focuses on the core needs of rapid adaptation to different projects and improved testing efficiency in continuous integration scenarios, providing solutions for key aspects of test case ranking, aiming to effectively solve the industry pain points of difficult multi-project migration and adaptation and high cost of manual intervention; after meta-training on a large number of tasks and obtaining rapid adaptive capabilities, the system can be directly applied to new continuous integration projects and efficiently generate optimal test sequences; Its rapid transfer capability, obtained through meta-training, eliminates the need for extensive secondary training in new projects, balancing efficiency and reliability, and demonstrating promising prospects for practical application. The method includes the following steps: 1) The meta-hyperparameter tuning module dynamically adjusts and optimizes the training hyperparameters of the deep reinforcement learning agent through an outer loop meta-training process. First, it starts meta-training and initializes a set of meta-hyperparameters, including the base learning rate. Basic discount factor Number of basic training cycles Adjustment factor Second, deep reinforcement learning training within an inner loop, using the current hyperparameter configuration in an inner loop approach. The deep reinforcement learning agent is trained; third, performance evaluation: after the inner loop training is completed, the agent's performance is evaluated on the validation set. The performance metric P is composed of the average percentage defect detection rate (APFD) and the normalized rank percentile mean (NRPA) to comprehensively evaluate the ranking effect; among them, the APFD metric is calculated by calculating the sum of the execution positions of all failed test cases in the test case sequence, and relating it to the total length of the sequence and the total number of defects, to represent the defect detection rate of the test case sequence. The calculation formula is as follows: ,in, This represents the sorted sequence of test case executions. For a specific test case in the sequence. Test cases The execution position in the sequence, Test cases The execution of the ruling is categorized as follows: failure is 1, success is 0. It is the total number of test cases in the sequence. This represents the total number of software defects revealed by the test case set in this cycle. The NRPA metric is calculated by comparing the ranking difference of each test case in the generated sequence with the optimal reference sequence, and normalization is used to represent how close the ranking result is to the optimal ranking. The calculation formula is as follows: in, This represents the sorted sequence generated by the model. This represents the optimal reference sequence. This indicates the total number of test cases. Represents a sequence Each test case in the database, Represents test cases In sequence Index in Represents test cases In the optimal sequence Indexes in the code; meta-feedback and dynamic hyperparameter tuning: meta-controller compares current performance. Compared with historical performance Calculate the change in performance ,according to With preset threshold The relationship between the training epochs and the hyperparameters is automatically adjusted. Among the hyperparameters are the number of training epochs and the number of training cycles. The learning rate is used to control the number of iterations in the intelligent training loop, directly affecting the model's convergence and training cost. The step size used to control the update of policy network parameters is related to the convergence speed and stability of model training; discount factor. This is used to weigh the importance of current rewards against long-term future returns, influencing the agent's decision-making perspective; the calculation formula is as follows: Number of training cycles The adjustment rules are given by the following formula: ； Learning rate The adjustment rules are given by the following formula: ； Discount factor The adjustment rules are given by the following formula: ； Fifth, meta-training convergence judgment: Repeat the inner loop training and hyperparameter tuning until the meta-hyperparameters and agent performance tend to stabilize on multiple tasks. At this time, the meta-training phase ends and the system gains the ability to quickly adapt to new tasks. 2) The pointer attention mechanism, combined with the superior action evaluation algorithm, forms the core of the agent that performs specific sorting operations and policy learning in each continuous integration cycle. It is responsible for mapping the test case feature sequence to the optimal execution order and using test feedback for self-optimization. First, the encoder generates a hidden state sequence, which maps the test case feature sequence... Input a bidirectional long short-term memory network encoder. The encoder processes the sequence from front to back and outputs a hidden state that incorporates the global context for each test case. Finally, the hidden state sequence is obtained. Second, decoder initialization and pointer attention calculation: the decoder is a long short-term memory network, and its initial state... Set as the hidden state at the final moment of the encoder In each step of decoding The decoder uses a pointer attention mechanism to calculate the probability of selecting all unselected test cases; where the attention score and the test case... In the The probability of a step being selected is calculated as follows: Attention Score The calculation formula is: ,in, It is a trainable parameter vector. and It is a trainable weight matrix. It is the first The encoder hidden state of each test case. Is the decoder in the 1st Hidden state of the step; test cases In the The formula for the probability of a step being selected is: ,in, It was before The generated partial sequence has been processed. This represents the overall state of the current test case set; the summation in the denominator applies to all test cases that have not yet been selected in the current step. Third, sequence generation: at each step The test case with the highest probability is selected as the next element in the sequence, removed from the candidate set, and its feature vector is input into the decoder to update its state. Repeat this process until all test cases are sorted, and output the complete sorted sequence. Fourth, multi-objective reward calculation and advantage function estimation: after executing the ranking sequence, the reward signal is calculated based on the test results. The reward function combines the normalized average percentage defect detection rate (NAPFD) with efficiency considerations based on execution time; and then calculates the advantage function. The methods for calculating NAPFD and the dominance function A are as follows: The NAPFD metric uses the relationship between the execution position of failed test cases in a test sequence and the total length of the sequence to represent the defect detection efficiency of the test case sequence. The calculation formula is as follows: ,middle, It is the percentage of defects detected within the testing budget. , It is a set of test examples that reveal defects. Test cases Position in the sequence, This is the original size of the test case set; the advantage function. The method used is to calculate the actual reward received. With commentator network predicting value The difference is used to represent the quality of the action performed under the current strategy relative to the average level. The calculation formula is as follows: ,in, It is the actual total reward obtained after executing the sorting sequence. It is a commentator network based on state Fifth, the strategy and value update of the dominant action evaluation algorithm: utilizing the dominance function. The parameters of the executor network (pointer network) are updated using the policy gradient ascent method. The parameters of the commenter network are updated by minimizing the mean squared error of the value function. The policy update is based on the policy gradient calculation formula, while the value update is based on the commenter network's loss function. The specific methods are as follows: The policy gradient is calculated by multiplying the probability gradient of the policy network's output action with the advantage function to represent the direction and magnitude of the parameter update. Its calculation formula is as follows: The commentator's network loss function uses the calculation of the actual reward. With network predictive value The mean squared error is used to represent the accuracy of value prediction, and its calculation formula is: .

2. The test case priority ranking method based on meta-reinforcement learning and adaptive pointer networks according to claim 1, characterized in that, In step 1), the meta-hyperparameter adjustment module dynamically adjusts the training hyperparameters of the deep reinforcement learning agent through the outer loop meta-training process, including initializing the base learning rate, base discount factor, base number of training epochs, and meta-adjustment factor; trains the agent using the current hyperparameter configuration through the inner loop deep reinforcement learning training; evaluates the agent's performance on the validation set, with the performance metric consisting of the average percentage defect detection rate (APFD) and the normalized rank percentile mean (NRPA); dynamically adjusts the hyperparameters based on performance changes, including the number of training epochs, learning rate, and discount factor; and repeats the inner loop training and hyperparameter adjustment until the meta-training converges.

3. The test case priority ranking method based on meta-reinforcement learning and adaptive pointer networks according to claim 1, characterized in that, In step 2), the pointer attention mechanism, combined with the dominant action evaluation algorithm, maps the test case feature sequence to the optimal execution order through an encoder-decoder structure. The encoder is a bidirectional long short-term memory network that generates a hidden state sequence. The decoder uses the pointer attention mechanism to calculate the selection probability of each test case and gradually generates a sorted sequence. After executing the sorted sequence, a reward signal is calculated based on the normalized average percentage defect detection rate (NAPFD) and the reward function based on the execution time. The parameters of the executor network and the commentator network are then updated using the dominant action evaluation algorithm.

Images