Time budget-oriented regression test task scheduling method and system

Abstract

The present application relates to software test scheduling and test equipment resource control, and discloses a time budget-oriented regression test task scheduling method and system. A scheduling server obtains a current round task, a test execution equipment available duration budget and a historical execution record formed before the round starts, determines a historical failure risk score and an estimated execution duration, generates a joint priority score and performs deterministic sorting. The sorting sequence remains unchanged in the budget determination process after being formed; the server accumulates the estimated execution duration from the first item in the sequence, stops at the first over-budget task, does not search for subsequent tasks without skipping the task, does not re-solve a task subset, obtains a continuous executable prefix and converts it into an equipment execution queue. The execution result and the actual execution duration returned by the equipment are written into the historical record for re-scoring and re-sorting in the subsequent round. The present application forms a determined control relationship among fixed sorting, budget interception and equipment queue formation.

Description

Technical Field

[0001] This invention relates to the fields of software testing, computer task scheduling, and test equipment resource control, specifically to a time-budget-oriented regression test task scheduling method and system. Background Technology

[0002] Software version iteration, continuous integration, and release verification typically require the repeated execution of existing test tasks. When the estimated total duration of test tasks exceeds the release window, night window, scheduled window, or available time of the test execution equipment, the test system needs to determine the actual execution order and scope within limited equipment resources and execution time. The execution time of different test tasks may vary significantly. When long-running tasks continuously occupy the test environment, it reduces the number of tasks that can yield results within the same execution window and delays subsequent diagnostics and re-execution.

[0003] Existing regression test task ranking schemes can prioritize high-risk tasks based on historical failure counts, recent failure patterns, or predicted risk, or select test tasks based on execution duration, test cost, or time constraints. When ranking solely based on historical failure information, the top ranks may be dominated by longer tasks; when ranking solely based on estimated execution duration, shorter tasks with lower failure risk may occupy the top ranks. Other schemes re-solve for a subset of objectives from candidate test tasks based on multiple objective constraints. This subset may not correspond to a continuous portion of the pre-ranked sequence and may generate new combined solutions and queue rearrangements when the budget changes.

[0004] Therefore, a technical solution is needed to connect historical failure risks, expected execution duration, and available time budget of test execution equipment according to a clear data relationship, so that the sorting results can be directly transformed into a device execution queue that maintains the semantics of order, and the execution results and actual execution duration can be used for historical updates in subsequent test rounds. Summary of the Invention

[0005] Technical problems to be solved The technical problem to be solved by this invention is: under the condition of limited test execution equipment and execution time, how to adjust the task duration composition at the beginning of the sorting while retaining historical failure risk information, determine the continuous executable range within the budget without changing the joint sorting semantics, avoid combining subsets or reordering candidate tasks in the budget determination stage, convert the continuous executable range into a task queue that the test execution equipment can directly execute, and form a cross-round historical update closed loop based on the execution results returned by the equipment and the actual execution time.

[0006] Technical solution To address the aforementioned technical problems, this invention provides a time-budget-oriented regression test task scheduling method executed by a scheduling server. The scheduling server is communicatively connected to a historical execution record storage device and at least one test execution device. The method includes the following steps:

[0007] S101, obtain the set of test tasks to be executed in the current test round, the available time budget of the test execution device, and the historical execution records formed before the start of the current test round.

[0008] S102, for each test task to be executed, determine the historical failure risk score based on the number of historical executions and the number of historical failures with the same test task identifier.

[0009] S103, determine the expected execution time based on the historical execution time with the same test task identifier; if there is no valid historical execution time, use the historical execution time statistics of the device, account, environment or task group as the time rollback value.

[0010] S104. Determine the joint priority score based on the historical failure risk score and the expected execution time, so that the joint priority score is positively correlated with the historical failure risk score and negatively correlated with the monotonically increasing duration penalty term corresponding to the expected execution time.

[0011] S105, the test tasks to be executed are deterministically sorted according to the joint priority score to obtain a sorting sequence; once the sorting sequence is formed, it remains unchanged during the budget determination process of the current test round.

[0012] S106, starting from the first item in the sorted sequence, the estimated execution time is accumulated sequentially.

[0013] S107, when adding the next test task would cause the cumulative estimated execution time to exceed the available time budget of the test execution device, terminate the prefix expansion, and do not skip the next test task but search for subsequent test tasks, and do not perform combined subset solution or reorder the unexecuted test tasks, to obtain a continuous executable prefix that maintains the sorted sequence order.

[0014] S108: Generate an execution queue that includes at least the test task identifier and execution order based on the continuous executable prefix, and send the execution queue to the test execution device for execution in sequence.

[0015] S109, Receive the execution result and actual execution time returned by the test execution device, and write the execution result and actual execution time into the historical execution record for re-scoring and sorting in subsequent test rounds.

[0016] The present invention also provides a regression test task scheduling system corresponding to the above method, as well as an electronic device, a computer-readable storage medium, and a computer program product for executing the above method.

[0017] Beneficial effects Since historical failure risk scores and expected execution times are both used in the generation of joint priority scores, the ranking sequence can adjust the task duration composition at the beginning of the ranking while retaining historical failure information, reducing the possibility that the ranking results are biased only towards high-risk long tasks or low-risk short tasks.

[0018] Because the budget control module keeps the sorting sequence unchanged after it is formed, it only expands the consecutive executable prefixes sequentially from the first item of the sorting sequence and stops at the first over-budget task. It does not search for the tasks that can be accommodated after the task, nor does it re-solve the candidate task combination or re-sort. Therefore, it can reduce the candidate traversal, combination solution and execution queue rearrangement operations in the budget determination stage. The device execution queue and the joint sorting sequence are in one-to-one correspondence, which is conducive to maintaining the consistency and reproducibility between task sequence number, device issuance order, execution log and result collection.

[0019] Because the execution results and actual execution time returned by the test execution device are associated with and written into the historical execution record, and only participate in scoring and duration estimation before the start of subsequent test rounds, the sorting input boundary of the current round is clear, thus forming a traceable technical closed loop between device execution and cross-round rescheduling.

[0020] This invention does not presuppose that the number of failed tasks discovered will necessarily increase in any testing scenario. Its technical function is to establish an implementable data processing and control relationship between failure risk, task duration, and device execution window, providing a stable, deterministic task sequence that can be directly executed by the test execution device for scheduling regression test tasks within a limited execution window. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the regression test task scheduling system architecture provided in an embodiment of the present invention.

[0022] Figure 2 A flowchart of a time-budget-oriented regression test task scheduling method provided in an embodiment of the present invention.

[0023] Figure 3 This is a schematic diagram illustrating the joint priority scoring generation relationship provided in an embodiment of the present invention.

[0024] Figure 4 This is a schematic diagram illustrating the determination of a continuously executable prefix under time budget conditions, as provided in an embodiment of the present invention.

[0025] Figure 5This is a schematic diagram illustrating the cross-round historical updates of execution results and actual execution time provided in the embodiments of the present invention. Detailed Implementation

[0026] The embodiments of the present invention will be described below with reference to the accompanying drawings. These embodiments are used to explain the technical solutions of the present invention and do not constitute a limitation on the scope of protection; where there is no technical conflict, technical features from different embodiments can be combined.

[0027] Example 1: System Architecture like Figure 1 As shown, the test system includes a scheduling server 100, a historical execution record storage 200, and at least one test execution device 300. It may also include a management terminal 400 or a resource management system. The scheduling server 100 is equipped with a data acquisition module 110, a risk scoring module 120, a duration estimation module 130, a joint sorting module 140, a budget control module 150, an execution control module 160, and a historical update module 170.

[0028] The historical execution record storage 200 can utilize a relational database, time-series database, object storage, or file storage. Each historical execution record must at least include a test task identifier, execution result, and actual execution duration. It may also include a test round identifier, execution device identifier, execution environment identifier, start time, end time, and record validity identifier. The test task identifier can use an anonymized identifier or a hash identifier; it is not required to store customer names, business names, source code, defect details, or original log content.

[0029] The test execution device 300 can be a physical test machine, virtual machine, container node, simulation device, or other computing resource capable of executing software testing tasks. The scheduling server 100 can send execution queues through message queues, remote procedure call interfaces, application programming interfaces, or inter-process communication interfaces. The test execution device 300 executes test tasks in the execution order and returns task identifiers, execution results, actual execution time, and execution status.

[0030] Example 2: Data Objects, Time Boundaries, and Execution Queues In step S101, the scheduling server obtains a set of test tasks to be executed. Each test task must have at least a test task identifier. It can also include device compatibility identifiers, environment identifiers, account identifiers, version identifiers, planned execution round identifiers, or original sequence identifiers. The scheduling server simultaneously obtains the available time budget for the test execution device. The available time budget can be provided by the publishing window, the scheduled time period, the maintenance window, or the resource management system, or it can be determined by the ratio of the total planned time to the preset budget.

[0031] The scoring inputs for the current test round include the test task to be executed, historical execution results generated before the start of the current test round, historical execution duration, and estimated device availability time; the output includes a sorted sequence. and consecutive executable prefixes The execution results and actual execution time generated in the current test round are not included in the current test round's score; they are only written into the historical execution record and used in subsequent test rounds.

[0032] The execution queue may include at least one of the following: queue identifier, target device identifier, test task identifier, sequence number, estimated execution duration, budget identifier, and execution parameters. The feedback message returned by the test execution device may include at least one of the following: queue identifier, test task identifier, execution result, actual execution duration, completion status, and exception type. The scheduling server associates the feedback message with the corresponding task in the sorted sequence based on the queue identifier and test task identifier.

[0033] Example 3: Historical Failure Risk Scoring For the One test task to be executed In the historical execution record set corresponding to its device, account, environment, or task group. In the statistics, those with the same test task identifier Historical execution count and number of historical failures Historical execution results can be represented by binary variables. This indicates that a value of 1 represents failure and 0 represents success; records that are abnormal, interrupted, or have missing results can be excluded according to preset validity rules.

[0034] When the number of historical executions is small or there is no historical record, directly using the failure frequency may result in significant fluctuations. Therefore, we first calculate the overall failure rate of the local historical record set. :

[0035] Then use the smoothing parameter λ to determine the historical failure risk score. : A preset smoothing parameter greater than zero can be set based on the historical sample size. For samples that meet the following criteria... and The cold start test task, Equal to the overall failure rate of local history Local historical sets are also spacetime. The system uses a historical overall failure rate or a preset prior failure rate to ensure that the score can be calculated.

[0036] Example 4: Expected Execution Duration and Joint Priority The duration estimation module 130 determines the estimated execution time based on the valid positive historical execution time corresponding to the same test task identifier. In a preferred embodiment, the historical median is used first; if the median is invalid or not positive, the historical mean is used; if no valid value is found, the positive historical execution duration statistics of the device, account, environment, or task group are used as the duration rollback value; if there is no valid historical duration for the corresponding group, the system-level positive historical execution duration statistics or the preset positive duration are used.

[0037] To prevent the duration penalty from going out of bounds due to zero duration or abnormally small values, a lower limit is applied to the expected execution duration: The lower limit for duration is greater than zero, and can be set to 1 second, 1 minute, or other positive values ​​according to the duration unit used by the system. In one embodiment, the joint sorting module 140 calculates the joint priority score according to the following formula. :

[0038] The duration penalty parameter is greater than zero. As the value increases, the impact of the expected execution time on the sorting becomes more significant. As the score approaches zero, the ranking gradually approaches the result based solely on historical failure risk scores. The value can be pre-selected from a preset candidate set including at least two values ​​from 0.25, 0.5, and 1.0, after offline verification in independent historical rounds, and fixed before the start of the current test round; execution results or actual execution time not yet generated in the current test round are not used for selection. . Figure 3 The lower bound of the estimated execution time is represented by the same symbols as in the above formula. .

[0039] In another embodiment, the duration penalty term can be any other monotonically increasing function determined by the expected execution duration, provided that the joint priority score is positively correlated with the historical failure risk score and negatively correlated with the duration penalty term. The joint sorting module 140 generates a sorting sequence in descending order of the joint priority score.

[0040] When joint priority scores are the same, a pre-defined parallel resolution rule is used. The parallel resolution rule can compare global order, round order, planned execution time, test task identifier, and original record position in sequence to ensure that the same input yields the same sorting result.

[0041] Example 5: Continuous executable prefix and non-skip rule Budget control module 150 in sorting sequence Once formed, this sorted sequence remains unchanged during the budget decision process in the current test round, and is then used to determine the order of items from the sorted sequence. The first item begins to accumulate the estimated execution time after the lower limit is applied. To meet the previous The sum of the estimated execution times of the items shall not exceed the available time budget. The largest non-negative integer; when the estimated execution time of the first item has exceeded hour, And the consecutive executable prefix is ​​empty. Add the first... The cumulative estimated execution time after the item exceeds When the expansion stops, a continuous executable prefix is ​​obtained:

[0042] This implementation method does not skip step [number]. The system searches for subsequent short tasks, without combining subsets of unexecuted test tasks or altering the sorting sequence. The actual distributed set remains a continuous prefix of the combined sorting sequence, ensuring a correspondence between sorting position, queue number, device execution order, and execution log. The execution control module 160 will then use this continuous executable prefix... The test task identifier and execution order are encapsulated into an execution queue and sent to the test execution device 300.

[0043] Example 6: Cross-round update of execution results and actual execution time like Figure 5 As shown, the test execution device 300 executes test tasks according to the execution queue of the current test round, and returns the execution results, actual execution time, and completion status of each test task to the scheduling server 100. The scheduling server 100 associates the execution results, actual execution time, and completion status with the corresponding test task identifier and writes them into the historical execution record.

[0044] The execution result and actual execution time returned by the current test round do not change the sorting sequence and consecutive executable prefix already formed in the current test round, nor are they used to add test tasks outside the original execution queue to the current test round.

[0045] Before the start of the next test round, the scheduling server 100 reads the updated historical execution records, re-determines the historical failure risk score and expected execution time of each test task to be executed, and generates the sorting sequence and continuous executable prefix for the next test round accordingly. This forms a cross-round closed loop of fixed execution queue, result and duration recycling, historical record update, and re-scoring and sorting in subsequent rounds.

[0046] Example 7: Exemplary Calculation Process The task numbers and values ​​below are exemplary parameters used only to explain the calculation steps and do not represent experimental results, nor do they correspond to any real enterprise, customer, device, dataset, or test record. Let the local historical overall failure rate be assumed. Smoothing parameters Minimum duration Minutes, duration penalty parameter Available time budget minute.

[0047] Based solely on historical failure risk scoring During sorting, T03 is first, with an estimated execution time of 15 minutes; after adding T01, the cumulative estimated execution time is 23 minutes, exceeding the 20-minute budget. The only consecutive executable prefix contains T03. Scoring is based on joint priority. During sorting, the sequence is T02, T01, T03, T06, T05, T04. The cumulative estimated execution time of T02 and T01 is 11 minutes, which becomes 26 minutes after adding T03. Therefore, the consecutive executable prefix is ​​{T02, T01}, and T06 is selected without skipping T03.

[0048] After T02 and T01 are executed, their execution results and actual execution time are associated with the corresponding task identifier and written into the historical execution record. Before the start of the next test round, they will be re-involved in the calculation of historical failure risk score and expected execution time. The data does not change the sorting sequence and consecutive executable prefix that have been formed in the current test round of this example.

[0049] Example 8: Multiple Devices and Compatibility Constraints When the testing system includes multiple test execution devices, each test task can carry a device type, operating system, version, interface, account, or environment compatibility identifier. The scheduling server forms a set of local tasks for each device based on the compatibility identifier, calculates the historical failure risk score and estimated execution time within the local historical record range corresponding to each device, and generates a corresponding sorting sequence and consecutive executable prefix based on the independent available time budget of each device.

[0050] The same test task can be limited to being assigned to only one compatible device; when multiple compatible devices exist, the target device can be selected by combining device load, task affinity, or mutual exclusion constraints. After the target device is determined, the joint priority is determined within the local task set of the corresponding device based on historical failure risk scores and expected execution time.

[0051] Example 9: Historical Updates, Exception Handling, and Data Minimization After the task is completed, the history update module 170 associates and stores the execution result, actual execution time, test round identifier, execution device identifier, and record validity identifier with the corresponding test task identifier. Before the next test round begins, the scheduling server can use all valid historical execution records, or it can form a rolling history window according to a fixed number of rounds, a fixed time range, or an exponential decay method.

[0052] Execution interruptions, equipment failures, invalid logs, or missing execution results can be marked as abnormal records and excluded from historical failure risk scores and estimated execution time statistics, or handled separately according to preset abnormal categories. Records with missing execution results are not included in the historical failure count; missing or non-positive execution times are handled using a time rollback value and a lower limit.

[0053] Historical execution records can save only the anonymized test task identifier, binary or enumerated execution results, execution duration, and necessary compatibility identifiers. It is not required to save the real customer name, business name, device serial number, account name, source code, defect details, or original log content.

[0054] Example 10: Electronic devices, storage media, and computer program products Electronic devices may include a processor, memory, a communication interface, and a bus. The memory stores a computer program, which, when executed by the processor, implements the methods described in any of the above embodiments. The computer-readable storage medium may be a read-only memory, random access memory, solid-state drive, magnetic disk, optical disk, or other medium capable of storing program instructions. The computer program product may be provided via network transmission, download, or other means, and implements the above methods when executed by the processor.

[0055] Without altering the current round's historical input boundaries, joint priority calculation relationships, fixed sorting sequences, budget continuous prefix semantics, rules against skipping or searching subsequent tasks, rules against resolving candidate task combination rules, and cross-round historical update relationships, the order of the above steps can be adaptively adjusted. The scope of protection of the claims is determined by the content of the claims statement.

Claims

1. A time-budget-oriented regression test task scheduling method, applied to a test system including a scheduling server, a historical execution record storage, and at least one test execution device, characterized in that, include: Obtain the set of test tasks to be executed in the current test round, the available time budget of the test execution device, and the historical execution records formed before the start of the current test round. The historical execution records include at least a test task identifier, historical execution result, and historical execution duration. For each test task to be executed in the set of test tasks to be executed, determine a historical failure risk score based on the historical execution results with the same test task identifier, and determine the expected execution duration based on the historical execution duration with the same test task identifier. Generate a joint priority score based on the historical failure risk score and the expected execution duration. The joint priority score is positively correlated with the historical failure risk score and negatively correlated with a monotonically increasing duration penalty term determined by the expected execution duration. Sort the joint priority scores in descending order, and when the joint priority scores are the same, perform deterministic sorting according to a preset parallel resolution order to obtain a sorting sequence. After the sorting sequence is formed, During the budget determination process for the current test round, the sorting sequence remains unchanged. Starting from the first item of the sorting sequence, the estimated execution time of each test task to be executed in the sorting sequence is accumulated sequentially. Expansion stops when adding the next test task would cause the accumulated estimated execution time to exceed the available time budget, and the next test task is not skipped while searching for subsequent test tasks. Unexecuted test tasks are not combined subsets or reordered, resulting in a continuous executable prefix that maintains the order of the sorting sequence. An execution queue, including at least the corresponding test task identifier and execution order, is generated based on the continuous executable prefix, and the execution queue is sent to the corresponding test execution device. The execution result and actual execution time returned by the test execution device are received, and the execution result and actual execution time are associated with the corresponding test task identifier and written into the historical execution record for subsequent test rounds to re-determine the historical failure risk score, estimated execution time, and sorting sequence.

2. The method according to claim 1, characterized in that, When determining the historical failure risk score and the estimated execution time, only the historical execution results and historical execution times that have been completed and recorded before the start of the current test round are used; the data generated by any test task to be executed in the current test round does not participate in the joint priority score of that test task in the current test round.

3. The method according to claim 1, characterized in that, The historical failure risk score is determined by the following formula: in, For the first Historical failure risk score for each test task to be executed For historical execution counts with the same test task identifier, For the corresponding number of historical failures, For the first The historical overall failure rate of the device, account, environment, or task group to which the test task to be executed belongs. A smoothing parameter greater than zero; when and hour, When the device, account, environment, or task group has no valid historical records, Use the system-level historical overall failure rate or a preset prior failure rate.

4. The method according to claim 3, characterized in that, The estimated execution time is preferentially based on the median of the positive historical execution time corresponding to the same test task identifier; when the median is invalid or not positive, the mean of the positive historical execution time is used; when there is no valid task-level historical execution time, the positive historical execution time statistics of the device, account, environment, or task group are used as the time rollback value; when there is no valid positive historical execution time for the device, account, environment, or task group, the system-level positive historical execution time statistics or the preset positive time are used; the estimated execution time is subjected to a lower limit processing to obtain: in, For the first The estimated execution time of each test task to be executed. The estimated execution time after lower limit processing. The duration is a lower limit greater than zero; the joint priority score is determined by the following formula: in, For the first The joint priority score of each test task to be executed. Assess the risk of historical failures. A duration penalty parameter greater than zero; the duration penalty parameter The selected value is chosen from a preset candidate set including at least two values ​​from 0.25, 0.5, and 1.0, and fixed before the start of the current test round. The execution results or actual execution time that have not yet been generated in the current test round will not be used.

5. The method according to claim 1, characterized in that, When the joint priority scores are the same, they are resolved sequentially according to the global order, round order, planned execution time, test task identifier, and original record position fixed before the start of the current test round; let the sorting sequence be... ,make To satisfy the sorting sequence before The sum of the expected execution times for each item does not exceed the largest non-negative integer of the available time budget; when When, the continuous executable prefix is ​​empty; when hour, This is the first task exceeding the budget; The consecutive executable prefix is ​​only composed of to The composition, and the budget determination process is incorrect. Subsequent test tasks will perform a capacity search.

6. The method according to claim 1, characterized in that, Updating the historical execution records includes: associating and storing the current execution result, actual execution duration, test round identifier, execution device identifier, and record validity identifier with the corresponding test task identifier; recalculating the historical failure risk score and expected execution duration in the next test round using a scrolling history window or all available historical execution records; and excluding execution interruptions, device failures, missing execution results, or non-positive execution durations based on preset exception categories, using rollback values, or applying lower limits.

7. A time-budget-oriented regression test task scheduling system, comprising a scheduling server, a historical execution record storage, and at least one test execution device, characterized in that, The scheduling server includes: a data acquisition module, used to acquire the set of test tasks to be executed in the current test round, the available time budget of the test execution equipment, and historical execution records formed before the start of the current test round; a risk scoring module, used to determine the historical failure risk score of each test task to be executed based on the historical execution results; a duration estimation module, used to determine the expected execution time of each test task to be executed based on the historical execution time; a joint sorting module, used to generate a joint priority score based on the historical failure risk score and the expected execution time and perform deterministic sorting to obtain a sorting sequence; and a budget control module, used to keep the sorting sequence unchanged during the budget determination process of the current test round after the sorting sequence is formed. Starting from the first item of the sorted sequence, the estimated execution time of each test task to be executed is accumulated sequentially. The process stops when adding the next test task would exceed the available time budget, without skipping the next test task and searching for subsequent test tasks. No subset solving or reordering of unexecuted test tasks is performed, resulting in a continuous executable prefix. An execution control module is used to generate an execution queue based on the continuous executable prefix and distribute it to the corresponding test execution device. A history update module is used to receive the execution results and actual execution time returned by the test execution device and write the execution results and actual execution time into the historical execution record for use in subsequent test rounds. The joint sorting module is configured to calculate the joint priority score according to the following formula: in, For the first The joint priority score of each test task to be executed. Assess the risk of historical failures. The estimated execution time after lower limit processing. The duration penalty parameter is greater than zero; the data written by the historical update module is only used in subsequent test rounds and does not participate in the joint priority scoring of the current test round.

8. An electronic device comprising a memory, a processor, a communication interface, and a computer program stored in the memory and capable of running on the processor, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 6.

Images