Method, device and computer equipment for scheduling test tasks

Abstract

The present disclosure provides a method for scheduling test tasks, relating to the technical field of computer, and the implementation scheme is: obtaining a test task to be scheduled, a test data set, and a state of computing power and bandwidth resources; calculating an algorithm feature vector, a data set feature vector, and a test strategy feature vector based on the test task, the test data set, and the state of computing power and bandwidth resources; constructing a ternary relationship graph based on the algorithm feature vector, the data set feature vector, and the test strategy feature vector; performing feasibility judgment and cost evaluation on each ternary relationship graph; selecting a target ternary relationship graph to generate a task execution scheme based on the feasibility judgment and the cost evaluation; and binding the test task to the matched resource slot and data set for execution according to the task execution scheme, which can realize the integration of algorithm features, data set features, and test strategies into decision-making, can prioritize the coverage of key scenarios, and can improve resource utilization, reduce queuing delay, and reduce data transfer cost.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, particularly to the fields of algorithm testing engineering and task scheduling, and especially to methods, apparatus, computer equipment, computer-readable storage media, and computer program products for scheduling test tasks. Background Technology

[0002] Existing computing power scheduling schemes typically take "task resource requirements (e.g., CPU, GPU, or memory) + available node resources" as input, selecting nodes through rules, reinforcement learning, or heuristics. GPU platform solutions often combine resource partitioning, queuing, training backhaul, and release tracking to improve utilization and reduce waiting time. Other solutions optimize GPU topology, non-uniform memory access (NUMA), and fragmentation at the hardware level, and improve utilization and fairness in multi-tenant scenarios through preemption / reordering. Algorithm automated testing / regression testing systems mainly focus on process automation (task splitting and distribution, environment and test case management, report generation, coverage evaluation, etc.), with scheduling often treated as a process step, and resource-side reliance still on general queues / schedulers. Summary of the Invention

[0003] This disclosure provides a method, apparatus, computer device, computer-readable storage medium, and computer program product for scheduling test tasks.

[0004] According to one aspect of this disclosure, a method for scheduling test tasks is provided. The method includes: obtaining the status of one or more test tasks to be scheduled, one or more test datasets, and computing power and bandwidth resources; calculating an algorithm feature vector, a dataset feature vector, and a test strategy feature vector based on the status of the one or more test tasks, one or more test datasets, and computing power and bandwidth resources; constructing one or more ternary relationship graphs between the one or more test tasks, one or more test datasets, and one or more resource slots based on the algorithm feature vectors, dataset feature vectors, and test strategy feature vectors, wherein the one or more resource slots are mathematical models of computing power and bandwidth resources; and performing feasibility judgment and cost evaluation on each ternary relationship graph, wherein the feasibility judgment indicates whether the test tasks, resource slots, and datasets in each ternary relationship graph can be matched and executed. Cost assessment indicates the cost of matching and executing test tasks, resource slots, and datasets in each ternary relation graph. Based on feasibility judgment and cost assessment, and with the condition of simultaneously reducing the cost associated with task time and data migration, while increasing the coverage of user-preset scenarios and the utilization efficiency of computing power and bandwidth resources, one or more target ternary relation graphs from one or more ternary relation graphs are selected to generate a task execution plan. The task execution plan includes batch grouping and execution order of one or more test tasks, and at least one resource slot from one or more resource slots and at least one test dataset from one or more test datasets that match one or more test tasks. According to the task execution plan, one or more test tasks are bound to at least one matched resource slot and at least one test dataset for execution in accordance with batch grouping and execution order.

[0005] In some embodiments, one or more ternary graphs use edges to represent the requirements and coverage gain of one or more test tasks for one or more test datasets, the data transfer cost of one or more test datasets to one or more resource slots, and the degree to which one or more resource slots satisfy the computing power and environmental dependencies of one or more test tasks.

[0006] In some embodiments, the algorithm feature vector includes the algorithm's model size, operator dependencies, and computational complexity; the dataset feature vector includes the dataset's data size, I / O read / write patterns, and storage location; and the test strategy feature vector includes the test type, priority, deadline, and scenario coverage target of the test task.

[0007] In some embodiments, selecting one or more target ternary graphs from one or more ternary graphs to generate a task execution plan, while simultaneously satisfying the conditions of reducing the cost associated with task time and data migration, increasing the coverage of user-preset scenarios, and improving the utilization efficiency of computing power and bandwidth resources, includes: constructing a multi-objective loss function by weighted summing of task timeout penalty, coverage gain, resource utilization, queuing latency, and data migration cost, wherein the coverage gain is used to represent the coverage of one or more test tasks to user-preset scenarios; and selecting one or more target ternary graphs from one or more ternary graphs to generate a task execution plan by calculating the multi-objective loss function.

[0008] In some embodiments, the method further includes monitoring performance metrics while executing one or more test tasks, and dynamically adjusting the task execution plan based on the performance metrics.

[0009] In some embodiments, performance metrics include task timeout penalties, coverage gain, resource utilization, queuing latency, and data migration costs.

[0010] In some embodiments, adjusting the task execution scheme includes at least one of the following: adjusting the batch grouping of one or more test tasks; adjusting the execution order of one or more test tasks; adjusting the cache; performing data migration; and rebinding one or more test tasks to at least one resource slot other than at least one resource slot in one or more resource slots.

[0011] In some embodiments, the feasibility assessment includes computing power feasibility, environmental dependency feasibility, bandwidth feasibility, and time-limit feasibility, and the cost assessment includes data transfer cost, bandwidth usage penalty, and I / O mode conflict penalty.

[0012] According to another aspect of this disclosure, an apparatus for scheduling test tasks is provided. The apparatus includes: an acquisition module configured to acquire the status of one or more test tasks to be scheduled, one or more test datasets, and computing power and bandwidth resources; a feature calculation module configured to calculate an algorithm feature vector, a dataset feature vector, and a test strategy feature vector based on the status of the one or more test tasks, one or more test datasets, and computing power and bandwidth resources; a ternary relationship graph construction module configured to construct one or more ternary relationship graphs between the one or more test tasks, one or more test datasets, and one or more resource slots based on the algorithm feature vectors, dataset feature vectors, and test strategy feature vectors, wherein the one or more resource slots are mathematical models of computing power and bandwidth resources; and a judgment and evaluation module configured to perform a feasibility judgment and cost evaluation on each ternary relationship graph, wherein the feasibility judgment indicates that the test tasks, resource slots, and datasets in each ternary relationship graph are feasible. Whether matching and execution are possible, the cost assessment indicates the cost of matching and executing test tasks, resource slots, and datasets in each ternary relation graph; the task execution plan generation module is configured to select one or more target ternary relation graphs from one or more ternary relation graphs to generate a task execution plan based on feasibility judgment and cost assessment, with the conditions of simultaneously reducing the cost associated with task time and data migration, increasing the coverage of user-preset scenarios, and the utilization efficiency of computing power and bandwidth resources. The task execution plan includes batch grouping and execution order of one or more test tasks, and at least one resource slot from one or more resource slots and at least one test dataset from one or more test datasets that match one or more test tasks; and the execution module is configured to bind one or more test tasks to at least one matched resource slot and at least one test dataset according to the task execution plan, so as to execute them in accordance with batch grouping and execution order.

[0013] According to another aspect of this disclosure, a computer device is provided, comprising: at least one processor; and a memory having a computer program stored thereon, wherein the computer program, when executed by the at least one processor, causes the at least one processor to perform the methods provided above in this disclosure.

[0014] According to another aspect of this disclosure, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, causes the processor to perform the methods provided above in this disclosure.

[0015] According to another aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, causes the processor to perform the methods provided above in this disclosure.

[0016] According to one or more embodiments of this disclosure, by incorporating algorithm features, dataset features, and testing strategies into decision-making, key versions or their releases can be completed on time, key scenario coverage can be prioritized, resource utilization can be improved, and queuing latency and data transfer costs can be reduced.

[0017] These and other aspects of this disclosure will be apparent from the embodiments described below, and will be elucidated with reference to the embodiments described below. Attached Figure Description

[0018] The accompanying drawings exemplify embodiments and form part of the specification, serving together with the textual description to explain exemplary implementations of the embodiments. The illustrated embodiments are for illustrative purposes only and do not limit the scope of this disclosure. Throughout the drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0019] Figure 1 This is a flowchart illustrating a method for scheduling test tasks according to an exemplary embodiment.

[0020] Figure 2 This is a flowchart illustrating the steps of a method for scheduling test tasks according to an exemplary embodiment.

[0021] Figure 3 This diagram illustrates the specific flow of a method for scheduling test tasks according to an exemplary embodiment.

[0022] Figure 4 This is a block diagram illustrating an apparatus for scheduling test tasks according to an exemplary embodiment.

[0023] Figure 5 An example computer device is shown in which any of the embodiments described herein may be implemented. Detailed Implementation

[0024] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0025] In this disclosure, unless otherwise stated, the use of terms such as "first," "second," etc., to describe various elements is not intended to limit the positional, temporal, or importance relationships of these elements; such terms are merely used to distinguish one element from another. In some examples, the first element and the second element may refer to the same instance of that element, while in other cases, based on the context, they may refer to different instances.

[0026] The terminology used in the description of the various examples described in this disclosure is for the purpose of describing particular examples only and is not intended to be limiting. Unless the context explicitly indicates otherwise, an element may be one or more unless the number of elements is specifically limited. As used herein, the term "multiple" means two or more, and the term "based on" should be interpreted as "at least partially based on". Furthermore, the terms "and / or" and "at least one of..." cover any one of the listed items and all possible combinations thereof.

[0027] Current technologies still focus on task requirements and computing power supply or hardware resource optimization, failing to incorporate the following dimensions of algorithm testing engineering into a unified scheduling model: the characteristics of the test dataset (size, I / O (input / output) mode, scene label / coverage gain) and the priority and time constraints of test strategies (regression / load testing / comparison). This leads to the following problems in testing scenarios in existing technologies: 1) I / O or network becomes a hidden bottleneck, with computing resources appearing idle but throughput limited; 2) Customer-preset key scenarios / key version regressions compete with ordinary tasks in the same queue, causing delays in critical tests; 3) Resource utilization is diluted by data waiting and queuing, resulting in "GPUs waiting for data, CPUs waiting for storage." Furthermore, existing technologies cannot solve the following problems: how to prioritize which test tasks to run under the same computing power to maximize "scene coverage completion," and how to allocate computing resources and datasets between regression / load testing / canary comparison before the release deadline to ensure deployment quality at the lowest cost.

[0028] The embodiments of this disclosure provide a method for scheduling test tasks, which incorporates algorithm features, dataset features, and test strategies into the decision-making process, enabling test tasks to be completed on time, prioritizing the coverage of key scenarios, improving resource utilization, and reducing queuing latency and data transfer costs.

[0029] Figure 1 This is a flowchart illustrating a method 100 for scheduling test tasks according to an exemplary embodiment.

[0030] like Figure 1As shown, this disclosure proposes a method 100 for scheduling test tasks, including the following steps: S102, obtaining the status of one or more test tasks to be scheduled, one or more test datasets, and computing power and bandwidth resources; S104, calculating algorithm feature vectors, dataset feature vectors, and test strategy feature vectors based on the status of one or more test tasks, one or more test datasets, and computing power and bandwidth resources; S106, constructing one or more ternary relationship graphs between one or more test tasks, one or more test datasets, and one or more resource slots based on the algorithm feature vectors, dataset feature vectors, and test strategy feature vectors, wherein one or more resource slots are mathematical models of computing power and bandwidth resources; S108, performing feasibility judgment and cost evaluation on each ternary relationship graph, wherein the feasibility judgment indicates whether the test tasks, resource slots, and datasets in each ternary relationship graph can be matched. The process involves matching and executing test tasks, resource slots, and datasets in each ternary relation graph, with a cost assessment indicating the cost of matching and executing them. S110: Based on feasibility judgment and cost assessment, and with the conditions of simultaneously reducing costs associated with task time and data migration while increasing coverage of user-preset scenarios and the efficiency of computing power and bandwidth resource utilization, one or more target ternary relation graphs from one or more ternary relation graphs are selected to generate a task execution plan. The task execution plan includes batch grouping and execution order of one or more test tasks, and at least one resource slot from one or more resource slots matched with one or more test tasks, and at least one test dataset from one or more test datasets. S112: According to the task execution plan, one or more test tasks are bound to at least one matched resource slot and at least one test dataset for execution according to batch grouping and execution order.

[0031] In step S102, the status of one or more test tasks to be scheduled, one or more test datasets, and computing power and bandwidth resources are obtained.

[0032] In the example, in the algorithm testing project, one or more test tasks for testing algorithms or models can be obtained from the test task access layer (e.g., triggered via command line interface (CLI), application programming interface (API), and continuous integration (CI)). These one or more test tasks can be test tasks under different user requirements, different scenarios, different types, or different versions of different algorithm models.

[0033] In some examples, the test task can be of the type of regression testing, stress testing, comparative testing, etc., without limitation. Regression testing is a test conducted to verify that the original functions and performance have not unexpectedly degraded or malfunctioned due to the modification of the algorithm code, model, or parameters. Stress testing is a test to evaluate the performance, stability, and bottlenecks of an algorithm or system under extreme or abnormal load conditions. Comparative testing involves running the new version of the algorithm (version A) and the baseline version (version B, usually the online stable version) on the same test set and environment, and comparing various performance indicators.

[0034] In the example, one or more test datasets can be obtained from the dataset management office. These test datasets can be for different scenarios, such as datasets for testing the sentiment analysis scenario of language models, datasets for testing the mathematical calculation ability of algorithm models, or datasets for any other scenario, or datasets of basic facial attributes for face recognition, etc. There are no restrictions here.

[0035] In the example, computing power and bandwidth resources can include computing power resources and bandwidth resources. Computing power resources can include hardware computing power resources such as GPUs, CPUs, or Sparse Processing Units (SPUs), and their status can include whether the computing power resources are idle, as well as the utilization rate, temperature, and power of the computing power resources. Bandwidth resources can include intra-node bandwidth, inter-node bandwidth, and storage bandwidth, and their status can include the available bandwidth, latency, current load, packet loss rate, queue depth, and throughput of the bandwidth resources.

[0036] In step S104, based on one or more test tasks, one or more test datasets, and the state of computing power and bandwidth resources, the algorithm feature vector, dataset feature vector, and test strategy feature vector are calculated.

[0037] In the example, features are extracted and profiled from one or more test tasks, one or more test datasets, and the status of computing power and bandwidth resources to generate algorithm feature vectors, dataset feature vectors, and test strategy feature vectors. The algorithm feature vectors are associated with the algorithms or models targeted by one or more test tasks, the dataset feature vectors are associated with one or more test datasets, and the test strategy feature vectors are associated with various operating metrics.

[0038] In step S106, based on the algorithm feature vector, dataset feature vector, and test strategy feature vector, one or more ternary relationship graphs are constructed between one or more test tasks, one or more test datasets, and one or more resource slots, wherein one or more resource slots are mathematical models of computing power and bandwidth resources.

[0039] In some examples, computing resources (e.g., CPU / GPU / SPU) and bandwidth resources (e.g., network bandwidth and storage bandwidth) are uniformly mathematically modeled into allocable resource slots. Furthermore, these resource slots also include constraints on computing and bandwidth resources, such as concurrency, affinity or anti-affinity, and time windows. Affinity and anti-affinity are constraints used in the scheduling system to control the placement relationship between tasks and resources. For example, tasks and resources located in different locations are considered anti-affinity (e.g., needing to retrieve data from a different location), while tasks and resources located in the same location are considered affinity (e.g., able to directly retrieve data). Concurrency indicates the constraint on how many tasks can share the same resource over time. A time window indicates a predetermined period of time during which a resource is available for a specific task.

[0040] By unifying the abstraction of resource slots, the computing, storage, and network bandwidth are explicitly modeled and allocated in an integrated manner. Combined with data prefetching, caching strategies, and I / O rate limiting, the phenomenon of "GPU waiting for data and CPU waiting for storage" is reduced. At the same time, through batch combination optimization and conflict avoidance, congestion and I / O interference in the shared bandwidth domain are reduced, and the overall resource throughput is improved.

[0041] In some examples, each ternary graph may include one test task from one or more test tasks, one test dataset from one or more test datasets, and one resource slot from one or more resource slots. For example, for task T1, dataset D1, and two resource slots R1 and R2, a first ternary graph including task T1, dataset D1, and resource slot R1 and a second ternary graph including task T1, dataset D1, and resource slot R2 can be generated.

[0042] Therefore, by incorporating algorithm features, datasets, and testing strategies into the scheduling decision, the limitations of traditional scheduling that only focuses on computing power supply are overcome, and the problem of "computing power idle and throughput limited" caused by data access bottlenecks in testing scenarios is solved from the root.

[0043] In step S108, a feasibility judgment and cost assessment are performed on each ternary relation graph. The feasibility judgment indicates whether the test task, resource slot and dataset in each ternary relation graph can be matched and executed. The cost assessment indicates the cost of matching and executing the test task, resource slot and dataset in each ternary relation graph.

[0044] In some examples, a feasibility assessment and cost evaluation can be performed on a ternary relationship graph including task T1, dataset D1, and resource slot R1. Specifically, the feasibility assessment indicates whether the test task T1, resource slot R1, and dataset D1 in the ternary relationship graph can be matched and executed, and the cost evaluation indicates the cost of matching and executing the test task T1, resource slot R1, and dataset D1 in the ternary relationship graph.

[0045] In step S110, based on feasibility judgment and cost assessment, and with the conditions of simultaneously reducing the cost associated with task time and data migration, while increasing the coverage of user-preset scenarios and the utilization efficiency of computing power and bandwidth resources, one or more target ternary relationship graphs from one or more ternary relationship graphs are selected to generate a task execution plan. The task execution plan includes batch grouping and execution order of one or more test tasks, and at least one resource slot from one or more resource slots that match one or more test tasks and at least one test dataset from one or more test datasets.

[0046] In the example, the user-preset scenarios can be key scenarios that the user specifies must be tested before the deadline. Therefore, test tasks that cover these scenarios can be tested first.

[0047] In the example, a task execution plan can be generated based on one or more target ternary graphs. For example, a corresponding task execution plan can be generated based on a target ternary graph including task T1, dataset D1, and resource slot R1. Specifically, the task execution plan may include binding T1 to D1 and R1 for batch execution. As another example, a corresponding task execution plan can be generated based on target ternary graphs including task T2, dataset D2, and resource slot R2 and target ternary graphs including task T3, dataset D2, and resource slot R2. Specifically, the task execution plan may include binding T2 and T3 to D2 and R3 for batch execution, with T2 executed first and T3 executed later.

[0048] Therefore, by taking the increase in coverage of user-preset scenarios as a clear scheduling goal, high-value testing tasks (such as critical version regression and core scenario coverage) can be prioritized to be completed on time, effectively avoiding the risk of delay caused by disorderly competition between critical tests and ordinary tasks in the resource queue.

[0049] In step S112, according to the task execution plan, one or more test tasks are bound to at least one matched resource slot and at least one test dataset for execution in batch grouping and execution order.

[0050] Therefore, incorporating algorithm features, dataset features, and testing strategies into the decision-making process can ensure that testing tasks are completed on time, key scenarios are covered first, resource utilization is improved, and queuing latency and data transfer costs are reduced.

[0051] In some embodiments, one or more ternary graphs use edges to represent the requirements and coverage gain of one or more test tasks for one or more test datasets, the data transfer cost of one or more test datasets to one or more resource slots, and the degree to which one or more resource slots satisfy the computing power and environmental dependencies of one or more test tasks.

[0052] In the example, the ternary graph including task T1, dataset D1, and resource slot R1 can include three edges T1—D1, D1—R1, and T1—R1. Edge T1—D1 can represent the requirement and coverage gain of test task T1 for test dataset D1. For example, task T1 needs dataset D1, and the coverage gain for key scenarios is high. Edge D1—R1 can represent the data transfer cost from dataset D1 to resource slot R1. For example, it can represent the reachability and transfer cost of data from the storage domain where D1 is located to resource slot R1. Edge T1—R1 can represent whether resource slot R1 meets the computing power or environmental dependencies of T1.

[0053] In some embodiments, the algorithm feature vector may include the model size, operator dependencies, and computational complexity of the algorithm; the dataset feature vector may include the data size, I / O read / write patterns, and storage location of the dataset; and the test strategy feature vector may include the test type, priority, deadline, and scenario coverage target of the test task.

[0054] Figure 2 This is a flowchart illustrating step S110 of a method 100 for scheduling test tasks according to an exemplary embodiment.

[0055] refer to Figure 2 S110 may include the following steps: S202, constructing a multi-objective loss function by weighted summation of task timeout penalty, coverage gain, resource utilization, queuing delay and data migration cost, wherein the coverage gain is used to represent the degree of coverage of one or more test tasks to the user-preset scenario; and S204, generating a task execution plan by selecting one or more target ternary relation graphs from one or more ternary relation graphs by calculating the multi-objective loss function.

[0056] In some examples, it can be calculated using the formula J = α·DeadlinePenalty + β·QueueDelay γ·Utilization The multi-objective loss function is constructed using η·CoverageGain + λ·DataMoveCost, where J is the multi-objective loss function, DeadlinePenalty can be the timeout penalty for critical versions of tasks, QueueDelay is the queuing latency, Utilization can be the utilization rate of computing resources and bandwidth slots such as CPU / GPU / SPU, CoverageGain can be the coverage gain and critical scenario completion rate, and DataMoveCost can be the cost of fetching across storage domains / data centers or the cost of cache misses (i.e., data migration cost). α, β, γ, η and λ are all preset weights.

[0057] Therefore, by taking coverage gain as one of the scheduling objectives, scheduling prioritizes the completion of key scenarios rather than simply pursuing full computing power utilization; this creates a structural difference from most computing power scheduling approaches that only optimize service metrics or utilization.

[0058] In some embodiments, method 100 may further include monitoring performance metrics while executing one or more test tasks, and dynamically adjusting the task execution plan based on the performance metrics.

[0059] In some embodiments, performance metrics may include task timeout penalties, coverage gain, resource utilization, queuing latency, and data migration costs. For example, when the above performance metrics approach a preset threshold, the task execution plan is adjusted.

[0060] In some embodiments, adjusting the task execution scheme may include at least one of the following: adjusting the batch grouping of one or more test tasks; adjusting the execution order of one or more test tasks; adjusting the cache; performing data migration; and rebinding one or more test tasks to at least one resource slot other than at least one resource slot in one or more resource slots.

[0061] In the example, when critical tasks are approaching deadlines or falling behind in coverage, the priority of critical scenario tasks can be increased and batches reorganized; prefetching and cache residency can be adjusted to reduce task waiting time; and I / O-intensive tasks can be moved to high-bandwidth slots or computationally intensive tasks can be moved to high-computing-power slots.

[0062] Therefore, by continuously monitoring various performance indicators (such as utilization, queuing latency, coverage, etc.) during operation, and dynamically triggering adjustment strategies such as reordering, migration, and queue jumping, the scheduling system has adaptive capabilities and can continuously approach the optimal multi-objective under changing environments and loads, ensuring stable and efficient operation.

[0063] In some embodiments, feasibility assessment may include computing power feasibility, environmental dependency feasibility, bandwidth feasibility, and time-limit feasibility, and cost assessment may include data transfer cost, bandwidth usage penalty, and I / O mode conflict penalty.

[0064] Specifically, computational feasibility refers to whether the computing resources of the resource slots meet the computational requirements of the task; bandwidth feasibility refers to whether the available bandwidth from the resource slots to the dataset storage domain is not less than the I / O throughput required by the task; time-limit feasibility refers to whether, under a given bandwidth, the sum of the data migration time and the task execution time does not exceed the task deadline; data transfer cost is calculated comprehensively based on the dataset size, the actual available bandwidth from the resource slots to the storage domain, and network latency; bandwidth occupancy penalty is calculated based on the proportion of the bandwidth required by the task to the total capacity of the shared link and the current link congestion level; and I / O mode conflict penalty is calculated based on the difference in I / O modes of the datasets executing concurrent tasks, with the penalty value increasing when random reads and sequential reads are concurrent.

[0065] Figure 3 The figure illustrates a specific flow 300 of a method 100 for scheduling test tasks according to an exemplary embodiment.

[0066] refer to Figure 3 The specific process 300 of the method 100 for scheduling test tasks may include the following steps: 301. Obtaining test tasks from the test task access layer (triggered via API / CLI / CI); 302. Obtaining datasets (including dataset size, IO (input / output), labels, and locations) from dataset management or metadata; 303. Calculating algorithm feature vectors, dataset feature vectors, and policy feature vectors based on the obtained test tasks and datasets (the status of computing and bandwidth resources is not shown) through the feature extraction and profiling module; 304. Constructing a task-dataset-resource slot graph, i.e., a ternary relationship graph, through the ternary relationship graph construction module; 305. Performing multi-objective optimization through a multi-objective scheduling engine, including multiple objectives such as deadlines, coverage, utilization, latency, and transport costs, to generate a task execution plan including batch grouping, resource matching, and dataset matching; 306. Resource slot management, where resource slots are a unified model of CPU / SPU / storage / network; 307. Monitoring during task execution, including collecting and feeding back operational metrics and performing adaptive dynamic reordering.

[0067] It should be noted that, Figure 3 The specific process 300 in the example is for illustrative purposes only and not for limiting purposes.

[0068] Figure 4 This is a block diagram illustrating an apparatus 400 for scheduling test tasks according to an exemplary embodiment.

[0069] like Figure 4 As shown, in some embodiments, the apparatus 400 may include: an acquisition module 410, configured to acquire the status of one or more test tasks to be scheduled, one or more test datasets, and computing power and bandwidth resources; a feature calculation module 420, configured to calculate algorithm feature vectors, dataset feature vectors, and test strategy feature vectors based on the status of one or more test tasks, one or more test datasets, and computing power and bandwidth resources; a ternary relationship graph construction module 430, configured to construct one or more ternary relationship graphs between one or more test tasks, one or more test datasets, and one or more resource slots based on the algorithm feature vectors, dataset feature vectors, and test strategy feature vectors, wherein one or more resource slots are mathematical models of computing power and bandwidth resources; and a judgment and evaluation module 440, configured to perform feasibility judgment and cost evaluation on each ternary relationship graph, wherein the feasibility judgment indicates whether the test tasks, resource slots, and datasets in each ternary relationship graph can be... The process involves matching and executing tasks, with a cost assessment indicating the cost of matching and executing test tasks, resource slots, and datasets in each ternary relation graph. A task execution plan generation module 450 is configured to select one or more target ternary relation graphs from one or more ternary relation graphs to generate a task execution plan based on feasibility judgment and cost assessment, while simultaneously reducing costs associated with task time and data migration, increasing coverage of user-preset scenarios, and improving the utilization efficiency of computing power and bandwidth resources. The task execution plan includes batch grouping and execution order of one or more test tasks, and at least one resource slot from one or more resource slots and at least one test dataset from one or more test datasets that match the one or more test tasks. An execution module 460 is configured to bind one or more test tasks to at least one matched resource slot and at least one test dataset according to the task execution plan, for execution in accordance with batch grouping and execution order.

[0070] In some embodiments, the task execution scheme generation module 450 may further include: a first task execution scheme generation submodule, configured to construct a multi-objective loss function by weighted summation of task timeout penalty, coverage gain, resource utilization, queuing latency, and data migration cost, wherein the coverage gain is used to represent the degree of coverage of one or more test tasks to a user-preset scenario; and a second task execution scheme generation submodule, configured to generate a task execution scheme by selecting one or more target ternary graphs from one or more ternary graphs by calculating the multi-objective loss function.

[0071] In some embodiments, the apparatus 400 may further include: a monitoring module configured to monitor performance metrics while executing one or more test tasks; and an adjustment module configured to dynamically adjust the task execution plan based on the performance metrics.

[0072] The operations of the aforementioned acquisition module 410, feature calculation module 420, ternary relation graph construction module 430, judgment and evaluation module 440, task execution plan generation module 450, and execution module 460 can be combined. Figure 1 The operations of steps S102, S104, S106, S108, S110 and S112 are the same, so the details of each aspect will not be repeated here.

[0073] According to one aspect of this disclosure, a computer device is also provided, comprising: at least one processor; and a memory storing a computer program thereon, wherein the computer program, when executed by the at least one processor, causes the at least one processor to perform the steps of any of the method embodiments described above.

[0074] According to one aspect of this disclosure, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, causes the processor to perform the steps of any of the method embodiments described above.

[0075] According to one aspect of this disclosure, a computer program product is also provided, which includes a computer program that, when executed by a processor, implements the steps of any of the method embodiments described above.

[0076] Figure 5 An example computer device 500 is shown in which any of the embodiments described herein may be implemented. The computer device 500 may be used to implement one or more components of the systems and methods described above. The computer device 500 may include a bus 502 or other communication mechanism for communicating information, and one or more processors 504 coupled to the bus 502 for processing information. The processor 504 may be, for example, one or more general-purpose microprocessors.

[0077] Computer device 500 may also include main memory 506, such as random access memory (RAM), cache, and / or other dynamic storage devices, coupled to bus 502, for storing information and instructions to be executed by processor 504. Main memory 506 may also be used to store temporary variables or other intermediate information during the execution of instructions to be executed by processor 504. Such instructions, when stored in a storage medium accessible to processor 504, can make computer device 500 a special-purpose machine customized to perform the operations specified in the instructions. Main memory 506 may include non-volatile media and / or volatile media. Non-volatile media may include, for example, optical discs or magnetic disks. Volatile media may include dynamic memory. Common media formats may include, for example, floppy disks, collapsible disks, hard disks, solid-state drives, magnetic tapes or any other magnetic data storage media, CD-ROMs (read-only optical disc drives), any other optical data storage media, any physical media with a perforated arrangement, RAM (random access memory), DRAM (dynamic random access memory), PROM (programmable read-only memory) and EPROM (erasable programmable read-only memory), FLASH-EPROM (fast erase programmable read-only memory), NVRAM (non-volatile random access memory), any other memory chips or tape cartridges, or network versions of the above.

[0078] Computer device 500 may implement the techniques described herein using custom hardwired logic, one or more ASICs (Application-Specific Integrated Circuits) or FPGAs (Field-Programmable Gate Arrays), firmware, and / or program logic, which, when combined with computer device 500, enable computer device 500 to become a special-purpose machine or to be programmed therein. According to one embodiment, the techniques described herein are executed by computer device 500 in response to processor 504 executing one or more sequences of one or more instructions contained in main memory 506. Such instructions may be read into main memory 506 from another storage medium, such as storage device 508. Executing the sequence of instructions contained in main memory 506 causes processor 504 to perform the processing steps described herein. For example, the processes / methods disclosed herein may be implemented by computer program instructions stored in main memory 506. When these instructions are executed by processor 504, they may perform the steps shown in the corresponding figures and as described above. In alternative embodiments, hardwired circuitry may be used in place of or in combination with software instructions.

[0079] Computer device 500 also includes a network interface 510 coupled to bus 502. Network interface 510 can provide bidirectional data communication coupled to one or more network links connected to one or more networks. As another example, network interface 510 can be a local area network (LAN) card to provide data communication connectivity with a compatible LAN (or a WAN component communicating with a WAN (wide area network)). Wireless links can also be implemented.

[0080] The performance of certain operations can be distributed across processors, not just residing within a single machine, but deployed across many machines. In some exemplary embodiments, the processor or the processor-implemented engine may reside in a single geographic location (e.g., in a home environment, office environment, or server farm). In other exemplary embodiments, the processor or the processor-implemented engine may be distributed across many geographic locations.

[0081] Each process, method, and algorithm described in the preceding sections can be embodied in a code module executed by one or more computer systems or computer processors including computer hardware, and can be fully or partially automated by them. These processes and algorithms can be implemented, in part or in whole, in a specific application circuit.

[0082] When the functions disclosed herein are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Specific technical solutions (all or part) disclosed herein, or aspects contributing to the prior art, can be embodied in the form of a software product. This software product can be stored in a storage medium and includes instructions to cause a computer device (which may be a personal computer, server, network device, etc.) to perform all or part of the steps of the methods described in the embodiments of this application. The storage medium may include a flash drive, a portable hard drive, ROM, RAM, a magnetic disk, an optical disk, another medium suitable for storing program code, or any combination thereof.

[0083] The embodiments disclosed herein can be implemented via a cloud platform, server, or group of servers that interact with a client. The client can be a terminal device or a client registered by a user on the platform, wherein the terminal device can be a mobile terminal, a personal computer (PC), or any device that can install platform applications.

[0084] The various features and processes described above can be used independently or combined in various ways. All possible combinations and sub-combinations are intended to fall within the scope of this disclosure. Furthermore, certain method or process blocks may be omitted in some embodiments. The methods and processes described herein are not limited to any particular order, and associated blocks or states may be executed in other suitable orders. For example, described blocks or states may be executed in a non-specifically disclosed order, or multiple blocks or states may be combined in a single block or state. Exemplary blocks or states may be executed serially, in parallel, or otherwise. Blocks or states may be added to or removed from the disclosed exemplary embodiments. The exemplary systems and components described herein may be configured differently from those described. For example, elements may be added, removed, or rearranged compared to the disclosed exemplary embodiments.

[0085] The various operations of the exemplary methods described herein can be performed at least in part by an algorithm. An algorithm may consist of program code or instructions stored in memory (such as the non-transitory computer-readable storage medium described above). Such an algorithm may include a machine learning algorithm. In some embodiments, the machine learning algorithm may not be explicitly programmed into the computer to perform the function, but may learn from training data to obtain a predictive model for performing that function.

[0086] The various operations of the exemplary methods described herein can be performed at least in part by one or more processors, which are temporarily configured (e.g., by software) or permanently configured to perform the relevant operations. Whether temporarily or permanently configured, such processors can constitute the engine of a processor implementation whose operation is to perform one or more of the operations or functions described herein.

[0087] Similarly, the methods described herein can be implemented at least partially by a processor, where a specific processor or one or more processors are examples of hardware. For example, at least some operations of the methods can be performed by one or more processors or an engine implemented by a processor. Furthermore, one or more processors can also run in a “cloud computing” environment or as “Software as a Service” (SaaS) to support the execution of the relevant operations. For example, at least some operations can be performed by a group of computers (as an example of a machine including processors), which can be accessed via a network (e.g., the Internet) and through one or more appropriate interfaces (e.g., application programming interfaces (APIs)).

[0088] The performance of certain operations can be distributed across processors, not just residing within a single machine, but deployed across many machines. In some exemplary embodiments, the processor or the processor-implemented engine may reside in a single geographic location (e.g., in a home environment, office environment, or server farm). In other exemplary embodiments, the processor or the processor-implemented engine may be distributed across many geographic locations.

[0089] In this specification, multiple instances may implement components, operations, or structures described as a single instance. Although individual operations of one or more methods are described and illustrated as independent operations, one or more individual operations may be performed concurrently, and these operations are not required to be performed in the order shown. Structures and functionalities presented as independent components in the example configuration may be implemented as combined structures or components. Similarly, structures and functionalities presented as individual components may be implemented as independent components. These and other variations, modifications, additions, and improvements are all within the scope of this document.

[0090] As used herein, “or” is inclusive rather than exclusive unless explicitly stated or indicated by context. Furthermore, “and” is both common and individual unless explicitly stated or indicated by context. Moreover, multiple instances may be provided for the resources, operations, or structures described herein as a single example. Furthermore, the boundaries between various resources, operations, engines, and data stores are somewhat arbitrary, and specific operations are illustrated within the context of a particular illustrative configuration. The allocation of other functionalities is conceivable and may fall within the scope of various embodiments of this disclosure. Generally, structures and functionalities presented as independent resources in example configurations may be implemented as combined structures or resources. Similarly, structures and functionalities presented as individual resources may be implemented as independent resources. These and other variations, modifications, additions, and improvements are all within the scope of embodiments of this disclosure. Therefore, this specification and accompanying drawings should be viewed in an illustrative rather than restrictive sense.

[0091] The terms “comprising” or “including” are used to indicate the presence of a subsequently stated feature, but do not preclude the addition of other features. Conditional language, in particular, such as “may,” “can,” or “may,” unless specifically stated or otherwise understood in the context of use, is generally intended to express that certain embodiments include certain features, elements, and / or steps, while other embodiments do not. Therefore, such conditional language generally does not imply that a feature, element, and / or step is necessary in any way for one or more embodiments, or that one or more embodiments must include logic that, with or without user input or prompting, determines whether such features, elements, and / or steps are included in any particular embodiment, or whether they are to be performed in any particular embodiment.

Claims

1. A method for scheduling test tasks, characterized in that, include: Obtain the status of one or more test tasks to be scheduled, one or more test datasets, and computing power and bandwidth resources; Based on the one or more test tasks, the one or more test datasets, and the state of computing power and bandwidth resources, calculate the algorithm feature vector, the dataset feature vector, and the test strategy feature vector; Based on the algorithm feature vector, the dataset feature vector, and the test strategy feature vector, construct one or more ternary relationship graphs between the one or more test tasks, the one or more test datasets, and the one or more resource slots, wherein the one or more resource slots are mathematical models of the computing power and bandwidth resources; For each ternary relation graph, a feasibility judgment and cost assessment are performed. The feasibility judgment indicates whether the test task, resource slot, and dataset in each ternary relation graph can be matched and executed. The cost assessment indicates the cost of matching and executing the test task, resource slot, and dataset in each ternary relation graph. Based on the feasibility assessment and cost evaluation, and with the conditions of simultaneously reducing the costs associated with task time and data migration, increasing the coverage of user-preset scenarios, and improving the utilization efficiency of computing power and bandwidth resources, one or more target ternary relationship graphs from the one or more ternary relationship graphs are selected to generate a task execution plan. The task execution plan includes batch grouping and execution order of the one or more test tasks, and at least one resource slot from the one or more resource slots matching the one or more test tasks, and at least one test dataset from the one or more test datasets; and According to the task execution plan, the one or more test tasks are bound to the matched at least one resource slot and the at least one test dataset for execution in accordance with the batch grouping and the execution order.

2. The method according to claim 1, characterized in that, The one or more ternary relationship graphs use edges to represent the one or more test tasks' requirements and coverage gain for the one or more test datasets, the data transfer cost from the one or more test datasets to the one or more resource slots, and the degree to which the one or more resource slots satisfy the computing power and environmental dependencies of the one or more test tasks.

3. The method according to claim 1, characterized in that, The algorithm feature vector includes the algorithm's model size, operator dependencies, and computational complexity; the dataset feature vector includes the dataset's data size, I / O read / write patterns, and storage location; and the test strategy feature vector includes the test type, priority, deadline, and scenario coverage target of the test task.

4. The method according to claim 1, characterized in that, To generate a task execution plan by selecting one or more target ternary graphs from the one or more ternary graphs, while simultaneously reducing the costs associated with task time and data migration and increasing the coverage of user-preset scenarios and the utilization efficiency of computing power and bandwidth resources, the following methods are employed: A multi-objective loss function is constructed by weighting and summing task timeout penalties, coverage gain, resource utilization, queuing latency, and data migration costs, wherein the coverage gain represents the degree of coverage of the one or more test tasks to the user-preset scenario; and The task execution plan is generated by selecting one or more target ternary graphs from the one or more ternary graphs by calculating the multi-objective loss function.

5. The method according to claim 4, characterized in that, The method also includes: Monitor performance metrics while executing the one or more test tasks; and The task execution plan is dynamically adjusted based on the aforementioned operational indicators.

6. The method according to claim 5, characterized in that, The operational metrics include the task timeout penalty, the coverage gain, the resource utilization, the queuing latency, and the data migration cost.

7. The method according to claim 5, characterized in that, The adjustment of the task execution plan includes at least one of the following: Adjust the batch grouping of the one or more test tasks; Adjust the execution order of the one or more test tasks; Adjust the cache; Perform data migration: and Rebind the one or more test tasks to at least one other resource slot in the one or more resource slots, other than the at least one resource slot.

8. The method according to claim 1, characterized in that, The feasibility assessment includes computing power feasibility, environmental dependency feasibility, bandwidth feasibility, and time-limit feasibility. The cost assessment includes data transfer cost, bandwidth usage penalty, and I / O mode conflict penalty.

9. An apparatus for scheduling test tasks, characterized in that, The device includes: The acquisition module is configured to acquire the status of one or more test tasks to be scheduled, one or more test datasets, and computing power and bandwidth resources. The feature calculation module is configured to calculate algorithm feature vectors, dataset feature vectors, and test strategy feature vectors based on the state of the one or more test tasks, the one or more test datasets, and the computing power and bandwidth resources. The ternary relationship graph construction module is configured to construct one or more ternary relationship graphs between the one or more test tasks, the one or more test datasets, and the one or more resource slots based on the algorithm feature vector, the dataset feature vector, and the test strategy feature vector, wherein the one or more resource slots are mathematical models of the computing power and bandwidth resources; The judgment and evaluation module is configured to perform feasibility judgment and cost evaluation for each ternary relation graph. The feasibility judgment indicates whether the test task, resource slot and dataset in each ternary relation graph can be matched and executed. The cost evaluation indicates the cost of matching and executing the test task, resource slot and dataset in each ternary relation graph. The task execution plan generation module is configured to generate a task execution plan based on the feasibility assessment and cost evaluation, while simultaneously reducing costs associated with task time and data migration, increasing coverage of user-preset scenarios, and improving the utilization efficiency of computing power and bandwidth resources. The task execution plan includes batch grouping and execution order of the one or more test tasks, and at least one resource slot from the one or more resource slots matching the one or more test tasks, and at least one test dataset from the one or more test datasets. The execution module is configured to bind one or more test tasks to the matched at least one resource slot and at least one test dataset according to the task execution scheme, so as to execute them in accordance with the batch grouping and the execution order.

10. A computer device, characterized in that, The computer device includes: At least one processor; A memory having a computer program stored thereon, wherein, when executed by the at least one processor, the computer program causes the at least one processor to perform the method of any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to perform the method of any one of claims 1-8.

12. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, causes the processor to perform the method of any one of claims 1-8.

Images