A GPU resource scheduling method based on reinforcement learning

Abstract

The application discloses a GPU resource scheduling method based on reinforcement learning, relates to the technical field of computer resource scheduling and high-performance computing, and comprises the following steps: obtaining all task information in a current task queue, including historical execution records and expected completion time, obtaining task demand difference data, performing historical behavior analysis on the task demand difference data, obtaining occupancy duration characteristics of various tasks, constructing a scheduling state vector based on the occupancy duration characteristics, inputting the scheduling state vector into a reinforcement learning scheduling model, and obtaining task continuous occupancy demand intensity and GPU binding relationship establishment basis; the GPU resource scheduling method based on reinforcement learning effectively improves overall GPU utilization, shortens average task waiting time, reduces resource idling and frequent switching overhead caused by unreasonable binding, and enhances stability and global optimization capability of the scheduling strategy in a complex multi-tenant environment.

Description

Technical Field

[0001] This invention relates to the field of computer resource scheduling and high-performance computing technology, specifically to a GPU resource scheduling method based on reinforcement learning. Background Technology

[0002] In multi-tenant high-performance computing and artificial intelligence training clusters, GPUs serve as core computing resources, and their allocation directly impacts task execution efficiency and overall system throughput. Current mainstream GPU scheduling systems typically allocate resources based on static rules, priority queues, or simple prediction mechanisms, determining the GPU allocation scheme according to task submission order, resource availability, or historical statistics.

[0003] However, in real-world operating environments, different types of tasks exhibit significant differences in their execution behavior. Some tasks are characterized by long-term continuous GPU occupancy, while others exhibit short-term bursts or intermittent resource demands. Existing scheduling methods typically rely on fixed thresholds or empirical rules to determine whether to establish a GPU binding relationship for a task. However, due to the lack of a systematic characterization of the dynamic evolution of task behavior, it is difficult to accurately reflect the changing trends in the intensity of continuous task occupancy, leading to unreasonable timing of binding. Furthermore, GPU resource states have strong temporal characteristics, with a coupling relationship between their current occupancy state and future release time. Existing scheduling methods often make decisions based solely on the current resource snapshot during resource allocation, lacking comprehensive modeling of future resource release trends, making it difficult to form stable resource allocation strategies under complex load environments. When multiple tasks compete for the same batch of GPU resources over time, conflicting task sets are easily generated. Existing technologies typically handle this through fixed priorities or simple sorting mechanisms, lacking adaptive optimization capabilities based on the overall system performance. Moreover, after the GPU binding relationship is established, the system often lacks a continuous evaluation and dynamic adjustment mechanism. When resource utilization decreases or task waiting time increases, the scheduling strategy cannot self-correct based on the actual operating effect, resulting in resource allocation being in a suboptimal state for a long time. Summary of the Invention

[0004] The purpose of this invention is to provide a GPU resource scheduling method based on reinforcement learning, thereby solving the problems existing in the prior art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a GPU resource scheduling method based on reinforcement learning, comprising: S1, acquiring all task information in the current task queue, including historical execution records and expected completion times, to obtain task demand difference data; S2, performing historical behavior analysis on the task demand difference data to obtain the occupancy duration characteristics of various tasks; S3, constructing a scheduling state vector based on the occupancy duration characteristics and inputting it into a reinforcement learning scheduling model to obtain the intensity of continuous task occupancy demand and the basis for establishing GPU binding relationships; S4, acquiring the current GPU occupancy state and future release time prediction to obtain resource release prediction results, which are used as the environment state input of the reinforcement learning scheduling model; S5 If the continuous demand intensity exceeds a preset threshold, the reinforcement learning scheduling model outputs a binding timing selection scheme and a binding relationship maintenance period; S6. Based on the binding relationship maintenance period and resource release prediction results, and combined with the output of the reinforcement learning scheduling model, a set of potential conflicting tasks is calculated to obtain a conflict priority list; S7. The conflict priority list is sorted by task queue to obtain a preemption decision process sequence; S8. The resource allocation state is adjusted according to the preemption decision process sequence to obtain the updated GPU binding relationship state; S9. The updated GPU binding relationship state is monitored, and a reward signal is generated based on resource utilization and task waiting time, which is fed back to the reinforcement learning scheduling model for policy update and the next round of loop processing.

[0006] Preferably, step S1 includes reading task queue metadata to obtain a task scheduling snapshot, and retrieving a historical feature set based on the task scheduling snapshot; calculating the estimated execution time using the historical feature set, and comparing the estimated execution time with the preset deadline in the task scheduling snapshot to generate a time deviation value; if the time deviation value exceeds a threshold, determining a resource scarcity index based on the resource demand matrix, and constructing a demand difference vector based on the resource scarcity index; aggregating the demand difference vector to generate a final dataset to obtain task demand difference data.

[0007] Preferably, step S2 includes indexing historical behavior logs based on task demand difference data, and extracting historical time series segments based on resource scarcity indexes in the task demand difference data; performing waveform decomposition on the historical time series segments to obtain a resource occupancy waveform set; if the steady-state occupancy period in the resource occupancy waveform set meets the stability criterion, then inputting it into a behavior pattern clustering model to obtain category feature vectors; using the category feature vectors to construct a feature mapping matrix, and parsing the time dimension data in the feature mapping matrix to generate occupancy duration features for various tasks.

[0008] Preferably, step S3 includes acquiring the duration characteristics of various tasks to construct a scheduling state vector; inputting the scheduling state vector into a reinforcement learning scheduling model, and combining it with the computing power topology to obtain the continuous occupancy demand intensity of tasks; if the continuous occupancy demand intensity of tasks is higher than a threshold, then constructing a resource contention graph; calculating the affinity probability distribution based on the resource contention graph, and generating a basis for establishing GPU binding relationships based on the affinity probability distribution by the reinforcement learning scheduling model.

[0009] Preferably, step S4 includes obtaining the memory utilization and computing load values ​​of cluster nodes, constructing a real-time resource snapshot based on the node topology graph; extracting process lifecycle features based on the real-time resource snapshot, and matching historical task patterns with historical running records; calculating future release time points based on historical task patterns and process lifecycle features, and generating a resource release schedule; aligning the resource release schedule with the memory fragmentation rate and transmission bandwidth values ​​to generate a composite resource state matrix; and mapping the composite resource state matrix to obtain a high-dimensional feature vector, which serves as the environment state input for the reinforcement learning scheduling model.

[0010] Preferably, step S5 includes obtaining the continuous occupancy time segments of the task to be scheduled, calculating the continuous occupancy demand intensity value based on the continuous occupancy time segments; if the continuous occupancy demand intensity value is higher than a preset threshold, generating a multi-dimensional state feature vector by combining resource contention rate and node affinity data; inputting the multi-dimensional state feature vector into the reinforcement learning scheduling model, and having the reinforcement learning scheduling model output a scheduling decision vector; performing action space mapping analysis on the scheduling decision vector, and outputting a binding timing selection scheme and binding relationship maintenance period.

[0011] Preferably, step S6 includes obtaining the scheduling decision vector output by the reinforcement learning scheduling model and a preset binding relationship maintenance period; determining the resource occupation time window of the task to be scheduled based on the scheduling decision vector and the binding relationship maintenance period; performing time-series alignment and comparison between the resource occupation time window and the resource release prediction result; selecting task combinations that meet preset conflict conditions to construct a potential conflict task set; obtaining the task urgency and conflict impact range data of each task in the potential conflict task set, and performing weighted calculation to obtain a priority weight value; and sorting the potential conflict task set according to the priority weight value to obtain a conflict priority table.

[0012] Preferably, step S7 includes acquiring a conflict priority list and task queue data, generating an initial sorted task sequence based on priority weight values; traversing the initial sorted task sequence to identify resource fragment locations that can be preempted, and calculating the preemption cost value of tasks associated with resource fragment locations; if the preemption cost value is lower than the expected delay loss value, generating a scheduling instruction set containing preemption actions, and arranging the tasks in the initial sorted task sequence in a time sequence based on the scheduling instruction set to obtain a preemption decision process sequence.

[0013] Preferably, step S8 includes receiving a preemption decision process sequence, parsing the target device handle according to the preemption decision process sequence, reading the video memory address mapping table of the target device handle to locate the physical slot; sending an interrupt instruction for the physical slot, serializing the process context to the virtual address space, and releasing the physical slot from locking the video memory page table; filtering the target computing unit set according to topology affinity, aggregating the video memory fragments of the target computing unit set to construct a usable video memory block; loading priority tasks into the computing unit corresponding to the usable video memory block and updating the resource registry to obtain the updated GPU binding relationship status.

[0014] Preferably, step S9 includes obtaining the updated GPU binding relationship state and a multi-dimensional state observation vector, wherein the multi-dimensional state observation vector is composed of memory occupancy, computing power utilization, and task queuing time; generating a scalar reward signal value based on the multi-dimensional state observation vector, and associating the scalar reward signal value with the corresponding binding state snapshot to construct a state transition tuple; calculating gradient descent parameters based on the state transition tuple, and using the gradient descent parameters to update the policy network weights of the reinforcement learning scheduling model for the next round of loop processing.

[0015] As can be seen from the above technical solution, the present invention has the following beneficial effects: This reinforcement learning-based GPU resource scheduling method constructs a scheduling state vector containing data on task demand differences, duration of occupancy, and predictions of the current GPU occupancy status and future release time. It then introduces a reinforcement learning scheduling model to adaptively decide on the intensity of continuous task occupancy demands, GPU binding timing, and maintenance cycle. This transforms resource allocation from a traditional static rule-driven approach to a strategy optimization method based on dynamic evolution of the environment state. Simultaneously, by adaptively calculating the priority of potentially conflicting task sets and generating a preemption decision sequence, it achieves dynamic coordination of resource conflicts in multi-task competition scenarios. This effectively improves overall GPU utilization, shortens average task waiting time, reduces resource idleness and frequent switching overhead caused by unreasonable binding, and enhances the stability and global optimization capability of the scheduling strategy in complex multi-tenant environments. Attached Figure Description

[0016] Figure 1 This is a flowchart of the GPU resource scheduling method of the present invention; Figure 2 This is a flowchart of the GPU resource release prediction and reinforcement learning environment state construction process of the present invention. Figure 3 This is a diagram illustrating the task conflict analysis and preemption decision generation process of this invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Example 1:

[0018] like Figure 1 and Figure 3 As shown, the present invention provides a technical solution: a GPU resource scheduling method based on reinforcement learning, comprising: S1. Obtain all task information in the current task queue, including historical execution records and expected completion times, to obtain task requirement difference data; S2. Perform historical behavior analysis on the data of different task requirements to obtain the duration characteristics of various tasks. S3. Construct a scheduling state vector based on the duration of task occupancy and input it into the reinforcement learning scheduling model to obtain the intensity of continuous task occupancy demand and the basis for establishing GPU binding relationship. S4. Obtain the current GPU occupancy status and future release time prediction to obtain the resource release prediction result, and use it as the environment state input for the reinforcement learning scheduling model; S5. If the intensity of continuous occupancy demand exceeds the preset threshold, the reinforcement learning scheduling model will output the binding timing selection scheme and the binding relationship maintenance period. S6. Based on the binding relationship maintenance period and resource release prediction results, and combined with the reinforcement learning scheduling model, calculate the set of potential conflict tasks to obtain the conflict priority list. S7. Sort the task queue of the conflict priority list to obtain the preemption decision process sequence; S8. Adjust the resource allocation status according to the preemption decision process sequence to obtain the updated GPU binding relationship status; S9. Monitor the updated GPU binding relationship status and generate reward signals based on resource utilization and task waiting time. Feedback these signals to the reinforcement learning scheduling model for policy updates and the next round of processing.

[0019] In this embodiment, specifically, during the GPU resource scheduling process, all task information in the current task queue is first obtained, including task historical execution records and expected completion times.

[0020] The task history execution record is used to record the GPU resource usage status, task execution duration, and task completion status of the task during the historical execution process; the expected completion time is used to indicate the completion time requirement set when the task is submitted; by processing the data of various task information in the task queue, the resource requirement differences between different tasks are analyzed to obtain task requirement difference data, which provides data input for subsequent task behavior analysis.

[0021] Furthermore, based on the differences in task requirements, the historical execution records of tasks are analyzed to obtain the duration characteristics of different types of tasks.

[0022] Specifically, the duration of resource occupation is used to represent the time parameter during which a task continuously occupies GPU resources during execution. By statistically processing the task running time, GPU occupation time, and task completion time in historical execution records, a correspondence between task type and resource occupation duration is established, enabling the scheduling process to determine the task resource occupation characteristics based on the task's historical behavior.

[0023] Furthermore, after obtaining the occupancy duration feature, a scheduling state vector is constructed based on the occupancy duration feature.

[0024] Specifically, the scheduling state vector is used to describe the current task state and GPU resource state, including task resource requirement information, task execution state information, task occupancy duration information, and GPU resource occupancy information. The constructed scheduling state vector is input into the reinforcement learning scheduling model, which calculates based on the current state information and outputs the task continuous occupancy requirement intensity and the basis for establishing the GPU binding relationship. Among them, the task continuous occupancy requirement intensity is used to determine whether the task needs to maintain a continuous GPU resource occupancy state, and the basis for establishing the GPU binding relationship is used to determine the binding relationship between the task and the GPU resource.

[0025] At the same time, it obtains the current GPU usage status and future release time prediction information.

[0026] Specifically, the current GPU occupancy status includes the current GPU device running status, task occupancy, and resource usage; the future release time prediction is calculated based on the current running task execution progress and historical task execution data to obtain the resource release prediction result; the resource release prediction result is used as the environment state input of the reinforcement learning scheduling model, so that the reinforcement learning scheduling model considers both the current GPU resource status and the predicted resource release status when generating the scheduling policy.

[0027] Specifically, when the intensity of continuous occupancy demand output by the reinforcement learning scheduling model exceeds a preset threshold, the binding timing selection scheme and the binding relationship maintenance period are determined based on the output of the reinforcement learning scheduling model.

[0028] The binding timing selection scheme is used to determine the time node for establishing a GPU binding relationship between a task and the GPU resource, and the binding relationship maintenance period is used to determine the duration range for which the task and GPU resource maintain a binding relationship, so that the task execution process maintains the corresponding resource access relationship according to the scheduling policy.

[0029] Furthermore, based on the binding relationship maintenance period and resource release prediction results, the GPU resource occupancy status and task execution status are analyzed to identify tasks with resource competition relationships under the current resource allocation status, forming a set of potential conflicting tasks.

[0030] Based on the output of the reinforcement learning scheduling model and the task attribute information, the priorities of tasks in the potential conflict task set are calculated to obtain a conflict priority list; the conflict priority list is used to represent the processing order of different conflict tasks in the resource adjustment process.

[0031] In addition, the task queue is sorted according to the conflict priority list to generate a preemption decision process sequence.

[0032] The preemption decision process sequence is used to determine the order of task resource adjustments, including the execution order of task resource retention, task resource release, and task resource reallocation. The GPU resource allocation state is adjusted according to the preemption decision process sequence, and the binding relationship between tasks and GPU resources is updated to obtain the updated GPU binding relationship state.

[0033] Furthermore, after the GPU binding relationship is updated, the updated GPU binding relationship status is monitored in real time to obtain GPU resource utilization, task waiting time, and task execution status information.

[0034] Specifically, a reward signal is generated based on the acquired running status information and fed back to the reinforcement learning scheduling model. The reinforcement learning scheduling model updates the current scheduling strategy based on the reward signal, enabling the model to adjust the subsequent scheduling process according to different task queue states and GPU resource states, and enter the next round of GPU resource scheduling loop. Through the above process, dynamic scheduling of GPU resources based on task historical behavior analysis, resource state prediction, and reinforcement learning strategy optimization is achieved. Example 2:

[0035] S1 includes reading task queue metadata to obtain a task scheduling snapshot, and retrieving a historical feature set based on the task scheduling snapshot; calculating the estimated execution time using the historical feature set, and comparing the estimated execution time with the preset deadline in the task scheduling snapshot to generate a time deviation value; if the time deviation value exceeds a threshold, determining a resource scarcity index based on the resource demand matrix, and constructing a demand difference vector based on the resource scarcity index; aggregating the demand difference vector to generate the final dataset to obtain task demand difference data.

[0036] This embodiment first reads the task metadata in the current task queue. The task queue metadata includes the task number, task type, task submission time, the amount of GPU computing resources requested by the task, the amount of video memory required, the preset deadline, and the current execution status.

[0037] A task scheduling snapshot is generated based on the read task metadata. The task scheduling snapshot is used to save the task status information in the task queue at the current scheduling moment.

[0038] Furthermore, after generating the task scheduling snapshot, the corresponding historical feature set is retrieved from the historical task execution records based on the task number, task type, and resource requirement information in the task scheduling snapshot.

[0039] The specific retrieval process is as follows: using the task number and task type of the current task as query conditions, searching for historical task records with corresponding relationships in the historical execution database, and extracting the execution duration, GPU usage time, video memory usage, computing resource usage, and task completion time generated during the execution of historical tasks to form a historical feature set.

[0040] When calculating the estimated execution time using the historical feature set, the historical execution records in the historical feature set are first filtered to remove historical records that differ from the current task's resource requirements by more than a preset range.

[0041] Subsequently, the execution time corresponding to each record is determined based on the retained historical execution records, and the calculation weight is assigned according to the matching degree between the historical records and the current task in terms of task type, resource requirements, and execution environment; the matching degree is determined by comparing the resource request amount, GPU usage status, and task execution type of the historical task and the current task.

[0042] Furthermore, after completing the weight allocation, the execution time in each historical execution record is multiplied by the corresponding weight, and then all the calculation results are summed to obtain the estimated execution time of the task.

[0043] The above calculation process ensures that the estimated execution time takes into account both historical execution time and current task resource requirements.

[0044] Furthermore, after obtaining the estimated execution time, the estimated execution time is compared with the preset deadline in the task scheduling snapshot to generate a time deviation value.

[0045] Specifically, the time difference between the estimated completion time and the preset deadline corresponding to the estimated execution time is calculated; when the estimated completion time is later than the preset deadline, the time difference between the two is taken as a positive time deviation value; when the estimated completion time is earlier than the preset deadline, the time difference between the two is taken as a negative time deviation value.

[0046] When the time deviation exceeds a preset threshold, a resource scarcity index is determined based on the resource demand matrix.

[0047] The preset threshold is determined based on the time deviation data in the historical task scheduling process. The specific determination process is as follows: obtain the time deviation records of tasks in multiple historical scheduling cycles, statistically analyze the task records that have resource reallocation, increased task waiting time, or failure to meet the deadline requirements during the task completion process, use the corresponding time deviation values ​​as candidate threshold ranges, and select the time deviation values ​​that can trigger resource adjustment operations as the preset threshold.

[0048] In this embodiment, the resource requirement matrix is ​​used to record the task's requirements for GPU computing resources, video memory resources, and other scheduling resources.

[0049] Specifically, based on the resource demand matrix corresponding to the current task, each item is compared with the current resource availability of the GPU, and the difference between the resource demand and available resources is calculated. When the resource demand exceeds the available resources, the corresponding resource category is recorded as a resource scarcity item, and the resource scarcity index is determined based on the difference between the resource demand and available resources.

[0050] Furthermore, a demand difference vector is constructed based on the resource scarcity index.

[0051] The specific process involves arranging the resource category, task resource requirement value, current available resource value, and the difference between the two corresponding to the resource scarcity index in a preset order to form a demand difference vector. This demand difference vector is used to represent the difference between the resources required during the current task execution and the resources that can be allocated by the GPU.

[0052] Finally, the requirement difference vectors corresponding to each task in the task queue are aggregated.

[0053] Specifically, the demand difference vectors are stored according to the task number, and the time deviation values, resource scarcity indexes, and resource demand difference parameters are combined to form the final dataset, thus obtaining the task demand difference data.

[0054] Through the above process, the task requirement difference data acquisition process in S1 clearly defines the specific implementation methods for reading task queue metadata, retrieving historical features, calculating estimated execution time, judging time deviation, determining resource scarcity index, and generating requirement difference vectors, so that the generated task requirement difference data can reflect the task execution time status and GPU resource requirement status. Example 3:

[0055] S2 includes indexing historical behavior logs based on task demand difference data, and extracting historical time series segments based on resource scarcity index in the task demand difference data; performing waveform decomposition on the historical time series segments to obtain a resource occupancy waveform set; if the steady-state occupancy period in the resource occupancy waveform set meets the stability criterion, then inputting it into the behavior pattern clustering model to obtain category feature vectors; using the category feature vectors to construct a feature mapping matrix, and parsing the time dimension data in the feature mapping matrix to generate occupancy duration features for various tasks.

[0056] This embodiment first indexes historical behavior logs based on task requirement difference data. The task requirement difference data includes task identification information, resource requirement parameters, and resource scarcity index.

[0057] Specifically, the storage location of the historical execution record of the corresponding task is determined based on the task identification information, and the type of resource to be analyzed is determined based on the resource scarcity index; for example, when the resource scarcity index corresponds to GPU computing resources, the GPU computing resource usage record is read from the historical behavior log; when the resource scarcity index corresponds to video memory resources, the video memory usage change record is read.

[0058] Furthermore, after obtaining the historical behavior logs for the corresponding resource type, historical time series segments are extracted based on the resource scarcity index.

[0059] The specific process is as follows: First, determine the complete execution time range based on the task start time and task end time in the historical behavior log. Then, read the resource usage data recorded in chronological order within this time range and arrange them according to a fixed sampling interval to form a historical time series segment. The historical time series segment includes resource usage values ​​corresponding to multiple consecutive sampling moments, which are used to describe the change process of resource usage status during task execution.

[0060] Furthermore, waveform decomposition is performed on historical time series segments to obtain a set of resource occupancy waveforms.

[0061] The specific process is as follows: the continuous resource occupancy values ​​in the historical time series segment are divided into multiple change intervals according to the time sequence, and the resource occupancy change trend in each change interval is analyzed; when the direction of resource occupancy change between consecutive time nodes is consistent, the corresponding time nodes are assigned to the same change waveform; when the resource occupancy status changes, a new change waveform is defined; through the above method, a set of resource occupancy waveforms consisting of multiple resource occupancy change stages is obtained.

[0062] Furthermore, after obtaining the set of resource occupancy waveforms, the steady-state occupancy period is determined.

[0063] The specific judgment process is as follows: obtain the resource occupation difference between adjacent sampling times in each resource occupation waveform, and count the length of time during which the resource occupation difference is within the preset range of change within consecutive sampling times; when the continuous time length reaches the duration requirement corresponding to the stability standard, the time interval is determined as the steady-state occupation period.

[0064] The stability criteria are determined based on historical mission execution data.

[0065] The specific determination process is as follows: collect resource usage waveforms of similar tasks in multiple historical execution processes, count the time interval during which resource usage remains in a continuous state during task execution, and set judgment conditions based on the duration range of the stable resource usage phase during the completion of historical tasks; when the steady-state usage cycle in the resource usage waveform set meets the judgment condition, it indicates that the resource usage waveform can reflect the behavioral characteristics of the task's stable resource usage, and input the corresponding resource usage waveform into the behavior pattern clustering model.

[0066] In this embodiment, the behavior pattern clustering model performs feature extraction and classification processing on the input resource occupancy waveform.

[0067] The specific process is as follows: extract the duration of resource occupation, the number of resource occupation changes, the resource release time node, and the duration of the stable occupation phase from the resource occupation waveform. Combine the above parameters in a preset order to form behavioral feature data and input them into the behavioral pattern clustering model. The behavioral pattern clustering model classifies tasks according to the similarity between different task behavioral features, divides tasks with the same resource occupation change pattern into the same category, and outputs the category feature vector of the corresponding category.

[0068] Subsequently, a feature mapping matrix is ​​constructed using the category feature vectors.

[0069] The specific process is as follows: Arrange the category feature vectors corresponding to each task category according to the task category number, and map the resource consumption features, execution time features, and resource release features in the category feature vectors to the corresponding positions in the matrix to form a feature mapping matrix; record the correspondence between task categories and resource consumption behavior through the feature mapping matrix.

[0070] Furthermore, the time dimension data in the feature mapping matrix is ​​analyzed.

[0071] The specific process is as follows: Read the time-related data items in the feature mapping matrix, extract the time node when the task starts occupying GPU resources, the time interval during which the resource occupancy is maintained, and the time node corresponding to the resource release. Determine the duration of task occupancy based on the time interval between the resource release time node and the resource start time node.

[0072] Finally, the tasks are categorized and organized according to their duration of occupation, generating duration characteristics for each type of task. These duration characteristics represent the time parameters during which different types of tasks maintain GPU resource occupation during execution. Example 4:

[0073] S3 includes acquiring the duration characteristics of various tasks to construct a scheduling state vector; inputting the scheduling state vector into a reinforcement learning scheduling model, and combining it with the computing power topology to obtain the intensity of continuous task occupancy demand; if the intensity of continuous task occupancy demand is higher than a threshold, then constructing a resource contention graph; calculating the affinity probability distribution based on the resource contention graph, and using the reinforcement learning scheduling model to generate the basis for establishing GPU binding relationships based on the affinity probability distribution.

[0074] This embodiment first obtains the duration characteristics of various tasks obtained in S2.

[0075] Specifically, the duration of resource occupation features include the continuous occupation time corresponding to the task category, the resource release time, the task execution stage, and the historical resource occupation status. A scheduling state vector is constructed based on these features.

[0076] Specifically, the scheduling state vector construction process is as follows: according to the preset data arrangement rules, the duration of occupation, resource requirements, current execution status, preset completion time, and GPU resource occupation status corresponding to each task are combined.

[0077] Among them, the task occupation duration is used to indicate the time information of the task maintaining GPU resource occupation; the resource requirement is used to indicate the amount of GPU computing resources and video memory resources required by the task; the GPU resource occupation status is used to indicate the available resources of the current GPU node; after arranging the above parameters in a fixed order, a scheduling status vector of the corresponding task is formed.

[0078] The scheduling state vector is input into the reinforcement learning scheduling model. After receiving the scheduling state vector, the reinforcement learning scheduling model processes the task state features and calculates the intensity of continuous task occupancy demand in combination with the computing power topology.

[0079] The computing topology includes the data transmission connection relationship between GPU nodes, the distribution of computing units, the memory access relationship, and the current load status of GPU nodes.

[0080] The specific processing steps are as follows: First, determine the type of GPU computing resources that the task needs to access based on the task resource requirement information. Then, obtain candidate GPU nodes that meet the task's computing requirements based on the computing power topology. Calculate the matching degree between the task and each candidate GPU node.

[0081] In calculating the matching degree, the task's computing resource requirements, the amount of available computing resources on the GPU node, the task's data transmission distance, and the duration of GPU node occupation during the historical task execution are used as calculation parameters.

[0082] Specifically, resource matching parameters are determined based on the difference between the task's computational resource requirements and the available computational resources of the GPU node; transmission overhead parameters are determined based on the task's data transmission distance; and continuous occupancy adaptation parameters are determined based on historical task execution data. The above parameters are then weighted according to preset weights to obtain the state evaluation value between the task and the candidate GPU node.

[0083] In this embodiment, the reinforcement learning scheduling model calculates the degree of need for the current task to maintain continuous GPU resource usage based on the state evaluation value, and outputs the task continuous usage demand intensity; the task continuous usage demand intensity is used to represent the degree of need for the task to maintain the same GPU resource binding relationship during execution.

[0084] Specifically, when the continuous demand intensity of a task exceeds a preset threshold, a resource contention graph is constructed.

[0085] The preset threshold is determined based on historical task scheduling results. The specific process is as follows: acquire the continuous task demand intensity data corresponding to the historical task execution process, and record the number of task switching, resource waiting time and task completion time under different demand intensity conditions; statistically analyze the historical data of task execution status improvement after meeting resource binding conditions, and determine the judgment value in the corresponding task continuous demand intensity range as the preset threshold.

[0086] Furthermore, after the threshold judgment condition is met, a resource competition graph is constructed based on the resource demand relationship of the tasks in the current task queue.

[0087] The specific process is as follows: tasks are treated as graph nodes, and the GPU resource competition relationship between two tasks is used as a connection relationship; when determining whether there is a competition relationship between tasks, the amount of GPU computing resources requested by the task, the amount of video memory required, and the current GPU resource usage are compared; when two tasks have the same GPU resource requirements, and the currently available GPU resources cannot satisfy the resource requests of the two tasks at the same time, a connection relationship is established between the corresponding task nodes.

[0088] Furthermore, after establishing the connection relationship between task nodes, the connection weight is calculated based on the difference in task resource requirements and resource occupancy status.

[0089] The specific process is as follows: calculate the difference between the sum of the resource requirements of the two tasks and the current available GPU resources, and determine the competition impact parameters by combining the overlapping interval of the execution time of the two tasks. Based on the competition impact parameters, set the connection weights between task nodes to form a resource competition graph.

[0090] Calculate the affinity probability distribution based on the resource competition map.

[0091] The specific process is as follows: For each task node, calculate its affinity score with the candidate GPU node. The affinity score calculation parameters include the matching relationship between the computing resources required by the task and the available computing resources of the GPU, the matching relationship between the task's memory requirements and the GPU's memory capacity, the task's data access path, and the binding status of the task with the GPU node during its historical execution.

[0092] Furthermore, after calculating the affinity score for each candidate GPU node, the score results for all candidate GPU nodes are normalized.

[0093] The specific process is as follows: divide the affinity score corresponding to a single GPU node by the sum of the affinity scores of all candidate GPU nodes to obtain the affinity probability value corresponding to that GPU node; generate the affinity probability distribution based on the probability values ​​corresponding to all GPU nodes.

[0094] Finally, the reinforcement learning scheduling model generates the basis for establishing GPU binding relationships based on the affinity probability distribution.

[0095] The specific process is as follows: the affinity probability distribution is input into the policy output layer of the reinforcement learning scheduling model, the corresponding GPU resource node is selected according to the probability distribution result, and the determination of whether to establish a GPU binding relationship is made in combination with the intensity of continuous task occupancy demand; when it is determined to establish a binding relationship, the binding target is generated according to the GPU resource node with the highest affinity probability, and the basis for establishing the corresponding GPU binding relationship is output.

[0096] Through the above processing, the task execution features in the scheduling state vector can be converted into state information that can be processed by the reinforcement learning scheduling model. Based on the GPU computing power topology and task resource competition relationship, the intensity of continuous task occupancy demand and the basis for establishing GPU binding relationship are calculated, providing a decision basis for the subsequent GPU resource scheduling process. Example 5:

[0097] S4 includes obtaining the memory utilization and computing load values ​​of cluster nodes, constructing a real-time resource snapshot based on the node topology graph; extracting process lifecycle features based on the real-time resource snapshot, and matching historical task patterns with historical running records; calculating future release time points based on historical task patterns and process lifecycle features to generate a resource release schedule; aligning the resource release schedule with memory fragmentation rate and transmission bandwidth values ​​to generate a composite resource state matrix; and mapping the composite resource state matrix to obtain a high-dimensional feature vector, which serves as the environment state input for the reinforcement learning scheduling model.

[0098] In this embodiment, the memory utilization and computing load of the cluster nodes are first obtained, and a real-time resource snapshot is constructed in combination with the node topology graph.

[0099] During the specific acquisition process, the GPU monitoring interface is used to read the used memory capacity, total memory capacity, computing core occupancy status, and number of currently running tasks of each cluster node.

[0100] The process of obtaining the video memory utilization rate is as follows: read the video memory capacity currently occupied by the node, compare the capacity with the total video memory capacity configured for the node, and determine the video memory utilization rate of the current node based on the ratio between the two. The process of obtaining the computing load value is as follows: collect the time percentage of the GPU computing core executing tasks within the current time period, and determine the node's computing resource occupancy status in combination with the number of currently running tasks.

[0101] Furthermore, after obtaining the memory utilization and computing load values ​​of each GPU node, a node topology graph is generated based on the data connection relationships between cluster nodes.

[0102] Specifically, the node topology graph includes GPU node numbers, node connection relationships, inter-node transmission paths, and corresponding transmission bandwidth information; the memory utilization, computing load value, and node topology relationships of each node are associated and stored to form a real-time resource snapshot; the real-time resource snapshot is used to record the cluster GPU resource status within the current scheduling cycle.

[0103] Furthermore, process lifecycle characteristics are extracted based on real-time resource snapshots.

[0104] The specific process is as follows: obtain the creation time, GPU resource usage start time, resource usage change record, and current execution time of the currently running process based on the task running information in the real-time resource snapshot; determine the process startup phase based on the process creation time; determine the resource usage phase based on the continuous GPU resource usage time; determine the resource release phase based on the resource usage decrease time node; and determine the task completion phase based on the task end record.

[0105] Furthermore, after obtaining the process lifecycle characteristics, historical task patterns are matched by combining historical execution records.

[0106] The specific matching process is as follows: read the lifecycle data corresponding to the completed tasks in the historical running records, and compare the start time, resource occupation duration, resource release time and task execution duration in the current process lifecycle characteristics with the corresponding parameters in the historical task mode.

[0107] The matching score between the current process and the historical task mode is calculated based on the comparison results.

[0108] The specific calculation process is as follows: calculate the difference between the current process lifecycle parameters and the corresponding parameters of the historical task modes, weight each parameter difference according to a preset weight, and sum the weighted results to obtain the pattern matching score; when the pattern matching score meets the preset matching conditions, the corresponding historical task mode is used as the matching mode of the current task.

[0109] The preset matching conditions are determined based on the results of historical task classification.

[0110] The specific determination process is as follows: collect the life cycle characteristics of known task types during the execution of historical tasks, calculate the range of feature differences between tasks of the same type, and use the scoring range that can distinguish different task operation modes as the matching judgment condition.

[0111] When calculating future release time points based on historical task patterns and process lifecycle characteristics, the total execution time and resource release time points of tasks matching the historical task patterns are first obtained.

[0112] Then, based on the current execution time of the task, determine the current running stage of the task, and calculate the remaining execution time of the corresponding stage in the historical task mode.

[0113] The specific calculation process is as follows: take the time between the start of the task and the release of resources in the historical task mode as the reference execution cycle, subtract the time that the current task has already run, and get the estimated remaining running time of the current task; add the current time node and the estimated remaining running time together to get the future resource release time point.

[0114] The above calculation process is executed on multiple running tasks in the cluster, and they are arranged according to task number, GPU node number and predicted release time to generate a resource release schedule.

[0115] The resource release schedule records the expected release time of GPU resources corresponding to each task, which is used to indicate the location and time of resources that can be released within future scheduling cycles.

[0116] Subsequently, the resource release schedule is aligned with the memory fragmentation rate and transmission bandwidth values ​​to generate a composite resource state matrix.

[0117] The specific process is as follows: based on the GPU node number in the resource release schedule, find the memory status and network transmission status of the corresponding node, and combine the resource release time, memory fragmentation rate and transmission bandwidth value corresponding to the same GPU node in a preset order to form node resource status data.

[0118] The memory fragmentation rate is calculated as follows: the size of all free memory regions in the current GPU node is counted, and the size of the contiguous allocable memory regions is determined. The size of the non-contiguous free memory regions is compared with the size of all free memory regions to obtain the memory fragmentation rate. The transmission bandwidth value is obtained by reading the data transmission link status between GPU nodes, including the current available transmission bandwidth and historical transmission status between nodes.

[0119] The resource release time, memory fragmentation rate, and transmission bandwidth values ​​corresponding to each GPU node are arranged in a matrix to form a composite resource status matrix; this matrix is ​​used to describe the current GPU resource status and future resource changes.

[0120] Finally, feature mapping is performed on the composite resource state matrix to obtain a high-dimensional feature vector, which is then used as the environment state input for the reinforcement learning scheduling model.

[0121] The specific process is as follows: First, the different types of parameters in the composite resource state matrix are standardized to convert the resource release time, memory fragmentation rate and transmission bandwidth values ​​into a unified data format; then, features are combined according to node number and resource parameter category to convert the multidimensional resource state in the matrix into a fixed-length data vector.

[0122] The high-dimensional feature vector obtained from the transformation is input into the reinforcement learning scheduling model as the current environmental state of the model.

[0123] The reinforcement learning scheduling model calculates the scheduling strategy based on the environment state and task state information, so that the subsequent GPU resource binding, task sorting and resource adjustment processes can be executed according to the current resource state and the predicted release state. Example 6:

[0124] S5 includes obtaining the continuous occupancy time segments of the tasks to be scheduled, calculating the continuous occupancy demand intensity value based on the continuous occupancy time segments; if the continuous occupancy demand intensity value is higher than a preset threshold, generating a multi-dimensional state feature vector by combining resource contention rate and node affinity data; inputting the multi-dimensional state feature vector into the reinforcement learning scheduling model, and having the reinforcement learning scheduling model output a scheduling decision vector; performing action space mapping analysis on the scheduling decision vector, and outputting the binding timing selection scheme and binding relationship maintenance period.

[0125] This embodiment first obtains the continuous time segments corresponding to the task to be scheduled. The continuous time segments are obtained by collecting GPU resource usage records during task execution.

[0126] The specific process is as follows: read the GPU computing resource occupancy status of the task to be scheduled during execution, and arrange the resource occupancy value corresponding to each sampling time point in chronological order; when the task continuously maintains the GPU resource occupancy status between adjacent sampling time points, and the resource occupancy status does not change, the corresponding continuous time interval is determined as a continuous occupancy time segment.

[0127] Furthermore, after obtaining the continuous occupancy time segments, the continuous occupancy demand intensity value is calculated based on the continuous occupancy time segments.

[0128] The specific calculation process is as follows: First, the duration of all consecutive time segments occupied by the current task is counted, and the duration of each consecutive time segment is added together to obtain the total continuous time occupied by the task; then, the total execution time of the task from the start of execution to the current moment is obtained, and the time continuity parameter is determined according to the ratio between the total continuous time occupied and the total execution time.

[0129] At the same time, based on the historical task operation records, the continuous occupation time data of tasks with the same type or similar resource requirements as the current task is obtained, and the total continuous occupation time of the current task is compared with the continuous occupation time of historical tasks to determine the historical behavior matching parameters.

[0130] Then, based on the task resource release records, calculate the time required for resources to be released again after continuous occupation ends, and determine the resource retention parameters.

[0131] In addition, the time continuity parameter, historical behavior matching parameter, and resource retention parameter are weighted according to preset weights to obtain the continuous occupancy demand intensity value.

[0132] The preset weights are determined based on historical scheduling data. The specific process is as follows: collect the continuous task occupancy time, GPU binding and holding time, task waiting time and resource switching times during historical task scheduling, statistically analyze the impact of different parameters on GPU binding effect, and use the corresponding parameter impact ratio as the basis for weight configuration.

[0133] When the continuous occupancy demand intensity value exceeds the preset threshold, a multi-dimensional state feature vector is generated by combining resource contention rate and node affinity data.

[0134] The preset threshold is determined based on the historical task scheduling results. The specific process is as follows: obtain the continuous occupancy demand intensity value calculated during the historical task execution process, and record the task execution result after adopting the GPU binding scheduling method; statistically analyze the task completion time, GPU resource switching times and resource waiting time corresponding to different continuous occupancy demand intensity values, and determine the minimum continuous occupancy demand intensity value that can make the task execution state meet the preset scheduling conditions as the preset threshold.

[0135] The resource contention rate is calculated based on the resource demand relationship between the current task and other tasks in the task queue.

[0136] The specific process is as follows: obtain the amount of GPU computing resources and video memory resources requested by the current task, and obtain the resource requests of other tasks in the task queue; determine the number of tasks with the same GPU resource requirements as the current task as the number of competing tasks, take the number of tasks participating in resource competition in the task queue as the total number of competing tasks, and obtain the resource contention rate based on the ratio between the number of competing tasks and the total number of competing tasks.

[0137] Node affinity data is obtained based on the matching relationship between tasks and GPU nodes.

[0138] The specific process is as follows: read the current computing resource status, video memory status, data transfer status, and historical task binding records of the GPU node, calculate the matching degree between the task resource requirements and the available resources of the GPU node, and generate node affinity data based on the matching results of each GPU node.

[0139] By combining continuous occupancy demand intensity, resource contention rate, and node affinity data, a multidimensional state feature vector is generated.

[0140] The specific process is as follows: the continuous occupancy demand intensity value of the task is used as the task behavior feature, the resource contention rate is used as the task competition state feature, and the node affinity data is used as the resource matching feature. These are combined in a preset order to form a multi-dimensional state feature vector, which contains the task occupancy demand and GPU resource state information.

[0141] Furthermore, the multidimensional state feature vector is input into the reinforcement learning scheduling model; after receiving the input state, the reinforcement learning scheduling model calculates the current state according to the scheduling strategy obtained during training.

[0142] The specific process is as follows: extract the task continuous occupancy demand feature, resource competition feature, and node matching feature from the multi-dimensional state feature vector, input the above features into the model policy network, calculate the action evaluation value corresponding to different scheduling actions by the policy network, and generate the scheduling decision vector based on the action evaluation value.

[0143] The scheduling decision vector includes GPU binding state selection parameters, binding target node selection parameters, and binding duration parameters. The GPU binding state selection parameters indicate whether to establish a GPU binding relationship. The binding target node selection parameters indicate the GPU resource node corresponding to the task. The binding duration parameters indicate the binding relationship maintenance period.

[0144] Furthermore, the scheduling decision vector is analyzed by action space mapping.

[0145] The specific process is as follows: establish a mapping relationship between scheduling action parameters and actual GPU scheduling operations; convert the binding state parameters in the scheduling decision vector into binding operation instructions; convert the target node selection parameters into the corresponding GPU node number; and convert the binding duration parameters into a time period range.

[0146] The binding timing selection scheme and binding relationship maintenance period are output based on the action space mapping results; the binding timing selection scheme is determined based on the current execution status of the task and the scheduling action instructions.

[0147] Specifically, when the binding status parameter in the scheduling decision vector meets the binding establishment conditions, the current task scheduling time node is obtained as the binding establishment time; when the task is already in the GPU execution state, the binding adjustment time is determined according to the next resource scheduling cycle.

[0148] As a preferred implementation, the binding relationship maintenance period is determined based on the binding duration parameter in the scheduling decision vector and adjusted in conjunction with the GPU resource release prediction results.

[0149] The specific process is as follows: obtain the estimated resource release time of the current task, and compare the estimated resource release time with the binding maintenance period output by the model; when the resource release time is earlier than the end time of the binding period, use the resource release time as the termination time of the binding relationship; when the resource release time is later than the end time of the binding period, maintain the GPU binding state according to the binding relationship maintenance period output by the model. Example 7:

[0150] S6 includes obtaining the scheduling decision vector output by the reinforcement learning scheduling model and the preset binding relationship maintenance period; determining the resource occupation time window of the task to be scheduled based on the scheduling decision vector and the binding relationship maintenance period; performing time-series alignment and comparison between the resource occupation time window and the resource release prediction results; selecting task combinations that meet the preset conflict conditions to construct a potential conflict task set; obtaining the task urgency and conflict impact range data of each task in the potential conflict task set, and performing weighted calculation to obtain the priority weight value; sorting the potential conflict task set according to the priority weight value to obtain the conflict priority table.

[0151] In this embodiment, the scheduling decision vector output by the reinforcement learning scheduling model and the preset binding relationship maintenance period are first obtained.

[0152] Specifically, the scheduling decision vector includes the GPU binding node corresponding to the task, the resource adjustment action, and the binding status information; the binding relationship maintenance period represents the duration for which the current task maintains the GPU resource binding relationship.

[0153] The resource occupation time window of the task to be scheduled is determined based on the scheduling decision vector and the binding relationship maintenance period.

[0154] The specific process is as follows: determine the target GPU resources occupied by the task based on the GPU binding node information in the scheduling decision vector; read the current task start scheduling time and use this time as the start time of the resource occupation time window; determine the resource maintenance end time based on the binding relationship maintenance period, that is, add the task start scheduling time and the binding relationship maintenance period together to obtain the resource occupation end time; the start time and end time constitute the resource occupation time window of the task to be scheduled.

[0155] Furthermore, after obtaining the resource occupancy time window, the time window is time-series aligned and compared with the resource release prediction results; the resource release prediction results include the expected release time corresponding to the running task, the released GPU node, and the type of released resource.

[0156] The specific comparison process is as follows: read the estimated release time of each running task in the resource release prediction results, and compare the estimated release time with the resource occupation time window of the task to be scheduled; when the estimated release time is within the resource occupation time window, it is determined that there is a time overlap between the two tasks; further obtain the GPU resource demand information corresponding to the two tasks, and when the GPU computing resources or video memory resources requested by the two tasks have an overlapping occupation relationship, it is determined that there is a resource conflict between the two tasks.

[0157] As a preferred embodiment, a set of potential conflicting tasks is constructed by filtering task combinations that meet the preset conflict conditions.

[0158] The preset conflict conditions are determined based on historical GPU resource scheduling data. The specific determination process is as follows: obtain the task records in the historical task scheduling process that have experienced resource waiting, task delay execution, or GPU resource reallocation, and statistically analyze the resource occupation time overlap range and resource demand overlap between tasks when the above situations occur. The time overlap conditions and resource competition conditions that meet the characteristics of historical conflict tasks are used as conflict judgment conditions.

[0159] After determining the conflict conditions, each task combination in the task queue is evaluated one by one; when a task combination simultaneously meets the conditions of overlapping resource occupation time and GPU resource contention, the corresponding task combination is added to the potential conflict task set.

[0160] Furthermore, data on the urgency and impact of each task in the potential conflict task set are obtained, and a weighted calculation is performed to obtain the priority weight value.

[0161] The urgency of the task is calculated based on the remaining execution time and the deadline requirement.

[0162] The specific process is as follows: obtain the estimated completion time of the current task and the preset deadline of the task, and calculate the time difference between the two; when the estimated completion time is close to the deadline, determine the remaining scheduling time of the task; at the same time, calculate the task execution progress parameter based on the relationship between the task's executed time and the estimated total execution time, and combine the remaining scheduling time parameter and the execution progress parameter to obtain the task urgency data.

[0163] The scope of conflict impact data is calculated based on the scope of resource contention within the task.

[0164] The specific process is as follows: count the number of tasks that compete with the current task for resources, and obtain the number of GPU nodes affected by the current task's resource usage; combine the number of competing tasks and the number of affected GPU nodes to obtain the conflict impact range data.

[0165] Priority weight values ​​are obtained by weighting the calculation based on the urgency of the task and the scope of the conflict.

[0166] The specific calculation process is as follows: First, the data on task urgency and conflict impact range are normalized to convert different types of parameters into a unified evaluation scale; then, the weight values ​​corresponding to task urgency and conflict impact range are calculated according to preset weights; finally, the two calculation results are summed to obtain the priority weight value of the corresponding task.

[0167] The preset weights are determined based on the results of historical conflict resolution.

[0168] The specific process is as follows: collect historical conflict task processing records, obtain resource adjustment results, task completion time and resource waiting time corresponding to tasks with different urgency and different conflict scopes, and determine the weight configuration based on the influence relationship of each parameter on the task scheduling results.

[0169] The set of potentially conflicting tasks is sorted according to their priority weight values ​​to obtain a conflict priority table.

[0170] The specific sorting process is as follows: read the priority weight value corresponding to each task in the potential conflict task set, sort the priority weight values ​​in descending order to determine the task processing order; when multiple tasks have the same priority weight value, further compare the task deadline time and current resource occupancy status, and arrange the task with the earlier deadline or stronger resource competition relationship first to generate the final conflict priority table.

[0171] The conflict priority table is used to represent the processing order of different conflicting tasks during the resource adjustment process, and serves as the data basis for subsequent task queue sorting and preemption decision-making processes. Example 8:

[0172] S7 includes acquiring a conflict priority list and task queue data, generating an initial sorted task sequence based on priority weight values; traversing the initial sorted task sequence to identify resource fragment locations that can be preempted, and calculating the preemption cost value of tasks associated with resource fragment locations; if the preemption cost value is lower than the expected latency loss value, generating a scheduling instruction set containing preemption actions, and arranging the tasks in the initial sorted task sequence in a time sequence according to the scheduling instruction set to obtain a preemption decision process sequence.

[0173] This embodiment first obtains the conflict priority list and task queue data.

[0174] Specifically, the conflict priority list includes task identifier, priority weight value, resource contention relationship, and conflict impact range; the task queue data includes tasks waiting to be executed, currently running tasks, task resource requirements, task execution status, and GPU resource occupancy status.

[0175] An initial sorting task sequence is generated based on priority weight values.

[0176] The specific process is as follows: read the priority weight value corresponding to each task in the conflict priority list, associate the priority weight value of each task with the task number; arrange the tasks in descending order of priority weight value to generate an initial sorted task sequence.

[0177] When multiple tasks have the same priority weight value, the tasks with the same weight are sorted a second time.

[0178] The specific process is as follows: obtain the remaining execution time, preset deadline, and current GPU resource usage for tasks with the same weight; determine the task time urgency parameter based on the remaining execution time, determine the task deadline constraint parameter based on the deadline, and determine the resource usage impact parameter based on the resource usage; compare the above parameters to determine the order of tasks with the same weight.

[0179] Subsequently, the initial sorting task sequence is traversed to identify the locations of resource fragments that can be preempted.

[0180] The specific process is as follows: Read the GPU node resource status according to the task order in the initial sorted task sequence, and obtain the positional relationship between the allocated resource area and the idle resource area in the current GPU node; analyze the capacity, starting address and corresponding task release status of each idle resource area to determine the location of resource fragments that can be reallocated.

[0181] The process of determining the location of resource fragments is as follows: obtain the GPU resource capacity required by the task to be scheduled, compare the capacity of each free resource region with the resource demand capacity of the task to be scheduled; when the capacity of the free resource region meets the resource demand of the task, and the region does not belong to the continuously occupied region of the currently executing task, the free region is determined as the location of the preemptible resource fragment.

[0182] Furthermore, after determining the location of the resource fragment, the preemption cost of the task associated with the resource fragment location is calculated.

[0183] The specific process is as follows: obtain the running status of the task corresponding to the resource fragment location, including the task's execution time, remaining execution time, current GPU resource usage, and task migration data volume.

[0184] Calculate the time overhead incurred during the preemption process based on the above parameters.

[0185] The specific process is as follows: determine the time loss caused by the task stopping execution based on the task pause time; determine the recovery overhead based on the amount of data to be reloaded and the computing resource initialization time required to resume task execution; and determine the data migration overhead based on the amount of data to be migrated and the transmission bandwidth between GPU nodes.

[0186] Furthermore, the preemption cost of tasks associated with resource fragment locations is calculated by weighting the task pause time loss, resumption execution cost, and data migration cost.

[0187] The weights of each expense are determined based on historical task preemption records. Specifically, the proportion of each expense's impact on task delay is determined by statistically analyzing the changes in task completion time after the execution of preemption operations for different task types.

[0188] Specifically, when the preemption cost is lower than the expected latency loss value, a scheduling instruction set containing the preemption action is generated; the expected latency loss value is used to limit the maximum latency impact allowed during task execution.

[0189] The process for determining the expected delay loss value is as follows: obtain historical task execution data, and calculate the completion time of the task when the preemption operation was not performed and the completion time after the preemption operation was performed; calculate the time difference between the two execution states to obtain historical preemption delay loss data; and determine the expected delay loss value based on the delay loss range that meets the task deadline requirements during historical scheduling.

[0190] When the calculated preemption cost is less than the expected delay loss, it means that the task impact of the preemption action meets the scheduling conditions, and the corresponding scheduling instruction set is generated.

[0191] The scheduling instruction set includes instructions to pause preempted tasks, instructions to release GPU resources, instructions to allocate resources to target tasks, and instructions to resume task execution.

[0192] Based on the scheduling instruction set, the tasks in the initial sorted task sequence are arranged in a time sequence to obtain the preemption decision process sequence.

[0193] The specific process is as follows: First, read the execution order of each action in the scheduling instruction set; determine the tasks that need to be processed in advance according to the resource release instruction, determine the tasks that need to be executed first according to the resource allocation instruction, and determine the recovery execution time node according to the task recovery instruction; rearrange the tasks in the initial sorted task sequence according to the action execution time order to generate the preemption decision process sequence.

[0194] The preemption decision-making process sequence includes the task pause order, resource release order, GPU resource allocation order, and task resumption order, which are used to guide the subsequent GPU binding relationship state adjustment process. Example 9:

[0195] S8 includes receiving a preemption decision process sequence, parsing the target device handle according to the preemption decision process sequence, reading the video memory address mapping table of the target device handle to locate the physical slot; sending an interrupt instruction for the physical slot, serializing the process context to the virtual address space, and releasing the physical slot from the video memory page table; filtering the target computing unit set according to topology affinity, aggregating the video memory fragments of the target computing unit set to construct a usable video memory block; loading priority tasks into the computing unit corresponding to the usable video memory block and updating the resource registry to obtain the updated GPU binding relationship status.

[0196] This embodiment first receives the preemption decision process sequence.

[0197] Specifically, the preemption decision process sequence includes task order adjustment, task pause operation, resource release operation, and task reloading operation; the target task that needs to perform resource adjustment is determined based on the task identification information in the preemption decision process sequence, and the target device handle is parsed.

[0198] The parsing process of the target device handle is as follows: read the task identifier in the preemption decision process sequence, query the current GPU resource registration information based on the task identifier, obtain the GPU device number, device context address and resource occupation record corresponding to the task, and combine the above information to form the target device handle; the target device handle is used to locate the GPU device resources currently occupied by the target task.

[0199] Furthermore, after obtaining the target device handle, the corresponding video memory address mapping table is read to locate the physical slot.

[0200] The specific process is as follows: access the memory management record of the corresponding GPU device based on the target device handle, and read the mapping relationship between the task's virtual memory address and physical memory address; query the corresponding physical memory address based on the virtual memory address, and then determine the physical slot currently occupied by the task based on the memory region to which the physical memory address belongs.

[0201] Send an interrupt command for the determined physical slot.

[0202] The specific process is as follows: an interrupt request is generated based on the task pause action in the preemption decision process sequence, and the interrupt request is sent to the target GPU device, causing the target GPU device to stop executing the subsequent calculation instructions of the corresponding task and enter the context saving state.

[0203] During interrupt handling, the process context is serialized into the virtual address space.

[0204] The specific process is as follows: read the computation register data, thread scheduling status, video memory access status, and task execution position in the current execution state of the target task; convert the above status data into continuous storage data according to the preset data arrangement order, and record the virtual address position corresponding to each status data to form recoverable process context data.

[0205] Furthermore, after saving the process context, the physical slot is released from locking the video memory page table.

[0206] The specific process is as follows: read the video memory page table entry corresponding to the physical slot to obtain the virtual address mapping relationship corresponding to the physical slot; delete the occupancy lock flag in the page table entry and update the video memory page table status so that the corresponding physical slot changes from the task-occupied state to the allocable state.

[0207] Furthermore, the target computing unit set is selected based on topological affinity.

[0208] The specific process is as follows: obtain the topology information of the computing units in the current GPU cluster, including the GPU node connection relationship, data transmission path between nodes, remaining computing resources, and available video memory capacity; and filter the candidate computing units according to the computing resource requirements and video memory requirements of the task to be loaded.

[0209] During the screening process, the topology affinity evaluation value is calculated based on the matching relationship between task requirements and candidate computing power units.

[0210] The specific calculation process is as follows: obtain the available computing resources, available video memory capacity, and node transmission bandwidth of the candidate computing power units respectively; compare the computing resources required by the task with the available computing resources of the candidate computing power units to obtain computing resource matching parameters; compare the video memory requirements of the task with the available video memory capacity of the candidate computing power units to obtain video memory matching parameters; determine the transmission matching parameters based on the task data transmission path length and node transmission bandwidth; and perform weighted calculations on the above parameters according to preset weights to obtain the topology affinity evaluation value.

[0211] Candidate computing units are sorted according to their topology affinity evaluation values. Computing units that meet resource requirements and whose evaluation values ​​meet scheduling conditions are selected to form a target computing unit set.

[0212] Subsequently, the memory fragments in the target computing unit set are aggregated to construct a usable memory block.

[0213] The specific process is as follows: read the memory allocation status of each GPU node in the target computing unit set, obtain the start address, end address and available capacity of each free memory region; and arrange the free memory regions according to the physical address order.

[0214] When merging adjacent video memory fragments, the address interval between the end address of the previous video memory region and the start address of the next video memory region is calculated. If there are no allocated resources in the region corresponding to the address interval, and the capacity of the merged video memory regions meets the task's video memory requirements, the two video memory regions are merged. The above process is repeated until a continuous available video memory block that meets the task's loading requirements is formed.

[0215] Finally, the priority tasks are loaded into the computing units corresponding to the available memory blocks, and the resource registry is updated to obtain the updated GPU binding relationship status.

[0216] The specific loading process is as follows: read the priority task information according to the task execution order in the preemption decision process sequence, obtain the computing resource requirements and video memory requirements corresponding to the task; select the video memory block whose capacity meets the task requirements from the set of available video memory blocks, and load the task context data into the corresponding GPU computing unit; establish a new binding relationship between the task identifier and the target GPU device.

[0217] After the task loading is complete, the resource registry is updated.

[0218] The specific process is as follows: Modify the resource occupancy status of the target GPU node in the resource registry, mark the memory slot occupied by the original task as released, mark the memory block occupied by the new task as occupied, and synchronously update the task identifier, GPU device number, memory address mapping relationship, and binding time information. Example 10:

[0219] S9 includes obtaining the updated GPU binding relationship state and multi-dimensional state observation vector, which is composed of memory usage, computing power utilization, and task queuing time; generating a scalar reward signal value based on the multi-dimensional state observation vector, and associating the scalar reward signal value with the corresponding binding state snapshot to construct a state transition tuple; calculating gradient descent parameters based on the state transition tuple, and using the gradient descent parameters to update the policy network weights of the reinforcement learning scheduling model for the next round of loop processing.

[0220] This embodiment first obtains the updated GPU binding relationship state and multi-dimensional state observation vector.

[0221] Specifically, the updated GPU binding relationship status includes the task identifier, the bound GPU device number, the video memory area occupied by the task, the binding duration, and the computing unit information where the task is located.

[0222] The multidimensional state observation vector is formed by a combination of memory usage, computing power utilization, and task queuing time.

[0223] The specific acquisition process is as follows: read the allocated memory capacity of the current GPU node and the total memory capacity of the GPU node, compare the allocated memory capacity with the total memory capacity to obtain the memory utilization rate of the current node.

[0224] The process of obtaining computing power utilization is as follows: collect the time length of the GPU computing unit executing computing tasks within the current scheduling period, and obtain the total time length of the current scheduling period; determine the current GPU computing power utilization based on the ratio between the execution time length of the computing tasks and the total time length of the scheduling period.

[0225] The process of obtaining the task queuing time is as follows: read the time node when the task enters the scheduling queue and obtain the time node when the task actually obtains GPU resources to execute; calculate the time difference between the two time nodes to obtain the task queuing time.

[0226] In addition, memory usage, computing power utilization, and task queuing time are combined in a preset order to form a multi-dimensional state observation vector. This multi-dimensional state observation vector is used to represent the current GPU resource usage status and task waiting status.

[0227] Furthermore, a scalar reward signal value is generated based on the multidimensional state observation vector.

[0228] The specific process is as follows: obtain the memory usage rate, computing power utilization rate and task queuing time parameters in the multi-dimensional state observation vector respectively, and determine the evaluation value corresponding to each parameter according to the preset reward calculation rules.

[0229] The video memory usage evaluation value is determined based on the video memory resource usage status.

[0230] The specific calculation process is as follows: the current video memory usage rate is compared with the target resource utilization range. When the video memory usage rate is within the target range, a corresponding resource utilization evaluation value is generated. When the video memory usage rate deviates from the target range, a resource utilization loss value is calculated based on the degree of deviation.

[0231] The computing power utilization evaluation value is determined based on the execution status of GPU computing resources.

[0232] The specific process is as follows: compare the current computing power utilization rate with the preset computing resource utilization target value, calculate the difference between the two, and determine the computing resource utilization evaluation value based on the difference.

[0233] The task waiting evaluation value is determined based on the task queuing time.

[0234] The specific process is as follows: obtain the task queuing time and compare it with the preset waiting time limit; when the task queuing time exceeds the waiting time limit, calculate the waiting loss value based on the excess.

[0235] The evaluation values ​​of video memory usage, computing power utilization, and task waiting are weighted according to preset weights to obtain the scalar reward signal value.

[0236] The specific calculation process is as follows: multiply the above three evaluation values ​​by their corresponding weights, and sum the three results to obtain the reward signal value corresponding to the current GPU resource scheduling action.

[0237] The preset weights are determined based on historical scheduling results.

[0238] The specific process is as follows: collect data on GPU utilization status, task completion time, and task waiting time during historical scheduling cycles, analyze the impact of memory resource utilization, computing resource utilization, and task waiting status on scheduling performance, and determine the reward calculation weight based on the corresponding impact ratio of each evaluation parameter.

[0239] Associate the scalar reward signal value with the corresponding bound state snapshot to construct a state transition tuple.

[0240] The specific process is as follows: read the GPU binding relationship state at the beginning of the current scheduling cycle and use it as the current state in the state transition tuple; obtain the scheduling action output by the reinforcement learning scheduling model based on the current state; record the updated GPU binding relationship state formed after executing the scheduling action and use it as the next state; combine the current state, scheduling action, reward signal value and the next state to form a state transition tuple.

[0241] The gradient descent parameters are calculated based on the state transition tuple. The specific process is as follows: input the state transition tuple into the reinforcement learning scheduling model to obtain the action evaluation result output by the policy network based on the current state; calculate the target evaluation value based on the reward signal value in the state transition tuple and the evaluation result of the next state; compare the difference between the action evaluation result and the target evaluation value to obtain the output error of the policy network.

[0242] Furthermore, backpropagation is performed based on the output error of the policy network to obtain the gradient of the corresponding parameters of each network layer.

[0243] The specific process is as follows: starting from the output layer of the policy network, calculate the direction of weight adjustment of the output layer based on the error value; then calculate the gradient change of the corresponding weight parameters of the hidden layer and the input layer in sequence to obtain the gradient descent parameters.

[0244] The policy network weights of the reinforcement learning scheduling model are updated using gradient descent parameters.

[0245] The specific process is as follows: read the weight parameters of each layer in the current policy network, calculate the weight adjustment amount according to the gradient descent parameters and the preset learning step size; update the original weight parameters with the corresponding adjustment amount to obtain the new policy network weights.

[0246] Furthermore, after updating the policy network weights, the updated reinforcement learning scheduling model is used for the next round of GPU resource scheduling.

[0247] In the next round of processing, the reinforcement learning scheduling model receives task status data and GPU resource status data according to the new policy network weights, re-outputs the task scheduling policy, and continues to execute the resource allocation, status feedback, and policy update process.

[0248] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A GPU resource scheduling method based on reinforcement learning, characterized in that, include: S1. Obtain all task information in the current task queue, including historical execution records and expected completion times, to obtain task requirement difference data; S2. Perform historical behavior analysis on the data of different task requirements to obtain the duration characteristics of various tasks. S3. Construct a scheduling state vector based on the duration of task occupancy and input it into the reinforcement learning scheduling model to obtain the intensity of continuous task occupancy demand and the basis for establishing GPU binding relationship. S4. Obtain the current GPU occupancy status and future release time prediction to obtain the resource release prediction result, and use it as the environment state input for the reinforcement learning scheduling model; S5. If the intensity of continuous occupancy demand exceeds the preset threshold, the reinforcement learning scheduling model will output the binding timing selection scheme and the binding relationship maintenance period. S6. Based on the binding relationship maintenance period and resource release prediction results, and combined with the reinforcement learning scheduling model, calculate the set of potential conflict tasks to obtain the conflict priority list. S7. Sort the task queue of the conflict priority list to obtain the preemption decision process sequence; S8. Adjust the resource allocation status according to the preemption decision process sequence to obtain the updated GPU binding relationship status; S9. Monitor the updated GPU binding relationship status and generate reward signals based on resource utilization and task waiting time. Feedback these signals to the reinforcement learning scheduling model for policy updates and the next round of processing.

2. The GPU resource scheduling method based on reinforcement learning according to claim 1, characterized in that: S1 includes: Read the task queue metadata to obtain a task scheduling snapshot, and retrieve the historical feature set based on the task scheduling snapshot; The estimated execution time is calculated using a set of historical features, and the estimated execution time is compared with the preset deadline in the task scheduling snapshot to generate a time deviation value. If the time deviation exceeds the threshold, a resource scarcity index is determined based on the resource demand matrix, and a demand difference vector is constructed based on the resource scarcity index. Aggregate the requirement difference vectors to generate the final dataset, thereby obtaining task requirement difference data.

3. The GPU resource scheduling method based on reinforcement learning according to claim 1, characterized in that: S2 includes: Historical behavior logs are indexed based on the task requirement difference data, and historical time series segments are extracted based on the resource scarcity index in the task requirement difference data. Waveform decomposition of historical time series segments yields a set of resource occupancy waveforms. If the steady-state occupancy period in the resource occupancy waveform set meets the stability criterion, it is input into the behavior pattern clustering model to obtain the category feature vector. A feature mapping matrix is ​​constructed using category feature vectors, and the time dimension data in the feature mapping matrix is ​​parsed to generate the duration features of various tasks.

4. The GPU resource scheduling method based on reinforcement learning according to claim 1, characterized in that: S3 includes: Obtain the duration characteristics of various tasks to construct a scheduling state vector; The scheduling state vector is input into the reinforcement learning scheduling model, and the intensity of continuous task occupancy demand is obtained by combining the computing power topology. If the continuous demand intensity of a task exceeds a threshold, a resource contention graph is constructed. The affinity probability distribution is calculated based on the resource contention graph, and the GPU binding relationship is established based on the affinity probability distribution generated by the reinforcement learning scheduling model.

5. The GPU resource scheduling method based on reinforcement learning according to claim 1, characterized in that: S4 includes: Obtain the memory utilization and computing load values ​​of cluster nodes, and construct a real-time resource snapshot based on the node topology graph; Extract process lifecycle characteristics based on real-time resource snapshots and match historical task patterns with historical execution records; Calculate future release time points based on historical task patterns and process lifecycle characteristics, and generate a resource release schedule; The resource release schedule is aligned with the memory fragmentation rate and transmission bandwidth values ​​to generate a composite resource state matrix. A high-dimensional feature vector is obtained by mapping the composite resource state matrix, which serves as the environment state input for the reinforcement learning scheduling model.

6. The GPU resource scheduling method based on reinforcement learning according to claim 1, characterized in that: S5 includes: Obtain the continuous occupancy time segments of the tasks to be scheduled, and calculate the continuous occupancy demand intensity value based on the continuous occupancy time segments; If the continuous occupancy demand intensity value is higher than the preset threshold, a multi-dimensional state feature vector is generated by combining the resource contention rate and node affinity data. The multidimensional state feature vector is input into the reinforcement learning scheduling model, and the reinforcement learning scheduling model outputs the scheduling decision vector. Perform action space mapping analysis on the scheduling decision vector, and output the binding timing selection scheme and the binding relationship maintenance period.

7. The GPU resource scheduling method based on reinforcement learning according to claim 1, characterized in that: S6 includes: Obtain the scheduling decision vector and the preset binding relationship maintenance period output by the reinforcement learning scheduling model, and determine the resource occupation time window of the task to be scheduled based on the scheduling decision vector and the binding relationship maintenance period; By comparing the resource occupation time window with the resource release prediction results in a time sequence, task combinations that meet the preset conflict conditions are selected to construct a set of potential conflict tasks. Obtain the task urgency and conflict impact data for each task in the potential conflict task set, and perform weighted calculations to obtain priority weight values; The set of potential conflicting tasks is sorted according to priority weight values ​​to obtain the conflict priority table.

8. The GPU resource scheduling method based on reinforcement learning according to claim 1, characterized in that: S7 includes: Obtain the conflict priority list and task queue data, and generate an initial sorted task sequence based on the priority weight values; Traverse the initial sorted task sequence to identify the locations of resource fragments that can be preempted, and calculate the preemption cost of the tasks associated with the resource fragment locations. If the preemption cost is lower than the expected delay loss, a scheduling instruction set containing preemption actions is generated. The tasks in the initial sorted task sequence are then time-ordered according to the scheduling instruction set to obtain the preemption decision process sequence.

9. The GPU resource scheduling method based on reinforcement learning according to claim 1, characterized in that: S8 includes: Receive the preemption decision process sequence, parse the target device handle according to the preemption decision process sequence, and read the video memory address mapping table of the target device handle to locate the physical slot; An interrupt command is sent to the physical slot, the process context is serialized to the virtual address space, and the physical slot is released from the lock on the video memory page table; The target computing unit set is selected based on topological affinity, and the memory fragments of the target computing unit set are aggregated to construct a usable memory block; The priority task is loaded into the computing unit corresponding to the available video memory block and the resource registry is updated to obtain the updated GPU binding relationship status.

10. The GPU resource scheduling method based on reinforcement learning according to claim 1, characterized in that: S9 includes: Obtain the updated GPU binding relationship status and multi-dimensional state observation vector, which is composed of memory usage, computing power utilization and task queuing time. A scalar reward signal value is generated based on the multidimensional state observation vector, and the scalar reward signal value is associated with the corresponding bound state snapshot to construct a state transition tuple; The gradient descent parameters are calculated based on the state transition tuple, and the policy network weights of the reinforcement learning scheduling model are updated using the gradient descent parameters for the next round of loop processing.

Images